2024-09-13T18:13:06.945 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T18:13:06.968 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T18:13:07.068 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903977 branch: wip-rishabh-testing-20240910.100730-reef description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} email: null first_in_suite: false job_id: '7903977' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-rishabh-testing-20240910.100730-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - Replacing daemon mds - SLOW_OPS - slow request sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240910.100730-reef sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 6439 sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 sleep_before_teardown: 0 subset: 964/1024 suite: fs suite_branch: wip-rishabh-testing-20240910.100730-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - mds_thrash: null - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_14:42:16 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T18:13:07.069 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa; will attempt to use it 2024-09-13T18:13:07.069 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa/tasks 2024-09-13T18:13:07.070 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T18:13:07.071 INFO:teuthology.task.internal:Checking packages... 2024-09-13T18:13:07.092 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3' 2024-09-13T18:13:07.092 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T18:13:07.092 INFO:teuthology.packaging:ref: None 2024-09-13T18:13:07.092 INFO:teuthology.packaging:tag: None 2024-09-13T18:13:07.092 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240910.100730-reef 2024-09-13T18:13:07.093 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:13:07.093 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-rishabh-testing-20240910.100730-reef 2024-09-13T18:13:07.241 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-860-g6c6b083f-1focal 2024-09-13T18:13:07.243 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T18:13:07.250 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T18:13:07.250 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T18:13:07.258 INFO:teuthology.task.internal:Saving configuration 2024-09-13T18:13:07.275 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T18:13:07.283 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T18:13:07.316 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903977', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 18:06:46.902328', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T18:13:07.346 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903977', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 18:06:46.900284', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T18:13:07.347 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T18:13:07.354 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-09-13T18:13:07.354 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-09-13T18:13:07.354 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T18:13:07.432 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f695b4efbe0>, signals=[15]) 2024-09-13T18:13:07.433 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T18:13:07.441 INFO:teuthology.task.internal:Opening connections... 2024-09-13T18:13:07.441 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2024-09-13T18:13:07.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:13:07.539 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2024-09-13T18:13:07.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:13:07.642 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T18:13:07.650 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2024-09-13T18:13:07.657 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2024-09-13T18:13:07.657 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2024-09-13T18:13:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="Ubuntu" 2024-09-13T18:13:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T18:13:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:ID=ubuntu 2024-09-13T18:13:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE=debian 2024-09-13T18:13:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T18:13:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="20.04" 2024-09-13T18:13:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T18:13:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T18:13:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T18:13:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T18:13:07.705 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_CODENAME=focal 2024-09-13T18:13:07.705 INFO:teuthology.orchestra.run.smithi146.stdout:UBUNTU_CODENAME=focal 2024-09-13T18:13:07.705 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2024-09-13T18:13:07.731 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2024-09-13T18:13:07.738 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2024-09-13T18:13:07.738 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2024-09-13T18:13:07.791 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="Ubuntu" 2024-09-13T18:13:07.791 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T18:13:07.791 INFO:teuthology.orchestra.run.smithi154.stdout:ID=ubuntu 2024-09-13T18:13:07.791 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE=debian 2024-09-13T18:13:07.791 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T18:13:07.791 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="20.04" 2024-09-13T18:13:07.791 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T18:13:07.791 INFO:teuthology.orchestra.run.smithi154.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T18:13:07.791 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T18:13:07.791 INFO:teuthology.orchestra.run.smithi154.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T18:13:07.792 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_CODENAME=focal 2024-09-13T18:13:07.792 INFO:teuthology.orchestra.run.smithi154.stdout:UBUNTU_CODENAME=focal 2024-09-13T18:13:07.792 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2024-09-13T18:13:07.831 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T18:13:07.842 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T18:13:07.849 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T18:13:07.849 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2024-09-13T18:13:07.852 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2024-09-13T18:13:07.857 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T18:13:07.865 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T18:13:07.865 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2024-09-13T18:13:07.867 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2024-09-13T18:13:07.909 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T18:13:07.976 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T18:13:07.992 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T18:13:07.992 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T18:13:07.993 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T18:13:07.993 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T18:13:07.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:13:07.998 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-09-13T18:13:07.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:13:08.000 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-09-13T18:13:08.005 INFO:teuthology.orchestra.run.smithi146.stdout:5.4.0-124-generic 2024-09-13T18:13:08.006 INFO:teuthology.task.kernel:Running kernel on smithi146: 5.4.0-124-generic 2024-09-13T18:13:08.006 DEBUG:teuthology.orchestra.run.smithi146:> sudo apt-get clean 2024-09-13T18:13:08.007 INFO:teuthology.orchestra.run.smithi154.stdout:5.4.0-124-generic 2024-09-13T18:13:08.007 INFO:teuthology.task.kernel:Running kernel on smithi154: 5.4.0-124-generic 2024-09-13T18:13:08.007 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get clean 2024-09-13T18:13:08.149 DEBUG:teuthology.orchestra.run.smithi146:> sudo apt-get update 2024-09-13T18:13:08.159 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2024-09-13T18:13:08.318 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T18:13:08.319 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-13T18:13:08.320 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-13T18:13:08.366 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T18:13:08.367 INFO:teuthology.orchestra.run.smithi146.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-13T18:13:08.367 INFO:teuthology.orchestra.run.smithi146.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-13T18:13:08.392 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-13T18:13:08.416 INFO:teuthology.orchestra.run.smithi146.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-13T18:13:08.700 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-13T18:13:08.751 INFO:teuthology.orchestra.run.smithi146.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-13T18:13:08.820 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-13T18:13:08.831 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-13T18:13:08.833 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-13T18:13:08.840 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-13T18:13:08.840 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-13T18:13:08.863 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-13T18:13:08.863 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-13T18:13:08.868 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T18:13:08.868 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-13T18:13:08.876 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-13T18:13:08.880 INFO:teuthology.orchestra.run.smithi146.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-13T18:13:08.881 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-13T18:13:08.882 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-13T18:13:08.882 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-13T18:13:08.887 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-13T18:13:08.887 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-13T18:13:08.888 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-13T18:13:08.890 INFO:teuthology.orchestra.run.smithi146.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-13T18:13:08.891 INFO:teuthology.orchestra.run.smithi146.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-13T18:13:08.896 INFO:teuthology.orchestra.run.smithi146.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-13T18:13:08.897 INFO:teuthology.orchestra.run.smithi146.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-13T18:13:08.923 INFO:teuthology.orchestra.run.smithi146.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-13T18:13:08.924 INFO:teuthology.orchestra.run.smithi146.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-13T18:13:08.927 INFO:teuthology.orchestra.run.smithi146.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T18:13:08.934 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-13T18:13:08.936 INFO:teuthology.orchestra.run.smithi146.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-13T18:13:08.941 INFO:teuthology.orchestra.run.smithi146.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-13T18:13:08.944 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-13T18:13:08.949 INFO:teuthology.orchestra.run.smithi146.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-13T18:13:08.950 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-13T18:13:08.950 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-13T18:13:08.950 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-13T18:13:08.951 INFO:teuthology.orchestra.run.smithi146.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-13T18:13:08.951 INFO:teuthology.orchestra.run.smithi146.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-13T18:13:08.952 INFO:teuthology.orchestra.run.smithi146.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-13T18:13:08.952 INFO:teuthology.orchestra.run.smithi146.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-13T18:13:08.952 INFO:teuthology.orchestra.run.smithi146.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-13T18:13:08.962 INFO:teuthology.orchestra.run.smithi146.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-13T18:13:08.971 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-13T18:13:08.975 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T18:13:08.975 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-13T18:13:08.983 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-13T18:13:08.990 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-13T18:13:08.993 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-13T18:13:08.993 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-13T18:13:08.993 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-13T18:13:08.994 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-13T18:13:09.001 INFO:teuthology.orchestra.run.smithi146.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-13T18:13:09.004 INFO:teuthology.orchestra.run.smithi154.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-13T18:13:09.004 INFO:teuthology.orchestra.run.smithi154.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-13T18:13:09.004 INFO:teuthology.orchestra.run.smithi154.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-13T18:13:09.005 INFO:teuthology.orchestra.run.smithi154.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-13T18:13:09.005 INFO:teuthology.orchestra.run.smithi146.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-13T18:13:09.005 INFO:teuthology.orchestra.run.smithi146.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-13T18:13:09.021 INFO:teuthology.orchestra.run.smithi154.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-13T18:13:09.022 INFO:teuthology.orchestra.run.smithi154.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-13T18:13:09.022 INFO:teuthology.orchestra.run.smithi154.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-13T18:13:09.025 INFO:teuthology.orchestra.run.smithi146.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-13T18:13:09.026 INFO:teuthology.orchestra.run.smithi146.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-13T18:13:09.029 INFO:teuthology.orchestra.run.smithi146.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T18:13:09.029 INFO:teuthology.orchestra.run.smithi146.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-13T18:13:09.035 INFO:teuthology.orchestra.run.smithi146.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-13T18:13:09.042 INFO:teuthology.orchestra.run.smithi146.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-13T18:13:09.045 INFO:teuthology.orchestra.run.smithi146.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-13T18:13:09.045 INFO:teuthology.orchestra.run.smithi146.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-13T18:13:09.045 INFO:teuthology.orchestra.run.smithi146.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-13T18:13:09.053 INFO:teuthology.orchestra.run.smithi146.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-13T18:13:09.054 INFO:teuthology.orchestra.run.smithi146.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-13T18:13:09.054 INFO:teuthology.orchestra.run.smithi146.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-13T18:13:09.054 INFO:teuthology.orchestra.run.smithi146.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-13T18:13:09.054 INFO:teuthology.orchestra.run.smithi146.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-13T18:13:09.070 INFO:teuthology.orchestra.run.smithi146.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-13T18:13:09.071 INFO:teuthology.orchestra.run.smithi146.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-13T18:13:09.071 INFO:teuthology.orchestra.run.smithi146.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-13T18:13:13.165 INFO:teuthology.orchestra.run.smithi146.stdout:Fetched 21.8 MB in 3s (7,086 kB/s) 2024-09-13T18:13:13.212 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 21.8 MB in 3s (7,225 kB/s) 2024-09-13T18:13:14.738 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-09-13T18:13:14.765 DEBUG:teuthology.orchestra.run.smithi146:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T18:13:14.798 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-09-13T18:13:14.826 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T18:13:14.842 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-09-13T18:13:14.901 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-09-13T18:13:15.059 INFO:teuthology.orchestra.run.smithi146.stdout:Building dependency tree... 2024-09-13T18:13:15.059 INFO:teuthology.orchestra.run.smithi146.stdout:Reading state information... 2024-09-13T18:13:15.121 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2024-09-13T18:13:15.122 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2024-09-13T18:13:15.259 INFO:teuthology.orchestra.run.smithi146.stdout:The following additional packages will be installed: 2024-09-13T18:13:15.260 INFO:teuthology.orchestra.run.smithi146.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T18:13:15.260 INFO:teuthology.orchestra.run.smithi146.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-13T18:13:15.260 INFO:teuthology.orchestra.run.smithi146.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-13T18:13:15.261 INFO:teuthology.orchestra.run.smithi146.stdout:Suggested packages: 2024-09-13T18:13:15.261 INFO:teuthology.orchestra.run.smithi146.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-13T18:13:15.261 INFO:teuthology.orchestra.run.smithi146.stdout:Recommended packages: 2024-09-13T18:13:15.261 INFO:teuthology.orchestra.run.smithi146.stdout: thermald 2024-09-13T18:13:15.330 INFO:teuthology.orchestra.run.smithi146.stdout:The following NEW packages will be installed: 2024-09-13T18:13:15.330 INFO:teuthology.orchestra.run.smithi146.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T18:13:15.330 INFO:teuthology.orchestra.run.smithi146.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-13T18:13:15.331 INFO:teuthology.orchestra.run.smithi146.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-13T18:13:15.332 INFO:teuthology.orchestra.run.smithi146.stdout:The following packages will be upgraded: 2024-09-13T18:13:15.333 INFO:teuthology.orchestra.run.smithi146.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T18:13:15.333 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2024-09-13T18:13:15.333 INFO:teuthology.orchestra.run.smithi154.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T18:13:15.333 INFO:teuthology.orchestra.run.smithi154.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-13T18:13:15.334 INFO:teuthology.orchestra.run.smithi154.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-13T18:13:15.334 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2024-09-13T18:13:15.335 INFO:teuthology.orchestra.run.smithi154.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-13T18:13:15.335 INFO:teuthology.orchestra.run.smithi154.stdout:Recommended packages: 2024-09-13T18:13:15.335 INFO:teuthology.orchestra.run.smithi154.stdout: thermald 2024-09-13T18:13:15.390 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2024-09-13T18:13:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T18:13:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-13T18:13:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-13T18:13:15.392 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages will be upgraded: 2024-09-13T18:13:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T18:13:15.515 INFO:teuthology.orchestra.run.smithi146.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T18:13:15.516 INFO:teuthology.orchestra.run.smithi146.stdout:Need to get 77.2 MB of archives. 2024-09-13T18:13:15.516 INFO:teuthology.orchestra.run.smithi146.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-13T18:13:15.516 INFO:teuthology.orchestra.run.smithi146.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-13T18:13:15.578 INFO:teuthology.orchestra.run.smithi154.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T18:13:15.578 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 77.2 MB of archives. 2024-09-13T18:13:15.578 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-13T18:13:15.578 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-13T18:13:16.689 INFO:teuthology.orchestra.run.smithi146.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-13T18:13:16.728 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-13T18:13:16.961 INFO:teuthology.orchestra.run.smithi146.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-13T18:13:17.030 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-13T18:13:18.318 INFO:teuthology.orchestra.run.smithi146.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-13T18:13:18.318 INFO:teuthology.orchestra.run.smithi146.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-13T18:13:18.319 INFO:teuthology.orchestra.run.smithi146.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-13T18:13:18.363 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-13T18:13:18.363 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-13T18:13:18.363 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-13T18:13:18.601 INFO:teuthology.orchestra.run.smithi146.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-13T18:13:18.614 INFO:teuthology.orchestra.run.smithi146.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-13T18:13:18.644 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-13T18:13:18.708 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-13T18:13:19.277 INFO:teuthology.orchestra.run.smithi146.stdout:Fetched 77.2 MB in 3s (23.6 MB/s) 2024-09-13T18:13:19.404 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 77.2 MB in 3s (23.3 MB/s) 2024-09-13T18:13:19.465 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-13T18:13:19.597 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-13T18:13:21.184 INFO:teuthology.orchestra.run.smithi146.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-13T18:13:21.189 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:13:21.278 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:21.327 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-13T18:13:21.332 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:13:21.469 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:23.186 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-13T18:13:23.202 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:13:23.349 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:23.478 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-13T18:13:23.493 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:13:23.591 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:24.405 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-13T18:13:24.419 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:13:24.463 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:24.698 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-13T18:13:24.713 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:13:24.781 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:29.200 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T18:13:29.325 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T18:13:29.676 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T18:13:29.739 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T18:13:29.803 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T18:13:29.888 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T18:13:30.156 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-13T18:13:30.171 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-13T18:13:30.214 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T18:13:30.255 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T18:13:30.383 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T18:13:30.745 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-13T18:13:30.761 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-13T18:13:30.803 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T18:13:35.515 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-13T18:13:35.537 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:13:35.581 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:36.271 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-13T18:13:36.294 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:13:36.337 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:38.216 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T18:13:38.343 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T18:13:39.244 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T18:13:39.399 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T18:13:39.759 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:39.777 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:41.012 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:41.057 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:42.184 INFO:teuthology.orchestra.run.smithi154.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-13T18:13:42.184 INFO:teuthology.orchestra.run.smithi154.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-13T18:13:42.218 INFO:teuthology.orchestra.run.smithi146.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-13T18:13:42.218 INFO:teuthology.orchestra.run.smithi146.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-13T18:13:42.340 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:42.359 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:43.528 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T18:13:43.628 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-13T18:13:43.746 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:43.872 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-13T18:13:44.032 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-13T18:13:44.157 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:44.229 INFO:teuthology.orchestra.run.smithi146.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T18:13:44.229 INFO:teuthology.orchestra.run.smithi146.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-13T18:13:44.758 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T18:13:44.884 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-13T18:13:45.043 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:45.169 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-13T18:13:45.320 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-13T18:13:45.447 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:13:45.550 INFO:teuthology.orchestra.run.smithi154.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T18:13:45.550 INFO:teuthology.orchestra.run.smithi154.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-13T18:14:05.914 INFO:teuthology.orchestra.run.smithi146.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T18:14:05.914 INFO:teuthology.orchestra.run.smithi146.stdout:Sourcing file `/etc/default/grub' 2024-09-13T18:14:05.922 INFO:teuthology.orchestra.run.smithi146.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T18:14:05.941 INFO:teuthology.orchestra.run.smithi146.stdout:Generating grub configuration file ... 2024-09-13T18:14:06.242 INFO:teuthology.orchestra.run.smithi146.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T18:14:06.274 INFO:teuthology.orchestra.run.smithi146.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T18:14:06.624 INFO:teuthology.orchestra.run.smithi146.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T18:14:06.635 INFO:teuthology.orchestra.run.smithi146.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T18:14:07.364 INFO:teuthology.orchestra.run.smithi154.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T18:14:07.364 INFO:teuthology.orchestra.run.smithi154.stdout:Sourcing file `/etc/default/grub' 2024-09-13T18:14:07.380 INFO:teuthology.orchestra.run.smithi154.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T18:14:07.401 INFO:teuthology.orchestra.run.smithi154.stdout:Generating grub configuration file ... 2024-09-13T18:14:07.758 INFO:teuthology.orchestra.run.smithi154.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T18:14:07.779 INFO:teuthology.orchestra.run.smithi154.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T18:14:08.184 INFO:teuthology.orchestra.run.smithi154.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T18:14:08.196 INFO:teuthology.orchestra.run.smithi154.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T18:14:08.425 INFO:teuthology.orchestra.run.smithi146.stdout:done 2024-09-13T18:14:08.863 DEBUG:teuthology.orchestra.run.smithi146:> dpkg -s linux-image-generic 2024-09-13T18:14:08.919 INFO:teuthology.orchestra.run.smithi146.stdout:Package: linux-image-generic 2024-09-13T18:14:08.919 INFO:teuthology.orchestra.run.smithi146.stdout:Status: install ok installed 2024-09-13T18:14:08.919 INFO:teuthology.orchestra.run.smithi146.stdout:Priority: optional 2024-09-13T18:14:08.919 INFO:teuthology.orchestra.run.smithi146.stdout:Section: kernel 2024-09-13T18:14:08.919 INFO:teuthology.orchestra.run.smithi146.stdout:Installed-Size: 20 2024-09-13T18:14:08.919 INFO:teuthology.orchestra.run.smithi146.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T18:14:08.920 INFO:teuthology.orchestra.run.smithi146.stdout:Architecture: amd64 2024-09-13T18:14:08.920 INFO:teuthology.orchestra.run.smithi146.stdout:Source: linux-meta 2024-09-13T18:14:08.920 INFO:teuthology.orchestra.run.smithi146.stdout:Version: 5.4.0.195.193 2024-09-13T18:14:08.920 INFO:teuthology.orchestra.run.smithi146.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T18:14:08.920 INFO:teuthology.orchestra.run.smithi146.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T18:14:08.920 INFO:teuthology.orchestra.run.smithi146.stdout:Recommends: thermald 2024-09-13T18:14:08.920 INFO:teuthology.orchestra.run.smithi146.stdout:Description: Generic Linux kernel image 2024-09-13T18:14:08.920 INFO:teuthology.orchestra.run.smithi146.stdout: This package will always depend on the latest generic kernel image 2024-09-13T18:14:08.920 INFO:teuthology.orchestra.run.smithi146.stdout: available. 2024-09-13T18:14:08.921 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-13T18:14:08.921 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T18:14:08.921 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T18:14:08.921 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T18:14:08.921 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi146.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T18:14:08.921 DEBUG:teuthology.orchestra.run.smithi146:> sudo apt-get clean 2024-09-13T18:14:09.011 DEBUG:teuthology.orchestra.run.smithi146:> sudo apt-get update 2024-09-13T18:14:09.138 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T18:14:09.146 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T18:14:09.147 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:3 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T18:14:09.163 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T18:14:10.945 INFO:teuthology.orchestra.run.smithi154.stdout:done 2024-09-13T18:14:11.000 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-09-13T18:14:11.025 DEBUG:teuthology.orchestra.run.smithi146:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T18:14:11.095 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-09-13T18:14:11.313 INFO:teuthology.orchestra.run.smithi146.stdout:Building dependency tree... 2024-09-13T18:14:11.314 INFO:teuthology.orchestra.run.smithi146.stdout:Reading state information... 2024-09-13T18:14:11.413 DEBUG:teuthology.orchestra.run.smithi154:> dpkg -s linux-image-generic 2024-09-13T18:14:11.480 INFO:teuthology.orchestra.run.smithi154.stdout:Package: linux-image-generic 2024-09-13T18:14:11.480 INFO:teuthology.orchestra.run.smithi154.stdout:Status: install ok installed 2024-09-13T18:14:11.480 INFO:teuthology.orchestra.run.smithi154.stdout:Priority: optional 2024-09-13T18:14:11.480 INFO:teuthology.orchestra.run.smithi154.stdout:Section: kernel 2024-09-13T18:14:11.480 INFO:teuthology.orchestra.run.smithi154.stdout:Installed-Size: 20 2024-09-13T18:14:11.480 INFO:teuthology.orchestra.run.smithi154.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T18:14:11.480 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: amd64 2024-09-13T18:14:11.480 INFO:teuthology.orchestra.run.smithi154.stdout:Source: linux-meta 2024-09-13T18:14:11.480 INFO:teuthology.orchestra.run.smithi154.stdout:Version: 5.4.0.195.193 2024-09-13T18:14:11.480 INFO:teuthology.orchestra.run.smithi154.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T18:14:11.481 INFO:teuthology.orchestra.run.smithi154.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T18:14:11.481 INFO:teuthology.orchestra.run.smithi154.stdout:Recommends: thermald 2024-09-13T18:14:11.481 INFO:teuthology.orchestra.run.smithi154.stdout:Description: Generic Linux kernel image 2024-09-13T18:14:11.481 INFO:teuthology.orchestra.run.smithi154.stdout: This package will always depend on the latest generic kernel image 2024-09-13T18:14:11.481 INFO:teuthology.orchestra.run.smithi154.stdout: available. 2024-09-13T18:14:11.481 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-13T18:14:11.481 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T18:14:11.481 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T18:14:11.481 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T18:14:11.481 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi154.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T18:14:11.482 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get clean 2024-09-13T18:14:11.552 INFO:teuthology.orchestra.run.smithi146.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-13T18:14:11.552 INFO:teuthology.orchestra.run.smithi146.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T18:14:11.554 DEBUG:teuthology.orchestra.run.smithi146:> dpkg -s linux-image-generic 2024-09-13T18:14:11.568 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2024-09-13T18:14:11.575 INFO:teuthology.orchestra.run.smithi146.stdout:Package: linux-image-generic 2024-09-13T18:14:11.575 INFO:teuthology.orchestra.run.smithi146.stdout:Status: install ok installed 2024-09-13T18:14:11.575 INFO:teuthology.orchestra.run.smithi146.stdout:Priority: optional 2024-09-13T18:14:11.575 INFO:teuthology.orchestra.run.smithi146.stdout:Section: kernel 2024-09-13T18:14:11.575 INFO:teuthology.orchestra.run.smithi146.stdout:Installed-Size: 20 2024-09-13T18:14:11.575 INFO:teuthology.orchestra.run.smithi146.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T18:14:11.575 INFO:teuthology.orchestra.run.smithi146.stdout:Architecture: amd64 2024-09-13T18:14:11.575 INFO:teuthology.orchestra.run.smithi146.stdout:Source: linux-meta 2024-09-13T18:14:11.575 INFO:teuthology.orchestra.run.smithi146.stdout:Version: 5.4.0.195.193 2024-09-13T18:14:11.576 INFO:teuthology.orchestra.run.smithi146.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T18:14:11.576 INFO:teuthology.orchestra.run.smithi146.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T18:14:11.576 INFO:teuthology.orchestra.run.smithi146.stdout:Recommends: thermald 2024-09-13T18:14:11.576 INFO:teuthology.orchestra.run.smithi146.stdout:Description: Generic Linux kernel image 2024-09-13T18:14:11.576 INFO:teuthology.orchestra.run.smithi146.stdout: This package will always depend on the latest generic kernel image 2024-09-13T18:14:11.576 INFO:teuthology.orchestra.run.smithi146.stdout: available. 2024-09-13T18:14:11.576 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-09-13T18:14:11.624 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.SxhkIQktAv 2024-09-13T18:14:11.625 DEBUG:teuthology.orchestra.run.smithi146:> sudo cp /boot/grub/grub.cfg /tmp/tmp.SxhkIQktAv 2024-09-13T18:14:11.649 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.SxhkIQktAv 2024-09-13T18:14:11.688 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.SxhkIQktAv is 9KB 2024-09-13T18:14:11.703 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.SxhkIQktAv 2024-09-13T18:14:11.710 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T18:14:11.765 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:14:11.766 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T18:14:11.766 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T18:14:11.796 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-13T18:14:11.796 DEBUG:teuthology.orchestra.run.smithi146:> sudo update-grub 2024-09-13T18:14:11.830 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T18:14:11.831 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T18:14:11.901 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T18:14:11.990 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T18:14:13.140 INFO:teuthology.orchestra.run.smithi146.stderr:Sourcing file `/etc/default/grub' 2024-09-13T18:14:13.150 INFO:teuthology.orchestra.run.smithi146.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T18:14:13.165 INFO:teuthology.orchestra.run.smithi146.stderr:Generating grub configuration file ... 2024-09-13T18:14:13.410 INFO:teuthology.orchestra.run.smithi146.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T18:14:13.422 INFO:teuthology.orchestra.run.smithi146.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T18:14:13.679 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-09-13T18:14:13.705 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T18:14:13.775 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-09-13T18:14:13.784 INFO:teuthology.orchestra.run.smithi146.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T18:14:13.795 INFO:teuthology.orchestra.run.smithi146.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T18:14:13.993 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2024-09-13T18:14:13.993 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2024-09-13T18:14:14.237 INFO:teuthology.orchestra.run.smithi154.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-13T18:14:14.238 INFO:teuthology.orchestra.run.smithi154.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T18:14:14.239 DEBUG:teuthology.orchestra.run.smithi154:> dpkg -s linux-image-generic 2024-09-13T18:14:14.261 INFO:teuthology.orchestra.run.smithi154.stdout:Package: linux-image-generic 2024-09-13T18:14:14.262 INFO:teuthology.orchestra.run.smithi154.stdout:Status: install ok installed 2024-09-13T18:14:14.262 INFO:teuthology.orchestra.run.smithi154.stdout:Priority: optional 2024-09-13T18:14:14.262 INFO:teuthology.orchestra.run.smithi154.stdout:Section: kernel 2024-09-13T18:14:14.262 INFO:teuthology.orchestra.run.smithi154.stdout:Installed-Size: 20 2024-09-13T18:14:14.262 INFO:teuthology.orchestra.run.smithi154.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T18:14:14.262 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: amd64 2024-09-13T18:14:14.262 INFO:teuthology.orchestra.run.smithi154.stdout:Source: linux-meta 2024-09-13T18:14:14.262 INFO:teuthology.orchestra.run.smithi154.stdout:Version: 5.4.0.195.193 2024-09-13T18:14:14.262 INFO:teuthology.orchestra.run.smithi154.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T18:14:14.262 INFO:teuthology.orchestra.run.smithi154.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T18:14:14.262 INFO:teuthology.orchestra.run.smithi154.stdout:Recommends: thermald 2024-09-13T18:14:14.262 INFO:teuthology.orchestra.run.smithi154.stdout:Description: Generic Linux kernel image 2024-09-13T18:14:14.263 INFO:teuthology.orchestra.run.smithi154.stdout: This package will always depend on the latest generic kernel image 2024-09-13T18:14:14.263 INFO:teuthology.orchestra.run.smithi154.stdout: available. 2024-09-13T18:14:14.263 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-09-13T18:14:14.269 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.x7FEe497nF 2024-09-13T18:14:14.269 DEBUG:teuthology.orchestra.run.smithi154:> sudo cp /boot/grub/grub.cfg /tmp/tmp.x7FEe497nF 2024-09-13T18:14:14.280 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.x7FEe497nF 2024-09-13T18:14:14.365 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.x7FEe497nF is 9KB 2024-09-13T18:14:14.380 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.x7FEe497nF 2024-09-13T18:14:14.386 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T18:14:14.443 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:14:14.443 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T18:14:14.443 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T18:14:14.473 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-13T18:14:14.473 DEBUG:teuthology.orchestra.run.smithi154:> sudo update-grub 2024-09-13T18:14:14.542 INFO:teuthology.orchestra.run.smithi146.stderr:done 2024-09-13T18:14:14.544 DEBUG:teuthology.orchestra.run.smithi146:> sudo shutdown -r now 2024-09-13T18:14:15.961 INFO:teuthology.orchestra.run.smithi154.stderr:Sourcing file `/etc/default/grub' 2024-09-13T18:14:15.970 INFO:teuthology.orchestra.run.smithi154.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T18:14:15.983 INFO:teuthology.orchestra.run.smithi154.stderr:Generating grub configuration file ... 2024-09-13T18:14:16.218 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T18:14:16.230 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T18:14:16.634 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T18:14:16.646 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T18:14:18.305 INFO:teuthology.orchestra.run.smithi154.stderr:done 2024-09-13T18:14:18.306 DEBUG:teuthology.orchestra.run.smithi154:> sudo shutdown -r now 2024-09-13T18:14:44.566 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T18:14:44.567 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-13T18:14:44.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:14:48.309 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T18:14:48.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-13T18:14:48.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:15:19.105 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-09-13T18:15:22.945 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2024-09-13T18:15:28.109 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-13T18:15:28.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:15:31.950 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-13T18:15:31.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:15:46.497 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-09-13T18:15:58.509 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-13T18:15:58.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:15:59.213 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-09-13T18:16:00.080 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-13T18:16:00.081 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-195-generic"... 2024-09-13T18:16:00.081 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-09-13T18:16:00.087 INFO:teuthology.orchestra.run.smithi146.stdout:5.4.0-195-generic 2024-09-13T18:16:00.087 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-13T18:16:00.087 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T18:16:00.087 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T18:16:01.088 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T18:16:01.088 DEBUG:teuthology.orchestra.run.smithi146:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T18:16:01.505 INFO:teuthology.orchestra.run.smithi146.stdout:ttyS1 2024-09-13T18:16:01.538 DEBUG:teuthology.parallel:result is None 2024-09-13T18:16:31.962 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T18:16:43.963 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-13T18:16:43.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:16:44.232 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-09-13T18:16:44.798 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-13T18:16:44.798 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-195-generic"... 2024-09-13T18:16:44.798 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-09-13T18:16:44.804 INFO:teuthology.orchestra.run.smithi154.stdout:5.4.0-195-generic 2024-09-13T18:16:44.805 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-13T18:16:44.805 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T18:16:44.805 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T18:16:45.806 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T18:16:45.806 DEBUG:teuthology.orchestra.run.smithi154:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T18:16:45.907 INFO:teuthology.orchestra.run.smithi154.stdout:ttyS1 2024-09-13T18:16:45.940 DEBUG:teuthology.parallel:result is None 2024-09-13T18:16:45.940 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T18:16:45.948 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T18:16:45.948 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T18:16:45.951 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T18:16:45.957 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T18:16:45.963 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T18:16:45.971 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T18:16:45.971 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T18:16:45.973 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T18:16:46.037 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T18:16:46.044 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T18:16:46.045 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T18:16:46.047 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T18:16:46.064 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:16:46.071 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:16:46.075 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:16:46.081 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:16:46.083 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T18:16:46.094 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T18:16:46.094 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T18:16:46.097 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T18:16:46.114 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T18:16:46.124 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T18:16:46.124 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T18:16:46.155 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T18:16:46.164 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T18:16:46.169 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T18:16:46.220 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:16:46.220 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T18:16:46.237 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T18:16:46.244 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T18:16:46.251 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:16:46.251 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T18:16:46.322 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2024-09-13T18:16:46.324 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2024-09-13T18:16:46.380 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T18:16:46.387 INFO:teuthology.task.internal:Starting timer... 2024-09-13T18:16:46.387 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T18:16:46.405 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T18:16:46.414 DEBUG:teuthology.task.selinux:Excluding smithi146: OS 'ubuntu' does not support SELinux 2024-09-13T18:16:46.414 DEBUG:teuthology.task.selinux:Excluding smithi154: OS 'ubuntu' does not support SELinux 2024-09-13T18:16:46.414 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T18:16:46.415 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T18:16:46.415 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T18:16:46.415 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T18:16:46.428 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T18:16:46.442 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T18:16:46.444 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi146.front.sepia.ceph.com,smithi154.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T18:22:28.637 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi146.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com')] 2024-09-13T18:22:28.639 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-13T18:22:28.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:22:28.739 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-09-13T18:22:28.862 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-13T18:22:28.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-13T18:22:28.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:22:28.968 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-09-13T18:22:29.094 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-13T18:22:29.094 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T18:22:29.104 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T18:22:29.105 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T18:22:29.105 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:22:29.108 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T18:22:29.108 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:22:29.139 INFO:teuthology.orchestra.run.smithi154.stdout:13 Sep 18:22:29 ntpd[13740]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T18:22:29.139 INFO:teuthology.orchestra.run.smithi154.stdout:13 Sep 18:22:29 ntpd[13740]: Command line: ntpd -gq 2024-09-13T18:22:29.140 INFO:teuthology.orchestra.run.smithi154.stdout:13 Sep 18:22:29 ntpd[13740]: proto: precision = 0.190 usec (-22) 2024-09-13T18:22:29.140 INFO:teuthology.orchestra.run.smithi154.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T18:22:29.141 INFO:teuthology.orchestra.run.smithi154.stdout:13 Sep 18:22:29 ntpd[13740]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T18:22:29.141 INFO:teuthology.orchestra.run.smithi154.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T18:22:29.142 INFO:teuthology.orchestra.run.smithi154.stdout:13 Sep 18:22:29 ntpd[13740]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T18:22:29.142 INFO:teuthology.orchestra.run.smithi154.stdout:13 Sep 18:22:29 ntpd[13740]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T18:22:29.142 INFO:teuthology.orchestra.run.smithi154.stdout:13 Sep 18:22:29 ntpd[13740]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T18:22:29.142 INFO:teuthology.orchestra.run.smithi154.stdout:13 Sep 18:22:29 ntpd[13740]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T18:22:29.143 INFO:teuthology.orchestra.run.smithi154.stdout:13 Sep 18:22:29 ntpd[13740]: Listen normally on 3 enp3s0f1 172.21.15.154:123 2024-09-13T18:22:29.143 INFO:teuthology.orchestra.run.smithi154.stdout:13 Sep 18:22:29 ntpd[13740]: Listen normally on 4 lo [::1]:123 2024-09-13T18:22:29.143 INFO:teuthology.orchestra.run.smithi154.stdout:13 Sep 18:22:29 ntpd[13740]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7761%5]:123 2024-09-13T18:22:29.143 INFO:teuthology.orchestra.run.smithi154.stdout:13 Sep 18:22:29 ntpd[13740]: Listening on routing socket on fd #22 for interface updates 2024-09-13T18:22:29.143 INFO:teuthology.orchestra.run.smithi146.stdout:13 Sep 18:22:29 ntpd[13739]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T18:22:29.143 INFO:teuthology.orchestra.run.smithi146.stdout:13 Sep 18:22:29 ntpd[13739]: Command line: ntpd -gq 2024-09-13T18:22:29.143 INFO:teuthology.orchestra.run.smithi146.stdout:13 Sep 18:22:29 ntpd[13739]: proto: precision = 0.064 usec (-24) 2024-09-13T18:22:29.144 INFO:teuthology.orchestra.run.smithi146.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T18:22:29.144 INFO:teuthology.orchestra.run.smithi146.stdout:13 Sep 18:22:29 ntpd[13739]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T18:22:29.144 INFO:teuthology.orchestra.run.smithi146.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T18:22:29.145 INFO:teuthology.orchestra.run.smithi146.stdout:13 Sep 18:22:29 ntpd[13739]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T18:22:29.145 INFO:teuthology.orchestra.run.smithi146.stdout:13 Sep 18:22:29 ntpd[13739]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T18:22:29.146 INFO:teuthology.orchestra.run.smithi146.stdout:13 Sep 18:22:29 ntpd[13739]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T18:22:29.146 INFO:teuthology.orchestra.run.smithi146.stdout:13 Sep 18:22:29 ntpd[13739]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T18:22:29.146 INFO:teuthology.orchestra.run.smithi146.stdout:13 Sep 18:22:29 ntpd[13739]: Listen normally on 3 enp3s0f1 172.21.15.146:123 2024-09-13T18:22:29.146 INFO:teuthology.orchestra.run.smithi146.stdout:13 Sep 18:22:29 ntpd[13739]: Listen normally on 4 lo [::1]:123 2024-09-13T18:22:29.146 INFO:teuthology.orchestra.run.smithi146.stdout:13 Sep 18:22:29 ntpd[13739]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:76fd%5]:123 2024-09-13T18:22:29.146 INFO:teuthology.orchestra.run.smithi146.stdout:13 Sep 18:22:29 ntpd[13739]: Listening on routing socket on fd #22 for interface updates 2024-09-13T18:22:30.140 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:30 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:30.140 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:30 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:30.140 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:30 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:30.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:30 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:30.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:30 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:30.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:30 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:30.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:30 ntpd[13740]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:30.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:30 ntpd[13740]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:30.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:30 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:30.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:30 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:30.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:30 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:30.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:30 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:30.143 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:30 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:30.143 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:30 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:30.143 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:30 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:30.143 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:30 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:31.141 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:31 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:31.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:31 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:31.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:31 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:31.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:31 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:31.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:31 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:31.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:31 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:32.140 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:32 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:32.140 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:32 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:32.140 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:32 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:32.140 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:32 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:32.140 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:32 ntpd[13740]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:32.140 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:32 ntpd[13740]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:32.141 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:32 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:32.141 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:32 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:32.141 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:32 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:32.141 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:32 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:32.141 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:32 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:32.141 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:32 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:33.141 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:33 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:33.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:33 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:33.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:33 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:33.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:33 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:33.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:33 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:33.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:33 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:34.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:34.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:34.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:34.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:34.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:34.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:34.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:34.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:34.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:34.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:34.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:34.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:34.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:34.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:34.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:34.143 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:34.143 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:34.143 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:34 ntpd[13740]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:34.144 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:34 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:34.144 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:34 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:34.144 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:34 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:34.145 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:34 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:34.145 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:34 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:34.145 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:34 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:34.145 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:34 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:34.145 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:34 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:34.145 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:34 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:34.145 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:34 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:34.145 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:34 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:34.145 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:34 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:35.141 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:35 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:35.141 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:35 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:35.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:35 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:35.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:35 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:35.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:35 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:35.142 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:35 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:36.140 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:36 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:36.140 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:36 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:36.140 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:36 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:36.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:36 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:36.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:36 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:36.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:36 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:36.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:36 ntpd[13740]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:36.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:36 ntpd[13740]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:36.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:36 ntpd[13740]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:36.141 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:36 ntpd[13740]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:36.141 INFO:teuthology.orchestra.run.smithi154.stdout:13 Sep 18:22:36 ntpd[13740]: ntpd: time slew -0.000819 s 2024-09-13T18:22:36.142 INFO:teuthology.orchestra.run.smithi154.stdout:ntpd: time slew -0.000819s 2024-09-13T18:22:36.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:36 ntpd[13740]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T18:22:36.142 INFO:teuthology.orchestra.run.smithi154.stderr:13 Sep 18:22:36 ntpd[13740]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T18:22:36.143 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:36 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:36.143 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:36 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:36.143 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:36 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:36.143 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:36 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:36.143 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:36 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:22:36.144 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:36 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:22:36.144 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:36 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:22:36.144 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:36 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:22:36.144 INFO:teuthology.orchestra.run.smithi146.stdout:13 Sep 18:22:36 ntpd[13739]: ntpd: time slew +0.009125 s 2024-09-13T18:22:36.144 INFO:teuthology.orchestra.run.smithi146.stdout:ntpd: time slew +0.009125s 2024-09-13T18:22:36.144 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:36 ntpd[13739]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T18:22:36.145 INFO:teuthology.orchestra.run.smithi146.stderr:13 Sep 18:22:36 ntpd[13739]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T18:22:36.211 INFO:teuthology.orchestra.run.smithi154.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T18:22:36.211 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================== 2024-09-13T18:22:36.211 INFO:teuthology.orchestra.run.smithi154.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:22:36.211 INFO:teuthology.orchestra.run.smithi154.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:22:36.211 INFO:teuthology.orchestra.run.smithi154.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:22:36.211 INFO:teuthology.orchestra.run.smithi154.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:22:36.221 INFO:teuthology.orchestra.run.smithi146.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T18:22:36.221 INFO:teuthology.orchestra.run.smithi146.stdout:============================================================================== 2024-09-13T18:22:36.221 INFO:teuthology.orchestra.run.smithi146.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:22:36.221 INFO:teuthology.orchestra.run.smithi146.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:22:36.221 INFO:teuthology.orchestra.run.smithi146.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:22:36.221 INFO:teuthology.orchestra.run.smithi146.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:22:36.222 INFO:teuthology.run_tasks:Running task install... 2024-09-13T18:22:36.231 DEBUG:teuthology.task.install:project ceph 2024-09-13T18:22:36.231 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}} 2024-09-13T18:22:36.231 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'} 2024-09-13T18:22:36.231 INFO:teuthology.task.install:Using flavor: default 2024-09-13T18:22:36.240 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T18:22:36.240 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-09-13T18:22:36.240 DEBUG:teuthology.orchestra.run.smithi146:> sudo apt-key list | grep Ceph 2024-09-13T18:22:36.241 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-key list | grep Ceph 2024-09-13T18:22:36.337 INFO:teuthology.orchestra.run.smithi154.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T18:22:36.340 INFO:teuthology.orchestra.run.smithi146.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T18:22:36.398 INFO:teuthology.orchestra.run.smithi154.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T18:22:36.398 INFO:teuthology.orchestra.run.smithi154.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T18:22:36.399 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-13T18:22:36.399 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-13T18:22:36.399 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:22:36.416 INFO:teuthology.orchestra.run.smithi146.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T18:22:36.416 INFO:teuthology.orchestra.run.smithi146.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T18:22:36.416 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-13T18:22:36.417 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-13T18:22:36.417 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:22:36.554 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default/ 2024-09-13T18:22:36.554 INFO:teuthology.task.install.deb:Package version is 18.2.4-860-g6c6b083f-1focal 2024-09-13T18:22:36.573 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default/ 2024-09-13T18:22:36.573 INFO:teuthology.task.install.deb:Package version is 18.2.4-860-g6c6b083f-1focal 2024-09-13T18:22:36.675 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:22:36.675 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T18:22:36.688 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:22:36.688 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T18:22:36.692 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2024-09-13T18:22:36.704 DEBUG:teuthology.orchestra.run.smithi146:> sudo apt-get update 2024-09-13T18:22:36.893 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T18:22:36.935 INFO:teuthology.orchestra.run.smithi154.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal InRelease 2024-09-13T18:22:36.948 INFO:teuthology.orchestra.run.smithi146.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal InRelease 2024-09-13T18:22:36.957 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T18:22:36.969 INFO:teuthology.orchestra.run.smithi146.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T18:22:36.981 INFO:teuthology.orchestra.run.smithi154.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T18:22:36.993 INFO:teuthology.orchestra.run.smithi146.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T18:22:37.009 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T18:22:37.010 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T18:22:37.010 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:5 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T18:22:37.010 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T18:22:37.014 INFO:teuthology.orchestra.run.smithi146.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main i386 Packages [3,433 B] 2024-09-13T18:22:37.042 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main i386 Packages [3,433 B] 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi146.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T18:22:37.080 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T18:22:37.091 INFO:teuthology.orchestra.run.smithi146.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-13T18:22:37.165 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T18:22:37.445 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T18:22:37.675 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 28.7 kB in 1s (37.6 kB/s) 2024-09-13T18:22:37.744 INFO:teuthology.orchestra.run.smithi146.stdout:Fetched 157 kB in 1s (189 kB/s) 2024-09-13T18:22:38.825 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-09-13T18:22:38.851 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-860-g6c6b083f-1focal cephadm=18.2.4-860-g6c6b083f-1focal ceph-mds=18.2.4-860-g6c6b083f-1focal ceph-mgr=18.2.4-860-g6c6b083f-1focal ceph-common=18.2.4-860-g6c6b083f-1focal ceph-fuse=18.2.4-860-g6c6b083f-1focal ceph-test=18.2.4-860-g6c6b083f-1focal ceph-volume=18.2.4-860-g6c6b083f-1focal radosgw=18.2.4-860-g6c6b083f-1focal python3-rados=18.2.4-860-g6c6b083f-1focal python3-rgw=18.2.4-860-g6c6b083f-1focal python3-cephfs=18.2.4-860-g6c6b083f-1focal python3-rbd=18.2.4-860-g6c6b083f-1focal libcephfs2=18.2.4-860-g6c6b083f-1focal libcephfs-dev=18.2.4-860-g6c6b083f-1focal librados2=18.2.4-860-g6c6b083f-1focal librbd1=18.2.4-860-g6c6b083f-1focal rbd-fuse=18.2.4-860-g6c6b083f-1focal python3-cephfs=18.2.4-860-g6c6b083f-1focal cephfs-shell=18.2.4-860-g6c6b083f-1focal cephfs-top=18.2.4-860-g6c6b083f-1focal cephfs-mirror=18.2.4-860-g6c6b083f-1focal 2024-09-13T18:22:38.895 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-09-13T18:22:38.920 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-09-13T18:22:38.922 DEBUG:teuthology.orchestra.run.smithi146:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-860-g6c6b083f-1focal cephadm=18.2.4-860-g6c6b083f-1focal ceph-mds=18.2.4-860-g6c6b083f-1focal ceph-mgr=18.2.4-860-g6c6b083f-1focal ceph-common=18.2.4-860-g6c6b083f-1focal ceph-fuse=18.2.4-860-g6c6b083f-1focal ceph-test=18.2.4-860-g6c6b083f-1focal ceph-volume=18.2.4-860-g6c6b083f-1focal radosgw=18.2.4-860-g6c6b083f-1focal python3-rados=18.2.4-860-g6c6b083f-1focal python3-rgw=18.2.4-860-g6c6b083f-1focal python3-cephfs=18.2.4-860-g6c6b083f-1focal python3-rbd=18.2.4-860-g6c6b083f-1focal libcephfs2=18.2.4-860-g6c6b083f-1focal libcephfs-dev=18.2.4-860-g6c6b083f-1focal librados2=18.2.4-860-g6c6b083f-1focal librbd1=18.2.4-860-g6c6b083f-1focal rbd-fuse=18.2.4-860-g6c6b083f-1focal python3-cephfs=18.2.4-860-g6c6b083f-1focal cephfs-shell=18.2.4-860-g6c6b083f-1focal cephfs-top=18.2.4-860-g6c6b083f-1focal cephfs-mirror=18.2.4-860-g6c6b083f-1focal 2024-09-13T18:22:38.995 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-09-13T18:22:39.136 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2024-09-13T18:22:39.137 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2024-09-13T18:22:39.208 INFO:teuthology.orchestra.run.smithi146.stdout:Building dependency tree... 2024-09-13T18:22:39.208 INFO:teuthology.orchestra.run.smithi146.stdout:Reading state information... 2024-09-13T18:22:39.342 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T18:22:39.344 INFO:teuthology.orchestra.run.smithi154.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T18:22:39.344 INFO:teuthology.orchestra.run.smithi154.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T18:22:39.344 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2024-09-13T18:22:39.344 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T18:22:39.344 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T18:22:39.345 INFO:teuthology.orchestra.run.smithi154.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T18:22:39.345 INFO:teuthology.orchestra.run.smithi154.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T18:22:39.345 INFO:teuthology.orchestra.run.smithi154.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T18:22:39.346 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T18:22:39.346 INFO:teuthology.orchestra.run.smithi154.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T18:22:39.346 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T18:22:39.346 INFO:teuthology.orchestra.run.smithi154.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T18:22:39.346 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-13T18:22:39.346 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-13T18:22:39.346 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-13T18:22:39.346 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-13T18:22:39.346 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-13T18:22:39.346 INFO:teuthology.orchestra.run.smithi154.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T18:22:39.346 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-13T18:22:39.346 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2024-09-13T18:22:39.347 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T18:22:39.347 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T18:22:39.347 INFO:teuthology.orchestra.run.smithi154.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T18:22:39.347 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T18:22:39.347 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T18:22:39.347 INFO:teuthology.orchestra.run.smithi154.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T18:22:39.347 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2024-09-13T18:22:39.347 INFO:teuthology.orchestra.run.smithi154.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T18:22:39.347 INFO:teuthology.orchestra.run.smithi154.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T18:22:39.347 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T18:22:39.348 INFO:teuthology.orchestra.run.smithi154.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T18:22:39.348 INFO:teuthology.orchestra.run.smithi154.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T18:22:39.348 INFO:teuthology.orchestra.run.smithi154.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T18:22:39.348 INFO:teuthology.orchestra.run.smithi154.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T18:22:39.348 INFO:teuthology.orchestra.run.smithi154.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T18:22:39.348 INFO:teuthology.orchestra.run.smithi154.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T18:22:39.348 INFO:teuthology.orchestra.run.smithi154.stdout:Recommended packages: 2024-09-13T18:22:39.348 INFO:teuthology.orchestra.run.smithi154.stdout: btrfs-tools 2024-09-13T18:22:39.399 INFO:teuthology.orchestra.run.smithi146.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T18:22:39.400 INFO:teuthology.orchestra.run.smithi146.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T18:22:39.401 INFO:teuthology.orchestra.run.smithi146.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T18:22:39.401 INFO:teuthology.orchestra.run.smithi146.stdout:The following additional packages will be installed: 2024-09-13T18:22:39.401 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T18:22:39.401 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T18:22:39.401 INFO:teuthology.orchestra.run.smithi146.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T18:22:39.401 INFO:teuthology.orchestra.run.smithi146.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T18:22:39.402 INFO:teuthology.orchestra.run.smithi146.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T18:22:39.402 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T18:22:39.403 INFO:teuthology.orchestra.run.smithi146.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T18:22:39.403 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T18:22:39.403 INFO:teuthology.orchestra.run.smithi146.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T18:22:39.403 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-13T18:22:39.403 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-13T18:22:39.403 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-13T18:22:39.403 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-13T18:22:39.403 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-13T18:22:39.403 INFO:teuthology.orchestra.run.smithi146.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T18:22:39.403 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-13T18:22:39.403 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2024-09-13T18:22:39.403 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T18:22:39.404 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T18:22:39.404 INFO:teuthology.orchestra.run.smithi146.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T18:22:39.404 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T18:22:39.404 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T18:22:39.404 INFO:teuthology.orchestra.run.smithi146.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T18:22:39.404 INFO:teuthology.orchestra.run.smithi146.stdout:Suggested packages: 2024-09-13T18:22:39.404 INFO:teuthology.orchestra.run.smithi146.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T18:22:39.404 INFO:teuthology.orchestra.run.smithi146.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T18:22:39.404 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T18:22:39.404 INFO:teuthology.orchestra.run.smithi146.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T18:22:39.405 INFO:teuthology.orchestra.run.smithi146.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T18:22:39.405 INFO:teuthology.orchestra.run.smithi146.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T18:22:39.405 INFO:teuthology.orchestra.run.smithi146.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T18:22:39.405 INFO:teuthology.orchestra.run.smithi146.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T18:22:39.405 INFO:teuthology.orchestra.run.smithi146.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T18:22:39.405 INFO:teuthology.orchestra.run.smithi146.stdout:Recommended packages: 2024-09-13T18:22:39.405 INFO:teuthology.orchestra.run.smithi146.stdout: btrfs-tools 2024-09-13T18:22:39.476 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2024-09-13T18:22:39.476 INFO:teuthology.orchestra.run.smithi154.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T18:22:39.476 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T18:22:39.476 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T18:22:39.477 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T18:22:39.478 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T18:22:39.478 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-09-13T18:22:39.478 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-09-13T18:22:39.478 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T18:22:39.478 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T18:22:39.478 INFO:teuthology.orchestra.run.smithi154.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T18:22:39.478 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T18:22:39.478 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T18:22:39.478 INFO:teuthology.orchestra.run.smithi154.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-13T18:22:39.478 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages will be upgraded: 2024-09-13T18:22:39.478 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 librbd1 2024-09-13T18:22:39.527 INFO:teuthology.orchestra.run.smithi146.stdout:The following NEW packages will be installed: 2024-09-13T18:22:39.527 INFO:teuthology.orchestra.run.smithi146.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T18:22:39.527 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T18:22:39.527 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T18:22:39.527 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-09-13T18:22:39.527 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-09-13T18:22:39.527 INFO:teuthology.orchestra.run.smithi146.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-09-13T18:22:39.527 INFO:teuthology.orchestra.run.smithi146.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-09-13T18:22:39.528 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-09-13T18:22:39.528 INFO:teuthology.orchestra.run.smithi146.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T18:22:39.528 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T18:22:39.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T18:22:39.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2024-09-13T18:22:39.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-09-13T18:22:39.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T18:22:39.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T18:22:39.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T18:22:39.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T18:22:39.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T18:22:39.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-09-13T18:22:39.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-09-13T18:22:39.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T18:22:39.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T18:22:39.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T18:22:39.530 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T18:22:39.530 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T18:22:39.530 INFO:teuthology.orchestra.run.smithi146.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-13T18:22:39.530 INFO:teuthology.orchestra.run.smithi146.stdout:The following packages will be upgraded: 2024-09-13T18:22:39.530 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 librbd1 2024-09-13T18:22:39.623 INFO:teuthology.orchestra.run.smithi154.stdout:2 upgraded, 116 newly installed, 0 to remove and 295 not upgraded. 2024-09-13T18:22:39.623 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 200 MB of archives. 2024-09-13T18:22:39.623 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 979 MB of additional disk space will be used. 2024-09-13T18:22:39.623 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-860-g6c6b083f-1focal [3,453 kB] 2024-09-13T18:22:39.668 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-09-13T18:22:39.678 INFO:teuthology.orchestra.run.smithi146.stdout:2 upgraded, 116 newly installed, 0 to remove and 295 not upgraded. 2024-09-13T18:22:39.678 INFO:teuthology.orchestra.run.smithi146.stdout:Need to get 200 MB of archives. 2024-09-13T18:22:39.678 INFO:teuthology.orchestra.run.smithi146.stdout:After this operation, 979 MB of additional disk space will be used. 2024-09-13T18:22:39.678 INFO:teuthology.orchestra.run.smithi146.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-860-g6c6b083f-1focal [3,453 kB] 2024-09-13T18:22:39.724 INFO:teuthology.orchestra.run.smithi146.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-09-13T18:22:39.753 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T18:22:39.787 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-09-13T18:22:39.807 INFO:teuthology.orchestra.run.smithi146.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T18:22:39.829 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-860-g6c6b083f-1focal [4,041 kB] 2024-09-13T18:22:39.842 INFO:teuthology.orchestra.run.smithi146.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-09-13T18:22:39.894 INFO:teuthology.orchestra.run.smithi146.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-860-g6c6b083f-1focal [4,041 kB] 2024-09-13T18:22:39.912 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T18:22:39.927 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-860-g6c6b083f-1focal [747 kB] 2024-09-13T18:22:39.938 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-860-g6c6b083f-1focal [388 kB] 2024-09-13T18:22:39.944 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-860-g6c6b083f-1focal [31.9 kB] 2024-09-13T18:22:39.945 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-860-g6c6b083f-1focal [200 kB] 2024-09-13T18:22:39.948 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-860-g6c6b083f-1focal [62.5 kB] 2024-09-13T18:22:39.949 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-860-g6c6b083f-1focal [368 kB] 2024-09-13T18:22:39.967 INFO:teuthology.orchestra.run.smithi146.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T18:22:39.982 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-860-g6c6b083f-1focal [6,634 kB] 2024-09-13T18:22:40.028 INFO:teuthology.orchestra.run.smithi146.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-860-g6c6b083f-1focal [747 kB] 2024-09-13T18:22:40.042 INFO:teuthology.orchestra.run.smithi146.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-860-g6c6b083f-1focal [388 kB] 2024-09-13T18:22:40.042 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T18:22:40.058 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T18:22:40.079 INFO:teuthology.orchestra.run.smithi146.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-860-g6c6b083f-1focal [31.9 kB] 2024-09-13T18:22:40.080 INFO:teuthology.orchestra.run.smithi146.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-860-g6c6b083f-1focal [200 kB] 2024-09-13T18:22:40.083 INFO:teuthology.orchestra.run.smithi146.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-860-g6c6b083f-1focal [62.5 kB] 2024-09-13T18:22:40.084 INFO:teuthology.orchestra.run.smithi146.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-860-g6c6b083f-1focal [368 kB] 2024-09-13T18:22:40.093 INFO:teuthology.orchestra.run.smithi146.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-860-g6c6b083f-1focal [6,634 kB] 2024-09-13T18:22:40.097 INFO:teuthology.orchestra.run.smithi146.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T18:22:40.113 INFO:teuthology.orchestra.run.smithi146.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T18:22:40.119 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T18:22:40.174 INFO:teuthology.orchestra.run.smithi146.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T18:22:40.181 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-860-g6c6b083f-1focal [113 kB] 2024-09-13T18:22:40.182 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-860-g6c6b083f-1focal [478 kB] 2024-09-13T18:22:40.189 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-860-g6c6b083f-1focal [25.2 MB] 2024-09-13T18:22:40.315 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T18:22:40.318 INFO:teuthology.orchestra.run.smithi146.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-860-g6c6b083f-1focal [113 kB] 2024-09-13T18:22:40.320 INFO:teuthology.orchestra.run.smithi146.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-860-g6c6b083f-1focal [478 kB] 2024-09-13T18:22:40.324 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T18:22:40.349 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T18:22:40.359 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T18:22:40.360 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T18:22:40.364 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T18:22:40.364 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T18:22:40.366 INFO:teuthology.orchestra.run.smithi146.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-860-g6c6b083f-1focal [25.2 MB] 2024-09-13T18:22:40.371 INFO:teuthology.orchestra.run.smithi146.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T18:22:40.377 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T18:22:40.377 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T18:22:40.380 INFO:teuthology.orchestra.run.smithi146.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T18:22:40.397 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T18:22:40.405 INFO:teuthology.orchestra.run.smithi146.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T18:22:40.413 INFO:teuthology.orchestra.run.smithi146.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T18:22:40.414 INFO:teuthology.orchestra.run.smithi146.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T18:22:40.419 INFO:teuthology.orchestra.run.smithi146.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T18:22:40.420 INFO:teuthology.orchestra.run.smithi146.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T18:22:40.432 INFO:teuthology.orchestra.run.smithi146.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T18:22:40.433 INFO:teuthology.orchestra.run.smithi146.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T18:22:40.453 INFO:teuthology.orchestra.run.smithi146.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T18:22:40.464 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T18:22:40.465 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T18:22:40.466 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T18:22:40.468 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T18:22:40.469 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T18:22:40.470 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T18:22:40.472 INFO:teuthology.orchestra.run.smithi154.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T18:22:40.473 INFO:teuthology.orchestra.run.smithi154.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T18:22:40.479 INFO:teuthology.orchestra.run.smithi154.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T18:22:40.523 INFO:teuthology.orchestra.run.smithi146.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T18:22:40.524 INFO:teuthology.orchestra.run.smithi146.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T18:22:40.525 INFO:teuthology.orchestra.run.smithi146.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T18:22:40.527 INFO:teuthology.orchestra.run.smithi146.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T18:22:40.528 INFO:teuthology.orchestra.run.smithi146.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T18:22:40.528 INFO:teuthology.orchestra.run.smithi146.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T18:22:40.531 INFO:teuthology.orchestra.run.smithi146.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T18:22:40.532 INFO:teuthology.orchestra.run.smithi146.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T18:22:40.536 INFO:teuthology.orchestra.run.smithi146.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T18:22:40.560 INFO:teuthology.orchestra.run.smithi154.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T18:22:40.561 INFO:teuthology.orchestra.run.smithi154.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T18:22:40.569 INFO:teuthology.orchestra.run.smithi154.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T18:22:40.570 INFO:teuthology.orchestra.run.smithi154.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T18:22:40.571 INFO:teuthology.orchestra.run.smithi154.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T18:22:40.572 INFO:teuthology.orchestra.run.smithi154.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T18:22:40.573 INFO:teuthology.orchestra.run.smithi154.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T18:22:40.575 INFO:teuthology.orchestra.run.smithi154.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T18:22:40.617 INFO:teuthology.orchestra.run.smithi146.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T18:22:40.617 INFO:teuthology.orchestra.run.smithi146.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T18:22:40.626 INFO:teuthology.orchestra.run.smithi146.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T18:22:40.627 INFO:teuthology.orchestra.run.smithi146.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T18:22:40.628 INFO:teuthology.orchestra.run.smithi146.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T18:22:40.629 INFO:teuthology.orchestra.run.smithi146.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T18:22:40.630 INFO:teuthology.orchestra.run.smithi146.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T18:22:40.633 INFO:teuthology.orchestra.run.smithi146.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T18:22:40.641 INFO:teuthology.orchestra.run.smithi154.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T18:22:40.646 INFO:teuthology.orchestra.run.smithi154.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T18:22:40.698 INFO:teuthology.orchestra.run.smithi146.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T18:22:40.703 INFO:teuthology.orchestra.run.smithi146.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T18:22:40.723 INFO:teuthology.orchestra.run.smithi154.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T18:22:40.725 INFO:teuthology.orchestra.run.smithi154.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T18:22:40.728 INFO:teuthology.orchestra.run.smithi154.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T18:22:40.729 INFO:teuthology.orchestra.run.smithi154.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T18:22:40.731 INFO:teuthology.orchestra.run.smithi154.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T18:22:40.731 INFO:teuthology.orchestra.run.smithi154.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T18:22:40.734 INFO:teuthology.orchestra.run.smithi154.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T18:22:40.735 INFO:teuthology.orchestra.run.smithi154.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T18:22:40.780 INFO:teuthology.orchestra.run.smithi146.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T18:22:40.783 INFO:teuthology.orchestra.run.smithi146.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T18:22:40.786 INFO:teuthology.orchestra.run.smithi146.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T18:22:40.786 INFO:teuthology.orchestra.run.smithi146.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T18:22:40.788 INFO:teuthology.orchestra.run.smithi146.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T18:22:40.788 INFO:teuthology.orchestra.run.smithi146.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T18:22:40.790 INFO:teuthology.orchestra.run.smithi146.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T18:22:40.791 INFO:teuthology.orchestra.run.smithi146.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T18:22:40.804 INFO:teuthology.orchestra.run.smithi154.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T18:22:40.807 INFO:teuthology.orchestra.run.smithi154.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T18:22:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T18:22:40.865 INFO:teuthology.orchestra.run.smithi146.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T18:22:40.885 INFO:teuthology.orchestra.run.smithi154.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T18:22:40.888 INFO:teuthology.orchestra.run.smithi154.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T18:22:40.890 INFO:teuthology.orchestra.run.smithi154.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T18:22:40.893 INFO:teuthology.orchestra.run.smithi154.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T18:22:40.894 INFO:teuthology.orchestra.run.smithi154.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T18:22:40.895 INFO:teuthology.orchestra.run.smithi154.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T18:22:40.943 INFO:teuthology.orchestra.run.smithi146.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T18:22:40.946 INFO:teuthology.orchestra.run.smithi146.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T18:22:40.948 INFO:teuthology.orchestra.run.smithi146.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T18:22:40.950 INFO:teuthology.orchestra.run.smithi146.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T18:22:40.951 INFO:teuthology.orchestra.run.smithi146.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T18:22:40.952 INFO:teuthology.orchestra.run.smithi146.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T18:22:41.121 INFO:teuthology.orchestra.run.smithi154.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-860-g6c6b083f-1focal [9,609 kB] 2024-09-13T18:22:41.217 INFO:teuthology.orchestra.run.smithi154.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T18:22:41.220 INFO:teuthology.orchestra.run.smithi146.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-860-g6c6b083f-1focal [9,609 kB] 2024-09-13T18:22:41.246 INFO:teuthology.orchestra.run.smithi154.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T18:22:41.275 INFO:teuthology.orchestra.run.smithi146.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T18:22:41.294 INFO:teuthology.orchestra.run.smithi154.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T18:22:41.305 INFO:teuthology.orchestra.run.smithi146.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T18:22:41.307 INFO:teuthology.orchestra.run.smithi146.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T18:22:41.315 INFO:teuthology.orchestra.run.smithi154.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T18:22:41.316 INFO:teuthology.orchestra.run.smithi154.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T18:22:41.317 INFO:teuthology.orchestra.run.smithi154.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T18:22:41.317 INFO:teuthology.orchestra.run.smithi154.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T18:22:41.317 INFO:teuthology.orchestra.run.smithi154.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T18:22:41.318 INFO:teuthology.orchestra.run.smithi154.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T18:22:41.318 INFO:teuthology.orchestra.run.smithi154.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T18:22:41.342 INFO:teuthology.orchestra.run.smithi154.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-860-g6c6b083f-1focal [238 kB] 2024-09-13T18:22:41.363 INFO:teuthology.orchestra.run.smithi154.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-860-g6c6b083f-1focal [147 kB] 2024-09-13T18:22:41.367 INFO:teuthology.orchestra.run.smithi154.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-860-g6c6b083f-1focal [1,576 kB] 2024-09-13T18:22:41.376 INFO:teuthology.orchestra.run.smithi146.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T18:22:41.376 INFO:teuthology.orchestra.run.smithi146.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T18:22:41.376 INFO:teuthology.orchestra.run.smithi146.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T18:22:41.376 INFO:teuthology.orchestra.run.smithi146.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T18:22:41.377 INFO:teuthology.orchestra.run.smithi146.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T18:22:41.377 INFO:teuthology.orchestra.run.smithi146.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T18:22:41.378 INFO:teuthology.orchestra.run.smithi146.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T18:22:41.381 INFO:teuthology.orchestra.run.smithi154.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T18:22:41.382 INFO:teuthology.orchestra.run.smithi154.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T18:22:41.419 INFO:teuthology.orchestra.run.smithi154.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-860-g6c6b083f-1focal [6,641 kB] 2024-09-13T18:22:41.442 INFO:teuthology.orchestra.run.smithi146.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T18:22:41.443 INFO:teuthology.orchestra.run.smithi146.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T18:22:41.462 INFO:teuthology.orchestra.run.smithi154.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T18:22:41.467 INFO:teuthology.orchestra.run.smithi154.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T18:22:41.470 INFO:teuthology.orchestra.run.smithi146.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-860-g6c6b083f-1focal [238 kB] 2024-09-13T18:22:41.471 INFO:teuthology.orchestra.run.smithi154.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T18:22:41.474 INFO:teuthology.orchestra.run.smithi154.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T18:22:41.476 INFO:teuthology.orchestra.run.smithi154.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T18:22:41.479 INFO:teuthology.orchestra.run.smithi154.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T18:22:41.481 INFO:teuthology.orchestra.run.smithi154.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-09-13T18:22:41.482 INFO:teuthology.orchestra.run.smithi154.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T18:22:41.490 INFO:teuthology.orchestra.run.smithi146.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-860-g6c6b083f-1focal [147 kB] 2024-09-13T18:22:41.493 INFO:teuthology.orchestra.run.smithi146.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-860-g6c6b083f-1focal [1,576 kB] 2024-09-13T18:22:41.523 INFO:teuthology.orchestra.run.smithi146.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T18:22:41.527 INFO:teuthology.orchestra.run.smithi146.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T18:22:41.533 INFO:teuthology.orchestra.run.smithi146.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T18:22:41.533 INFO:teuthology.orchestra.run.smithi146.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T18:22:41.533 INFO:teuthology.orchestra.run.smithi146.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T18:22:41.537 INFO:teuthology.orchestra.run.smithi146.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T18:22:41.539 INFO:teuthology.orchestra.run.smithi146.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-09-13T18:22:41.539 INFO:teuthology.orchestra.run.smithi146.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T18:22:41.546 INFO:teuthology.orchestra.run.smithi146.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-860-g6c6b083f-1focal [6,641 kB] 2024-09-13T18:22:41.551 INFO:teuthology.orchestra.run.smithi154.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T18:22:41.556 INFO:teuthology.orchestra.run.smithi154.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T18:22:41.613 INFO:teuthology.orchestra.run.smithi146.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T18:22:41.618 INFO:teuthology.orchestra.run.smithi146.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T18:22:41.625 INFO:teuthology.orchestra.run.smithi154.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T18:22:41.628 INFO:teuthology.orchestra.run.smithi154.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-860-g6c6b083f-1focal [24.5 MB] 2024-09-13T18:22:41.687 INFO:teuthology.orchestra.run.smithi146.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T18:22:41.753 INFO:teuthology.orchestra.run.smithi146.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-860-g6c6b083f-1focal [24.5 MB] 2024-09-13T18:22:41.791 INFO:teuthology.orchestra.run.smithi154.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T18:22:41.793 INFO:teuthology.orchestra.run.smithi154.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T18:22:41.794 INFO:teuthology.orchestra.run.smithi154.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T18:22:41.795 INFO:teuthology.orchestra.run.smithi154.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T18:22:41.799 INFO:teuthology.orchestra.run.smithi146.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T18:22:41.811 INFO:teuthology.orchestra.run.smithi154.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T18:22:41.811 INFO:teuthology.orchestra.run.smithi154.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T18:22:41.812 INFO:teuthology.orchestra.run.smithi154.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T18:22:41.813 INFO:teuthology.orchestra.run.smithi154.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T18:22:41.838 INFO:teuthology.orchestra.run.smithi154.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T18:22:41.853 INFO:teuthology.orchestra.run.smithi146.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T18:22:41.853 INFO:teuthology.orchestra.run.smithi146.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T18:22:41.854 INFO:teuthology.orchestra.run.smithi146.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T18:22:41.870 INFO:teuthology.orchestra.run.smithi146.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T18:22:41.871 INFO:teuthology.orchestra.run.smithi146.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T18:22:41.872 INFO:teuthology.orchestra.run.smithi146.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T18:22:41.872 INFO:teuthology.orchestra.run.smithi146.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T18:22:41.873 INFO:teuthology.orchestra.run.smithi146.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T18:22:42.525 INFO:teuthology.orchestra.run.smithi154.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-860-g6c6b083f-1focal [13.7 kB] 2024-09-13T18:22:42.584 INFO:teuthology.orchestra.run.smithi154.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-860-g6c6b083f-1focal [857 kB] 2024-09-13T18:22:42.599 INFO:teuthology.orchestra.run.smithi154.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-860-g6c6b083f-1focal [2,296 kB] 2024-09-13T18:22:42.615 INFO:teuthology.orchestra.run.smithi154.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-860-g6c6b083f-1focal [239 kB] 2024-09-13T18:22:42.617 INFO:teuthology.orchestra.run.smithi154.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-860-g6c6b083f-1focal [143 kB] 2024-09-13T18:22:42.618 INFO:teuthology.orchestra.run.smithi154.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-860-g6c6b083f-1focal [7,015 kB] 2024-09-13T18:22:42.640 INFO:teuthology.orchestra.run.smithi146.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-860-g6c6b083f-1focal [13.7 kB] 2024-09-13T18:22:42.641 INFO:teuthology.orchestra.run.smithi146.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-860-g6c6b083f-1focal [857 kB] 2024-09-13T18:22:42.653 INFO:teuthology.orchestra.run.smithi146.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-860-g6c6b083f-1focal [2,296 kB] 2024-09-13T18:22:42.683 INFO:teuthology.orchestra.run.smithi146.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-860-g6c6b083f-1focal [239 kB] 2024-09-13T18:22:42.686 INFO:teuthology.orchestra.run.smithi146.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-860-g6c6b083f-1focal [143 kB] 2024-09-13T18:22:42.688 INFO:teuthology.orchestra.run.smithi146.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-860-g6c6b083f-1focal [7,015 kB] 2024-09-13T18:22:42.799 INFO:teuthology.orchestra.run.smithi154.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-860-g6c6b083f-1focal [7,894 kB] 2024-09-13T18:22:42.899 INFO:teuthology.orchestra.run.smithi146.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-860-g6c6b083f-1focal [7,894 kB] 2024-09-13T18:22:43.046 INFO:teuthology.orchestra.run.smithi154.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-860-g6c6b083f-1focal [13.9 kB] 2024-09-13T18:22:43.046 INFO:teuthology.orchestra.run.smithi154.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-860-g6c6b083f-1focal [47.1 MB] 2024-09-13T18:22:43.164 INFO:teuthology.orchestra.run.smithi146.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-860-g6c6b083f-1focal [13.9 kB] 2024-09-13T18:22:43.164 INFO:teuthology.orchestra.run.smithi146.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-860-g6c6b083f-1focal [47.1 MB] 2024-09-13T18:22:44.720 INFO:teuthology.orchestra.run.smithi154.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-860-g6c6b083f-1focal [120 kB] 2024-09-13T18:22:44.722 INFO:teuthology.orchestra.run.smithi154.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.4-860-g6c6b083f-1focal [271 kB] 2024-09-13T18:22:44.727 INFO:teuthology.orchestra.run.smithi154.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.4-860-g6c6b083f-1focal [33.0 kB] 2024-09-13T18:22:44.791 INFO:teuthology.orchestra.run.smithi154.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.4-860-g6c6b083f-1focal [22.8 kB] 2024-09-13T18:22:44.792 INFO:teuthology.orchestra.run.smithi154.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-860-g6c6b083f-1focal [37.4 kB] 2024-09-13T18:22:44.793 INFO:teuthology.orchestra.run.smithi154.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-860-g6c6b083f-1focal [13.5 MB] 2024-09-13T18:22:44.947 INFO:teuthology.orchestra.run.smithi146.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-860-g6c6b083f-1focal [120 kB] 2024-09-13T18:22:44.949 INFO:teuthology.orchestra.run.smithi146.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.4-860-g6c6b083f-1focal [271 kB] 2024-09-13T18:22:44.954 INFO:teuthology.orchestra.run.smithi146.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.4-860-g6c6b083f-1focal [33.0 kB] 2024-09-13T18:22:44.955 INFO:teuthology.orchestra.run.smithi146.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.4-860-g6c6b083f-1focal [22.8 kB] 2024-09-13T18:22:44.956 INFO:teuthology.orchestra.run.smithi146.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-860-g6c6b083f-1focal [37.4 kB] 2024-09-13T18:22:44.957 INFO:teuthology.orchestra.run.smithi146.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-860-g6c6b083f-1focal [13.5 MB] 2024-09-13T18:22:44.972 INFO:teuthology.orchestra.run.smithi154.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-860-g6c6b083f-1focal [108 kB] 2024-09-13T18:22:45.135 INFO:teuthology.orchestra.run.smithi146.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-860-g6c6b083f-1focal [108 kB] 2024-09-13T18:22:45.623 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 200 MB in 5s (36.4 MB/s) 2024-09-13T18:22:45.768 INFO:teuthology.orchestra.run.smithi146.stdout:Fetched 200 MB in 6s (35.6 MB/s) 2024-09-13T18:22:45.794 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-13T18:22:45.852 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-13T18:22:45.856 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-09-13T18:22:45.877 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-13T18:22:45.901 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-09-13T18:22:45.936 INFO:teuthology.orchestra.run.smithi146.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-13T18:22:45.940 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-09-13T18:22:45.984 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-09-13T18:22:46.212 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T18:22:46.228 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T18:22:46.262 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T18:22:46.286 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T18:22:46.302 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T18:22:46.345 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T18:22:46.623 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cmd2. 2024-09-13T18:22:46.640 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-09-13T18:22:46.647 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-cmd2. 2024-09-13T18:22:46.663 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-09-13T18:22:46.681 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-09-13T18:22:46.705 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-09-13T18:22:47.050 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T18:22:47.059 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T18:22:47.068 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T18:22:47.075 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T18:22:47.150 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T18:22:47.184 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T18:22:47.445 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T18:22:47.462 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T18:22:47.495 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T18:22:47.563 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T18:22:47.580 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T18:22:47.646 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T18:22:47.848 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T18:22:47.865 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T18:22:47.906 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T18:22:48.007 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T18:22:48.024 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T18:22:48.065 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T18:22:48.385 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T18:22:48.401 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T18:22:48.453 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:22:48.603 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T18:22:48.620 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T18:22:48.671 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:22:48.946 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T18:22:48.961 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T18:22:48.996 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:22:49.206 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T18:22:49.224 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T18:22:49.264 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:22:49.357 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T18:22:49.373 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T18:22:49.415 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:22:49.700 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T18:22:49.717 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T18:22:49.767 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:22:49.818 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T18:22:49.838 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T18:22:49.876 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T18:22:50.198 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../010-librbd1_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:50.212 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T18:22:50.229 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T18:22:50.270 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T18:22:50.321 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking librbd1 (18.2.4-860-g6c6b083f-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T18:22:50.681 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../010-librbd1_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:50.831 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librbd1 (18.2.4-860-g6c6b083f-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T18:22:51.025 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../011-librados2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:51.144 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking librados2 (18.2.4-860-g6c6b083f-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T18:22:51.526 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../011-librados2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:51.661 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librados2 (18.2.4-860-g6c6b083f-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T18:22:51.873 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T18:22:51.891 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../012-libcephfs2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:51.931 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libcephfs2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:52.267 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-rados. 2024-09-13T18:22:52.283 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../013-python3-rados_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:52.325 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-rados (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:52.448 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T18:22:52.464 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../012-libcephfs2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:52.506 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcephfs2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:52.669 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T18:22:52.687 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:22:52.719 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-ceph-argparse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:52.842 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rados. 2024-09-13T18:22:52.862 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../013-python3-rados_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:52.925 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rados (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:52.955 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T18:22:52.971 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../015-python3-cephfs_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:53.013 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-cephfs (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:53.278 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T18:22:53.294 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:22:53.336 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-ceph-argparse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:53.374 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T18:22:53.390 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:22:53.432 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:53.622 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T18:22:53.649 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../015-python3-cephfs_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:53.706 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cephfs (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:53.752 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T18:22:53.769 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T18:22:53.810 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T18:22:54.066 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T18:22:54.082 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:22:54.088 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T18:22:54.107 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../018-python3-rbd_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:54.124 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:54.146 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-rbd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:54.457 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T18:22:54.461 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T18:22:54.476 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T18:22:54.477 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T18:22:54.515 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T18:22:54.552 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T18:22:54.813 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T18:22:54.831 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../018-python3-rbd_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:54.863 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rbd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:54.893 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T18:22:54.909 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T18:22:54.951 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T18:22:55.216 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T18:22:55.235 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T18:22:55.274 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T18:22:55.288 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T18:22:55.304 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T18:22:55.346 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T18:22:55.686 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T18:22:55.690 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package librgw2. 2024-09-13T18:22:55.703 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T18:22:55.708 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../022-librgw2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:55.740 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking librgw2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:55.743 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T18:22:56.088 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T18:22:56.107 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T18:22:56.171 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T18:22:56.491 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librgw2. 2024-09-13T18:22:56.508 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../022-librgw2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:56.549 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librgw2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:56.584 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T18:22:56.601 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../023-python3-rgw_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:56.642 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-rgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:56.995 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package liboath0. 2024-09-13T18:22:57.012 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T18:22:57.054 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T18:22:57.340 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T18:22:57.357 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../025-libradosstriper1_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:57.389 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libradosstriper1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:57.635 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T18:22:57.654 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../023-python3-rgw_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:57.693 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:57.843 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-common. 2024-09-13T18:22:57.858 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../026-ceph-common_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:57.900 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:58.063 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liboath0. 2024-09-13T18:22:58.080 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T18:22:58.146 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T18:22:58.482 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T18:22:58.502 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../025-libradosstriper1_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:58.540 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libradosstriper1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:22:59.019 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-common. 2024-09-13T18:22:59.036 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../026-ceph-common_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:22:59.068 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:00.331 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-base. 2024-09-13T18:23:00.347 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../027-ceph-base_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:00.391 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-base (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:01.569 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T18:23:01.584 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T18:23:01.615 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-base. 2024-09-13T18:23:01.618 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T18:23:01.631 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../027-ceph-base_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:01.684 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-base (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:01.963 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T18:23:01.979 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T18:23:02.039 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T18:23:02.341 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T18:23:02.358 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T18:23:02.399 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T18:23:02.702 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T18:23:02.718 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T18:23:02.760 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T18:23:02.986 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T18:23:03.002 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T18:23:03.045 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T18:23:03.088 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-mako. 2024-09-13T18:23:03.104 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T18:23:03.146 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T18:23:03.423 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T18:23:03.440 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T18:23:03.475 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T18:23:03.491 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T18:23:03.506 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T18:23:03.533 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T18:23:03.811 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T18:23:03.827 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T18:23:03.844 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T18:23:03.860 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T18:23:03.879 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T18:23:03.901 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T18:23:04.180 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-webob. 2024-09-13T18:23:04.197 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T18:23:04.229 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T18:23:04.238 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T18:23:04.246 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T18:23:04.287 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T18:23:04.566 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T18:23:04.582 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T18:23:04.623 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-mako. 2024-09-13T18:23:04.624 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T18:23:04.640 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T18:23:04.681 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T18:23:04.952 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T18:23:04.967 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T18:23:05.010 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T18:23:05.051 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T18:23:05.071 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T18:23:05.101 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T18:23:05.363 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T18:23:05.379 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T18:23:05.424 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T18:23:05.470 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T18:23:05.487 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T18:23:05.529 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T18:23:05.766 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T18:23:05.782 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T18:23:05.824 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T18:23:05.865 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webob. 2024-09-13T18:23:05.881 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T18:23:05.923 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T18:23:06.127 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-paste. 2024-09-13T18:23:06.143 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T18:23:06.185 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T18:23:06.301 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T18:23:06.319 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T18:23:06.359 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T18:23:06.497 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T18:23:06.514 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T18:23:06.563 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T18:23:06.695 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T18:23:06.712 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T18:23:06.753 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T18:23:06.874 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T18:23:06.891 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T18:23:06.924 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T18:23:07.157 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T18:23:07.173 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T18:23:07.235 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T18:23:07.277 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T18:23:07.574 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T18:23:07.590 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T18:23:07.593 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T18:23:07.612 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T18:23:07.648 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T18:23:07.659 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T18:23:07.998 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T18:23:08.001 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-paste. 2024-09-13T18:23:08.014 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T18:23:08.019 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T18:23:08.055 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T18:23:08.059 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T18:23:08.375 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T18:23:08.394 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:08.432 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T18:23:08.433 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:08.451 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T18:23:08.496 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T18:23:08.830 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T18:23:08.832 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T18:23:08.846 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:08.848 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T18:23:08.888 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:08.890 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T18:23:09.241 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T18:23:09.258 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../048-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T18:23:09.285 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T18:23:09.299 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T18:23:09.301 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T18:23:09.343 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T18:23:09.569 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T18:23:09.585 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../049-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T18:23:09.619 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T18:23:09.671 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T18:23:09.691 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T18:23:09.729 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T18:23:09.964 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T18:23:09.980 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../050-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T18:23:10.015 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T18:23:10.143 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T18:23:10.160 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T18:23:10.209 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T18:23:10.387 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T18:23:10.403 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../051-ceph-mgr_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:10.445 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mgr (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:10.550 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T18:23:10.568 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:10.599 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:10.949 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T18:23:10.966 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../052-ceph-mon_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:11.023 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mon (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:11.038 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T18:23:11.053 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:11.096 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:11.481 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T18:23:11.499 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../048-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T18:23:11.539 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T18:23:11.867 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T18:23:11.883 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../049-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T18:23:11.925 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T18:23:12.354 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T18:23:12.370 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../050-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T18:23:12.412 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T18:23:12.592 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T18:23:12.608 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../053-ceph-osd_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:12.650 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-osd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:12.791 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T18:23:12.808 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../051-ceph-mgr_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:12.874 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:14.105 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T18:23:14.122 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../052-ceph-mon_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:14.160 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mon (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:14.811 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph. 2024-09-13T18:23:14.830 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../054-ceph_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:14.869 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:15.097 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T18:23:15.115 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../053-ceph-osd_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:15.122 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T18:23:15.139 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../055-ceph-fuse_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:15.172 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:15.180 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-osd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:15.626 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T18:23:15.642 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../056-ceph-mds_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:15.684 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mds (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:16.203 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package cephadm. 2024-09-13T18:23:16.219 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../057-cephadm_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:16.261 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:16.531 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T18:23:16.549 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../058-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T18:23:16.581 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T18:23:16.934 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T18:23:16.950 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T18:23:16.992 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T18:23:17.395 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T18:23:17.413 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:17.453 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:17.466 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph. 2024-09-13T18:23:17.483 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../054-ceph_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:17.544 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:17.806 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T18:23:17.822 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T18:23:17.864 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T18:23:17.919 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T18:23:17.937 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../055-ceph-fuse_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:17.978 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:18.192 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-attr. 2024-09-13T18:23:18.210 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2024-09-13T18:23:18.250 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T18:23:18.472 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T18:23:18.489 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../056-ceph-mds_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:18.530 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mds (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:18.578 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T18:23:18.594 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T18:23:18.636 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T18:23:18.931 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-py. 2024-09-13T18:23:18.947 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T18:23:18.989 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T18:23:19.108 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephadm. 2024-09-13T18:23:19.127 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../057-cephadm_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:19.158 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:19.342 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T18:23:19.358 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T18:23:19.401 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T18:23:19.534 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T18:23:19.552 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../058-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T18:23:19.597 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T18:23:19.703 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T18:23:19.719 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2024-09-13T18:23:19.761 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T18:23:19.997 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T18:23:20.013 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T18:23:20.047 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T18:23:20.080 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T18:23:20.096 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T18:23:20.138 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T18:23:20.452 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T18:23:20.470 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:20.476 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T18:23:20.493 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T18:23:20.508 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:20.534 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T18:23:20.779 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T18:23:20.797 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T18:23:20.828 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T18:23:20.922 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T18:23:20.942 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T18:23:20.980 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T18:23:21.106 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T18:23:21.122 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T18:23:21.164 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T18:23:21.316 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-attr. 2024-09-13T18:23:21.333 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2024-09-13T18:23:21.375 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T18:23:21.483 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-routes. 2024-09-13T18:23:21.499 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2024-09-13T18:23:21.541 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T18:23:21.787 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T18:23:21.806 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T18:23:21.845 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T18:23:21.886 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T18:23:21.902 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:21.944 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:22.140 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-py. 2024-09-13T18:23:22.159 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T18:23:22.198 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T18:23:22.559 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T18:23:22.576 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T18:23:22.642 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T18:23:22.987 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T18:23:23.004 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2024-09-13T18:23:23.045 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T18:23:23.299 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T18:23:23.316 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T18:23:23.357 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T18:23:23.381 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T18:23:23.397 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T18:23:23.464 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T18:23:23.660 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T18:23:23.680 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T18:23:23.718 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T18:23:23.842 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T18:23:23.858 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T18:23:23.900 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T18:23:24.029 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T18:23:24.045 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T18:23:24.087 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T18:23:24.178 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T18:23:24.194 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T18:23:24.236 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T18:23:24.556 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T18:23:24.575 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T18:23:24.641 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T18:23:24.966 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-routes. 2024-09-13T18:23:24.984 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2024-09-13T18:23:25.099 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T18:23:25.396 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T18:23:25.414 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T18:23:25.453 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T18:23:25.494 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T18:23:25.510 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:25.552 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:25.966 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T18:23:25.983 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T18:23:26.015 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T18:23:26.402 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T18:23:26.419 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T18:23:26.460 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T18:23:27.041 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T18:23:27.057 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T18:23:27.124 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T18:23:27.177 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T18:23:27.197 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:27.235 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:27.452 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T18:23:27.469 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T18:23:27.510 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T18:23:27.855 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T18:23:27.870 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T18:23:27.946 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T18:23:28.389 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T18:23:28.405 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T18:23:28.447 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T18:23:28.836 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T18:23:28.852 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T18:23:28.894 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T18:23:29.250 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T18:23:29.267 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T18:23:29.308 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T18:23:29.620 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T18:23:29.638 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2024-09-13T18:23:29.678 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T18:23:29.872 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T18:23:29.889 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T18:23:29.930 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T18:23:30.001 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T18:23:30.021 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T18:23:30.059 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T18:23:30.362 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T18:23:30.382 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T18:23:30.412 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T18:23:30.517 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T18:23:30.535 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T18:23:30.592 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T18:23:30.724 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T18:23:30.740 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T18:23:30.790 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T18:23:30.988 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T18:23:31.004 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T18:23:31.046 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T18:23:31.144 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T18:23:31.160 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T18:23:31.202 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T18:23:31.514 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T18:23:31.534 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T18:23:31.571 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T18:23:31.816 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T18:23:31.833 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:31.866 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:31.925 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T18:23:31.942 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T18:23:31.983 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T18:23:32.535 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T18:23:32.554 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:32.599 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:32.929 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T18:23:32.947 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T18:23:32.986 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T18:23:33.052 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T18:23:33.069 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T18:23:33.110 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T18:23:33.347 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T18:23:33.364 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:23:33.405 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:23:33.515 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T18:23:33.532 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T18:23:33.606 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T18:23:33.685 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package jq. 2024-09-13T18:23:33.703 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:23:33.745 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:23:33.944 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T18:23:33.963 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T18:23:33.987 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package socat. 2024-09-13T18:23:34.002 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T18:23:34.007 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T18:23:34.045 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T18:23:34.348 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T18:23:34.365 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2024-09-13T18:23:34.365 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T18:23:34.382 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T18:23:34.423 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T18:23:34.423 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T18:23:34.760 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T18:23:34.761 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-test. 2024-09-13T18:23:34.777 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T18:23:34.781 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../096-ceph-test_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:34.817 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T18:23:34.819 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-test (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:35.133 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T18:23:35.152 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T18:23:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T18:23:35.536 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T18:23:35.553 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T18:23:35.594 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T18:23:36.006 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T18:23:36.022 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T18:23:36.064 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T18:23:36.394 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T18:23:36.414 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T18:23:36.452 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T18:23:36.831 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T18:23:36.851 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T18:23:36.888 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T18:23:37.502 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T18:23:37.520 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:37.560 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:37.921 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T18:23:37.937 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T18:23:38.021 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T18:23:38.399 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T18:23:38.417 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:23:38.457 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:23:38.760 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package jq. 2024-09-13T18:23:38.777 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:23:38.835 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:23:38.989 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T18:23:39.009 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:39.047 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-volume (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:39.129 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package socat. 2024-09-13T18:23:39.148 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T18:23:39.196 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T18:23:39.410 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-13T18:23:39.427 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:39.468 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking cephfs-mirror (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:39.533 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T18:23:39.550 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T18:23:39.590 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T18:23:39.829 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-colorama. 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2024-09-13T18:23:39.887 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-09-13T18:23:39.995 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-test. 2024-09-13T18:23:40.013 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../096-ceph-test_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:40.053 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-test (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:40.149 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package cephfs-shell. 2024-09-13T18:23:40.165 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../100-cephfs-shell_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:40.207 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking cephfs-shell (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:40.452 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package cephfs-top. 2024-09-13T18:23:40.470 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../101-cephfs-top_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:40.510 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking cephfs-top (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:40.764 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T18:23:40.782 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T18:23:40.814 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:23:41.301 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T18:23:41.318 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:41.359 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libcephfs-dev (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:41.771 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T18:23:41.787 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T18:23:41.849 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T18:23:42.170 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T18:23:42.187 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T18:23:42.228 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T18:23:42.614 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T18:23:42.634 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T18:23:42.672 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:23:44.141 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T18:23:44.163 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T18:23:44.225 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T18:23:44.243 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T18:23:44.261 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:44.301 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-volume (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:44.535 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T18:23:44.555 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T18:23:44.593 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T18:23:44.739 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-13T18:23:44.759 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:44.797 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephfs-mirror (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:44.896 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T18:23:44.915 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T18:23:44.954 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T18:23:45.170 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-colorama. 2024-09-13T18:23:45.188 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2024-09-13T18:23:45.253 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-09-13T18:23:45.325 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T18:23:45.346 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T18:23:45.382 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T18:23:45.547 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephfs-shell. 2024-09-13T18:23:45.565 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../100-cephfs-shell_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:45.605 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephfs-shell (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:45.863 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T18:23:45.880 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T18:23:45.891 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephfs-top. 2024-09-13T18:23:45.908 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../101-cephfs-top_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:23:45.921 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T18:23:45.949 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephfs-top (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:46.221 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T18:23:46.237 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T18:23:46.249 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T18:23:46.270 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T18:23:46.287 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:23:46.307 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T18:23:46.669 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T18:23:46.685 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T18:23:46.727 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T18:23:46.816 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T18:23:46.833 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:46.874 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcephfs-dev (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:47.029 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T18:23:47.050 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T18:23:47.087 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T18:23:47.319 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T18:23:47.336 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T18:23:47.348 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T18:23:47.368 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T18:23:47.394 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T18:23:47.406 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T18:23:47.748 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T18:23:47.765 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T18:23:47.798 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T18:23:48.079 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package radosgw. 2024-09-13T18:23:48.099 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../116-radosgw_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:48.137 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking radosgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:48.209 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T18:23:48.228 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T18:23:48.292 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:23:49.574 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T18:23:49.594 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../117-rbd-fuse_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:49.631 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking rbd-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:49.932 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T18:23:50.047 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T18:23:50.065 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T18:23:50.105 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T18:23:50.236 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T18:23:50.363 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T18:23:50.523 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T18:23:50.527 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T18:23:50.544 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T18:23:50.585 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T18:23:50.691 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:50.896 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T18:23:50.913 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T18:23:50.954 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T18:23:51.225 INFO:teuthology.orchestra.run.smithi146.stdout:Adding system user cephadm....done 2024-09-13T18:23:51.352 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T18:23:51.372 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T18:23:51.405 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:23:51.409 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T18:23:51.531 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T18:23:51.866 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T18:23:51.921 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T18:23:51.940 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T18:23:51.979 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T18:23:52.125 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:23:52.257 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T18:23:52.341 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T18:23:52.360 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T18:23:52.386 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T18:23:52.433 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T18:23:52.712 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T18:23:52.803 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T18:23:52.819 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T18:23:52.860 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T18:23:53.114 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T18:23:53.240 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T18:23:53.260 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T18:23:53.289 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T18:23:53.407 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T18:23:53.576 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T18:23:53.596 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T18:23:53.634 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T18:23:53.666 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-09-13T18:23:53.951 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T18:23:54.260 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T18:23:54.380 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package radosgw. 2024-09-13T18:23:54.397 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../116-radosgw_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:54.437 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking radosgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:54.545 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-ceph-argparse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:54.846 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T18:23:54.997 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-09-13T18:23:55.281 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T18:23:55.557 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T18:23:55.841 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T18:23:55.885 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T18:23:55.899 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../117-rbd-fuse_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:23:55.941 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking rbd-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:55.968 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T18:23:56.283 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T18:23:56.352 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T18:23:56.612 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T18:23:56.645 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T18:23:56.763 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T18:23:56.889 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T18:23:56.954 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T18:23:57.099 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:23:57.281 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T18:23:57.548 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T18:23:57.692 INFO:teuthology.orchestra.run.smithi154.stdout:Adding system user cephadm....done 2024-09-13T18:23:57.880 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:23:57.882 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T18:23:58.006 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T18:23:58.176 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T18:23:58.325 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T18:23:58.643 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:23:58.810 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T18:23:58.935 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T18:23:58.963 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T18:23:59.078 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T18:23:59.204 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T18:23:59.305 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T18:23:59.488 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T18:23:59.715 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T18:23:59.865 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-09-13T18:24:00.124 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T18:24:00.486 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T18:24:00.502 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T18:24:00.854 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T18:24:00.976 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-09-13T18:24:00.980 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:01.253 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T18:24:01.307 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:24:01.416 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T18:24:01.537 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T18:24:01.751 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T18:24:01.830 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-ceph-argparse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:01.877 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T18:24:02.003 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T18:24:02.123 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T18:24:02.248 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-09-13T18:24:02.321 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T18:24:02.600 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T18:24:02.638 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:24:02.764 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T18:24:02.865 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T18:24:02.901 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T18:24:03.192 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T18:24:03.193 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T18:24:03.311 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T18:24:03.477 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T18:24:03.729 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T18:24:03.778 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T18:24:03.929 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T18:24:04.022 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T18:24:04.213 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T18:24:04.290 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T18:24:04.498 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T18:24:04.649 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T18:24:04.807 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T18:24:04.934 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T18:24:05.199 INFO:teuthology.orchestra.run.smithi146.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T18:24:05.268 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T18:24:05.435 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T18:24:05.561 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T18:24:05.586 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T18:24:05.887 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T18:24:05.912 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T18:24:06.038 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T18:24:06.186 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T18:24:06.247 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T18:24:06.372 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T18:24:06.482 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T18:24:06.490 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T18:24:06.791 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T18:24:06.917 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-09-13T18:24:07.059 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T18:24:07.276 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T18:24:07.636 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T18:24:07.996 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T18:24:08.156 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:08.474 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:24:08.523 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:24:08.624 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T18:24:08.641 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T18:24:08.976 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T18:24:08.983 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T18:24:09.085 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T18:24:09.253 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T18:24:09.310 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T18:24:09.546 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T18:24:09.611 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:24:09.737 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T18:24:09.872 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:24:10.023 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T18:24:10.096 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T18:24:10.150 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T18:24:10.457 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T18:24:10.485 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T18:24:10.794 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T18:24:10.816 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T18:24:11.135 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T18:24:11.155 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T18:24:11.279 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T18:24:11.564 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T18:24:11.890 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T18:24:12.181 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T18:24:12.250 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T18:24:12.282 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T18:24:12.383 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:24:12.492 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T18:24:12.611 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T18:24:12.810 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T18:24:12.885 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T18:24:12.961 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T18:24:13.032 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T18:24:13.263 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T18:24:13.371 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T18:24:13.555 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T18:24:13.731 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T18:24:13.857 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T18:24:13.882 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T18:24:14.025 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T18:24:14.217 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up librados2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:14.359 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T18:24:14.428 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T18:24:14.571 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up librgw2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:14.694 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T18:24:14.722 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:14.848 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libcephfs2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:14.957 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T18:24:15.242 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libradosstriper1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:15.343 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T18:24:15.661 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up librbd1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:15.787 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:15.913 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:16.132 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T18:24:16.132 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T18:24:16.183 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:24:16.310 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T18:24:16.556 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libcephfs-dev (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:16.682 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-rados (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:16.710 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T18:24:16.783 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up cephfs-top (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:16.883 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T18:24:17.003 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T18:24:17.296 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:24:17.422 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T18:24:17.779 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-rbd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:17.823 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T18:24:17.905 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up rbd-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:18.031 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-rgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:18.157 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-cephfs (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:18.158 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T18:24:18.291 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up cephfs-shell (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:18.392 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:18.543 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T18:24:18.903 INFO:teuthology.orchestra.run.smithi146.stdout:Adding group ceph....done 2024-09-13T18:24:18.903 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T18:24:19.305 INFO:teuthology.orchestra.run.smithi146.stdout:Adding system user ceph....done 2024-09-13T18:24:19.472 INFO:teuthology.orchestra.run.smithi146.stdout:Setting system user ceph properties....done 2024-09-13T18:24:19.501 INFO:teuthology.orchestra.run.smithi146.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T18:24:19.651 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T18:24:19.933 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T18:24:19.953 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T18:24:20.093 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T18:24:20.238 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:24:20.353 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T18:24:20.376 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up cephfs-mirror (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:20.605 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:24:20.605 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:24:20.646 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T18:24:20.805 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T18:24:21.011 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-test (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:21.115 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T18:24:21.137 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up radosgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:21.408 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T18:24:21.611 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:24:21.611 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:24:21.726 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T18:24:22.064 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-base (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:22.069 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librados2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:22.306 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T18:24:22.456 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T18:24:22.474 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librgw2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:22.600 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:22.751 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcephfs2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:22.877 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T18:24:22.899 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mds (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:23.134 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T18:24:23.134 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T18:24:23.203 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libradosstriper1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:23.363 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T18:24:23.550 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mgr (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:23.664 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librbd1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:23.800 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T18:24:23.800 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T18:24:23.824 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:23.950 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:24.196 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T18:24:24.197 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T18:24:24.252 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-osd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:24.652 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcephfs-dev (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:24.777 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rados (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:24.929 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephfs-top (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:25.046 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T18:24:25.073 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T18:24:25.073 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T18:24:25.496 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:25.622 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:25.765 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mon (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:25.975 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rbd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:25.991 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T18:24:25.991 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T18:24:26.101 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up rbd-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:26.261 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:26.387 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cephfs (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:26.416 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:26.538 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephfs-shell (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:26.558 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:26.655 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:26.676 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:26.811 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-volume (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:27.213 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T18:24:27.258 INFO:teuthology.orchestra.run.smithi154.stdout:Adding group ceph....done 2024-09-13T18:24:27.365 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T18:24:27.641 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T18:24:27.668 INFO:teuthology.orchestra.run.smithi154.stdout:Adding system user ceph....done 2024-09-13T18:24:27.869 INFO:teuthology.orchestra.run.smithi154.stdout:Setting system user ceph properties....done 2024-09-13T18:24:27.889 INFO:teuthology.orchestra.run.smithi154.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T18:24:28.037 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T18:24:28.350 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T18:24:28.840 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephfs-mirror (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:29.098 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:24:29.098 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:24:29.550 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-test (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:29.676 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up radosgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:30.184 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:24:30.184 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:24:30.575 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T18:24:30.645 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-base (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:31.052 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T18:24:31.184 INFO:teuthology.orchestra.run.smithi146.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T18:24:31.186 DEBUG:teuthology.orchestra.run.smithi146:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-13T18:24:31.256 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-09-13T18:24:31.474 INFO:teuthology.orchestra.run.smithi146.stdout:Building dependency tree... 2024-09-13T18:24:31.475 INFO:teuthology.orchestra.run.smithi146.stdout:Reading state information... 2024-09-13T18:24:31.676 INFO:teuthology.orchestra.run.smithi146.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-09-13T18:24:31.676 INFO:teuthology.orchestra.run.smithi146.stdout:flex is already the newest version (2.6.4-6.2). 2024-09-13T18:24:31.676 INFO:teuthology.orchestra.run.smithi146.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-09-13T18:24:31.676 INFO:teuthology.orchestra.run.smithi146.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-09-13T18:24:31.676 INFO:teuthology.orchestra.run.smithi146.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-09-13T18:24:31.676 INFO:teuthology.orchestra.run.smithi146.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T18:24:31.677 INFO:teuthology.orchestra.run.smithi146.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T18:24:31.677 INFO:teuthology.orchestra.run.smithi146.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T18:24:31.677 INFO:teuthology.orchestra.run.smithi146.stdout:The following additional packages will be installed: 2024-09-13T18:24:31.677 INFO:teuthology.orchestra.run.smithi146.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-09-13T18:24:31.678 INFO:teuthology.orchestra.run.smithi146.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2024-09-13T18:24:31.678 INFO:teuthology.orchestra.run.smithi146.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2024-09-13T18:24:31.678 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2024-09-13T18:24:31.678 INFO:teuthology.orchestra.run.smithi146.stdout: usb-modeswitch-data wpasupplicant 2024-09-13T18:24:31.680 INFO:teuthology.orchestra.run.smithi146.stdout:Suggested packages: 2024-09-13T18:24:31.680 INFO:teuthology.orchestra.run.smithi146.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2024-09-13T18:24:31.680 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2024-09-13T18:24:31.680 INFO:teuthology.orchestra.run.smithi146.stdout: libengine-pkcs11-openssl 2024-09-13T18:24:31.696 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mds (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:31.740 INFO:teuthology.orchestra.run.smithi146.stdout:The following NEW packages will be installed: 2024-09-13T18:24:31.740 INFO:teuthology.orchestra.run.smithi146.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-09-13T18:24:31.741 INFO:teuthology.orchestra.run.smithi146.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2024-09-13T18:24:31.741 INFO:teuthology.orchestra.run.smithi146.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-09-13T18:24:31.741 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2024-09-13T18:24:31.741 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2024-09-13T18:24:31.741 INFO:teuthology.orchestra.run.smithi146.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T18:24:31.742 INFO:teuthology.orchestra.run.smithi146.stdout:The following packages will be upgraded: 2024-09-13T18:24:31.743 INFO:teuthology.orchestra.run.smithi146.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-09-13T18:24:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:6 upgraded, 31 newly installed, 0 to remove and 289 not upgraded. 2024-09-13T18:24:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:Need to get 25.4 MB of archives. 2024-09-13T18:24:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:After this operation, 80.0 MB of additional disk space will be used. 2024-09-13T18:24:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-09-13T18:24:31.947 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T18:24:31.947 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T18:24:32.374 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:32.543 INFO:teuthology.orchestra.run.smithi146.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-09-13T18:24:32.545 INFO:teuthology.orchestra.run.smithi146.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-09-13T18:24:32.546 INFO:teuthology.orchestra.run.smithi146.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-09-13T18:24:32.558 INFO:teuthology.orchestra.run.smithi146.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-09-13T18:24:32.561 INFO:teuthology.orchestra.run.smithi146.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-09-13T18:24:32.601 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T18:24:32.601 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T18:24:32.665 INFO:teuthology.orchestra.run.smithi146.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-09-13T18:24:32.667 INFO:teuthology.orchestra.run.smithi146.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-09-13T18:24:32.674 INFO:teuthology.orchestra.run.smithi146.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-09-13T18:24:32.679 INFO:teuthology.orchestra.run.smithi146.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-09-13T18:24:32.680 INFO:teuthology.orchestra.run.smithi146.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-09-13T18:24:32.706 INFO:teuthology.orchestra.run.smithi146.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-09-13T18:24:32.708 INFO:teuthology.orchestra.run.smithi146.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-09-13T18:24:32.711 INFO:teuthology.orchestra.run.smithi146.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-09-13T18:24:32.712 INFO:teuthology.orchestra.run.smithi146.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-09-13T18:24:32.787 INFO:teuthology.orchestra.run.smithi146.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-09-13T18:24:32.788 INFO:teuthology.orchestra.run.smithi146.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-09-13T18:24:32.797 INFO:teuthology.orchestra.run.smithi146.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.20-0ubuntu0.20.04.1 [117 kB] 2024-09-13T18:24:32.800 INFO:teuthology.orchestra.run.smithi146.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-09-13T18:24:32.809 INFO:teuthology.orchestra.run.smithi146.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-09-13T18:24:32.809 INFO:teuthology.orchestra.run.smithi146.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-09-13T18:24:32.869 INFO:teuthology.orchestra.run.smithi146.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-09-13T18:24:32.869 INFO:teuthology.orchestra.run.smithi146.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-09-13T18:24:32.886 INFO:teuthology.orchestra.run.smithi146.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-09-13T18:24:32.909 INFO:teuthology.orchestra.run.smithi146.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-09-13T18:24:32.973 INFO:teuthology.orchestra.run.smithi146.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-09-13T18:24:32.978 INFO:teuthology.orchestra.run.smithi146.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-09-13T18:24:32.979 INFO:teuthology.orchestra.run.smithi146.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-09-13T18:24:32.980 INFO:teuthology.orchestra.run.smithi146.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2024-09-13T18:24:32.981 INFO:teuthology.orchestra.run.smithi146.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.20-0ubuntu0.20.04.1 [1,055 kB] 2024-09-13T18:24:33.000 INFO:teuthology.orchestra.run.smithi146.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2024-09-13T18:24:33.035 INFO:teuthology.orchestra.run.smithi146.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.20-0ubuntu0.20.04.1 [13.5 MB] 2024-09-13T18:24:33.075 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-osd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:33.405 INFO:teuthology.orchestra.run.smithi146.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2024-09-13T18:24:33.405 INFO:teuthology.orchestra.run.smithi146.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2024-09-13T18:24:33.475 INFO:teuthology.orchestra.run.smithi146.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2024-09-13T18:24:33.475 INFO:teuthology.orchestra.run.smithi146.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-09-13T18:24:33.476 INFO:teuthology.orchestra.run.smithi146.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-09-13T18:24:33.583 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T18:24:33.583 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T18:24:33.884 INFO:teuthology.orchestra.run.smithi146.stdout:Preconfiguring packages ... 2024-09-13T18:24:34.027 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:34.074 INFO:teuthology.orchestra.run.smithi146.stdout:Fetched 25.4 MB in 2s (14.7 MB/s) 2024-09-13T18:24:34.153 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:34.321 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mon (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:34.344 INFO:teuthology.orchestra.run.smithi146.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T18:24:34.349 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T18:24:34.529 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T18:24:34.545 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T18:24:34.545 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T18:24:35.005 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:35.156 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:35.450 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-volume (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:24:35.804 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-09-13T18:24:35.853 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T18:24:36.005 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T18:24:36.306 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T18:24:37.190 INFO:teuthology.orchestra.run.smithi146.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T18:24:37.194 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T18:24:37.335 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T18:24:37.725 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T18:24:37.890 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T18:24:38.245 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T18:24:38.652 INFO:teuthology.orchestra.run.smithi146.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T18:24:38.656 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T18:24:38.790 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T18:24:39.193 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T18:24:39.203 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T18:24:39.320 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T18:24:39.828 INFO:teuthology.orchestra.run.smithi154.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T18:24:39.831 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-13T18:24:39.833 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T18:24:39.908 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-09-13T18:24:39.966 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-09-13T18:24:40.131 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2024-09-13T18:24:40.132 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2024-09-13T18:24:40.351 INFO:teuthology.orchestra.run.smithi154.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-09-13T18:24:40.351 INFO:teuthology.orchestra.run.smithi154.stdout:flex is already the newest version (2.6.4-6.2). 2024-09-13T18:24:40.351 INFO:teuthology.orchestra.run.smithi154.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-09-13T18:24:40.351 INFO:teuthology.orchestra.run.smithi154.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-09-13T18:24:40.352 INFO:teuthology.orchestra.run.smithi154.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-09-13T18:24:40.352 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T18:24:40.352 INFO:teuthology.orchestra.run.smithi154.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T18:24:40.352 INFO:teuthology.orchestra.run.smithi154.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T18:24:40.353 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2024-09-13T18:24:40.353 INFO:teuthology.orchestra.run.smithi154.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-09-13T18:24:40.353 INFO:teuthology.orchestra.run.smithi154.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2024-09-13T18:24:40.354 INFO:teuthology.orchestra.run.smithi154.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2024-09-13T18:24:40.354 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2024-09-13T18:24:40.354 INFO:teuthology.orchestra.run.smithi154.stdout: usb-modeswitch-data wpasupplicant 2024-09-13T18:24:40.355 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2024-09-13T18:24:40.355 INFO:teuthology.orchestra.run.smithi154.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2024-09-13T18:24:40.355 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2024-09-13T18:24:40.355 INFO:teuthology.orchestra.run.smithi154.stdout: libengine-pkcs11-openssl 2024-09-13T18:24:40.418 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2024-09-13T18:24:40.418 INFO:teuthology.orchestra.run.smithi154.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-09-13T18:24:40.418 INFO:teuthology.orchestra.run.smithi154.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2024-09-13T18:24:40.419 INFO:teuthology.orchestra.run.smithi154.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-09-13T18:24:40.419 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2024-09-13T18:24:40.419 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2024-09-13T18:24:40.420 INFO:teuthology.orchestra.run.smithi154.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T18:24:40.420 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages will be upgraded: 2024-09-13T18:24:40.421 INFO:teuthology.orchestra.run.smithi154.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-09-13T18:24:40.477 INFO:teuthology.orchestra.run.smithi154.stdout:6 upgraded, 31 newly installed, 0 to remove and 289 not upgraded. 2024-09-13T18:24:40.478 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 25.4 MB of archives. 2024-09-13T18:24:40.478 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 80.0 MB of additional disk space will be used. 2024-09-13T18:24:40.478 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-09-13T18:24:40.486 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-13T18:24:40.509 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-09-13T18:24:40.609 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-09-13T18:24:40.610 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-09-13T18:24:40.611 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-09-13T18:24:40.616 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-09-13T18:24:40.617 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-09-13T18:24:40.634 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-09-13T18:24:40.635 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-09-13T18:24:40.636 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-09-13T18:24:40.638 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-09-13T18:24:40.638 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-09-13T18:24:40.643 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-09-13T18:24:40.644 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-09-13T18:24:40.645 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-09-13T18:24:40.646 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-09-13T18:24:40.660 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-09-13T18:24:40.661 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-09-13T18:24:40.666 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.20-0ubuntu0.20.04.1 [117 kB] 2024-09-13T18:24:40.668 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-09-13T18:24:40.676 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-09-13T18:24:40.676 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-09-13T18:24:40.690 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-09-13T18:24:40.690 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-09-13T18:24:40.698 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-09-13T18:24:40.708 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-09-13T18:24:40.723 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-09-13T18:24:40.726 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-09-13T18:24:40.727 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-09-13T18:24:40.727 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2024-09-13T18:24:40.727 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.20-0ubuntu0.20.04.1 [1,055 kB] 2024-09-13T18:24:40.736 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2024-09-13T18:24:40.738 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.20-0ubuntu0.20.04.1 [13.5 MB] 2024-09-13T18:24:40.787 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-09-13T18:24:40.847 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2024-09-13T18:24:40.847 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2024-09-13T18:24:40.922 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2024-09-13T18:24:40.923 INFO:teuthology.orchestra.run.smithi154.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-09-13T18:24:40.923 INFO:teuthology.orchestra.run.smithi154.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-09-13T18:24:41.320 INFO:teuthology.orchestra.run.smithi154.stdout:Preconfiguring packages ... 2024-09-13T18:24:41.516 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 25.4 MB in 0s (51.2 MB/s) 2024-09-13T18:24:41.607 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package dump. 2024-09-13T18:24:41.625 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-09-13T18:24:41.665 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking dump (0.4b46-6) ... 2024-09-13T18:24:41.944 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package indent. 2024-09-13T18:24:41.961 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:24:41.989 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T18:24:41.994 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T18:24:41.994 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T18:24:42.187 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T18:24:42.289 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-13T18:24:42.306 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-09-13T18:24:42.347 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T18:24:42.691 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-13T18:24:42.709 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-09-13T18:24:42.750 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T18:24:43.019 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-13T18:24:43.039 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T18:24:43.069 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:24:43.339 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-13T18:24:43.359 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:24:43.379 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-09-13T18:24:43.430 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:24:43.716 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-13T18:24:43.733 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:24:43.791 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:24:44.118 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-13T18:24:44.135 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:24:44.177 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:24:44.521 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-13T18:24:44.540 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-09-13T18:24:44.571 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T18:24:44.857 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T18:24:44.862 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T18:24:44.882 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-13T18:24:44.899 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T18:24:44.940 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T18:24:44.977 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T18:24:45.335 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-13T18:24:45.353 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../14-libpq5_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:24:45.404 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:24:45.442 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T18:24:45.608 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T18:24:45.769 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-13T18:24:45.769 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:24:45.771 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:24:45.996 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T18:24:46.090 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-13T18:24:46.111 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:24:46.148 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:24:46.401 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-13T18:24:46.419 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T18:24:46.457 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T18:24:46.459 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T18:24:46.462 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T18:24:46.610 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T18:24:47.148 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T18:24:47.214 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-13T18:24:47.230 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2024-09-13T18:24:47.272 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-09-13T18:24:47.650 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package modemmanager. 2024-09-13T18:24:47.669 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:24:47.719 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:24:47.834 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T18:24:48.348 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package wpasupplicant. 2024-09-13T18:24:48.366 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-09-13T18:24:48.406 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T18:24:48.851 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T18:24:48.960 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package network-manager. 2024-09-13T18:24:48.978 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T18:24:49.031 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-09-13T18:24:49.041 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T18:24:49.569 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-13T18:24:49.588 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-09-13T18:24:49.626 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-09-13T18:24:49.767 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ppp. 2024-09-13T18:24:49.785 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-09-13T18:24:49.988 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package dump. 2024-09-13T18:24:50.007 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-09-13T18:24:50.043 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T18:24:50.072 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking dump (0.4b46-6) ... 2024-09-13T18:24:50.372 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package pptp-linux. 2024-09-13T18:24:50.390 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-09-13T18:24:50.416 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package indent. 2024-09-13T18:24:50.430 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-09-13T18:24:50.433 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:24:50.475 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T18:24:50.769 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-13T18:24:50.786 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-09-13T18:24:50.817 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-13T18:24:50.821 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T18:24:50.836 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-09-13T18:24:50.893 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-09-13T18:24:51.189 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-13T18:24:51.206 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2024-09-13T18:24:51.214 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-13T18:24:51.235 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-09-13T18:24:51.247 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2024-09-13T18:24:51.274 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T18:24:51.576 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-13T18:24:51.576 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package postgresql-client-12. 2024-09-13T18:24:51.593 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T18:24:51.593 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../26-postgresql-client-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:24:51.634 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:24:51.634 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:24:52.029 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-13T18:24:52.048 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:24:52.087 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:24:52.406 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-13T18:24:52.422 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:24:52.464 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:24:52.540 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package postgresql-common. 2024-09-13T18:24:52.560 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2024-09-13T18:24:52.622 INFO:teuthology.orchestra.run.smithi146.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-13T18:24:52.650 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2024-09-13T18:24:52.859 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-13T18:24:52.877 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:24:52.917 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:24:53.122 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package postgresql-12. 2024-09-13T18:24:53.141 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../28-postgresql-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:24:53.183 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:24:53.337 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-13T18:24:53.356 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-09-13T18:24:53.395 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T18:24:54.550 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-13T18:24:54.571 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T18:24:54.774 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package postgresql. 2024-09-13T18:24:54.793 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2024-09-13T18:24:54.832 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2024-09-13T18:24:54.833 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T18:24:55.085 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package postgresql-client. 2024-09-13T18:24:55.105 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2024-09-13T18:24:55.143 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2024-09-13T18:24:55.329 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-13T18:24:55.348 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../14-libpq5_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:24:55.387 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-13T18:24:55.403 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:24:55.404 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2024-09-13T18:24:55.445 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T18:24:55.681 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-13T18:24:55.700 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2024-09-13T18:24:55.739 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-09-13T18:24:55.798 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-13T18:24:55.816 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:24:55.889 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:24:56.125 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-13T18:24:56.145 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-09-13T18:24:56.194 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T18:24:56.259 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-13T18:24:56.277 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:24:56.317 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:24:56.486 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2024-09-13T18:24:56.603 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-13T18:24:56.621 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T18:24:56.682 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T18:24:56.697 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T18:24:57.007 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:24:57.133 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-09-13T18:24:57.257 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-13T18:24:57.259 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T18:24:57.276 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2024-09-13T18:24:57.315 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-09-13T18:24:57.485 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T18:24:57.727 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package modemmanager. 2024-09-13T18:24:57.745 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:24:57.829 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:24:57.905 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T18:24:57.905 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T18:24:58.347 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:24:58.483 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package wpasupplicant. 2024-09-13T18:24:58.500 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-09-13T18:24:58.541 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T18:24:59.054 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package network-manager. 2024-09-13T18:24:59.073 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T18:24:59.137 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T18:24:59.246 INFO:teuthology.orchestra.run.smithi146.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-13T18:24:59.336 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T18:24:59.461 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-09-13T18:24:59.612 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2024-09-13T18:25:00.129 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ppp. 2024-09-13T18:25:00.148 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-09-13T18:25:00.617 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T18:25:00.631 INFO:teuthology.orchestra.run.smithi146.stdout:Adding user postgres to group ssl-cert 2024-09-13T18:25:00.906 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:00.907 INFO:teuthology.orchestra.run.smithi146.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-13T18:25:01.002 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package pptp-linux. 2024-09-13T18:25:01.020 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-09-13T18:25:01.061 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-09-13T18:25:01.162 INFO:teuthology.orchestra.run.smithi146.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-13T18:25:01.162 INFO:teuthology.orchestra.run.smithi146.stdout:Removing obsolete dictionary files: 2024-09-13T18:25:01.481 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-13T18:25:01.483 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-13T18:25:01.497 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-09-13T18:25:01.556 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-09-13T18:25:01.909 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-13T18:25:01.926 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2024-09-13T18:25:02.009 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2024-09-13T18:25:02.211 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T18:25:02.340 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-09-13T18:25:02.355 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-client-12. 2024-09-13T18:25:02.374 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../26-postgresql-client-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:25:02.413 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:25:02.767 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T18:25:03.093 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-common. 2024-09-13T18:25:03.112 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2024-09-13T18:25:03.173 INFO:teuthology.orchestra.run.smithi154.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-13T18:25:03.203 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2024-09-13T18:25:03.459 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T18:25:03.733 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-12. 2024-09-13T18:25:03.752 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../28-postgresql-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:25:03.794 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:25:04.387 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-09-13T18:25:04.606 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up dump (0.4b46-6) ... 2024-09-13T18:25:04.701 INFO:teuthology.orchestra.run.smithi146.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-13T18:25:04.758 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:25:04.884 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T18:25:05.035 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T18:25:05.161 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T18:25:05.354 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T18:25:05.488 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:25:05.667 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql. 2024-09-13T18:25:05.688 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2024-09-13T18:25:05.836 INFO:teuthology.orchestra.run.smithi146.stdout:Creating new PostgreSQL cluster 12/main ... 2024-09-13T18:25:05.850 INFO:teuthology.orchestra.run.smithi146.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2024-09-13T18:25:05.916 INFO:teuthology.orchestra.run.smithi146.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-13T18:25:05.916 INFO:teuthology.orchestra.run.smithi146.stdout:This user must also own the server process. 2024-09-13T18:25:05.916 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:05.916 INFO:teuthology.orchestra.run.smithi146.stdout:The database cluster will be initialized with locale "en_US". 2024-09-13T18:25:05.916 INFO:teuthology.orchestra.run.smithi146.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-13T18:25:05.916 INFO:teuthology.orchestra.run.smithi146.stdout:The default text search configuration will be set to "english". 2024-09-13T18:25:05.916 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:05.916 INFO:teuthology.orchestra.run.smithi146.stdout:Data page checksums are disabled. 2024-09-13T18:25:05.917 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:05.917 INFO:teuthology.orchestra.run.smithi146.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2024-09-13T18:25:05.918 INFO:teuthology.orchestra.run.smithi146.stdout:creating subdirectories ... ok 2024-09-13T18:25:05.918 INFO:teuthology.orchestra.run.smithi146.stdout:selecting dynamic shared memory implementation ... posix 2024-09-13T18:25:05.935 INFO:teuthology.orchestra.run.smithi146.stdout:selecting default max_connections ... 100 2024-09-13T18:25:05.964 INFO:teuthology.orchestra.run.smithi146.stdout:selecting default shared_buffers ... 128MB 2024-09-13T18:25:05.980 INFO:teuthology.orchestra.run.smithi146.stdout:selecting default time zone ... Etc/UTC 2024-09-13T18:25:05.980 INFO:teuthology.orchestra.run.smithi146.stdout:creating configuration files ... ok 2024-09-13T18:25:06.072 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2024-09-13T18:25:06.084 INFO:teuthology.orchestra.run.smithi146.stdout:running bootstrap script ... ok 2024-09-13T18:25:06.455 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-client. 2024-09-13T18:25:06.475 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2024-09-13T18:25:06.530 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2024-09-13T18:25:06.604 INFO:teuthology.orchestra.run.smithi146.stdout:performing post-bootstrap initialization ... ok 2024-09-13T18:25:06.799 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-13T18:25:06.816 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2024-09-13T18:25:06.857 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T18:25:07.126 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-13T18:25:07.143 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2024-09-13T18:25:07.176 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-09-13T18:25:07.314 INFO:teuthology.orchestra.run.smithi146.stdout:syncing data to disk ... ok 2024-09-13T18:25:07.314 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:07.314 INFO:teuthology.orchestra.run.smithi146.stdout:Success. You can now start the database server using: 2024-09-13T18:25:07.314 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:07.314 INFO:teuthology.orchestra.run.smithi146.stdout: pg_ctlcluster 12 main start 2024-09-13T18:25:07.314 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:07.621 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-13T18:25:07.628 INFO:teuthology.orchestra.run.smithi146.stdout:Ver Cluster Port Status Owner Data directory Log file 2024-09-13T18:25:07.628 INFO:teuthology.orchestra.run.smithi146.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2024-09-13T18:25:07.640 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-09-13T18:25:07.690 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T18:25:07.715 INFO:teuthology.orchestra.run.smithi146.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-13T18:25:08.017 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2024-09-13T18:25:08.268 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T18:25:08.570 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:25:08.729 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-09-13T18:25:08.855 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T18:25:09.006 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T18:25:09.357 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T18:25:09.358 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T18:25:09.851 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:25:10.653 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:25:10.709 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-13T18:25:10.787 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:25:10.798 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T18:25:10.897 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-09-13T18:25:10.957 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-09-13T18:25:11.065 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:25:11.083 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2024-09-13T18:25:11.190 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2024-09-13T18:25:11.341 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T18:25:12.039 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:12.039 INFO:teuthology.orchestra.run.smithi146.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-13T18:25:12.039 INFO:teuthology.orchestra.run.smithi146.stdout:which means they are currently configured by ifupdown: 2024-09-13T18:25:12.039 INFO:teuthology.orchestra.run.smithi146.stdout:- enp3s0f1 2024-09-13T18:25:12.040 INFO:teuthology.orchestra.run.smithi146.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-13T18:25:12.040 INFO:teuthology.orchestra.run.smithi146.stdout:remove their configuration from /etc/network/interfaces. 2024-09-13T18:25:12.040 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:12.201 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-13T18:25:12.515 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-13T18:25:12.584 INFO:teuthology.orchestra.run.smithi154.stdout:Adding user postgres to group ssl-cert 2024-09-13T18:25:12.887 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:12.887 INFO:teuthology.orchestra.run.smithi154.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-13T18:25:13.028 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-13T18:25:13.150 INFO:teuthology.orchestra.run.smithi154.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-13T18:25:13.150 INFO:teuthology.orchestra.run.smithi154.stdout:Removing obsolete dictionary files: 2024-09-13T18:25:13.488 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-13T18:25:13.502 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:25:13.646 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-09-13T18:25:13.780 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:25:13.881 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T18:25:14.007 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2024-09-13T18:25:14.125 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:25:14.251 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:25:14.270 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T18:25:14.371 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-09-13T18:25:14.377 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:25:14.530 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T18:25:14.649 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T18:25:14.649 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T18:25:14.656 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T18:25:15.138 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-09-13T18:25:15.239 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-13T18:25:15.365 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T18:25:15.491 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T18:25:15.619 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-09-13T18:25:15.793 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T18:25:15.881 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up dump (0.4b46-6) ... 2024-09-13T18:25:15.985 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-13T18:25:16.056 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:25:16.207 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T18:25:16.333 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T18:25:16.509 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T18:25:16.677 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T18:25:16.836 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:25:17.216 INFO:teuthology.orchestra.run.smithi154.stdout:Creating new PostgreSQL cluster 12/main ... 2024-09-13T18:25:17.236 INFO:teuthology.orchestra.run.smithi154.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2024-09-13T18:25:17.305 INFO:teuthology.orchestra.run.smithi154.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-13T18:25:17.305 INFO:teuthology.orchestra.run.smithi154.stdout:This user must also own the server process. 2024-09-13T18:25:17.306 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:17.306 INFO:teuthology.orchestra.run.smithi154.stdout:The database cluster will be initialized with locale "en_US". 2024-09-13T18:25:17.306 INFO:teuthology.orchestra.run.smithi154.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-13T18:25:17.306 INFO:teuthology.orchestra.run.smithi154.stdout:The default text search configuration will be set to "english". 2024-09-13T18:25:17.306 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:17.306 INFO:teuthology.orchestra.run.smithi154.stdout:Data page checksums are disabled. 2024-09-13T18:25:17.306 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:17.306 INFO:teuthology.orchestra.run.smithi154.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2024-09-13T18:25:17.307 INFO:teuthology.orchestra.run.smithi154.stdout:creating subdirectories ... ok 2024-09-13T18:25:17.307 INFO:teuthology.orchestra.run.smithi154.stdout:selecting dynamic shared memory implementation ... posix 2024-09-13T18:25:17.324 INFO:teuthology.orchestra.run.smithi154.stdout:selecting default max_connections ... 100 2024-09-13T18:25:17.343 INFO:teuthology.orchestra.run.smithi154.stdout:selecting default shared_buffers ... 128MB 2024-09-13T18:25:17.350 INFO:teuthology.orchestra.run.smithi154.stdout:selecting default time zone ... Etc/UTC 2024-09-13T18:25:17.350 INFO:teuthology.orchestra.run.smithi154.stdout:creating configuration files ... ok 2024-09-13T18:25:17.459 INFO:teuthology.orchestra.run.smithi154.stdout:running bootstrap script ... ok 2024-09-13T18:25:17.979 INFO:teuthology.orchestra.run.smithi154.stdout:performing post-bootstrap initialization ... ok 2024-09-13T18:25:18.762 INFO:teuthology.orchestra.run.smithi154.stdout:syncing data to disk ... ok 2024-09-13T18:25:18.762 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:18.762 INFO:teuthology.orchestra.run.smithi154.stdout:Success. You can now start the database server using: 2024-09-13T18:25:18.762 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:18.762 INFO:teuthology.orchestra.run.smithi154.stdout: pg_ctlcluster 12 main start 2024-09-13T18:25:18.762 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:19.078 INFO:teuthology.orchestra.run.smithi154.stdout:Ver Cluster Port Status Owner Data directory Log file 2024-09-13T18:25:19.079 INFO:teuthology.orchestra.run.smithi154.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2024-09-13T18:25:19.169 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-13T18:25:22.293 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:25:22.419 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:25:22.578 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-09-13T18:25:22.771 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:25:22.872 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2024-09-13T18:25:22.972 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-09-13T18:25:22.973 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T18:25:23.072 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-09-13T18:25:23.675 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:23.675 INFO:teuthology.orchestra.run.smithi154.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-13T18:25:23.676 INFO:teuthology.orchestra.run.smithi154.stdout:which means they are currently configured by ifupdown: 2024-09-13T18:25:23.676 INFO:teuthology.orchestra.run.smithi154.stdout:- enp3s0f1 2024-09-13T18:25:23.676 INFO:teuthology.orchestra.run.smithi154.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-13T18:25:23.676 INFO:teuthology.orchestra.run.smithi154.stdout:remove their configuration from /etc/network/interfaces. 2024-09-13T18:25:23.676 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:23.835 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-13T18:25:23.949 INFO:teuthology.orchestra.run.smithi146.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T18:25:23.955 DEBUG:teuthology.parallel:result is None 2024-09-13T18:25:24.149 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-13T18:25:24.673 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-13T18:25:25.187 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:25:25.345 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-09-13T18:25:25.513 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:25:25.622 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T18:25:25.723 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2024-09-13T18:25:25.857 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:25:25.984 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:25:26.143 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:25:26.434 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T18:25:26.434 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T18:25:26.948 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-09-13T18:25:27.113 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-13T18:25:27.239 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T18:25:27.424 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T18:25:27.718 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T18:25:34.938 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-09-13T18:25:35.080 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-09-13T18:25:35.939 INFO:teuthology.orchestra.run.smithi154.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T18:25:35.944 DEBUG:teuthology.parallel:result is None 2024-09-13T18:25:35.945 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:25:36.102 DEBUG:teuthology.orchestra.run.smithi146:> dpkg-query -W -f '${Version}' ceph 2024-09-13T18:25:36.127 INFO:teuthology.orchestra.run.smithi146.stdout:18.2.4-860-g6c6b083f-1focal 2024-09-13T18:25:36.128 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860-g6c6b083f-1focal 2024-09-13T18:25:36.128 INFO:teuthology.task.install:The correct ceph version 18.2.4-860-g6c6b083f-1focal is installed. 2024-09-13T18:25:36.130 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:25:36.278 DEBUG:teuthology.orchestra.run.smithi154:> dpkg-query -W -f '${Version}' ceph 2024-09-13T18:25:36.302 INFO:teuthology.orchestra.run.smithi154.stdout:18.2.4-860-g6c6b083f-1focal 2024-09-13T18:25:36.303 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860-g6c6b083f-1focal 2024-09-13T18:25:36.303 INFO:teuthology.task.install:The correct ceph version 18.2.4-860-g6c6b083f-1focal is installed. 2024-09-13T18:25:36.305 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T18:25:36.305 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:25:36.305 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T18:25:36.322 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:25:36.322 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T18:25:36.361 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T18:25:36.361 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:25:36.361 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T18:25:36.378 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T18:25:36.389 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:25:36.389 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T18:25:36.404 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T18:25:36.463 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T18:25:36.464 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:25:36.464 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T18:25:36.478 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T18:25:36.535 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:25:36.536 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T18:25:36.551 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T18:25:36.561 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T18:25:36.561 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:25:36.561 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T18:25:36.577 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T18:25:36.632 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:25:36.632 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T18:25:36.648 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T18:25:36.659 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T18:25:36.738 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T18:25:36.738 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /var/log/ceph 2024-09-13T18:25:36.740 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /var/log/ceph 2024-09-13T18:25:36.756 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T18:25:36.756 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T18:25:36.758 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T18:25:36.815 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T18:25:36.815 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T18:25:36.817 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T18:25:36.836 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-13T18:25:36.837 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-13T18:25:36.837 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-13T18:25:36.837 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:25:36.837 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T18:25:36.843 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T18:25:36.924 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:25:36.924 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T18:25:36.931 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T18:25:36.975 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T18:25:36.975 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds', 'SLOW_OPS', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T18:25:36.976 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903977', 'branch': 'wip-rishabh-testing-20240910.100730-reef', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}}', 'email': None, 'first_in_suite': False, 'job_id': '7903977', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240910.100730-reef'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds', 'SLOW_OPS', 'slow request'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240910.100730-reef', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 6439, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'sleep_before_teardown': 0, 'subset': '964/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240910.100730-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi146.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi154.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds', 'SLOW_OPS', 'slow request'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['suites/ffsb.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-13_14:42:16', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T18:25:36.976 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T18:25:36.979 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T18:25:36.984 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T18:25:36.986 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T18:25:37.039 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:25:37.039 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T18:25:37.047 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T18:25:37.047 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2024-09-13T18:25:37.097 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T18:25:37.097 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:25:37.097 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 368 Links: 1 2024-09-13T18:25:37.097 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:25:37.097 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-13 18:20:51.897165489 +0000 2024-09-13T18:25:37.097 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-13 18:20:51.697170095 +0000 2024-09-13T18:25:37.097 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-13 18:20:51.697170095 +0000 2024-09-13T18:25:37.097 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: - 2024-09-13T18:25:37.098 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T18:25:37.110 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-13T18:25:37.110 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-13T18:25:37.110 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000433006 s, 1.2 MB/s 2024-09-13T18:25:37.111 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T18:25:37.119 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2024-09-13T18:25:37.126 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T18:25:37.126 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:25:37.126 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 376 Links: 1 2024-09-13T18:25:37.126 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:25:37.126 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-13 18:20:52.117160421 +0000 2024-09-13T18:25:37.126 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-13 18:20:52.113160513 +0000 2024-09-13T18:25:37.126 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-13 18:20:52.113160513 +0000 2024-09-13T18:25:37.126 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: - 2024-09-13T18:25:37.127 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T18:25:37.140 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-13T18:25:37.140 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-13T18:25:37.140 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000426834 s, 1.2 MB/s 2024-09-13T18:25:37.141 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T18:25:37.191 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2024-09-13T18:25:37.198 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T18:25:37.199 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:25:37.199 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 383 Links: 1 2024-09-13T18:25:37.199 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:25:37.199 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-13 18:20:52.749145863 +0000 2024-09-13T18:25:37.199 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-13 18:20:52.561150193 +0000 2024-09-13T18:25:37.199 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-13 18:20:52.561150193 +0000 2024-09-13T18:25:37.199 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: - 2024-09-13T18:25:37.199 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T18:25:37.211 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-13T18:25:37.211 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-13T18:25:37.211 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000410789 s, 1.2 MB/s 2024-09-13T18:25:37.212 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T18:25:37.262 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2024-09-13T18:25:37.270 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T18:25:37.270 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:25:37.270 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 390 Links: 1 2024-09-13T18:25:37.270 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:25:37.270 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-13 18:20:52.993140242 +0000 2024-09-13T18:25:37.270 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-13 18:20:52.989140335 +0000 2024-09-13T18:25:37.270 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-13 18:20:52.989140335 +0000 2024-09-13T18:25:37.270 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: - 2024-09-13T18:25:37.271 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T18:25:37.329 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-13T18:25:37.329 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-13T18:25:37.329 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000366365 s, 1.4 MB/s 2024-09-13T18:25:37.330 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T18:25:37.339 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:25:37.339 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:25:37.339 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T18:25:37.346 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T18:25:37.346 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2024-09-13T18:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T18:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 344 Links: 1 2024-09-13T18:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-09-13 18:21:01.622776713 +0000 2024-09-13T18:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-09-13 18:21:01.414781498 +0000 2024-09-13T18:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-09-13 18:21:01.414781498 +0000 2024-09-13T18:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-09-13T18:25:37.396 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T18:25:37.408 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-09-13T18:25:37.408 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-09-13T18:25:37.408 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000451406 s, 1.1 MB/s 2024-09-13T18:25:37.409 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T18:25:37.418 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2024-09-13T18:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T18:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 348 Links: 1 2024-09-13T18:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-09-13 18:21:01.834771836 +0000 2024-09-13T18:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-09-13 18:21:01.830771927 +0000 2024-09-13T18:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-09-13 18:21:01.830771927 +0000 2024-09-13T18:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-09-13T18:25:37.426 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T18:25:37.481 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-09-13T18:25:37.481 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-09-13T18:25:37.481 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000389165 s, 1.3 MB/s 2024-09-13T18:25:37.482 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T18:25:37.491 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2024-09-13T18:25:37.497 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T18:25:37.498 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:25:37.498 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 354 Links: 1 2024-09-13T18:25:37.498 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:25:37.498 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-09-13 18:21:02.494756650 +0000 2024-09-13T18:25:37.498 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-09-13 18:21:02.270761804 +0000 2024-09-13T18:25:37.498 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-09-13 18:21:02.270761804 +0000 2024-09-13T18:25:37.498 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-09-13T18:25:37.498 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T18:25:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-09-13T18:25:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-09-13T18:25:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000391268 s, 1.3 MB/s 2024-09-13T18:25:37.513 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T18:25:37.521 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2024-09-13T18:25:37.526 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T18:25:37.526 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:25:37.526 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 361 Links: 1 2024-09-13T18:25:37.526 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:25:37.526 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-09-13 18:21:02.950746160 +0000 2024-09-13T18:25:37.526 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-09-13 18:21:02.726751313 +0000 2024-09-13T18:25:37.526 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-09-13 18:21:02.726751313 +0000 2024-09-13T18:25:37.526 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-09-13T18:25:37.527 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T18:25:37.537 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-09-13T18:25:37.537 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-09-13T18:25:37.537 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000362819 s, 1.4 MB/s 2024-09-13T18:25:37.538 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T18:25:37.546 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T18:25:37.546 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T18:25:37.546 INFO:tasks.ceph:Generating config... 2024-09-13T18:25:37.548 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-13T18:25:37.548 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-09-13T18:25:37.548 INFO:tasks.ceph:[client] debug client = 20 2024-09-13T18:25:37.548 INFO:tasks.ceph:[client] debug ms = 1 2024-09-13T18:25:37.548 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-13T18:25:37.548 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-13T18:25:37.548 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-13T18:25:37.548 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-13T18:25:37.548 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-13T18:25:37.548 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-09-13T18:25:37.548 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-09-13T18:25:37.548 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-09-13T18:25:37.548 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] ms inject delay max = 1 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-13T18:25:37.549 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-13T18:25:37.550 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-13T18:25:37.550 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-13T18:25:37.550 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T18:25:37.550 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T18:25:37.550 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T18:25:37.550 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T18:25:37.550 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T18:25:37.550 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-13T18:25:37.550 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-13T18:25:37.550 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T18:25:37.550 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-13T18:25:37.550 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T18:25:37.550 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-13T18:25:37.550 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-13T18:25:37.550 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] filestore flush min = 0 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] ms inject delay max = 1 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T18:25:37.551 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-13T18:25:37.552 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T18:25:37.552 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-13T18:25:37.552 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T18:25:37.552 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T18:25:37.592 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T18:25:37.610 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T18:25:37.692 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T18:25:37.706 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.146'), ('mon.b', '172.21.15.154'), ('mon.c', '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]')] 2024-09-13T18:25:37.706 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.146,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-13T18:25:37.707 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:25:37.707 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T18:25:37.714 DEBUG:teuthology.orchestra.run.smithi146:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.146 --add b 172.21.15.154 --addv c '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:25:37.797 INFO:teuthology.orchestra.run.smithi146.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:25:37.798 INFO:teuthology.orchestra.run.smithi146.stdout:monmaptool: generated fsid 509e851e-cdbc-4e57-a792-d3d1aeeeba86 2024-09-13T18:25:37.798 INFO:teuthology.orchestra.run.smithi146.stdout:setting min_mon_release = pacific 2024-09-13T18:25:37.798 INFO:teuthology.orchestra.run.smithi146.stdout:epoch 0 2024-09-13T18:25:37.798 INFO:teuthology.orchestra.run.smithi146.stdout:fsid 509e851e-cdbc-4e57-a792-d3d1aeeeba86 2024-09-13T18:25:37.798 INFO:teuthology.orchestra.run.smithi146.stdout:last_changed 2024-09-13T18:25:37.792892+0000 2024-09-13T18:25:37.798 INFO:teuthology.orchestra.run.smithi146.stdout:created 2024-09-13T18:25:37.792892+0000 2024-09-13T18:25:37.798 INFO:teuthology.orchestra.run.smithi146.stdout:min_mon_release 16 (pacific) 2024-09-13T18:25:37.798 INFO:teuthology.orchestra.run.smithi146.stdout:election_strategy: 1 2024-09-13T18:25:37.798 INFO:teuthology.orchestra.run.smithi146.stdout:0: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.a 2024-09-13T18:25:37.798 INFO:teuthology.orchestra.run.smithi146.stdout:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-09-13T18:25:37.798 INFO:teuthology.orchestra.run.smithi146.stdout:2: [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] mon.c 2024-09-13T18:25:37.798 INFO:teuthology.orchestra.run.smithi146.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T18:25:37.800 DEBUG:teuthology.orchestra.run.smithi146:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T18:25:37.806 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 509e851e-cdbc-4e57-a792-d3d1aeeeba86... 2024-09-13T18:25:37.807 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T18:25:37.809 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T18:25:37.839 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2024-09-13T18:25:37.839 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.840 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2024-09-13T18:25:37.841 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.146,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]" 2024-09-13T18:25:37.842 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = 509e851e-cdbc-4e57-a792-d3d1aeeeba86 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2024-09-13T18:25:37.843 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.844 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore compression mode = aggressive 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 20 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 20 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 10 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: filestore flush min = 0 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: mon client directed command retry = 5 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2024-09-13T18:25:37.845 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout: ms inject delay max = 1 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout: ms inject delay probability = 0.005 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout: ms inject delay type = client mds 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout: ms inject socket failures = 2500 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout: osd blocked scrub grace period = 3600 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout: osd max markdown count = 1000 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2024-09-13T18:25:37.846 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:25:37.847 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2024-09-13T18:25:37.848 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on pool no app = False 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout: client_shutdown_timeout = 120 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-13T18:25:37.849 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.e] 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.f] 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2024-09-13T18:25:37.850 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: mds bal fragment size max = 10000 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: mds bal merge size = 5 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: mds bal split bits = 3 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: mds bal split size = 100 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: mds oft prefetch dirfrags = True 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: mds_dir_prefetch = True 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: mds_sleep_rank_change = 5000000.0 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: mon client directed command retry = 5 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: ms inject delay max = 1 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: ms inject delay probability = 0.005 2024-09-13T18:25:37.851 INFO:teuthology.orchestra.run.smithi146.stdout: ms inject delay type = client mds osd 2024-09-13T18:25:37.852 INFO:teuthology.orchestra.run.smithi146.stdout: ms inject socket failures = 2500 2024-09-13T18:25:37.852 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-13T18:25:37.852 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-13T18:25:37.852 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-13T18:25:37.853 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2024-09-13T18:25:37.853 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2024-09-13T18:25:37.853 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T18:25:37.853 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2024-09-13T18:25:37.854 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T18:25:37.855 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.146,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]" 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 509e851e-cdbc-4e57-a792-d3d1aeeeba86 2024-09-13T18:25:37.856 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2024-09-13T18:25:37.857 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2024-09-13T18:25:37.858 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore compression mode = aggressive 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: filestore flush min = 0 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: mon client directed command retry = 5 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2024-09-13T18:25:37.859 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject delay max = 1 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject delay probability = 0.005 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject delay type = client mds 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject socket failures = 2500 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout: osd blocked scrub grace period = 3600 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout: osd max markdown count = 1000 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-09-13T18:25:37.860 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:25:37.861 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2024-09-13T18:25:37.862 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2024-09-13T18:25:37.862 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:25:37.862 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.862 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T18:25:37.862 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T18:25:37.862 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T18:25:37.862 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.862 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T18:25:37.862 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T18:25:37.862 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T18:25:37.862 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.862 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2024-09-13T18:25:37.862 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on pool no app = False 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout: client_shutdown_timeout = 120 2024-09-13T18:25:37.863 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.e] 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.f] 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2024-09-13T18:25:37.864 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: mds bal fragment size max = 10000 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: mds bal merge size = 5 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: mds bal split bits = 3 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: mds bal split size = 100 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = True 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: mds oft prefetch dirfrags = True 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = True 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: mds_dir_prefetch = True 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: mds_sleep_rank_change = 5000000.0 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: mon client directed command retry = 5 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject delay max = 1 2024-09-13T18:25:37.865 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject delay probability = 0.005 2024-09-13T18:25:37.866 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject delay type = client mds osd 2024-09-13T18:25:37.866 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject socket failures = 2500 2024-09-13T18:25:37.866 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2024-09-13T18:25:37.866 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2024-09-13T18:25:37.866 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2024-09-13T18:25:37.866 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T18:25:37.866 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T18:25:37.950 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T18:25:37.951 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:25:37.951 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T18:25:37.957 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:25:37.957 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T18:25:37.964 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi146.front.sepia.ceph.com 2024-09-13T18:25:37.964 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:25:37.964 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T18:25:37.964 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T18:25:37.988 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:25:37.988 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:25:38.037 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi154.front.sepia.ceph.com 2024-09-13T18:25:38.037 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:25:38.037 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T18:25:38.037 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T18:25:38.065 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:25:38.065 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:25:38.070 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T18:25:38.070 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T18:25:38.070 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T18:25:38.123 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T18:25:38.126 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T18:25:38.175 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T18:25:38.193 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T18:25:38.193 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T18:25:38.243 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T18:25:38.246 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-13T18:25:38.261 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T18:25:38.308 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T18:25:38.311 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-13T18:25:38.324 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-09-13T18:25:38.368 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-09-13T18:25:38.371 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-09-13T18:25:38.381 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T18:25:38.435 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T18:25:38.438 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-13T18:25:38.452 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T18:25:38.502 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T18:25:38.505 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-13T18:25:38.517 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-09-13T18:25:38.564 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-09-13T18:25:38.567 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-09-13T18:25:38.578 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T18:25:38.578 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T18:25:38.620 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T18:25:38.633 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T18:25:38.633 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T18:25:38.634 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T18:25:38.691 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:25:38.691 INFO:tasks.ceph:role: osd.0 2024-09-13T18:25:38.691 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-13T18:25:38.691 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T18:25:39.214 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:25:39.215 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:25:39.215 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:25:39.215 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 2024-09-13T18:25:39.215 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:25:39.215 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:25:39.215 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:25:39.215 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:25:39.215 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:25:39.215 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:25:39.220 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-09-13T18:25:39.220 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T18:25:39.298 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T18:25:39.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:25:39.311 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:25:39.311 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T18:25:39.323 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:25:39.323 INFO:tasks.ceph:role: osd.1 2024-09-13T18:25:39.323 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-13T18:25:39.323 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T18:25:39.849 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:25:39.850 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:25:39.850 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:25:39.850 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 2024-09-13T18:25:39.850 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:25:39.850 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:25:39.850 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:25:39.850 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:25:39.850 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:25:39.850 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:25:39.851 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-09-13T18:25:39.852 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T18:25:39.907 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T18:25:39.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:25:39.919 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:25:39.920 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T18:25:39.978 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:25:39.978 INFO:tasks.ceph:role: osd.2 2024-09-13T18:25:39.978 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-13T18:25:39.978 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T18:25:40.449 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:25:40.449 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:25:40.449 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:25:40.450 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 2024-09-13T18:25:40.450 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:25:40.450 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:25:40.450 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:25:40.450 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:25:40.450 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:25:40.450 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:25:40.451 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-09-13T18:25:40.451 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T18:25:40.510 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T18:25:40.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:25:40.551 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:25:40.551 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T18:25:40.619 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:25:40.620 INFO:tasks.ceph:role: osd.3 2024-09-13T18:25:40.620 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-13T18:25:40.620 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T18:25:41.162 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:25:41.162 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:25:41.163 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:25:41.163 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 2024-09-13T18:25:41.163 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:25:41.163 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:25:41.163 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:25:41.163 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:25:41.163 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:25:41.163 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:25:41.164 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-09-13T18:25:41.164 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T18:25:41.247 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T18:25:41.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:25:41.260 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:25:41.261 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:25:41.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:41.335+0000 7f5149915880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T18:25:41.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:41.335+0000 7f5149915880 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T18:25:41.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:41.335+0000 7f5149915880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T18:25:41.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:41.335+0000 7f5149915880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T18:25:41.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:41.335+0000 7f5149915880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T18:25:41.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:41.343+0000 7f5149915880 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T18:25:44.390 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T18:25:44.404 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:25:44.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:44.439+0000 7f50060a3880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T18:25:44.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:44.439+0000 7f50060a3880 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T18:25:44.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:44.439+0000 7f50060a3880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T18:25:44.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:44.439+0000 7f50060a3880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T18:25:44.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:44.439+0000 7f50060a3880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T18:25:44.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:44.439+0000 7f50060a3880 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T18:25:47.429 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T18:25:47.446 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:25:47.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:47.479+0000 7f5b3094d880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T18:25:47.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:47.479+0000 7f5b3094d880 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T18:25:47.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:47.479+0000 7f5b3094d880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T18:25:47.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:47.479+0000 7f5b3094d880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T18:25:47.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:47.479+0000 7f5b3094d880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T18:25:47.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:47.479+0000 7f5b3094d880 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T18:25:50.478 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T18:25:50.535 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:25:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:50.567+0000 7fe86c5b1880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T18:25:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:50.567+0000 7fe86c5b1880 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T18:25:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:50.567+0000 7fe86c5b1880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T18:25:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:50.567+0000 7fe86c5b1880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T18:25:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:50.567+0000 7fe86c5b1880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T18:25:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:25:50.571+0000 7fe86c5b1880 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T18:25:53.566 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T18:25:53.582 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T18:25:53.599 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T18:25:53.600 INFO:tasks.ceph:role: osd.4 2024-09-13T18:25:53.600 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-13T18:25:53.600 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T18:25:54.153 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:25:54.153 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:25:54.153 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:25:54.153 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 2024-09-13T18:25:54.153 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:25:54.153 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:25:54.153 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:25:54.154 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:25:54.154 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:25:54.154 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:25:54.155 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-09-13T18:25:54.155 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T18:25:54.247 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T18:25:54.260 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:25:54.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:25:54.261 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T18:25:54.320 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T18:25:54.320 INFO:tasks.ceph:role: osd.5 2024-09-13T18:25:54.320 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-13T18:25:54.321 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T18:25:54.846 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:25:54.847 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:25:54.847 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:25:54.847 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 2024-09-13T18:25:54.847 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:25:54.847 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:25:54.847 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:25:54.847 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:25:54.847 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:25:54.847 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:25:54.849 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-09-13T18:25:54.849 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T18:25:54.888 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T18:25:54.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:25:54.898 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:25:54.898 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T18:25:54.956 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T18:25:54.956 INFO:tasks.ceph:role: osd.6 2024-09-13T18:25:54.956 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-13T18:25:54.957 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T18:25:55.422 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:25:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:25:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:25:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 2024-09-13T18:25:55.422 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:25:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:25:55.422 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:25:55.422 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:25:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:25:55.423 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:25:55.424 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-09-13T18:25:55.425 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T18:25:55.479 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T18:25:55.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:25:55.492 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:25:55.493 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T18:25:55.509 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T18:25:55.509 INFO:tasks.ceph:role: osd.7 2024-09-13T18:25:55.509 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-13T18:25:55.509 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T18:25:56.024 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:25:56.024 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:25:56.024 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:25:56.024 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 2024-09-13T18:25:56.024 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:25:56.025 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:25:56.025 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:25:56.025 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:25:56.025 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:25:56.025 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:25:56.026 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-09-13T18:25:56.026 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T18:25:56.086 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T18:25:56.330 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:25:56.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:25:56.331 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:25:56.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:25:56.423+0000 7f126d8d5880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T18:25:56.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:25:56.427+0000 7f126d8d5880 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T18:25:56.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:25:56.427+0000 7f126d8d5880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T18:25:56.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:25:56.427+0000 7f126d8d5880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T18:25:56.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:25:56.427+0000 7f126d8d5880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T18:25:56.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:25:56.431+0000 7f126d8d5880 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T18:25:59.424 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T18:25:59.441 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:25:59.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:25:59.478+0000 7f0c0e471880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T18:25:59.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:25:59.478+0000 7f0c0e471880 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T18:25:59.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:25:59.478+0000 7f0c0e471880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T18:25:59.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:25:59.478+0000 7f0c0e471880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T18:25:59.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:25:59.478+0000 7f0c0e471880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T18:25:59.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:25:59.482+0000 7f0c0e471880 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T18:26:02.484 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T18:26:02.498 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:26:02.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:02.542+0000 7faa8448b880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T18:26:02.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:02.542+0000 7faa8448b880 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T18:26:02.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:02.542+0000 7faa8448b880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T18:26:02.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:02.542+0000 7faa8448b880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T18:26:02.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:02.542+0000 7faa8448b880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T18:26:02.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:02.542+0000 7faa8448b880 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T18:26:05.531 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T18:26:05.546 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:26:05.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:05.630+0000 7fe7316a7880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T18:26:05.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:05.630+0000 7fe7316a7880 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T18:26:05.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:05.630+0000 7fe7316a7880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T18:26:05.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:05.630+0000 7fe7316a7880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T18:26:05.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:05.630+0000 7fe7316a7880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T18:26:05.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:05.630+0000 7fe7316a7880 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T18:26:06.978 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:26:06.986 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:26:08.627 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T18:26:08.642 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T18:26:08.643 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:08.643 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T18:26:08.659 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:08.659 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-13T18:26:08.674 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:08.674 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-13T18:26:08.686 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:08.686 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-09-13T18:26:08.702 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:08.702 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T18:26:08.761 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:08.761 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T18:26:08.775 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:08.775 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T18:26:08.789 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:08.789 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T18:26:08.845 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:26:08.845 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T18:26:08.861 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:26:08.861 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-13T18:26:08.921 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:26:08.921 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-13T18:26:08.930 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:26:08.931 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-09-13T18:26:08.941 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:26:08.941 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T18:26:08.952 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:26:08.952 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T18:26:09.008 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:26:09.008 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T18:26:09.021 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:26:09.021 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T18:26:09.033 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:09.033 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T18:26:09.040 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T18:26:09.040 DEBUG:teuthology.orchestra.run.smithi146:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T18:26:09.042 DEBUG:teuthology.orchestra.run.smithi154:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T18:26:09.079 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.y] 2024-09-13T18:26:09.079 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCig+RmbnsMBxAABPQDYnSAmiscIpIfkG1dXA== 2024-09-13T18:26:09.079 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-09-13T18:26:09.080 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCig+RmY801DhAAY+4iA4x+/yXbi+kvrxGbVg== 2024-09-13T18:26:09.080 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-09-13T18:26:09.080 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCig+RmDk4TEhAAbM3P9cnlXofLdm8HiPcFKA== 2024-09-13T18:26:09.080 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.e] 2024-09-13T18:26:09.080 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCig+RmhP+sFRAA3Zo6UiHmvRG7I1hbU6A3Lg== 2024-09-13T18:26:09.080 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.0] 2024-09-13T18:26:09.080 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQClg+RmRiFPFBAAFW/+UiyasB4VBfdE39JgKg== 2024-09-13T18:26:09.080 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.1] 2024-09-13T18:26:09.080 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCog+RmcPhuGhAAfh1T1iMpp0JtwU4MfIBMVQ== 2024-09-13T18:26:09.080 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.2] 2024-09-13T18:26:09.080 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCrg+RmLj7QHBAAeeOzRjcV9jTMG0jHQevVEQ== 2024-09-13T18:26:09.080 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.3] 2024-09-13T18:26:09.080 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCug+Rm8s0mIhAA9UWvFcT0BV2Ibv2T3PV0Kw== 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.x] 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCig+RmHCCCChAAEoLfKfKTmfIZeWl/z42Rbg== 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCig+RmBigFGhAAk4nL/5JGfB5pK3UuGAI2Uw== 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCig+RmpGP9HRAA9FvEbW4aEDBIHBdlVYoxVA== 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.f] 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCig+RmzBWsIRAA/6xuWSx5+vwHCyreTCCs1g== 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.4] 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQC0g+Rm03uXGRAAuQRRLg6ZtbgGTngfmOv2Zg== 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.5] 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQC3g+RmvhvfHBAAiQ7r8FoBroD+PxPuHzqknA== 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.6] 2024-09-13T18:26:09.081 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQC6g+RmZQ+KIBAABS8QNQ1pCT///o9JIEYXvA== 2024-09-13T18:26:09.082 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.7] 2024-09-13T18:26:09.082 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQC9g+RmMsfYJRAAZRBjd1XeRlt+H/89ZEL5Ig== 2024-09-13T18:26:09.082 INFO:teuthology.orchestra.run.smithi146.stdout:[client.0] 2024-09-13T18:26:09.082 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCig+Rm+cOuJBAAyUoWr6HZES9YyGSEE0Ji7A== 2024-09-13T18:26:09.090 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.y] 2024-09-13T18:26:09.090 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCig+RmbnsMBxAABPQDYnSAmiscIpIfkG1dXA== 2024-09-13T18:26:09.090 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2024-09-13T18:26:09.091 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCig+RmY801DhAAY+4iA4x+/yXbi+kvrxGbVg== 2024-09-13T18:26:09.091 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2024-09-13T18:26:09.091 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCig+RmDk4TEhAAbM3P9cnlXofLdm8HiPcFKA== 2024-09-13T18:26:09.091 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.e] 2024-09-13T18:26:09.091 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCig+RmhP+sFRAA3Zo6UiHmvRG7I1hbU6A3Lg== 2024-09-13T18:26:09.091 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.0] 2024-09-13T18:26:09.091 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQClg+RmRiFPFBAAFW/+UiyasB4VBfdE39JgKg== 2024-09-13T18:26:09.091 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.1] 2024-09-13T18:26:09.091 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCog+RmcPhuGhAAfh1T1iMpp0JtwU4MfIBMVQ== 2024-09-13T18:26:09.091 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.2] 2024-09-13T18:26:09.091 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCrg+RmLj7QHBAAeeOzRjcV9jTMG0jHQevVEQ== 2024-09-13T18:26:09.091 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.3] 2024-09-13T18:26:09.092 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCug+Rm8s0mIhAA9UWvFcT0BV2Ibv2T3PV0Kw== 2024-09-13T18:26:09.092 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.x] 2024-09-13T18:26:09.092 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCig+RmHCCCChAAEoLfKfKTmfIZeWl/z42Rbg== 2024-09-13T18:26:09.092 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2024-09-13T18:26:09.092 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCig+RmBigFGhAAk4nL/5JGfB5pK3UuGAI2Uw== 2024-09-13T18:26:09.092 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2024-09-13T18:26:09.092 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCig+RmpGP9HRAA9FvEbW4aEDBIHBdlVYoxVA== 2024-09-13T18:26:09.092 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.f] 2024-09-13T18:26:09.092 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCig+RmzBWsIRAA/6xuWSx5+vwHCyreTCCs1g== 2024-09-13T18:26:09.092 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.4] 2024-09-13T18:26:09.092 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC0g+Rm03uXGRAAuQRRLg6ZtbgGTngfmOv2Zg== 2024-09-13T18:26:09.092 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.5] 2024-09-13T18:26:09.092 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC3g+RmvhvfHBAAiQ7r8FoBroD+PxPuHzqknA== 2024-09-13T18:26:09.093 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.6] 2024-09-13T18:26:09.093 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC6g+RmZQ+KIBAABS8QNQ1pCT///o9JIEYXvA== 2024-09-13T18:26:09.093 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.7] 2024-09-13T18:26:09.093 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC9g+RmMsfYJRAAZRBjd1XeRlt+H/89ZEL5Ig== 2024-09-13T18:26:09.093 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2024-09-13T18:26:09.093 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCig+Rm+cOuJBAAyUoWr6HZES9YyGSEE0Ji7A== 2024-09-13T18:26:09.094 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T18:26:09.128 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T18:26:09.179 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:26:09.181 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:26:09.232 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:26:09.234 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:26:09.288 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:26:09.290 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:26:09.344 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.346 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.393 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.395 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.485 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.487 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.537 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.540 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.589 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T18:26:09.591 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T18:26:09.643 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:26:09.645 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:26:09.696 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:26:09.698 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:26:09.748 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:26:09.750 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:26:09.801 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.803 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.851 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.854 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.899 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.901 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.993 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:09.996 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:26:10.048 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T18:26:10.050 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T18:26:10.104 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T18:26:10.105 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T18:26:10.115 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T18:26:10.200 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T18:26:10.217 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T18:26:10.235 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T18:26:10.314 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T18:26:10.331 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T18:26:10.340 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T18:26:10.423 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T18:26:10.438 DEBUG:teuthology.orchestra.run.smithi146:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:26:10.440 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:26:10.445 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T18:26:10.445 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T18:26:10.445 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T18:26:10.447 INFO:tasks.ceph.mon.a:Started 2024-09-13T18:26:10.448 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T18:26:10.448 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T18:26:10.450 INFO:tasks.ceph.mon.b:Started 2024-09-13T18:26:10.450 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T18:26:10.450 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T18:26:10.452 INFO:tasks.ceph.mon.c:Started 2024-09-13T18:26:10.452 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T18:26:10.452 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T18:26:10.453 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T18:26:10.455 INFO:tasks.ceph.mgr.y:Started 2024-09-13T18:26:10.455 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T18:26:10.455 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T18:26:10.457 INFO:tasks.ceph.mgr.x:Started 2024-09-13T18:26:10.457 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T18:26:10.457 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T18:26:10.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.552+0000 7f63e4d25700 1 Processor -- start 2024-09-13T18:26:10.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.552+0000 7f63e4d25700 1 -- start start 2024-09-13T18:26:10.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.552+0000 7f63e4d25700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:10.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63e4d25700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:10.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63e4d25700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e006e480 con 0x7f63e00c2670 2024-09-13T18:26:10.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63e4d25700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e006e2a0 con 0x7f63e00c2b90 2024-09-13T18:26:10.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63e4d25700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e006e700 con 0x7f63e01506d0 2024-09-13T18:26:10.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63de59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:10.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63de59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:45578/0 (socket says 172.21.15.146:45578) 2024-09-13T18:26:10.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63de59c700 1 -- 172.21.15.146:0/2270669520 learned_addr learned my addr 172.21.15.146:0/2270669520 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:10.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63ded9d700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:10.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63de59c700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 msgr2=0x7f63e0066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:26:10.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63de59c700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 msgr2=0x7f63e0066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:26:10.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63de59c700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:26:10.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63de59c700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:26:10.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63ded9d700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 msgr2=0x7f63e00c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:26:10.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63ded9d700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 msgr2=0x7f63e00c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:26:10.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63ded9d700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:26:10.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.556+0000 7f63ded9d700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:26:10.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.756+0000 7f63de59c700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:10.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.756+0000 7f63ded9d700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:10.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.756+0000 7f63de59c700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 msgr2=0x7f63e0066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:26:10.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.756+0000 7f63de59c700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 msgr2=0x7f63e0066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:26:10.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.756+0000 7f63de59c700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:26:10.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.756+0000 7f63de59c700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:26:10.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.756+0000 7f63ded9d700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 msgr2=0x7f63e00c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:26:10.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.756+0000 7f63ded9d700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 msgr2=0x7f63e00c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:26:10.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.756+0000 7f63ded9d700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:26:10.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:10.756+0000 7f63ded9d700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:26:11.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.156+0000 7f63ded9d700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:11.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.156+0000 7f63de59c700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:11.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.156+0000 7f63de59c700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 msgr2=0x7f63e0066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:26:11.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.156+0000 7f63de59c700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 msgr2=0x7f63e0066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:26:11.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.156+0000 7f63ded9d700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 msgr2=0x7f63e00c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:26:11.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.156+0000 7f63de59c700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:26:11.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.156+0000 7f63ded9d700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 msgr2=0x7f63e00c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:26:11.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.156+0000 7f63ded9d700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:26:11.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.156+0000 7f63de59c700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:26:11.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.156+0000 7f63ded9d700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:26:11.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.956+0000 7f63ded9d700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:11.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.960+0000 7f63de59c700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:11.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.960+0000 7f63ded9d700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 msgr2=0x7f63e00c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:26:11.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.960+0000 7f63ded9d700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 msgr2=0x7f63e00c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:26:11.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.960+0000 7f63ded9d700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:26:11.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.960+0000 7f63de59c700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 msgr2=0x7f63e0066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:26:11.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.960+0000 7f63de59c700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 msgr2=0x7f63e0066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:26:11.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.960+0000 7f63ded9d700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:26:11.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.960+0000 7f63de59c700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:26:11.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:11.960+0000 7f63de59c700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:26:13.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.556+0000 7f63cffff700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 msgr2=0x7f63e0066090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:26:13.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.556+0000 7f63cffff700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.154:3300/0 conn(0x7f63e01506d0 0x7f63e0066090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:13.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.556+0000 7f63cffff700 1 -- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 msgr2=0x7f63e00c7410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:26:13.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.556+0000 7f63cffff700 1 --2- 172.21.15.146:0/2270669520 >> v2:172.21.15.146:3300/0 conn(0x7f63e00c2b90 0x7f63e00c7410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:13.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.556+0000 7f63cffff700 1 -- 172.21.15.146:0/2270669520 >> v1:172.21.15.154:6789/0 conn(0x7f63e00c2670 legacy=0x7f63e00c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:13.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.556+0000 7f63cffff700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:13.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.556+0000 7f63cffff700 1 -- 172.21.15.146:0/2270669520 --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e00c46e0 con 0x7f63c0003720 2024-09-13T18:26:13.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.556+0000 7f63cffff700 1 -- 172.21.15.146:0/2270669520 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e006e2a0 con 0x7f63c00073e0 2024-09-13T18:26:13.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.556+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:13.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.556+0000 7f63ddd9b700 1 -- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 msgr2=0x7f63c00098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:26:13.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.556+0000 7f63ddd9b700 1 -- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 msgr2=0x7f63c00098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:26:13.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.556+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:26:13.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.556+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:26:13.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.756+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:13.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.756+0000 7f63ddd9b700 1 -- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 msgr2=0x7f63c00098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:26:13.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.756+0000 7f63ddd9b700 1 -- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 msgr2=0x7f63c00098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:26:13.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.756+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:26:13.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:13.756+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:26:14.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:14.156+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:14.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:14.156+0000 7f63ddd9b700 1 -- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 msgr2=0x7f63c00098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:26:14.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:14.156+0000 7f63ddd9b700 1 -- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 msgr2=0x7f63c00098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:26:14.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:14.156+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:26:14.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:14.156+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:26:14.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:14.960+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:14.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:14.960+0000 7f63ddd9b700 1 -- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 msgr2=0x7f63c00098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:26:14.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:14.960+0000 7f63ddd9b700 1 -- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 msgr2=0x7f63c00098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:26:14.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:14.960+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:26:14.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:14.960+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:26:15.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.636+0000 7f63dd59a700 1 -- 172.21.15.146:0/2270669520 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417772111 0 0) 0x7f63e00c46e0 con 0x7f63c0003720 2024-09-13T18:26:15.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.636+0000 7f63dd59a700 1 -- 172.21.15.146:0/2270669520 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63c80035d0 con 0x7f63c0003720 2024-09-13T18:26:15.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 -- 172.21.15.146:0/2270669520 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3661136075 0 0) 0x7f63c80035d0 con 0x7f63c0003720 2024-09-13T18:26:15.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 -- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 msgr2=0x7f63c00098a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:26:15.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c00073e0 0x7f63c00098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 -- 172.21.15.146:0/2270669520 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c000b080 con 0x7f63c0003720 2024-09-13T18:26:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 -- 172.21.15.146:0/2270669520 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f63d0009590 con 0x7f63c0003720 2024-09-13T18:26:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 -- 172.21.15.146:0/2270669520 >> v1:172.21.15.146:6789/0 conn(0x7f63c0003720 legacy=0x7f63c0005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63ded9d700 1 -- 172.21.15.146:0/2270669520 reap_dead start 2024-09-13T18:26:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63c8003a20 0x7f63c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c8004870 0x7f63c8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c8005300 0x7f63c800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 -- 172.21.15.146:0/2270669520 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e006e2a0 con 0x7f63c8004870 2024-09-13T18:26:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 -- 172.21.15.146:0/2270669520 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c800c250 con 0x7f63c8003a20 2024-09-13T18:26:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 -- 172.21.15.146:0/2270669520 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c800c410 con 0x7f63c8005300 2024-09-13T18:26:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c8004870 0x7f63c8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63de59c700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63c8003a20 0x7f63c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:15.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63ded9d700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c8005300 0x7f63c800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:15.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63ddd9b700 1 -- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c8005300 msgr2=0x7f63c800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:15.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c8005300 0x7f63c800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:15.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63ddd9b700 1 -- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63c8003a20 msgr2=0x7f63c8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:15.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63c8003a20 0x7f63c8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:15.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63ddd9b700 1 -- 172.21.15.146:0/2270669520 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c80035d0 con 0x7f63c8004870 2024-09-13T18:26:15.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63ddd9b700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c8004870 0x7f63c8004cf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f63d8001d60 tx=0x7f63c80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3965c731ae9ced server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:15.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 -- 172.21.15.146:0/2270669520 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8005540 con 0x7f63c8004870 2024-09-13T18:26:15.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 -- 172.21.15.146:0/2270669520 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d80056a0 con 0x7f63c8004870 2024-09-13T18:26:15.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63dd59a700 1 -- 172.21.15.146:0/2270669520 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8016450 con 0x7f63c8004870 2024-09-13T18:26:15.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63e4d25700 1 -- 172.21.15.146:0/2270669520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c8004870 msgr2=0x7f63c8004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:15.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.640+0000 7f63e4d25700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c8004870 0x7f63c8004cf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f63d8001d60 tx=0x7f63c80058a0 comp rx=0 tx=0).stop 2024-09-13T18:26:15.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 -- 172.21.15.146:0/2270669520 shutdown_connections 2024-09-13T18:26:15.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c8005300 0x7f63c800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:15.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c8004870 0x7f63c8004cf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:15.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 --2- 172.21.15.146:0/2270669520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63c8003a20 0x7f63c8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:15.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 -- 172.21.15.146:0/2270669520 >> 172.21.15.146:0/2270669520 conn(0x7f63e0066920 msgr2=0x7f63e006acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:15.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 -- 172.21.15.146:0/2270669520 shutdown_connections 2024-09-13T18:26:15.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 -- 172.21.15.146:0/2270669520 wait complete. 2024-09-13T18:26:15.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 Processor -- start 2024-09-13T18:26:15.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 -- start start 2024-09-13T18:26:15.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63c8003a20 0x7f63e01ed090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:15.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c8004870 0x7f63e01459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:15.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c8005300 0x7f63e00cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:15.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e00ca4e0 con 0x7f63c8005300 2024-09-13T18:26:15.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e00c9630 con 0x7f63c8003a20 2024-09-13T18:26:15.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63e4d25700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e00c98d0 con 0x7f63c8004870 2024-09-13T18:26:15.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63ded9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c8005300 0x7f63e00cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:15.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63de59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63c8003a20 0x7f63e01ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:15.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63ddd9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c8004870 0x7f63e01459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:15.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63ded9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c8005300 0x7f63e00cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34610/0 (socket says 172.21.15.146:34610) 2024-09-13T18:26:15.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63de59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63c8003a20 0x7f63e01ed090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:54810/0 (socket says 172.21.15.146:54810) 2024-09-13T18:26:15.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.644+0000 7f63ded9d700 1 -- 172.21.15.146:0/1507508239 learned_addr learned my addr 172.21.15.146:0/1507508239 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:15.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63ded9d700 1 -- 172.21.15.146:0/1507508239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c8004870 msgr2=0x7f63e01459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:15.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63ded9d700 1 --2- 172.21.15.146:0/1507508239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c8004870 0x7f63e01459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:15.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63ded9d700 1 -- 172.21.15.146:0/1507508239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63c8003a20 msgr2=0x7f63e01ed090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:15.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63ded9d700 1 --2- 172.21.15.146:0/1507508239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63c8003a20 0x7f63e01ed090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:15.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63ded9d700 1 -- 172.21.15.146:0/1507508239 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e00c9bb0 con 0x7f63c8005300 2024-09-13T18:26:15.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63ded9d700 1 --2- 172.21.15.146:0/1507508239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c8005300 0x7f63e00cab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f63c0007320 tx=0x7f63d4012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:15.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63ceffd700 1 -- 172.21.15.146:0/1507508239 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d4012ea0 con 0x7f63c8005300 2024-09-13T18:26:15.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63ceffd700 1 -- 172.21.15.146:0/1507508239 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d400c040 con 0x7f63c8005300 2024-09-13T18:26:15.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63e4d25700 1 -- 172.21.15.146:0/1507508239 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e00c9e90 con 0x7f63c8005300 2024-09-13T18:26:15.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63ceffd700 1 -- 172.21.15.146:0/1507508239 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d4006ca0 con 0x7f63c8005300 2024-09-13T18:26:15.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63e4d25700 1 -- 172.21.15.146:0/1507508239 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e014c8c0 con 0x7f63c8005300 2024-09-13T18:26:15.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63ceffd700 1 -- 172.21.15.146:0/1507508239 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f63d401e070 con 0x7f63c8005300 2024-09-13T18:26:15.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63ceffd700 1 -- 172.21.15.146:0/1507508239 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f63d4006520 con 0x7f63c8005300 2024-09-13T18:26:15.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.648+0000 7f63e4d25700 1 -- 172.21.15.146:0/1507508239 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e006e700 con 0x7f63c8005300 2024-09-13T18:26:15.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.656+0000 7f63ceffd700 1 -- 172.21.15.146:0/1507508239 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f63d400f040 con 0x7f63c8005300 2024-09-13T18:26:15.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.728+0000 7f63e4d25700 1 -- 172.21.15.146:0/1507508239 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f63e00cc4c0 con 0x7f63c8005300 2024-09-13T18:26:15.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.732+0000 7f63ceffd700 1 -- 172.21.15.146:0/1507508239 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f63d40066e0 con 0x7f63c8005300 2024-09-13T18:26:15.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.732+0000 7f63e4d25700 1 -- 172.21.15.146:0/1507508239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c8005300 msgr2=0x7f63e00cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:15.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.732+0000 7f63e4d25700 1 --2- 172.21.15.146:0/1507508239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c8005300 0x7f63e00cab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f63c0007320 tx=0x7f63d4012760 comp rx=0 tx=0).stop 2024-09-13T18:26:15.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.732+0000 7f63e4d25700 1 -- 172.21.15.146:0/1507508239 shutdown_connections 2024-09-13T18:26:15.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.732+0000 7f63e4d25700 1 --2- 172.21.15.146:0/1507508239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c8005300 0x7f63e00cab30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:15.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.732+0000 7f63e4d25700 1 --2- 172.21.15.146:0/1507508239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63c8004870 0x7f63e01459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:15.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.732+0000 7f63e4d25700 1 --2- 172.21.15.146:0/1507508239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63c8003a20 0x7f63e01ed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:15.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.732+0000 7f63e4d25700 1 -- 172.21.15.146:0/1507508239 >> 172.21.15.146:0/1507508239 conn(0x7f63e0066920 msgr2=0x7f63e00c57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:15.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.732+0000 7f63e4d25700 1 -- 172.21.15.146:0/1507508239 shutdown_connections 2024-09-13T18:26:15.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:15.732+0000 7f63e4d25700 1 -- 172.21.15.146:0/1507508239 wait complete. 2024-09-13T18:26:15.743 INFO:teuthology.orchestra.run.smithi146.stderr:adjusted tunables profile to default 2024-09-13T18:26:15.755 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T18:26:15.755 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T18:26:15.755 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:15.756 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T18:26:15.770 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:15.771 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T18:26:15.783 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:15.783 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T18:26:15.797 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:15.797 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T18:26:15.809 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:26:15.809 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T18:26:15.825 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:26:15.825 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T18:26:15.844 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:15.838+0000 7fc5ad3b02c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:26:15.857 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:26:15.857 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T18:26:15.917 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-13T18:26:15.917 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T18:26:15.929 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new e2f8470a-1ce4-4248-bed6-4636f5d458e7 0 2024-09-13T18:26:16.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.034+0000 7f92b21c2700 1 Processor -- start 2024-09-13T18:26:16.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.034+0000 7f92b21c2700 1 -- start start 2024-09-13T18:26:16.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.034+0000 7f92b21c2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f92ac14fc80 0x7f92ac1590a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.034+0000 7f92b21c2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f92ac156730 0x7f92ac0c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.034+0000 7f92b21c2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac06d7e0 con 0x7f92ac06e740 2024-09-13T18:26:16.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.034+0000 7f92b21c2700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ac06cf80 con 0x7f92ac14fc80 2024-09-13T18:26:16.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.034+0000 7f92b21c2700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ac06d220 con 0x7f92ac156730 2024-09-13T18:26:16.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.034+0000 7f92abfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f92ac156730 0x7f92ac0c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.034+0000 7f92abfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f92ac156730 0x7f92ac0c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38636/0 (socket says 172.21.15.154:38636) 2024-09-13T18:26:16.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.034+0000 7f92abfff700 1 -- 172.21.15.154:0/2103792735 learned_addr learned my addr 172.21.15.154:0/2103792735 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-13T18:26:16.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.034+0000 7f92ab7fe700 1 --2- 172.21.15.154:0/2103792735 >> v2:172.21.15.146:3300/0 conn(0x7f92ac14fc80 0x7f92ac1590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92abfff700 1 -- 172.21.15.154:0/2103792735 >> v2:172.21.15.146:3300/0 conn(0x7f92ac14fc80 msgr2=0x7f92ac1590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:16.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92abfff700 1 --2- 172.21.15.154:0/2103792735 >> v2:172.21.15.146:3300/0 conn(0x7f92ac14fc80 0x7f92ac1590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92abfff700 1 -- 172.21.15.154:0/2103792735 >> v1:172.21.15.146:6789/0 conn(0x7f92ac06e740 legacy=0x7f92ac0c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92abfff700 1 -- 172.21.15.154:0/2103792735 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac06d500 con 0x7f92ac156730 2024-09-13T18:26:16.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92abfff700 1 --2- 172.21.15.154:0/2103792735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac156730 0x7f92ac0c7de0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f92ac15a1a0 tx=0x7f92a400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af25d6aa7dcc0da2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:16.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92aa7fc700 1 -- 172.21.15.154:0/2103792735 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a400e070 con 0x7f92ac156730 2024-09-13T18:26:16.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92aa7fc700 1 -- 172.21.15.154:0/2103792735 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a400b910 con 0x7f92ac156730 2024-09-13T18:26:16.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92aa7fc700 1 -- 172.21.15.154:0/2103792735 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a400bcc0 con 0x7f92ac156730 2024-09-13T18:26:16.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92b21c2700 1 -- 172.21.15.154:0/2103792735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac156730 msgr2=0x7f92ac0c7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92b21c2700 1 --2- 172.21.15.154:0/2103792735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac156730 0x7f92ac0c7de0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f92ac15a1a0 tx=0x7f92a400b330 comp rx=0 tx=0).stop 2024-09-13T18:26:16.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92b21c2700 1 -- 172.21.15.154:0/2103792735 shutdown_connections 2024-09-13T18:26:16.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92b21c2700 1 --2- 172.21.15.154:0/2103792735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac156730 0x7f92ac0c7de0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92b21c2700 1 --2- 172.21.15.154:0/2103792735 >> v2:172.21.15.146:3300/0 conn(0x7f92ac14fc80 0x7f92ac1590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92b21c2700 1 -- 172.21.15.154:0/2103792735 >> 172.21.15.154:0/2103792735 conn(0x7f92ac066920 msgr2=0x7f92ac06acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:16.044 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92b21c2700 1 -- 172.21.15.154:0/2103792735 shutdown_connections 2024-09-13T18:26:16.044 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92b21c2700 1 -- 172.21.15.154:0/2103792735 wait complete. 2024-09-13T18:26:16.044 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92b21c2700 1 Processor -- start 2024-09-13T18:26:16.044 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.038+0000 7f92b21c2700 1 -- start start 2024-09-13T18:26:16.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92b21c2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92ac06e740 0x7f92ac140570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92b21c2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92ac14fc80 0x7f92ac0c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92b21c2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac156730 0x7f92ac1ed5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92b21c2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0c5c00 con 0x7f92ac06e740 2024-09-13T18:26:16.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92b21c2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac157750 con 0x7f92ac156730 2024-09-13T18:26:16.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92b21c2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0c3e50 con 0x7f92ac14fc80 2024-09-13T18:26:16.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92abfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac156730 0x7f92ac1ed5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92aaffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92ac14fc80 0x7f92ac0c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92abfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac156730 0x7f92ac1ed5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38652/0 (socket says 172.21.15.154:38652) 2024-09-13T18:26:16.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92abfff700 1 -- 172.21.15.154:0/2350268799 learned_addr learned my addr 172.21.15.154:0/2350268799 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-13T18:26:16.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92ab7fe700 1 --2- 172.21.15.154:0/2350268799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92ac06e740 0x7f92ac140570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92abfff700 1 -- 172.21.15.154:0/2350268799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92ac14fc80 msgr2=0x7f92ac0c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92abfff700 1 --2- 172.21.15.154:0/2350268799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92ac14fc80 0x7f92ac0c4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92abfff700 1 -- 172.21.15.154:0/2350268799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92ac06e740 msgr2=0x7f92ac140570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92abfff700 1 --2- 172.21.15.154:0/2350268799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92ac06e740 0x7f92ac140570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92abfff700 1 -- 172.21.15.154:0/2350268799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a4009b30 con 0x7f92ac156730 2024-09-13T18:26:16.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92ab7fe700 1 --2- 172.21.15.154:0/2350268799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92ac06e740 0x7f92ac140570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:16.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92aaffd700 1 --2- 172.21.15.154:0/2350268799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92ac14fc80 0x7f92ac0c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:16.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92abfff700 1 --2- 172.21.15.154:0/2350268799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac156730 0x7f92ac1ed5d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f92ac06d360 tx=0x7f92a4004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:16.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f9293fff700 1 -- 172.21.15.154:0/2350268799 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4004930 con 0x7f92ac156730 2024-09-13T18:26:16.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f9293fff700 1 -- 172.21.15.154:0/2350268799 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a40092e0 con 0x7f92ac156730 2024-09-13T18:26:16.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92b21c2700 1 -- 172.21.15.154:0/2350268799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ac0c40f0 con 0x7f92ac156730 2024-09-13T18:26:16.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f9293fff700 1 -- 172.21.15.154:0/2350268799 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a401b650 con 0x7f92ac156730 2024-09-13T18:26:16.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92b21c2700 1 -- 172.21.15.154:0/2350268799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ac156f50 con 0x7f92ac156730 2024-09-13T18:26:16.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f9293fff700 1 -- 172.21.15.154:0/2350268799 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f92a401bab0 con 0x7f92ac156730 2024-09-13T18:26:16.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f9293fff700 1 -- 172.21.15.154:0/2350268799 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f92a4024ce0 con 0x7f92ac156730 2024-09-13T18:26:16.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.042+0000 7f92b21c2700 1 -- 172.21.15.154:0/2350268799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ac0c86b0 con 0x7f92ac156730 2024-09-13T18:26:16.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.046+0000 7f9293fff700 1 -- 172.21.15.154:0/2350268799 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f92a401f070 con 0x7f92ac156730 2024-09-13T18:26:16.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.122+0000 7f92b21c2700 1 -- 172.21.15.154:0/2350268799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e2f8470a-1ce4-4248-bed6-4636f5d458e7", "id": 0} v 0) v1 -- 0x7f92ac1558b0 con 0x7f92ac156730 2024-09-13T18:26:16.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.126+0000 7f9293fff700 1 -- 172.21.15.154:0/2350268799 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e2f8470a-1ce4-4248-bed6-4636f5d458e7", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f92a4024450 con 0x7f92ac156730 2024-09-13T18:26:16.133 INFO:teuthology.orchestra.run.smithi154.stdout:0 2024-09-13T18:26:16.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.130+0000 7f92b21c2700 1 -- 172.21.15.154:0/2350268799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac156730 msgr2=0x7f92ac1ed5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.130+0000 7f92b21c2700 1 --2- 172.21.15.154:0/2350268799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac156730 0x7f92ac1ed5d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f92ac06d360 tx=0x7f92a4004af0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.130+0000 7f92b21c2700 1 -- 172.21.15.154:0/2350268799 shutdown_connections 2024-09-13T18:26:16.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.130+0000 7f92b21c2700 1 --2- 172.21.15.154:0/2350268799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac156730 0x7f92ac1ed5d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.130+0000 7f92b21c2700 1 --2- 172.21.15.154:0/2350268799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92ac14fc80 0x7f92ac0c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.130+0000 7f92b21c2700 1 --2- 172.21.15.154:0/2350268799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92ac06e740 0x7f92ac140570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.130+0000 7f92b21c2700 1 -- 172.21.15.154:0/2350268799 >> 172.21.15.154:0/2350268799 conn(0x7f92ac066920 msgr2=0x7f92ac06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:16.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.130+0000 7f92b21c2700 1 -- 172.21.15.154:0/2350268799 shutdown_connections 2024-09-13T18:26:16.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.130+0000 7f92b21c2700 1 -- 172.21.15.154:0/2350268799 wait complete. 2024-09-13T18:26:16.149 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 78bcc5f7-e643-4530-9f06-c9669e2db81e 1 2024-09-13T18:26:16.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78ff59e700 1 Processor -- start 2024-09-13T18:26:16.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78ff59e700 1 -- start start 2024-09-13T18:26:16.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78ff59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7900153760 0x7f790006cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78ff59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f79000c7900 0x7f790006bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78ff59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f790006c490 0x7f79000c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78ff59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790015a440 con 0x7f79000c7900 2024-09-13T18:26:16.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78ff59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7900153b40 con 0x7f790006c490 2024-09-13T18:26:16.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78ff59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790006d020 con 0x7f7900153760 2024-09-13T18:26:16.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fe59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7900153760 0x7f790006cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fed9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f790006c490 0x7f79000c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fed9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f790006c490 0x7f79000c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38664/0 (socket says 172.21.15.154:38664) 2024-09-13T18:26:16.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fed9d700 1 -- 172.21.15.154:0/2947619592 learned_addr learned my addr 172.21.15.154:0/2947619592 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-13T18:26:16.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fe59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7900153760 0x7f790006cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:42878/0 (socket says 172.21.15.154:42878) 2024-09-13T18:26:16.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fdd9b700 1 --2- 172.21.15.154:0/2947619592 >> v2:172.21.15.146:3300/0 conn(0x7f79000c7900 0x7f790006bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fe59c700 1 -- 172.21.15.154:0/2947619592 >> v2:172.21.15.154:3300/0 conn(0x7f790006c490 msgr2=0x7f79000c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:16.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fe59c700 1 --2- 172.21.15.154:0/2947619592 >> v2:172.21.15.154:3300/0 conn(0x7f790006c490 0x7f79000c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fe59c700 1 -- 172.21.15.154:0/2947619592 >> v2:172.21.15.146:3300/0 conn(0x7f79000c7900 msgr2=0x7f790006bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:16.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fe59c700 1 --2- 172.21.15.154:0/2947619592 >> v2:172.21.15.146:3300/0 conn(0x7f79000c7900 0x7f790006bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fe59c700 1 -- 172.21.15.154:0/2947619592 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790015cd50 con 0x7f7900153760 2024-09-13T18:26:16.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fed9d700 1 --2- 172.21.15.154:0/2947619592 >> v2:172.21.15.154:3300/0 conn(0x7f790006c490 0x7f79000c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:16.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fdd9b700 1 --2- 172.21.15.154:0/2947619592 >> v2:172.21.15.146:3300/0 conn(0x7f79000c7900 0x7f790006bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:26:16.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.254+0000 7f78fe59c700 1 --2- 172.21.15.154:0/2947619592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7900153760 0x7f790006cae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79000b2f60 tx=0x7f78f0009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c31def4ab81da3aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:16.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78fd59a700 1 -- 172.21.15.154:0/2947619592 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0011830 con 0x7f7900153760 2024-09-13T18:26:16.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78fd59a700 1 -- 172.21.15.154:0/2947619592 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f0011990 con 0x7f7900153760 2024-09-13T18:26:16.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78fd59a700 1 -- 172.21.15.154:0/2947619592 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0011d20 con 0x7f7900153760 2024-09-13T18:26:16.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78ff59e700 1 -- 172.21.15.154:0/2947619592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7900153760 msgr2=0x7f790006cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78ff59e700 1 --2- 172.21.15.154:0/2947619592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7900153760 0x7f790006cae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79000b2f60 tx=0x7f78f0009b30 comp rx=0 tx=0).stop 2024-09-13T18:26:16.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78ff59e700 1 -- 172.21.15.154:0/2947619592 shutdown_connections 2024-09-13T18:26:16.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78ff59e700 1 --2- 172.21.15.154:0/2947619592 >> v2:172.21.15.154:3300/0 conn(0x7f790006c490 0x7f79000c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78ff59e700 1 --2- 172.21.15.154:0/2947619592 >> v2:172.21.15.146:3300/0 conn(0x7f79000c7900 0x7f790006bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78ff59e700 1 --2- 172.21.15.154:0/2947619592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7900153760 0x7f790006cae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78ff59e700 1 -- 172.21.15.154:0/2947619592 >> 172.21.15.154:0/2947619592 conn(0x7f79000ccf00 msgr2=0x7f79000cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:16.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78ff59e700 1 -- 172.21.15.154:0/2947619592 shutdown_connections 2024-09-13T18:26:16.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78ff59e700 1 -- 172.21.15.154:0/2947619592 wait complete. 2024-09-13T18:26:16.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78ff59e700 1 Processor -- start 2024-09-13T18:26:16.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.258+0000 7f78ff59e700 1 -- start start 2024-09-13T18:26:16.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78ff59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f790006c490 0x7f790015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78ff59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79000c7900 0x7f79001e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78ff59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7900153760 0x7f790015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78ff59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7900144e60 con 0x7f790006c490 2024-09-13T18:26:16.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78ff59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790015a660 con 0x7f7900153760 2024-09-13T18:26:16.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78ff59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79000cc3b0 con 0x7f79000c7900 2024-09-13T18:26:16.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78fdd9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79000c7900 0x7f79001e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78fdd9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79000c7900 0x7f79001e3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:42884/0 (socket says 172.21.15.154:42884) 2024-09-13T18:26:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78fdd9b700 1 -- 172.21.15.154:0/2646376596 learned_addr learned my addr 172.21.15.154:0/2646376596 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-13T18:26:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78fed9d700 1 --2- 172.21.15.154:0/2646376596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7900153760 0x7f790015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78fe59c700 1 --2- 172.21.15.154:0/2646376596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f790006c490 0x7f790015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78fdd9b700 1 -- 172.21.15.154:0/2646376596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7900153760 msgr2=0x7f790015adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78fdd9b700 1 --2- 172.21.15.154:0/2646376596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7900153760 0x7f790015adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78fdd9b700 1 -- 172.21.15.154:0/2646376596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f790006c490 msgr2=0x7f790015c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78fdd9b700 1 --2- 172.21.15.154:0/2646376596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f790006c490 0x7f790015c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78fdd9b700 1 -- 172.21.15.154:0/2646376596 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f8009b30 con 0x7f79000c7900 2024-09-13T18:26:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78fdd9b700 1 --2- 172.21.15.154:0/2646376596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79000c7900 0x7f79001e3c10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f78f400c820 tx=0x7f78f400cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78eeffd700 1 -- 172.21.15.154:0/2646376596 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4004220 con 0x7f79000c7900 2024-09-13T18:26:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78eeffd700 1 -- 172.21.15.154:0/2646376596 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f4004380 con 0x7f79000c7900 2024-09-13T18:26:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78eeffd700 1 -- 172.21.15.154:0/2646376596 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f40036c0 con 0x7f79000c7900 2024-09-13T18:26:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78ff59e700 1 -- 172.21.15.154:0/2646376596 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f00094b0 con 0x7f79000c7900 2024-09-13T18:26:16.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78ff59e700 1 -- 172.21.15.154:0/2646376596 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79000cc650 con 0x7f79000c7900 2024-09-13T18:26:16.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78eeffd700 1 -- 172.21.15.154:0/2646376596 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f78f4003820 con 0x7f79000c7900 2024-09-13T18:26:16.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78eeffd700 1 -- 172.21.15.154:0/2646376596 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f78f4019400 con 0x7f79000c7900 2024-09-13T18:26:16.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.262+0000 7f78ff59e700 1 -- 172.21.15.154:0/2646376596 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7900067a60 con 0x7f79000c7900 2024-09-13T18:26:16.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.266+0000 7f78eeffd700 1 -- 172.21.15.154:0/2646376596 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f78f4038e30 con 0x7f79000c7900 2024-09-13T18:26:16.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.338+0000 7f78ff59e700 1 -- 172.21.15.154:0/2646376596 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "78bcc5f7-e643-4530-9f06-c9669e2db81e", "id": 1} v 0) v1 -- 0x7f79000cdd00 con 0x7f79000c7900 2024-09-13T18:26:16.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.346+0000 7f78eeffd700 1 -- 172.21.15.154:0/2646376596 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "78bcc5f7-e643-4530-9f06-c9669e2db81e", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f78f4027030 con 0x7f79000c7900 2024-09-13T18:26:16.351 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-09-13T18:26:16.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.346+0000 7f78ff59e700 1 -- 172.21.15.154:0/2646376596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79000c7900 msgr2=0x7f79001e3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.346+0000 7f78ff59e700 1 --2- 172.21.15.154:0/2646376596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79000c7900 0x7f79001e3c10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f78f400c820 tx=0x7f78f400cb30 comp rx=0 tx=0).stop 2024-09-13T18:26:16.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.346+0000 7f78ff59e700 1 -- 172.21.15.154:0/2646376596 shutdown_connections 2024-09-13T18:26:16.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.346+0000 7f78ff59e700 1 --2- 172.21.15.154:0/2646376596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7900153760 0x7f790015adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.346+0000 7f78ff59e700 1 --2- 172.21.15.154:0/2646376596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79000c7900 0x7f79001e3c10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.346+0000 7f78ff59e700 1 --2- 172.21.15.154:0/2646376596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f790006c490 0x7f790015c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.346+0000 7f78ff59e700 1 -- 172.21.15.154:0/2646376596 >> 172.21.15.154:0/2646376596 conn(0x7f79000ccf00 msgr2=0x7f7900151fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:16.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.346+0000 7f78ff59e700 1 -- 172.21.15.154:0/2646376596 shutdown_connections 2024-09-13T18:26:16.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.346+0000 7f78ff59e700 1 -- 172.21.15.154:0/2646376596 wait complete. 2024-09-13T18:26:16.369 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new ad13f832-2def-402e-b36b-e68088f63502 2 2024-09-13T18:26:16.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44e1310700 1 Processor -- start 2024-09-13T18:26:16.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44e1310700 1 -- start start 2024-09-13T18:26:16.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44e1310700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f44dc0b1170 0x7f44dc0c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44e1310700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44dc06a740 con 0x7f44dc1517e0 2024-09-13T18:26:16.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44e1310700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44dc0b05d0 con 0x7f44dc0c79e0 2024-09-13T18:26:16.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44e1310700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44dc0b02a0 con 0x7f44dc0b1170 2024-09-13T18:26:16.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44e0b0f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f44dc0b1170 0x7f44dc0c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44db7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f44dc0c79e0 0x7f44dc0b0a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43056/0 (socket says 172.21.15.154:43056) 2024-09-13T18:26:16.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44db7fe700 1 -- 172.21.15.154:0/2771544451 learned_addr learned my addr 172.21.15.154:0/2771544451 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-09-13T18:26:16.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44e0b0f700 1 -- 172.21.15.154:0/2771544451 >> v1:172.21.15.154:6789/0 conn(0x7f44dc0c79e0 legacy=0x7f44dc0b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44e0b0f700 1 -- 172.21.15.154:0/2771544451 >> v1:172.21.15.146:6789/0 conn(0x7f44dc1517e0 legacy=0x7f44dc0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44e0b0f700 1 -- 172.21.15.154:0/2771544451 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc157af0 con 0x7f44dc0b1170 2024-09-13T18:26:16.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44e0b0f700 1 --2- 172.21.15.154:0/2771544451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44dc0b1170 0x7f44dc0c92f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009850 tx=0x7f44cc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54c89829a7da960f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:16.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44daffd700 1 -- 172.21.15.154:0/2771544451 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc011830 con 0x7f44dc0b1170 2024-09-13T18:26:16.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.478+0000 7f44daffd700 1 -- 172.21.15.154:0/2771544451 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc011990 con 0x7f44dc0b1170 2024-09-13T18:26:16.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.482+0000 7f44daffd700 1 -- 172.21.15.154:0/2771544451 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc011d20 con 0x7f44dc0b1170 2024-09-13T18:26:16.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.482+0000 7f44e1310700 1 -- 172.21.15.154:0/2771544451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44dc0b1170 msgr2=0x7f44dc0c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.482+0000 7f44e1310700 1 --2- 172.21.15.154:0/2771544451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44dc0b1170 0x7f44dc0c92f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009850 tx=0x7f44cc009b60 comp rx=0 tx=0).stop 2024-09-13T18:26:16.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.482+0000 7f44e1310700 1 -- 172.21.15.154:0/2771544451 shutdown_connections 2024-09-13T18:26:16.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.482+0000 7f44e1310700 1 --2- 172.21.15.154:0/2771544451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44dc0b1170 0x7f44dc0c92f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.482+0000 7f44e1310700 1 -- 172.21.15.154:0/2771544451 >> 172.21.15.154:0/2771544451 conn(0x7f44dc06c2d0 msgr2=0x7f44dc06c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:16.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.482+0000 7f44e1310700 1 -- 172.21.15.154:0/2771544451 shutdown_connections 2024-09-13T18:26:16.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.482+0000 7f44e1310700 1 -- 172.21.15.154:0/2771544451 wait complete. 2024-09-13T18:26:16.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.482+0000 7f44e1310700 1 Processor -- start 2024-09-13T18:26:16.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.482+0000 7f44e1310700 1 -- start start 2024-09-13T18:26:16.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44e1310700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44dc0b1170 0x7f44dc1df160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44e1310700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44dc0c79e0 0x7f44dc1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44e1310700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44dc1517e0 0x7f44dc1ddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44e1310700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1543c0 con 0x7f44dc0b1170 2024-09-13T18:26:16.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44e1310700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1ed800 con 0x7f44dc1517e0 2024-09-13T18:26:16.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44e1310700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1deba0 con 0x7f44dc0c79e0 2024-09-13T18:26:16.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44db7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44dc0c79e0 0x7f44dc1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44db7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44dc0c79e0 0x7f44dc1563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:42890/0 (socket says 172.21.15.154:42890) 2024-09-13T18:26:16.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44db7fe700 1 -- 172.21.15.154:0/1163145117 learned_addr learned my addr 172.21.15.154:0/1163145117 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-13T18:26:16.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44e0b0f700 1 --2- 172.21.15.154:0/1163145117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44dc1517e0 0x7f44dc1ddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44dbfff700 1 --2- 172.21.15.154:0/1163145117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44dc0b1170 0x7f44dc1df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44db7fe700 1 -- 172.21.15.154:0/1163145117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44dc1517e0 msgr2=0x7f44dc1ddbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44db7fe700 1 --2- 172.21.15.154:0/1163145117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44dc1517e0 0x7f44dc1ddbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44db7fe700 1 -- 172.21.15.154:0/1163145117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44dc0b1170 msgr2=0x7f44dc1df160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44db7fe700 1 --2- 172.21.15.154:0/1163145117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44dc0b1170 0x7f44dc1df160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44db7fe700 1 -- 172.21.15.154:0/1163145117 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc0094b0 con 0x7f44dc0c79e0 2024-09-13T18:26:16.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44e0b0f700 1 --2- 172.21.15.154:0/1163145117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44dc1517e0 0x7f44dc1ddbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:16.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44db7fe700 1 --2- 172.21.15.154:0/1163145117 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44dc0c79e0 0x7f44dc1563d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f44c400c730 tx=0x7f44c400ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:16.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44d8ff9700 1 -- 172.21.15.154:0/1163145117 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c4005d90 con 0x7f44dc0c79e0 2024-09-13T18:26:16.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44d8ff9700 1 -- 172.21.15.154:0/1163145117 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c40070d0 con 0x7f44dc0c79e0 2024-09-13T18:26:16.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44e1310700 1 -- 172.21.15.154:0/1163145117 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44dc1dee80 con 0x7f44dc0c79e0 2024-09-13T18:26:16.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44d8ff9700 1 -- 172.21.15.154:0/1163145117 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c4007440 con 0x7f44dc0c79e0 2024-09-13T18:26:16.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44e1310700 1 -- 172.21.15.154:0/1163145117 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44dc1df6a0 con 0x7f44dc0c79e0 2024-09-13T18:26:16.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44d8ff9700 1 -- 172.21.15.154:0/1163145117 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f44c401c440 con 0x7f44dc0c79e0 2024-09-13T18:26:16.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.486+0000 7f44d8ff9700 1 -- 172.21.15.154:0/1163145117 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f44c4012820 con 0x7f44dc0c79e0 2024-09-13T18:26:16.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.490+0000 7f44e1310700 1 -- 172.21.15.154:0/1163145117 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44dc1dd8f0 con 0x7f44dc0c79e0 2024-09-13T18:26:16.497 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.494+0000 7f44d8ff9700 1 -- 172.21.15.154:0/1163145117 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f44dc1dd8f0 con 0x7f44dc0c79e0 2024-09-13T18:26:16.560 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:16.554+0000 7fc5ad3b02c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:26:16.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.570+0000 7f44e1310700 1 -- 172.21.15.154:0/1163145117 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ad13f832-2def-402e-b36b-e68088f63502", "id": 2} v 0) v1 -- 0x7f44dc0c92f0 con 0x7f44dc0c79e0 2024-09-13T18:26:16.582 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.578+0000 7f44d8ff9700 1 -- 172.21.15.154:0/1163145117 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad13f832-2def-402e-b36b-e68088f63502", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f44c4021060 con 0x7f44dc0c79e0 2024-09-13T18:26:16.582 INFO:teuthology.orchestra.run.smithi154.stdout:2 2024-09-13T18:26:16.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.578+0000 7f44e1310700 1 -- 172.21.15.154:0/1163145117 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44dc0c79e0 msgr2=0x7f44dc1563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.578+0000 7f44e1310700 1 --2- 172.21.15.154:0/1163145117 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44dc0c79e0 0x7f44dc1563d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f44c400c730 tx=0x7f44c400ca40 comp rx=0 tx=0).stop 2024-09-13T18:26:16.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.578+0000 7f44e1310700 1 -- 172.21.15.154:0/1163145117 shutdown_connections 2024-09-13T18:26:16.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.578+0000 7f44e1310700 1 --2- 172.21.15.154:0/1163145117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44dc1517e0 0x7f44dc1ddbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.578+0000 7f44e1310700 1 --2- 172.21.15.154:0/1163145117 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44dc0c79e0 0x7f44dc1563d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.578+0000 7f44e1310700 1 --2- 172.21.15.154:0/1163145117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44dc0b1170 0x7f44dc1df160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.578+0000 7f44e1310700 1 -- 172.21.15.154:0/1163145117 >> 172.21.15.154:0/1163145117 conn(0x7f44dc06c2d0 msgr2=0x7f44dc06c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:16.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.578+0000 7f44e1310700 1 -- 172.21.15.154:0/1163145117 shutdown_connections 2024-09-13T18:26:16.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.578+0000 7f44e1310700 1 -- 172.21.15.154:0/1163145117 wait complete. 2024-09-13T18:26:16.602 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 905c1019-7640-42d4-8c40-1363a8039b04 3 2024-09-13T18:26:16.619 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:16.614+0000 7fc5ad3b02c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:26:16.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.714+0000 7fb7dcf8e700 1 Processor -- start 2024-09-13T18:26:16.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.714+0000 7fb7dcf8e700 1 -- start start 2024-09-13T18:26:16.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7dcf8e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7d80c83e0 0x7fb7d80c7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7dcf8e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb7d80c6b40 0x7fb7d80c2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7dcf8e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d80716e0 con 0x7fb7d80c87c0 2024-09-13T18:26:16.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7dcf8e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d80c1880 con 0x7fb7d80c6b40 2024-09-13T18:26:16.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7dcf8e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d8072260 con 0x7fb7d80c83e0 2024-09-13T18:26:16.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d5d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb7d80c87c0 0x7fb7d80c26f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43072/0 (socket says 172.21.15.154:43072) 2024-09-13T18:26:16.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d5d9b700 1 -- 172.21.15.154:0/2979344100 learned_addr learned my addr 172.21.15.154:0/2979344100 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-09-13T18:26:16.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d659c700 1 --2- 172.21.15.154:0/2979344100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7d80c83e0 0x7fb7d80c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d6d9d700 1 --2- 172.21.15.154:0/2979344100 >> v2:172.21.15.146:3300/0 conn(0x7fb7d80c6b40 0x7fb7d80c2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d559a700 1 -- 172.21.15.154:0/2979344100 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212614026 0 0) 0x7fb7d80716e0 con 0x7fb7d80c87c0 2024-09-13T18:26:16.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d559a700 1 -- 172.21.15.154:0/2979344100 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7c00035d0 con 0x7fb7d80c87c0 2024-09-13T18:26:16.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d659c700 1 -- 172.21.15.154:0/2979344100 >> v2:172.21.15.146:3300/0 conn(0x7fb7d80c6b40 msgr2=0x7fb7d80c2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:16.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d659c700 1 --2- 172.21.15.154:0/2979344100 >> v2:172.21.15.146:3300/0 conn(0x7fb7d80c6b40 0x7fb7d80c2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d659c700 1 -- 172.21.15.154:0/2979344100 >> v1:172.21.15.154:6789/0 conn(0x7fb7d80c87c0 legacy=0x7fb7d80c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d659c700 1 -- 172.21.15.154:0/2979344100 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d80c7a90 con 0x7fb7d80c83e0 2024-09-13T18:26:16.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d6d9d700 1 --2- 172.21.15.154:0/2979344100 >> v2:172.21.15.146:3300/0 conn(0x7fb7d80c6b40 0x7fb7d80c2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:16.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d659c700 1 --2- 172.21.15.154:0/2979344100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7d80c83e0 0x7fb7d80c7550 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009850 tx=0x7fb7c8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=490d0ee5cc67105e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:16.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d559a700 1 -- 172.21.15.154:0/2979344100 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8011830 con 0x7fb7d80c83e0 2024-09-13T18:26:16.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d559a700 1 -- 172.21.15.154:0/2979344100 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c8011990 con 0x7fb7d80c83e0 2024-09-13T18:26:16.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7d559a700 1 -- 172.21.15.154:0/2979344100 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8011d40 con 0x7fb7d80c83e0 2024-09-13T18:26:16.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/2979344100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7d80c83e0 msgr2=0x7fb7d80c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7dcf8e700 1 --2- 172.21.15.154:0/2979344100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7d80c83e0 0x7fb7d80c7550 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009850 tx=0x7fb7c8009b60 comp rx=0 tx=0).stop 2024-09-13T18:26:16.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/2979344100 shutdown_connections 2024-09-13T18:26:16.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7dcf8e700 1 --2- 172.21.15.154:0/2979344100 >> v2:172.21.15.146:3300/0 conn(0x7fb7d80c6b40 0x7fb7d80c2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7dcf8e700 1 --2- 172.21.15.154:0/2979344100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7d80c83e0 0x7fb7d80c7550 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/2979344100 >> 172.21.15.154:0/2979344100 conn(0x7fb7d80cc140 msgr2=0x7fb7d80cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:16.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/2979344100 shutdown_connections 2024-09-13T18:26:16.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.718+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/2979344100 wait complete. 2024-09-13T18:26:16.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7dcf8e700 1 Processor -- start 2024-09-13T18:26:16.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7dcf8e700 1 -- start start 2024-09-13T18:26:16.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7dcf8e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7d80c6b40 0x7fb7d8151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7dcf8e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7d80c83e0 0x7fb7d81528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7dcf8e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7d80c87c0 0x7fb7d8152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7dcf8e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d80b15b0 con 0x7fb7d80c6b40 2024-09-13T18:26:16.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7dcf8e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81eb880 con 0x7fb7d80c87c0 2024-09-13T18:26:16.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7dcf8e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81536a0 con 0x7fb7d80c83e0 2024-09-13T18:26:16.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7d6d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7d80c87c0 0x7fb7d8152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7d6d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7d80c87c0 0x7fb7d8152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38700/0 (socket says 172.21.15.154:38700) 2024-09-13T18:26:16.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7d6d9d700 1 -- 172.21.15.154:0/593331402 learned_addr learned my addr 172.21.15.154:0/593331402 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-13T18:26:16.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7d5d9b700 1 --2- 172.21.15.154:0/593331402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7d80c83e0 0x7fb7d81528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7d659c700 1 --2- 172.21.15.154:0/593331402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7d80c6b40 0x7fb7d8151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7d6d9d700 1 -- 172.21.15.154:0/593331402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7d80c83e0 msgr2=0x7fb7d81528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7d6d9d700 1 --2- 172.21.15.154:0/593331402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7d80c83e0 0x7fb7d81528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7d6d9d700 1 -- 172.21.15.154:0/593331402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7d80c6b40 msgr2=0x7fb7d8151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7d6d9d700 1 --2- 172.21.15.154:0/593331402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7d80c6b40 0x7fb7d8151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7d6d9d700 1 -- 172.21.15.154:0/593331402 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c80094b0 con 0x7fb7d80c87c0 2024-09-13T18:26:16.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7d659c700 1 --2- 172.21.15.154:0/593331402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7d80c6b40 0x7fb7d8151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:16.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7d5d9b700 1 --2- 172.21.15.154:0/593331402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7d80c83e0 0x7fb7d81528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:16.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7d6d9d700 1 --2- 172.21.15.154:0/593331402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7d80c87c0 0x7fb7d8152df0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0007e40 tx=0x7fb7d000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:16.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7c6ffd700 1 -- 172.21.15.154:0/593331402 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000ddd0 con 0x7fb7d80c87c0 2024-09-13T18:26:16.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7c6ffd700 1 -- 172.21.15.154:0/593331402 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d000f040 con 0x7fb7d80c87c0 2024-09-13T18:26:16.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7c6ffd700 1 -- 172.21.15.154:0/593331402 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0003c80 con 0x7fb7d80c87c0 2024-09-13T18:26:16.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/593331402 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d8153980 con 0x7fb7d80c87c0 2024-09-13T18:26:16.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/593331402 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d81ebb60 con 0x7fb7d80c87c0 2024-09-13T18:26:16.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7c6ffd700 1 -- 172.21.15.154:0/593331402 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb7d0004330 con 0x7fb7d80c87c0 2024-09-13T18:26:16.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.722+0000 7fb7c6ffd700 1 -- 172.21.15.154:0/593331402 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fb7d0003de0 con 0x7fb7d80c87c0 2024-09-13T18:26:16.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.726+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/593331402 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d80af430 con 0x7fb7d80c87c0 2024-09-13T18:26:16.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.730+0000 7fb7c6ffd700 1 -- 172.21.15.154:0/593331402 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fb7d80af430 con 0x7fb7d80c87c0 2024-09-13T18:26:16.772 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:16.764+0000 7fdce42d52c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:26:16.789 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:16.782+0000 7fc5ad3b02c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:26:16.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.802+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/593331402 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "905c1019-7640-42d4-8c40-1363a8039b04", "id": 3} v 0) v1 -- 0x7fb7d80c1390 con 0x7fb7d80c87c0 2024-09-13T18:26:16.813 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.810+0000 7fb7c6ffd700 1 -- 172.21.15.154:0/593331402 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "905c1019-7640-42d4-8c40-1363a8039b04", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb7d001a070 con 0x7fb7d80c87c0 2024-09-13T18:26:16.814 INFO:teuthology.orchestra.run.smithi154.stdout:3 2024-09-13T18:26:16.816 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.810+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/593331402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7d80c87c0 msgr2=0x7fb7d8152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.816 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.810+0000 7fb7dcf8e700 1 --2- 172.21.15.154:0/593331402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7d80c87c0 0x7fb7d8152df0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0007e40 tx=0x7fb7d000d3b0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.816 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.810+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/593331402 shutdown_connections 2024-09-13T18:26:16.816 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.810+0000 7fb7dcf8e700 1 --2- 172.21.15.154:0/593331402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7d80c87c0 0x7fb7d8152df0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.816 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.810+0000 7fb7dcf8e700 1 --2- 172.21.15.154:0/593331402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7d80c83e0 0x7fb7d81528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.816 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.810+0000 7fb7dcf8e700 1 --2- 172.21.15.154:0/593331402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7d80c6b40 0x7fb7d8151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.816 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.810+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/593331402 >> 172.21.15.154:0/593331402 conn(0x7fb7d80cc140 msgr2=0x7fb7d814f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:16.816 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.810+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/593331402 shutdown_connections 2024-09-13T18:26:16.816 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.810+0000 7fb7dcf8e700 1 -- 172.21.15.154:0/593331402 wait complete. 2024-09-13T18:26:16.826 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new e1178506-9350-4efa-8e99-27d251a1d966 4 2024-09-13T18:26:16.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf46719700 1 Processor -- start 2024-09-13T18:26:16.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf46719700 1 -- start start 2024-09-13T18:26:16.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf46719700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf40156ca0 0x7fcf4006b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf46719700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcf400706d0 0x7fcf40070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf46719700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf400ae090 con 0x7fcf400adc70 2024-09-13T18:26:16.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf46719700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf40157080 con 0x7fcf400706d0 2024-09-13T18:26:16.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf46719700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400713f0 con 0x7fcf40156ca0 2024-09-13T18:26:16.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf45717700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf40156ca0 0x7fcf4006b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf45717700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf40156ca0 0x7fcf4006b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:42924/0 (socket says 172.21.15.154:42924) 2024-09-13T18:26:16.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf45717700 1 -- 172.21.15.154:0/2646860200 learned_addr learned my addr 172.21.15.154:0/2646860200 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-13T18:26:16.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf45f18700 1 --2- 172.21.15.154:0/2646860200 >> v2:172.21.15.146:3300/0 conn(0x7fcf400706d0 0x7fcf40070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf45717700 1 -- 172.21.15.154:0/2646860200 >> v2:172.21.15.146:3300/0 conn(0x7fcf400706d0 msgr2=0x7fcf40070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:16.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf45717700 1 --2- 172.21.15.154:0/2646860200 >> v2:172.21.15.146:3300/0 conn(0x7fcf400706d0 0x7fcf40070b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf45717700 1 -- 172.21.15.154:0/2646860200 >> v1:172.21.15.146:6789/0 conn(0x7fcf400adc70 legacy=0x7fcf400c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.930+0000 7fcf45717700 1 -- 172.21.15.154:0/2646860200 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4006a850 con 0x7fcf40156ca0 2024-09-13T18:26:16.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf45717700 1 --2- 172.21.15.154:0/2646860200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf40156ca0 0x7fcf4006b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009910 tx=0x7fcf3c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ac3b399c1bbac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:16.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf37fff700 1 -- 172.21.15.154:0/2646860200 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00ba60 con 0x7fcf40156ca0 2024-09-13T18:26:16.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf37fff700 1 -- 172.21.15.154:0/2646860200 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c00bbc0 con 0x7fcf40156ca0 2024-09-13T18:26:16.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf37fff700 1 -- 172.21.15.154:0/2646860200 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c009e60 con 0x7fcf40156ca0 2024-09-13T18:26:16.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf46719700 1 -- 172.21.15.154:0/2646860200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf40156ca0 msgr2=0x7fcf4006b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf46719700 1 --2- 172.21.15.154:0/2646860200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf40156ca0 0x7fcf4006b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009910 tx=0x7fcf3c009c20 comp rx=0 tx=0).stop 2024-09-13T18:26:16.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf46719700 1 -- 172.21.15.154:0/2646860200 shutdown_connections 2024-09-13T18:26:16.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf46719700 1 --2- 172.21.15.154:0/2646860200 >> v2:172.21.15.146:3300/0 conn(0x7fcf400706d0 0x7fcf40070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf46719700 1 --2- 172.21.15.154:0/2646860200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf40156ca0 0x7fcf4006b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf46719700 1 -- 172.21.15.154:0/2646860200 >> 172.21.15.154:0/2646860200 conn(0x7fcf4006c2d0 msgr2=0x7fcf4006c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:16.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf46719700 1 -- 172.21.15.154:0/2646860200 shutdown_connections 2024-09-13T18:26:16.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf46719700 1 -- 172.21.15.154:0/2646860200 wait complete. 2024-09-13T18:26:16.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf46719700 1 Processor -- start 2024-09-13T18:26:16.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.934+0000 7fcf46719700 1 -- start start 2024-09-13T18:26:16.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf46719700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf400706d0 0x7fcf400c2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf46719700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400adc70 0x7fcf401efa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf46719700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf40156ca0 0x7fcf400c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:16.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf46719700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf401ef060 con 0x7fcf400adc70 2024-09-13T18:26:16.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf46719700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf401ef5e0 con 0x7fcf400706d0 2024-09-13T18:26:16.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf46719700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4015d9c0 con 0x7fcf40156ca0 2024-09-13T18:26:16.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf45717700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf400706d0 0x7fcf400c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf45f18700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf40156ca0 0x7fcf400c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf45717700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf400706d0 0x7fcf400c2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38702/0 (socket says 172.21.15.154:38702) 2024-09-13T18:26:16.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf45717700 1 -- 172.21.15.154:0/1915835141 learned_addr learned my addr 172.21.15.154:0/1915835141 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-13T18:26:16.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf44f16700 1 --2- 172.21.15.154:0/1915835141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400adc70 0x7fcf401efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:16.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf45f18700 1 -- 172.21.15.154:0/1915835141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf400706d0 msgr2=0x7fcf400c2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf45f18700 1 --2- 172.21.15.154:0/1915835141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf400706d0 0x7fcf400c2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf45f18700 1 -- 172.21.15.154:0/1915835141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400adc70 msgr2=0x7fcf401efa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:16.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf45f18700 1 --2- 172.21.15.154:0/1915835141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400adc70 0x7fcf401efa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:16.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf45f18700 1 -- 172.21.15.154:0/1915835141 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c009570 con 0x7fcf40156ca0 2024-09-13T18:26:16.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf45717700 1 --2- 172.21.15.154:0/1915835141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf400706d0 0x7fcf400c2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:16.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf45f18700 1 --2- 172.21.15.154:0/1915835141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf40156ca0 0x7fcf400c2fd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcf38007cf0 tx=0x7fcf3800e0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:16.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf35ffb700 1 -- 172.21.15.154:0/1915835141 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38018070 con 0x7fcf40156ca0 2024-09-13T18:26:16.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf35ffb700 1 -- 172.21.15.154:0/1915835141 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf38003a40 con 0x7fcf40156ca0 2024-09-13T18:26:16.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf46719700 1 -- 172.21.15.154:0/1915835141 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf400c29a0 con 0x7fcf40156ca0 2024-09-13T18:26:16.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf35ffb700 1 -- 172.21.15.154:0/1915835141 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38003dd0 con 0x7fcf40156ca0 2024-09-13T18:26:16.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf46719700 1 -- 172.21.15.154:0/1915835141 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf401f0010 con 0x7fcf40156ca0 2024-09-13T18:26:16.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf35ffb700 1 -- 172.21.15.154:0/1915835141 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fcf3800ec70 con 0x7fcf40156ca0 2024-09-13T18:26:16.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf35ffb700 1 -- 172.21.15.154:0/1915835141 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fcf38014930 con 0x7fcf40156ca0 2024-09-13T18:26:16.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.938+0000 7fcf46719700 1 -- 172.21.15.154:0/1915835141 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf401ef790 con 0x7fcf40156ca0 2024-09-13T18:26:16.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:16.942+0000 7fcf35ffb700 1 -- 172.21.15.154:0/1915835141 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fcf401ef790 con 0x7fcf40156ca0 2024-09-13T18:26:17.008 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:17.002+0000 7fc5ad3b02c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:26:17.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.018+0000 7fcf46719700 1 -- 172.21.15.154:0/1915835141 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e1178506-9350-4efa-8e99-27d251a1d966", "id": 4} v 0) v1 -- 0x7fcf4006bbc0 con 0x7fcf40156ca0 2024-09-13T18:26:17.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.026+0000 7fcf35ffb700 1 -- 172.21.15.154:0/1915835141 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e1178506-9350-4efa-8e99-27d251a1d966", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcf38014bf0 con 0x7fcf40156ca0 2024-09-13T18:26:17.031 INFO:teuthology.orchestra.run.smithi154.stdout:4 2024-09-13T18:26:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.026+0000 7fcf46719700 1 -- 172.21.15.154:0/1915835141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf40156ca0 msgr2=0x7fcf400c2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.026+0000 7fcf46719700 1 --2- 172.21.15.154:0/1915835141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf40156ca0 0x7fcf400c2fd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcf38007cf0 tx=0x7fcf3800e0e0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.026+0000 7fcf46719700 1 -- 172.21.15.154:0/1915835141 shutdown_connections 2024-09-13T18:26:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.026+0000 7fcf46719700 1 --2- 172.21.15.154:0/1915835141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf40156ca0 0x7fcf400c2fd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.026+0000 7fcf46719700 1 --2- 172.21.15.154:0/1915835141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400adc70 0x7fcf401efa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.026+0000 7fcf46719700 1 --2- 172.21.15.154:0/1915835141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf400706d0 0x7fcf400c2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.026+0000 7fcf46719700 1 -- 172.21.15.154:0/1915835141 >> 172.21.15.154:0/1915835141 conn(0x7fcf4006c2d0 msgr2=0x7fcf40073c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:17.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.026+0000 7fcf46719700 1 -- 172.21.15.154:0/1915835141 shutdown_connections 2024-09-13T18:26:17.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.026+0000 7fcf46719700 1 -- 172.21.15.154:0/1915835141 wait complete. 2024-09-13T18:26:17.044 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new cef9a2a4-ce47-4ebf-a64d-fb332e0637a7 5 2024-09-13T18:26:17.061 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:17.058+0000 7fc5ad3b02c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:26:17.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3d10e700 1 Processor -- start 2024-09-13T18:26:17.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3d10e700 1 -- start start 2024-09-13T18:26:17.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3d10e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9e38151840 0x7f9e3806d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3d10e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9e3806f5c0 0x7f9e3806ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3d10e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e380c85f0 0x7f9e380704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3d10e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e380c8160 con 0x7f9e38151840 2024-09-13T18:26:17.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3d10e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e380c7850 con 0x7f9e3806f5c0 2024-09-13T18:26:17.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3d10e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3815b5b0 con 0x7f9e380c85f0 2024-09-13T18:26:17.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3c90d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e380c85f0 0x7f9e380704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3c90d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e380c85f0 0x7f9e380704c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:42936/0 (socket says 172.21.15.154:42936) 2024-09-13T18:26:17.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3c90d700 1 -- 172.21.15.154:0/2200777095 learned_addr learned my addr 172.21.15.154:0/2200777095 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-13T18:26:17.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e377fe700 1 --2- 172.21.15.154:0/2200777095 >> v2:172.21.15.154:3300/0 conn(0x7f9e3806f5c0 0x7f9e3806ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e37fff700 1 --2- 172.21.15.154:0/2200777095 >> v2:172.21.15.146:3300/0 conn(0x7f9e38151840 0x7f9e3806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3c90d700 1 -- 172.21.15.154:0/2200777095 >> v2:172.21.15.154:3300/0 conn(0x7f9e3806f5c0 msgr2=0x7f9e3806ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:17.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3c90d700 1 --2- 172.21.15.154:0/2200777095 >> v2:172.21.15.154:3300/0 conn(0x7f9e3806f5c0 0x7f9e3806ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3c90d700 1 -- 172.21.15.154:0/2200777095 >> v2:172.21.15.146:3300/0 conn(0x7f9e38151840 msgr2=0x7f9e3806d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3c90d700 1 --2- 172.21.15.154:0/2200777095 >> v2:172.21.15.146:3300/0 conn(0x7f9e38151840 0x7f9e3806d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3c90d700 1 -- 172.21.15.154:0/2200777095 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e380c7b10 con 0x7f9e380c85f0 2024-09-13T18:26:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e37fff700 1 --2- 172.21.15.154:0/2200777095 >> v2:172.21.15.146:3300/0 conn(0x7f9e38151840 0x7f9e3806d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:26:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e3c90d700 1 --2- 172.21.15.154:0/2200777095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e380c85f0 0x7f9e380704c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9e38070c50 tx=0x7f9e3000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96cb6c7daf57922 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e36ffd700 1 -- 172.21.15.154:0/2200777095 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000e070 con 0x7f9e380c85f0 2024-09-13T18:26:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e36ffd700 1 -- 172.21.15.154:0/2200777095 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3000b910 con 0x7f9e380c85f0 2024-09-13T18:26:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.150+0000 7f9e36ffd700 1 -- 172.21.15.154:0/2200777095 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000bcc0 con 0x7f9e380c85f0 2024-09-13T18:26:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 -- 172.21.15.154:0/2200777095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e380c85f0 msgr2=0x7f9e380704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 --2- 172.21.15.154:0/2200777095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e380c85f0 0x7f9e380704c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9e38070c50 tx=0x7f9e3000b330 comp rx=0 tx=0).stop 2024-09-13T18:26:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 -- 172.21.15.154:0/2200777095 shutdown_connections 2024-09-13T18:26:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 --2- 172.21.15.154:0/2200777095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e380c85f0 0x7f9e380704c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 --2- 172.21.15.154:0/2200777095 >> v2:172.21.15.154:3300/0 conn(0x7f9e3806f5c0 0x7f9e3806ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 --2- 172.21.15.154:0/2200777095 >> v2:172.21.15.146:3300/0 conn(0x7f9e38151840 0x7f9e3806d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 -- 172.21.15.154:0/2200777095 >> 172.21.15.154:0/2200777095 conn(0x7f9e380c8a60 msgr2=0x7f9e380c8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 -- 172.21.15.154:0/2200777095 shutdown_connections 2024-09-13T18:26:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 -- 172.21.15.154:0/2200777095 wait complete. 2024-09-13T18:26:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 Processor -- start 2024-09-13T18:26:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 -- start start 2024-09-13T18:26:17.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e3806f5c0 0x7f9e38146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e380c85f0 0x7f9e3814c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e38151840 0x7f9e3814ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e38153c80 con 0x7f9e3806f5c0 2024-09-13T18:26:17.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e38151c40 con 0x7f9e380c85f0 2024-09-13T18:26:17.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3d10e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e38146cd0 con 0x7f9e38151840 2024-09-13T18:26:17.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e377fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e380c85f0 0x7f9e3814c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e3c90d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e38151840 0x7f9e3814ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e377fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e380c85f0 0x7f9e3814c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38724/0 (socket says 172.21.15.154:38724) 2024-09-13T18:26:17.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.154+0000 7f9e377fe700 1 -- 172.21.15.154:0/887538500 learned_addr learned my addr 172.21.15.154:0/887538500 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-13T18:26:17.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e37fff700 1 --2- 172.21.15.154:0/887538500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e3806f5c0 0x7f9e38146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e377fe700 1 -- 172.21.15.154:0/887538500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e38151840 msgr2=0x7f9e3814ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e377fe700 1 --2- 172.21.15.154:0/887538500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e38151840 0x7f9e3814ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e377fe700 1 -- 172.21.15.154:0/887538500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e3806f5c0 msgr2=0x7f9e38146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e377fe700 1 --2- 172.21.15.154:0/887538500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e3806f5c0 0x7f9e38146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e377fe700 1 -- 172.21.15.154:0/887538500 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30009b30 con 0x7f9e380c85f0 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e377fe700 1 --2- 172.21.15.154:0/887538500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e380c85f0 0x7f9e3814c970 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00e960 tx=0x7f9e2c00ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e34ff9700 1 -- 172.21.15.154:0/887538500 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c0096c0 con 0x7f9e380c85f0 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e34ff9700 1 -- 172.21.15.154:0/887538500 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c00eeb0 con 0x7f9e380c85f0 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e34ff9700 1 -- 172.21.15.154:0/887538500 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c00ca50 con 0x7f9e380c85f0 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e3d10e700 1 -- 172.21.15.154:0/887538500 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3815af00 con 0x7f9e380c85f0 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e3d10e700 1 -- 172.21.15.154:0/887538500 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e38152170 con 0x7f9e380c85f0 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e34ff9700 1 -- 172.21.15.154:0/887538500 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9e2c005a40 con 0x7f9e380c85f0 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e34ff9700 1 -- 172.21.15.154:0/887538500 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f9e2c0067f0 con 0x7f9e380c85f0 2024-09-13T18:26:17.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.158+0000 7f9e3d10e700 1 -- 172.21.15.154:0/887538500 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e380c6960 con 0x7f9e380c85f0 2024-09-13T18:26:17.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.162+0000 7f9e34ff9700 1 -- 172.21.15.154:0/887538500 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f9e2c021060 con 0x7f9e380c85f0 2024-09-13T18:26:17.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.238+0000 7f9e3d10e700 1 -- 172.21.15.154:0/887538500 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cef9a2a4-ce47-4ebf-a64d-fb332e0637a7", "id": 5} v 0) v1 -- 0x7f9e380c7060 con 0x7f9e380c85f0 2024-09-13T18:26:17.248 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:17.242+0000 7fc5ad3b02c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:26:17.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.246+0000 7f9e34ff9700 1 -- 172.21.15.154:0/887538500 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cef9a2a4-ce47-4ebf-a64d-fb332e0637a7", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9e2c017070 con 0x7f9e380c85f0 2024-09-13T18:26:17.251 INFO:teuthology.orchestra.run.smithi154.stdout:5 2024-09-13T18:26:17.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.246+0000 7f9e3d10e700 1 -- 172.21.15.154:0/887538500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e380c85f0 msgr2=0x7f9e3814c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.246+0000 7f9e3d10e700 1 --2- 172.21.15.154:0/887538500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e380c85f0 0x7f9e3814c970 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00e960 tx=0x7f9e2c00ec70 comp rx=0 tx=0).stop 2024-09-13T18:26:17.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.246+0000 7f9e3d10e700 1 -- 172.21.15.154:0/887538500 shutdown_connections 2024-09-13T18:26:17.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.246+0000 7f9e3d10e700 1 --2- 172.21.15.154:0/887538500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e38151840 0x7f9e3814ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.246+0000 7f9e3d10e700 1 --2- 172.21.15.154:0/887538500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e380c85f0 0x7f9e3814c970 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.246+0000 7f9e3d10e700 1 --2- 172.21.15.154:0/887538500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e3806f5c0 0x7f9e38146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.246+0000 7f9e3d10e700 1 -- 172.21.15.154:0/887538500 >> 172.21.15.154:0/887538500 conn(0x7f9e380c8a60 msgr2=0x7f9e380c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:17.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.246+0000 7f9e3d10e700 1 -- 172.21.15.154:0/887538500 shutdown_connections 2024-09-13T18:26:17.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.246+0000 7f9e3d10e700 1 -- 172.21.15.154:0/887538500 wait complete. 2024-09-13T18:26:17.265 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 5aef7701-3a54-4e2d-af07-6fe296078938 6 2024-09-13T18:26:17.313 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:17.306+0000 7fc5ad3b02c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:26:17.376 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:17.370+0000 7fc5ad3b02c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:26:17.421 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 Processor -- start 2024-09-13T18:26:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 -- start start 2024-09-13T18:26:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700cc520 0x7fbc70071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbc70150760 0x7fbc701565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc7006e680 con 0x7fbc700cc900 2024-09-13T18:26:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc70071b30 con 0x7fbc70150760 2024-09-13T18:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7006e960 con 0x7fbc700cc520 2024-09-13T18:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc74db9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700cc520 0x7fbc70071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc6ffff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbc700cc900 0x7fbc70150050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43088/0 (socket says 172.21.15.154:43088) 2024-09-13T18:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc6ffff700 1 -- 172.21.15.154:0/2708472830 learned_addr learned my addr 172.21.15.154:0/2708472830 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-09-13T18:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc755ba700 1 --2- 172.21.15.154:0/2708472830 >> v2:172.21.15.146:3300/0 conn(0x7fbc70150760 0x7fbc701565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc74db9700 1 -- 172.21.15.154:0/2708472830 >> v2:172.21.15.146:3300/0 conn(0x7fbc70150760 msgr2=0x7fbc701565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc74db9700 1 --2- 172.21.15.154:0/2708472830 >> v2:172.21.15.146:3300/0 conn(0x7fbc70150760 0x7fbc701565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc74db9700 1 -- 172.21.15.154:0/2708472830 >> v1:172.21.15.154:6789/0 conn(0x7fbc700cc900 legacy=0x7fbc70150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc6f7fe700 1 -- 172.21.15.154:0/2708472830 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036560569 0 0) 0x7fbc7006e680 con 0x7fbc700cc900 2024-09-13T18:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc74db9700 1 -- 172.21.15.154:0/2708472830 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7006ec40 con 0x7fbc700cc520 2024-09-13T18:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc755ba700 1 --2- 172.21.15.154:0/2708472830 >> v2:172.21.15.146:3300/0 conn(0x7fbc70150760 0x7fbc701565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc74db9700 1 --2- 172.21.15.154:0/2708472830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700cc520 0x7fbc70071480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009880 tx=0x7fbc60009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0c3c256e50bc6b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc6f7fe700 1 -- 172.21.15.154:0/2708472830 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc60011830 con 0x7fbc700cc520 2024-09-13T18:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc6f7fe700 1 -- 172.21.15.154:0/2708472830 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc60011990 con 0x7fbc700cc520 2024-09-13T18:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc6f7fe700 1 -- 172.21.15.154:0/2708472830 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc60011d20 con 0x7fbc700cc520 2024-09-13T18:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 -- 172.21.15.154:0/2708472830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700cc520 msgr2=0x7fbc70071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 --2- 172.21.15.154:0/2708472830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700cc520 0x7fbc70071480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009880 tx=0x7fbc60009b90 comp rx=0 tx=0).stop 2024-09-13T18:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 -- 172.21.15.154:0/2708472830 shutdown_connections 2024-09-13T18:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 --2- 172.21.15.154:0/2708472830 >> v2:172.21.15.146:3300/0 conn(0x7fbc70150760 0x7fbc701565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 --2- 172.21.15.154:0/2708472830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700cc520 0x7fbc70071480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 -- 172.21.15.154:0/2708472830 >> 172.21.15.154:0/2708472830 conn(0x7fbc70071ed0 msgr2=0x7fbc700722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 -- 172.21.15.154:0/2708472830 shutdown_connections 2024-09-13T18:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.418+0000 7fbc75dbb700 1 -- 172.21.15.154:0/2708472830 wait complete. 2024-09-13T18:26:17.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc75dbb700 1 Processor -- start 2024-09-13T18:26:17.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc75dbb700 1 -- start start 2024-09-13T18:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc75dbb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc700cc520 0x7fbc70155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc75dbb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700cc900 0x7fbc700c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc75dbb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc700c5450 0x7fbc70152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc75dbb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc700ca9b0 con 0x7fbc700cc520 2024-09-13T18:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc75dbb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70156230 con 0x7fbc700c5450 2024-09-13T18:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc75dbb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70155270 con 0x7fbc700cc900 2024-09-13T18:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc755ba700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc700c5450 0x7fbc70152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc6ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700cc900 0x7fbc700c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc755ba700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc700c5450 0x7fbc70152d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38738/0 (socket says 172.21.15.154:38738) 2024-09-13T18:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc755ba700 1 -- 172.21.15.154:0/3988806593 learned_addr learned my addr 172.21.15.154:0/3988806593 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-13T18:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc74db9700 1 --2- 172.21.15.154:0/3988806593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc700cc520 0x7fbc70155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc755ba700 1 -- 172.21.15.154:0/3988806593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700cc900 msgr2=0x7fbc700c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc755ba700 1 --2- 172.21.15.154:0/3988806593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700cc900 0x7fbc700c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc755ba700 1 -- 172.21.15.154:0/3988806593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc700cc520 msgr2=0x7fbc70155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc755ba700 1 --2- 172.21.15.154:0/3988806593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc700cc520 0x7fbc70155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc755ba700 1 -- 172.21.15.154:0/3988806593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc600094b0 con 0x7fbc700c5450 2024-09-13T18:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc755ba700 1 --2- 172.21.15.154:0/3988806593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc700c5450 0x7fbc70152d60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800cc60 tx=0x7fbc6800cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc6d7fa700 1 -- 172.21.15.154:0/3988806593 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6800ad90 con 0x7fbc700c5450 2024-09-13T18:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc6d7fa700 1 -- 172.21.15.154:0/3988806593 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc68003a40 con 0x7fbc700c5450 2024-09-13T18:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc75dbb700 1 -- 172.21.15.154:0/3988806593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc700cabc0 con 0x7fbc700c5450 2024-09-13T18:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc6d7fa700 1 -- 172.21.15.154:0/3988806593 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68003df0 con 0x7fbc700c5450 2024-09-13T18:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc75dbb700 1 -- 172.21.15.154:0/3988806593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc701554f0 con 0x7fbc700c5450 2024-09-13T18:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc6d7fa700 1 -- 172.21.15.154:0/3988806593 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fbc6800b040 con 0x7fbc700c5450 2024-09-13T18:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.422+0000 7fbc6d7fa700 1 -- 172.21.15.154:0/3988806593 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fbc6800fbb0 con 0x7fbc700c5450 2024-09-13T18:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.426+0000 7fbc75dbb700 1 -- 172.21.15.154:0/3988806593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7006e680 con 0x7fbc700c5450 2024-09-13T18:26:17.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.426+0000 7fbc6d7fa700 1 -- 172.21.15.154:0/3988806593 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fbc6801d060 con 0x7fbc700c5450 2024-09-13T18:26:17.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.502+0000 7fbc75dbb700 1 -- 172.21.15.154:0/3988806593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5aef7701-3a54-4e2d-af07-6fe296078938", "id": 6} v 0) v1 -- 0x7fbc700c9e90 con 0x7fbc700c5450 2024-09-13T18:26:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.510+0000 7fbc6d7fa700 1 -- 172.21.15.154:0/3988806593 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5aef7701-3a54-4e2d-af07-6fe296078938", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbc680270b0 con 0x7fbc700c5450 2024-09-13T18:26:17.517 INFO:teuthology.orchestra.run.smithi154.stdout:6 2024-09-13T18:26:17.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.514+0000 7fbc75dbb700 1 -- 172.21.15.154:0/3988806593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc700c5450 msgr2=0x7fbc70152d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.514+0000 7fbc75dbb700 1 --2- 172.21.15.154:0/3988806593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc700c5450 0x7fbc70152d60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800cc60 tx=0x7fbc6800cf70 comp rx=0 tx=0).stop 2024-09-13T18:26:17.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.514+0000 7fbc75dbb700 1 -- 172.21.15.154:0/3988806593 shutdown_connections 2024-09-13T18:26:17.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.514+0000 7fbc75dbb700 1 --2- 172.21.15.154:0/3988806593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc700c5450 0x7fbc70152d60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.514+0000 7fbc75dbb700 1 --2- 172.21.15.154:0/3988806593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700cc900 0x7fbc700c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.514+0000 7fbc75dbb700 1 --2- 172.21.15.154:0/3988806593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc700cc520 0x7fbc70155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.514+0000 7fbc75dbb700 1 -- 172.21.15.154:0/3988806593 >> 172.21.15.154:0/3988806593 conn(0x7fbc70071ed0 msgr2=0x7fbc7014eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:17.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.514+0000 7fbc75dbb700 1 -- 172.21.15.154:0/3988806593 shutdown_connections 2024-09-13T18:26:17.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.514+0000 7fbc75dbb700 1 -- 172.21.15.154:0/3988806593 wait complete. 2024-09-13T18:26:17.519 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:17.512+0000 7fdce42d52c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:26:17.531 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f 7 2024-09-13T18:26:17.580 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:17.572+0000 7fdce42d52c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:26:17.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df3fff700 1 Processor -- start 2024-09-13T18:26:17.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df3fff700 1 -- start start 2024-09-13T18:26:17.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df3fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8df4151560 0x7f8df415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8df415b510 0x7f8df40c7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df3fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8df40b2e10 0x7f8df40c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df3fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df40c3960 con 0x7f8df4151560 2024-09-13T18:26:17.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df3fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df415a860 con 0x7f8df40b2e10 2024-09-13T18:26:17.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df41572f0 con 0x7f8df415b510 2024-09-13T18:26:17.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df37fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8df40b2e10 0x7f8df40c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df27fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8df415b510 0x7f8df40c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df37fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8df40b2e10 0x7f8df40c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38754/0 (socket says 172.21.15.154:38754) 2024-09-13T18:26:17.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df37fe700 1 -- 172.21.15.154:0/3958178187 learned_addr learned my addr 172.21.15.154:0/3958178187 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-13T18:26:17.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df2ffd700 1 --2- 172.21.15.154:0/3958178187 >> v2:172.21.15.146:3300/0 conn(0x7f8df4151560 0x7f8df415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df37fe700 1 -- 172.21.15.154:0/3958178187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8df415b510 msgr2=0x7f8df40c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:17.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df37fe700 1 --2- 172.21.15.154:0/3958178187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8df415b510 0x7f8df40c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df37fe700 1 -- 172.21.15.154:0/3958178187 >> v2:172.21.15.146:3300/0 conn(0x7f8df4151560 msgr2=0x7f8df415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:17.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df37fe700 1 --2- 172.21.15.154:0/3958178187 >> v2:172.21.15.146:3300/0 conn(0x7f8df4151560 0x7f8df415afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df37fe700 1 -- 172.21.15.154:0/3958178187 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df40c1f20 con 0x7f8df40b2e10 2024-09-13T18:26:17.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df27fc700 1 --2- 172.21.15.154:0/3958178187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8df415b510 0x7f8df40c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:17.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.638+0000 7f8df37fe700 1 --2- 172.21.15.154:0/3958178187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8df40b2e10 0x7f8df40c7e40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009910 tx=0x7f8dec009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97c8d958a749c5f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:17.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df1ffb700 1 -- 172.21.15.154:0/3958178187 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec011830 con 0x7f8df40b2e10 2024-09-13T18:26:17.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df1ffb700 1 -- 172.21.15.154:0/3958178187 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec011990 con 0x7f8df40b2e10 2024-09-13T18:26:17.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df1ffb700 1 -- 172.21.15.154:0/3958178187 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec011d40 con 0x7f8df40b2e10 2024-09-13T18:26:17.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 -- 172.21.15.154:0/3958178187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8df40b2e10 msgr2=0x7f8df40c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 --2- 172.21.15.154:0/3958178187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8df40b2e10 0x7f8df40c7e40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009910 tx=0x7f8dec009c20 comp rx=0 tx=0).stop 2024-09-13T18:26:17.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 -- 172.21.15.154:0/3958178187 shutdown_connections 2024-09-13T18:26:17.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 --2- 172.21.15.154:0/3958178187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8df415b510 0x7f8df40c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 --2- 172.21.15.154:0/3958178187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8df40b2e10 0x7f8df40c7e40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 --2- 172.21.15.154:0/3958178187 >> v2:172.21.15.146:3300/0 conn(0x7f8df4151560 0x7f8df415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 -- 172.21.15.154:0/3958178187 >> 172.21.15.154:0/3958178187 conn(0x7f8df40ccf00 msgr2=0x7f8df40cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:17.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 -- 172.21.15.154:0/3958178187 shutdown_connections 2024-09-13T18:26:17.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 -- 172.21.15.154:0/3958178187 wait complete. 2024-09-13T18:26:17.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 Processor -- start 2024-09-13T18:26:17.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 -- start start 2024-09-13T18:26:17.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8df40b2e10 0x7f8df4159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8df4151560 0x7f8df40c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8df415b510 0x7f8df40c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df41efa80 con 0x7f8df40b2e10 2024-09-13T18:26:17.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4153f10 con 0x7f8df415b510 2024-09-13T18:26:17.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df41eea70 con 0x7f8df4151560 2024-09-13T18:26:17.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df27fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8df4151560 0x7f8df40c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df37fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8df415b510 0x7f8df40c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df27fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8df4151560 0x7f8df40c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:42986/0 (socket says 172.21.15.154:42986) 2024-09-13T18:26:17.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df27fc700 1 -- 172.21.15.154:0/437662161 learned_addr learned my addr 172.21.15.154:0/437662161 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-13T18:26:17.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.642+0000 7f8df2ffd700 1 --2- 172.21.15.154:0/437662161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8df40b2e10 0x7f8df4159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8df27fc700 1 -- 172.21.15.154:0/437662161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8df415b510 msgr2=0x7f8df40c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8df27fc700 1 --2- 172.21.15.154:0/437662161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8df415b510 0x7f8df40c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8df27fc700 1 -- 172.21.15.154:0/437662161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8df40b2e10 msgr2=0x7f8df4159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8df27fc700 1 --2- 172.21.15.154:0/437662161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8df40b2e10 0x7f8df4159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8df27fc700 1 -- 172.21.15.154:0/437662161 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc0094b0 con 0x7f8df4151560 2024-09-13T18:26:17.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8df27fc700 1 --2- 172.21.15.154:0/437662161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8df4151560 0x7f8df40c4640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00ebb0 tx=0x7f8ddc00eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:17.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8deb7fe700 1 -- 172.21.15.154:0/437662161 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc00c850 con 0x7f8df4151560 2024-09-13T18:26:17.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8deb7fe700 1 -- 172.21.15.154:0/437662161 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ddc00c9b0 con 0x7f8df4151560 2024-09-13T18:26:17.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8df3fff700 1 -- 172.21.15.154:0/437662161 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dec009570 con 0x7f8df4151560 2024-09-13T18:26:17.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8deb7fe700 1 -- 172.21.15.154:0/437662161 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc00cd40 con 0x7f8df4151560 2024-09-13T18:26:17.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8df3fff700 1 -- 172.21.15.154:0/437662161 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df41eecc0 con 0x7f8df4151560 2024-09-13T18:26:17.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8deb7fe700 1 -- 172.21.15.154:0/437662161 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f8ddc005d40 con 0x7f8df4151560 2024-09-13T18:26:17.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8deb7fe700 1 -- 172.21.15.154:0/437662161 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f8ddc006890 con 0x7f8df4151560 2024-09-13T18:26:17.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.646+0000 7f8df3fff700 1 -- 172.21.15.154:0/437662161 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df4067a60 con 0x7f8df4151560 2024-09-13T18:26:17.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.650+0000 7f8deb7fe700 1 -- 172.21.15.154:0/437662161 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f8ddc021060 con 0x7f8df4151560 2024-09-13T18:26:17.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.734+0000 7f8df3fff700 1 -- 172.21.15.154:0/437662161 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f", "id": 7} v 0) v1 -- 0x7f8df41580f0 con 0x7f8df4151560 2024-09-13T18:26:17.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.742+0000 7f8deb7fe700 1 -- 172.21.15.154:0/437662161 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8ddc017070 con 0x7f8df4151560 2024-09-13T18:26:17.748 INFO:teuthology.orchestra.run.smithi154.stdout:7 2024-09-13T18:26:17.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.746+0000 7f8df3fff700 1 -- 172.21.15.154:0/437662161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8df4151560 msgr2=0x7f8df40c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.746+0000 7f8df3fff700 1 --2- 172.21.15.154:0/437662161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8df4151560 0x7f8df40c4640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00ebb0 tx=0x7f8ddc00eec0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.746+0000 7f8df3fff700 1 -- 172.21.15.154:0/437662161 shutdown_connections 2024-09-13T18:26:17.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.746+0000 7f8df3fff700 1 --2- 172.21.15.154:0/437662161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8df415b510 0x7f8df40c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.746+0000 7f8df3fff700 1 --2- 172.21.15.154:0/437662161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8df4151560 0x7f8df40c4640 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.746+0000 7f8df3fff700 1 --2- 172.21.15.154:0/437662161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8df40b2e10 0x7f8df4159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.746+0000 7f8df3fff700 1 -- 172.21.15.154:0/437662161 >> 172.21.15.154:0/437662161 conn(0x7f8df40ccf00 msgr2=0x7f8df40c20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:17.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.746+0000 7f8df3fff700 1 -- 172.21.15.154:0/437662161 shutdown_connections 2024-09-13T18:26:17.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-13T18:26:17.746+0000 7f8df3fff700 1 -- 172.21.15.154:0/437662161 wait complete. 2024-09-13T18:26:17.752 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:17.744+0000 7fdce42d52c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:26:17.763 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T18:26:17.763 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T18:26:17.765 INFO:tasks.ceph.osd.0:Started 2024-09-13T18:26:17.766 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T18:26:17.766 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T18:26:17.768 INFO:tasks.ceph.osd.1:Started 2024-09-13T18:26:17.768 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T18:26:17.768 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T18:26:17.770 INFO:tasks.ceph.osd.2:Started 2024-09-13T18:26:17.770 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T18:26:17.770 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T18:26:17.772 INFO:tasks.ceph.osd.3:Started 2024-09-13T18:26:17.773 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T18:26:17.773 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T18:26:17.775 INFO:tasks.ceph.osd.4:Started 2024-09-13T18:26:17.775 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T18:26:17.775 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T18:26:17.777 INFO:tasks.ceph.osd.5:Started 2024-09-13T18:26:17.777 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T18:26:17.777 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T18:26:17.780 INFO:tasks.ceph.osd.6:Started 2024-09-13T18:26:17.780 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T18:26:17.780 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T18:26:17.783 INFO:tasks.ceph.osd.7:Started 2024-09-13T18:26:17.783 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:26:17.794 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:17.790+0000 7fc5ad3b02c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:26:17.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.904+0000 7f0342c72700 1 Processor -- start 2024-09-13T18:26:17.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.904+0000 7f0342c72700 1 -- start start 2024-09-13T18:26:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.904+0000 7f0342c72700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f033c1516c0 0x7f033c157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.904+0000 7f0342c72700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f033c0afae0 0x7f033c0c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.904+0000 7f0342c72700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033c06ac80 con 0x7f033c151aa0 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.904+0000 7f0342c72700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033c0c89d0 con 0x7f033c0afae0 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.904+0000 7f0342c72700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033c0c3b20 con 0x7f033c1516c0 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.904+0000 7f034120f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f033c0afae0 0x7f033c0c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.904+0000 7f033bfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f033c151aa0 0x7f033c0c2360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:46478/0 (socket says 172.21.15.146:46478) 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.904+0000 7f033bfff700 1 -- 172.21.15.146:0/2926180487 learned_addr learned my addr 172.21.15.146:0/2926180487 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.904+0000 7f0340a0e700 1 --2- 172.21.15.146:0/2926180487 >> v2:172.21.15.154:3300/0 conn(0x7f033c1516c0 0x7f033c157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f034120f700 1 -- 172.21.15.146:0/2926180487 >> v2:172.21.15.154:3300/0 conn(0x7f033c1516c0 msgr2=0x7f033c157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f034120f700 1 --2- 172.21.15.146:0/2926180487 >> v2:172.21.15.154:3300/0 conn(0x7f033c1516c0 0x7f033c157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f034120f700 1 -- 172.21.15.146:0/2926180487 >> v1:172.21.15.146:6789/0 conn(0x7f033c151aa0 legacy=0x7f033c0c2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f034120f700 1 -- 172.21.15.146:0/2926180487 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c06a390 con 0x7f033c0afae0 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0340a0e700 1 --2- 172.21.15.146:0/2926180487 >> v2:172.21.15.154:3300/0 conn(0x7f033c1516c0 0x7f033c157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f033b7fe700 1 -- 172.21.15.146:0/2926180487 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809391095 0 0) 0x7f033c06ac80 con 0x7f033c151aa0 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f034120f700 1 --2- 172.21.15.146:0/2926180487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f033c0afae0 0x7f033c0c7d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0334009d90 tx=0x7f033400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae789e4827f8fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f033b7fe700 1 -- 172.21.15.146:0/2926180487 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033400e070 con 0x7f033c0afae0 2024-09-13T18:26:17.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f033b7fe700 1 -- 172.21.15.146:0/2926180487 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033400bb90 con 0x7f033c0afae0 2024-09-13T18:26:17.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f033b7fe700 1 -- 172.21.15.146:0/2926180487 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334012430 con 0x7f033c0afae0 2024-09-13T18:26:17.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 -- 172.21.15.146:0/2926180487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f033c0afae0 msgr2=0x7f033c0c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 --2- 172.21.15.146:0/2926180487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f033c0afae0 0x7f033c0c7d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0334009d90 tx=0x7f033400b330 comp rx=0 tx=0).stop 2024-09-13T18:26:17.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 -- 172.21.15.146:0/2926180487 shutdown_connections 2024-09-13T18:26:17.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 --2- 172.21.15.146:0/2926180487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f033c0afae0 0x7f033c0c7d40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 --2- 172.21.15.146:0/2926180487 >> v2:172.21.15.154:3300/0 conn(0x7f033c1516c0 0x7f033c157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 -- 172.21.15.146:0/2926180487 >> 172.21.15.146:0/2926180487 conn(0x7f033c06c4e0 msgr2=0x7f033c06c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:17.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 -- 172.21.15.146:0/2926180487 shutdown_connections 2024-09-13T18:26:17.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 -- 172.21.15.146:0/2926180487 wait complete. 2024-09-13T18:26:17.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 Processor -- start 2024-09-13T18:26:17.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 -- start start 2024-09-13T18:26:17.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f033c0afae0 0x7f033c0cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c1516c0 0x7f033c0c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f033c151aa0 0x7f033c1492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:17.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c146b10 con 0x7f033c151aa0 2024-09-13T18:26:17.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0b3780 con 0x7f033c1516c0 2024-09-13T18:26:17.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.908+0000 7f0342c72700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0b3010 con 0x7f033c0afae0 2024-09-13T18:26:17.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f034120f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f033c151aa0 0x7f033c1492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f034120f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f033c151aa0 0x7f033c1492d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34682/0 (socket says 172.21.15.146:34682) 2024-09-13T18:26:17.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f034120f700 1 -- 172.21.15.146:0/3560598647 learned_addr learned my addr 172.21.15.146:0/3560598647 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:17.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f0340a0e700 1 --2- 172.21.15.146:0/3560598647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f033c0afae0 0x7f033c0cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f033bfff700 1 --2- 172.21.15.146:0/3560598647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c1516c0 0x7f033c0c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:17.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f034120f700 1 -- 172.21.15.146:0/3560598647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f033c0afae0 msgr2=0x7f033c0cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f034120f700 1 --2- 172.21.15.146:0/3560598647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f033c0afae0 0x7f033c0cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f034120f700 1 -- 172.21.15.146:0/3560598647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c1516c0 msgr2=0x7f033c0c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f034120f700 1 --2- 172.21.15.146:0/3560598647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c1516c0 0x7f033c0c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f034120f700 1 -- 172.21.15.146:0/3560598647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0334009a70 con 0x7f033c151aa0 2024-09-13T18:26:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f034120f700 1 --2- 172.21.15.146:0/3560598647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f033c151aa0 0x7f033c1492d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f033400b730 tx=0x7f0334004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f03397fa700 1 -- 172.21.15.146:0/3560598647 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033401c810 con 0x7f033c151aa0 2024-09-13T18:26:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f03397fa700 1 -- 172.21.15.146:0/3560598647 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0334020070 con 0x7f033c151aa0 2024-09-13T18:26:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f0342c72700 1 -- 172.21.15.146:0/3560598647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033c1f0750 con 0x7f033c151aa0 2024-09-13T18:26:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f03397fa700 1 -- 172.21.15.146:0/3560598647 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033401cbb0 con 0x7f033c151aa0 2024-09-13T18:26:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f0342c72700 1 -- 172.21.15.146:0/3560598647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c1f0100 con 0x7f033c151aa0 2024-09-13T18:26:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f03397fa700 1 -- 172.21.15.146:0/3560598647 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f033401cd10 con 0x7f033c151aa0 2024-09-13T18:26:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f03397fa700 1 -- 172.21.15.146:0/3560598647 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f0334022070 con 0x7f033c151aa0 2024-09-13T18:26:17.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.912+0000 7f0342c72700 1 -- 172.21.15.146:0/3560598647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c1f08e0 con 0x7f033c151aa0 2024-09-13T18:26:17.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.916+0000 7f03397fa700 1 -- 172.21.15.146:0/3560598647 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f0334046e30 con 0x7f033c151aa0 2024-09-13T18:26:17.927 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:17.922+0000 7fc5ad3b02c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:26:17.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.984+0000 7f0342c72700 1 -- 172.21.15.146:0/3560598647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f033c0b3240 con 0x7f033c151aa0 2024-09-13T18:26:17.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.984+0000 7f03397fa700 1 -- 172.21.15.146:0/3560598647 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0334035030 con 0x7f033c151aa0 2024-09-13T18:26:17.994 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:17.994 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":10,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:17.734910+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:26:17.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.988+0000 7f0342c72700 1 -- 172.21.15.146:0/3560598647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f033c151aa0 msgr2=0x7f033c1492d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:17.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.988+0000 7f0342c72700 1 --2- 172.21.15.146:0/3560598647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f033c151aa0 0x7f033c1492d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f033400b730 tx=0x7f0334004af0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.988+0000 7f0342c72700 1 -- 172.21.15.146:0/3560598647 shutdown_connections 2024-09-13T18:26:17.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.988+0000 7f0342c72700 1 --2- 172.21.15.146:0/3560598647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f033c151aa0 0x7f033c1492d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.988+0000 7f0342c72700 1 --2- 172.21.15.146:0/3560598647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c1516c0 0x7f033c0c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.988+0000 7f0342c72700 1 --2- 172.21.15.146:0/3560598647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f033c0afae0 0x7f033c0cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:17.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.988+0000 7f0342c72700 1 -- 172.21.15.146:0/3560598647 >> 172.21.15.146:0/3560598647 conn(0x7f033c06c4e0 msgr2=0x7f033c155e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:17.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.988+0000 7f0342c72700 1 -- 172.21.15.146:0/3560598647 shutdown_connections 2024-09-13T18:26:17.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:17.988+0000 7f0342c72700 1 -- 172.21.15.146:0/3560598647 wait complete. 2024-09-13T18:26:18.007 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T18:26:18.007 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T18:26:18.015 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:18.008+0000 7fdce42d52c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:26:18.071 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:18.064+0000 7fdce42d52c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:26:18.142 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-09-13T18:26:18.138+0000 7ff0ee8ba880 -1 Falling back to public interface 2024-09-13T18:26:18.148 INFO:tasks.ceph.osd.2.smithi146.stderr:2024-09-13T18:26:18.140+0000 7f5e6e5f5880 -1 Falling back to public interface 2024-09-13T18:26:18.149 INFO:tasks.ceph.osd.1.smithi146.stderr:2024-09-13T18:26:18.140+0000 7f0ab47f7880 -1 Falling back to public interface 2024-09-13T18:26:18.156 INFO:tasks.ceph.osd.0.smithi146.stderr:2024-09-13T18:26:18.148+0000 7f4840c10880 -1 Falling back to public interface 2024-09-13T18:26:18.174 INFO:tasks.ceph.osd.3.smithi146.stderr:2024-09-13T18:26:18.168+0000 7f3b0684a880 -1 Falling back to public interface 2024-09-13T18:26:18.266 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:18.260+0000 7fdce42d52c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:26:18.289 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:18.286+0000 7fc5ad3b02c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:26:18.335 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:18.328+0000 7fdce42d52c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:26:18.379 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-09-13T18:26:18.374+0000 7f2c8b44e880 -1 Falling back to public interface 2024-09-13T18:26:18.405 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:18.396+0000 7fdce42d52c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:26:18.406 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-13T18:26:18.402+0000 7f9e854ca880 -1 Falling back to public interface 2024-09-13T18:26:18.406 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-09-13T18:26:18.402+0000 7fe1b4e94880 -1 Falling back to public interface 2024-09-13T18:26:18.472 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:18.465+0000 7fc5ad3b02c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:26:18.540 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:18.533+0000 7fc5ad3b02c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:26:18.703 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:18.697+0000 7fc5ad3b02c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:26:18.813 DEBUG:teuthology.orchestra.run.smithi146:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T18:26:18.850 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:18.844+0000 7fdce42d52c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:26:18.913 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.904+0000 7f14baf09700 1 Processor -- start 2024-09-13T18:26:18.913 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.904+0000 7f14baf09700 1 -- start start 2024-09-13T18:26:18.914 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14baf09700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f14b406d690 0x7f14b40c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:18.914 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14baf09700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b40c5f40 con 0x7f14b415a1a0 2024-09-13T18:26:18.914 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14baf09700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b40cc3e0 con 0x7f14b4153500 2024-09-13T18:26:18.914 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14baf09700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b415a5f0 con 0x7f14b406d690 2024-09-13T18:26:18.914 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14b9706700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f14b415a1a0 0x7f14b406da70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:46490/0 (socket says 172.21.15.146:46490) 2024-09-13T18:26:18.914 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14b9f07700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f14b406d690 0x7f14b40c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:18.914 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14b9706700 1 -- 172.21.15.146:0/170328817 learned_addr learned my addr 172.21.15.146:0/170328817 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:18.916 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14b8f05700 1 -- 172.21.15.146:0/170328817 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191852381 0 0) 0x7f14b40c5f40 con 0x7f14b415a1a0 2024-09-13T18:26:18.916 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14b8f05700 1 -- 172.21.15.146:0/170328817 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14a00035d0 con 0x7f14b415a1a0 2024-09-13T18:26:18.916 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14b9f07700 1 -- 172.21.15.146:0/170328817 >> v1:172.21.15.154:6789/0 conn(0x7f14b4153500 legacy=0x7f14b4153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:18.916 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14b9f07700 1 -- 172.21.15.146:0/170328817 >> v1:172.21.15.146:6789/0 conn(0x7f14b415a1a0 legacy=0x7f14b406da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:18.916 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14b9f07700 1 -- 172.21.15.146:0/170328817 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b40c6190 con 0x7f14b406d690 2024-09-13T18:26:18.916 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14b9f07700 1 --2- 172.21.15.146:0/170328817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14b406d690 0x7f14b40c56a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009850 tx=0x7f14b0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27142f04a01d35f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:18.916 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14b8f05700 1 -- 172.21.15.146:0/170328817 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0011830 con 0x7f14b406d690 2024-09-13T18:26:18.916 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14b8f05700 1 -- 172.21.15.146:0/170328817 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b0011990 con 0x7f14b406d690 2024-09-13T18:26:18.916 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14b8f05700 1 -- 172.21.15.146:0/170328817 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0011d40 con 0x7f14b406d690 2024-09-13T18:26:18.916 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14baf09700 1 -- 172.21.15.146:0/170328817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14b406d690 msgr2=0x7f14b40c56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:18.917 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14baf09700 1 --2- 172.21.15.146:0/170328817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14b406d690 0x7f14b40c56a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009850 tx=0x7f14b0009b60 comp rx=0 tx=0).stop 2024-09-13T18:26:18.917 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14baf09700 1 -- 172.21.15.146:0/170328817 shutdown_connections 2024-09-13T18:26:18.917 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14baf09700 1 --2- 172.21.15.146:0/170328817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14b406d690 0x7f14b40c56a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:18.917 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14baf09700 1 -- 172.21.15.146:0/170328817 >> 172.21.15.146:0/170328817 conn(0x7f14b40721b0 msgr2=0x7f14b40cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:18.917 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14baf09700 1 -- 172.21.15.146:0/170328817 shutdown_connections 2024-09-13T18:26:18.917 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14baf09700 1 -- 172.21.15.146:0/170328817 wait complete. 2024-09-13T18:26:18.917 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14baf09700 1 Processor -- start 2024-09-13T18:26:18.917 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.908+0000 7f14baf09700 1 -- start start 2024-09-13T18:26:18.920 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14baf09700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14b406d690 0x7f14b41eff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:18.920 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14baf09700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14b4153500 0x7f14b415c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:18.920 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14baf09700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14b415a1a0 0x7f14b415d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:18.920 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14baf09700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b415bf10 con 0x7f14b4153500 2024-09-13T18:26:18.920 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14baf09700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b415c870 con 0x7f14b406d690 2024-09-13T18:26:18.920 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14baf09700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b415b8e0 con 0x7f14b415a1a0 2024-09-13T18:26:18.920 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14b9706700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14b4153500 0x7f14b415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:18.920 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14b9706700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14b4153500 0x7f14b415c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34698/0 (socket says 172.21.15.146:34698) 2024-09-13T18:26:18.921 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14b9706700 1 -- 172.21.15.146:0/3920262897 learned_addr learned my addr 172.21.15.146:0/3920262897 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:18.921 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14b9f07700 1 --2- 172.21.15.146:0/3920262897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14b406d690 0x7f14b41eff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:18.921 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14ba708700 1 --2- 172.21.15.146:0/3920262897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14b415a1a0 0x7f14b415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:18.921 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14b9706700 1 -- 172.21.15.146:0/3920262897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14b415a1a0 msgr2=0x7f14b415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:18.921 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14b9706700 1 --2- 172.21.15.146:0/3920262897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14b415a1a0 0x7f14b415d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:18.921 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14b9706700 1 -- 172.21.15.146:0/3920262897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14b406d690 msgr2=0x7f14b41eff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:18.921 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14b9706700 1 --2- 172.21.15.146:0/3920262897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14b406d690 0x7f14b41eff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:18.921 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14b9706700 1 -- 172.21.15.146:0/3920262897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b00094b0 con 0x7f14b4153500 2024-09-13T18:26:18.921 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14b9706700 1 --2- 172.21.15.146:0/3920262897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14b4153500 0x7f14b415c330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f14a400c4d0 tx=0x7f14a400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:18.921 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14aa7fc700 1 -- 172.21.15.146:0/3920262897 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4005d90 con 0x7f14b4153500 2024-09-13T18:26:18.921 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14aa7fc700 1 -- 172.21.15.146:0/3920262897 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a40070d0 con 0x7f14b4153500 2024-09-13T18:26:18.921 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14baf09700 1 -- 172.21.15.146:0/3920262897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b41f2bc0 con 0x7f14b4153500 2024-09-13T18:26:18.921 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14aa7fc700 1 -- 172.21.15.146:0/3920262897 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4007480 con 0x7f14b4153500 2024-09-13T18:26:18.922 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14baf09700 1 -- 172.21.15.146:0/3920262897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b415bb30 con 0x7f14b4153500 2024-09-13T18:26:18.922 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14aa7fc700 1 -- 172.21.15.146:0/3920262897 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f14a401c440 con 0x7f14b4153500 2024-09-13T18:26:18.922 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14aa7fc700 1 -- 172.21.15.146:0/3920262897 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f14a4012ad0 con 0x7f14b4153500 2024-09-13T18:26:18.922 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.912+0000 7f14baf09700 1 -- 172.21.15.146:0/3920262897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b415c050 con 0x7f14b4153500 2024-09-13T18:26:18.923 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.916+0000 7f14aa7fc700 1 -- 172.21.15.146:0/3920262897 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f14a4017070 con 0x7f14b4153500 2024-09-13T18:26:18.983 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:18.976+0000 7fdce42d52c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:26:18.990 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.984+0000 7f14baf09700 1 -- 172.21.15.146:0/3920262897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14b40c3480 con 0x7f14b4153500 2024-09-13T18:26:18.991 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.984+0000 7f14aa7fc700 1 -- 172.21.15.146:0/3920262897 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f14a4012400 con 0x7f14b4153500 2024-09-13T18:26:18.991 INFO:teuthology.misc.health.smithi146.stdout: 2024-09-13T18:26:18.992 INFO:teuthology.misc.health.smithi146.stdout:{"epoch":10,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:17.734910+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:26:18.993 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.984+0000 7f14baf09700 1 -- 172.21.15.146:0/3920262897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14b4153500 msgr2=0x7f14b415c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:18.993 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.984+0000 7f14baf09700 1 --2- 172.21.15.146:0/3920262897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14b4153500 0x7f14b415c330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f14a400c4d0 tx=0x7f14a400c7e0 comp rx=0 tx=0).stop 2024-09-13T18:26:18.993 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.984+0000 7f14baf09700 1 -- 172.21.15.146:0/3920262897 shutdown_connections 2024-09-13T18:26:18.993 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.984+0000 7f14baf09700 1 --2- 172.21.15.146:0/3920262897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14b415a1a0 0x7f14b415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:18.993 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.984+0000 7f14baf09700 1 --2- 172.21.15.146:0/3920262897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14b4153500 0x7f14b415c330 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:18.993 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.984+0000 7f14baf09700 1 --2- 172.21.15.146:0/3920262897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14b406d690 0x7f14b41eff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:18.994 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.984+0000 7f14baf09700 1 -- 172.21.15.146:0/3920262897 >> 172.21.15.146:0/3920262897 conn(0x7f14b40721b0 msgr2=0x7f14b4153950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:18.994 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.984+0000 7f14baf09700 1 -- 172.21.15.146:0/3920262897 shutdown_connections 2024-09-13T18:26:18.994 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:18.984+0000 7f14baf09700 1 -- 172.21.15.146:0/3920262897 wait complete. 2024-09-13T18:26:19.009 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T18:26:19.355 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:19.348+0000 7fdce42d52c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:26:19.407 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:19.401+0000 7fc5ad3b02c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T18:26:19.475 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:19.469+0000 7fc5ad3b02c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:26:19.545 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:19.536+0000 7fdce42d52c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:26:19.612 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:19.604+0000 7fdce42d52c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:26:19.624 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:19.617+0000 7fc5ad3b02c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:26:19.690 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:19.685+0000 7fc5ad3b02c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:26:19.759 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:19.753+0000 7fc5ad3b02c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:26:19.776 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:19.768+0000 7fdce42d52c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:26:20.055 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:20.049+0000 7fc5ad3b02c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:26:20.127 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:20.121+0000 7fc5ad3b02c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:26:20.518 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:20.512+0000 7fdce42d52c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T18:26:20.587 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:20.580+0000 7fdce42d52c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:26:20.769 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:20.760+0000 7fdce42d52c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:26:20.783 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:20.777+0000 7fc5ad3b02c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:26:20.840 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:20.832+0000 7fdce42d52c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:26:20.914 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:20.908+0000 7fdce42d52c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:26:21.272 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:21.264+0000 7fdce42d52c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:26:21.343 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:21.336+0000 7fdce42d52c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:26:21.572 INFO:tasks.ceph.osd.1.smithi146.stderr:2024-09-13T18:26:21.564+0000 7f0ab47f7880 -1 osd.1 0 log_to_monitors true 2024-09-13T18:26:21.722 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-09-13T18:26:21.717+0000 7ff0ee8ba880 -1 osd.5 0 log_to_monitors true 2024-09-13T18:26:21.800 INFO:tasks.ceph.osd.0.smithi146.stderr:2024-09-13T18:26:21.792+0000 7f4840c10880 -1 osd.0 0 log_to_monitors true 2024-09-13T18:26:21.824 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:21.817+0000 7fc593f24700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T18:26:21.824 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-09-13T18:26:21.817+0000 7fc593f24700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T18:26:21.894 INFO:tasks.ceph.osd.3.smithi146.stderr:2024-09-13T18:26:21.888+0000 7f3b0684a880 -1 osd.3 0 log_to_monitors true 2024-09-13T18:26:21.924 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-09-13T18:26:21.917+0000 7fe1b4e94880 -1 osd.7 0 log_to_monitors true 2024-09-13T18:26:21.929 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-09-13T18:26:21.925+0000 7f2c8b44e880 -1 osd.4 0 log_to_monitors true 2024-09-13T18:26:21.999 INFO:tasks.ceph.osd.2.smithi146.stderr:2024-09-13T18:26:21.988+0000 7f5e6e5f5880 -1 osd.2 0 log_to_monitors true 2024-09-13T18:26:22.037 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-09-13T18:26:22.028+0000 7fdce42d52c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:26:22.178 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-13T18:26:22.173+0000 7f9e854ca880 -1 osd.6 0 log_to_monitors true 2024-09-13T18:26:23.768 INFO:tasks.ceph.osd.1.smithi146.stderr:2024-09-13T18:26:23.760+0000 7f0ab0636700 -1 osd.1 0 waiting for initial osdmap 2024-09-13T18:26:23.771 INFO:tasks.ceph.osd.1.smithi146.stderr:2024-09-13T18:26:23.764+0000 7f0aa80e5700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:26:23.784 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-09-13T18:26:23.777+0000 7ff0eb080700 -1 osd.5 0 waiting for initial osdmap 2024-09-13T18:26:23.790 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-09-13T18:26:23.785+0000 7ff0e21a8700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:26:23.803 INFO:tasks.ceph.osd.0.smithi146.stderr:2024-09-13T18:26:23.796+0000 7f483ca4f700 -1 osd.0 0 waiting for initial osdmap 2024-09-13T18:26:23.809 INFO:tasks.ceph.osd.0.smithi146.stderr:2024-09-13T18:26:23.804+0000 7f48344fe700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:26:24.777 INFO:tasks.ceph.osd.3.smithi146.stderr:2024-09-13T18:26:24.768+0000 7f3b02689700 -1 osd.3 0 waiting for initial osdmap 2024-09-13T18:26:24.780 INFO:tasks.ceph.osd.2.smithi146.stderr:2024-09-13T18:26:24.772+0000 7f5e6adbb700 -1 osd.2 0 waiting for initial osdmap 2024-09-13T18:26:24.781 INFO:tasks.ceph.osd.3.smithi146.stderr:2024-09-13T18:26:24.776+0000 7f3afa138700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:26:24.784 INFO:tasks.ceph.osd.2.smithi146.stderr:2024-09-13T18:26:24.776+0000 7f5e61ee3700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:26:24.905 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-09-13T18:26:24.897+0000 7f2c87c14700 -1 osd.4 0 waiting for initial osdmap 2024-09-13T18:26:24.912 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-09-13T18:26:24.905+0000 7f2c7ed3c700 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:26:24.951 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-09-13T18:26:24.945+0000 7fe1b0cd3700 -1 osd.7 0 waiting for initial osdmap 2024-09-13T18:26:24.958 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-13T18:26:24.953+0000 7f9e81309700 -1 osd.6 0 waiting for initial osdmap 2024-09-13T18:26:24.959 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-09-13T18:26:24.953+0000 7fe1a8782700 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:26:24.963 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-13T18:26:24.957+0000 7f9e78db8700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:26:25.817 DEBUG:teuthology.orchestra.run.smithi146:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T18:26:25.927 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.920+0000 7fe7ff59e700 1 Processor -- start 2024-09-13T18:26:25.927 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.920+0000 7fe7ff59e700 1 -- start start 2024-09-13T18:26:25.929 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.920+0000 7fe7ff59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe8000cb1e0 0x7fe8000c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:25.929 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.920+0000 7fe7ff59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8000cb660 con 0x7fe800153740 2024-09-13T18:26:25.929 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.920+0000 7fe7ff59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe800159700 con 0x7fe8000cadd0 2024-09-13T18:26:25.929 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.920+0000 7fe7ff59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8000cb940 con 0x7fe8000cb1e0 2024-09-13T18:26:25.929 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.920+0000 7fe7fe59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fe800153740 0x7fe8000c4020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43936/0 (socket says 172.21.15.146:43936) 2024-09-13T18:26:25.929 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.920+0000 7fe7fe59c700 1 -- 172.21.15.146:0/3595711849 learned_addr learned my addr 172.21.15.146:0/3595711849 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:25.929 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.920+0000 7fe7fed9d700 1 --2- 172.21.15.146:0/3595711849 >> v2:172.21.15.154:3300/0 conn(0x7fe8000cb1e0 0x7fe8000c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:25.930 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.920+0000 7fe7fd59a700 1 -- 172.21.15.146:0/3595711849 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2753461646 0 0) 0x7fe8000cb660 con 0x7fe800153740 2024-09-13T18:26:25.930 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.920+0000 7fe7fd59a700 1 -- 172.21.15.146:0/3595711849 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7e80035d0 con 0x7fe800153740 2024-09-13T18:26:25.931 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.920+0000 7fe7fd59a700 1 -- 172.21.15.146:0/3595711849 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844761925 0 0) 0x7fe800159700 con 0x7fe8000cadd0 2024-09-13T18:26:25.931 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.920+0000 7fe7fd59a700 1 -- 172.21.15.146:0/3595711849 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8000cb660 con 0x7fe8000cadd0 2024-09-13T18:26:25.931 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7fed9d700 1 -- 172.21.15.146:0/3595711849 >> v1:172.21.15.154:6789/0 conn(0x7fe8000cadd0 legacy=0x7fe800070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:25.931 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7fd59a700 1 -- 172.21.15.146:0/3595711849 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233053236 0 0) 0x7fe8000cb660 con 0x7fe8000cadd0 2024-09-13T18:26:25.931 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7fed9d700 1 -- 172.21.15.146:0/3595711849 >> v1:172.21.15.146:6789/0 conn(0x7fe800153740 legacy=0x7fe8000c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:25.931 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7fed9d700 1 -- 172.21.15.146:0/3595711849 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8001599e0 con 0x7fe8000cb1e0 2024-09-13T18:26:25.931 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7fed9d700 1 --2- 172.21.15.146:0/3595711849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8000cb1e0 0x7fe8000c9db0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009dc0 tx=0x7fe7f800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34ffa5ee23cffdd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:25.931 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7fd59a700 1 -- 172.21.15.146:0/3595711849 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f800e070 con 0x7fe8000cb1e0 2024-09-13T18:26:25.931 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7fd59a700 1 -- 172.21.15.146:0/3595711849 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f8003a40 con 0x7fe8000cb1e0 2024-09-13T18:26:25.931 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7fd59a700 1 -- 172.21.15.146:0/3595711849 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0001670 con 0x7fe8000cb1e0 2024-09-13T18:26:25.932 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3595711849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8000cb1e0 msgr2=0x7fe8000c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:25.932 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7ff59e700 1 --2- 172.21.15.146:0/3595711849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8000cb1e0 0x7fe8000c9db0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009dc0 tx=0x7fe7f800b330 comp rx=0 tx=0).stop 2024-09-13T18:26:25.932 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3595711849 shutdown_connections 2024-09-13T18:26:25.932 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7ff59e700 1 --2- 172.21.15.146:0/3595711849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8000cb1e0 0x7fe8000c9db0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:25.932 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3595711849 >> 172.21.15.146:0/3595711849 conn(0x7fe800066620 msgr2=0x7fe800066a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:25.932 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3595711849 shutdown_connections 2024-09-13T18:26:25.933 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3595711849 wait complete. 2024-09-13T18:26:25.933 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7ff59e700 1 Processor -- start 2024-09-13T18:26:25.933 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.924+0000 7fe7ff59e700 1 -- start start 2024-09-13T18:26:25.936 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.928+0000 7fe7ff59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8000cadd0 0x7fe800158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:25.936 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.928+0000 7fe7ff59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8000cb1e0 0x7fe800155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:25.936 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.928+0000 7fe7ff59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe800153740 0x7fe8001e5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:25.936 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.928+0000 7fe7ff59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8001f24c0 con 0x7fe800153740 2024-09-13T18:26:25.936 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.928+0000 7fe7ff59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8001561e0 con 0x7fe8000cadd0 2024-09-13T18:26:25.937 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.928+0000 7fe7ff59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8001f2d20 con 0x7fe8000cb1e0 2024-09-13T18:26:25.937 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.928+0000 7fe7fed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe800153740 0x7fe8001e5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:25.937 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.928+0000 7fe7fed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe800153740 0x7fe8001e5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60340/0 (socket says 172.21.15.146:60340) 2024-09-13T18:26:25.938 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.928+0000 7fe7fed9d700 1 -- 172.21.15.146:0/3544446966 learned_addr learned my addr 172.21.15.146:0/3544446966 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:25.938 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.928+0000 7fe7fed9d700 1 -- 172.21.15.146:0/3544446966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8000cb1e0 msgr2=0x7fe800155790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:26:25.938 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.932+0000 7fe7fed9d700 1 --2- 172.21.15.146:0/3544446966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8000cb1e0 0x7fe800155790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:25.938 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.932+0000 7fe7fed9d700 1 -- 172.21.15.146:0/3544446966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8000cadd0 msgr2=0x7fe800158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:25.938 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.932+0000 7fe7fed9d700 1 --2- 172.21.15.146:0/3544446966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8000cadd0 0x7fe800158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:25.938 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.932+0000 7fe7fed9d700 1 -- 172.21.15.146:0/3544446966 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f8009a70 con 0x7fe800153740 2024-09-13T18:26:25.938 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.932+0000 7fe7fed9d700 1 --2- 172.21.15.146:0/3544446966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe800153740 0x7fe8001e5d90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8015040 tx=0x7fe7f8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:25.939 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.932+0000 7fe7eeffd700 1 -- 172.21.15.146:0/3544446966 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4001670 con 0x7fe800153740 2024-09-13T18:26:25.939 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.932+0000 7fe7eeffd700 1 -- 172.21.15.146:0/3544446966 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f8004340 con 0x7fe800153740 2024-09-13T18:26:25.939 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.932+0000 7fe7eeffd700 1 -- 172.21.15.146:0/3544446966 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8004710 con 0x7fe800153740 2024-09-13T18:26:25.939 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.932+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3544446966 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8000708b0 con 0x7fe800153740 2024-09-13T18:26:25.939 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.932+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3544446966 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe800157920 con 0x7fe800153740 2024-09-13T18:26:25.940 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.932+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3544446966 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe800153b20 con 0x7fe800153740 2024-09-13T18:26:25.941 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.932+0000 7fe7eeffd700 1 -- 172.21.15.146:0/3544446966 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7fe7f8012450 con 0x7fe800153740 2024-09-13T18:26:25.942 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.936+0000 7fe7eeffd700 1 --2- 172.21.15.146:0/3544446966 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe7e00666a0 0x7fe7e0068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:25.942 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.936+0000 7fe7eeffd700 1 -- 172.21.15.146:0/3544446966 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe7f80852a0 con 0x7fe800153740 2024-09-13T18:26:25.942 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.936+0000 7fe7fe59c700 1 --2- 172.21.15.146:0/3544446966 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe7e00666a0 0x7fe7e0068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:25.942 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.936+0000 7fe7fe59c700 1 --2- 172.21.15.146:0/3544446966 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe7e00666a0 0x7fe7e0068b60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0007fd0 tx=0x7fe7f00078c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:25.948 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:25.940+0000 7fe7eeffd700 1 -- 172.21.15.146:0/3544446966 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe7f8052a50 con 0x7fe800153740 2024-09-13T18:26:26.116 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.108+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3544446966 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8001f2e60 con 0x7fe800153740 2024-09-13T18:26:26.118 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.112+0000 7fe7eeffd700 1 -- 172.21.15.146:0/3544446966 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+12842 (secure 0 0 0) 0x7fe7f8056070 con 0x7fe800153740 2024-09-13T18:26:26.118 INFO:teuthology.misc.health.smithi146.stdout: 2024-09-13T18:26:26.118 INFO:teuthology.misc.health.smithi146.stdout:{"epoch":15,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:25.888175+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:26:26.123 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.116+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3544446966 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe7e00666a0 msgr2=0x7fe7e0068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.123 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.116+0000 7fe7ff59e700 1 --2- 172.21.15.146:0/3544446966 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe7e00666a0 0x7fe7e0068b60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0007fd0 tx=0x7fe7f00078c0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.123 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.116+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3544446966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe800153740 msgr2=0x7fe8001e5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.123 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.116+0000 7fe7ff59e700 1 --2- 172.21.15.146:0/3544446966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe800153740 0x7fe8001e5d90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8015040 tx=0x7fe7f8004af0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.123 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.116+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3544446966 shutdown_connections 2024-09-13T18:26:26.123 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.116+0000 7fe7ff59e700 1 --2- 172.21.15.146:0/3544446966 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe7e00666a0 0x7fe7e0068b60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.123 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.116+0000 7fe7ff59e700 1 --2- 172.21.15.146:0/3544446966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe800153740 0x7fe8001e5d90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.123 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.116+0000 7fe7ff59e700 1 --2- 172.21.15.146:0/3544446966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8000cb1e0 0x7fe800155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.123 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.116+0000 7fe7ff59e700 1 --2- 172.21.15.146:0/3544446966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8000cadd0 0x7fe800158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.123 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.116+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3544446966 >> 172.21.15.146:0/3544446966 conn(0x7fe800066620 msgr2=0x7fe8000cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:26.123 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.116+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3544446966 shutdown_connections 2024-09-13T18:26:26.123 INFO:teuthology.misc.health.smithi146.stderr:2024-09-13T18:26:26.116+0000 7fe7ff59e700 1 -- 172.21.15.146:0/3544446966 wait complete. 2024-09-13T18:26:26.136 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T18:26:26.136 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T18:26:26.137 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T18:26:26.137 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T18:26:26.139 INFO:tasks.ceph.mds.a:Started 2024-09-13T18:26:26.139 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T18:26:26.139 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T18:26:26.141 INFO:tasks.ceph.mds.c:Started 2024-09-13T18:26:26.141 INFO:tasks.ceph.mds.e:Restarting daemon 2024-09-13T18:26:26.141 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-09-13T18:26:26.143 INFO:tasks.ceph.mds.e:Started 2024-09-13T18:26:26.143 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T18:26:26.143 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T18:26:26.145 INFO:tasks.ceph.mds.b:Started 2024-09-13T18:26:26.146 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T18:26:26.146 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T18:26:26.148 INFO:tasks.ceph.mds.d:Started 2024-09-13T18:26:26.148 INFO:tasks.ceph.mds.f:Restarting daemon 2024-09-13T18:26:26.148 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-09-13T18:26:26.150 INFO:tasks.ceph.mds.f:Started 2024-09-13T18:26:26.150 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-13T18:26:26.151 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:26:26.210 INFO:tasks.ceph.mds.e.smithi146.stdout:starting mds.e at 2024-09-13T18:26:26.225 INFO:tasks.ceph.mds.a.smithi146.stdout:starting mds.a at 2024-09-13T18:26:26.226 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2024-09-13T18:26:26.234 INFO:tasks.ceph.mds.c.smithi146.stdout:starting mds.c at 2024-09-13T18:26:26.244 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2024-09-13T18:26:26.246 INFO:tasks.ceph.mds.f.smithi154.stdout:starting mds.f at 2024-09-13T18:26:26.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.248+0000 7f36123ed700 1 Processor -- start 2024-09-13T18:26:26.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.248+0000 7f36123ed700 1 -- start start 2024-09-13T18:26:26.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f36123ed700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c6770 0x7f360c159190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f36123ed700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c156810 con 0x7f360c14fb50 2024-09-13T18:26:26.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f36123ed700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c1559b0 con 0x7f360c0c7b00 2024-09-13T18:26:26.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f36123ed700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c155c90 con 0x7f360c0c6770 2024-09-13T18:26:26.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f36113eb700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f360c14fb50 0x7f360c0c7fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43950/0 (socket says 172.21.15.146:43950) 2024-09-13T18:26:26.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f36113eb700 1 -- 172.21.15.146:0/2016584748 learned_addr learned my addr 172.21.15.146:0/2016584748 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:26.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217235504 0 0) 0x7f360c156810 con 0x7f360c14fb50 2024-09-13T18:26:26.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35f80046e0 con 0x7f360c14fb50 2024-09-13T18:26:26.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3611bec700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c6770 0x7f360c159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3649221487 0 0) 0x7f35f80046e0 con 0x7f360c14fb50 2024-09-13T18:26:26.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c6770 msgr2=0x7f360c159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:26.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c6770 0x7f360c159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 >> v1:172.21.15.154:6789/0 conn(0x7f360c0c7b00 legacy=0x7f360c0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c155f70 con 0x7f360c14fb50 2024-09-13T18:26:26.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f3608001690 con 0x7f360c14fb50 2024-09-13T18:26:26.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3611bec700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c6770 0x7f360c159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:26.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 >> v1:172.21.15.146:6789/0 conn(0x7f360c14fb50 legacy=0x7f360c0c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f8004bd0 0x7f35f8004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35f8005980 0x7f35f8005e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c6770 0x7f35f8006340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c155c90 con 0x7f35f8005980 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3610bea700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35f8005980 0x7f35f8005e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f800cc20 con 0x7f35f8004bd0 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f800cde0 con 0x7f360c0c6770 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f36113eb700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f8004bd0 0x7f35f8004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3611bec700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c6770 0x7f35f8006340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3610bea700 1 -- 172.21.15.146:0/2016584748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c6770 msgr2=0x7f35f8006340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3610bea700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c6770 0x7f35f8006340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3610bea700 1 -- 172.21.15.146:0/2016584748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f8004bd0 msgr2=0x7f35f8004fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3610bea700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f8004bd0 0x7f35f8004fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3610bea700 1 -- 172.21.15.146:0/2016584748 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f80046e0 con 0x7f35f8005980 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3611bec700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c6770 0x7f35f8006340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3610bea700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35f8005980 0x7f35f8005e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f35fc007fd0 tx=0x7f35f80069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2308a7090dfa8ad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0052d0 con 0x7f35f8005980 2024-09-13T18:26:26.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35fc005450 con 0x7f35f8005980 2024-09-13T18:26:26.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3603fff700 1 -- 172.21.15.146:0/2016584748 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0038e0 con 0x7f35f8005980 2024-09-13T18:26:26.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f36123ed700 1 -- 172.21.15.146:0/2016584748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35f8005980 msgr2=0x7f35f8005e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f36123ed700 1 --2- 172.21.15.146:0/2016584748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35f8005980 0x7f35f8005e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f35fc007fd0 tx=0x7f35f80069b0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f3611bec700 1 -- 172.21.15.146:0/2016584748 reap_dead start 2024-09-13T18:26:26.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f36123ed700 1 -- 172.21.15.146:0/2016584748 shutdown_connections 2024-09-13T18:26:26.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f36123ed700 1 -- 172.21.15.146:0/2016584748 >> 172.21.15.146:0/2016584748 conn(0x7f360c066b00 msgr2=0x7f360c06aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:26.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f36123ed700 1 -- 172.21.15.146:0/2016584748 shutdown_connections 2024-09-13T18:26:26.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.252+0000 7f36123ed700 1 -- 172.21.15.146:0/2016584748 wait complete. 2024-09-13T18:26:26.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f36123ed700 1 Processor -- start 2024-09-13T18:26:26.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f36123ed700 1 -- start start 2024-09-13T18:26:26.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f36123ed700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f8005980 0x7f360c1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f36123ed700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c4c40 0x7f360c1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f36123ed700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f360c14c7b0 0x7f360c141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f36123ed700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c150480 con 0x7f360c14c7b0 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f36123ed700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c1ee730 con 0x7f35f8005980 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f36123ed700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c145920 con 0x7f360c0c4c40 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f3611bec700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f360c14c7b0 0x7f360c141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f36113eb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f8005980 0x7f360c1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f3611bec700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f360c14c7b0 0x7f360c141de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60402/0 (socket says 172.21.15.146:60402) 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f36113eb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f8005980 0x7f360c1569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:55114/0 (socket says 172.21.15.146:55114) 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f3610bea700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c4c40 0x7f360c1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f3611bec700 1 -- 172.21.15.146:0/4106096235 learned_addr learned my addr 172.21.15.146:0/4106096235 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f3611bec700 1 -- 172.21.15.146:0/4106096235 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c4c40 msgr2=0x7f360c1574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f3611bec700 1 --2- 172.21.15.146:0/4106096235 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c4c40 0x7f360c1574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f3611bec700 1 -- 172.21.15.146:0/4106096235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f8005980 msgr2=0x7f360c1569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f3611bec700 1 --2- 172.21.15.146:0/4106096235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f8005980 0x7f360c1569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f3611bec700 1 -- 172.21.15.146:0/4106096235 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c06d640 con 0x7f360c14c7b0 2024-09-13T18:26:26.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f36113eb700 1 --2- 172.21.15.146:0/4106096235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f8005980 0x7f360c1569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:26.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f3611bec700 1 --2- 172.21.15.146:0/4106096235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f360c14c7b0 0x7f360c141de0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f360c06e950 tx=0x7f3604004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f3601ffb700 1 -- 172.21.15.146:0/4106096235 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36040079c0 con 0x7f360c14c7b0 2024-09-13T18:26:26.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f3601ffb700 1 -- 172.21.15.146:0/4106096235 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360400f050 con 0x7f360c14c7b0 2024-09-13T18:26:26.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f36123ed700 1 -- 172.21.15.146:0/4106096235 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c06d920 con 0x7f360c14c7b0 2024-09-13T18:26:26.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f3601ffb700 1 -- 172.21.15.146:0/4106096235 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604007d60 con 0x7f360c14c7b0 2024-09-13T18:26:26.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.256+0000 7f36123ed700 1 -- 172.21.15.146:0/4106096235 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c1ee980 con 0x7f360c14c7b0 2024-09-13T18:26:26.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.260+0000 7f36123ed700 1 -- 172.21.15.146:0/4106096235 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c0c6060 con 0x7f360c14c7b0 2024-09-13T18:26:26.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.260+0000 7f3601ffb700 1 -- 172.21.15.146:0/4106096235 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f360401e460 con 0x7f360c14c7b0 2024-09-13T18:26:26.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.260+0000 7f3601ffb700 1 --2- 172.21.15.146:0/4106096235 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f35f00666d0 0x7f35f0068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.260+0000 7f3601ffb700 1 -- 172.21.15.146:0/4106096235 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3604083f20 con 0x7f360c14c7b0 2024-09-13T18:26:26.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.260+0000 7f36113eb700 1 --2- 172.21.15.146:0/4106096235 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f35f00666d0 0x7f35f0068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.260+0000 7f36113eb700 1 --2- 172.21.15.146:0/4106096235 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f35f00666d0 0x7f35f0068b90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3608001050 tx=0x7f360800a040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.268+0000 7f3601ffb700 1 -- 172.21.15.146:0/4106096235 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f36040516b0 con 0x7f360c14c7b0 2024-09-13T18:26:26.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.437+0000 7f36123ed700 1 -- 172.21.15.146:0/4106096235 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f360c0c6c30 con 0x7f360c14c7b0 2024-09-13T18:26:26.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.441+0000 7f3601ffb700 1 -- 172.21.15.146:0/4106096235 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+12842 (secure 0 0 0) 0x7f3604054cd0 con 0x7f360c14c7b0 2024-09-13T18:26:26.446 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:26.446 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":15,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:25.888175+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:26:26.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.441+0000 7f36123ed700 1 -- 172.21.15.146:0/4106096235 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f35f00666d0 msgr2=0x7f35f0068b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.441+0000 7f36123ed700 1 --2- 172.21.15.146:0/4106096235 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f35f00666d0 0x7f35f0068b90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3608001050 tx=0x7f360800a040 comp rx=0 tx=0).stop 2024-09-13T18:26:26.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.441+0000 7f36123ed700 1 -- 172.21.15.146:0/4106096235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f360c14c7b0 msgr2=0x7f360c141de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.441+0000 7f36123ed700 1 --2- 172.21.15.146:0/4106096235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f360c14c7b0 0x7f360c141de0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f360c06e950 tx=0x7f3604004690 comp rx=0 tx=0).stop 2024-09-13T18:26:26.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.441+0000 7f36123ed700 1 -- 172.21.15.146:0/4106096235 shutdown_connections 2024-09-13T18:26:26.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.441+0000 7f36123ed700 1 --2- 172.21.15.146:0/4106096235 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f35f00666d0 0x7f35f0068b90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.441+0000 7f36123ed700 1 --2- 172.21.15.146:0/4106096235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f360c14c7b0 0x7f360c141de0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.441+0000 7f36123ed700 1 --2- 172.21.15.146:0/4106096235 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360c0c4c40 0x7f360c1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.441+0000 7f36123ed700 1 --2- 172.21.15.146:0/4106096235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f8005980 0x7f360c1569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.441+0000 7f36123ed700 1 -- 172.21.15.146:0/4106096235 >> 172.21.15.146:0/4106096235 conn(0x7f360c066b00 msgr2=0x7f360c0c7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:26.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.441+0000 7f36123ed700 1 -- 172.21.15.146:0/4106096235 shutdown_connections 2024-09-13T18:26:26.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.441+0000 7f36123ed700 1 -- 172.21.15.146:0/4106096235 wait complete. 2024-09-13T18:26:26.461 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:26:24.867515+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:26:26.461 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:26:26.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361997700 1 Processor -- start 2024-09-13T18:26:26.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361997700 1 -- start start 2024-09-13T18:26:26.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361997700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f535c070500 0x7f535c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361997700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f535c0aa360 0x7f535c0c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361997700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c0c94a0 con 0x7f535c159b70 2024-09-13T18:26:26.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361997700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535c0c7df0 con 0x7f535c0aa360 2024-09-13T18:26:26.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361997700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535c0c8090 con 0x7f535c070500 2024-09-13T18:26:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361196700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f535c0aa360 0x7f535c0c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361196700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f535c0aa360 0x7f535c0c5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60404/0 (socket says 172.21.15.146:60404) 2024-09-13T18:26:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361196700 1 -- 172.21.15.146:0/1081555877 learned_addr learned my addr 172.21.15.146:0/1081555877 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f535bfff700 1 --2- 172.21.15.146:0/1081555877 >> v2:172.21.15.154:3300/0 conn(0x7f535c070500 0x7f535c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361196700 1 -- 172.21.15.146:0/1081555877 >> v2:172.21.15.154:3300/0 conn(0x7f535c070500 msgr2=0x7f535c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361196700 1 --2- 172.21.15.146:0/1081555877 >> v2:172.21.15.154:3300/0 conn(0x7f535c070500 0x7f535c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361196700 1 -- 172.21.15.146:0/1081555877 >> v1:172.21.15.154:6789/0 conn(0x7f535c159b70 legacy=0x7f535c0a9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361196700 1 -- 172.21.15.146:0/1081555877 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0c8b70 con 0x7f535c0aa360 2024-09-13T18:26:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f535bfff700 1 --2- 172.21.15.146:0/1081555877 >> v2:172.21.15.154:3300/0 conn(0x7f535c070500 0x7f535c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f5361196700 1 --2- 172.21.15.146:0/1081555877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f535c0aa360 0x7f535c0c5d70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f535c0c5390 tx=0x7f5354009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9821de5b4ef777 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.573+0000 7f535b7fe700 1 -- 172.21.15.146:0/1081555877 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535400b8a0 con 0x7f535c0aa360 2024-09-13T18:26:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f535b7fe700 1 -- 172.21.15.146:0/1081555877 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535400ba00 con 0x7f535c0aa360 2024-09-13T18:26:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f535b7fe700 1 -- 172.21.15.146:0/1081555877 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535400bd90 con 0x7f535c0aa360 2024-09-13T18:26:26.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 -- 172.21.15.146:0/1081555877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f535c0aa360 msgr2=0x7f535c0c5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 --2- 172.21.15.146:0/1081555877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f535c0aa360 0x7f535c0c5d70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f535c0c5390 tx=0x7f5354009bf0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 -- 172.21.15.146:0/1081555877 shutdown_connections 2024-09-13T18:26:26.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 --2- 172.21.15.146:0/1081555877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f535c0aa360 0x7f535c0c5d70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 --2- 172.21.15.146:0/1081555877 >> v2:172.21.15.154:3300/0 conn(0x7f535c070500 0x7f535c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 -- 172.21.15.146:0/1081555877 >> 172.21.15.146:0/1081555877 conn(0x7f535c066090 msgr2=0x7f535c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:26.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 -- 172.21.15.146:0/1081555877 shutdown_connections 2024-09-13T18:26:26.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 -- 172.21.15.146:0/1081555877 wait complete. 2024-09-13T18:26:26.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 Processor -- start 2024-09-13T18:26:26.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 -- start start 2024-09-13T18:26:26.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c070500 0x7f535c157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f535c0aa360 0x7f535c0c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f535c159b70 0x7f535c0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c1f1840 con 0x7f535c0aa360 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c1f13b0 con 0x7f535c070500 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.577+0000 7f5361997700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c1534f0 con 0x7f535c159b70 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f535bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f535c0aa360 0x7f535c0c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f535bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f535c0aa360 0x7f535c0c20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60418/0 (socket says 172.21.15.146:60418) 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f535bfff700 1 -- 172.21.15.146:0/2484448329 learned_addr learned my addr 172.21.15.146:0/2484448329 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f5361196700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f535c159b70 0x7f535c0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f5360995700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c070500 0x7f535c157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f535bfff700 1 -- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f535c159b70 msgr2=0x7f535c0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f535bfff700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f535c159b70 0x7f535c0ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f535bfff700 1 -- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c070500 msgr2=0x7f535c157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f535bfff700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c070500 0x7f535c157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f535bfff700 1 -- 172.21.15.146:0/2484448329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5354009570 con 0x7f535c0aa360 2024-09-13T18:26:26.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f535bfff700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f535c0aa360 0x7f535c0c20b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f534400e760 tx=0x7f534400ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f53597fa700 1 -- 172.21.15.146:0/2484448329 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534400c070 con 0x7f535c0aa360 2024-09-13T18:26:26.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f53597fa700 1 -- 172.21.15.146:0/2484448329 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5344004250 con 0x7f535c0aa360 2024-09-13T18:26:26.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f5361997700 1 -- 172.21.15.146:0/2484448329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c1f0a80 con 0x7f535c0aa360 2024-09-13T18:26:26.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f53597fa700 1 -- 172.21.15.146:0/2484448329 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53440036c0 con 0x7f535c0aa360 2024-09-13T18:26:26.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f5361997700 1 -- 172.21.15.146:0/2484448329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c1f1040 con 0x7f535c0aa360 2024-09-13T18:26:26.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f5361997700 1 -- 172.21.15.146:0/2484448329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c06f6c0 con 0x7f535c0aa360 2024-09-13T18:26:26.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.581+0000 7f53597fa700 1 -- 172.21.15.146:0/2484448329 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f5344003820 con 0x7f535c0aa360 2024-09-13T18:26:26.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.585+0000 7f53597fa700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53400666a0 0x7f5340068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.585+0000 7f53597fa700 1 -- 172.21.15.146:0/2484448329 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5344083a40 con 0x7f535c0aa360 2024-09-13T18:26:26.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.585+0000 7f5360995700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53400666a0 0x7f5340068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.585+0000 7f5360995700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53400666a0 0x7f5340068b60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f534c00f290 tx=0x7f534c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.589+0000 7f53597fa700 1 -- 172.21.15.146:0/2484448329 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5344051140 con 0x7f535c0aa360 2024-09-13T18:26:26.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.753+0000 7f5361997700 1 -- 172.21.15.146:0/2484448329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f535c0aa140 con 0x7f535c0aa360 2024-09-13T18:26:26.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.757+0000 7f53597fa700 1 -- 172.21.15.146:0/2484448329 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5344054760 con 0x7f535c0aa360 2024-09-13T18:26:26.762 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-09-13T18:26:26.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.757+0000 7f5361997700 1 -- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53400666a0 msgr2=0x7f5340068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.757+0000 7f5361997700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53400666a0 0x7f5340068b60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f534c00f290 tx=0x7f534c009040 comp rx=0 tx=0).stop 2024-09-13T18:26:26.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.757+0000 7f5361997700 1 -- 172.21.15.146:0/2484448329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f535c0aa360 msgr2=0x7f535c0c20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.757+0000 7f5361997700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f535c0aa360 0x7f535c0c20b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f534400e760 tx=0x7f534400ea70 comp rx=0 tx=0).stop 2024-09-13T18:26:26.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.757+0000 7f5361997700 1 -- 172.21.15.146:0/2484448329 shutdown_connections 2024-09-13T18:26:26.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.757+0000 7f5361997700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53400666a0 0x7f5340068b60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.757+0000 7f5361997700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f535c159b70 0x7f535c0ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.757+0000 7f5361997700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f535c0aa360 0x7f535c0c20b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.757+0000 7f5361997700 1 --2- 172.21.15.146:0/2484448329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c070500 0x7f535c157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.757+0000 7f5361997700 1 -- 172.21.15.146:0/2484448329 >> 172.21.15.146:0/2484448329 conn(0x7f535c066090 msgr2=0x7f535c156340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:26.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.757+0000 7f5361997700 1 -- 172.21.15.146:0/2484448329 shutdown_connections 2024-09-13T18:26:26.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.757+0000 7f5361997700 1 -- 172.21.15.146:0/2484448329 wait complete. 2024-09-13T18:26:26.777 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:26:26.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89db59e700 1 Processor -- start 2024-09-13T18:26:26.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89db59e700 1 -- start start 2024-09-13T18:26:26.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89db59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f89dc0cf170 0x7f89dc1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89db59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f89dc0cb5b0 0x7f89dc1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89db59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89dc15a540 con 0x7f89dc0cf550 2024-09-13T18:26:26.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89db59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89dc0c5220 con 0x7f89dc0cb5b0 2024-09-13T18:26:26.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89db59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89dc159ca0 con 0x7f89dc0cf170 2024-09-13T18:26:26.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89dad9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f89dc0cb5b0 0x7f89dc1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89dad9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f89dc0cb5b0 0x7f89dc1534c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60424/0 (socket says 172.21.15.146:60424) 2024-09-13T18:26:26.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89dad9d700 1 -- 172.21.15.146:0/837665670 learned_addr learned my addr 172.21.15.146:0/837665670 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:26.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89da59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f89dc0cf170 0x7f89dc1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89dad9d700 1 -- 172.21.15.146:0/837665670 >> v2:172.21.15.154:3300/0 conn(0x7f89dc0cf170 msgr2=0x7f89dc1595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:26.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89dad9d700 1 --2- 172.21.15.146:0/837665670 >> v2:172.21.15.154:3300/0 conn(0x7f89dc0cf170 0x7f89dc1595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89dad9d700 1 -- 172.21.15.146:0/837665670 >> v1:172.21.15.154:6789/0 conn(0x7f89dc0cf550 legacy=0x7f89dc0c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89dad9d700 1 -- 172.21.15.146:0/837665670 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc159f80 con 0x7f89dc0cb5b0 2024-09-13T18:26:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89dad9d700 1 --2- 172.21.15.146:0/837665670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89dc0cb5b0 0x7f89dc1534c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f89dc08c1e0 tx=0x7f89d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4edf758925f9674 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89d9d9b700 1 -- 172.21.15.146:0/837665670 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d400e070 con 0x7f89dc0cb5b0 2024-09-13T18:26:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89d9d9b700 1 -- 172.21.15.146:0/837665670 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d400b910 con 0x7f89dc0cb5b0 2024-09-13T18:26:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.877+0000 7f89d9d9b700 1 -- 172.21.15.146:0/837665670 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d400bcc0 con 0x7f89dc0cb5b0 2024-09-13T18:26:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 -- 172.21.15.146:0/837665670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89dc0cb5b0 msgr2=0x7f89dc1534c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 --2- 172.21.15.146:0/837665670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89dc0cb5b0 0x7f89dc1534c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f89dc08c1e0 tx=0x7f89d400b330 comp rx=0 tx=0).stop 2024-09-13T18:26:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 -- 172.21.15.146:0/837665670 shutdown_connections 2024-09-13T18:26:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 --2- 172.21.15.146:0/837665670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89dc0cb5b0 0x7f89dc1534c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 --2- 172.21.15.146:0/837665670 >> v2:172.21.15.154:3300/0 conn(0x7f89dc0cf170 0x7f89dc1595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 -- 172.21.15.146:0/837665670 >> 172.21.15.146:0/837665670 conn(0x7f89dc072220 msgr2=0x7f89dc066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:26.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 -- 172.21.15.146:0/837665670 shutdown_connections 2024-09-13T18:26:26.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 -- 172.21.15.146:0/837665670 wait complete. 2024-09-13T18:26:26.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 Processor -- start 2024-09-13T18:26:26.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 -- start start 2024-09-13T18:26:26.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89dc0cb5b0 0x7f89dc1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89dc0cf170 0x7f89dc1e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc0cf550 0x7f89dc0b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc1ef3d0 con 0x7f89dc0cf170 2024-09-13T18:26:26.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc159050 con 0x7f89dc0cf550 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc158440 con 0x7f89dc0cb5b0 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89d3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89dc0cf170 0x7f89dc1e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89d3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89dc0cf170 0x7f89dc1e0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60434/0 (socket says 172.21.15.146:60434) 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89d3fff700 1 -- 172.21.15.146:0/392626858 learned_addr learned my addr 172.21.15.146:0/392626858 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89da59c700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89dc0cb5b0 0x7f89dc1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89dad9d700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc0cf550 0x7f89dc0b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89d3fff700 1 -- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89dc0cb5b0 msgr2=0x7f89dc1594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89d3fff700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89dc0cb5b0 0x7f89dc1594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89d3fff700 1 -- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc0cf550 msgr2=0x7f89dc0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89d3fff700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc0cf550 0x7f89dc0b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89d3fff700 1 -- 172.21.15.146:0/392626858 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d4009b30 con 0x7f89dc0cf170 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89dad9d700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc0cf550 0x7f89dc0b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89d3fff700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89dc0cf170 0x7f89dc1e0a70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f89c400c500 tx=0x7f89c400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89d2ffd700 1 -- 172.21.15.146:0/392626858 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c40052d0 con 0x7f89dc0cf170 2024-09-13T18:26:26.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89d2ffd700 1 -- 172.21.15.146:0/392626858 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c4005430 con 0x7f89dc0cf170 2024-09-13T18:26:26.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 -- 172.21.15.146:0/392626858 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89dc158720 con 0x7f89dc0cf170 2024-09-13T18:26:26.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89d2ffd700 1 -- 172.21.15.146:0/392626858 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c40055a0 con 0x7f89dc0cf170 2024-09-13T18:26:26.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.881+0000 7f89db59e700 1 -- 172.21.15.146:0/392626858 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89dc158ce0 con 0x7f89dc0cf170 2024-09-13T18:26:26.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.885+0000 7f89db59e700 1 -- 172.21.15.146:0/392626858 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89dc1591e0 con 0x7f89dc0cf170 2024-09-13T18:26:26.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.885+0000 7f89d2ffd700 1 -- 172.21.15.146:0/392626858 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f89c40038e0 con 0x7f89dc0cf170 2024-09-13T18:26:26.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.885+0000 7f89d2ffd700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f89bc0666a0 0x7f89bc068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.885+0000 7f89d2ffd700 1 -- 172.21.15.146:0/392626858 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f89c4083200 con 0x7f89dc0cf170 2024-09-13T18:26:26.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.885+0000 7f89da59c700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f89bc0666a0 0x7f89bc068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.885+0000 7f89da59c700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f89bc0666a0 0x7f89bc068b60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009bf0 tx=0x7f89cc009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:26.893+0000 7f89d2ffd700 1 -- 172.21.15.146:0/392626858 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f89c4050870 con 0x7f89dc0cf170 2024-09-13T18:26:27.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.097+0000 7f89db59e700 1 -- 172.21.15.146:0/392626858 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89dc066850 con 0x7f89dc0cf170 2024-09-13T18:26:27.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.101+0000 7f89d2ffd700 1 -- 172.21.15.146:0/392626858 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1259 (secure 0 0 0) 0x7f89c4017090 con 0x7f89dc0cf170 2024-09-13T18:26:27.106 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:27.106 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T18:26:27.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.101+0000 7f89db59e700 1 -- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f89bc0666a0 msgr2=0x7f89bc068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:27.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.101+0000 7f89db59e700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f89bc0666a0 0x7f89bc068b60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009bf0 tx=0x7f89cc009240 comp rx=0 tx=0).stop 2024-09-13T18:26:27.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.101+0000 7f89db59e700 1 -- 172.21.15.146:0/392626858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89dc0cf170 msgr2=0x7f89dc1e0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:27.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.101+0000 7f89db59e700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89dc0cf170 0x7f89dc1e0a70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f89c400c500 tx=0x7f89c400c810 comp rx=0 tx=0).stop 2024-09-13T18:26:27.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.101+0000 7f89db59e700 1 -- 172.21.15.146:0/392626858 shutdown_connections 2024-09-13T18:26:27.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.101+0000 7f89db59e700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f89bc0666a0 0x7f89bc068b60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:27.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.101+0000 7f89db59e700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc0cf550 0x7f89dc0b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:27.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.101+0000 7f89db59e700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89dc0cf170 0x7f89dc1e0a70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:27.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.101+0000 7f89db59e700 1 --2- 172.21.15.146:0/392626858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89dc0cb5b0 0x7f89dc1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:27.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.101+0000 7f89db59e700 1 -- 172.21.15.146:0/392626858 >> 172.21.15.146:0/392626858 conn(0x7f89dc072220 msgr2=0x7f89dc0c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:27.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.101+0000 7f89db59e700 1 -- 172.21.15.146:0/392626858 shutdown_connections 2024-09-13T18:26:27.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:27.101+0000 7f89db59e700 1 -- 172.21.15.146:0/392626858 wait complete. 2024-09-13T18:26:27.110 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 2 2024-09-13T18:26:29.121 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:26:29.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e821a9700 1 Processor -- start 2024-09-13T18:26:29.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e821a9700 1 -- start start 2024-09-13T18:26:29.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e821a9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8e7c0c6df0 0x7f8e7c0c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e821a9700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e7c0c7280 con 0x7f8e7c070570 2024-09-13T18:26:29.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e821a9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e7c0adfe0 con 0x7f8e7c157e10 2024-09-13T18:26:29.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e821a9700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e7c15e690 con 0x7f8e7c0c6df0 2024-09-13T18:26:29.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e72ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f8e7c070570 0x7f8e7c0c5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43962/0 (socket says 172.21.15.146:43962) 2024-09-13T18:26:29.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e72ffd700 1 -- 172.21.15.146:0/2374186181 learned_addr learned my addr 172.21.15.146:0/2374186181 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:29.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7bfff700 1 --2- 172.21.15.146:0/2374186181 >> v2:172.21.15.146:3300/0 conn(0x7f8e7c0c6df0 0x7f8e7c0c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221138923 0 0) 0x7f8e7c0c7280 con 0x7f8e7c070570 2024-09-13T18:26:29.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e680035d0 con 0x7f8e7c070570 2024-09-13T18:26:29.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2276209983 0 0) 0x7f8e680035d0 con 0x7f8e7c070570 2024-09-13T18:26:29.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 >> v2:172.21.15.146:3300/0 conn(0x7f8e7c0c6df0 msgr2=0x7f8e7c0c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 --2- 172.21.15.146:0/2374186181 >> v2:172.21.15.146:3300/0 conn(0x7f8e7c0c6df0 0x7f8e7c0c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 >> v1:172.21.15.154:6789/0 conn(0x7f8e7c157e10 legacy=0x7f8e7c06fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c0c7570 con 0x7f8e7c070570 2024-09-13T18:26:29.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f8e64001670 con 0x7f8e7c070570 2024-09-13T18:26:29.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 >> v1:172.21.15.146:6789/0 conn(0x7f8e7c070570 legacy=0x7f8e7c0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 --2- 172.21.15.146:0/2374186181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e68003a20 0x7f8e68003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 --2- 172.21.15.146:0/2374186181 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e68004420 0x7f8e680048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 --2- 172.21.15.146:0/2374186181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e680050f0 0x7f8e6800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c15e690 con 0x7f8e680050f0 2024-09-13T18:26:29.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6800c050 con 0x7f8e68003a20 2024-09-13T18:26:29.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6800c210 con 0x7f8e68004420 2024-09-13T18:26:29.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7b7fe700 1 --2- 172.21.15.146:0/2374186181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e68003a20 0x7f8e68003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7bfff700 1 --2- 172.21.15.146:0/2374186181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e680050f0 0x7f8e6800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e72ffd700 1 --2- 172.21.15.146:0/2374186181 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e68004420 0x7f8e680048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7b7fe700 1 -- 172.21.15.146:0/2374186181 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e68004420 msgr2=0x7f8e680048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7b7fe700 1 --2- 172.21.15.146:0/2374186181 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e68004420 0x7f8e680048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7b7fe700 1 -- 172.21.15.146:0/2374186181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e680050f0 msgr2=0x7f8e6800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7b7fe700 1 --2- 172.21.15.146:0/2374186181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e680050f0 0x7f8e6800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7b7fe700 1 -- 172.21.15.146:0/2374186181 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e680035d0 con 0x7f8e68003a20 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7bfff700 1 --2- 172.21.15.146:0/2374186181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e680050f0 0x7f8e6800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7bfff700 1 -- 172.21.15.146:0/2374186181 reap_dead start 2024-09-13T18:26:29.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7b7fe700 1 --2- 172.21.15.146:0/2374186181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e68003a20 0x7f8e68003e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0019a0 tx=0x7f8e680058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d23586f227a1547 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.237+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c001440 con 0x7f8e68003a20 2024-09-13T18:26:29.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.241+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6c0052f0 con 0x7f8e68003a20 2024-09-13T18:26:29.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.241+0000 7f8e7affd700 1 -- 172.21.15.146:0/2374186181 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c0036a0 con 0x7f8e68003a20 2024-09-13T18:26:29.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.241+0000 7f8e821a9700 1 -- 172.21.15.146:0/2374186181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e68003a20 msgr2=0x7f8e68003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.241+0000 7f8e821a9700 1 --2- 172.21.15.146:0/2374186181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e68003a20 0x7f8e68003e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0019a0 tx=0x7f8e680058a0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.241+0000 7f8e821a9700 1 -- 172.21.15.146:0/2374186181 shutdown_connections 2024-09-13T18:26:29.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.241+0000 7f8e821a9700 1 --2- 172.21.15.146:0/2374186181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e68003a20 0x7f8e68003e00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.241+0000 7f8e821a9700 1 -- 172.21.15.146:0/2374186181 >> 172.21.15.146:0/2374186181 conn(0x7f8e7c0cb1d0 msgr2=0x7f8e7c0cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.241+0000 7f8e821a9700 1 -- 172.21.15.146:0/2374186181 shutdown_connections 2024-09-13T18:26:29.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.241+0000 7f8e821a9700 1 -- 172.21.15.146:0/2374186181 wait complete. 2024-09-13T18:26:29.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.241+0000 7f8e821a9700 1 Processor -- start 2024-09-13T18:26:29.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.241+0000 7f8e821a9700 1 -- start start 2024-09-13T18:26:29.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e821a9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e68003a20 0x7f8e7c1f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e821a9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e7c151130 0x7f8e7c15a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e821a9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e7c15a840 0x7f8e7c152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e821a9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c15acc0 con 0x7f8e7c151130 2024-09-13T18:26:29.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e821a9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c152390 con 0x7f8e7c15a840 2024-09-13T18:26:29.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e821a9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c151690 con 0x7f8e68003a20 2024-09-13T18:26:29.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e72ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e7c151130 0x7f8e7c15a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e72ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e7c151130 0x7f8e7c15a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60466/0 (socket says 172.21.15.146:60466) 2024-09-13T18:26:29.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e72ffd700 1 -- 172.21.15.146:0/3783887243 learned_addr learned my addr 172.21.15.146:0/3783887243 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:29.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e7b7fe700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e68003a20 0x7f8e7c1f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e7bfff700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e7c15a840 0x7f8e7c152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e72ffd700 1 -- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e68003a20 msgr2=0x7f8e7c1f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e72ffd700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e68003a20 0x7f8e7c1f3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e72ffd700 1 -- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e7c15a840 msgr2=0x7f8e7c152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e72ffd700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e7c15a840 0x7f8e7c152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e72ffd700 1 -- 172.21.15.146:0/3783887243 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c151970 con 0x7f8e7c151130 2024-09-13T18:26:29.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e72ffd700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e7c151130 0x7f8e7c15a300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e640072c0 tx=0x7f8e640075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e78ff9700 1 -- 172.21.15.146:0/3783887243 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64011580 con 0x7f8e7c151130 2024-09-13T18:26:29.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e78ff9700 1 -- 172.21.15.146:0/3783887243 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e64011700 con 0x7f8e7c151130 2024-09-13T18:26:29.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e821a9700 1 -- 172.21.15.146:0/3783887243 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7c151c50 con 0x7f8e7c151130 2024-09-13T18:26:29.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e78ff9700 1 -- 172.21.15.146:0/3783887243 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64011ab0 con 0x7f8e7c151130 2024-09-13T18:26:29.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e821a9700 1 -- 172.21.15.146:0/3783887243 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7c1525e0 con 0x7f8e7c151130 2024-09-13T18:26:29.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.245+0000 7f8e821a9700 1 -- 172.21.15.146:0/3783887243 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7c06f940 con 0x7f8e7c151130 2024-09-13T18:26:29.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.249+0000 7f8e78ff9700 1 -- 172.21.15.146:0/3783887243 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f8e64011c10 con 0x7f8e7c151130 2024-09-13T18:26:29.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.249+0000 7f8e78ff9700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e5c0666a0 0x7f8e5c068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.249+0000 7f8e78ff9700 1 -- 172.21.15.146:0/3783887243 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8e640846c0 con 0x7f8e7c151130 2024-09-13T18:26:29.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.249+0000 7f8e7b7fe700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e5c0666a0 0x7f8e5c068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.249+0000 7f8e7b7fe700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e5c0666a0 0x7f8e5c068b60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c015040 tx=0x7f8e6c00a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.253+0000 7f8e78ff9700 1 -- 172.21.15.146:0/3783887243 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8e64051ef0 con 0x7f8e7c151130 2024-09-13T18:26:29.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.461+0000 7f8e821a9700 1 -- 172.21.15.146:0/3783887243 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e7c0c1f70 con 0x7f8e7c151130 2024-09-13T18:26:29.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.461+0000 7f8e78ff9700 1 -- 172.21.15.146:0/3783887243 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5045 (secure 0 0 0) 0x7f8e64055510 con 0x7f8e7c151130 2024-09-13T18:26:29.469 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:29.469 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T18:26:29.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.465+0000 7f8e821a9700 1 -- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e5c0666a0 msgr2=0x7f8e5c068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.465+0000 7f8e821a9700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e5c0666a0 0x7f8e5c068b60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c015040 tx=0x7f8e6c00a000 comp rx=0 tx=0).stop 2024-09-13T18:26:29.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.465+0000 7f8e821a9700 1 -- 172.21.15.146:0/3783887243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e7c151130 msgr2=0x7f8e7c15a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.465+0000 7f8e821a9700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e7c151130 0x7f8e7c15a300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e640072c0 tx=0x7f8e640075d0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.465+0000 7f8e821a9700 1 -- 172.21.15.146:0/3783887243 shutdown_connections 2024-09-13T18:26:29.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.465+0000 7f8e821a9700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e5c0666a0 0x7f8e5c068b60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.465+0000 7f8e821a9700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e7c15a840 0x7f8e7c152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.465+0000 7f8e821a9700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e7c151130 0x7f8e7c15a300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.465+0000 7f8e821a9700 1 --2- 172.21.15.146:0/3783887243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e68003a20 0x7f8e7c1f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.465+0000 7f8e821a9700 1 -- 172.21.15.146:0/3783887243 >> 172.21.15.146:0/3783887243 conn(0x7f8e7c0cb1d0 msgr2=0x7f8e7c0ca1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.465+0000 7f8e821a9700 1 -- 172.21.15.146:0/3783887243 shutdown_connections 2024-09-13T18:26:29.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.465+0000 7f8e821a9700 1 -- 172.21.15.146:0/3783887243 wait complete. 2024-09-13T18:26:29.473 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 3 2024-09-13T18:26:29.484 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:26:29.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.585+0000 7f945bfff700 1 Processor -- start 2024-09-13T18:26:29.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.585+0000 7f945bfff700 1 -- start start 2024-09-13T18:26:29.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f945c0ca210 0x7f945c0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f945c071390 0x7f945c0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945bfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945c0cc670 con 0x7f945c0cbd50 2024-09-13T18:26:29.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945bfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945c0caa30 con 0x7f945c0ca210 2024-09-13T18:26:29.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945c0cb810 con 0x7f945c071390 2024-09-13T18:26:29.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f94527fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f945c0ca210 0x7f945c0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945affd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f945c0cbd50 0x7f945c0701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43966/0 (socket says 172.21.15.146:43966) 2024-09-13T18:26:29.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f94527fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f945c0ca210 0x7f945c0b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60468/0 (socket says 172.21.15.146:60468) 2024-09-13T18:26:29.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945affd700 1 -- 172.21.15.146:0/263856933 learned_addr learned my addr 172.21.15.146:0/263856933 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:29.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945b7fe700 1 --2- 172.21.15.146:0/263856933 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f945c071390 0x7f945c0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f94527fc700 1 -- 172.21.15.146:0/263856933 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f945c071390 msgr2=0x7f945c0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f94527fc700 1 --2- 172.21.15.146:0/263856933 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f945c071390 0x7f945c0cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945a7fc700 1 -- 172.21.15.146:0/263856933 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 168408353 0 0) 0x7f944c001110 con 0x7f945c0cbd50 2024-09-13T18:26:29.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f94527fc700 1 -- 172.21.15.146:0/263856933 >> v1:172.21.15.146:6789/0 conn(0x7f945c0cbd50 legacy=0x7f945c0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f94527fc700 1 -- 172.21.15.146:0/263856933 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c15da20 con 0x7f945c0ca210 2024-09-13T18:26:29.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f94527fc700 1 --2- 172.21.15.146:0/263856933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f945c0ca210 0x7f945c0b1c40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9444009850 tx=0x7f9444009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45fc49b454d7fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945a7fc700 1 -- 172.21.15.146:0/263856933 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444011830 con 0x7f945c0ca210 2024-09-13T18:26:29.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945a7fc700 1 -- 172.21.15.146:0/263856933 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9444011990 con 0x7f945c0ca210 2024-09-13T18:26:29.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945a7fc700 1 -- 172.21.15.146:0/263856933 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444011d40 con 0x7f945c0ca210 2024-09-13T18:26:29.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945bfff700 1 -- 172.21.15.146:0/263856933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f945c0ca210 msgr2=0x7f945c0b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945bfff700 1 --2- 172.21.15.146:0/263856933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f945c0ca210 0x7f945c0b1c40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9444009850 tx=0x7f9444009b60 comp rx=0 tx=0).stop 2024-09-13T18:26:29.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945bfff700 1 -- 172.21.15.146:0/263856933 shutdown_connections 2024-09-13T18:26:29.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945bfff700 1 --2- 172.21.15.146:0/263856933 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f945c071390 0x7f945c0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945bfff700 1 --2- 172.21.15.146:0/263856933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f945c0ca210 0x7f945c0b1c40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945bfff700 1 -- 172.21.15.146:0/263856933 >> 172.21.15.146:0/263856933 conn(0x7f945c066270 msgr2=0x7f945c066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945bfff700 1 -- 172.21.15.146:0/263856933 shutdown_connections 2024-09-13T18:26:29.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.589+0000 7f945bfff700 1 -- 172.21.15.146:0/263856933 wait complete. 2024-09-13T18:26:29.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945bfff700 1 Processor -- start 2024-09-13T18:26:29.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945bfff700 1 -- start start 2024-09-13T18:26:29.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f945c071390 0x7f945c15baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f945c0ca210 0x7f945c148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f945c0cbd50 0x7f945c1efa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945bfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c1490b0 con 0x7f945c0cbd50 2024-09-13T18:26:29.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945bfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c1f3b70 con 0x7f945c0ca210 2024-09-13T18:26:29.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945c0cd170 con 0x7f945c071390 2024-09-13T18:26:29.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f945c0cbd50 0x7f945c1efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f945c0cbd50 0x7f945c1efa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60476/0 (socket says 172.21.15.146:60476) 2024-09-13T18:26:29.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945b7fe700 1 -- 172.21.15.146:0/2302983789 learned_addr learned my addr 172.21.15.146:0/2302983789 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:29.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945affd700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f945c071390 0x7f945c15baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f94527fc700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f945c0ca210 0x7f945c148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945b7fe700 1 -- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f945c071390 msgr2=0x7f945c15baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945b7fe700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f945c071390 0x7f945c15baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945b7fe700 1 -- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f945c0ca210 msgr2=0x7f945c148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945b7fe700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f945c0ca210 0x7f945c148670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945b7fe700 1 -- 172.21.15.146:0/2302983789 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94440094b0 con 0x7f945c0cbd50 2024-09-13T18:26:29.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f94527fc700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f945c0ca210 0x7f945c148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:26:29.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f945b7fe700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f945c0cbd50 0x7f945c1efa70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f945400efc0 tx=0x7f945400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.593+0000 7f9453fff700 1 -- 172.21.15.146:0/2302983789 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945400cd40 con 0x7f945c0cbd50 2024-09-13T18:26:29.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.597+0000 7f9453fff700 1 -- 172.21.15.146:0/2302983789 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945400f040 con 0x7f945c0cbd50 2024-09-13T18:26:29.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.597+0000 7f945bfff700 1 -- 172.21.15.146:0/2302983789 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945c1f2f60 con 0x7f945c0cbd50 2024-09-13T18:26:29.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.597+0000 7f9453fff700 1 -- 172.21.15.146:0/2302983789 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94540146c0 con 0x7f945c0cbd50 2024-09-13T18:26:29.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.597+0000 7f945bfff700 1 -- 172.21.15.146:0/2302983789 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945c1f3520 con 0x7f945c0cbd50 2024-09-13T18:26:29.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.597+0000 7f945bfff700 1 -- 172.21.15.146:0/2302983789 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945c15d500 con 0x7f945c0cbd50 2024-09-13T18:26:29.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.597+0000 7f9453fff700 1 -- 172.21.15.146:0/2302983789 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f9454014820 con 0x7f945c0cbd50 2024-09-13T18:26:29.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.597+0000 7f9453fff700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f943c0666a0 0x7f943c068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.601+0000 7f9453fff700 1 -- 172.21.15.146:0/2302983789 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9454084a40 con 0x7f945c0cbd50 2024-09-13T18:26:29.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.601+0000 7f945affd700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f943c0666a0 0x7f943c068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.601+0000 7f945affd700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f943c0666a0 0x7f943c068b60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f944c000f60 tx=0x7f944c007cb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.613+0000 7f9453fff700 1 -- 172.21.15.146:0/2302983789 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9454052150 con 0x7f945c0cbd50 2024-09-13T18:26:29.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.621+0000 7f9453fff700 1 -- 172.21.15.146:0/2302983789 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9454052520 con 0x7f945c0cbd50 2024-09-13T18:26:29.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.781+0000 7f945bfff700 1 -- 172.21.15.146:0/2302983789 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f945c15a420 con 0x7f945c0cbd50 2024-09-13T18:26:29.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.781+0000 7f9453fff700 1 -- 172.21.15.146:0/2302983789 <== mon.0 v2:172.21.15.146:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12899 (secure 0 0 0) 0x7f9454081ce0 con 0x7f945c0cbd50 2024-09-13T18:26:29.789 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:29.789 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":17,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:27.912268+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:26:29.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.785+0000 7f945bfff700 1 -- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f943c0666a0 msgr2=0x7f943c068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.785+0000 7f945bfff700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f943c0666a0 0x7f943c068b60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f944c000f60 tx=0x7f944c007cb0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.785+0000 7f945bfff700 1 -- 172.21.15.146:0/2302983789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f945c0cbd50 msgr2=0x7f945c1efa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.785+0000 7f945bfff700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f945c0cbd50 0x7f945c1efa70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f945400efc0 tx=0x7f945400c3a0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.785+0000 7f945bfff700 1 -- 172.21.15.146:0/2302983789 shutdown_connections 2024-09-13T18:26:29.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.785+0000 7f945bfff700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f943c0666a0 0x7f943c068b60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.785+0000 7f945bfff700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f945c0cbd50 0x7f945c1efa70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.785+0000 7f945bfff700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f945c0ca210 0x7f945c148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.785+0000 7f945bfff700 1 --2- 172.21.15.146:0/2302983789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f945c071390 0x7f945c15baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.785+0000 7f945bfff700 1 -- 172.21.15.146:0/2302983789 >> 172.21.15.146:0/2302983789 conn(0x7f945c066270 msgr2=0x7f945c066650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.785+0000 7f945bfff700 1 -- 172.21.15.146:0/2302983789 shutdown_connections 2024-09-13T18:26:29.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.785+0000 7f945bfff700 1 -- 172.21.15.146:0/2302983789 wait complete. 2024-09-13T18:26:29.804 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:26:24.867515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:26:29.804 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:26:29.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fb1047700 1 Processor -- start 2024-09-13T18:26:29.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fb1047700 1 -- start start 2024-09-13T18:26:29.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fb1047700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1fac151740 0x7f1fac0c2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fb1047700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac158070 0x7f1fac15a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fb1047700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fac0c26b0 con 0x7f1fac15aeb0 2024-09-13T18:26:29.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fb1047700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac157c50 con 0x7f1fac151740 2024-09-13T18:26:29.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fb1047700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0c3ba0 con 0x7f1fac158070 2024-09-13T18:26:29.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fabfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1fac151740 0x7f1fac0c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fabfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1fac151740 0x7f1fac0c2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60482/0 (socket says 172.21.15.146:60482) 2024-09-13T18:26:29.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fabfff700 1 -- 172.21.15.146:0/3036492689 learned_addr learned my addr 172.21.15.146:0/3036492689 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:29.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fab7fe700 1 --2- 172.21.15.146:0/3036492689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac158070 0x7f1fac15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fabfff700 1 -- 172.21.15.146:0/3036492689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac158070 msgr2=0x7f1fac15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fabfff700 1 --2- 172.21.15.146:0/3036492689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac158070 0x7f1fac15a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fabfff700 1 -- 172.21.15.146:0/3036492689 >> v1:172.21.15.154:6789/0 conn(0x7f1fac15aeb0 legacy=0x7f1fac0c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fabfff700 1 -- 172.21.15.146:0/3036492689 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac0c8a70 con 0x7f1fac151740 2024-09-13T18:26:29.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fabfff700 1 --2- 172.21.15.146:0/3036492689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fac151740 0x7f1fac0c2170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009910 tx=0x7f1fa0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a270f19040453770 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1faaffd700 1 -- 172.21.15.146:0/3036492689 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa000ba60 con 0x7f1fac151740 2024-09-13T18:26:29.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1faaffd700 1 -- 172.21.15.146:0/3036492689 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa000bbc0 con 0x7f1fac151740 2024-09-13T18:26:29.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1faaffd700 1 -- 172.21.15.146:0/3036492689 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0009e60 con 0x7f1fac151740 2024-09-13T18:26:29.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fb1047700 1 -- 172.21.15.146:0/3036492689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fac151740 msgr2=0x7f1fac0c2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.921+0000 7f1fb1047700 1 --2- 172.21.15.146:0/3036492689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fac151740 0x7f1fac0c2170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009910 tx=0x7f1fa0009c20 comp rx=0 tx=0).stop 2024-09-13T18:26:29.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 -- 172.21.15.146:0/3036492689 shutdown_connections 2024-09-13T18:26:29.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 --2- 172.21.15.146:0/3036492689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac158070 0x7f1fac15a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 --2- 172.21.15.146:0/3036492689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fac151740 0x7f1fac0c2170 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 -- 172.21.15.146:0/3036492689 >> 172.21.15.146:0/3036492689 conn(0x7f1fac06f5c0 msgr2=0x7f1fac06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 -- 172.21.15.146:0/3036492689 shutdown_connections 2024-09-13T18:26:29.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 -- 172.21.15.146:0/3036492689 wait complete. 2024-09-13T18:26:29.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 Processor -- start 2024-09-13T18:26:29.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 -- start start 2024-09-13T18:26:29.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac151740 0x7f1fac0cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fac158070 0x7f1fac1e6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fac15aeb0 0x7f1fac0718d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac071160 con 0x7f1fac158070 2024-09-13T18:26:29.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0cc4e0 con 0x7f1fac15aeb0 2024-09-13T18:26:29.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb1047700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac070ba0 con 0x7f1fac151740 2024-09-13T18:26:29.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fab7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fac158070 0x7f1fac1e6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fabfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac151740 0x7f1fac0cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fb0846700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fac15aeb0 0x7f1fac0718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fabfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac151740 0x7f1fac0cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:45766/0 (socket says 172.21.15.146:45766) 2024-09-13T18:26:29.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fabfff700 1 -- 172.21.15.146:0/1271925659 learned_addr learned my addr 172.21.15.146:0/1271925659 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:29.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fab7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fac158070 0x7f1fac1e6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60498/0 (socket says 172.21.15.146:60498) 2024-09-13T18:26:29.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fab7fe700 1 -- 172.21.15.146:0/1271925659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac151740 msgr2=0x7f1fac0cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fab7fe700 1 --2- 172.21.15.146:0/1271925659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac151740 0x7f1fac0cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fab7fe700 1 -- 172.21.15.146:0/1271925659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fac15aeb0 msgr2=0x7f1fac0718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fab7fe700 1 --2- 172.21.15.146:0/1271925659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fac15aeb0 0x7f1fac0718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.925+0000 7f1fab7fe700 1 -- 172.21.15.146:0/1271925659 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa0009570 con 0x7f1fac158070 2024-09-13T18:26:29.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.929+0000 7f1fab7fe700 1 --2- 172.21.15.146:0/1271925659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fac158070 0x7f1fac1e6010 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00e810 tx=0x7f1f9c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.929+0000 7f1fa8ff9700 1 -- 172.21.15.146:0/1271925659 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c00c070 con 0x7f1fac158070 2024-09-13T18:26:29.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.929+0000 7f1fa8ff9700 1 -- 172.21.15.146:0/1271925659 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c013610 con 0x7f1fac158070 2024-09-13T18:26:29.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.929+0000 7f1fb1047700 1 -- 172.21.15.146:0/1271925659 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fac070e80 con 0x7f1fac158070 2024-09-13T18:26:29.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.929+0000 7f1fa8ff9700 1 -- 172.21.15.146:0/1271925659 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0139c0 con 0x7f1fac158070 2024-09-13T18:26:29.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.929+0000 7f1fb1047700 1 -- 172.21.15.146:0/1271925659 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fac156ab0 con 0x7f1fac158070 2024-09-13T18:26:29.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.929+0000 7f1fb1047700 1 -- 172.21.15.146:0/1271925659 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fac156cd0 con 0x7f1fac158070 2024-09-13T18:26:29.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.929+0000 7f1fa8ff9700 1 -- 172.21.15.146:0/1271925659 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f9c013b20 con 0x7f1fac158070 2024-09-13T18:26:29.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.929+0000 7f1fa8ff9700 1 --2- 172.21.15.146:0/1271925659 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1f98066730 0x7f1f98068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.933+0000 7f1fa8ff9700 1 -- 172.21.15.146:0/1271925659 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1f9c083b20 con 0x7f1fac158070 2024-09-13T18:26:29.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.933+0000 7f1fabfff700 1 --2- 172.21.15.146:0/1271925659 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1f98066730 0x7f1f98068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.933+0000 7f1fabfff700 1 --2- 172.21.15.146:0/1271925659 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1f98066730 0x7f1f98068bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009150 tx=0x7f1fa0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:29.937+0000 7f1fa8ff9700 1 -- 172.21.15.146:0/1271925659 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1f9c0511e0 con 0x7f1fac158070 2024-09-13T18:26:30.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.105+0000 7f1fb1047700 1 -- 172.21.15.146:0/1271925659 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1fac06c0b0 con 0x7f1fac158070 2024-09-13T18:26:30.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.105+0000 7f1fa8ff9700 1 -- 172.21.15.146:0/1271925659 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f9c054800 con 0x7f1fac158070 2024-09-13T18:26:30.113 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-09-13T18:26:30.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.109+0000 7f1fb1047700 1 -- 172.21.15.146:0/1271925659 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1f98066730 msgr2=0x7f1f98068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.109+0000 7f1fb1047700 1 --2- 172.21.15.146:0/1271925659 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1f98066730 0x7f1f98068bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009150 tx=0x7f1fa0008030 comp rx=0 tx=0).stop 2024-09-13T18:26:30.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.109+0000 7f1fb1047700 1 -- 172.21.15.146:0/1271925659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fac158070 msgr2=0x7f1fac1e6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.109+0000 7f1fb1047700 1 --2- 172.21.15.146:0/1271925659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fac158070 0x7f1fac1e6010 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00e810 tx=0x7f1f9c00eb20 comp rx=0 tx=0).stop 2024-09-13T18:26:30.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.109+0000 7f1fb1047700 1 -- 172.21.15.146:0/1271925659 shutdown_connections 2024-09-13T18:26:30.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.109+0000 7f1fb1047700 1 --2- 172.21.15.146:0/1271925659 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1f98066730 0x7f1f98068bf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.109+0000 7f1fb1047700 1 --2- 172.21.15.146:0/1271925659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fac15aeb0 0x7f1fac0718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.109+0000 7f1fb1047700 1 --2- 172.21.15.146:0/1271925659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fac158070 0x7f1fac1e6010 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.109+0000 7f1fb1047700 1 --2- 172.21.15.146:0/1271925659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac151740 0x7f1fac0cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.109+0000 7f1fb1047700 1 -- 172.21.15.146:0/1271925659 >> 172.21.15.146:0/1271925659 conn(0x7f1fac06f5c0 msgr2=0x7f1fac073c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:30.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.109+0000 7f1fb1047700 1 -- 172.21.15.146:0/1271925659 shutdown_connections 2024-09-13T18:26:30.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.109+0000 7f1fb1047700 1 -- 172.21.15.146:0/1271925659 wait complete. 2024-09-13T18:26:30.128 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T18:26:30.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.229+0000 7f91e359e700 1 Processor -- start 2024-09-13T18:26:30.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.229+0000 7f91e359e700 1 -- start start 2024-09-13T18:26:30.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.229+0000 7f91e359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e4153a60 0x7f91e4159700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.229+0000 7f91e359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f91e40c9ed0 0x7f91e40b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.229+0000 7f91e359e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e40cc2d0 con 0x7f91e40cf540 2024-09-13T18:26:30.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.229+0000 7f91e359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e4153e40 con 0x7f91e40c9ed0 2024-09-13T18:26:30.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.229+0000 7f91e359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e40cb3e0 con 0x7f91e4153a60 2024-09-13T18:26:30.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e1d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f91e40c9ed0 0x7f91e40b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e1d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f91e40c9ed0 0x7f91e40b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60510/0 (socket says 172.21.15.146:60510) 2024-09-13T18:26:30.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e1d9b700 1 -- 172.21.15.146:0/2476451285 learned_addr learned my addr 172.21.15.146:0/2476451285 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:30.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e259c700 1 --2- 172.21.15.146:0/2476451285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e4153a60 0x7f91e4159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e1d9b700 1 -- 172.21.15.146:0/2476451285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e4153a60 msgr2=0x7f91e4159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:30.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e1d9b700 1 --2- 172.21.15.146:0/2476451285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e4153a60 0x7f91e4159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e1d9b700 1 -- 172.21.15.146:0/2476451285 >> v1:172.21.15.146:6789/0 conn(0x7f91e40cf540 legacy=0x7f91e415cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e1d9b700 1 -- 172.21.15.146:0/2476451285 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e40cb6c0 con 0x7f91e40c9ed0 2024-09-13T18:26:30.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e259c700 1 --2- 172.21.15.146:0/2476451285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e4153a60 0x7f91e4159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:26:30.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e1d9b700 1 --2- 172.21.15.146:0/2476451285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91e40c9ed0 0x7f91e40b3280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009910 tx=0x7f91cc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e4d89366fd8d56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:30.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e159a700 1 -- 172.21.15.146:0/2476451285 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00ba60 con 0x7f91e40c9ed0 2024-09-13T18:26:30.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e159a700 1 -- 172.21.15.146:0/2476451285 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc00bbc0 con 0x7f91e40c9ed0 2024-09-13T18:26:30.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e159a700 1 -- 172.21.15.146:0/2476451285 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc009e60 con 0x7f91e40c9ed0 2024-09-13T18:26:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 -- 172.21.15.146:0/2476451285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91e40c9ed0 msgr2=0x7f91e40b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 --2- 172.21.15.146:0/2476451285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91e40c9ed0 0x7f91e40b3280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009910 tx=0x7f91cc009c20 comp rx=0 tx=0).stop 2024-09-13T18:26:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 -- 172.21.15.146:0/2476451285 shutdown_connections 2024-09-13T18:26:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 --2- 172.21.15.146:0/2476451285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91e40c9ed0 0x7f91e40b3280 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 --2- 172.21.15.146:0/2476451285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e4153a60 0x7f91e4159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 -- 172.21.15.146:0/2476451285 >> 172.21.15.146:0/2476451285 conn(0x7f91e4071ec0 msgr2=0x7f91e40722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 -- 172.21.15.146:0/2476451285 shutdown_connections 2024-09-13T18:26:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 -- 172.21.15.146:0/2476451285 wait complete. 2024-09-13T18:26:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 Processor -- start 2024-09-13T18:26:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 -- start start 2024-09-13T18:26:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e40cf540 0x7f91e40c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91e4153a60 0x7f91e40c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e414a610 0x7f91e40c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e414c720 con 0x7f91e4153a60 2024-09-13T18:26:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e40c92f0 con 0x7f91e40cf540 2024-09-13T18:26:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.233+0000 7f91e359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e4150200 con 0x7f91e414a610 2024-09-13T18:26:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91e4153a60 0x7f91e40c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91e4153a60 0x7f91e40c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60522/0 (socket says 172.21.15.146:60522) 2024-09-13T18:26:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e1d9b700 1 -- 172.21.15.146:0/76160928 learned_addr learned my addr 172.21.15.146:0/76160928 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e2d9d700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e414a610 0x7f91e40c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e259c700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e40cf540 0x7f91e40c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e1d9b700 1 -- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e414a610 msgr2=0x7f91e40c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e1d9b700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e414a610 0x7f91e40c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e1d9b700 1 -- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e40cf540 msgr2=0x7f91e40c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e1d9b700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e40cf540 0x7f91e40c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e1d9b700 1 -- 172.21.15.146:0/76160928 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc009570 con 0x7f91e4153a60 2024-09-13T18:26:30.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e1d9b700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91e4153a60 0x7f91e40c8bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f91cc000c00 tx=0x7f91cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:30.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91daffd700 1 -- 172.21.15.146:0/76160928 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc004310 con 0x7f91e4153a60 2024-09-13T18:26:30.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91daffd700 1 -- 172.21.15.146:0/76160928 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc01f070 con 0x7f91e4153a60 2024-09-13T18:26:30.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e359e700 1 -- 172.21.15.146:0/76160928 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e4149cb0 con 0x7f91e4153a60 2024-09-13T18:26:30.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91daffd700 1 -- 172.21.15.146:0/76160928 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0036c0 con 0x7f91e4153a60 2024-09-13T18:26:30.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e359e700 1 -- 172.21.15.146:0/76160928 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e414c0d0 con 0x7f91e4153a60 2024-09-13T18:26:30.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e359e700 1 -- 172.21.15.146:0/76160928 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e40ce010 con 0x7f91e4153a60 2024-09-13T18:26:30.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91daffd700 1 -- 172.21.15.146:0/76160928 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f91cc021030 con 0x7f91e4153a60 2024-09-13T18:26:30.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91daffd700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f91c4066730 0x7f91c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91daffd700 1 -- 172.21.15.146:0/76160928 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f91cc085640 con 0x7f91e4153a60 2024-09-13T18:26:30.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.237+0000 7f91e259c700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f91c4066730 0x7f91c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.241+0000 7f91e259c700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f91c4066730 0x7f91c4068bf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f91d400a9a0 tx=0x7f91d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:30.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.249+0000 7f91daffd700 1 -- 172.21.15.146:0/76160928 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f91cc052d90 con 0x7f91e4153a60 2024-09-13T18:26:30.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.421+0000 7f91e359e700 1 -- 172.21.15.146:0/76160928 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f91e40cc760 con 0x7f91e4153a60 2024-09-13T18:26:30.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.421+0000 7f91daffd700 1 -- 172.21.15.146:0/76160928 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f91cc015020 con 0x7f91e4153a60 2024-09-13T18:26:30.427 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:30.427 INFO:teuthology.orchestra.run.smithi146.stdout:[ 2024-09-13T18:26:30.427 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-13T18:26:30.427 INFO:teuthology.orchestra.run.smithi146.stdout: "poolnum": 1, 2024-09-13T18:26:30.427 INFO:teuthology.orchestra.run.smithi146.stdout: "poolname": ".mgr" 2024-09-13T18:26:30.427 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-13T18:26:30.427 INFO:teuthology.orchestra.run.smithi146.stdout:] 2024-09-13T18:26:30.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.421+0000 7f91e359e700 1 -- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f91c4066730 msgr2=0x7f91c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.421+0000 7f91e359e700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f91c4066730 0x7f91c4068bf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f91d400a9a0 tx=0x7f91d4008040 comp rx=0 tx=0).stop 2024-09-13T18:26:30.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.421+0000 7f91e359e700 1 -- 172.21.15.146:0/76160928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91e4153a60 msgr2=0x7f91e40c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.421+0000 7f91e359e700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91e4153a60 0x7f91e40c8bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f91cc000c00 tx=0x7f91cc004770 comp rx=0 tx=0).stop 2024-09-13T18:26:30.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.421+0000 7f91e359e700 1 -- 172.21.15.146:0/76160928 shutdown_connections 2024-09-13T18:26:30.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.421+0000 7f91e359e700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f91c4066730 0x7f91c4068bf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.421+0000 7f91e359e700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e414a610 0x7f91e40c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.421+0000 7f91e359e700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91e4153a60 0x7f91e40c8bf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.421+0000 7f91e359e700 1 --2- 172.21.15.146:0/76160928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e40cf540 0x7f91e40c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.425+0000 7f91e359e700 1 -- 172.21.15.146:0/76160928 >> 172.21.15.146:0/76160928 conn(0x7f91e4071ec0 msgr2=0x7f91e40b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:30.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.425+0000 7f91e359e700 1 -- 172.21.15.146:0/76160928 shutdown_connections 2024-09-13T18:26:30.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.425+0000 7f91e359e700 1 -- 172.21.15.146:0/76160928 wait complete. 2024-09-13T18:26:30.441 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T18:26:30.441 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T18:26:30.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.557+0000 7f8881ffb700 1 Processor -- start 2024-09-13T18:26:30.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.557+0000 7f8881ffb700 1 -- start start 2024-09-13T18:26:30.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8881ffb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f887c0c26d0 0x7f887c159820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8881ffb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f887c0c2ab0 0x7f887c071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8881ffb700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c0715b0 con 0x7f887c1562b0 2024-09-13T18:26:30.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8881ffb700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c0c1bb0 con 0x7f887c0c2ab0 2024-09-13T18:26:30.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8881ffb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c15a750 con 0x7f887c0c26d0 2024-09-13T18:26:30.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f887bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f887c0c2ab0 0x7f887c071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f887bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f887c0c2ab0 0x7f887c071070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60538/0 (socket says 172.21.15.146:60538) 2024-09-13T18:26:30.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f887bfff700 1 -- 172.21.15.146:0/731445213 learned_addr learned my addr 172.21.15.146:0/731445213 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:30.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8880ff9700 1 --2- 172.21.15.146:0/731445213 >> v2:172.21.15.154:3300/0 conn(0x7f887c0c26d0 0x7f887c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f887bfff700 1 -- 172.21.15.146:0/731445213 >> v2:172.21.15.154:3300/0 conn(0x7f887c0c26d0 msgr2=0x7f887c159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f887bfff700 1 --2- 172.21.15.146:0/731445213 >> v2:172.21.15.154:3300/0 conn(0x7f887c0c26d0 0x7f887c159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f887bfff700 1 -- 172.21.15.146:0/731445213 >> v1:172.21.15.154:6789/0 conn(0x7f887c1562b0 legacy=0x7f887c06dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f887bfff700 1 -- 172.21.15.146:0/731445213 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c159eb0 con 0x7f887c0c2ab0 2024-09-13T18:26:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8880ff9700 1 --2- 172.21.15.146:0/731445213 >> v2:172.21.15.154:3300/0 conn(0x7f887c0c26d0 0x7f887c159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f887bfff700 1 --2- 172.21.15.146:0/731445213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887c0c2ab0 0x7f887c071070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f887c0c89f0 tx=0x7f886c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755b3d371cb49785 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f887b7fe700 1 -- 172.21.15.146:0/731445213 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00dd90 con 0x7f887c0c2ab0 2024-09-13T18:26:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f887b7fe700 1 -- 172.21.15.146:0/731445213 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c010470 con 0x7f887c0c2ab0 2024-09-13T18:26:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f887b7fe700 1 -- 172.21.15.146:0/731445213 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c010800 con 0x7f887c0c2ab0 2024-09-13T18:26:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8881ffb700 1 -- 172.21.15.146:0/731445213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887c0c2ab0 msgr2=0x7f887c071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8881ffb700 1 --2- 172.21.15.146:0/731445213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887c0c2ab0 0x7f887c071070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f887c0c89f0 tx=0x7f886c00be90 comp rx=0 tx=0).stop 2024-09-13T18:26:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8881ffb700 1 -- 172.21.15.146:0/731445213 shutdown_connections 2024-09-13T18:26:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8881ffb700 1 --2- 172.21.15.146:0/731445213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887c0c2ab0 0x7f887c071070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8881ffb700 1 --2- 172.21.15.146:0/731445213 >> v2:172.21.15.154:3300/0 conn(0x7f887c0c26d0 0x7f887c159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8881ffb700 1 -- 172.21.15.146:0/731445213 >> 172.21.15.146:0/731445213 conn(0x7f887c06b220 msgr2=0x7f887c06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8881ffb700 1 -- 172.21.15.146:0/731445213 shutdown_connections 2024-09-13T18:26:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.561+0000 7f8881ffb700 1 -- 172.21.15.146:0/731445213 wait complete. 2024-09-13T18:26:30.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f8881ffb700 1 Processor -- start 2024-09-13T18:26:30.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f8881ffb700 1 -- start start 2024-09-13T18:26:30.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f8881ffb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f887c0c26d0 0x7f887c0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f8881ffb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f887c0c2ab0 0x7f887c1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f8881ffb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887c1562b0 0x7f887c157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f8881ffb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0c8260 con 0x7f887c1562b0 2024-09-13T18:26:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f8881ffb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c145090 con 0x7f887c0c26d0 2024-09-13T18:26:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f8881ffb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0c7f10 con 0x7f887c0c2ab0 2024-09-13T18:26:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f88817fa700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887c1562b0 0x7f887c157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f88817fa700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887c1562b0 0x7f887c157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60540/0 (socket says 172.21.15.146:60540) 2024-09-13T18:26:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f88817fa700 1 -- 172.21.15.146:0/3972982134 learned_addr learned my addr 172.21.15.146:0/3972982134 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f8880ff9700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f887c0c26d0 0x7f887c0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f887bfff700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f887c0c2ab0 0x7f887c1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f88817fa700 1 -- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f887c0c2ab0 msgr2=0x7f887c1553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f88817fa700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f887c0c2ab0 0x7f887c1553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f88817fa700 1 -- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f887c0c26d0 msgr2=0x7f887c0b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f88817fa700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f887c0c26d0 0x7f887c0b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f88817fa700 1 -- 172.21.15.146:0/3972982134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c00b810 con 0x7f887c1562b0 2024-09-13T18:26:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f887bfff700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f887c0c2ab0 0x7f887c1553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f8880ff9700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f887c0c26d0 0x7f887c0b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f88817fa700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887c1562b0 0x7f887c157410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f887400cb80 tx=0x7f887400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f88797fa700 1 -- 172.21.15.146:0/3972982134 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874003c20 con 0x7f887c1562b0 2024-09-13T18:26:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f88797fa700 1 -- 172.21.15.146:0/3972982134 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8874003d80 con 0x7f887c1562b0 2024-09-13T18:26:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f8881ffb700 1 -- 172.21.15.146:0/3972982134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887c0c62e0 con 0x7f887c1562b0 2024-09-13T18:26:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f88797fa700 1 -- 172.21.15.146:0/3972982134 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400abb0 con 0x7f887c1562b0 2024-09-13T18:26:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.565+0000 7f8881ffb700 1 -- 172.21.15.146:0/3972982134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c1ebe50 con 0x7f887c1562b0 2024-09-13T18:26:30.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.569+0000 7f8881ffb700 1 -- 172.21.15.146:0/3972982134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c0c83a0 con 0x7f887c1562b0 2024-09-13T18:26:30.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.569+0000 7f88797fa700 1 -- 172.21.15.146:0/3972982134 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f887400ad10 con 0x7f887c1562b0 2024-09-13T18:26:30.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.569+0000 7f88797fa700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8868066730 0x7f8868068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.569+0000 7f88797fa700 1 -- 172.21.15.146:0/3972982134 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8874083ef0 con 0x7f887c1562b0 2024-09-13T18:26:30.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.569+0000 7f8880ff9700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8868066730 0x7f8868068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.569+0000 7f8880ff9700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8868066730 0x7f8868068bf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f88700037d0 tx=0x7f8870008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:30.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.573+0000 7f88797fa700 1 -- 172.21.15.146:0/3972982134 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f887c0c83a0 con 0x7f887c1562b0 2024-09-13T18:26:30.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.741+0000 7f8881ffb700 1 -- 172.21.15.146:0/3972982134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f887c0c83a0 con 0x7f887c1562b0 2024-09-13T18:26:30.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.941+0000 7f88797fa700 1 -- 172.21.15.146:0/3972982134 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f887c0c83a0 con 0x7f887c1562b0 2024-09-13T18:26:30.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.953+0000 7f8881ffb700 1 -- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8868066730 msgr2=0x7f8868068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.953+0000 7f8881ffb700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8868066730 0x7f8868068bf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f88700037d0 tx=0x7f8870008040 comp rx=0 tx=0).stop 2024-09-13T18:26:30.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.953+0000 7f8881ffb700 1 -- 172.21.15.146:0/3972982134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887c1562b0 msgr2=0x7f887c157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.953+0000 7f8881ffb700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887c1562b0 0x7f887c157410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f887400cb80 tx=0x7f887400cf40 comp rx=0 tx=0).stop 2024-09-13T18:26:30.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.953+0000 7f8881ffb700 1 -- 172.21.15.146:0/3972982134 shutdown_connections 2024-09-13T18:26:30.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.953+0000 7f8881ffb700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8868066730 0x7f8868068bf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.953+0000 7f8881ffb700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887c1562b0 0x7f887c157410 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.953+0000 7f8881ffb700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f887c0c2ab0 0x7f887c1553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.953+0000 7f8881ffb700 1 --2- 172.21.15.146:0/3972982134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f887c0c26d0 0x7f887c0b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.953+0000 7f8881ffb700 1 -- 172.21.15.146:0/3972982134 >> 172.21.15.146:0/3972982134 conn(0x7f887c06b220 msgr2=0x7f887c1496d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:30.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.953+0000 7f8881ffb700 1 -- 172.21.15.146:0/3972982134 shutdown_connections 2024-09-13T18:26:30.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:30.953+0000 7f8881ffb700 1 -- 172.21.15.146:0/3972982134 wait complete. 2024-09-13T18:26:30.960 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_metadata' created 2024-09-13T18:26:30.970 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-13T18:26:31.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.089+0000 7fae28a3f700 1 Processor -- start 2024-09-13T18:26:31.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.089+0000 7fae28a3f700 1 -- start start 2024-09-13T18:26:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.089+0000 7fae28a3f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fae24153810 0x7fae240c8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.089+0000 7fae28a3f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae240c9d00 0x7fae240ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.089+0000 7fae28a3f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae240c43c0 con 0x7fae2415cbd0 2024-09-13T18:26:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.089+0000 7fae28a3f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae241595e0 con 0x7fae24153810 2024-09-13T18:26:31.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.089+0000 7fae28a3f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae241597d0 con 0x7fae240c9d00 2024-09-13T18:26:31.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.089+0000 7fae22d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fae24153810 0x7fae240c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae2359e700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fae2415cbd0 0x7fae2415d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:36998/0 (socket says 172.21.15.146:36998) 2024-09-13T18:26:31.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae2359e700 1 -- 172.21.15.146:0/425024865 learned_addr learned my addr 172.21.15.146:0/425024865 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:31.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae1a59c700 1 --2- 172.21.15.146:0/425024865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae240c9d00 0x7fae240ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae22d9d700 1 -- 172.21.15.146:0/425024865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae240c9d00 msgr2=0x7fae240ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae22d9d700 1 --2- 172.21.15.146:0/425024865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae240c9d00 0x7fae240ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae2259c700 1 -- 172.21.15.146:0/425024865 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742309860 0 0) 0x7fae240c43c0 con 0x7fae2415cbd0 2024-09-13T18:26:31.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae22d9d700 1 -- 172.21.15.146:0/425024865 >> v1:172.21.15.146:6789/0 conn(0x7fae2415cbd0 legacy=0x7fae2415d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae22d9d700 1 -- 172.21.15.146:0/425024865 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae240c8c70 con 0x7fae24153810 2024-09-13T18:26:31.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae22d9d700 1 --2- 172.21.15.146:0/425024865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae24153810 0x7fae240c8730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009850 tx=0x7fae0c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbec9ff4e844705e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae2259c700 1 -- 172.21.15.146:0/425024865 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c011830 con 0x7fae24153810 2024-09-13T18:26:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae2259c700 1 -- 172.21.15.146:0/425024865 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c011990 con 0x7fae24153810 2024-09-13T18:26:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae2259c700 1 -- 172.21.15.146:0/425024865 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c011d40 con 0x7fae24153810 2024-09-13T18:26:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae28a3f700 1 -- 172.21.15.146:0/425024865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae24153810 msgr2=0x7fae240c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae28a3f700 1 --2- 172.21.15.146:0/425024865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae24153810 0x7fae240c8730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009850 tx=0x7fae0c009b60 comp rx=0 tx=0).stop 2024-09-13T18:26:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae28a3f700 1 -- 172.21.15.146:0/425024865 shutdown_connections 2024-09-13T18:26:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae28a3f700 1 --2- 172.21.15.146:0/425024865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae240c9d00 0x7fae240ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae28a3f700 1 --2- 172.21.15.146:0/425024865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae24153810 0x7fae240c8730 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae28a3f700 1 -- 172.21.15.146:0/425024865 >> 172.21.15.146:0/425024865 conn(0x7fae2406f5c0 msgr2=0x7fae2406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae28a3f700 1 -- 172.21.15.146:0/425024865 shutdown_connections 2024-09-13T18:26:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.093+0000 7fae28a3f700 1 -- 172.21.15.146:0/425024865 wait complete. 2024-09-13T18:26:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae28a3f700 1 Processor -- start 2024-09-13T18:26:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae28a3f700 1 -- start start 2024-09-13T18:26:31.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae28a3f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae240c9d00 0x7fae2414ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae28a3f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae24153810 0x7fae240adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae28a3f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae2415cbd0 0x7fae241498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae28a3f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae241f26a0 con 0x7fae2415cbd0 2024-09-13T18:26:31.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae28a3f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae241f2960 con 0x7fae24153810 2024-09-13T18:26:31.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae28a3f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae241ef240 con 0x7fae240c9d00 2024-09-13T18:26:31.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae2359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae2415cbd0 0x7fae241498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae2359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae2415cbd0 0x7fae241498e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60782/0 (socket says 172.21.15.146:60782) 2024-09-13T18:26:31.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae2359e700 1 -- 172.21.15.146:0/3324336606 learned_addr learned my addr 172.21.15.146:0/3324336606 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:31.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae22d9d700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae240c9d00 0x7fae2414ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae1a59c700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae24153810 0x7fae240adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae2359e700 1 -- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae240c9d00 msgr2=0x7fae2414ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae2359e700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae240c9d00 0x7fae2414ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae2359e700 1 -- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae24153810 msgr2=0x7fae240adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae2359e700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae24153810 0x7fae240adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae2359e700 1 -- 172.21.15.146:0/3324336606 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c0094b0 con 0x7fae2415cbd0 2024-09-13T18:26:31.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae2359e700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae2415cbd0 0x7fae241498e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00cb60 tx=0x7fae1c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae1bfff700 1 -- 172.21.15.146:0/3324336606 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c00ab50 con 0x7fae2415cbd0 2024-09-13T18:26:31.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae1bfff700 1 -- 172.21.15.146:0/3324336606 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c00acb0 con 0x7fae2415cbd0 2024-09-13T18:26:31.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae28a3f700 1 -- 172.21.15.146:0/3324336606 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae241ef520 con 0x7fae2415cbd0 2024-09-13T18:26:31.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae1bfff700 1 -- 172.21.15.146:0/3324336606 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c007310 con 0x7fae2415cbd0 2024-09-13T18:26:31.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.097+0000 7fae28a3f700 1 -- 172.21.15.146:0/3324336606 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae241efae0 con 0x7fae2415cbd0 2024-09-13T18:26:31.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.101+0000 7fae28a3f700 1 -- 172.21.15.146:0/3324336606 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae241efd00 con 0x7fae2415cbd0 2024-09-13T18:26:31.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.101+0000 7fae1bfff700 1 -- 172.21.15.146:0/3324336606 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fae1c007470 con 0x7fae2415cbd0 2024-09-13T18:26:31.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.101+0000 7fae1bfff700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fae04066730 0x7fae04068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.101+0000 7fae1bfff700 1 -- 172.21.15.146:0/3324336606 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fae1c083b60 con 0x7fae2415cbd0 2024-09-13T18:26:31.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.101+0000 7fae22d9d700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fae04066730 0x7fae04068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.101+0000 7fae22d9d700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fae04066730 0x7fae04068bf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0090b0 tx=0x7fae0c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.109+0000 7fae1bfff700 1 -- 172.21.15.146:0/3324336606 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fae1c051030 con 0x7fae2415cbd0 2024-09-13T18:26:31.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.281+0000 7fae28a3f700 1 -- 172.21.15.146:0/3324336606 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fae240c7de0 con 0x7fae2415cbd0 2024-09-13T18:26:31.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.953+0000 7fae1bfff700 1 -- 172.21.15.146:0/3324336606 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7fae1c018020 con 0x7fae2415cbd0 2024-09-13T18:26:31.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.973+0000 7fae28a3f700 1 -- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fae04066730 msgr2=0x7fae04068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.973+0000 7fae28a3f700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fae04066730 0x7fae04068bf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0090b0 tx=0x7fae0c008030 comp rx=0 tx=0).stop 2024-09-13T18:26:31.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.973+0000 7fae28a3f700 1 -- 172.21.15.146:0/3324336606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae2415cbd0 msgr2=0x7fae241498e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.973+0000 7fae28a3f700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae2415cbd0 0x7fae241498e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00cb60 tx=0x7fae1c00cf20 comp rx=0 tx=0).stop 2024-09-13T18:26:31.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.973+0000 7fae28a3f700 1 -- 172.21.15.146:0/3324336606 shutdown_connections 2024-09-13T18:26:31.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.973+0000 7fae28a3f700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fae04066730 0x7fae04068bf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.973+0000 7fae28a3f700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae2415cbd0 0x7fae241498e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.973+0000 7fae28a3f700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae24153810 0x7fae240adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.973+0000 7fae28a3f700 1 --2- 172.21.15.146:0/3324336606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae240c9d00 0x7fae2414ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.973+0000 7fae28a3f700 1 -- 172.21.15.146:0/3324336606 >> 172.21.15.146:0/3324336606 conn(0x7fae2406f5c0 msgr2=0x7fae240c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.973+0000 7fae28a3f700 1 -- 172.21.15.146:0/3324336606 shutdown_connections 2024-09-13T18:26:31.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:31.973+0000 7fae28a3f700 1 -- 172.21.15.146:0/3324336606 wait complete. 2024-09-13T18:26:31.982 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_data' created 2024-09-13T18:26:31.992 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T18:26:32.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 Processor -- start 2024-09-13T18:26:32.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 -- start start 2024-09-13T18:26:32.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa0e00cf6c0 0x7fa0e00cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:32.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa0e00cb770 0x7fa0e015e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:32.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e00cb480 con 0x7fa0e00cfaa0 2024-09-13T18:26:32.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e0161410 con 0x7fa0e00cf6c0 2024-09-13T18:26:32.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e01616f0 con 0x7fa0e00cb770 2024-09-13T18:26:32.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e637c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa0e00cf6c0 0x7fa0e00cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:32.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e637c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa0e00cf6c0 0x7fa0e00cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60794/0 (socket says 172.21.15.146:60794) 2024-09-13T18:26:32.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e637c700 1 -- 172.21.15.146:0/1188685972 learned_addr learned my addr 172.21.15.146:0/1188685972 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:32.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e5b7b700 1 --2- 172.21.15.146:0/1188685972 >> v2:172.21.15.154:3300/0 conn(0x7fa0e00cb770 0x7fa0e015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:32.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e637c700 1 -- 172.21.15.146:0/1188685972 >> v2:172.21.15.154:3300/0 conn(0x7fa0e00cb770 msgr2=0x7fa0e015e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:32.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e637c700 1 --2- 172.21.15.146:0/1188685972 >> v2:172.21.15.154:3300/0 conn(0x7fa0e00cb770 0x7fa0e015e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:32.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e637c700 1 -- 172.21.15.146:0/1188685972 >> v1:172.21.15.154:6789/0 conn(0x7fa0e00cfaa0 legacy=0x7fa0e015f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:32.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e637c700 1 -- 172.21.15.146:0/1188685972 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e01619d0 con 0x7fa0e00cf6c0 2024-09-13T18:26:32.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e5b7b700 1 --2- 172.21.15.146:0/1188685972 >> v2:172.21.15.154:3300/0 conn(0x7fa0e00cb770 0x7fa0e015e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:32.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e637c700 1 --2- 172.21.15.146:0/1188685972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00cf6c0 0x7fa0e00cae60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa0e008c770 tx=0x7fa0dc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ead2c8a9bed6cc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:32.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e537a700 1 -- 172.21.15.146:0/1188685972 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc00b8a0 con 0x7fa0e00cf6c0 2024-09-13T18:26:32.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e537a700 1 -- 172.21.15.146:0/1188685972 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0dc00ba00 con 0x7fa0e00cf6c0 2024-09-13T18:26:32.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e537a700 1 -- 172.21.15.146:0/1188685972 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc00bd90 con 0x7fa0e00cf6c0 2024-09-13T18:26:32.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 -- 172.21.15.146:0/1188685972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00cf6c0 msgr2=0x7fa0e00cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:32.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 --2- 172.21.15.146:0/1188685972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00cf6c0 0x7fa0e00cae60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa0e008c770 tx=0x7fa0dc009bf0 comp rx=0 tx=0).stop 2024-09-13T18:26:32.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 -- 172.21.15.146:0/1188685972 shutdown_connections 2024-09-13T18:26:32.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 --2- 172.21.15.146:0/1188685972 >> v2:172.21.15.154:3300/0 conn(0x7fa0e00cb770 0x7fa0e015e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 --2- 172.21.15.146:0/1188685972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00cf6c0 0x7fa0e00cae60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 -- 172.21.15.146:0/1188685972 >> 172.21.15.146:0/1188685972 conn(0x7fa0e0071d90 msgr2=0x7fa0e00721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 -- 172.21.15.146:0/1188685972 shutdown_connections 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.097+0000 7fa0e85e0700 1 -- 172.21.15.146:0/1188685972 wait complete. 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e85e0700 1 Processor -- start 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e85e0700 1 -- start start 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e85e0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00cb770 0x7fa0e00c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e85e0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0e00cf6c0 0x7fa0e01f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e85e0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0e00cfaa0 0x7fa0e01593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e85e0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e01e74f0 con 0x7fa0e00cb770 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e85e0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00c6850 con 0x7fa0e00cfaa0 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e85e0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00c2fc0 con 0x7fa0e00cf6c0 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e637c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00cb770 0x7fa0e00c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:32.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e637c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00cb770 0x7fa0e00c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60806/0 (socket says 172.21.15.146:60806) 2024-09-13T18:26:32.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e637c700 1 -- 172.21.15.146:0/131701633 learned_addr learned my addr 172.21.15.146:0/131701633 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:32.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e5b7b700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0e00cf6c0 0x7fa0e01f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:32.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e6b7d700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0e00cfaa0 0x7fa0e01593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:32.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e637c700 1 -- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0e00cf6c0 msgr2=0x7fa0e01f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:32.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e637c700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0e00cf6c0 0x7fa0e01f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:32.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e637c700 1 -- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0e00cfaa0 msgr2=0x7fa0e01593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:32.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e637c700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0e00cfaa0 0x7fa0e01593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:32.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e637c700 1 -- 172.21.15.146:0/131701633 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc009570 con 0x7fa0e00cb770 2024-09-13T18:26:32.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e637c700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00cb770 0x7fa0e00c41e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0075ca0 tx=0x7fa0dc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:32.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0d6ffd700 1 -- 172.21.15.146:0/131701633 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc0036c0 con 0x7fa0e00cb770 2024-09-13T18:26:32.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0d6ffd700 1 -- 172.21.15.146:0/131701633 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0dc01f070 con 0x7fa0e00cb770 2024-09-13T18:26:32.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0d6ffd700 1 -- 172.21.15.146:0/131701633 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc003a80 con 0x7fa0e00cb770 2024-09-13T18:26:32.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e85e0700 1 -- 172.21.15.146:0/131701633 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e01e7210 con 0x7fa0e00cb770 2024-09-13T18:26:32.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.101+0000 7fa0e85e0700 1 -- 172.21.15.146:0/131701633 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e00c34b0 con 0x7fa0e00cb770 2024-09-13T18:26:32.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.105+0000 7fa0e85e0700 1 -- 172.21.15.146:0/131701633 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e00c36d0 con 0x7fa0e00cb770 2024-09-13T18:26:32.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.105+0000 7fa0d6ffd700 1 -- 172.21.15.146:0/131701633 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa0dc021030 con 0x7fa0e00cb770 2024-09-13T18:26:32.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.105+0000 7fa0d6ffd700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa0c4066730 0x7fa0c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:32.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.105+0000 7fa0d6ffd700 1 -- 172.21.15.146:0/131701633 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fa0dc0856a0 con 0x7fa0e00cb770 2024-09-13T18:26:32.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.105+0000 7fa0e5b7b700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa0c4066730 0x7fa0c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:32.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.109+0000 7fa0e5b7b700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa0c4066730 0x7fa0c4068bf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00b2cf0 tx=0x7fa0d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:32.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.109+0000 7fa0d6ffd700 1 -- 172.21.15.146:0/131701633 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa0dc052a70 con 0x7fa0e00cb770 2024-09-13T18:26:32.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.321+0000 7fa0e85e0700 1 -- 172.21.15.146:0/131701633 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa0e00caa80 con 0x7fa0e00cb770 2024-09-13T18:26:32.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.333+0000 7fa0d6ffd700 1 -- 172.21.15.146:0/131701633 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa0dc056090 con 0x7fa0e00cb770 2024-09-13T18:26:32.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.349+0000 7fa0e85e0700 1 -- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa0c4066730 msgr2=0x7fa0c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:32.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.349+0000 7fa0e85e0700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa0c4066730 0x7fa0c4068bf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00b2cf0 tx=0x7fa0d0008040 comp rx=0 tx=0).stop 2024-09-13T18:26:32.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.349+0000 7fa0e85e0700 1 -- 172.21.15.146:0/131701633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00cb770 msgr2=0x7fa0e00c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:32.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.349+0000 7fa0e85e0700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00cb770 0x7fa0e00c41e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0075ca0 tx=0x7fa0dc004770 comp rx=0 tx=0).stop 2024-09-13T18:26:32.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.349+0000 7fa0e85e0700 1 -- 172.21.15.146:0/131701633 shutdown_connections 2024-09-13T18:26:32.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.349+0000 7fa0e85e0700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa0c4066730 0x7fa0c4068bf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:32.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.349+0000 7fa0e85e0700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0e00cfaa0 0x7fa0e01593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:32.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.349+0000 7fa0e85e0700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0e00cf6c0 0x7fa0e01f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:32.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.349+0000 7fa0e85e0700 1 --2- 172.21.15.146:0/131701633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00cb770 0x7fa0e00c41e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:32.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.349+0000 7fa0e85e0700 1 -- 172.21.15.146:0/131701633 >> 172.21.15.146:0/131701633 conn(0x7fa0e0071d90 msgr2=0x7fa0e015f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:32.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.349+0000 7fa0e85e0700 1 -- 172.21.15.146:0/131701633 shutdown_connections 2024-09-13T18:26:32.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.349+0000 7fa0e85e0700 1 -- 172.21.15.146:0/131701633 wait complete. 2024-09-13T18:26:32.356 INFO:teuthology.orchestra.run.smithi146.stderr:new fs with metadata pool 2 and data pool 3 2024-09-13T18:26:32.368 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-13T18:26:32.368 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-13T18:26:32.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.477+0000 7f9057184700 1 Processor -- start 2024-09-13T18:26:32.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.477+0000 7f9057184700 1 -- start start 2024-09-13T18:26:32.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9057184700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f90501539e0 0x7f90500b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:32.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9057184700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90500c5e50 con 0x7f905015d960 2024-09-13T18:26:32.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9057184700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90500cb480 con 0x7f90500cf580 2024-09-13T18:26:32.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9057184700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90500cc3a0 con 0x7f90501539e0 2024-09-13T18:26:32.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9056182700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f90501539e0 0x7f90500b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:32.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9055981700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f905015d960 0x7f90500c9fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37006/0 (socket says 172.21.15.146:37006) 2024-09-13T18:26:32.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9055981700 1 -- 172.21.15.146:0/3935053960 learned_addr learned my addr 172.21.15.146:0/3935053960 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:32.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9055180700 1 -- 172.21.15.146:0/3935053960 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605726331 0 0) 0x7f904c001110 con 0x7f905015d960 2024-09-13T18:26:32.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9055180700 1 -- 172.21.15.146:0/3935053960 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f903c0035d0 con 0x7f905015d960 2024-09-13T18:26:32.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9056182700 1 -- 172.21.15.146:0/3935053960 >> v1:172.21.15.154:6789/0 conn(0x7f90500cf580 legacy=0x7f90500cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:32.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9056182700 1 -- 172.21.15.146:0/3935053960 >> v1:172.21.15.146:6789/0 conn(0x7f905015d960 legacy=0x7f90500c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:32.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9056182700 1 -- 172.21.15.146:0/3935053960 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90500cb720 con 0x7f90501539e0 2024-09-13T18:26:32.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9056182700 1 --2- 172.21.15.146:0/3935053960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90501539e0 0x7f90500b3370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9040009880 tx=0x7f9040009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3e03da91e8b8e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:32.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9055180700 1 -- 172.21.15.146:0/3935053960 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040011830 con 0x7f90501539e0 2024-09-13T18:26:32.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9055180700 1 -- 172.21.15.146:0/3935053960 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9040011990 con 0x7f90501539e0 2024-09-13T18:26:32.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9055180700 1 -- 172.21.15.146:0/3935053960 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040011d20 con 0x7f90501539e0 2024-09-13T18:26:32.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9057184700 1 -- 172.21.15.146:0/3935053960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90501539e0 msgr2=0x7f90500b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:32.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9057184700 1 --2- 172.21.15.146:0/3935053960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90501539e0 0x7f90500b3370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9040009880 tx=0x7f9040009b90 comp rx=0 tx=0).stop 2024-09-13T18:26:32.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9057184700 1 -- 172.21.15.146:0/3935053960 shutdown_connections 2024-09-13T18:26:32.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9057184700 1 --2- 172.21.15.146:0/3935053960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90501539e0 0x7f90500b3370 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:32.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9057184700 1 -- 172.21.15.146:0/3935053960 >> 172.21.15.146:0/3935053960 conn(0x7f9050071ec0 msgr2=0x7f90500722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:32.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9057184700 1 -- 172.21.15.146:0/3935053960 shutdown_connections 2024-09-13T18:26:32.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.481+0000 7f9057184700 1 -- 172.21.15.146:0/3935053960 wait complete. 2024-09-13T18:26:32.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.485+0000 7f9057184700 1 Processor -- start 2024-09-13T18:26:32.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.485+0000 7f9057184700 1 -- start start 2024-09-13T18:26:32.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.485+0000 7f9057184700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90500cf580 0x7f905014c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:32.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.485+0000 7f9057184700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90501539e0 0x7f90500b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:32.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.485+0000 7f9057184700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905015d960 0x7f9050146e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:32.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.485+0000 7f9057184700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905014cf30 con 0x7f90501539e0 2024-09-13T18:26:32.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.485+0000 7f9057184700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90500c6920 con 0x7f90500cf580 2024-09-13T18:26:32.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.485+0000 7f9057184700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90500c85b0 con 0x7f905015d960 2024-09-13T18:26:32.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.485+0000 7f9055981700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90501539e0 0x7f90500b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:32.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.485+0000 7f9056182700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90500cf580 0x7f905014c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:32.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.485+0000 7f9055981700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90501539e0 0x7f90500b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60824/0 (socket says 172.21.15.146:60824) 2024-09-13T18:26:32.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.485+0000 7f9055981700 1 -- 172.21.15.146:0/597473138 learned_addr learned my addr 172.21.15.146:0/597473138 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:32.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.485+0000 7f9056983700 1 --2- 172.21.15.146:0/597473138 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905015d960 0x7f9050146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:32.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.489+0000 7f9055981700 1 -- 172.21.15.146:0/597473138 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905015d960 msgr2=0x7f9050146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:32.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.489+0000 7f9055981700 1 --2- 172.21.15.146:0/597473138 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905015d960 0x7f9050146e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:32.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.489+0000 7f9055981700 1 -- 172.21.15.146:0/597473138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90500cf580 msgr2=0x7f905014c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:32.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.489+0000 7f9055981700 1 --2- 172.21.15.146:0/597473138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90500cf580 0x7f905014c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:32.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.489+0000 7f9055981700 1 -- 172.21.15.146:0/597473138 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90400094b0 con 0x7f90501539e0 2024-09-13T18:26:32.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.489+0000 7f9055981700 1 --2- 172.21.15.146:0/597473138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90501539e0 0x7f90500b4540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f904c001270 tx=0x7f904c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:32.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.489+0000 7f9046ffd700 1 -- 172.21.15.146:0/597473138 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c005d90 con 0x7f90501539e0 2024-09-13T18:26:32.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.489+0000 7f9046ffd700 1 -- 172.21.15.146:0/597473138 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c0070d0 con 0x7f90501539e0 2024-09-13T18:26:32.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.489+0000 7f9057184700 1 -- 172.21.15.146:0/597473138 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90500b19e0 con 0x7f90501539e0 2024-09-13T18:26:32.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.489+0000 7f9046ffd700 1 -- 172.21.15.146:0/597473138 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c007480 con 0x7f90501539e0 2024-09-13T18:26:32.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.489+0000 7f9057184700 1 -- 172.21.15.146:0/597473138 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90501488f0 con 0x7f90501539e0 2024-09-13T18:26:32.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.489+0000 7f9057184700 1 -- 172.21.15.146:0/597473138 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9050146b30 con 0x7f90501539e0 2024-09-13T18:26:32.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.493+0000 7f9046ffd700 1 -- 172.21.15.146:0/597473138 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f904c0075e0 con 0x7f90501539e0 2024-09-13T18:26:32.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.493+0000 7f9046ffd700 1 --2- 172.21.15.146:0/597473138 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9034066730 0x7f9034068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:32.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.493+0000 7f9046ffd700 1 -- 172.21.15.146:0/597473138 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f904c0833d0 con 0x7f90501539e0 2024-09-13T18:26:32.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.493+0000 7f9056182700 1 --2- 172.21.15.146:0/597473138 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9034066730 0x7f9034068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:32.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.493+0000 7f9056182700 1 --2- 172.21.15.146:0/597473138 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9034066730 0x7f9034068bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f90400090b0 tx=0x7f9040008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:32.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.501+0000 7f9046ffd700 1 -- 172.21.15.146:0/597473138 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9050146b30 con 0x7f90501539e0 2024-09-13T18:26:32.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:32.677+0000 7f9057184700 1 -- 172.21.15.146:0/597473138 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f90500c5e50 con 0x7f90501539e0 2024-09-13T18:26:33.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.341+0000 7f9046ffd700 1 -- 172.21.15.146:0/597473138 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f90500c5e50 con 0x7f90501539e0 2024-09-13T18:26:33.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.345+0000 7f9057184700 1 -- 172.21.15.146:0/597473138 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9034066730 msgr2=0x7f9034068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:33.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.345+0000 7f9057184700 1 --2- 172.21.15.146:0/597473138 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9034066730 0x7f9034068bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f90400090b0 tx=0x7f9040008030 comp rx=0 tx=0).stop 2024-09-13T18:26:33.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.345+0000 7f9057184700 1 -- 172.21.15.146:0/597473138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90501539e0 msgr2=0x7f90500b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:33.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.345+0000 7f9057184700 1 --2- 172.21.15.146:0/597473138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90501539e0 0x7f90500b4540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f904c001270 tx=0x7f904c00c7d0 comp rx=0 tx=0).stop 2024-09-13T18:26:33.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.345+0000 7f9057184700 1 -- 172.21.15.146:0/597473138 shutdown_connections 2024-09-13T18:26:33.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.345+0000 7f9057184700 1 --2- 172.21.15.146:0/597473138 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9034066730 0x7f9034068bf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:33.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.345+0000 7f9057184700 1 --2- 172.21.15.146:0/597473138 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905015d960 0x7f9050146e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:33.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.345+0000 7f9057184700 1 --2- 172.21.15.146:0/597473138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90501539e0 0x7f90500b4540 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:33.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.345+0000 7f9057184700 1 --2- 172.21.15.146:0/597473138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90500cf580 0x7f905014c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:33.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.345+0000 7f9057184700 1 -- 172.21.15.146:0/597473138 >> 172.21.15.146:0/597473138 conn(0x7f9050071ec0 msgr2=0x7f90500b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:33.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.345+0000 7f9057184700 1 -- 172.21.15.146:0/597473138 shutdown_connections 2024-09-13T18:26:33.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.345+0000 7f9057184700 1 -- 172.21.15.146:0/597473138 wait complete. 2024-09-13T18:26:33.366 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T18:26:33.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb086b18700 1 Processor -- start 2024-09-13T18:26:33.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb086b18700 1 -- start start 2024-09-13T18:26:33.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb086b18700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0800c68c0 0x7fb0800cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:33.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb086b18700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb080159b30 con 0x7fb080153af0 2024-09-13T18:26:33.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb086b18700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb080159e10 con 0x7fb0800cbd20 2024-09-13T18:26:33.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb086b18700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0800cc100 con 0x7fb0800c68c0 2024-09-13T18:26:33.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb085315700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0800c68c0 0x7fb0800cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:33.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb085b16700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb080153af0 0x7fb0800667a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37016/0 (socket says 172.21.15.146:37016) 2024-09-13T18:26:33.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb085b16700 1 -- 172.21.15.146:0/4133564386 learned_addr learned my addr 172.21.15.146:0/4133564386 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:33.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb084b14700 1 -- 172.21.15.146:0/4133564386 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836480634 0 0) 0x7fb080159b30 con 0x7fb080153af0 2024-09-13T18:26:33.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb084b14700 1 -- 172.21.15.146:0/4133564386 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb06c0035d0 con 0x7fb080153af0 2024-09-13T18:26:33.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb084b14700 1 -- 172.21.15.146:0/4133564386 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3296196328 0 0) 0x7fb06c0035d0 con 0x7fb080153af0 2024-09-13T18:26:33.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb085315700 1 -- 172.21.15.146:0/4133564386 >> v1:172.21.15.154:6789/0 conn(0x7fb0800cbd20 legacy=0x7fb0800c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:33.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb085315700 1 -- 172.21.15.146:0/4133564386 >> v1:172.21.15.146:6789/0 conn(0x7fb080153af0 legacy=0x7fb0800667a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:33.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb085315700 1 -- 172.21.15.146:0/4133564386 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08015a0f0 con 0x7fb0800c68c0 2024-09-13T18:26:33.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.493+0000 7fb085315700 1 --2- 172.21.15.146:0/4133564386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0800c68c0 0x7fb0800cb7e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb070009850 tx=0x7fb070009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daab044fb8a79fea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:33.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.497+0000 7fb084b14700 1 -- 172.21.15.146:0/4133564386 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c001670 con 0x7fb0800c68c0 2024-09-13T18:26:33.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.497+0000 7fb084b14700 1 -- 172.21.15.146:0/4133564386 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb070004030 con 0x7fb0800c68c0 2024-09-13T18:26:33.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.497+0000 7fb084b14700 1 -- 172.21.15.146:0/4133564386 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0700043c0 con 0x7fb0800c68c0 2024-09-13T18:26:33.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.497+0000 7fb086b18700 1 -- 172.21.15.146:0/4133564386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0800c68c0 msgr2=0x7fb0800cb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:33.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.497+0000 7fb086b18700 1 --2- 172.21.15.146:0/4133564386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0800c68c0 0x7fb0800cb7e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb070009850 tx=0x7fb070009b60 comp rx=0 tx=0).stop 2024-09-13T18:26:33.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.497+0000 7fb086b18700 1 -- 172.21.15.146:0/4133564386 shutdown_connections 2024-09-13T18:26:33.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.497+0000 7fb086b18700 1 --2- 172.21.15.146:0/4133564386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0800c68c0 0x7fb0800cb7e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:33.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.497+0000 7fb086b18700 1 -- 172.21.15.146:0/4133564386 >> 172.21.15.146:0/4133564386 conn(0x7fb080072170 msgr2=0x7fb080066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:33.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.497+0000 7fb086b18700 1 -- 172.21.15.146:0/4133564386 shutdown_connections 2024-09-13T18:26:33.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.497+0000 7fb086b18700 1 -- 172.21.15.146:0/4133564386 wait complete. 2024-09-13T18:26:33.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.497+0000 7fb086b18700 1 Processor -- start 2024-09-13T18:26:33.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.497+0000 7fb086b18700 1 -- start start 2024-09-13T18:26:33.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb086b18700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0800c68c0 0x7fb08014d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:33.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb086b18700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0800cbd20 0x7fb0800c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:33.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb086b18700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb080153af0 0x7fb0801563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:33.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb086b18700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08014e4b0 con 0x7fb0800c68c0 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb086b18700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08014fac0 con 0x7fb080153af0 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb086b18700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08014ed20 con 0x7fb0800cbd20 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb085b16700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0800c68c0 0x7fb08014d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb085b16700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0800c68c0 0x7fb08014d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60842/0 (socket says 172.21.15.146:60842) 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb085b16700 1 -- 172.21.15.146:0/2579863689 learned_addr learned my addr 172.21.15.146:0/2579863689 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb085315700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0800cbd20 0x7fb0800c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb086317700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb080153af0 0x7fb0801563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb085b16700 1 -- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0800cbd20 msgr2=0x7fb0800c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb085b16700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0800cbd20 0x7fb0800c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb085b16700 1 -- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb080153af0 msgr2=0x7fb0801563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb085b16700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb080153af0 0x7fb0801563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb085b16700 1 -- 172.21.15.146:0/2579863689 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0700094b0 con 0x7fb0800c68c0 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb086317700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb080153af0 0x7fb0801563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb085315700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0800cbd20 0x7fb0800c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:33.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb085b16700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0800c68c0 0x7fb08014d1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00c540 tx=0x7fb07c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:33.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb0767fc700 1 -- 172.21.15.146:0/2579863689 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c004ca0 con 0x7fb0800c68c0 2024-09-13T18:26:33.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb0767fc700 1 -- 172.21.15.146:0/2579863689 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07c004e00 con 0x7fb0800c68c0 2024-09-13T18:26:33.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb086b18700 1 -- 172.21.15.146:0/2579863689 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08014ea40 con 0x7fb0800c68c0 2024-09-13T18:26:33.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb0767fc700 1 -- 172.21.15.146:0/2579863689 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c007310 con 0x7fb0800c68c0 2024-09-13T18:26:33.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.501+0000 7fb086b18700 1 -- 172.21.15.146:0/2579863689 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0801efd90 con 0x7fb0800c68c0 2024-09-13T18:26:33.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.505+0000 7fb0767fc700 1 -- 172.21.15.146:0/2579863689 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb07c007470 con 0x7fb0800c68c0 2024-09-13T18:26:33.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.505+0000 7fb086b18700 1 -- 172.21.15.146:0/2579863689 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb080074f30 con 0x7fb0800c68c0 2024-09-13T18:26:33.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.505+0000 7fb0767fc700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb064066730 0x7fb064068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:33.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.505+0000 7fb0767fc700 1 -- 172.21.15.146:0/2579863689 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5935+0+0 (secure 0 0 0) 0x7fb07c083f40 con 0x7fb0800c68c0 2024-09-13T18:26:33.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.505+0000 7fb085315700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb064066730 0x7fb064068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:33.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.505+0000 7fb085315700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb064066730 0x7fb064068bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb0700090b0 tx=0x7fb070008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:33.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.509+0000 7fb0767fc700 1 -- 172.21.15.146:0/2579863689 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb07c051260 con 0x7fb0800c68c0 2024-09-13T18:26:33.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:33.681+0000 7fb086b18700 1 -- 172.21.15.146:0/2579863689 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb0800c6530 con 0x7fb0800c68c0 2024-09-13T18:26:35.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.361+0000 7fb0767fc700 1 -- 172.21.15.146:0/2579863689 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7fb07c007720 con 0x7fb0800c68c0 2024-09-13T18:26:35.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.373+0000 7fb086b18700 1 -- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb064066730 msgr2=0x7fb064068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.373+0000 7fb086b18700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb064066730 0x7fb064068bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb0700090b0 tx=0x7fb070008030 comp rx=0 tx=0).stop 2024-09-13T18:26:35.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.373+0000 7fb086b18700 1 -- 172.21.15.146:0/2579863689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0800c68c0 msgr2=0x7fb08014d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.373+0000 7fb086b18700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0800c68c0 0x7fb08014d1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00c540 tx=0x7fb07c00c900 comp rx=0 tx=0).stop 2024-09-13T18:26:35.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.373+0000 7fb086b18700 1 -- 172.21.15.146:0/2579863689 shutdown_connections 2024-09-13T18:26:35.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.373+0000 7fb086b18700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb064066730 0x7fb064068bf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.373+0000 7fb086b18700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb080153af0 0x7fb0801563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.373+0000 7fb086b18700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0800cbd20 0x7fb0800c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.373+0000 7fb086b18700 1 --2- 172.21.15.146:0/2579863689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0800c68c0 0x7fb08014d1b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.373+0000 7fb086b18700 1 -- 172.21.15.146:0/2579863689 >> 172.21.15.146:0/2579863689 conn(0x7fb080072170 msgr2=0x7fb080066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:35.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.373+0000 7fb086b18700 1 -- 172.21.15.146:0/2579863689 shutdown_connections 2024-09-13T18:26:35.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.373+0000 7fb086b18700 1 -- 172.21.15.146:0/2579863689 wait complete. 2024-09-13T18:26:35.382 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_data_ec' created 2024-09-13T18:26:35.394 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-13T18:26:35.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.497+0000 7fc7a0099700 1 Processor -- start 2024-09-13T18:26:35.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.497+0000 7fc7a0099700 1 -- start start 2024-09-13T18:26:35.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc7a0099700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc798151750 0x7fc7981575f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc7a0099700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc79806c8a0 0x7fc79806db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc7a0099700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79806d120 con 0x7fc798151b30 2024-09-13T18:26:35.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc7a0099700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79806d750 con 0x7fc79806c8a0 2024-09-13T18:26:35.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc7a0099700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79806e0b0 con 0x7fc798151750 2024-09-13T18:26:35.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc79d634700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc79806c8a0 0x7fc79806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc79de35700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc798151750 0x7fc7981575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc79e636700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc798151b30 0x7fc7980c5c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:54180/0 (socket says 172.21.15.146:54180) 2024-09-13T18:26:35.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc79e636700 1 -- 172.21.15.146:0/2891535691 learned_addr learned my addr 172.21.15.146:0/2891535691 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:35.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc79d634700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc79806c8a0 0x7fc79806db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:44886/0 (socket says 172.21.15.146:44886) 2024-09-13T18:26:35.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc79d634700 1 -- 172.21.15.146:0/2891535691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc798151750 msgr2=0x7fc7981575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:35.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc79d634700 1 --2- 172.21.15.146:0/2891535691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc798151750 0x7fc7981575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc79d634700 1 -- 172.21.15.146:0/2891535691 >> v1:172.21.15.154:6789/0 conn(0x7fc798151b30 legacy=0x7fc7980c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc79d634700 1 -- 172.21.15.146:0/2891535691 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79815b140 con 0x7fc79806c8a0 2024-09-13T18:26:35.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc79d634700 1 --2- 172.21.15.146:0/2891535691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc79806c8a0 0x7fc79806db70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc788009850 tx=0x7fc788009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c1be76f67816067 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:35.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc79ce33700 1 -- 172.21.15.146:0/2891535691 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788011830 con 0x7fc79806c8a0 2024-09-13T18:26:35.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc79ce33700 1 -- 172.21.15.146:0/2891535691 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788011990 con 0x7fc79806c8a0 2024-09-13T18:26:35.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc79ce33700 1 -- 172.21.15.146:0/2891535691 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788011d40 con 0x7fc79806c8a0 2024-09-13T18:26:35.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc7a0099700 1 -- 172.21.15.146:0/2891535691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc79806c8a0 msgr2=0x7fc79806db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc7a0099700 1 --2- 172.21.15.146:0/2891535691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc79806c8a0 0x7fc79806db70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc788009850 tx=0x7fc788009b60 comp rx=0 tx=0).stop 2024-09-13T18:26:35.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc7a0099700 1 -- 172.21.15.146:0/2891535691 shutdown_connections 2024-09-13T18:26:35.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc7a0099700 1 --2- 172.21.15.146:0/2891535691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc79806c8a0 0x7fc79806db70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc7a0099700 1 --2- 172.21.15.146:0/2891535691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc798151750 0x7fc7981575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc7a0099700 1 -- 172.21.15.146:0/2891535691 >> 172.21.15.146:0/2891535691 conn(0x7fc79806f5c0 msgr2=0x7fc79806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:35.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc7a0099700 1 -- 172.21.15.146:0/2891535691 shutdown_connections 2024-09-13T18:26:35.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.501+0000 7fc7a0099700 1 -- 172.21.15.146:0/2891535691 wait complete. 2024-09-13T18:26:35.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc7a0099700 1 Processor -- start 2024-09-13T18:26:35.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc7a0099700 1 -- start start 2024-09-13T18:26:35.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc7a0099700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc79806c8a0 0x7fc798159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc7a0099700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc798151750 0x7fc7981ed010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc7a0099700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc798151b30 0x7fc7981ed550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc7a0099700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798156ea0 con 0x7fc798151750 2024-09-13T18:26:35.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc7a0099700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798154220 con 0x7fc79806c8a0 2024-09-13T18:26:35.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc7a0099700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc798156670 con 0x7fc798151b30 2024-09-13T18:26:35.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc79d634700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc798151750 0x7fc7981ed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc79d634700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc798151750 0x7fc7981ed010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60852/0 (socket says 172.21.15.146:60852) 2024-09-13T18:26:35.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc79d634700 1 -- 172.21.15.146:0/3990064414 learned_addr learned my addr 172.21.15.146:0/3990064414 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:35.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc79e636700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc798151b30 0x7fc7981ed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc79de35700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc79806c8a0 0x7fc798159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc79d634700 1 -- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc798151b30 msgr2=0x7fc7981ed550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc79d634700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc798151b30 0x7fc7981ed550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc79d634700 1 -- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc79806c8a0 msgr2=0x7fc798159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc79d634700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc79806c8a0 0x7fc798159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.505+0000 7fc79d634700 1 -- 172.21.15.146:0/3990064414 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7880094b0 con 0x7fc798151750 2024-09-13T18:26:35.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.509+0000 7fc79d634700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc798151750 0x7fc7981ed010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc7880090b0 tx=0x7fc788004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:35.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.509+0000 7fc78e7fc700 1 -- 172.21.15.146:0/3990064414 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788004550 con 0x7fc798151750 2024-09-13T18:26:35.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.509+0000 7fc78e7fc700 1 -- 172.21.15.146:0/3990064414 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7880036c0 con 0x7fc798151750 2024-09-13T18:26:35.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.509+0000 7fc7a0099700 1 -- 172.21.15.146:0/3990064414 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc798156c00 con 0x7fc798151750 2024-09-13T18:26:35.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.509+0000 7fc78e7fc700 1 -- 172.21.15.146:0/3990064414 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788003a70 con 0x7fc798151750 2024-09-13T18:26:35.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.509+0000 7fc7a0099700 1 -- 172.21.15.146:0/3990064414 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7981f0f10 con 0x7fc798151750 2024-09-13T18:26:35.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.509+0000 7fc7a0099700 1 -- 172.21.15.146:0/3990064414 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7981567b0 con 0x7fc798151750 2024-09-13T18:26:35.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.509+0000 7fc78e7fc700 1 -- 172.21.15.146:0/3990064414 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc788003bd0 con 0x7fc798151750 2024-09-13T18:26:35.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.513+0000 7fc78e7fc700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc77c066730 0x7fc77c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.513+0000 7fc78e7fc700 1 -- 172.21.15.146:0/3990064414 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fc7880856f0 con 0x7fc798151750 2024-09-13T18:26:35.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.513+0000 7fc79de35700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc77c066730 0x7fc77c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.513+0000 7fc79de35700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc77c066730 0x7fc77c068bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc794009bf0 tx=0x7fc794009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:35.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.517+0000 7fc78e7fc700 1 -- 172.21.15.146:0/3990064414 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc7981567b0 con 0x7fc798151750 2024-09-13T18:26:35.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:35.693+0000 7fc7a0099700 1 -- 172.21.15.146:0/3990064414 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fc7981567b0 con 0x7fc798151750 2024-09-13T18:26:36.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.377+0000 7fc78e7fc700 1 -- 172.21.15.146:0/3990064414 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc7981567b0 con 0x7fc798151750 2024-09-13T18:26:36.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.393+0000 7fc7a0099700 1 -- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc77c066730 msgr2=0x7fc77c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.393+0000 7fc7a0099700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc77c066730 0x7fc77c068bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc794009bf0 tx=0x7fc794009240 comp rx=0 tx=0).stop 2024-09-13T18:26:36.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.393+0000 7fc7a0099700 1 -- 172.21.15.146:0/3990064414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc798151750 msgr2=0x7fc7981ed010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.393+0000 7fc7a0099700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc798151750 0x7fc7981ed010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc7880090b0 tx=0x7fc788004770 comp rx=0 tx=0).stop 2024-09-13T18:26:36.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.393+0000 7fc7a0099700 1 -- 172.21.15.146:0/3990064414 shutdown_connections 2024-09-13T18:26:36.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.393+0000 7fc7a0099700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc77c066730 0x7fc77c068bf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.393+0000 7fc7a0099700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc798151b30 0x7fc7981ed550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.393+0000 7fc7a0099700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc798151750 0x7fc7981ed010 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.393+0000 7fc7a0099700 1 --2- 172.21.15.146:0/3990064414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc79806c8a0 0x7fc798159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.393+0000 7fc7a0099700 1 -- 172.21.15.146:0/3990064414 >> 172.21.15.146:0/3990064414 conn(0x7fc79806f5c0 msgr2=0x7fc798070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:36.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.401+0000 7fc7a0099700 1 -- 172.21.15.146:0/3990064414 shutdown_connections 2024-09-13T18:26:36.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.401+0000 7fc7a0099700 1 -- 172.21.15.146:0/3990064414 wait complete. 2024-09-13T18:26:36.417 INFO:teuthology.orchestra.run.smithi146.stderr:set pool 4 allow_ec_overwrites to true 2024-09-13T18:26:36.423 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-13T18:26:36.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.525+0000 7f0f28957700 1 Processor -- start 2024-09-13T18:26:36.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.525+0000 7f0f28957700 1 -- start start 2024-09-13T18:26:36.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.525+0000 7f0f28957700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0f2406d990 0x7f0f24161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:36.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.525+0000 7f0f28957700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f24071470 con 0x7f0f24150470 2024-09-13T18:26:36.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.525+0000 7f0f28957700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f2406e7c0 con 0x7f0f240c2ef0 2024-09-13T18:26:36.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.525+0000 7f0f28957700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f240c8730 con 0x7f0f2406d990 2024-09-13T18:26:36.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.525+0000 7f0f237fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0f24150470 0x7f0f2406ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37024/0 (socket says 172.21.15.146:37024) 2024-09-13T18:26:36.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.525+0000 7f0f237fe700 1 -- 172.21.15.146:0/4072649411 learned_addr learned my addr 172.21.15.146:0/4072649411 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:36.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.525+0000 7f0f23fff700 1 --2- 172.21.15.146:0/4072649411 >> v2:172.21.15.154:3300/0 conn(0x7f0f2406d990 0x7f0f24161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:36.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.525+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830705550 0 0) 0x7f0f24071470 con 0x7f0f24150470 2024-09-13T18:26:36.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.525+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f0c0035d0 con 0x7f0f24150470 2024-09-13T18:26:36.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.525+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 257569939 0 0) 0x7f0f0c0035d0 con 0x7f0f24150470 2024-09-13T18:26:36.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.525+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 >> v2:172.21.15.154:3300/0 conn(0x7f0f2406d990 msgr2=0x7f0f24161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:36.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 --2- 172.21.15.146:0/4072649411 >> v2:172.21.15.154:3300/0 conn(0x7f0f2406d990 0x7f0f24161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 >> v1:172.21.15.154:6789/0 conn(0x7f0f240c2ef0 legacy=0x7f0f240cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2406e100 con 0x7f0f24150470 2024-09-13T18:26:36.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f0f14001670 con 0x7f0f24150470 2024-09-13T18:26:36.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 >> v1:172.21.15.146:6789/0 conn(0x7f0f24150470 legacy=0x7f0f2406ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 --2- 172.21.15.146:0/4072649411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f0c003a90 0x7f0f0c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:36.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 --2- 172.21.15.146:0/4072649411 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f0c004870 0x7f0f0c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:36.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 --2- 172.21.15.146:0/4072649411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f0c005230 0x7f0f0c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:36.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c00c110 con 0x7f0f0c005230 2024-09-13T18:26:36.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c00c280 con 0x7f0f0c003a90 2024-09-13T18:26:36.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c00c440 con 0x7f0f0c004870 2024-09-13T18:26:36.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f23fff700 1 --2- 172.21.15.146:0/4072649411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f0c005230 0x7f0f0c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:36.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f237fe700 1 --2- 172.21.15.146:0/4072649411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f0c003a90 0x7f0f0c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:36.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f22ffd700 1 --2- 172.21.15.146:0/4072649411 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f0c004870 0x7f0f0c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f23fff700 1 -- 172.21.15.146:0/4072649411 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f0c004870 msgr2=0x7f0f0c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f23fff700 1 --2- 172.21.15.146:0/4072649411 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f0c004870 0x7f0f0c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f23fff700 1 -- 172.21.15.146:0/4072649411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f0c003a90 msgr2=0x7f0f0c003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f23fff700 1 --2- 172.21.15.146:0/4072649411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f0c003a90 0x7f0f0c003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f23fff700 1 -- 172.21.15.146:0/4072649411 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0035d0 con 0x7f0f0c005230 2024-09-13T18:26:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f23fff700 1 -- 172.21.15.146:0/4072649411 reap_dead start 2024-09-13T18:26:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f23fff700 1 --2- 172.21.15.146:0/4072649411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f0c005230 0x7f0f0c00ba30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009fc0 tx=0x7f0f0c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d54e6492c63f68d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c003a40 con 0x7f0f0c005230 2024-09-13T18:26:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c00f040 con 0x7f0f0c005230 2024-09-13T18:26:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f227fc700 1 -- 172.21.15.146:0/4072649411 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c003e00 con 0x7f0f0c005230 2024-09-13T18:26:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f28957700 1 -- 172.21.15.146:0/4072649411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f0c005230 msgr2=0x7f0f0c00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f28957700 1 --2- 172.21.15.146:0/4072649411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f0c005230 0x7f0f0c00ba30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009fc0 tx=0x7f0f0c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f28957700 1 -- 172.21.15.146:0/4072649411 shutdown_connections 2024-09-13T18:26:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f28957700 1 --2- 172.21.15.146:0/4072649411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f0c005230 0x7f0f0c00ba30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f28957700 1 -- 172.21.15.146:0/4072649411 >> 172.21.15.146:0/4072649411 conn(0x7f0f24071ed0 msgr2=0x7f0f240722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f28957700 1 -- 172.21.15.146:0/4072649411 shutdown_connections 2024-09-13T18:26:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.529+0000 7f0f28957700 1 -- 172.21.15.146:0/4072649411 wait complete. 2024-09-13T18:26:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f28957700 1 Processor -- start 2024-09-13T18:26:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f28957700 1 -- start start 2024-09-13T18:26:36.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f28957700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f0c005230 0x7f0f24155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:36.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f28957700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f2414bac0 0x7f0f241f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:36.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f28957700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f240c2980 0x7f0f241455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:36.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f28957700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f24072810 con 0x7f0f0c005230 2024-09-13T18:26:36.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f28957700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2414abb0 con 0x7f0f2414bac0 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f28957700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f241565b0 con 0x7f0f240c2980 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f237fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f0c005230 0x7f0f24155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f237fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f0c005230 0x7f0f24155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60880/0 (socket says 172.21.15.146:60880) 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f237fe700 1 -- 172.21.15.146:0/732774518 learned_addr learned my addr 172.21.15.146:0/732774518 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f23fff700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f240c2980 0x7f0f241455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f22ffd700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f2414bac0 0x7f0f241f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f237fe700 1 -- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f240c2980 msgr2=0x7f0f241455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f237fe700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f240c2980 0x7f0f241455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f237fe700 1 -- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f2414bac0 msgr2=0x7f0f241f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f237fe700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f2414bac0 0x7f0f241f2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f237fe700 1 -- 172.21.15.146:0/732774518 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f241eef50 con 0x7f0f0c005230 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f23fff700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f240c2980 0x7f0f241455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f22ffd700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f2414bac0 0x7f0f241f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f237fe700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f0c005230 0x7f0f24155830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400eb50 tx=0x7f0f1400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f0bfff700 1 -- 172.21.15.146:0/732774518 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f14005e00 con 0x7f0f0c005230 2024-09-13T18:26:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f0bfff700 1 -- 172.21.15.146:0/732774518 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1400c760 con 0x7f0f0c005230 2024-09-13T18:26:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f28957700 1 -- 172.21.15.146:0/732774518 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f241562d0 con 0x7f0f0c005230 2024-09-13T18:26:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f0bfff700 1 -- 172.21.15.146:0/732774518 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1400caf0 con 0x7f0f0c005230 2024-09-13T18:26:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.533+0000 7f0f28957700 1 -- 172.21.15.146:0/732774518 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f241f2660 con 0x7f0f0c005230 2024-09-13T18:26:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.537+0000 7f0f28957700 1 -- 172.21.15.146:0/732774518 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f241f24c0 con 0x7f0f0c005230 2024-09-13T18:26:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.537+0000 7f0f0bfff700 1 -- 172.21.15.146:0/732774518 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f1400cc50 con 0x7f0f0c005230 2024-09-13T18:26:36.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.537+0000 7f0f0bfff700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0f10066730 0x7f0f10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:36.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.537+0000 7f0f0bfff700 1 -- 172.21.15.146:0/732774518 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f0f140841f0 con 0x7f0f0c005230 2024-09-13T18:26:36.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.537+0000 7f0f22ffd700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0f10066730 0x7f0f10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:36.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.541+0000 7f0f22ffd700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0f10066730 0x7f0f10068bf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0f241439f0 tx=0x7f0f18009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:36.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.545+0000 7f0f0bfff700 1 -- 172.21.15.146:0/732774518 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0f241f24c0 con 0x7f0f0c005230 2024-09-13T18:26:36.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.753+0000 7f0f28957700 1 -- 172.21.15.146:0/732774518 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f0f241f24c0 con 0x7f0f0c005230 2024-09-13T18:26:36.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.761+0000 7f0f0bfff700 1 -- 172.21.15.146:0/732774518 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v5) v1 ==== 137+0+0 (secure 0 0 0) 0x7f0f241f24c0 con 0x7f0f0c005230 2024-09-13T18:26:36.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.765+0000 7f0f28957700 1 -- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0f10066730 msgr2=0x7f0f10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.765+0000 7f0f28957700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0f10066730 0x7f0f10068bf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0f241439f0 tx=0x7f0f18009040 comp rx=0 tx=0).stop 2024-09-13T18:26:36.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.765+0000 7f0f28957700 1 -- 172.21.15.146:0/732774518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f0c005230 msgr2=0x7f0f24155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.765+0000 7f0f28957700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f0c005230 0x7f0f24155830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400eb50 tx=0x7f0f1400ef10 comp rx=0 tx=0).stop 2024-09-13T18:26:36.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.765+0000 7f0f28957700 1 -- 172.21.15.146:0/732774518 shutdown_connections 2024-09-13T18:26:36.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.765+0000 7f0f28957700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0f10066730 0x7f0f10068bf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.765+0000 7f0f28957700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f240c2980 0x7f0f241455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.765+0000 7f0f28957700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f2414bac0 0x7f0f241f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.769+0000 7f0f28957700 1 --2- 172.21.15.146:0/732774518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f0c005230 0x7f0f24155830 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.769+0000 7f0f28957700 1 -- 172.21.15.146:0/732774518 >> 172.21.15.146:0/732774518 conn(0x7f0f24071ed0 msgr2=0x7f0f24149980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:36.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.769+0000 7f0f28957700 1 -- 172.21.15.146:0/732774518 shutdown_connections 2024-09-13T18:26:36.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.769+0000 7f0f28957700 1 -- 172.21.15.146:0/732774518 wait complete. 2024-09-13T18:26:36.778 INFO:teuthology.orchestra.run.smithi146.stderr:added data pool 4 to fsmap 2024-09-13T18:26:36.785 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:26:36.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.885+0000 7f647e68d700 1 Processor -- start 2024-09-13T18:26:36.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.885+0000 7f647e68d700 1 -- start start 2024-09-13T18:26:36.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647e68d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f64780c6a80 0x7f6478161c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:36.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647e68d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6478153fa0 con 0x7f64780c6610 2024-09-13T18:26:36.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647e68d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64780aa500 con 0x7f6478153bc0 2024-09-13T18:26:36.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647e68d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647815a7e0 con 0x7f64780c6a80 2024-09-13T18:26:36.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647ce8a700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f64780c6610 0x7f64780cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37040/0 (socket says 172.21.15.146:37040) 2024-09-13T18:26:36.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647ce8a700 1 -- 172.21.15.146:0/4007062889 learned_addr learned my addr 172.21.15.146:0/4007062889 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:36.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647de8c700 1 --2- 172.21.15.146:0/4007062889 >> v2:172.21.15.146:3300/0 conn(0x7f64780c6a80 0x7f6478161c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:36.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f646ffff700 1 -- 172.21.15.146:0/4007062889 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474281495 0 0) 0x7f6478153fa0 con 0x7f64780c6610 2024-09-13T18:26:36.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f646ffff700 1 -- 172.21.15.146:0/4007062889 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64640035d0 con 0x7f64780c6610 2024-09-13T18:26:36.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647de8c700 1 -- 172.21.15.146:0/4007062889 >> v1:172.21.15.154:6789/0 conn(0x7f6478153bc0 legacy=0x7f64780c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647de8c700 1 -- 172.21.15.146:0/4007062889 >> v1:172.21.15.146:6789/0 conn(0x7f64780c6610 legacy=0x7f64780cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647de8c700 1 -- 172.21.15.146:0/4007062889 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64780aa220 con 0x7f64780c6a80 2024-09-13T18:26:36.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647de8c700 1 --2- 172.21.15.146:0/4007062889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780c6a80 0x7f6478161c30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6470009d90 tx=0x7f647000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5fb12f49585e51 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:36.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f646ffff700 1 -- 172.21.15.146:0/4007062889 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647000e070 con 0x7f64780c6a80 2024-09-13T18:26:36.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f646ffff700 1 -- 172.21.15.146:0/4007062889 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647000bb20 con 0x7f64780c6a80 2024-09-13T18:26:36.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f646ffff700 1 -- 172.21.15.146:0/4007062889 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6470012430 con 0x7f64780c6a80 2024-09-13T18:26:36.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647e68d700 1 -- 172.21.15.146:0/4007062889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780c6a80 msgr2=0x7f6478161c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647e68d700 1 --2- 172.21.15.146:0/4007062889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780c6a80 0x7f6478161c30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6470009d90 tx=0x7f647000b330 comp rx=0 tx=0).stop 2024-09-13T18:26:36.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647e68d700 1 -- 172.21.15.146:0/4007062889 shutdown_connections 2024-09-13T18:26:36.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647e68d700 1 --2- 172.21.15.146:0/4007062889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780c6a80 0x7f6478161c30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647e68d700 1 -- 172.21.15.146:0/4007062889 >> 172.21.15.146:0/4007062889 conn(0x7f6478066090 msgr2=0x7f6478066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:36.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647e68d700 1 -- 172.21.15.146:0/4007062889 shutdown_connections 2024-09-13T18:26:36.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.889+0000 7f647e68d700 1 -- 172.21.15.146:0/4007062889 wait complete. 2024-09-13T18:26:36.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.893+0000 7f647e68d700 1 Processor -- start 2024-09-13T18:26:36.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.893+0000 7f647e68d700 1 -- start start 2024-09-13T18:26:36.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.893+0000 7f647e68d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780c6610 0x7f64780c0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:36.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647e68d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64780c6a80 0x7f64780c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:36.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647e68d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6478153bc0 0x7f64780ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:36.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647e68d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64780c3a10 con 0x7f64780c6610 2024-09-13T18:26:36.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647e68d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64780abb90 con 0x7f64780c6a80 2024-09-13T18:26:36.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647e68d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64780c0d80 con 0x7f6478153bc0 2024-09-13T18:26:36.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647d68b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780c6610 0x7f64780c0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:36.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647d68b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780c6610 0x7f64780c0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60910/0 (socket says 172.21.15.146:60910) 2024-09-13T18:26:36.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647d68b700 1 -- 172.21.15.146:0/4218787114 learned_addr learned my addr 172.21.15.146:0/4218787114 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:36.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647ce8a700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64780c6a80 0x7f64780c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:36.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647de8c700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6478153bc0 0x7f64780ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:36.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647d68b700 1 -- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6478153bc0 msgr2=0x7f64780ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647d68b700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6478153bc0 0x7f64780ac020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647d68b700 1 -- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64780c6a80 msgr2=0x7f64780c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:36.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647d68b700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64780c6a80 0x7f64780c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:36.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647d68b700 1 -- 172.21.15.146:0/4218787114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6470009a70 con 0x7f64780c6610 2024-09-13T18:26:36.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647de8c700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6478153bc0 0x7f64780ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:36.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647ce8a700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64780c6a80 0x7f64780c21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:36.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647d68b700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780c6610 0x7f64780c0560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f647400c500 tx=0x7f647400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:36.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f646dffb700 1 -- 172.21.15.146:0/4218787114 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64740052d0 con 0x7f64780c6610 2024-09-13T18:26:36.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f646dffb700 1 -- 172.21.15.146:0/4218787114 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6474005430 con 0x7f64780c6610 2024-09-13T18:26:36.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647e68d700 1 -- 172.21.15.146:0/4218787114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64780c1020 con 0x7f64780c6610 2024-09-13T18:26:36.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f646dffb700 1 -- 172.21.15.146:0/4218787114 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64740055a0 con 0x7f64780c6610 2024-09-13T18:26:36.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647e68d700 1 -- 172.21.15.146:0/4218787114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64780c1980 con 0x7f64780c6610 2024-09-13T18:26:36.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f647e68d700 1 -- 172.21.15.146:0/4218787114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64780a9de0 con 0x7f64780c6610 2024-09-13T18:26:36.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.897+0000 7f646dffb700 1 -- 172.21.15.146:0/4218787114 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f64740038e0 con 0x7f64780c6610 2024-09-13T18:26:36.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.901+0000 7f646dffb700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f645c066760 0x7f645c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:36.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.901+0000 7f646dffb700 1 -- 172.21.15.146:0/4218787114 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6474083820 con 0x7f64780c6610 2024-09-13T18:26:36.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.901+0000 7f647ce8a700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f645c066760 0x7f645c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:36.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.901+0000 7f647ce8a700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f645c066760 0x7f645c068c20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f647814d4e0 tx=0x7f6468009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:36.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:36.905+0000 7f646dffb700 1 -- 172.21.15.146:0/4218787114 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6474050930 con 0x7f64780c6610 2024-09-13T18:26:37.058 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:26:37.061 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:26:37.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.109+0000 7f647e68d700 1 -- 172.21.15.146:0/4218787114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f647806f690 con 0x7f64780c6610 2024-09-13T18:26:37.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.113+0000 7f646dffb700 1 -- 172.21.15.146:0/4218787114 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+6253 (secure 0 0 0) 0x7f6474053f50 con 0x7f64780c6610 2024-09-13T18:26:37.118 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:37.118 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:26:36.757864+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:26:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.113+0000 7f647e68d700 1 -- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f645c066760 msgr2=0x7f645c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.113+0000 7f647e68d700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f645c066760 0x7f645c068c20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f647814d4e0 tx=0x7f6468009040 comp rx=0 tx=0).stop 2024-09-13T18:26:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.113+0000 7f647e68d700 1 -- 172.21.15.146:0/4218787114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780c6610 msgr2=0x7f64780c0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.113+0000 7f647e68d700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780c6610 0x7f64780c0560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f647400c500 tx=0x7f647400c8c0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.113+0000 7f647e68d700 1 -- 172.21.15.146:0/4218787114 shutdown_connections 2024-09-13T18:26:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.113+0000 7f647e68d700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f645c066760 0x7f645c068c20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.113+0000 7f647e68d700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6478153bc0 0x7f64780ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.113+0000 7f647e68d700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64780c6a80 0x7f64780c21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.113+0000 7f647e68d700 1 --2- 172.21.15.146:0/4218787114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780c6610 0x7f64780c0560 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.113+0000 7f647e68d700 1 -- 172.21.15.146:0/4218787114 >> 172.21.15.146:0/4218787114 conn(0x7f6478066090 msgr2=0x7f6478066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:37.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.113+0000 7f647e68d700 1 -- 172.21.15.146:0/4218787114 shutdown_connections 2024-09-13T18:26:37.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.113+0000 7f647e68d700 1 -- 172.21.15.146:0/4218787114 wait complete. 2024-09-13T18:26:37.122 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 5 2024-09-13T18:26:37.133 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:26:37.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d909e700 1 Processor -- start 2024-09-13T18:26:37.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d909e700 1 -- start start 2024-09-13T18:26:37.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d909e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f38d40cbd50 0x7f38d40b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d909e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38d40ca210 0x7f38d40cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d909e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d40cc670 con 0x7f38d4071390 2024-09-13T18:26:37.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d909e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d40caa30 con 0x7f38d40cbd50 2024-09-13T18:26:37.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d909e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40cb810 con 0x7f38d40ca210 2024-09-13T18:26:37.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d889d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f38d4071390 0x7f38d40701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37046/0 (socket says 172.21.15.146:37046) 2024-09-13T18:26:37.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d889d700 1 -- 172.21.15.146:0/1059114031 learned_addr learned my addr 172.21.15.146:0/1059114031 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:37.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d3fff700 1 --2- 172.21.15.146:0/1059114031 >> v2:172.21.15.154:3300/0 conn(0x7f38d40cbd50 0x7f38d40b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d37fe700 1 --2- 172.21.15.146:0/1059114031 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38d40ca210 0x7f38d40cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d2ffd700 1 -- 172.21.15.146:0/1059114031 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554928590 0 0) 0x7f38c8001110 con 0x7f38d4071390 2024-09-13T18:26:37.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d2ffd700 1 -- 172.21.15.146:0/1059114031 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38bc0035d0 con 0x7f38d4071390 2024-09-13T18:26:37.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d37fe700 1 -- 172.21.15.146:0/1059114031 >> v2:172.21.15.154:3300/0 conn(0x7f38d40cbd50 msgr2=0x7f38d40b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:37.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d37fe700 1 --2- 172.21.15.146:0/1059114031 >> v2:172.21.15.154:3300/0 conn(0x7f38d40cbd50 0x7f38d40b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d37fe700 1 -- 172.21.15.146:0/1059114031 >> v1:172.21.15.146:6789/0 conn(0x7f38d4071390 legacy=0x7f38d40701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d2ffd700 1 -- 172.21.15.146:0/1059114031 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2630116456 0 0) 0x7f38bc0035d0 con 0x7f38d4071390 2024-09-13T18:26:37.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d37fe700 1 -- 172.21.15.146:0/1059114031 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d415da20 con 0x7f38d40ca210 2024-09-13T18:26:37.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d37fe700 1 --2- 172.21.15.146:0/1059114031 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38d40ca210 0x7f38d40cc130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009d90 tx=0x7f38c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94895c8d3a130a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:37.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d2ffd700 1 -- 172.21.15.146:0/1059114031 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c400e070 con 0x7f38d40ca210 2024-09-13T18:26:37.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d2ffd700 1 -- 172.21.15.146:0/1059114031 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c4003a40 con 0x7f38d40ca210 2024-09-13T18:26:37.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.241+0000 7f38d2ffd700 1 -- 172.21.15.146:0/1059114031 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8001800 con 0x7f38d40ca210 2024-09-13T18:26:37.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 -- 172.21.15.146:0/1059114031 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38d40ca210 msgr2=0x7f38d40cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 --2- 172.21.15.146:0/1059114031 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38d40ca210 0x7f38d40cc130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009d90 tx=0x7f38c400b330 comp rx=0 tx=0).stop 2024-09-13T18:26:37.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 -- 172.21.15.146:0/1059114031 shutdown_connections 2024-09-13T18:26:37.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 --2- 172.21.15.146:0/1059114031 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38d40ca210 0x7f38d40cc130 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 --2- 172.21.15.146:0/1059114031 >> v2:172.21.15.154:3300/0 conn(0x7f38d40cbd50 0x7f38d40b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 -- 172.21.15.146:0/1059114031 >> 172.21.15.146:0/1059114031 conn(0x7f38d4066270 msgr2=0x7f38d4066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:37.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 -- 172.21.15.146:0/1059114031 shutdown_connections 2024-09-13T18:26:37.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 -- 172.21.15.146:0/1059114031 wait complete. 2024-09-13T18:26:37.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 Processor -- start 2024-09-13T18:26:37.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 -- start start 2024-09-13T18:26:37.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38d4071390 0x7f38d40c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38d40ca210 0x7f38d40c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38d40cbd50 0x7f38d40c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d41ef130 con 0x7f38d40ca210 2024-09-13T18:26:37.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d41f0100 con 0x7f38d4071390 2024-09-13T18:26:37.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d909e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d414d6e0 con 0x7f38d40cbd50 2024-09-13T18:26:37.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38d40ca210 0x7f38d40c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38d40ca210 0x7f38d40c67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60918/0 (socket says 172.21.15.146:60918) 2024-09-13T18:26:37.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d37fe700 1 -- 172.21.15.146:0/217762321 learned_addr learned my addr 172.21.15.146:0/217762321 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:37.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d3fff700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38d4071390 0x7f38d40c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d889d700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38d40cbd50 0x7f38d40c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d37fe700 1 -- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38d40cbd50 msgr2=0x7f38d40c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d37fe700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38d40cbd50 0x7f38d40c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d37fe700 1 -- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38d4071390 msgr2=0x7f38d40c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d37fe700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38d4071390 0x7f38d40c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.245+0000 7f38d37fe700 1 -- 172.21.15.146:0/217762321 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c4009a70 con 0x7f38d40ca210 2024-09-13T18:26:37.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.249+0000 7f38d37fe700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38d40ca210 0x7f38d40c67f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f38c400b680 tx=0x7f38c4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:37.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.249+0000 7f38d0ff9700 1 -- 172.21.15.146:0/217762321 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4004670 con 0x7f38d40ca210 2024-09-13T18:26:37.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.249+0000 7f38d0ff9700 1 -- 172.21.15.146:0/217762321 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c40047d0 con 0x7f38d40ca210 2024-09-13T18:26:37.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.249+0000 7f38d909e700 1 -- 172.21.15.146:0/217762321 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d41ef340 con 0x7f38d40ca210 2024-09-13T18:26:37.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.249+0000 7f38d0ff9700 1 -- 172.21.15.146:0/217762321 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4012670 con 0x7f38d40ca210 2024-09-13T18:26:37.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.249+0000 7f38d909e700 1 -- 172.21.15.146:0/217762321 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d41ef900 con 0x7f38d40ca210 2024-09-13T18:26:37.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.249+0000 7f38d909e700 1 -- 172.21.15.146:0/217762321 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d414a640 con 0x7f38d40ca210 2024-09-13T18:26:37.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.249+0000 7f38d0ff9700 1 -- 172.21.15.146:0/217762321 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f38c40127d0 con 0x7f38d40ca210 2024-09-13T18:26:37.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.249+0000 7f38d0ff9700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f38c0066760 0x7f38c0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.249+0000 7f38d0ff9700 1 -- 172.21.15.146:0/217762321 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f38c4086140 con 0x7f38d40ca210 2024-09-13T18:26:37.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.249+0000 7f38d3fff700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f38c0066760 0x7f38c0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.253+0000 7f38d3fff700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f38c0066760 0x7f38c0068c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009ae0 tx=0x7f38cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:37.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.257+0000 7f38d0ff9700 1 -- 172.21.15.146:0/217762321 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f38c4056c50 con 0x7f38d40ca210 2024-09-13T18:26:37.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.437+0000 7f38d909e700 1 -- 172.21.15.146:0/217762321 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38d415d500 con 0x7f38d40ca210 2024-09-13T18:26:37.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.441+0000 7f38d0ff9700 1 -- 172.21.15.146:0/217762321 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f38c40567e0 con 0x7f38d40ca210 2024-09-13T18:26:37.447 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:37.448 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:26:37.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.441+0000 7f38d909e700 1 -- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f38c0066760 msgr2=0x7f38c0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.441+0000 7f38d909e700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f38c0066760 0x7f38c0068c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009ae0 tx=0x7f38cc008040 comp rx=0 tx=0).stop 2024-09-13T18:26:37.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.441+0000 7f38d909e700 1 -- 172.21.15.146:0/217762321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38d40ca210 msgr2=0x7f38d40c67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.441+0000 7f38d909e700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38d40ca210 0x7f38d40c67f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f38c400b680 tx=0x7f38c4004af0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.441+0000 7f38d909e700 1 -- 172.21.15.146:0/217762321 shutdown_connections 2024-09-13T18:26:37.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.441+0000 7f38d909e700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f38c0066760 0x7f38c0068c20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.441+0000 7f38d909e700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38d40cbd50 0x7f38d40c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.441+0000 7f38d909e700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38d40ca210 0x7f38d40c67f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.441+0000 7f38d909e700 1 --2- 172.21.15.146:0/217762321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38d4071390 0x7f38d40c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.441+0000 7f38d909e700 1 -- 172.21.15.146:0/217762321 >> 172.21.15.146:0/217762321 conn(0x7f38d4066270 msgr2=0x7f38d40b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:37.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.445+0000 7f38d909e700 1 -- 172.21.15.146:0/217762321 shutdown_connections 2024-09-13T18:26:37.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.445+0000 7f38d909e700 1 -- 172.21.15.146:0/217762321 wait complete. 2024-09-13T18:26:37.462 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:26:37.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f891b4f3700 1 Processor -- start 2024-09-13T18:26:37.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f891b4f3700 1 -- start start 2024-09-13T18:26:37.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f891b4f3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f89140ca020 0x7f89140ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f891b4f3700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89140c5bd0 con 0x7f89140cad90 2024-09-13T18:26:37.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f891b4f3700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89140c9bb0 con 0x7f89140c57f0 2024-09-13T18:26:37.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f891b4f3700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891415a160 con 0x7f89140ca020 2024-09-13T18:26:37.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8919a90700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f89140cad90 0x7f89140cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37050/0 (socket says 172.21.15.146:37050) 2024-09-13T18:26:37.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8919a90700 1 -- 172.21.15.146:0/973877705 learned_addr learned my addr 172.21.15.146:0/973877705 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:37.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8918a8e700 1 --2- 172.21.15.146:0/973877705 >> v2:172.21.15.154:3300/0 conn(0x7f89140ca020 0x7f89140ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011977109 0 0) 0x7f89140c5bd0 con 0x7f89140cad90 2024-09-13T18:26:37.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88fc0035d0 con 0x7f89140cad90 2024-09-13T18:26:37.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4076720127 0 0) 0x7f88fc0035d0 con 0x7f89140cad90 2024-09-13T18:26:37.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 >> v2:172.21.15.154:3300/0 conn(0x7f89140ca020 msgr2=0x7f89140ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:37.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 --2- 172.21.15.146:0/973877705 >> v2:172.21.15.154:3300/0 conn(0x7f89140ca020 0x7f89140ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 >> v1:172.21.15.154:6789/0 conn(0x7f89140c57f0 legacy=0x7f89140cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8914159830 con 0x7f89140cad90 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f8904001670 con 0x7f89140cad90 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 >> v1:172.21.15.146:6789/0 conn(0x7f89140cad90 legacy=0x7f89140cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 --2- 172.21.15.146:0/973877705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88fc003a20 0x7f88fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 --2- 172.21.15.146:0/973877705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88fc0043d0 0x7f88fc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 --2- 172.21.15.146:0/973877705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc005090 0x7f88fc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891415a160 con 0x7f88fc005090 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc00bf70 con 0x7f88fc0043d0 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc00c100 con 0x7f88fc003a20 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8919a90700 1 --2- 172.21.15.146:0/973877705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc005090 0x7f88fc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f891928f700 1 --2- 172.21.15.146:0/973877705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88fc003a20 0x7f88fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8918a8e700 1 --2- 172.21.15.146:0/973877705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88fc0043d0 0x7f88fc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8919a90700 1 -- 172.21.15.146:0/973877705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88fc003a20 msgr2=0x7f88fc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:37.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8919a90700 1 --2- 172.21.15.146:0/973877705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88fc003a20 0x7f88fc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8919a90700 1 -- 172.21.15.146:0/973877705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88fc0043d0 msgr2=0x7f88fc004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:37.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8919a90700 1 --2- 172.21.15.146:0/973877705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88fc0043d0 0x7f88fc004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8919a90700 1 -- 172.21.15.146:0/973877705 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc0035d0 con 0x7f88fc005090 2024-09-13T18:26:37.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f891928f700 1 --2- 172.21.15.146:0/973877705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88fc003a20 0x7f88fc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:37.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8919a90700 1 -- 172.21.15.146:0/973877705 reap_dead start 2024-09-13T18:26:37.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8919a90700 1 --2- 172.21.15.146:0/973877705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc005090 0x7f88fc00ba30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89040050f0 tx=0x7f88fc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863aabbbd2d8ebed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:37.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8904005b80 con 0x7f88fc005090 2024-09-13T18:26:37.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8904005ce0 con 0x7f88fc005090 2024-09-13T18:26:37.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f8913fff700 1 -- 172.21.15.146:0/973877705 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8904008300 con 0x7f88fc005090 2024-09-13T18:26:37.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f891b4f3700 1 -- 172.21.15.146:0/973877705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc005090 msgr2=0x7f88fc00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.569+0000 7f891b4f3700 1 --2- 172.21.15.146:0/973877705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc005090 0x7f88fc00ba30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89040050f0 tx=0x7f88fc0058a0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891b4f3700 1 -- 172.21.15.146:0/973877705 shutdown_connections 2024-09-13T18:26:37.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891b4f3700 1 --2- 172.21.15.146:0/973877705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc005090 0x7f88fc00ba30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891b4f3700 1 -- 172.21.15.146:0/973877705 >> 172.21.15.146:0/973877705 conn(0x7f8914071d30 msgr2=0x7f8914072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:37.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891b4f3700 1 -- 172.21.15.146:0/973877705 shutdown_connections 2024-09-13T18:26:37.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891b4f3700 1 -- 172.21.15.146:0/973877705 wait complete. 2024-09-13T18:26:37.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891b4f3700 1 Processor -- start 2024-09-13T18:26:37.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891b4f3700 1 -- start start 2024-09-13T18:26:37.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891b4f3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc005090 0x7f89140c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891b4f3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89141564d0 0x7f89140aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891b4f3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f891414f3f0 0x7f891414f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891b4f3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891414fde0 con 0x7f88fc005090 2024-09-13T18:26:37.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891b4f3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89140ab0c0 con 0x7f891414f3f0 2024-09-13T18:26:37.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891b4f3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89141eef20 con 0x7f89141564d0 2024-09-13T18:26:37.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891928f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc005090 0x7f89140c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f8919a90700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f891414f3f0 0x7f891414f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891928f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc005090 0x7f89140c6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60948/0 (socket says 172.21.15.146:60948) 2024-09-13T18:26:37.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891928f700 1 -- 172.21.15.146:0/1666119319 learned_addr learned my addr 172.21.15.146:0/1666119319 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:37.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f8918a8e700 1 --2- 172.21.15.146:0/1666119319 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89141564d0 0x7f89140aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891928f700 1 -- 172.21.15.146:0/1666119319 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89141564d0 msgr2=0x7f89140aa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891928f700 1 --2- 172.21.15.146:0/1666119319 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89141564d0 0x7f89140aa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891928f700 1 -- 172.21.15.146:0/1666119319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f891414f3f0 msgr2=0x7f891414f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891928f700 1 --2- 172.21.15.146:0/1666119319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f891414f3f0 0x7f891414f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.573+0000 7f891928f700 1 -- 172.21.15.146:0/1666119319 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89141ef200 con 0x7f88fc005090 2024-09-13T18:26:37.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.577+0000 7f891928f700 1 --2- 172.21.15.146:0/1666119319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc005090 0x7f89140c6650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f890c001fc0 tx=0x7f890c00c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:37.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.577+0000 7f8911ffb700 1 -- 172.21.15.146:0/1666119319 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c003c30 con 0x7f88fc005090 2024-09-13T18:26:37.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.577+0000 7f8911ffb700 1 -- 172.21.15.146:0/1666119319 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890c003d90 con 0x7f88fc005090 2024-09-13T18:26:37.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.577+0000 7f891b4f3700 1 -- 172.21.15.146:0/1666119319 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89141ef4e0 con 0x7f88fc005090 2024-09-13T18:26:37.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.577+0000 7f8911ffb700 1 -- 172.21.15.146:0/1666119319 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c00cb60 con 0x7f88fc005090 2024-09-13T18:26:37.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.577+0000 7f891b4f3700 1 -- 172.21.15.146:0/1666119319 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89141eebd0 con 0x7f88fc005090 2024-09-13T18:26:37.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.577+0000 7f891b4f3700 1 -- 172.21.15.146:0/1666119319 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89141efa00 con 0x7f88fc005090 2024-09-13T18:26:37.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.577+0000 7f8911ffb700 1 -- 172.21.15.146:0/1666119319 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f890c00ccc0 con 0x7f88fc005090 2024-09-13T18:26:37.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.577+0000 7f8911ffb700 1 --2- 172.21.15.146:0/1666119319 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8900066760 0x7f8900068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.577+0000 7f8911ffb700 1 -- 172.21.15.146:0/1666119319 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f890c084690 con 0x7f88fc005090 2024-09-13T18:26:37.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.577+0000 7f8918a8e700 1 --2- 172.21.15.146:0/1666119319 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8900066760 0x7f8900068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.581+0000 7f8918a8e700 1 --2- 172.21.15.146:0/1666119319 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8900066760 0x7f8900068c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f89140c7360 tx=0x7f8908008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:37.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.589+0000 7f8911ffb700 1 -- 172.21.15.146:0/1666119319 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f89141efa00 con 0x7f88fc005090 2024-09-13T18:26:37.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.765+0000 7f891b4f3700 1 -- 172.21.15.146:0/1666119319 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89140cb570 con 0x7f88fc005090 2024-09-13T18:26:37.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.765+0000 7f8911ffb700 1 -- 172.21.15.146:0/1666119319 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f890c0517a0 con 0x7f88fc005090 2024-09-13T18:26:37.773 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:37.773 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:26:37.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.769+0000 7f891b4f3700 1 -- 172.21.15.146:0/1666119319 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8900066760 msgr2=0x7f8900068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.769+0000 7f891b4f3700 1 --2- 172.21.15.146:0/1666119319 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8900066760 0x7f8900068c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f89140c7360 tx=0x7f8908008040 comp rx=0 tx=0).stop 2024-09-13T18:26:37.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.769+0000 7f891b4f3700 1 -- 172.21.15.146:0/1666119319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc005090 msgr2=0x7f89140c6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.769+0000 7f891b4f3700 1 --2- 172.21.15.146:0/1666119319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc005090 0x7f89140c6650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f890c001fc0 tx=0x7f890c00c3c0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.769+0000 7f891b4f3700 1 -- 172.21.15.146:0/1666119319 shutdown_connections 2024-09-13T18:26:37.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.769+0000 7f891b4f3700 1 --2- 172.21.15.146:0/1666119319 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8900066760 0x7f8900068c20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.769+0000 7f891b4f3700 1 --2- 172.21.15.146:0/1666119319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f891414f3f0 0x7f891414f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.769+0000 7f891b4f3700 1 --2- 172.21.15.146:0/1666119319 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89141564d0 0x7f89140aa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.769+0000 7f891b4f3700 1 --2- 172.21.15.146:0/1666119319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc005090 0x7f89140c6650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.769+0000 7f891b4f3700 1 -- 172.21.15.146:0/1666119319 >> 172.21.15.146:0/1666119319 conn(0x7f8914071d30 msgr2=0x7f8914072110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:37.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.769+0000 7f891b4f3700 1 -- 172.21.15.146:0/1666119319 shutdown_connections 2024-09-13T18:26:37.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.769+0000 7f891b4f3700 1 -- 172.21.15.146:0/1666119319 wait complete. 2024-09-13T18:26:37.804 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:26:37.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcafa83e700 1 Processor -- start 2024-09-13T18:26:37.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcafa83e700 1 -- start start 2024-09-13T18:26:37.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcafa83e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcaf40caed0 0x7fcaf40723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcafa83e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcaf40cb2b0 0x7fcaf41619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcafa83e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf4161550 con 0x7fcaf40a9840 2024-09-13T18:26:37.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcafa83e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf4072a30 con 0x7fcaf40caed0 2024-09-13T18:26:37.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcafa83e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf40aa800 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcaf903b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcaf40cb2b0 0x7fcaf41619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcaf983c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcaf40caed0 0x7fcaf40723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcaf903b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcaf40cb2b0 0x7fcaf41619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:44470/0 (socket says 172.21.15.146:44470) 2024-09-13T18:26:37.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcaf903b700 1 -- 172.21.15.146:0/3154384020 learned_addr learned my addr 172.21.15.146:0/3154384020 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:37.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcaf903b700 1 -- 172.21.15.146:0/3154384020 >> v2:172.21.15.154:3300/0 conn(0x7fcaf40caed0 msgr2=0x7fcaf40723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:37.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcaf903b700 1 --2- 172.21.15.146:0/3154384020 >> v2:172.21.15.154:3300/0 conn(0x7fcaf40caed0 0x7fcaf40723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcaf903b700 1 -- 172.21.15.146:0/3154384020 >> v1:172.21.15.154:6789/0 conn(0x7fcaf40a9840 legacy=0x7fcaf415e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcaf903b700 1 -- 172.21.15.146:0/3154384020 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf4161220 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcaf903b700 1 --2- 172.21.15.146:0/3154384020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcaf40cb2b0 0x7fcaf41619e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009e50 tx=0x7fcaf000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac2f09c1d7d2409 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:37.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.917+0000 7fcaf883a700 1 -- 172.21.15.146:0/3154384020 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000e070 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcaf883a700 1 -- 172.21.15.146:0/3154384020 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf000b910 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcaf883a700 1 -- 172.21.15.146:0/3154384020 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000bcc0 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 -- 172.21.15.146:0/3154384020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcaf40cb2b0 msgr2=0x7fcaf41619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 --2- 172.21.15.146:0/3154384020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcaf40cb2b0 0x7fcaf41619e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009e50 tx=0x7fcaf000b330 comp rx=0 tx=0).stop 2024-09-13T18:26:37.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 -- 172.21.15.146:0/3154384020 shutdown_connections 2024-09-13T18:26:37.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 --2- 172.21.15.146:0/3154384020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcaf40cb2b0 0x7fcaf41619e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 --2- 172.21.15.146:0/3154384020 >> v2:172.21.15.154:3300/0 conn(0x7fcaf40caed0 0x7fcaf40723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 -- 172.21.15.146:0/3154384020 >> 172.21.15.146:0/3154384020 conn(0x7fcaf4066090 msgr2=0x7fcaf4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:37.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 -- 172.21.15.146:0/3154384020 shutdown_connections 2024-09-13T18:26:37.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 -- 172.21.15.146:0/3154384020 wait complete. 2024-09-13T18:26:37.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 Processor -- start 2024-09-13T18:26:37.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 -- start start 2024-09-13T18:26:37.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf40a9840 0x7fcaf40aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcaf40caed0 0x7fcaf41f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcaf40cb2b0 0x7fcaf40ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf41f3bf0 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf41f3550 con 0x7fcaf40a9840 2024-09-13T18:26:37.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa83e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf40af840 con 0x7fcaf40caed0 2024-09-13T18:26:37.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa03d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcaf40cb2b0 0x7fcaf40ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa03d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcaf40cb2b0 0x7fcaf40ad1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60964/0 (socket says 172.21.15.146:60964) 2024-09-13T18:26:37.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcaf983c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf40a9840 0x7fcaf40aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcaf903b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcaf40caed0 0x7fcaf41f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.921+0000 7fcafa03d700 1 -- 172.21.15.146:0/435747878 learned_addr learned my addr 172.21.15.146:0/435747878 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:37.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.925+0000 7fcafa03d700 1 -- 172.21.15.146:0/435747878 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcaf40caed0 msgr2=0x7fcaf41f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.925+0000 7fcafa03d700 1 --2- 172.21.15.146:0/435747878 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcaf40caed0 0x7fcaf41f4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.925+0000 7fcafa03d700 1 -- 172.21.15.146:0/435747878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf40a9840 msgr2=0x7fcaf40aa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:37.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.925+0000 7fcafa03d700 1 --2- 172.21.15.146:0/435747878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf40a9840 0x7fcaf40aa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:37.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.925+0000 7fcafa03d700 1 -- 172.21.15.146:0/435747878 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf0009b30 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.925+0000 7fcaf903b700 1 --2- 172.21.15.146:0/435747878 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcaf40caed0 0x7fcaf41f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:37.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.925+0000 7fcafa03d700 1 --2- 172.21.15.146:0/435747878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcaf40cb2b0 0x7fcaf40ad1d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00c430 tx=0x7fcaec00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:37.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.925+0000 7fcaea7fc700 1 -- 172.21.15.146:0/435747878 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0052d0 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.925+0000 7fcaea7fc700 1 -- 172.21.15.146:0/435747878 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec005430 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.925+0000 7fcafa83e700 1 -- 172.21.15.146:0/435747878 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf40afb20 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.925+0000 7fcaea7fc700 1 -- 172.21.15.146:0/435747878 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0055a0 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.925+0000 7fcafa83e700 1 -- 172.21.15.146:0/435747878 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf41f3810 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.925+0000 7fcafa83e700 1 -- 172.21.15.146:0/435747878 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf41f3d30 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.929+0000 7fcaea7fc700 1 -- 172.21.15.146:0/435747878 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcaec0038e0 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.929+0000 7fcaea7fc700 1 --2- 172.21.15.146:0/435747878 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcad8066730 0x7fcad8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:37.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.929+0000 7fcaea7fc700 1 -- 172.21.15.146:0/435747878 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcaec083640 con 0x7fcaf40cb2b0 2024-09-13T18:26:37.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.929+0000 7fcaf983c700 1 --2- 172.21.15.146:0/435747878 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcad8066730 0x7fcad8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:37.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.929+0000 7fcaf983c700 1 --2- 172.21.15.146:0/435747878 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcad8066730 0x7fcad8068bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009b50 tx=0x7fcae4009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:37.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:37.933+0000 7fcaea7fc700 1 -- 172.21.15.146:0/435747878 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcaec050750 con 0x7fcaf40cb2b0 2024-09-13T18:26:38.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcafa83e700 1 -- 172.21.15.146:0/435747878 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcaf40ca4f0 con 0x7fcaf40cb2b0 2024-09-13T18:26:38.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcaea7fc700 1 -- 172.21.15.146:0/435747878 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+6251 (secure 0 0 0) 0x7fcaec053d70 con 0x7fcaf40cb2b0 2024-09-13T18:26:38.147 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:38.147 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:26:37.760214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-13T18:26:38.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcafa83e700 1 -- 172.21.15.146:0/435747878 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcad8066730 msgr2=0x7fcad8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:38.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcafa83e700 1 --2- 172.21.15.146:0/435747878 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcad8066730 0x7fcad8068bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009b50 tx=0x7fcae4009240 comp rx=0 tx=0).stop 2024-09-13T18:26:38.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcafa83e700 1 -- 172.21.15.146:0/435747878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcaf40cb2b0 msgr2=0x7fcaf40ad1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:38.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcafa83e700 1 --2- 172.21.15.146:0/435747878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcaf40cb2b0 0x7fcaf40ad1d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00c430 tx=0x7fcaec00c740 comp rx=0 tx=0).stop 2024-09-13T18:26:38.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcafa83e700 1 -- 172.21.15.146:0/435747878 shutdown_connections 2024-09-13T18:26:38.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcafa83e700 1 --2- 172.21.15.146:0/435747878 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcad8066730 0x7fcad8068bf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:38.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcafa83e700 1 --2- 172.21.15.146:0/435747878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcaf40cb2b0 0x7fcaf40ad1d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:38.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcafa83e700 1 --2- 172.21.15.146:0/435747878 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcaf40caed0 0x7fcaf41f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:38.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcafa83e700 1 --2- 172.21.15.146:0/435747878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf40a9840 0x7fcaf40aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:38.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcafa83e700 1 -- 172.21.15.146:0/435747878 >> 172.21.15.146:0/435747878 conn(0x7fcaf4066090 msgr2=0x7fcaf415a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:38.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcafa83e700 1 -- 172.21.15.146:0/435747878 shutdown_connections 2024-09-13T18:26:38.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.141+0000 7fcafa83e700 1 -- 172.21.15.146:0/435747878 wait complete. 2024-09-13T18:26:38.150 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 7 2024-09-13T18:26:38.161 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:26:38.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.265+0000 7fddd2cc8700 1 Processor -- start 2024-09-13T18:26:38.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.265+0000 7fddd2cc8700 1 -- start start 2024-09-13T18:26:38.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd2cc8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fddcc0ca210 0x7fddcc0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:38.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd2cc8700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc15da20 con 0x7fddcc071390 2024-09-13T18:26:38.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd2cc8700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc0ca750 con 0x7fddcc0cbd50 2024-09-13T18:26:38.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd2cc8700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddcc0caa30 con 0x7fddcc0ca210 2024-09-13T18:26:38.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd24c7700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fddcc071390 0x7fddcc163e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37058/0 (socket says 172.21.15.146:37058) 2024-09-13T18:26:38.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd24c7700 1 -- 172.21.15.146:0/835587684 learned_addr learned my addr 172.21.15.146:0/835587684 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:38.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd14c5700 1 --2- 172.21.15.146:0/835587684 >> v2:172.21.15.154:3300/0 conn(0x7fddcc0ca210 0x7fddcc0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:38.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd0cc4700 1 -- 172.21.15.146:0/835587684 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890952043 0 0) 0x7fddcc15da20 con 0x7fddcc071390 2024-09-13T18:26:38.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd0cc4700 1 -- 172.21.15.146:0/835587684 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddb40035d0 con 0x7fddcc071390 2024-09-13T18:26:38.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd0cc4700 1 -- 172.21.15.146:0/835587684 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356334122 0 0) 0x7fddcc0ca750 con 0x7fddcc0cbd50 2024-09-13T18:26:38.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd0cc4700 1 -- 172.21.15.146:0/835587684 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc15da20 con 0x7fddcc0cbd50 2024-09-13T18:26:38.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd14c5700 1 -- 172.21.15.146:0/835587684 >> v1:172.21.15.154:6789/0 conn(0x7fddcc0cbd50 legacy=0x7fddcc0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:38.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd14c5700 1 -- 172.21.15.146:0/835587684 >> v1:172.21.15.146:6789/0 conn(0x7fddcc071390 legacy=0x7fddcc163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:38.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd14c5700 1 -- 172.21.15.146:0/835587684 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc15cc60 con 0x7fddcc0ca210 2024-09-13T18:26:38.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd14c5700 1 --2- 172.21.15.146:0/835587684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddcc0ca210 0x7fddcc0b1c40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009d90 tx=0x7fddc000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b524b2cd241d5d41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:38.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd0cc4700 1 -- 172.21.15.146:0/835587684 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc000e070 con 0x7fddcc0ca210 2024-09-13T18:26:38.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd0cc4700 1 -- 172.21.15.146:0/835587684 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc000bb20 con 0x7fddcc0ca210 2024-09-13T18:26:38.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd0cc4700 1 -- 172.21.15.146:0/835587684 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc000be90 con 0x7fddcc0ca210 2024-09-13T18:26:38.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd2cc8700 1 -- 172.21.15.146:0/835587684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddcc0ca210 msgr2=0x7fddcc0b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:38.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd2cc8700 1 --2- 172.21.15.146:0/835587684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddcc0ca210 0x7fddcc0b1c40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009d90 tx=0x7fddc000b330 comp rx=0 tx=0).stop 2024-09-13T18:26:38.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd2cc8700 1 -- 172.21.15.146:0/835587684 shutdown_connections 2024-09-13T18:26:38.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd2cc8700 1 --2- 172.21.15.146:0/835587684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddcc0ca210 0x7fddcc0b1c40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:38.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd2cc8700 1 -- 172.21.15.146:0/835587684 >> 172.21.15.146:0/835587684 conn(0x7fddcc066270 msgr2=0x7fddcc066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:38.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd2cc8700 1 -- 172.21.15.146:0/835587684 shutdown_connections 2024-09-13T18:26:38.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.269+0000 7fddd2cc8700 1 -- 172.21.15.146:0/835587684 wait complete. 2024-09-13T18:26:38.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.273+0000 7fddd2cc8700 1 Processor -- start 2024-09-13T18:26:38.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.273+0000 7fddd2cc8700 1 -- start start 2024-09-13T18:26:38.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.273+0000 7fddd2cc8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddcc071390 0x7fddcc15c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:38.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.273+0000 7fddd2cc8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddcc0ca210 0x7fddcc15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:38.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.273+0000 7fddd2cc8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddcc0cbd50 0x7fddcc147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:38.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.273+0000 7fddd2cc8700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc0c6c60 con 0x7fddcc0ca210 2024-09-13T18:26:38.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.273+0000 7fddd2cc8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc0c91b0 con 0x7fddcc071390 2024-09-13T18:26:38.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.273+0000 7fddd2cc8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc0c7170 con 0x7fddcc0cbd50 2024-09-13T18:26:38.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.273+0000 7fddd14c5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddcc0ca210 0x7fddcc15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:38.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.273+0000 7fddd14c5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddcc0ca210 0x7fddcc15b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60974/0 (socket says 172.21.15.146:60974) 2024-09-13T18:26:38.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.273+0000 7fddd14c5700 1 -- 172.21.15.146:0/1315434886 learned_addr learned my addr 172.21.15.146:0/1315434886 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:38.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.273+0000 7fddd1cc6700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddcc071390 0x7fddcc15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:38.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.273+0000 7fddd24c7700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddcc0cbd50 0x7fddcc147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:38.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddd14c5700 1 -- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddcc0cbd50 msgr2=0x7fddcc147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:38.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddd14c5700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddcc0cbd50 0x7fddcc147b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:38.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddd14c5700 1 -- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddcc071390 msgr2=0x7fddcc15c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:38.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddd14c5700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddcc071390 0x7fddcc15c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:38.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddd14c5700 1 -- 172.21.15.146:0/1315434886 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0009a70 con 0x7fddcc0ca210 2024-09-13T18:26:38.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddd14c5700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddcc0ca210 0x7fddcc15b950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fddc0000f80 tx=0x7fddc0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:38.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddca7fc700 1 -- 172.21.15.146:0/1315434886 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0025600 con 0x7fddcc0ca210 2024-09-13T18:26:38.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddca7fc700 1 -- 172.21.15.146:0/1315434886 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc001f070 con 0x7fddcc0ca210 2024-09-13T18:26:38.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddd2cc8700 1 -- 172.21.15.146:0/1315434886 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddcc15ca40 con 0x7fddcc0ca210 2024-09-13T18:26:38.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddca7fc700 1 -- 172.21.15.146:0/1315434886 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc00259c0 con 0x7fddcc0ca210 2024-09-13T18:26:38.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddd2cc8700 1 -- 172.21.15.146:0/1315434886 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddcc0c8880 con 0x7fddcc0ca210 2024-09-13T18:26:38.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddd2cc8700 1 -- 172.21.15.146:0/1315434886 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddcc14a870 con 0x7fddcc0ca210 2024-09-13T18:26:38.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddca7fc700 1 -- 172.21.15.146:0/1315434886 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fddc001b3f0 con 0x7fddcc0ca210 2024-09-13T18:26:38.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.277+0000 7fddca7fc700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fddb8066730 0x7fddb8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:38.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.281+0000 7fddca7fc700 1 -- 172.21.15.146:0/1315434886 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fddc0085c20 con 0x7fddcc0ca210 2024-09-13T18:26:38.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.281+0000 7fddd1cc6700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fddb8066730 0x7fddb8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:38.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.281+0000 7fddd1cc6700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fddb8066730 0x7fddb8068bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0011e0 tx=0x7fddbc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:38.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.285+0000 7fddca7fc700 1 -- 172.21.15.146:0/1315434886 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fddcc14a870 con 0x7fddcc0ca210 2024-09-13T18:26:38.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.461+0000 7fddd2cc8700 1 -- 172.21.15.146:0/1315434886 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddcc14a870 con 0x7fddcc0ca210 2024-09-13T18:26:38.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.465+0000 7fddca7fc700 1 -- 172.21.15.146:0/1315434886 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7fddcc14a870 con 0x7fddcc0ca210 2024-09-13T18:26:38.471 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:38.471 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:26:38.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.465+0000 7fddd2cc8700 1 -- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fddb8066730 msgr2=0x7fddb8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:38.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.465+0000 7fddd2cc8700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fddb8066730 0x7fddb8068bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0011e0 tx=0x7fddbc009040 comp rx=0 tx=0).stop 2024-09-13T18:26:38.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.469+0000 7fddd2cc8700 1 -- 172.21.15.146:0/1315434886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddcc0ca210 msgr2=0x7fddcc15b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:38.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.469+0000 7fddd2cc8700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddcc0ca210 0x7fddcc15b950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fddc0000f80 tx=0x7fddc0004af0 comp rx=0 tx=0).stop 2024-09-13T18:26:38.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.469+0000 7fddd2cc8700 1 -- 172.21.15.146:0/1315434886 shutdown_connections 2024-09-13T18:26:38.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.469+0000 7fddd2cc8700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fddb8066730 0x7fddb8068bf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:38.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.469+0000 7fddd2cc8700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddcc0cbd50 0x7fddcc147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:38.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.469+0000 7fddd2cc8700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddcc0ca210 0x7fddcc15b950 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:38.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.469+0000 7fddd2cc8700 1 --2- 172.21.15.146:0/1315434886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddcc071390 0x7fddcc15c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:38.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.469+0000 7fddd2cc8700 1 -- 172.21.15.146:0/1315434886 >> 172.21.15.146:0/1315434886 conn(0x7fddcc066270 msgr2=0x7fddcc0b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:38.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.469+0000 7fddd2cc8700 1 -- 172.21.15.146:0/1315434886 shutdown_connections 2024-09-13T18:26:38.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:38.469+0000 7fddd2cc8700 1 -- 172.21.15.146:0/1315434886 wait complete. 2024-09-13T18:26:39.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:26:32.323712+0000', 'modified': '2024-09-13T18:26:37.760214+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4290}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4290': {'gid': 4290, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.146:6837/31041557', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 31041557}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 31041557}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-09-13T18:26:39.092 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T18:26:39.092 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T18:26:39.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd66ac6a700 1 Processor -- start 2024-09-13T18:26:39.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd66ac6a700 1 -- start start 2024-09-13T18:26:39.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd66ac6a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd664153810 0x7fd6640c8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd66ac6a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd6640c9d00 0x7fd6640ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd66ac6a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd66415cbd0 0x7fd66415d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd66ac6a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6641595e0 con 0x7fd66415cbd0 2024-09-13T18:26:39.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd66ac6a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6640c43c0 con 0x7fd6640c9d00 2024-09-13T18:26:39.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd66ac6a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640c8c70 con 0x7fd664153810 2024-09-13T18:26:39.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd66a469700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd66415cbd0 0x7fd66415d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd66a469700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd66415cbd0 0x7fd66415d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60984/0 (socket says 172.21.15.146:60984) 2024-09-13T18:26:39.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd669c68700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd664153810 0x7fd6640c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd66a469700 1 -- 172.21.15.146:0/141696908 learned_addr learned my addr 172.21.15.146:0/141696908 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:39.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd669467700 1 --2- 172.21.15.146:0/141696908 >> v2:172.21.15.154:3300/0 conn(0x7fd6640c9d00 0x7fd6640ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd669c68700 1 -- 172.21.15.146:0/141696908 >> v2:172.21.15.154:3300/0 conn(0x7fd6640c9d00 msgr2=0x7fd6640ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:39.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd669c68700 1 --2- 172.21.15.146:0/141696908 >> v2:172.21.15.154:3300/0 conn(0x7fd6640c9d00 0x7fd6640ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd669c68700 1 -- 172.21.15.146:0/141696908 >> v2:172.21.15.146:3300/0 conn(0x7fd66415cbd0 msgr2=0x7fd66415d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:39.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd669c68700 1 --2- 172.21.15.146:0/141696908 >> v2:172.21.15.146:3300/0 conn(0x7fd66415cbd0 0x7fd66415d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd669c68700 1 -- 172.21.15.146:0/141696908 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6641597d0 con 0x7fd664153810 2024-09-13T18:26:39.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.209+0000 7fd66a469700 1 --2- 172.21.15.146:0/141696908 >> v2:172.21.15.146:3300/0 conn(0x7fd66415cbd0 0x7fd66415d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:39.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd669c68700 1 --2- 172.21.15.146:0/141696908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd664153810 0x7fd6640c8730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd66406c0b0 tx=0x7fd66000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b111e3b556bef96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:39.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd668c66700 1 -- 172.21.15.146:0/141696908 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66000e070 con 0x7fd664153810 2024-09-13T18:26:39.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd668c66700 1 -- 172.21.15.146:0/141696908 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66000bb20 con 0x7fd664153810 2024-09-13T18:26:39.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd668c66700 1 -- 172.21.15.146:0/141696908 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660012430 con 0x7fd664153810 2024-09-13T18:26:39.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd66ac6a700 1 -- 172.21.15.146:0/141696908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd664153810 msgr2=0x7fd6640c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:39.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd66ac6a700 1 --2- 172.21.15.146:0/141696908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd664153810 0x7fd6640c8730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd66406c0b0 tx=0x7fd66000b330 comp rx=0 tx=0).stop 2024-09-13T18:26:39.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd66ac6a700 1 -- 172.21.15.146:0/141696908 shutdown_connections 2024-09-13T18:26:39.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd66ac6a700 1 --2- 172.21.15.146:0/141696908 >> v2:172.21.15.146:3300/0 conn(0x7fd66415cbd0 0x7fd66415d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd66ac6a700 1 --2- 172.21.15.146:0/141696908 >> v2:172.21.15.154:3300/0 conn(0x7fd6640c9d00 0x7fd6640ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd66ac6a700 1 --2- 172.21.15.146:0/141696908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd664153810 0x7fd6640c8730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd66ac6a700 1 -- 172.21.15.146:0/141696908 >> 172.21.15.146:0/141696908 conn(0x7fd66406f5c0 msgr2=0x7fd66406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:39.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd66ac6a700 1 -- 172.21.15.146:0/141696908 shutdown_connections 2024-09-13T18:26:39.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd66ac6a700 1 -- 172.21.15.146:0/141696908 wait complete. 2024-09-13T18:26:39.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd66ac6a700 1 Processor -- start 2024-09-13T18:26:39.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.213+0000 7fd66ac6a700 1 -- start start 2024-09-13T18:26:39.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66ac6a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6640c9d00 0x7fd66414f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66ac6a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd664153810 0x7fd6640cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66ac6a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd66415cbd0 0x7fd6640cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66ac6a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640ccdd0 con 0x7fd66415cbd0 2024-09-13T18:26:39.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66ac6a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640ade90 con 0x7fd6640c9d00 2024-09-13T18:26:39.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66ac6a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640ad940 con 0x7fd664153810 2024-09-13T18:26:39.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd669467700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd664153810 0x7fd6640cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66a469700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd66415cbd0 0x7fd6640cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd669467700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd664153810 0x7fd6640cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:44492/0 (socket says 172.21.15.146:44492) 2024-09-13T18:26:39.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd669467700 1 -- 172.21.15.146:0/49246280 learned_addr learned my addr 172.21.15.146:0/49246280 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:39.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd669c68700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6640c9d00 0x7fd66414f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66a469700 1 -- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd664153810 msgr2=0x7fd6640cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:39.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66a469700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd664153810 0x7fd6640cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66a469700 1 -- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6640c9d00 msgr2=0x7fd66414f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:39.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66a469700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6640c9d00 0x7fd66414f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66a469700 1 -- 172.21.15.146:0/49246280 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c009570 con 0x7fd66415cbd0 2024-09-13T18:26:39.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd669467700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd664153810 0x7fd6640cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:39.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66a469700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd66415cbd0 0x7fd6640cb870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00e930 tx=0x7fd65c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:39.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd6567fc700 1 -- 172.21.15.146:0/49246280 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c013680 con 0x7fd66415cbd0 2024-09-13T18:26:39.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd6567fc700 1 -- 172.21.15.146:0/49246280 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c013800 con 0x7fd66415cbd0 2024-09-13T18:26:39.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd6567fc700 1 -- 172.21.15.146:0/49246280 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c013bb0 con 0x7fd66415cbd0 2024-09-13T18:26:39.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66ac6a700 1 -- 172.21.15.146:0/49246280 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd660009a70 con 0x7fd66415cbd0 2024-09-13T18:26:39.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.217+0000 7fd66ac6a700 1 -- 172.21.15.146:0/49246280 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6640adbb0 con 0x7fd66415cbd0 2024-09-13T18:26:39.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.221+0000 7fd6567fc700 1 -- 172.21.15.146:0/49246280 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd65c013d10 con 0x7fd66415cbd0 2024-09-13T18:26:39.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.221+0000 7fd6567fc700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd648066760 0x7fd648068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.221+0000 7fd6567fc700 1 -- 172.21.15.146:0/49246280 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd65c084450 con 0x7fd66415cbd0 2024-09-13T18:26:39.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.221+0000 7fd669c68700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd648066760 0x7fd648068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.221+0000 7fd669c68700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd648066760 0x7fd648068c20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd660014040 tx=0x7fd660011000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:39.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 Processor -- start 2024-09-13T18:26:39.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 -- start start 2024-09-13T18:26:39.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d417c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 0x1d420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d42a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c8cd80 con 0x1d42620 2024-09-13T18:26:39.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c8a510 con 0x1d55ba0 2024-09-13T18:26:39.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c85e00 con 0x1d41d00 2024-09-13T18:26:39.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66a469700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d42a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66a469700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d42a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32770/0 (socket says 172.21.15.146:32770) 2024-09-13T18:26:39.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66a469700 1 -- 172.21.15.146:0/4054515517 learned_addr learned my addr 172.21.15.146:0/4054515517 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:39.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd669c68700 1 --2- 172.21.15.146:0/4054515517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd669467700 1 --2- 172.21.15.146:0/4054515517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 0x1d420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66a469700 1 -- 172.21.15.146:0/4054515517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 msgr2=0x1d420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:39.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66a469700 1 --2- 172.21.15.146:0/4054515517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 0x1d420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66a469700 1 -- 172.21.15.146:0/4054515517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 msgr2=0x1d417c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:39.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66a469700 1 --2- 172.21.15.146:0/4054515517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d417c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66a469700 1 -- 172.21.15.146:0/4054515517 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1af8960 con 0x1d42620 2024-09-13T18:26:39.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd669c68700 1 --2- 172.21.15.146:0/4054515517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:39.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66a469700 1 --2- 172.21.15.146:0/4054515517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d42a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd65c05cfb0 tx=0x7fd65c054b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5318de473ef03301 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:39.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd6477fe700 1 -- 172.21.15.146:0/4054515517 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c05d4e0 con 0x1d42620 2024-09-13T18:26:39.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd6477fe700 1 -- 172.21.15.146:0/4054515517 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c053490 con 0x1d42620 2024-09-13T18:26:39.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd6477fe700 1 -- 172.21.15.146:0/4054515517 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0840b0 con 0x1d42620 2024-09-13T18:26:39.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 -- 172.21.15.146:0/4054515517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 msgr2=0x1d42a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:39.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 --2- 172.21.15.146:0/4054515517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d42a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd65c05cfb0 tx=0x7fd65c054b80 comp rx=0 tx=0).stop 2024-09-13T18:26:39.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 -- 172.21.15.146:0/4054515517 shutdown_connections 2024-09-13T18:26:39.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 --2- 172.21.15.146:0/4054515517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d42a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 --2- 172.21.15.146:0/4054515517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 0x1d420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 --2- 172.21.15.146:0/4054515517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.225+0000 7fd66f17d740 1 -- 172.21.15.146:0/4054515517 >> 172.21.15.146:0/4054515517 conn(0x1aad4d0 msgr2=0x1d58990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:39.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 1 -- 172.21.15.146:0/4054515517 shutdown_connections 2024-09-13T18:26:39.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 1 -- 172.21.15.146:0/4054515517 wait complete. 2024-09-13T18:26:39.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 1 -- start start 2024-09-13T18:26:39.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 1 Processor -- start 2024-09-13T18:26:39.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 0x1d1d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d41090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c43730 con 0x1d42620 2024-09-13T18:26:39.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c495b0 con 0x1d55ba0 2024-09-13T18:26:39.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b31aa0 con 0x1d41d00 2024-09-13T18:26:39.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 10 client.0 _ll_register_callbacks cb 0x1c7f610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:39.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd669467700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d41090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd669467700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d41090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32776/0 (socket says 172.21.15.146:32776) 2024-09-13T18:26:39.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd669467700 1 -- 172.21.15.146:0/1026199909 learned_addr learned my addr 172.21.15.146:0/1026199909 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:39.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66a469700 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd669c68700 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 0x1d1d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd669467700 1 -- 172.21.15.146:0/1026199909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 msgr2=0x1d1d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:39.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd669467700 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 0x1d1d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd669467700 1 -- 172.21.15.146:0/1026199909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 msgr2=0x1d415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:39.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd669467700 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:39.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd669467700 1 -- 172.21.15.146:0/1026199909 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6640adbb0 con 0x1d42620 2024-09-13T18:26:39.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd669467700 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d41090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd6580149f0 tx=0x7fd658014db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e8e107fe78d193 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:39.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd644ff9700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-13T18:26:39.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd644ff9700 1 -- 172.21.15.146:0/1026199909 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658012af0 con 0x1d42620 2024-09-13T18:26:39.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd644ff9700 1 -- 172.21.15.146:0/1026199909 <== mon.0 v2:172.21.15.146:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd65801a070 con 0x1d42620 2024-09-13T18:26:39.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 1 -- 172.21.15.146:0/1026199909 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1aba1e0 con 0x1d42620 2024-09-13T18:26:39.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd644ff9700 1 -- 172.21.15.146:0/1026199909 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd658012c50 con 0x1d42620 2024-09-13T18:26:39.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 10 client.4452 fetch_fsmap learned FSMap version 7 2024-09-13T18:26:39.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.229+0000 7fd66f17d740 1 -- 172.21.15.146:0/1026199909 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x1c4c220 con 0x1d42620 2024-09-13T18:26:39.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.233+0000 7fd644ff9700 1 -- 172.21.15.146:0/1026199909 <== mon.0 v2:172.21.15.146:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1aba1e0 con 0x1d42620 2024-09-13T18:26:39.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.233+0000 7fd644ff9700 1 -- 172.21.15.146:0/1026199909 <== mon.0 v2:172.21.15.146:3300/0 5 ==== fsmap(e 7) v1 ==== 4212+0+0 (secure 0 0 0) 0x7fd658011030 con 0x1d42620 2024-09-13T18:26:39.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.233+0000 7fd66f17d740 10 client.4452 fetch_fsmap finished waiting for FSMap version 7 2024-09-13T18:26:39.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.233+0000 7fd66f17d740 10 client.4452 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T18:26:39.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.233+0000 7fd66f17d740 20 client.4452 populate_metadata read hostname 'smithi146' 2024-09-13T18:26:39.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.233+0000 7fd66f17d740 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0x1d6a4d0 0x1d6a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:39.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.233+0000 7fd66f17d740 4 client.4452 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:26:39.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.233+0000 7fd66f17d740 1 -- 172.21.15.146:0/1026199909 --> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1b04460 con 0x1d6a4d0 2024-09-13T18:26:39.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.233+0000 7fd669c68700 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0x1d6a4d0 0x1d6a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.233+0000 7fd669c68700 1 -- 172.21.15.146:0/1026199909 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0x1d6a4d0 msgr2=0x1d6a8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-13T18:26:39.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.233+0000 7fd669c68700 1 -- 172.21.15.146:0/1026199909 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0x1d6a4d0 msgr2=0x1d6a8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:26:39.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.233+0000 7fd669c68700 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0x1d6a4d0 0x1d6a8b0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:26:39.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.233+0000 7fd669c68700 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0x1d6a4d0 0x1d6a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:26:39.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.433+0000 7fd669c68700 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0x1d6a4d0 0x1d6a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:39.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.433+0000 7fd669c68700 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0x1d6a4d0 0x1d6a8b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc43d20ea01b33a4 server_cookie=23a381f8f8c6a4d5 in_seq=0 out_seq=0 2024-09-13T18:26:39.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:39.433+0000 7fd644ff9700 10 client.4452 ms_handle_connect on v2:172.21.15.146:6836/31041557 2024-09-13T18:26:42.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd644ff9700 1 -- 172.21.15.146:0/1026199909 <== mds.0 v2:172.21.15.146:6836/31041557 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x1b04460 con 0x1d6a4d0 2024-09-13T18:26:42.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd644ff9700 10 client.4452 handle_command_reply: tid=0 2024-09-13T18:26:42.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd66f17d740 1 client.4452 shutdown 2024-09-13T18:26:42.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd66f17d740 1 -- 172.21.15.146:0/1026199909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 msgr2=0x1d41090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:42.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd66f17d740 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d41090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd6580149f0 tx=0x7fd658014db0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd66f17d740 1 -- 172.21.15.146:0/1026199909 shutdown_connections 2024-09-13T18:26:42.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd66f17d740 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0x1d6a4d0 0x1d6a8b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd66f17d740 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd644ff9700 0 client.4452 ms_handle_reset on v2:172.21.15.146:6836/31041557 2024-09-13T18:26:42.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd66f17d740 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d41090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd66f17d740 1 --2- 172.21.15.146:0/1026199909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 0x1d1d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd66f17d740 1 -- 172.21.15.146:0/1026199909 >> 172.21.15.146:0/1026199909 conn(0x1aad4d0 msgr2=0x1d55f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:42.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd66f17d740 1 -- 172.21.15.146:0/1026199909 shutdown_connections 2024-09-13T18:26:42.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd66f17d740 1 -- 172.21.15.146:0/1026199909 wait complete. 2024-09-13T18:26:42.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.765+0000 7fd66f17d740 20 client.4452 trim_cache size 0 max 16384 2024-09-13T18:26:42.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.785+0000 7fd66f17d740 1 Processor -- start 2024-09-13T18:26:42.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.785+0000 7fd66f17d740 1 -- start start 2024-09-13T18:26:42.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.785+0000 7fd66f17d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d41d00 0x1d1fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:42.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.785+0000 7fd66f17d740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d42620 0x1d79770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:42.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.785+0000 7fd66f17d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d40560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:42.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.785+0000 7fd66f17d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c504f0 con 0x1d41d00 2024-09-13T18:26:42.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.785+0000 7fd66f17d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d67270 con 0x1d55ba0 2024-09-13T18:26:42.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.785+0000 7fd66f17d740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c56660 con 0x1d42620 2024-09-13T18:26:42.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.785+0000 7fd669c68700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d41d00 0x1d1fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:42.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.785+0000 7fd669c68700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d41d00 0x1d1fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51064/0 (socket says 172.21.15.146:51064) 2024-09-13T18:26:42.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.785+0000 7fd669c68700 1 -- 172.21.15.146:0/115082772 learned_addr learned my addr 172.21.15.146:0/115082772 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:42.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.785+0000 7fd669467700 1 --2- 172.21.15.146:0/115082772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d42620 0x1d79770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:42.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.785+0000 7fd66a469700 1 --2- 172.21.15.146:0/115082772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d40560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:42.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd669c68700 1 -- 172.21.15.146:0/115082772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d42620 msgr2=0x1d79770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:42.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd669c68700 1 --2- 172.21.15.146:0/115082772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d42620 0x1d79770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd669c68700 1 -- 172.21.15.146:0/115082772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 msgr2=0x1d40560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:42.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd669c68700 1 --2- 172.21.15.146:0/115082772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d40560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd669c68700 1 -- 172.21.15.146:0/115082772 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1c32450 con 0x1d41d00 2024-09-13T18:26:42.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd669467700 1 --2- 172.21.15.146:0/115082772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d42620 0x1d79770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:42.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd669c68700 1 --2- 172.21.15.146:0/115082772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d41d00 0x1d1fba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd66000d010 tx=0x7fd660010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1bf0f74262bbea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:42.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd647fff700 1 -- 172.21.15.146:0/115082772 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66002c040 con 0x1d41d00 2024-09-13T18:26:42.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd647fff700 1 -- 172.21.15.146:0/115082772 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd660003c30 con 0x1d41d00 2024-09-13T18:26:42.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd647fff700 1 -- 172.21.15.146:0/115082772 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660012ea0 con 0x1d41d00 2024-09-13T18:26:42.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd66f17d740 1 -- 172.21.15.146:0/115082772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d41d00 msgr2=0x1d1fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:42.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd66f17d740 1 --2- 172.21.15.146:0/115082772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d41d00 0x1d1fba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd66000d010 tx=0x7fd660010000 comp rx=0 tx=0).stop 2024-09-13T18:26:42.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd66f17d740 1 -- 172.21.15.146:0/115082772 shutdown_connections 2024-09-13T18:26:42.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd66f17d740 1 --2- 172.21.15.146:0/115082772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d40560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd66f17d740 1 --2- 172.21.15.146:0/115082772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d42620 0x1d79770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd66f17d740 1 --2- 172.21.15.146:0/115082772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d41d00 0x1d1fba0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd66f17d740 1 -- 172.21.15.146:0/115082772 >> 172.21.15.146:0/115082772 conn(0x1aad4d0 msgr2=0x1d1f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:42.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd66f17d740 1 -- 172.21.15.146:0/115082772 shutdown_connections 2024-09-13T18:26:42.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.789+0000 7fd66f17d740 1 -- 172.21.15.146:0/115082772 wait complete. 2024-09-13T18:26:42.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd66f17d740 1 -- start start 2024-09-13T18:26:42.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd66f17d740 1 Processor -- start 2024-09-13T18:26:42.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd66f17d740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 0x1d606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:42.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd66f17d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d77c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:42.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd66f17d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d1f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:42.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd66f17d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d1cd90 con 0x1d42620 2024-09-13T18:26:42.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd66f17d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d1c7a0 con 0x1d55ba0 2024-09-13T18:26:42.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd66f17d740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d77360 con 0x1d41d00 2024-09-13T18:26:42.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd66f17d740 10 client.0 _ll_register_callbacks cb 0x7fd65c051430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:42.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd669467700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d77c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:42.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd669467700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d77c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51070/0 (socket says 172.21.15.146:51070) 2024-09-13T18:26:42.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd669467700 1 -- 172.21.15.146:0/511382304 learned_addr learned my addr 172.21.15.146:0/511382304 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:42.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd669c68700 1 --2- 172.21.15.146:0/511382304 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 0x1d606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:42.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd66a469700 1 --2- 172.21.15.146:0/511382304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d1f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:42.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd669467700 1 -- 172.21.15.146:0/511382304 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 msgr2=0x1d606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:42.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd669467700 1 --2- 172.21.15.146:0/511382304 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 0x1d606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd669467700 1 -- 172.21.15.146:0/511382304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 msgr2=0x1d1f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:42.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd669467700 1 --2- 172.21.15.146:0/511382304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d1f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.793+0000 7fd669467700 1 -- 172.21.15.146:0/511382304 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1c4c220 con 0x1d42620 2024-09-13T18:26:42.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd669467700 1 --2- 172.21.15.146:0/511382304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d77c20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd6580143d0 tx=0x7fd658004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef595441e5f3e8ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:42.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd644ff9700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-13T18:26:42.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd644ff9700 1 -- 172.21.15.146:0/511382304 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6580125b0 con 0x1d42620 2024-09-13T18:26:42.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd644ff9700 1 -- 172.21.15.146:0/511382304 <== mon.0 v2:172.21.15.146:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd658012e00 con 0x1d42620 2024-09-13T18:26:42.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 -- 172.21.15.146:0/511382304 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd65c053490 con 0x1d42620 2024-09-13T18:26:42.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd644ff9700 1 -- 172.21.15.146:0/511382304 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6580169e0 con 0x1d42620 2024-09-13T18:26:42.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 10 client.4458 fetch_fsmap learned FSMap version 7 2024-09-13T18:26:42.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 -- 172.21.15.146:0/511382304 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x7fd660023070 con 0x1d42620 2024-09-13T18:26:42.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd644ff9700 1 -- 172.21.15.146:0/511382304 <== mon.0 v2:172.21.15.146:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c053490 con 0x1d42620 2024-09-13T18:26:42.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd644ff9700 1 -- 172.21.15.146:0/511382304 <== mon.0 v2:172.21.15.146:3300/0 5 ==== fsmap(e 7) v1 ==== 4212+0+0 (secure 0 0 0) 0x7fd658014ad0 con 0x1d42620 2024-09-13T18:26:42.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 10 client.4458 fetch_fsmap finished waiting for FSMap version 7 2024-09-13T18:26:42.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 10 client.4458 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T18:26:42.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 20 client.4458 populate_metadata read hostname 'smithi146' 2024-09-13T18:26:42.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 --2- 172.21.15.146:0/511382304 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0x1dc6340 0x1dbf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:42.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 4 client.4458 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:26:42.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 -- 172.21.15.146:0/511382304 --> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1b291e0 con 0x1dc6340 2024-09-13T18:26:42.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd669c68700 1 --2- 172.21.15.146:0/511382304 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0x1dc6340 0x1dbf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:42.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd644ff9700 10 client.4458 ms_handle_connect on v2:172.21.15.146:6836/31041557 2024-09-13T18:26:42.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd669c68700 1 --2- 172.21.15.146:0/511382304 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0x1dc6340 0x1dbf030 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc4db74bb345ae50 server_cookie=9590ea44e3c76be8 in_seq=0 out_seq=0 2024-09-13T18:26:42.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd644ff9700 1 -- 172.21.15.146:0/511382304 <== mds.0 v2:172.21.15.146:6836/31041557 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x1b291e0 con 0x1dc6340 2024-09-13T18:26:42.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd644ff9700 10 client.4458 handle_command_reply: tid=0 2024-09-13T18:26:42.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 client.4458 shutdown 2024-09-13T18:26:42.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 -- 172.21.15.146:0/511382304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 msgr2=0x1d77c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:42.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 --2- 172.21.15.146:0/511382304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d77c20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd6580143d0 tx=0x7fd658004770 comp rx=0 tx=0).stop 2024-09-13T18:26:42.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 -- 172.21.15.146:0/511382304 shutdown_connections 2024-09-13T18:26:42.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 --2- 172.21.15.146:0/511382304 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0x1dc6340 0x1dbf030 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 --2- 172.21.15.146:0/511382304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1d55ba0 0x1d1f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 --2- 172.21.15.146:0/511382304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1d42620 0x1d77c20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 --2- 172.21.15.146:0/511382304 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1d41d00 0x1d606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd644ff9700 0 client.4458 ms_handle_reset on v2:172.21.15.146:6836/31041557 2024-09-13T18:26:42.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 -- 172.21.15.146:0/511382304 >> 172.21.15.146:0/511382304 conn(0x1aad4d0 msgr2=0x1d63600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:42.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 -- 172.21.15.146:0/511382304 shutdown_connections 2024-09-13T18:26:42.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.797+0000 7fd66f17d740 1 -- 172.21.15.146:0/511382304 wait complete. 2024-09-13T18:26:42.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.801+0000 7fd66f17d740 20 client.4458 trim_cache size 0 max 16384 2024-09-13T18:26:42.807 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:26:42.807 INFO:teuthology.orchestra.run.smithi146.stdout: "cluster_fsid": "509e851e-cdbc-4e57-a792-d3d1aeeeba86", 2024-09-13T18:26:42.807 INFO:teuthology.orchestra.run.smithi146.stdout: "whoami": 0, 2024-09-13T18:26:42.807 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4290, 2024-09-13T18:26:42.808 INFO:teuthology.orchestra.run.smithi146.stdout: "want_state": "up:active", 2024-09-13T18:26:42.808 INFO:teuthology.orchestra.run.smithi146.stdout: "state": "up:active", 2024-09-13T18:26:42.808 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs", 2024-09-13T18:26:42.808 INFO:teuthology.orchestra.run.smithi146.stdout: "rank_uptime": 10.473463257000001, 2024-09-13T18:26:42.808 INFO:teuthology.orchestra.run.smithi146.stdout: "mdsmap_epoch": 6, 2024-09-13T18:26:42.808 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_epoch": 25, 2024-09-13T18:26:42.808 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_epoch_barrier": 25, 2024-09-13T18:26:42.808 INFO:teuthology.orchestra.run.smithi146.stdout: "uptime": 16.568392709000001 2024-09-13T18:26:42.808 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:26:42.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.801+0000 7fd66ac6a700 1 -- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd648066760 msgr2=0x7fd648068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:42.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.801+0000 7fd66ac6a700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd648066760 0x7fd648068c20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd660014040 tx=0x7fd660011000 comp rx=0 tx=0).stop 2024-09-13T18:26:42.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.801+0000 7fd66ac6a700 1 -- 172.21.15.146:0/49246280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd66415cbd0 msgr2=0x7fd6640cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:42.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.801+0000 7fd66ac6a700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd66415cbd0 0x7fd6640cb870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00e930 tx=0x7fd65c00ec40 comp rx=0 tx=0).stop 2024-09-13T18:26:42.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.801+0000 7fd66ac6a700 1 -- 172.21.15.146:0/49246280 shutdown_connections 2024-09-13T18:26:42.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.801+0000 7fd66ac6a700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd648066760 0x7fd648068c20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.801+0000 7fd66ac6a700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd66415cbd0 0x7fd6640cb870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.801+0000 7fd66ac6a700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd664153810 0x7fd6640cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.801+0000 7fd66ac6a700 1 --2- 172.21.15.146:0/49246280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6640c9d00 0x7fd66414f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.801+0000 7fd66ac6a700 1 -- 172.21.15.146:0/49246280 >> 172.21.15.146:0/49246280 conn(0x7fd66406f5c0 msgr2=0x7fd664070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:42.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.801+0000 7fd66ac6a700 1 -- 172.21.15.146:0/49246280 shutdown_connections 2024-09-13T18:26:42.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.801+0000 7fd66ac6a700 1 -- 172.21.15.146:0/49246280 wait complete. 2024-09-13T18:26:42.823 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T18:26:42.823 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:26:42.823 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T18:26:42.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.929+0000 7ff7f7fff700 1 Processor -- start 2024-09-13T18:26:42.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.929+0000 7ff7f7fff700 1 -- start start 2024-09-13T18:26:42.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.929+0000 7ff7f7fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff7f8153bc0 0x7ff7f80c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:42.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.929+0000 7ff7f7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff7f80c6610 0x7ff7f80cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:42.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.929+0000 7ff7f7fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f8153fa0 con 0x7ff7f80c6a80 2024-09-13T18:26:42.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.929+0000 7ff7f7fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f80c6e60 con 0x7ff7f80c6610 2024-09-13T18:26:42.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.929+0000 7ff7f7fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f80aa500 con 0x7ff7f8153bc0 2024-09-13T18:26:42.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.929+0000 7ff7f67fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff7f80c6610 0x7ff7f80cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:42.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.929+0000 7ff7f67fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff7f80c6610 0x7ff7f80cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51082/0 (socket says 172.21.15.146:51082) 2024-09-13T18:26:42.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.929+0000 7ff7f67fc700 1 -- 172.21.15.146:0/2248709726 learned_addr learned my addr 172.21.15.146:0/2248709726 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:42.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f6ffd700 1 --2- 172.21.15.146:0/2248709726 >> v2:172.21.15.154:3300/0 conn(0x7ff7f8153bc0 0x7ff7f80c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:42.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f67fc700 1 -- 172.21.15.146:0/2248709726 >> v2:172.21.15.154:3300/0 conn(0x7ff7f8153bc0 msgr2=0x7ff7f80c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f67fc700 1 --2- 172.21.15.146:0/2248709726 >> v2:172.21.15.154:3300/0 conn(0x7ff7f8153bc0 0x7ff7f80c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f67fc700 1 -- 172.21.15.146:0/2248709726 >> v1:172.21.15.154:6789/0 conn(0x7ff7f80c6a80 legacy=0x7ff7f80c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f67fc700 1 -- 172.21.15.146:0/2248709726 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f815a7e0 con 0x7ff7f80c6610 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f67fc700 1 --2- 172.21.15.146:0/2248709726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7f80c6610 0x7ff7f80cac60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8072ac0 tx=0x7ff7ec009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e17d88e66b6ebce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f5ffb700 1 -- 172.21.15.146:0/2248709726 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec011830 con 0x7ff7f80c6610 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f5ffb700 1 -- 172.21.15.146:0/2248709726 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec011990 con 0x7ff7f80c6610 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f5ffb700 1 -- 172.21.15.146:0/2248709726 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec011d20 con 0x7ff7f80c6610 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f7fff700 1 -- 172.21.15.146:0/2248709726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7f80c6610 msgr2=0x7ff7f80cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f7fff700 1 --2- 172.21.15.146:0/2248709726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7f80c6610 0x7ff7f80cac60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8072ac0 tx=0x7ff7ec009bf0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f7fff700 1 -- 172.21.15.146:0/2248709726 shutdown_connections 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f7fff700 1 --2- 172.21.15.146:0/2248709726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7f80c6610 0x7ff7f80cac60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f7fff700 1 --2- 172.21.15.146:0/2248709726 >> v2:172.21.15.154:3300/0 conn(0x7ff7f8153bc0 0x7ff7f80c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f7fff700 1 -- 172.21.15.146:0/2248709726 >> 172.21.15.146:0/2248709726 conn(0x7ff7f8066090 msgr2=0x7ff7f8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f7fff700 1 -- 172.21.15.146:0/2248709726 shutdown_connections 2024-09-13T18:26:42.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f7fff700 1 -- 172.21.15.146:0/2248709726 wait complete. 2024-09-13T18:26:42.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f7fff700 1 Processor -- start 2024-09-13T18:26:42.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.933+0000 7ff7f7fff700 1 -- start start 2024-09-13T18:26:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7f80c6610 0x7ff7f80acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f80c6a80 0x7ff7f814f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f7fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f8153bc0 0x7ff7f814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f7fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f80ad4e0 con 0x7ff7f80c6610 2024-09-13T18:26:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f7fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f80aa8a0 con 0x7ff7f80c6a80 2024-09-13T18:26:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f7fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f80aab80 con 0x7ff7f8153bc0 2024-09-13T18:26:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f6ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7f80c6610 0x7ff7f80acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:42.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f6ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7f80c6610 0x7ff7f80acfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51094/0 (socket says 172.21.15.146:51094) 2024-09-13T18:26:42.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f6ffd700 1 -- 172.21.15.146:0/985395157 learned_addr learned my addr 172.21.15.146:0/985395157 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:42.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f77fe700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f8153bc0 0x7ff7f814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:42.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f67fc700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f80c6a80 0x7ff7f814f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:42.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f6ffd700 1 -- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f8153bc0 msgr2=0x7ff7f814fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:42.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f6ffd700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f8153bc0 0x7ff7f814fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f6ffd700 1 -- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f80c6a80 msgr2=0x7ff7f814f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:42.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f6ffd700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f80c6a80 0x7ff7f814f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:42.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f6ffd700 1 -- 172.21.15.146:0/985395157 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec009570 con 0x7ff7f80c6610 2024-09-13T18:26:42.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f77fe700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f8153bc0 0x7ff7f814fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:26:42.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f67fc700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f80c6a80 0x7ff7f814f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:26:42.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f6ffd700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7f80c6610 0x7ff7f80acfa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000efc0 tx=0x7ff7f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:42.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7df7fe700 1 -- 172.21.15.146:0/985395157 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0009b80 con 0x7ff7f80c6610 2024-09-13T18:26:42.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7df7fe700 1 -- 172.21.15.146:0/985395157 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f000f040 con 0x7ff7f80c6610 2024-09-13T18:26:42.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7df7fe700 1 -- 172.21.15.146:0/985395157 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0014480 con 0x7ff7f80c6610 2024-09-13T18:26:42.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f7fff700 1 -- 172.21.15.146:0/985395157 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f80aae60 con 0x7ff7f80c6610 2024-09-13T18:26:42.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f7fff700 1 -- 172.21.15.146:0/985395157 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f80ab420 con 0x7ff7f80c6610 2024-09-13T18:26:42.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.937+0000 7ff7f7fff700 1 -- 172.21.15.146:0/985395157 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f8159fe0 con 0x7ff7f80c6610 2024-09-13T18:26:42.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.941+0000 7ff7df7fe700 1 -- 172.21.15.146:0/985395157 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff7f00145e0 con 0x7ff7f80c6610 2024-09-13T18:26:42.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.941+0000 7ff7df7fe700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff7e4066730 0x7ff7e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:42.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.941+0000 7ff7df7fe700 1 -- 172.21.15.146:0/985395157 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff7f0084d30 con 0x7ff7f80c6610 2024-09-13T18:26:42.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.941+0000 7ff7f67fc700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff7e4066730 0x7ff7e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:42.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.941+0000 7ff7f67fc700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff7e4066730 0x7ff7e4068bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80c4ef0 tx=0x7ff7ec008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:42.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:42.949+0000 7ff7df7fe700 1 -- 172.21.15.146:0/985395157 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff7f0051e40 con 0x7ff7f80c6610 2024-09-13T18:26:43.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.153+0000 7ff7f7fff700 1 -- 172.21.15.146:0/985395157 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff7f806f690 con 0x7ff7f80c6610 2024-09-13T18:26:43.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.153+0000 7ff7df7fe700 1 -- 172.21.15.146:0/985395157 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff7f0055460 con 0x7ff7f80c6610 2024-09-13T18:26:43.160 INFO:teuthology.orchestra.run.smithi146.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T18:26:43.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.157+0000 7ff7f7fff700 1 -- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff7e4066730 msgr2=0x7ff7e4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:43.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.157+0000 7ff7f7fff700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff7e4066730 0x7ff7e4068bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80c4ef0 tx=0x7ff7ec008030 comp rx=0 tx=0).stop 2024-09-13T18:26:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.157+0000 7ff7f7fff700 1 -- 172.21.15.146:0/985395157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7f80c6610 msgr2=0x7ff7f80acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.157+0000 7ff7f7fff700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7f80c6610 0x7ff7f80acfa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000efc0 tx=0x7ff7f000c3a0 comp rx=0 tx=0).stop 2024-09-13T18:26:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.157+0000 7ff7f7fff700 1 -- 172.21.15.146:0/985395157 shutdown_connections 2024-09-13T18:26:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.157+0000 7ff7f7fff700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff7e4066730 0x7ff7e4068bf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.157+0000 7ff7f7fff700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f8153bc0 0x7ff7f814fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.157+0000 7ff7f7fff700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f80c6a80 0x7ff7f814f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.157+0000 7ff7f7fff700 1 --2- 172.21.15.146:0/985395157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7f80c6610 0x7ff7f80acfa0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.157+0000 7ff7f7fff700 1 -- 172.21.15.146:0/985395157 >> 172.21.15.146:0/985395157 conn(0x7ff7f8066090 msgr2=0x7ff7f80c0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.157+0000 7ff7f7fff700 1 -- 172.21.15.146:0/985395157 shutdown_connections 2024-09-13T18:26:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:43.157+0000 7ff7f7fff700 1 -- 172.21.15.146:0/985395157 wait complete. 2024-09-13T18:26:43.175 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T18:26:43.175 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T18:26:43.175 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-13T18:26:43.175 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-13T18:26:43.175 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:26:43.175 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T18:26:43.175 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T18:26:43.175 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T18:26:43.175 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T18:26:43.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:43.175 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-13T18:26:43.183 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:26:43.183 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:26:43.183 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:26:43.183 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:26:43.183 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-13T18:26:43.183 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:26:43.183 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:26:43.184 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-13T18:26:43.184 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:26:43.184 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-13T18:26:43.184 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:26:43.184 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-13T18:26:43.184 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T18:26:43.184 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-13T18:26:43.184 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T18:26:43.184 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42553sec preferred_lft 42553sec 2024-09-13T18:26:43.184 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2024-09-13T18:26:43.184 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:26:43.185 INFO:teuthology.orchestra.run.smithi146.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T18:26:43.185 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 02:42:8a:41:83:6b brd ff:ff:ff:ff:ff:ff 2024-09-13T18:26:43.185 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T18:26:43.185 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:26:43.185 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T18:26:43.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:43.186 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-09-13T18:26:43.186 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-13T18:26:43.186 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add name ceph-brx type bridge 2024-09-13T18:26:43.186 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr flush dev ceph-brx 2024-09-13T18:26:43.186 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set ceph-brx up 2024-09-13T18:26:43.186 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T18:26:43.186 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-13T18:26:43.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:43.228 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T18:26:43.238 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-13T18:26:43.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:43.239 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-13T18:26:43.260 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-13T18:26:43.260 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T18:26:43.260 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T18:26:43.260 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T18:26:43.261 INFO:teuthology.orchestra.run.smithi146.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T18:26:43.261 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-09-13T18:26:43.261 INFO:teuthology.orchestra.run.smithi146.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T18:26:43.261 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T18:26:43.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:43.261 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-09-13T18:26:43.261 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-13T18:26:43.261 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T18:26:43.262 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T18:26:43.262 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T18:26:43.262 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-13T18:26:43.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:43.302 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-13T18:26:43.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:43.310 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list-id 2024-09-13T18:26:43.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:43.360 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-09-13T18:26:43.360 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-13T18:26:43.360 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T18:26:43.360 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-13T18:26:43.360 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-13T18:26:43.390 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-13T18:26:43.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:43.391 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-09-13T18:26:43.391 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-13T18:26:43.391 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-13T18:26:43.391 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T18:26:43.391 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-13T18:26:43.391 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-13T18:26:43.391 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-13T18:26:43.391 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-13T18:26:43.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:43.455 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-09-13T18:26:43.455 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-13T18:26:43.455 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-09-13T18:26:43.455 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T18:26:43.455 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-13T18:26:43.479 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-13T18:26:43.479 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:26:43.479 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:26:43.496 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T18:26:43.497 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:26:43.497 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:26:43.543 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-13T18:26:43.561 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:26:43.561 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:26:43.621 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:26:43.622 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:26:43.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:43.623 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-13T18:26:43.629 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T18:26:43.630 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-13T18:26:43.632 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-13T18:26:43.651 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:26:43.651 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:26:43.709 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:26:43.710 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:26:43.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:43.711 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-13T18:26:43.715 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi146.stderr:2024-09-13T18:26:43.709+0000 7fc8985d9640 -1 init, newargv = 0x562cdb9e93d0 newargc=15 2024-09-13T18:26:43.716 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi146.stderr:ceph-fuse[31113]: starting ceph client 2024-09-13T18:26:43.725 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi146.stderr:ceph-fuse[31113]: starting fuse 2024-09-13T18:26:44.818 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-13T18:26:44.837 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:26:44.837 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:26:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:26:44.898 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:26:44.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:44.898 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-13T18:26:44.906 INFO:teuthology.orchestra.run.smithi146.stdout:52 2024-09-13T18:26:44.906 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-09-13T18:26:44.906 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:26:44.906 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-13T18:26:44.906 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-13T18:26:44.906 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-13T18:26:44.906 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-13T18:26:44.906 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T18:26:44.907 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-13T18:26:44.908 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-13T18:26:44.908 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-13T18:26:44.908 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T18:26:44.908 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-13T18:26:44.908 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.admin")) 2024-09-13T18:26:44.908 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-13T18:26:44.989 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.admin.31113.asok 2024-09-13T18:26:45.176 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.31113.asok 2024-09-13T18:26:45.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:45.176 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.31113.asok status 2024-09-13T18:26:45.317 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:26:45.317 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-13T18:26:45.317 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T18:26:45.317 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T18:26:45.317 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "admin", 2024-09-13T18:26:45.317 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-13T18:26:45.317 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-13T18:26:45.317 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "31113", 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4470, 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 4470 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-13T18:26:45.318 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 108018343 2024-09-13T18:26:45.319 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-13T18:26:45.319 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-13T18:26:45.319 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-13T18:26:45.319 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-13T18:26:45.319 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-13T18:26:45.319 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 108018343 2024-09-13T18:26:45.319 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-13T18:26:45.319 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.4470 192.168.0.1:0/108018343", 2024-09-13T18:26:45.319 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/108018343", 2024-09-13T18:26:45.319 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-13T18:26:45.319 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 6, 2024-09-13T18:26:45.319 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 25, 2024-09-13T18:26:45.320 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-13T18:26:45.320 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-13T18:26:45.320 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-13T18:26:45.320 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:26:45.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:45.327 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-13T18:26:45.327 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:26:45.335 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-13T18:26:45.335 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:26:45.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:45.335 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:26:45.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:45.348 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-13T18:26:45.396 INFO:teuthology.orchestra.run.smithi146.stdout:# file: . 2024-09-13T18:26:45.396 INFO:teuthology.orchestra.run.smithi146.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-13T18:26:45.396 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:45.397 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:45.398 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:26:45.405 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:26:45.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.565+0000 7f6c4759e700 1 Processor -- start 2024-09-13T18:26:45.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.565+0000 7f6c4759e700 1 -- start start 2024-09-13T18:26:45.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c4759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c480c8690 0x7f6c48159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:45.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c4759e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c48159120 con 0x7f6c4806cef0 2024-09-13T18:26:45.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c4759e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c481596e0 con 0x7f6c480c82b0 2024-09-13T18:26:45.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c4759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4815a0b0 con 0x7f6c480c8690 2024-09-13T18:26:45.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c46d9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6c4806cef0 0x7f6c480c7760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50616/0 (socket says 172.21.15.146:50616) 2024-09-13T18:26:45.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c46d9d700 1 -- 172.21.15.146:0/3286184773 learned_addr learned my addr 172.21.15.146:0/3286184773 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:45.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c3ffff700 1 --2- 172.21.15.146:0/3286184773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c480c8690 0x7f6c48159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:45.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c45d9b700 1 -- 172.21.15.146:0/3286184773 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4279482340 0 0) 0x7f6c481596e0 con 0x7f6c480c82b0 2024-09-13T18:26:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c45d9b700 1 -- 172.21.15.146:0/3286184773 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c340034f0 con 0x7f6c480c82b0 2024-09-13T18:26:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c45d9b700 1 -- 172.21.15.146:0/3286184773 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532187576 0 0) 0x7f6c48159120 con 0x7f6c4806cef0 2024-09-13T18:26:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c45d9b700 1 -- 172.21.15.146:0/3286184773 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c481596e0 con 0x7f6c4806cef0 2024-09-13T18:26:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c3ffff700 1 -- 172.21.15.146:0/3286184773 >> v1:172.21.15.154:6789/0 conn(0x7f6c480c82b0 legacy=0x7f6c48155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c3ffff700 1 -- 172.21.15.146:0/3286184773 >> v1:172.21.15.146:6789/0 conn(0x7f6c4806cef0 legacy=0x7f6c480c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c3ffff700 1 -- 172.21.15.146:0/3286184773 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c480c7e70 con 0x7f6c480c8690 2024-09-13T18:26:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c3ffff700 1 --2- 172.21.15.146:0/3286184773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c480c8690 0x7f6c48159b70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009880 tx=0x7f6c30009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69022cc9778dc2b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c45d9b700 1 -- 172.21.15.146:0/3286184773 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30011830 con 0x7f6c480c8690 2024-09-13T18:26:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c45d9b700 1 -- 172.21.15.146:0/3286184773 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c30011990 con 0x7f6c480c8690 2024-09-13T18:26:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.569+0000 7f6c45d9b700 1 -- 172.21.15.146:0/3286184773 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30011d00 con 0x7f6c480c8690 2024-09-13T18:26:45.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.573+0000 7f6c4759e700 1 -- 172.21.15.146:0/3286184773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c480c8690 msgr2=0x7f6c48159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:45.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.573+0000 7f6c4759e700 1 --2- 172.21.15.146:0/3286184773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c480c8690 0x7f6c48159b70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009880 tx=0x7f6c30009b90 comp rx=0 tx=0).stop 2024-09-13T18:26:45.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.573+0000 7f6c4759e700 1 -- 172.21.15.146:0/3286184773 shutdown_connections 2024-09-13T18:26:45.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.573+0000 7f6c4759e700 1 --2- 172.21.15.146:0/3286184773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c480c8690 0x7f6c48159b70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:45.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.573+0000 7f6c4759e700 1 -- 172.21.15.146:0/3286184773 >> 172.21.15.146:0/3286184773 conn(0x7f6c48072270 msgr2=0x7f6c4806ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:45.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.573+0000 7f6c4759e700 1 -- 172.21.15.146:0/3286184773 shutdown_connections 2024-09-13T18:26:45.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.573+0000 7f6c4759e700 1 -- 172.21.15.146:0/3286184773 wait complete. 2024-09-13T18:26:45.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c4759e700 1 Processor -- start 2024-09-13T18:26:45.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c4759e700 1 -- start start 2024-09-13T18:26:45.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c4759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c4806cef0 0x7f6c4814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:45.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c4759e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c480c82b0 0x7f6c48152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:45.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c4759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c480c8690 0x7f6c48152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:45.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c4759e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c481ef210 con 0x7f6c480c8690 2024-09-13T18:26:45.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c4759e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c48151600 con 0x7f6c480c82b0 2024-09-13T18:26:45.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c4759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4814a020 con 0x7f6c4806cef0 2024-09-13T18:26:45.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c46d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c480c8690 0x7f6c48152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:45.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c46d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c480c8690 0x7f6c48152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51102/0 (socket says 172.21.15.146:51102) 2024-09-13T18:26:45.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c46d9d700 1 -- 172.21.15.146:0/2518259244 learned_addr learned my addr 172.21.15.146:0/2518259244 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:45.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c4659c700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c4806cef0 0x7f6c4814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:45.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c3ffff700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c480c82b0 0x7f6c48152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:45.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c46d9d700 1 -- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c4806cef0 msgr2=0x7f6c4814c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:45.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c46d9d700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c4806cef0 0x7f6c4814c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:45.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c46d9d700 1 -- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c480c82b0 msgr2=0x7f6c48152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:45.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c46d9d700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c480c82b0 0x7f6c48152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:45.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c46d9d700 1 -- 172.21.15.146:0/2518259244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c300094b0 con 0x7f6c480c8690 2024-09-13T18:26:45.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c46d9d700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c480c8690 0x7f6c48152680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000cb40 tx=0x7f6c4000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:45.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c3effd700 1 -- 172.21.15.146:0/2518259244 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40007500 con 0x7f6c480c8690 2024-09-13T18:26:45.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c3effd700 1 -- 172.21.15.146:0/2518259244 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c40007660 con 0x7f6c480c8690 2024-09-13T18:26:45.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c4759e700 1 -- 172.21.15.146:0/2518259244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4814a2e0 con 0x7f6c480c8690 2024-09-13T18:26:45.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c3effd700 1 -- 172.21.15.146:0/2518259244 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400079f0 con 0x7f6c480c8690 2024-09-13T18:26:45.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.577+0000 7f6c4759e700 1 -- 172.21.15.146:0/2518259244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c48151c20 con 0x7f6c480c8690 2024-09-13T18:26:45.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.581+0000 7f6c4759e700 1 -- 172.21.15.146:0/2518259244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c480c3210 con 0x7f6c480c8690 2024-09-13T18:26:45.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.581+0000 7f6c3effd700 1 -- 172.21.15.146:0/2518259244 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6c4001b3b0 con 0x7f6c480c8690 2024-09-13T18:26:45.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.581+0000 7f6c3effd700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c28066730 0x7f6c28068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:45.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.581+0000 7f6c3effd700 1 -- 172.21.15.146:0/2518259244 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6c40084230 con 0x7f6c480c8690 2024-09-13T18:26:45.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.581+0000 7f6c4659c700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c28066730 0x7f6c28068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:45.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.585+0000 7f6c4659c700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c28066730 0x7f6c28068bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c380011e0 tx=0x7f6c38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:45.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.589+0000 7f6c3effd700 1 -- 172.21.15.146:0/2518259244 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6c400513a0 con 0x7f6c480c8690 2024-09-13T18:26:45.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.757+0000 7f6c4759e700 1 -- 172.21.15.146:0/2518259244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c48159b70 con 0x7f6c480c8690 2024-09-13T18:26:45.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.761+0000 7f6c3effd700 1 -- 172.21.15.146:0/2518259244 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f6c400549c0 con 0x7f6c480c8690 2024-09-13T18:26:45.768 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:45.768 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:26:45.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.765+0000 7f6c4759e700 1 -- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c28066730 msgr2=0x7f6c28068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:45.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.765+0000 7f6c4759e700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c28066730 0x7f6c28068bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c380011e0 tx=0x7f6c38009040 comp rx=0 tx=0).stop 2024-09-13T18:26:45.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.765+0000 7f6c4759e700 1 -- 172.21.15.146:0/2518259244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c480c8690 msgr2=0x7f6c48152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:45.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.765+0000 7f6c4759e700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c480c8690 0x7f6c48152680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000cb40 tx=0x7f6c4000cb70 comp rx=0 tx=0).stop 2024-09-13T18:26:45.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.765+0000 7f6c4759e700 1 -- 172.21.15.146:0/2518259244 shutdown_connections 2024-09-13T18:26:45.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.765+0000 7f6c4759e700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c28066730 0x7f6c28068bf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:45.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.765+0000 7f6c4759e700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c480c8690 0x7f6c48152680 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:45.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.765+0000 7f6c4759e700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c480c82b0 0x7f6c48152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:45.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.765+0000 7f6c4759e700 1 --2- 172.21.15.146:0/2518259244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c4806cef0 0x7f6c4814c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:45.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.765+0000 7f6c4759e700 1 -- 172.21.15.146:0/2518259244 >> 172.21.15.146:0/2518259244 conn(0x7f6c48072270 msgr2=0x7f6c4806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:45.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.765+0000 7f6c4759e700 1 -- 172.21.15.146:0/2518259244 shutdown_connections 2024-09-13T18:26:45.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.765+0000 7f6c4759e700 1 -- 172.21.15.146:0/2518259244 wait complete. 2024-09-13T18:26:45.783 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:26:24.867515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:26:30.782269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:26:31.323277+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:26:34.393386+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:26:45.783 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:26:45.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.901+0000 7ff31759e700 1 Processor -- start 2024-09-13T18:26:45.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.901+0000 7ff31759e700 1 -- start start 2024-09-13T18:26:45.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff31815d960 0x7ff3180c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:45.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31759e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3180ca500 0x7ff3180cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:45.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31759e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3180c5e50 con 0x7ff3181539e0 2024-09-13T18:26:45.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31759e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3180cc3a0 con 0x7ff3180ca500 2024-09-13T18:26:45.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3180cb480 con 0x7ff31815d960 2024-09-13T18:26:45.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff30ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff31815d960 0x7ff3180c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:45.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff316d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3180ca500 0x7ff3180cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:45.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff30ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff31815d960 0x7ff3180c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:50668/0 (socket says 172.21.15.146:50668) 2024-09-13T18:26:45.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff30ffff700 1 -- 172.21.15.146:0/3428923666 learned_addr learned my addr 172.21.15.146:0/3428923666 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:45.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31659c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff3181539e0 0x7ff3180b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:50656/0 (socket says 172.21.15.146:50656) 2024-09-13T18:26:45.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff30ffff700 1 -- 172.21.15.146:0/3428923666 >> v2:172.21.15.154:3300/0 conn(0x7ff3180ca500 msgr2=0x7ff3180cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:45.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff30ffff700 1 --2- 172.21.15.146:0/3428923666 >> v2:172.21.15.154:3300/0 conn(0x7ff3180ca500 0x7ff3180cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:45.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff30ffff700 1 -- 172.21.15.146:0/3428923666 >> v1:172.21.15.154:6789/0 conn(0x7ff3181539e0 legacy=0x7ff3180b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:45.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff30ffff700 1 -- 172.21.15.146:0/3428923666 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3180cb720 con 0x7ff31815d960 2024-09-13T18:26:45.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff30ffff700 1 --2- 172.21.15.146:0/3428923666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff31815d960 0x7ff3180c9fc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff300009910 tx=0x7ff300009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6945b08117a205f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:45.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff315d9b700 1 -- 172.21.15.146:0/3428923666 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300011830 con 0x7ff31815d960 2024-09-13T18:26:45.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff315d9b700 1 -- 172.21.15.146:0/3428923666 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff300011990 con 0x7ff31815d960 2024-09-13T18:26:45.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff315d9b700 1 -- 172.21.15.146:0/3428923666 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300011d40 con 0x7ff31815d960 2024-09-13T18:26:45.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31759e700 1 -- 172.21.15.146:0/3428923666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff31815d960 msgr2=0x7ff3180c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:45.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31759e700 1 --2- 172.21.15.146:0/3428923666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff31815d960 0x7ff3180c9fc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff300009910 tx=0x7ff300009c20 comp rx=0 tx=0).stop 2024-09-13T18:26:45.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31759e700 1 -- 172.21.15.146:0/3428923666 shutdown_connections 2024-09-13T18:26:45.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31759e700 1 --2- 172.21.15.146:0/3428923666 >> v2:172.21.15.154:3300/0 conn(0x7ff3180ca500 0x7ff3180cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:45.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31759e700 1 --2- 172.21.15.146:0/3428923666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff31815d960 0x7ff3180c9fc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:45.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31759e700 1 -- 172.21.15.146:0/3428923666 >> 172.21.15.146:0/3428923666 conn(0x7ff318071ec0 msgr2=0x7ff3180722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:45.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31759e700 1 -- 172.21.15.146:0/3428923666 shutdown_connections 2024-09-13T18:26:45.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.905+0000 7ff31759e700 1 -- 172.21.15.146:0/3428923666 wait complete. 2024-09-13T18:26:45.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31759e700 1 Processor -- start 2024-09-13T18:26:45.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31759e700 1 -- start start 2024-09-13T18:26:45.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3180ca500 0x7ff3180c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:45.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31759e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3181539e0 0x7ff3180c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:45.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff31815d960 0x7ff3180c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:45.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31759e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3181f3ef0 con 0x7ff31815d960 2024-09-13T18:26:45.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31759e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3181f7590 con 0x7ff3181539e0 2024-09-13T18:26:45.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3181f32e0 con 0x7ff3180ca500 2024-09-13T18:26:45.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff316d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff31815d960 0x7ff3180c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:45.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff30ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3181539e0 0x7ff3180c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:45.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31659c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3180ca500 0x7ff3180c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:45.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31659c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3180ca500 0x7ff3180c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:50670/0 (socket says 172.21.15.146:50670) 2024-09-13T18:26:45.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31659c700 1 -- 172.21.15.146:0/350659470 learned_addr learned my addr 172.21.15.146:0/350659470 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:45.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff316d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff31815d960 0x7ff3180c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51106/0 (socket says 172.21.15.146:51106) 2024-09-13T18:26:45.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31659c700 1 -- 172.21.15.146:0/350659470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3181539e0 msgr2=0x7ff3180c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:45.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31659c700 1 --2- 172.21.15.146:0/350659470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3181539e0 0x7ff3180c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:45.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31659c700 1 -- 172.21.15.146:0/350659470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff31815d960 msgr2=0x7ff3180c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:45.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31659c700 1 --2- 172.21.15.146:0/350659470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff31815d960 0x7ff3180c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:45.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31659c700 1 -- 172.21.15.146:0/350659470 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff300009570 con 0x7ff3180ca500 2024-09-13T18:26:45.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31659c700 1 --2- 172.21.15.146:0/350659470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3180ca500 0x7ff3180c41c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff30800c700 tx=0x7ff30800ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:45.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff30effd700 1 -- 172.21.15.146:0/350659470 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308005cd0 con 0x7ff3180ca500 2024-09-13T18:26:45.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff30effd700 1 -- 172.21.15.146:0/350659470 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff308005e30 con 0x7ff3180ca500 2024-09-13T18:26:45.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31759e700 1 -- 172.21.15.146:0/350659470 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3181f35c0 con 0x7ff3180ca500 2024-09-13T18:26:45.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff30effd700 1 -- 172.21.15.146:0/350659470 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3080072d0 con 0x7ff3180ca500 2024-09-13T18:26:45.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.909+0000 7ff31759e700 1 -- 172.21.15.146:0/350659470 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3181f3b80 con 0x7ff3180ca500 2024-09-13T18:26:45.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.913+0000 7ff31759e700 1 -- 172.21.15.146:0/350659470 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3180cc7c0 con 0x7ff3180ca500 2024-09-13T18:26:45.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.913+0000 7ff30effd700 1 -- 172.21.15.146:0/350659470 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff308007430 con 0x7ff3180ca500 2024-09-13T18:26:45.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.913+0000 7ff30effd700 1 --2- 172.21.15.146:0/350659470 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff2fc066730 0x7ff2fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:45.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.913+0000 7ff30effd700 1 -- 172.21.15.146:0/350659470 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff308083500 con 0x7ff3180ca500 2024-09-13T18:26:45.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.913+0000 7ff30ffff700 1 --2- 172.21.15.146:0/350659470 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff2fc066730 0x7ff2fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:45.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.917+0000 7ff30ffff700 1 --2- 172.21.15.146:0/350659470 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff2fc066730 0x7ff2fc068bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff300000c00 tx=0x7ff300008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:45.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:45.925+0000 7ff30effd700 1 -- 172.21.15.146:0/350659470 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff308050630 con 0x7ff3180ca500 2024-09-13T18:26:46.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.097+0000 7ff31759e700 1 -- 172.21.15.146:0/350659470 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3180c5e30 con 0x7ff3180ca500 2024-09-13T18:26:46.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.097+0000 7ff30effd700 1 -- 172.21.15.146:0/350659470 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff308017020 con 0x7ff3180ca500 2024-09-13T18:26:46.102 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-09-13T18:26:46.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.101+0000 7ff31759e700 1 -- 172.21.15.146:0/350659470 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff2fc066730 msgr2=0x7ff2fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.101+0000 7ff31759e700 1 --2- 172.21.15.146:0/350659470 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff2fc066730 0x7ff2fc068bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff300000c00 tx=0x7ff300008030 comp rx=0 tx=0).stop 2024-09-13T18:26:46.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.101+0000 7ff31759e700 1 -- 172.21.15.146:0/350659470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3180ca500 msgr2=0x7ff3180c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.101+0000 7ff31759e700 1 --2- 172.21.15.146:0/350659470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3180ca500 0x7ff3180c41c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff30800c700 tx=0x7ff30800ca10 comp rx=0 tx=0).stop 2024-09-13T18:26:46.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.101+0000 7ff31759e700 1 -- 172.21.15.146:0/350659470 shutdown_connections 2024-09-13T18:26:46.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.101+0000 7ff31759e700 1 --2- 172.21.15.146:0/350659470 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff2fc066730 0x7ff2fc068bf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.101+0000 7ff31759e700 1 --2- 172.21.15.146:0/350659470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff31815d960 0x7ff3180c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.101+0000 7ff31759e700 1 --2- 172.21.15.146:0/350659470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3181539e0 0x7ff3180c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.101+0000 7ff31759e700 1 --2- 172.21.15.146:0/350659470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3180ca500 0x7ff3180c41c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.101+0000 7ff31759e700 1 -- 172.21.15.146:0/350659470 >> 172.21.15.146:0/350659470 conn(0x7ff318071ec0 msgr2=0x7ff3180722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:46.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.101+0000 7ff31759e700 1 -- 172.21.15.146:0/350659470 shutdown_connections 2024-09-13T18:26:46.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.101+0000 7ff31759e700 1 -- 172.21.15.146:0/350659470 wait complete. 2024-09-13T18:26:46.123 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:26:46.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 Processor -- start 2024-09-13T18:26:46.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 -- start start 2024-09-13T18:26:46.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa25c1539c0 0x7fa25c0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa25c06ffd0 0x7fa25c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25c0aa010 con 0x7fa25c15a570 2024-09-13T18:26:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25c0aa5d0 con 0x7fa25c1539c0 2024-09-13T18:26:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25c0aa2f0 con 0x7fa25c06ffd0 2024-09-13T18:26:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25affd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa25c1539c0 0x7fa25c0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25a7fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa25c15a570 0x7fa25c0c6610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50624/0 (socket says 172.21.15.146:50624) 2024-09-13T18:26:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25a7fc700 1 -- 172.21.15.146:0/587985516 learned_addr learned my addr 172.21.15.146:0/587985516 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25b7fe700 1 --2- 172.21.15.146:0/587985516 >> v2:172.21.15.154:3300/0 conn(0x7fa25c06ffd0 0x7fa25c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25affd700 1 -- 172.21.15.146:0/587985516 >> v2:172.21.15.154:3300/0 conn(0x7fa25c06ffd0 msgr2=0x7fa25c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25affd700 1 --2- 172.21.15.146:0/587985516 >> v2:172.21.15.154:3300/0 conn(0x7fa25c06ffd0 0x7fa25c0caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa259ffb700 1 -- 172.21.15.146:0/587985516 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811289519 0 0) 0x7fa25c0aa010 con 0x7fa25c15a570 2024-09-13T18:26:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25affd700 1 -- 172.21.15.146:0/587985516 >> v1:172.21.15.146:6789/0 conn(0x7fa25c15a570 legacy=0x7fa25c0c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25affd700 1 -- 172.21.15.146:0/587985516 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c159a90 con 0x7fa25c1539c0 2024-09-13T18:26:46.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25b7fe700 1 --2- 172.21.15.146:0/587985516 >> v2:172.21.15.154:3300/0 conn(0x7fa25c06ffd0 0x7fa25c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:46.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25affd700 1 --2- 172.21.15.146:0/587985516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c1539c0 0x7fa25c0c8de0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa25c06eca0 tx=0x7fa24c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f494f9d8eef3f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:46.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa259ffb700 1 -- 172.21.15.146:0/587985516 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c011830 con 0x7fa25c1539c0 2024-09-13T18:26:46.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa259ffb700 1 -- 172.21.15.146:0/587985516 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24c011990 con 0x7fa25c1539c0 2024-09-13T18:26:46.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa259ffb700 1 -- 172.21.15.146:0/587985516 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c011d40 con 0x7fa25c1539c0 2024-09-13T18:26:46.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 -- 172.21.15.146:0/587985516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c1539c0 msgr2=0x7fa25c0c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 --2- 172.21.15.146:0/587985516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c1539c0 0x7fa25c0c8de0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa25c06eca0 tx=0x7fa24c009b30 comp rx=0 tx=0).stop 2024-09-13T18:26:46.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 -- 172.21.15.146:0/587985516 shutdown_connections 2024-09-13T18:26:46.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 --2- 172.21.15.146:0/587985516 >> v2:172.21.15.154:3300/0 conn(0x7fa25c06ffd0 0x7fa25c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 --2- 172.21.15.146:0/587985516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c1539c0 0x7fa25c0c8de0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 -- 172.21.15.146:0/587985516 >> 172.21.15.146:0/587985516 conn(0x7fa25c066090 msgr2=0x7fa25c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:46.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 -- 172.21.15.146:0/587985516 shutdown_connections 2024-09-13T18:26:46.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.237+0000 7fa25bfff700 1 -- 172.21.15.146:0/587985516 wait complete. 2024-09-13T18:26:46.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25bfff700 1 Processor -- start 2024-09-13T18:26:46.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25bfff700 1 -- start start 2024-09-13T18:26:46.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa25c06ffd0 0x7fa25c0ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c1539c0 0x7fa25c06d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c15a570 0x7fa25c14fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25bfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c06dbc0 con 0x7fa25c15a570 2024-09-13T18:26:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25bfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c1ec470 con 0x7fa25c06ffd0 2024-09-13T18:26:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0aad70 con 0x7fa25c1539c0 2024-09-13T18:26:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c15a570 0x7fa25c14fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c15a570 0x7fa25c14fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51126/0 (socket says 172.21.15.146:51126) 2024-09-13T18:26:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25b7fe700 1 -- 172.21.15.146:0/993737251 learned_addr learned my addr 172.21.15.146:0/993737251 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25affd700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa25c06ffd0 0x7fa25c0ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25a7fc700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c1539c0 0x7fa25c06d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25b7fe700 1 -- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c1539c0 msgr2=0x7fa25c06d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25b7fe700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c1539c0 0x7fa25c06d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25b7fe700 1 -- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa25c06ffd0 msgr2=0x7fa25c0ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25b7fe700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa25c06ffd0 0x7fa25c0ac470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25b7fe700 1 -- 172.21.15.146:0/993737251 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c0094b0 con 0x7fa25c15a570 2024-09-13T18:26:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25a7fc700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c1539c0 0x7fa25c06d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25b7fe700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c15a570 0x7fa25c14fc60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa254007ef0 tx=0x7fa25400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa2437fe700 1 -- 172.21.15.146:0/993737251 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25400de30 con 0x7fa25c15a570 2024-09-13T18:26:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa2437fe700 1 -- 172.21.15.146:0/993737251 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25400f040 con 0x7fa25c15a570 2024-09-13T18:26:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25bfff700 1 -- 172.21.15.146:0/993737251 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25c154500 con 0x7fa25c15a570 2024-09-13T18:26:46.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa2437fe700 1 -- 172.21.15.146:0/993737251 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254003c80 con 0x7fa25c15a570 2024-09-13T18:26:46.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.241+0000 7fa25bfff700 1 -- 172.21.15.146:0/993737251 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c0ab310 con 0x7fa25c15a570 2024-09-13T18:26:46.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.245+0000 7fa25bfff700 1 -- 172.21.15.146:0/993737251 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25c1ec2d0 con 0x7fa25c15a570 2024-09-13T18:26:46.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.245+0000 7fa2437fe700 1 -- 172.21.15.146:0/993737251 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa254004030 con 0x7fa25c15a570 2024-09-13T18:26:46.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.249+0000 7fa2437fe700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa248066760 0x7fa248068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.249+0000 7fa2437fe700 1 -- 172.21.15.146:0/993737251 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa254084870 con 0x7fa25c15a570 2024-09-13T18:26:46.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.249+0000 7fa25affd700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa248066760 0x7fa248068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.249+0000 7fa25affd700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa248066760 0x7fa248068c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0090b0 tx=0x7fa24c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:46.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.253+0000 7fa2437fe700 1 -- 172.21.15.146:0/993737251 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa2540518d0 con 0x7fa25c15a570 2024-09-13T18:26:46.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.425+0000 7fa25bfff700 1 -- 172.21.15.146:0/993737251 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa25c1ec2d0 con 0x7fa25c15a570 2024-09-13T18:26:46.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.425+0000 7fa2437fe700 1 -- 172.21.15.146:0/993737251 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa25401a020 con 0x7fa25c15a570 2024-09-13T18:26:46.431 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:26:46.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.425+0000 7fa25bfff700 1 -- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa248066760 msgr2=0x7fa248068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.425+0000 7fa25bfff700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa248066760 0x7fa248068c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0090b0 tx=0x7fa24c008030 comp rx=0 tx=0).stop 2024-09-13T18:26:46.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.429+0000 7fa25bfff700 1 -- 172.21.15.146:0/993737251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c15a570 msgr2=0x7fa25c14fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.429+0000 7fa25bfff700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c15a570 0x7fa25c14fc60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa254007ef0 tx=0x7fa25400d3b0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.429+0000 7fa25bfff700 1 -- 172.21.15.146:0/993737251 shutdown_connections 2024-09-13T18:26:46.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.429+0000 7fa25bfff700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa248066760 0x7fa248068c20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.429+0000 7fa25bfff700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c15a570 0x7fa25c14fc60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.429+0000 7fa25bfff700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c1539c0 0x7fa25c06d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.429+0000 7fa25bfff700 1 --2- 172.21.15.146:0/993737251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa25c06ffd0 0x7fa25c0ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.429+0000 7fa25bfff700 1 -- 172.21.15.146:0/993737251 >> 172.21.15.146:0/993737251 conn(0x7fa25c066090 msgr2=0x7fa25c0c9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:46.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.429+0000 7fa25bfff700 1 -- 172.21.15.146:0/993737251 shutdown_connections 2024-09-13T18:26:46.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.429+0000 7fa25bfff700 1 -- 172.21.15.146:0/993737251 wait complete. 2024-09-13T18:26:46.446 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:26:46.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.553+0000 7f3cdffff700 1 Processor -- start 2024-09-13T18:26:46.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.553+0000 7f3cdffff700 1 -- start start 2024-09-13T18:26:46.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.553+0000 7f3cdffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ce00c8f70 0x7f3ce015a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.553+0000 7f3cdffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3ce00a9490 0x7f3ce00c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.553+0000 7f3cdffff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce00c8650 con 0x7f3ce00c9350 2024-09-13T18:26:46.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.553+0000 7f3cdffff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce00aa1a0 con 0x7f3ce00a9490 2024-09-13T18:26:46.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.553+0000 7f3cdffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00aa480 con 0x7f3ce00c8f70 2024-09-13T18:26:46.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdf7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3ce00a9490 0x7f3ce00c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cde7fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3ce00c9350 0x7f3ce00cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:50672/0 (socket says 172.21.15.146:50672) 2024-09-13T18:26:46.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cde7fc700 1 -- 172.21.15.146:0/1675468747 learned_addr learned my addr 172.21.15.146:0/1675468747 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:46.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdeffd700 1 --2- 172.21.15.146:0/1675468747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ce00c8f70 0x7f3ce015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdf7fe700 1 -- 172.21.15.146:0/1675468747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ce00c8f70 msgr2=0x7f3ce015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdf7fe700 1 --2- 172.21.15.146:0/1675468747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ce00c8f70 0x7f3ce015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdf7fe700 1 -- 172.21.15.146:0/1675468747 >> v1:172.21.15.154:6789/0 conn(0x7f3ce00c9350 legacy=0x7f3ce00cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdf7fe700 1 -- 172.21.15.146:0/1675468747 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce0159890 con 0x7f3ce00a9490 2024-09-13T18:26:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cddffb700 1 -- 172.21.15.146:0/1675468747 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956844616 0 0) 0x7f3ce00c8650 con 0x7f3ce00c9350 2024-09-13T18:26:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdf7fe700 1 --2- 172.21.15.146:0/1675468747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce00a9490 0x7f3ce00c76c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009d90 tx=0x7f3cd800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=379b48c98922e315 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cddffb700 1 -- 172.21.15.146:0/1675468747 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd800e070 con 0x7f3ce00a9490 2024-09-13T18:26:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cddffb700 1 -- 172.21.15.146:0/1675468747 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd800bb20 con 0x7f3ce00a9490 2024-09-13T18:26:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cddffb700 1 -- 172.21.15.146:0/1675468747 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8012430 con 0x7f3ce00a9490 2024-09-13T18:26:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdffff700 1 -- 172.21.15.146:0/1675468747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce00a9490 msgr2=0x7f3ce00c76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdffff700 1 --2- 172.21.15.146:0/1675468747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce00a9490 0x7f3ce00c76c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009d90 tx=0x7f3cd800b330 comp rx=0 tx=0).stop 2024-09-13T18:26:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdffff700 1 -- 172.21.15.146:0/1675468747 shutdown_connections 2024-09-13T18:26:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdffff700 1 --2- 172.21.15.146:0/1675468747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce00a9490 0x7f3ce00c76c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdffff700 1 --2- 172.21.15.146:0/1675468747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ce00c8f70 0x7f3ce015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdffff700 1 -- 172.21.15.146:0/1675468747 >> 172.21.15.146:0/1675468747 conn(0x7f3ce0066090 msgr2=0x7f3ce0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdffff700 1 -- 172.21.15.146:0/1675468747 shutdown_connections 2024-09-13T18:26:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.557+0000 7f3cdffff700 1 -- 172.21.15.146:0/1675468747 wait complete. 2024-09-13T18:26:46.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdffff700 1 Processor -- start 2024-09-13T18:26:46.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdffff700 1 -- start start 2024-09-13T18:26:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce00a9490 0x7f3ce00c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ce00c8f70 0x7f3ce00c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ce00c9350 0x7f3ce0157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce006d550 con 0x7f3ce00c9350 2024-09-13T18:26:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00c3d40 con 0x7f3ce00a9490 2024-09-13T18:26:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce015d790 con 0x7f3ce00c8f70 2024-09-13T18:26:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ce00c9350 0x7f3ce0157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ce00c9350 0x7f3ce0157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51136/0 (socket says 172.21.15.146:51136) 2024-09-13T18:26:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdf7fe700 1 -- 172.21.15.146:0/1774184647 learned_addr learned my addr 172.21.15.146:0/1774184647 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:46.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdeffd700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce00a9490 0x7f3ce00c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cde7fc700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ce00c8f70 0x7f3ce00c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdf7fe700 1 -- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ce00c8f70 msgr2=0x7f3ce00c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdf7fe700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ce00c8f70 0x7f3ce00c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdf7fe700 1 -- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce00a9490 msgr2=0x7f3ce00c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdf7fe700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce00a9490 0x7f3ce00c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdf7fe700 1 -- 172.21.15.146:0/1774184647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd8009a70 con 0x7f3ce00c9350 2024-09-13T18:26:46.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdf7fe700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ce00c9350 0x7f3ce0157e40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b680 tx=0x7f3cd8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:46.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cd77fe700 1 -- 172.21.15.146:0/1774184647 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8012430 con 0x7f3ce00c9350 2024-09-13T18:26:46.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cd77fe700 1 -- 172.21.15.146:0/1774184647 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8020070 con 0x7f3ce00c9350 2024-09-13T18:26:46.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cd77fe700 1 -- 172.21.15.146:0/1774184647 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8004280 con 0x7f3ce00c9350 2024-09-13T18:26:46.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdffff700 1 -- 172.21.15.146:0/1774184647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce015da30 con 0x7f3ce00c9350 2024-09-13T18:26:46.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.561+0000 7f3cdffff700 1 -- 172.21.15.146:0/1774184647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce006da80 con 0x7f3ce00c9350 2024-09-13T18:26:46.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.565+0000 7f3cdffff700 1 -- 172.21.15.146:0/1774184647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce00c3ef0 con 0x7f3ce00c9350 2024-09-13T18:26:46.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.565+0000 7f3cd77fe700 1 -- 172.21.15.146:0/1774184647 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3cd80043e0 con 0x7f3ce00c9350 2024-09-13T18:26:46.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.565+0000 7f3cd77fe700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3cc4066730 0x7f3cc4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.565+0000 7f3cd77fe700 1 -- 172.21.15.146:0/1774184647 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3cd8022070 con 0x7f3ce00c9350 2024-09-13T18:26:46.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.565+0000 7f3cdeffd700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3cc4066730 0x7f3cc4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.565+0000 7f3cdeffd700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3cc4066730 0x7f3cc4068bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009b80 tx=0x7f3cd0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:46.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.573+0000 7f3cd77fe700 1 -- 172.21.15.146:0/1774184647 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3cd80b7db0 con 0x7f3ce00c9350 2024-09-13T18:26:46.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.745+0000 7f3cdffff700 1 -- 172.21.15.146:0/1774184647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ce00c12a0 con 0x7f3ce00c9350 2024-09-13T18:26:46.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.745+0000 7f3cd77fe700 1 -- 172.21.15.146:0/1774184647 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ce00c3ef0 con 0x7f3ce00c9350 2024-09-13T18:26:46.750 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-09-13T18:26:46.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.745+0000 7f3cdffff700 1 -- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3cc4066730 msgr2=0x7f3cc4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.745+0000 7f3cdffff700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3cc4066730 0x7f3cc4068bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009b80 tx=0x7f3cd0008040 comp rx=0 tx=0).stop 2024-09-13T18:26:46.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.745+0000 7f3cdffff700 1 -- 172.21.15.146:0/1774184647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ce00c9350 msgr2=0x7f3ce0157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.745+0000 7f3cdffff700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ce00c9350 0x7f3ce0157e40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b680 tx=0x7f3cd8004af0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.745+0000 7f3cdffff700 1 -- 172.21.15.146:0/1774184647 shutdown_connections 2024-09-13T18:26:46.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.745+0000 7f3cdffff700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3cc4066730 0x7f3cc4068bf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.745+0000 7f3cdffff700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ce00c9350 0x7f3ce0157e40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.745+0000 7f3cdffff700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ce00c8f70 0x7f3ce00c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.745+0000 7f3cdffff700 1 --2- 172.21.15.146:0/1774184647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce00a9490 0x7f3ce00c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.745+0000 7f3cdffff700 1 -- 172.21.15.146:0/1774184647 >> 172.21.15.146:0/1774184647 conn(0x7f3ce0066090 msgr2=0x7f3ce00cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:46.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.745+0000 7f3cdffff700 1 -- 172.21.15.146:0/1774184647 shutdown_connections 2024-09-13T18:26:46.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.749+0000 7f3cdffff700 1 -- 172.21.15.146:0/1774184647 wait complete. 2024-09-13T18:26:46.765 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:26:46.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.877+0000 7fd217fff700 1 Processor -- start 2024-09-13T18:26:46.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.877+0000 7fd217fff700 1 -- start start 2024-09-13T18:26:46.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.877+0000 7fd217fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd21815d960 0x7fd2180c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.877+0000 7fd217fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd2180ca500 0x7fd2180cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.877+0000 7fd217fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2180c5e50 con 0x7fd2181539e0 2024-09-13T18:26:46.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.877+0000 7fd217fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2180cc3a0 con 0x7fd2180ca500 2024-09-13T18:26:46.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.877+0000 7fd217fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180cb480 con 0x7fd21815d960 2024-09-13T18:26:46.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd216ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd2181539e0 0x7fd2180b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50638/0 (socket says 172.21.15.146:50638) 2024-09-13T18:26:46.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd2167fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd21815d960 0x7fd2180c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd216ffd700 1 -- 172.21.15.146:0/3507979434 learned_addr learned my addr 172.21.15.146:0/3507979434 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:46.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd2177fe700 1 --2- 172.21.15.146:0/3507979434 >> v2:172.21.15.154:3300/0 conn(0x7fd2180ca500 0x7fd2180cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd215ffb700 1 -- 172.21.15.146:0/3507979434 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1154632512 0 0) 0x7fd20c001110 con 0x7fd2181539e0 2024-09-13T18:26:46.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd2167fc700 1 -- 172.21.15.146:0/3507979434 >> v2:172.21.15.154:3300/0 conn(0x7fd2180ca500 msgr2=0x7fd2180cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:46.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd2167fc700 1 --2- 172.21.15.146:0/3507979434 >> v2:172.21.15.154:3300/0 conn(0x7fd2180ca500 0x7fd2180cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd2167fc700 1 -- 172.21.15.146:0/3507979434 >> v1:172.21.15.146:6789/0 conn(0x7fd2181539e0 legacy=0x7fd2180b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd2167fc700 1 -- 172.21.15.146:0/3507979434 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180cb720 con 0x7fd21815d960 2024-09-13T18:26:46.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd2167fc700 1 --2- 172.21.15.146:0/3507979434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd21815d960 0x7fd2180c9fc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd208009850 tx=0x7fd208009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=568b89e06be51bc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:46.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd215ffb700 1 -- 172.21.15.146:0/3507979434 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208011830 con 0x7fd21815d960 2024-09-13T18:26:46.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd215ffb700 1 -- 172.21.15.146:0/3507979434 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd208011990 con 0x7fd21815d960 2024-09-13T18:26:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd215ffb700 1 -- 172.21.15.146:0/3507979434 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208011d40 con 0x7fd21815d960 2024-09-13T18:26:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd217fff700 1 -- 172.21.15.146:0/3507979434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd21815d960 msgr2=0x7fd2180c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd217fff700 1 --2- 172.21.15.146:0/3507979434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd21815d960 0x7fd2180c9fc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd208009850 tx=0x7fd208009b60 comp rx=0 tx=0).stop 2024-09-13T18:26:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd217fff700 1 -- 172.21.15.146:0/3507979434 shutdown_connections 2024-09-13T18:26:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd217fff700 1 --2- 172.21.15.146:0/3507979434 >> v2:172.21.15.154:3300/0 conn(0x7fd2180ca500 0x7fd2180cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd217fff700 1 --2- 172.21.15.146:0/3507979434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd21815d960 0x7fd2180c9fc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd217fff700 1 -- 172.21.15.146:0/3507979434 >> 172.21.15.146:0/3507979434 conn(0x7fd218071ec0 msgr2=0x7fd2180722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd217fff700 1 -- 172.21.15.146:0/3507979434 shutdown_connections 2024-09-13T18:26:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.881+0000 7fd217fff700 1 -- 172.21.15.146:0/3507979434 wait complete. 2024-09-13T18:26:46.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd217fff700 1 Processor -- start 2024-09-13T18:26:46.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd217fff700 1 -- start start 2024-09-13T18:26:46.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd217fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2180ca500 0x7fd2180cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd217fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2181539e0 0x7fd21814ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd217fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd21815d960 0x7fd21814e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd217fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21814b960 con 0x7fd2180ca500 2024-09-13T18:26:46.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd217fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21814cc30 con 0x7fd21815d960 2024-09-13T18:26:46.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd217fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21814b5e0 con 0x7fd2181539e0 2024-09-13T18:26:46.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd216ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2180ca500 0x7fd2180cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd216ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2180ca500 0x7fd2180cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51142/0 (socket says 172.21.15.146:51142) 2024-09-13T18:26:46.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd216ffd700 1 -- 172.21.15.146:0/3132532271 learned_addr learned my addr 172.21.15.146:0/3132532271 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:46.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd2167fc700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2181539e0 0x7fd21814ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd2177fe700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd21815d960 0x7fd21814e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd216ffd700 1 -- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2181539e0 msgr2=0x7fd21814ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd216ffd700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2181539e0 0x7fd21814ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd216ffd700 1 -- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd21815d960 msgr2=0x7fd21814e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:46.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd216ffd700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd21815d960 0x7fd21814e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:46.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd216ffd700 1 -- 172.21.15.146:0/3132532271 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2080094b0 con 0x7fd2180ca500 2024-09-13T18:26:46.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd216ffd700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2180ca500 0x7fd2180cd100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00c750 tx=0x7fd20c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:46.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd1ff7fe700 1 -- 172.21.15.146:0/3132532271 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c004ca0 con 0x7fd2180ca500 2024-09-13T18:26:46.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd1ff7fe700 1 -- 172.21.15.146:0/3132532271 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20c004e00 con 0x7fd2180ca500 2024-09-13T18:26:46.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd1ff7fe700 1 -- 172.21.15.146:0/3132532271 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c007310 con 0x7fd2180ca500 2024-09-13T18:26:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd217fff700 1 -- 172.21.15.146:0/3132532271 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2181ef6b0 con 0x7fd2180ca500 2024-09-13T18:26:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.885+0000 7fd217fff700 1 -- 172.21.15.146:0/3132532271 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2181efbe0 con 0x7fd2180ca500 2024-09-13T18:26:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.889+0000 7fd217fff700 1 -- 172.21.15.146:0/3132532271 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21814de00 con 0x7fd2180ca500 2024-09-13T18:26:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.889+0000 7fd1ff7fe700 1 -- 172.21.15.146:0/3132532271 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd20c007470 con 0x7fd2180ca500 2024-09-13T18:26:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.889+0000 7fd1ff7fe700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd204066730 0x7fd204068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.889+0000 7fd1ff7fe700 1 -- 172.21.15.146:0/3132532271 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd20c083f00 con 0x7fd2180ca500 2024-09-13T18:26:46.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.889+0000 7fd2167fc700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd204066730 0x7fd204068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:46.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.893+0000 7fd2167fc700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd204066730 0x7fd204068bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd2080090b0 tx=0x7fd20801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:46.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:46.897+0000 7fd1ff7fe700 1 -- 172.21.15.146:0/3132532271 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd21814de00 con 0x7fd2180ca500 2024-09-13T18:26:47.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.065+0000 7fd217fff700 1 -- 172.21.15.146:0/3132532271 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd21814de00 con 0x7fd2180ca500 2024-09-13T18:26:47.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.065+0000 7fd1ff7fe700 1 -- 172.21.15.146:0/3132532271 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd21814de00 con 0x7fd2180ca500 2024-09-13T18:26:47.071 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:26:47.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.065+0000 7fd217fff700 1 -- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd204066730 msgr2=0x7fd204068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.065+0000 7fd217fff700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd204066730 0x7fd204068bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd2080090b0 tx=0x7fd20801c040 comp rx=0 tx=0).stop 2024-09-13T18:26:47.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.065+0000 7fd217fff700 1 -- 172.21.15.146:0/3132532271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2180ca500 msgr2=0x7fd2180cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.065+0000 7fd217fff700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2180ca500 0x7fd2180cd100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00c750 tx=0x7fd20c00ca60 comp rx=0 tx=0).stop 2024-09-13T18:26:47.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.065+0000 7fd217fff700 1 -- 172.21.15.146:0/3132532271 shutdown_connections 2024-09-13T18:26:47.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.065+0000 7fd217fff700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd204066730 0x7fd204068bf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.065+0000 7fd217fff700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd21815d960 0x7fd21814e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.065+0000 7fd217fff700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2181539e0 0x7fd21814ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.065+0000 7fd217fff700 1 --2- 172.21.15.146:0/3132532271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2180ca500 0x7fd2180cd100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.065+0000 7fd217fff700 1 -- 172.21.15.146:0/3132532271 >> 172.21.15.146:0/3132532271 conn(0x7fd218071ec0 msgr2=0x7fd2180b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:47.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.069+0000 7fd217fff700 1 -- 172.21.15.146:0/3132532271 shutdown_connections 2024-09-13T18:26:47.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.069+0000 7fd217fff700 1 -- 172.21.15.146:0/3132532271 wait complete. 2024-09-13T18:26:47.086 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:26:47.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9c0252b700 1 Processor -- start 2024-09-13T18:26:47.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9c0252b700 1 -- start start 2024-09-13T18:26:47.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9c0252b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bfc0712a0 0x7f9bfc1694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9c0252b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc0c6380 con 0x7f9bfc15e650 2024-09-13T18:26:47.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9c0252b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc158360 con 0x7f9bfc157f80 2024-09-13T18:26:47.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9c0252b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0c60a0 con 0x7f9bfc0712a0 2024-09-13T18:26:47.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfb7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9bfc15e650 0x7f9bfc070b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50652/0 (socket says 172.21.15.146:50652) 2024-09-13T18:26:47.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfb7fe700 1 -- 172.21.15.146:0/4294638598 learned_addr learned my addr 172.21.15.146:0/4294638598 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:47.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9c00ac8700 1 --2- 172.21.15.146:0/4294638598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bfc0712a0 0x7f9bfc1694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231746671 0 0) 0x7f9bfc0c6380 con 0x7f9bfc15e650 2024-09-13T18:26:47.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9be40035d0 con 0x7f9bfc15e650 2024-09-13T18:26:47.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1865746091 0 0) 0x7f9be40035d0 con 0x7f9bfc15e650 2024-09-13T18:26:47.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bfc0712a0 msgr2=0x7f9bfc1694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:47.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 --2- 172.21.15.146:0/4294638598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bfc0712a0 0x7f9bfc1694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 >> v1:172.21.15.154:6789/0 conn(0x7f9bfc157f80 legacy=0x7f9bfc0c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0c0ba0 con 0x7f9bfc15e650 2024-09-13T18:26:47.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f9bf40019f0 con 0x7f9bfc15e650 2024-09-13T18:26:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 >> v1:172.21.15.146:6789/0 conn(0x7f9bfc15e650 legacy=0x7f9bfc070b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 --2- 172.21.15.146:0/4294638598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bfc0712a0 0x7f9be4003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 --2- 172.21.15.146:0/4294638598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9be4003f90 0x7f9be4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 --2- 172.21.15.146:0/4294638598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9be4004db0 0x7f9be4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0c60a0 con 0x7f9be4004db0 2024-09-13T18:26:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be400bb40 con 0x7f9be4003f90 2024-09-13T18:26:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be400bd00 con 0x7f9bfc0712a0 2024-09-13T18:26:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9c00ac8700 1 --2- 172.21.15.146:0/4294638598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9be4004db0 0x7f9be4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfbfff700 1 --2- 172.21.15.146:0/4294638598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bfc0712a0 0x7f9be4003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9bfb7fe700 1 --2- 172.21.15.146:0/4294638598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9be4003f90 0x7f9be4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9c00ac8700 1 -- 172.21.15.146:0/4294638598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bfc0712a0 msgr2=0x7f9be4003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9c00ac8700 1 --2- 172.21.15.146:0/4294638598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bfc0712a0 0x7f9be4003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9c00ac8700 1 -- 172.21.15.146:0/4294638598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9be4003f90 msgr2=0x7f9be4004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:47.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9c00ac8700 1 --2- 172.21.15.146:0/4294638598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9be4003f90 0x7f9be4004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.189+0000 7f9c00ac8700 1 -- 172.21.15.146:0/4294638598 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be40035d0 con 0x7f9be4004db0 2024-09-13T18:26:47.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.193+0000 7f9c00ac8700 1 --2- 172.21.15.146:0/4294638598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9be4004db0 0x7f9be4005230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009b50 tx=0x7f9be40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a0cbc65ec2e8b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:47.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.193+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00e6c0 con 0x7f9be4004db0 2024-09-13T18:26:47.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.193+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec00e820 con 0x7f9be4004db0 2024-09-13T18:26:47.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.193+0000 7f9bfaffd700 1 -- 172.21.15.146:0/4294638598 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00ebd0 con 0x7f9be4004db0 2024-09-13T18:26:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.193+0000 7f9c0252b700 1 -- 172.21.15.146:0/4294638598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9be4004db0 msgr2=0x7f9be4005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.193+0000 7f9c0252b700 1 --2- 172.21.15.146:0/4294638598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9be4004db0 0x7f9be4005230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009b50 tx=0x7f9be40058a0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.193+0000 7f9c00ac8700 1 -- 172.21.15.146:0/4294638598 reap_dead start 2024-09-13T18:26:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.193+0000 7f9c0252b700 1 -- 172.21.15.146:0/4294638598 shutdown_connections 2024-09-13T18:26:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.193+0000 7f9c0252b700 1 -- 172.21.15.146:0/4294638598 >> 172.21.15.146:0/4294638598 conn(0x7f9bfc06f5c0 msgr2=0x7f9bfc06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.193+0000 7f9c0252b700 1 -- 172.21.15.146:0/4294638598 shutdown_connections 2024-09-13T18:26:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.193+0000 7f9c0252b700 1 -- 172.21.15.146:0/4294638598 wait complete. 2024-09-13T18:26:47.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9c0252b700 1 Processor -- start 2024-09-13T18:26:47.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9c0252b700 1 -- start start 2024-09-13T18:26:47.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9c0252b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9be4004db0 0x7f9bfc0aa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9c0252b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bfc0c5280 0x7f9bfc0b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9c0252b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bfc0af1b0 0x7f9bfc0af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9c0252b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0b0ef0 con 0x7f9be4004db0 2024-09-13T18:26:47.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9c0252b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0afb70 con 0x7f9bfc0c5280 2024-09-13T18:26:47.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9c0252b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0b0c50 con 0x7f9bfc0af1b0 2024-09-13T18:26:47.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9bfbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9be4004db0 0x7f9bfc0aa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9bfbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9be4004db0 0x7f9bfc0aa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51158/0 (socket says 172.21.15.146:51158) 2024-09-13T18:26:47.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9bfbfff700 1 -- 172.21.15.146:0/1043144296 learned_addr learned my addr 172.21.15.146:0/1043144296 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:47.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9bfb7fe700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bfc0c5280 0x7f9bfc0b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9c00ac8700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bfc0af1b0 0x7f9bfc0af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9bfbfff700 1 -- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bfc0af1b0 msgr2=0x7f9bfc0af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9bfbfff700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bfc0af1b0 0x7f9bfc0af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9bfbfff700 1 -- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bfc0c5280 msgr2=0x7f9bfc0b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9bfbfff700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bfc0c5280 0x7f9bfc0b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9bfbfff700 1 -- 172.21.15.146:0/1043144296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc1f3ab0 con 0x7f9be4004db0 2024-09-13T18:26:47.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.197+0000 7f9bfbfff700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9be4004db0 0x7f9bfc0aa010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000f890 tx=0x7f9bf000fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:47.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.201+0000 7f9bf8ff9700 1 -- 172.21.15.146:0/1043144296 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00052e0 con 0x7f9be4004db0 2024-09-13T18:26:47.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.201+0000 7f9bf8ff9700 1 -- 172.21.15.146:0/1043144296 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf0005460 con 0x7f9be4004db0 2024-09-13T18:26:47.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.201+0000 7f9c0252b700 1 -- 172.21.15.146:0/1043144296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bfc1f3d90 con 0x7f9be4004db0 2024-09-13T18:26:47.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.201+0000 7f9bf8ff9700 1 -- 172.21.15.146:0/1043144296 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00038e0 con 0x7f9be4004db0 2024-09-13T18:26:47.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.201+0000 7f9c0252b700 1 -- 172.21.15.146:0/1043144296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc0ae0c0 con 0x7f9be4004db0 2024-09-13T18:26:47.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.201+0000 7f9c0252b700 1 -- 172.21.15.146:0/1043144296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bfc0aeed0 con 0x7f9be4004db0 2024-09-13T18:26:47.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.201+0000 7f9bf8ff9700 1 -- 172.21.15.146:0/1043144296 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9bf0003a40 con 0x7f9be4004db0 2024-09-13T18:26:47.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.201+0000 7f9bf8ff9700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9be8066730 0x7f9be8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.201+0000 7f9bf8ff9700 1 -- 172.21.15.146:0/1043144296 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9bf0084d50 con 0x7f9be4004db0 2024-09-13T18:26:47.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.201+0000 7f9bfb7fe700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9be8066730 0x7f9be8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.205+0000 7f9bfb7fe700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9be8066730 0x7f9be8068bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40018a0 tx=0x7f9bf4001830 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:47.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.209+0000 7f9bf8ff9700 1 -- 172.21.15.146:0/1043144296 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9bf0051ca0 con 0x7f9be4004db0 2024-09-13T18:26:47.238 INFO:tasks.ceph.mds.c.smithi146.stderr:2024-09-13T18:26:47.233+0000 7f6be472a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:26:47.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.389+0000 7f9c0252b700 1 -- 172.21.15.146:0/1043144296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9bfc0aeed0 con 0x7f9be4004db0 2024-09-13T18:26:47.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.389+0000 7f9bf8ff9700 1 -- 172.21.15.146:0/1043144296 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9bfc0aeed0 con 0x7f9be4004db0 2024-09-13T18:26:47.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.393+0000 7f9c0252b700 1 -- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9be8066730 msgr2=0x7f9be8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.393+0000 7f9c0252b700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9be8066730 0x7f9be8068bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40018a0 tx=0x7f9bf4001830 comp rx=0 tx=0).stop 2024-09-13T18:26:47.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.393+0000 7f9c0252b700 1 -- 172.21.15.146:0/1043144296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9be4004db0 msgr2=0x7f9bfc0aa010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.393+0000 7f9c0252b700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9be4004db0 0x7f9bfc0aa010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000f890 tx=0x7f9bf000fba0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.393+0000 7f9c0252b700 1 -- 172.21.15.146:0/1043144296 shutdown_connections 2024-09-13T18:26:47.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.393+0000 7f9c0252b700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9be8066730 0x7f9be8068bf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.393+0000 7f9c0252b700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bfc0af1b0 0x7f9bfc0af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.393+0000 7f9c0252b700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bfc0c5280 0x7f9bfc0b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.393+0000 7f9c0252b700 1 --2- 172.21.15.146:0/1043144296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9be4004db0 0x7f9bfc0aa010 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.393+0000 7f9c0252b700 1 -- 172.21.15.146:0/1043144296 >> 172.21.15.146:0/1043144296 conn(0x7f9bfc06f5c0 msgr2=0x7f9bfc06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:47.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.393+0000 7f9c0252b700 1 -- 172.21.15.146:0/1043144296 shutdown_connections 2024-09-13T18:26:47.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.393+0000 7f9c0252b700 1 -- 172.21.15.146:0/1043144296 wait complete. 2024-09-13T18:26:47.402 INFO:teuthology.orchestra.run.smithi146.stderr:listed 0 entries 2024-09-13T18:26:47.412 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:26:47.412 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:26:47.417 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:26:47.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.521+0000 7f422e35e700 1 Processor -- start 2024-09-13T18:26:47.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.521+0000 7f422e35e700 1 -- start start 2024-09-13T18:26:47.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4228154a40 0x7f42280721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f422806f7d0 0x7f4228157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42280cac60 0x7f42280cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4228072810 con 0x7f422806f7d0 2024-09-13T18:26:47.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4228157910 con 0x7f4228154a40 2024-09-13T18:26:47.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42280c8c70 con 0x7f42280cac60 2024-09-13T18:26:47.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422cb5b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f422806f7d0 0x7f4228157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422cb5b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f422806f7d0 0x7f4228157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51160/0 (socket says 172.21.15.146:51160) 2024-09-13T18:26:47.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422cb5b700 1 -- 172.21.15.146:0/3548924340 learned_addr learned my addr 172.21.15.146:0/3548924340 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:47.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422db5d700 1 --2- 172.21.15.146:0/3548924340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42280cac60 0x7f42280cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422d35c700 1 --2- 172.21.15.146:0/3548924340 >> v2:172.21.15.154:3300/0 conn(0x7f4228154a40 0x7f42280721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422cb5b700 1 -- 172.21.15.146:0/3548924340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42280cac60 msgr2=0x7f42280cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:47.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422cb5b700 1 --2- 172.21.15.146:0/3548924340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42280cac60 0x7f42280cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422cb5b700 1 -- 172.21.15.146:0/3548924340 >> v2:172.21.15.154:3300/0 conn(0x7f4228154a40 msgr2=0x7f42280721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:47.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422cb5b700 1 --2- 172.21.15.146:0/3548924340 >> v2:172.21.15.154:3300/0 conn(0x7f4228154a40 0x7f42280721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422cb5b700 1 -- 172.21.15.146:0/3548924340 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42280c8f50 con 0x7f422806f7d0 2024-09-13T18:26:47.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422d35c700 1 --2- 172.21.15.146:0/3548924340 >> v2:172.21.15.154:3300/0 conn(0x7f4228154a40 0x7f42280721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:26:47.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422cb5b700 1 --2- 172.21.15.146:0/3548924340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f422806f7d0 0x7f4228157d30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f42280cbbe0 tx=0x7f422400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db38f75f8510d176 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:47.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f421bfff700 1 -- 172.21.15.146:0/3548924340 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422400e070 con 0x7f422806f7d0 2024-09-13T18:26:47.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f421bfff700 1 -- 172.21.15.146:0/3548924340 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422400b9c0 con 0x7f422806f7d0 2024-09-13T18:26:47.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f421bfff700 1 -- 172.21.15.146:0/3548924340 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422400bd70 con 0x7f422806f7d0 2024-09-13T18:26:47.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 -- 172.21.15.146:0/3548924340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f422806f7d0 msgr2=0x7f4228157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 --2- 172.21.15.146:0/3548924340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f422806f7d0 0x7f4228157d30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f42280cbbe0 tx=0x7f422400b330 comp rx=0 tx=0).stop 2024-09-13T18:26:47.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 -- 172.21.15.146:0/3548924340 shutdown_connections 2024-09-13T18:26:47.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 --2- 172.21.15.146:0/3548924340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42280cac60 0x7f42280cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 --2- 172.21.15.146:0/3548924340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f422806f7d0 0x7f4228157d30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 --2- 172.21.15.146:0/3548924340 >> v2:172.21.15.154:3300/0 conn(0x7f4228154a40 0x7f42280721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 -- 172.21.15.146:0/3548924340 >> 172.21.15.146:0/3548924340 conn(0x7f4228066090 msgr2=0x7f4228066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:47.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 -- 172.21.15.146:0/3548924340 shutdown_connections 2024-09-13T18:26:47.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.525+0000 7f422e35e700 1 -- 172.21.15.146:0/3548924340 wait complete. 2024-09-13T18:26:47.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422e35e700 1 Processor -- start 2024-09-13T18:26:47.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422e35e700 1 -- start start 2024-09-13T18:26:47.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422e35e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f422806f7d0 0x7f42280c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422e35e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42280cac60 0x7f42281f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422e35e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4228154a40 0x7f42280c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422e35e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42280afc40 con 0x7f4228154a40 2024-09-13T18:26:47.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422e35e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42280c2fe0 con 0x7f42280cac60 2024-09-13T18:26:47.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422e35e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42280c3660 con 0x7f422806f7d0 2024-09-13T18:26:47.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422db5d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4228154a40 0x7f42280c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422db5d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4228154a40 0x7f42280c3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51164/0 (socket says 172.21.15.146:51164) 2024-09-13T18:26:47.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422db5d700 1 -- 172.21.15.146:0/2879929084 learned_addr learned my addr 172.21.15.146:0/2879929084 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:47.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422d35c700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f422806f7d0 0x7f42280c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422cb5b700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42280cac60 0x7f42281f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422db5d700 1 -- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f422806f7d0 msgr2=0x7f42280c2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422db5d700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f422806f7d0 0x7f42280c2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422db5d700 1 -- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42280cac60 msgr2=0x7f42281f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.529+0000 7f422db5d700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42280cac60 0x7f42281f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.533+0000 7f422db5d700 1 -- 172.21.15.146:0/2879929084 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4224009b30 con 0x7f4228154a40 2024-09-13T18:26:47.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.533+0000 7f422db5d700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4228154a40 0x7f42280c3af0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f422000d830 tx=0x7f422000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:47.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.533+0000 7f4219ffb700 1 -- 172.21.15.146:0/2879929084 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200125f0 con 0x7f4228154a40 2024-09-13T18:26:47.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.533+0000 7f4219ffb700 1 -- 172.21.15.146:0/2879929084 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4220012770 con 0x7f4228154a40 2024-09-13T18:26:47.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.533+0000 7f422e35e700 1 -- 172.21.15.146:0/2879929084 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42280c3360 con 0x7f4228154a40 2024-09-13T18:26:47.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.533+0000 7f4219ffb700 1 -- 172.21.15.146:0/2879929084 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220012b20 con 0x7f4228154a40 2024-09-13T18:26:47.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.533+0000 7f422e35e700 1 -- 172.21.15.146:0/2879929084 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42280707b0 con 0x7f4228154a40 2024-09-13T18:26:47.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.533+0000 7f422e35e700 1 -- 172.21.15.146:0/2879929084 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42280c9e00 con 0x7f4228154a40 2024-09-13T18:26:47.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.533+0000 7f4219ffb700 1 -- 172.21.15.146:0/2879929084 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4220012c80 con 0x7f4228154a40 2024-09-13T18:26:47.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.533+0000 7f4219ffb700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f420c066730 0x7f420c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.533+0000 7f4219ffb700 1 -- 172.21.15.146:0/2879929084 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f42200843f0 con 0x7f4228154a40 2024-09-13T18:26:47.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.533+0000 7f422d35c700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f420c066730 0x7f420c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.537+0000 7f422d35c700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f420c066730 0x7f420c068bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f421c006fd0 tx=0x7f421c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:47.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.541+0000 7f4219ffb700 1 -- 172.21.15.146:0/2879929084 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4220051500 con 0x7f4228154a40 2024-09-13T18:26:47.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.713+0000 7f422e35e700 1 -- 172.21.15.146:0/2879929084 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42280c6bd0 con 0x7f4228154a40 2024-09-13T18:26:47.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.717+0000 7f4219ffb700 1 -- 172.21.15.146:0/2879929084 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f4220054b20 con 0x7f4228154a40 2024-09-13T18:26:47.723 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:47.723 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:26:47.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.717+0000 7f422e35e700 1 -- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f420c066730 msgr2=0x7f420c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.717+0000 7f422e35e700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f420c066730 0x7f420c068bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f421c006fd0 tx=0x7f421c008040 comp rx=0 tx=0).stop 2024-09-13T18:26:47.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.717+0000 7f422e35e700 1 -- 172.21.15.146:0/2879929084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4228154a40 msgr2=0x7f42280c3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.717+0000 7f422e35e700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4228154a40 0x7f42280c3af0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f422000d830 tx=0x7f422000db40 comp rx=0 tx=0).stop 2024-09-13T18:26:47.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.717+0000 7f422e35e700 1 -- 172.21.15.146:0/2879929084 shutdown_connections 2024-09-13T18:26:47.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.717+0000 7f422e35e700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f420c066730 0x7f420c068bf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.717+0000 7f422e35e700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4228154a40 0x7f42280c3af0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.717+0000 7f422e35e700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42280cac60 0x7f42281f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.717+0000 7f422e35e700 1 --2- 172.21.15.146:0/2879929084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f422806f7d0 0x7f42280c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.717+0000 7f422e35e700 1 -- 172.21.15.146:0/2879929084 >> 172.21.15.146:0/2879929084 conn(0x7f4228066090 msgr2=0x7f42280c5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:47.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.717+0000 7f422e35e700 1 -- 172.21.15.146:0/2879929084 shutdown_connections 2024-09-13T18:26:47.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.721+0000 7f422e35e700 1 -- 172.21.15.146:0/2879929084 wait complete. 2024-09-13T18:26:47.738 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:26:24.867515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:26:30.782269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:26:31.323277+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:26:34.393386+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:26:47.739 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:26:47.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.849+0000 7fcbdffff700 1 Processor -- start 2024-09-13T18:26:47.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.849+0000 7fcbdffff700 1 -- start start 2024-09-13T18:26:47.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcbe01539e0 0x7fcbe00b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe00c5e50 con 0x7fcbe00cf580 2024-09-13T18:26:47.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdffff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe00cb480 con 0x7fcbe015d960 2024-09-13T18:26:47.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdffff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe00cc3a0 con 0x7fcbe01539e0 2024-09-13T18:26:47.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdf7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fcbe00cf580 0x7fcbe00cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50662/0 (socket says 172.21.15.146:50662) 2024-09-13T18:26:47.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdf7fe700 1 -- 172.21.15.146:0/3756609473 learned_addr learned my addr 172.21.15.146:0/3756609473 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:47.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdeffd700 1 --2- 172.21.15.146:0/3756609473 >> v2:172.21.15.146:3300/0 conn(0x7fcbe01539e0 0x7fcbe00b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbddffb700 1 -- 172.21.15.146:0/3756609473 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169690086 0 0) 0x7fcbd8001110 con 0x7fcbe00cf580 2024-09-13T18:26:47.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbddffb700 1 -- 172.21.15.146:0/3756609473 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbc80035d0 con 0x7fcbe00cf580 2024-09-13T18:26:47.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdeffd700 1 -- 172.21.15.146:0/3756609473 >> v1:172.21.15.154:6789/0 conn(0x7fcbe015d960 legacy=0x7fcbe00c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdeffd700 1 -- 172.21.15.146:0/3756609473 >> v1:172.21.15.146:6789/0 conn(0x7fcbe00cf580 legacy=0x7fcbe00cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdeffd700 1 -- 172.21.15.146:0/3756609473 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe00cb720 con 0x7fcbe01539e0 2024-09-13T18:26:47.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdeffd700 1 --2- 172.21.15.146:0/3756609473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe01539e0 0x7fcbe00b3370 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009dc0 tx=0x7fcbd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821131c0656ff275 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:47.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbddffb700 1 -- 172.21.15.146:0/3756609473 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000e070 con 0x7fcbe01539e0 2024-09-13T18:26:47.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbddffb700 1 -- 172.21.15.146:0/3756609473 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd000bae0 con 0x7fcbe01539e0 2024-09-13T18:26:47.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbddffb700 1 -- 172.21.15.146:0/3756609473 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000be90 con 0x7fcbe01539e0 2024-09-13T18:26:47.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdffff700 1 -- 172.21.15.146:0/3756609473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe01539e0 msgr2=0x7fcbe00b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdffff700 1 --2- 172.21.15.146:0/3756609473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe01539e0 0x7fcbe00b3370 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009dc0 tx=0x7fcbd000b330 comp rx=0 tx=0).stop 2024-09-13T18:26:47.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdffff700 1 -- 172.21.15.146:0/3756609473 shutdown_connections 2024-09-13T18:26:47.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdffff700 1 --2- 172.21.15.146:0/3756609473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe01539e0 0x7fcbe00b3370 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdffff700 1 -- 172.21.15.146:0/3756609473 >> 172.21.15.146:0/3756609473 conn(0x7fcbe0071ec0 msgr2=0x7fcbe00722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:47.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdffff700 1 -- 172.21.15.146:0/3756609473 shutdown_connections 2024-09-13T18:26:47.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.853+0000 7fcbdffff700 1 -- 172.21.15.146:0/3756609473 wait complete. 2024-09-13T18:26:47.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.857+0000 7fcbdffff700 1 Processor -- start 2024-09-13T18:26:47.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.857+0000 7fcbdffff700 1 -- start start 2024-09-13T18:26:47.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.857+0000 7fcbdffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbe00cf580 0x7fcbe014b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.857+0000 7fcbdffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe015d960 0x7fcbe00c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.857+0000 7fcbdffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbe00c6a80 0x7fcbe01447f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.857+0000 7fcbdffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe00c6f20 con 0x7fcbe015d960 2024-09-13T18:26:47.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.857+0000 7fcbdffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe01444a0 con 0x7fcbe00cf580 2024-09-13T18:26:47.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.857+0000 7fcbdffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe0148950 con 0x7fcbe00c6a80 2024-09-13T18:26:47.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbde7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe015d960 0x7fcbe00c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbde7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe015d960 0x7fcbe00c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51184/0 (socket says 172.21.15.146:51184) 2024-09-13T18:26:47.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbde7fc700 1 -- 172.21.15.146:0/1349917816 learned_addr learned my addr 172.21.15.146:0/1349917816 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:47.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbdf7fe700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbe00c6a80 0x7fcbe01447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbdeffd700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbe00cf580 0x7fcbe014b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbdf7fe700 1 -- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbe00cf580 msgr2=0x7fcbe014b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbdf7fe700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbe00cf580 0x7fcbe014b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbdf7fe700 1 -- 172.21.15.146:0/1349917816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe015d960 msgr2=0x7fcbe00c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:47.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbdf7fe700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe015d960 0x7fcbe00c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:47.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbdf7fe700 1 -- 172.21.15.146:0/1349917816 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd0009a70 con 0x7fcbe00c6a80 2024-09-13T18:26:47.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbde7fc700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe015d960 0x7fcbe00c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:47.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbdf7fe700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbe00c6a80 0x7fcbe01447f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800c730 tx=0x7fcbd800ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:47.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbc7fff700 1 -- 172.21.15.146:0/1349917816 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8005d90 con 0x7fcbe00c6a80 2024-09-13T18:26:47.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbc7fff700 1 -- 172.21.15.146:0/1349917816 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd8001d30 con 0x7fcbe00c6a80 2024-09-13T18:26:47.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbdffff700 1 -- 172.21.15.146:0/1349917816 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe0146bd0 con 0x7fcbe00c6a80 2024-09-13T18:26:47.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbc7fff700 1 -- 172.21.15.146:0/1349917816 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd80072f0 con 0x7fcbe00c6a80 2024-09-13T18:26:47.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbdffff700 1 -- 172.21.15.146:0/1349917816 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe0147190 con 0x7fcbe00c6a80 2024-09-13T18:26:47.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbdffff700 1 -- 172.21.15.146:0/1349917816 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe00cab20 con 0x7fcbe00c6a80 2024-09-13T18:26:47.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.861+0000 7fcbc7fff700 1 -- 172.21.15.146:0/1349917816 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcbd8007450 con 0x7fcbe00c6a80 2024-09-13T18:26:47.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.865+0000 7fcbc7fff700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcbcc066730 0x7fcbcc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:47.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.865+0000 7fcbc7fff700 1 -- 172.21.15.146:0/1349917816 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcbd8083440 con 0x7fcbe00c6a80 2024-09-13T18:26:47.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.865+0000 7fcbdeffd700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcbcc066730 0x7fcbcc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:47.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.865+0000 7fcbdeffd700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcbcc066730 0x7fcbcc068bf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00d0180 tx=0x7fcbd0011000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:47.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:47.873+0000 7fcbc7fff700 1 -- 172.21.15.146:0/1349917816 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcbd80504c0 con 0x7fcbe00c6a80 2024-09-13T18:26:48.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.057+0000 7fcbdffff700 1 -- 172.21.15.146:0/1349917816 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcbe00c5e50 con 0x7fcbe00c6a80 2024-09-13T18:26:48.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.057+0000 7fcbc7fff700 1 -- 172.21.15.146:0/1349917816 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcbe00c5e50 con 0x7fcbe00c6a80 2024-09-13T18:26:48.065 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-09-13T18:26:48.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.061+0000 7fcbdffff700 1 -- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcbcc066730 msgr2=0x7fcbcc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.061+0000 7fcbdffff700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcbcc066730 0x7fcbcc068bf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00d0180 tx=0x7fcbd0011000 comp rx=0 tx=0).stop 2024-09-13T18:26:48.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.061+0000 7fcbdffff700 1 -- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbe00c6a80 msgr2=0x7fcbe01447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.061+0000 7fcbdffff700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbe00c6a80 0x7fcbe01447f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800c730 tx=0x7fcbd800ca40 comp rx=0 tx=0).stop 2024-09-13T18:26:48.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.061+0000 7fcbdffff700 1 -- 172.21.15.146:0/1349917816 shutdown_connections 2024-09-13T18:26:48.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.061+0000 7fcbdffff700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcbcc066730 0x7fcbcc068bf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.061+0000 7fcbdffff700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbe00c6a80 0x7fcbe01447f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.061+0000 7fcbdffff700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe015d960 0x7fcbe00c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.061+0000 7fcbdffff700 1 --2- 172.21.15.146:0/1349917816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbe00cf580 0x7fcbe014b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.061+0000 7fcbdffff700 1 -- 172.21.15.146:0/1349917816 >> 172.21.15.146:0/1349917816 conn(0x7fcbe0071ec0 msgr2=0x7fcbe00b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:48.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.061+0000 7fcbdffff700 1 -- 172.21.15.146:0/1349917816 shutdown_connections 2024-09-13T18:26:48.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.061+0000 7fcbdffff700 1 -- 172.21.15.146:0/1349917816 wait complete. 2024-09-13T18:26:48.080 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:26:48.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.189+0000 7f035759e700 1 Processor -- start 2024-09-13T18:26:48.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.189+0000 7f035759e700 1 -- start start 2024-09-13T18:26:48.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035759e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f035815d960 0x7f03580c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035759e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03580c5e50 con 0x7f03581539e0 2024-09-13T18:26:48.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035759e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03580cb480 con 0x7f03580ca500 2024-09-13T18:26:48.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035759e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03580cc3a0 con 0x7f035815d960 2024-09-13T18:26:48.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f0355d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f035815d960 0x7f03580c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f0355d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f035815d960 0x7f03580c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51188/0 (socket says 172.21.15.146:51188) 2024-09-13T18:26:48.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f0355d9b700 1 -- 172.21.15.146:0/1389028008 learned_addr learned my addr 172.21.15.146:0/1389028008 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:48.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f0355d9b700 1 -- 172.21.15.146:0/1389028008 >> v1:172.21.15.154:6789/0 conn(0x7f03580ca500 legacy=0x7f03580cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:48.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f0355d9b700 1 -- 172.21.15.146:0/1389028008 >> v1:172.21.15.146:6789/0 conn(0x7f03581539e0 legacy=0x7f03580b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f0355d9b700 1 -- 172.21.15.146:0/1389028008 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03580cb720 con 0x7f035815d960 2024-09-13T18:26:48.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f0355d9b700 1 --2- 172.21.15.146:0/1389028008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f035815d960 0x7f03580c9fc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0348009e80 tx=0x7f034800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281467cb07038e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:48.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035559a700 1 -- 172.21.15.146:0/1389028008 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034800e070 con 0x7f035815d960 2024-09-13T18:26:48.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035559a700 1 -- 172.21.15.146:0/1389028008 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0348003a40 con 0x7f035815d960 2024-09-13T18:26:48.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035559a700 1 -- 172.21.15.146:0/1389028008 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348003df0 con 0x7f035815d960 2024-09-13T18:26:48.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035759e700 1 -- 172.21.15.146:0/1389028008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f035815d960 msgr2=0x7f03580c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035759e700 1 --2- 172.21.15.146:0/1389028008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f035815d960 0x7f03580c9fc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0348009e80 tx=0x7f034800b330 comp rx=0 tx=0).stop 2024-09-13T18:26:48.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035759e700 1 -- 172.21.15.146:0/1389028008 shutdown_connections 2024-09-13T18:26:48.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035759e700 1 --2- 172.21.15.146:0/1389028008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f035815d960 0x7f03580c9fc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035759e700 1 -- 172.21.15.146:0/1389028008 >> 172.21.15.146:0/1389028008 conn(0x7f0358071ec0 msgr2=0x7f03580722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:48.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035759e700 1 -- 172.21.15.146:0/1389028008 shutdown_connections 2024-09-13T18:26:48.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.193+0000 7f035759e700 1 -- 172.21.15.146:0/1389028008 wait complete. 2024-09-13T18:26:48.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f035759e700 1 Processor -- start 2024-09-13T18:26:48.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f035759e700 1 -- start start 2024-09-13T18:26:48.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f035759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03580ca500 0x7f03580ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f035759e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03581539e0 0x7f03580c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f035759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f035815d960 0x7f03580c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f035759e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03580adbf0 con 0x7f035815d960 2024-09-13T18:26:48.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f035759e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03580c7530 con 0x7f03581539e0 2024-09-13T18:26:48.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f035759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03580c77d0 con 0x7f03580ca500 2024-09-13T18:26:48.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f0356d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f035815d960 0x7f03580c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f0356d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f035815d960 0x7f03580c4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51192/0 (socket says 172.21.15.146:51192) 2024-09-13T18:26:48.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f0356d9d700 1 -- 172.21.15.146:0/4271631146 learned_addr learned my addr 172.21.15.146:0/4271631146 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:48.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f035659c700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03580ca500 0x7f03580ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f0355d9b700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03581539e0 0x7f03580c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f0356d9d700 1 -- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03580ca500 msgr2=0x7f03580ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f0356d9d700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03580ca500 0x7f03580ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f0356d9d700 1 -- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03581539e0 msgr2=0x7f03580c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f0356d9d700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03581539e0 0x7f03580c41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f0356d9d700 1 -- 172.21.15.146:0/4271631146 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0348009b30 con 0x7f035815d960 2024-09-13T18:26:48.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.197+0000 7f0356d9d700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f035815d960 0x7f03580c4700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f035000c470 tx=0x7f035000c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:48.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.201+0000 7f034effd700 1 -- 172.21.15.146:0/4271631146 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350011790 con 0x7f035815d960 2024-09-13T18:26:48.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.201+0000 7f034effd700 1 -- 172.21.15.146:0/4271631146 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03500118f0 con 0x7f035815d960 2024-09-13T18:26:48.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.201+0000 7f034effd700 1 -- 172.21.15.146:0/4271631146 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350011ca0 con 0x7f035815d960 2024-09-13T18:26:48.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.201+0000 7f035759e700 1 -- 172.21.15.146:0/4271631146 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03580c84f0 con 0x7f035815d960 2024-09-13T18:26:48.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.201+0000 7f035759e700 1 -- 172.21.15.146:0/4271631146 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03580c8ab0 con 0x7f035815d960 2024-09-13T18:26:48.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.201+0000 7f035759e700 1 -- 172.21.15.146:0/4271631146 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03580c4c40 con 0x7f035815d960 2024-09-13T18:26:48.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.201+0000 7f034effd700 1 -- 172.21.15.146:0/4271631146 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f035001c460 con 0x7f035815d960 2024-09-13T18:26:48.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.205+0000 7f034effd700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0338066760 0x7f0338068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.205+0000 7f034effd700 1 -- 172.21.15.146:0/4271631146 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0350084010 con 0x7f035815d960 2024-09-13T18:26:48.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.205+0000 7f035659c700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0338066760 0x7f0338068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.205+0000 7f035659c700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0338066760 0x7f0338068c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f03400018a0 tx=0x7f0340009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:48.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.209+0000 7f034effd700 1 -- 172.21.15.146:0/4271631146 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f03580c4c40 con 0x7f035815d960 2024-09-13T18:26:48.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f035759e700 1 -- 172.21.15.146:0/4271631146 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03581f0780 con 0x7f035815d960 2024-09-13T18:26:48.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f034effd700 1 -- 172.21.15.146:0/4271631146 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03581f0780 con 0x7f035815d960 2024-09-13T18:26:48.391 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:26:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f035759e700 1 -- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0338066760 msgr2=0x7f0338068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f035759e700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0338066760 0x7f0338068c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f03400018a0 tx=0x7f0340009040 comp rx=0 tx=0).stop 2024-09-13T18:26:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f035759e700 1 -- 172.21.15.146:0/4271631146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f035815d960 msgr2=0x7f03580c4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f035759e700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f035815d960 0x7f03580c4700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f035000c470 tx=0x7f035000c780 comp rx=0 tx=0).stop 2024-09-13T18:26:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f035759e700 1 -- 172.21.15.146:0/4271631146 shutdown_connections 2024-09-13T18:26:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f035759e700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0338066760 0x7f0338068c20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f035759e700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f035815d960 0x7f03580c4700 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f035759e700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03581539e0 0x7f03580c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f035759e700 1 --2- 172.21.15.146:0/4271631146 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03580ca500 0x7f03580ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f035759e700 1 -- 172.21.15.146:0/4271631146 >> 172.21.15.146:0/4271631146 conn(0x7f0358071ec0 msgr2=0x7f03580b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f035759e700 1 -- 172.21.15.146:0/4271631146 shutdown_connections 2024-09-13T18:26:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.385+0000 7f035759e700 1 -- 172.21.15.146:0/4271631146 wait complete. 2024-09-13T18:26:48.410 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:26:48.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.521+0000 7f08a359e700 1 Processor -- start 2024-09-13T18:26:48.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.521+0000 7f08a359e700 1 -- start start 2024-09-13T18:26:48.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f08a40cf580 0x7f08a40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a359e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a40c5e50 con 0x7f08a415d960 2024-09-13T18:26:48.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a359e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a40cff30 con 0x7f08a41539e0 2024-09-13T18:26:48.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a40cc3a0 con 0x7f08a40cf580 2024-09-13T18:26:48.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a1d9b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f08a415d960 0x7f08a40c9fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50688/0 (socket says 172.21.15.146:50688) 2024-09-13T18:26:48.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a1d9b700 1 -- 172.21.15.146:0/4001908871 learned_addr learned my addr 172.21.15.146:0/4001908871 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:48.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a2d9d700 1 --2- 172.21.15.146:0/4001908871 >> v2:172.21.15.154:3300/0 conn(0x7f08a40cf580 0x7f08a40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345293479 0 0) 0x7f0894001110 con 0x7f08a415d960 2024-09-13T18:26:48.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f088c0035d0 con 0x7f08a415d960 2024-09-13T18:26:48.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2642827118 0 0) 0x7f088c0035d0 con 0x7f08a415d960 2024-09-13T18:26:48.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 >> v2:172.21.15.154:3300/0 conn(0x7f08a40cf580 msgr2=0x7f08a40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:48.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 --2- 172.21.15.146:0/4001908871 >> v2:172.21.15.154:3300/0 conn(0x7f08a40cf580 0x7f08a40cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 >> v1:172.21.15.154:6789/0 conn(0x7f08a41539e0 legacy=0x7f08a40b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a40cb480 con 0x7f08a415d960 2024-09-13T18:26:48.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f0894001800 con 0x7f08a415d960 2024-09-13T18:26:48.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 >> v1:172.21.15.146:6789/0 conn(0x7f08a415d960 legacy=0x7f08a40c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 --2- 172.21.15.146:0/4001908871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f088c003a90 0x7f088c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 --2- 172.21.15.146:0/4001908871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088c004870 0x7f088c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 --2- 172.21.15.146:0/4001908871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c005230 0x7f088c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40cc3a0 con 0x7f088c004870 2024-09-13T18:26:48.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c00c160 con 0x7f088c003a90 2024-09-13T18:26:48.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c00c320 con 0x7f088c005230 2024-09-13T18:26:48.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a1d9b700 1 --2- 172.21.15.146:0/4001908871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088c004870 0x7f088c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a259c700 1 --2- 172.21.15.146:0/4001908871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f088c003a90 0x7f088c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a2d9d700 1 --2- 172.21.15.146:0/4001908871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c005230 0x7f088c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a1d9b700 1 -- 172.21.15.146:0/4001908871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c005230 msgr2=0x7f088c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a1d9b700 1 --2- 172.21.15.146:0/4001908871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c005230 0x7f088c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a1d9b700 1 -- 172.21.15.146:0/4001908871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f088c003a90 msgr2=0x7f088c003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a1d9b700 1 --2- 172.21.15.146:0/4001908871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f088c003a90 0x7f088c003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a1d9b700 1 -- 172.21.15.146:0/4001908871 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c0035d0 con 0x7f088c004870 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a2d9d700 1 -- 172.21.15.146:0/4001908871 reap_dead start 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a1d9b700 1 --2- 172.21.15.146:0/4001908871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088c004870 0x7f088c004cf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f089400c240 tx=0x7f088c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51eb860b33e0814 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894007da0 con 0x7f088c004870 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089400cd60 con 0x7f088c004870 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.525+0000 7f08a159a700 1 -- 172.21.15.146:0/4001908871 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894005bc0 con 0x7f088c004870 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 -- 172.21.15.146:0/4001908871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088c004870 msgr2=0x7f088c004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 --2- 172.21.15.146:0/4001908871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088c004870 0x7f088c004cf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f089400c240 tx=0x7f088c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 -- 172.21.15.146:0/4001908871 shutdown_connections 2024-09-13T18:26:48.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 --2- 172.21.15.146:0/4001908871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088c004870 0x7f088c004cf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 -- 172.21.15.146:0/4001908871 >> 172.21.15.146:0/4001908871 conn(0x7f08a4071ec0 msgr2=0x7f08a40722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:48.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 -- 172.21.15.146:0/4001908871 shutdown_connections 2024-09-13T18:26:48.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 -- 172.21.15.146:0/4001908871 wait complete. 2024-09-13T18:26:48.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 Processor -- start 2024-09-13T18:26:48.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 -- start start 2024-09-13T18:26:48.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088c004870 0x7f08a41ef800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08a40c8740 0x7f08a414ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08a40c7f10 0x7f08a4142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a41e3390 con 0x7f088c004870 2024-09-13T18:26:48.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40c8f10 con 0x7f08a40c7f10 2024-09-13T18:26:48.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40c9440 con 0x7f08a40c8740 2024-09-13T18:26:48.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088c004870 0x7f08a41ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.529+0000 7f08a259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088c004870 0x7f08a41ef800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51216/0 (socket says 172.21.15.146:51216) 2024-09-13T18:26:48.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f08a259c700 1 -- 172.21.15.146:0/3913303817 learned_addr learned my addr 172.21.15.146:0/3913303817 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:48.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f08a1d9b700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08a40c8740 0x7f08a414ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f08a2d9d700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08a40c7f10 0x7f08a4142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f08a259c700 1 -- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08a40c8740 msgr2=0x7f08a414ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f08a259c700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08a40c8740 0x7f08a414ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f08a259c700 1 -- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08a40c7f10 msgr2=0x7f08a4142bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f08a259c700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08a40c7f10 0x7f08a4142bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f08a259c700 1 -- 172.21.15.146:0/3913303817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a40c9160 con 0x7f088c004870 2024-09-13T18:26:48.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f08a259c700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088c004870 0x7f08a41ef800 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f089800f7d0 tx=0x7f089800fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:48.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f0892ffd700 1 -- 172.21.15.146:0/3913303817 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08980052e0 con 0x7f088c004870 2024-09-13T18:26:48.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f0892ffd700 1 -- 172.21.15.146:0/3913303817 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0898005440 con 0x7f088c004870 2024-09-13T18:26:48.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f08a359e700 1 -- 172.21.15.146:0/3913303817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a41e2cd0 con 0x7f088c004870 2024-09-13T18:26:48.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f0892ffd700 1 -- 172.21.15.146:0/3913303817 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08980038e0 con 0x7f088c004870 2024-09-13T18:26:48.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f08a359e700 1 -- 172.21.15.146:0/3913303817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a414cb40 con 0x7f088c004870 2024-09-13T18:26:48.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.533+0000 7f08a359e700 1 -- 172.21.15.146:0/3913303817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a4153dc0 con 0x7f088c004870 2024-09-13T18:26:48.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.537+0000 7f0892ffd700 1 -- 172.21.15.146:0/3913303817 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0898003a40 con 0x7f088c004870 2024-09-13T18:26:48.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.537+0000 7f0892ffd700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0884066730 0x7f0884068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.537+0000 7f0892ffd700 1 -- 172.21.15.146:0/3913303817 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0898084100 con 0x7f088c004870 2024-09-13T18:26:48.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.537+0000 7f08a1d9b700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0884066730 0x7f0884068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.537+0000 7f08a1d9b700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0884066730 0x7f0884068bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0894015040 tx=0x7f089400a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:48.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.541+0000 7f0892ffd700 1 -- 172.21.15.146:0/3913303817 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0898051210 con 0x7f088c004870 2024-09-13T18:26:48.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.709+0000 7f08a359e700 1 -- 172.21.15.146:0/3913303817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08a40c8520 con 0x7f088c004870 2024-09-13T18:26:48.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.709+0000 7f0892ffd700 1 -- 172.21.15.146:0/3913303817 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0898054830 con 0x7f088c004870 2024-09-13T18:26:48.715 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-09-13T18:26:48.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.713+0000 7f08a359e700 1 -- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0884066730 msgr2=0x7f0884068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.713+0000 7f08a359e700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0884066730 0x7f0884068bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0894015040 tx=0x7f089400a000 comp rx=0 tx=0).stop 2024-09-13T18:26:48.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.713+0000 7f08a359e700 1 -- 172.21.15.146:0/3913303817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088c004870 msgr2=0x7f08a41ef800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.713+0000 7f08a359e700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088c004870 0x7f08a41ef800 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f089800f7d0 tx=0x7f089800fb90 comp rx=0 tx=0).stop 2024-09-13T18:26:48.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.713+0000 7f08a359e700 1 -- 172.21.15.146:0/3913303817 shutdown_connections 2024-09-13T18:26:48.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.713+0000 7f08a359e700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0884066730 0x7f0884068bf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.713+0000 7f08a359e700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08a40c7f10 0x7f08a4142bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.713+0000 7f08a359e700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08a40c8740 0x7f08a414ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.713+0000 7f08a359e700 1 --2- 172.21.15.146:0/3913303817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088c004870 0x7f08a41ef800 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.713+0000 7f08a359e700 1 -- 172.21.15.146:0/3913303817 >> 172.21.15.146:0/3913303817 conn(0x7f08a4071ec0 msgr2=0x7f08a40ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:48.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.713+0000 7f08a359e700 1 -- 172.21.15.146:0/3913303817 shutdown_connections 2024-09-13T18:26:48.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.713+0000 7f08a359e700 1 -- 172.21.15.146:0/3913303817 wait complete. 2024-09-13T18:26:48.737 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:26:48.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f025d8700 1 Processor -- start 2024-09-13T18:26:48.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f025d8700 1 -- start start 2024-09-13T18:26:48.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f025d8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7efc1538c0 0x7f7efc0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f025d8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7efc15a080 0x7f7efc0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f025d8700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7efc0cb340 con 0x7f7efc0cc680 2024-09-13T18:26:48.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f025d8700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7efc15a510 con 0x7f7efc1538c0 2024-09-13T18:26:48.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f025d8700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7efc0cb5e0 con 0x7f7efc15a080 2024-09-13T18:26:48.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f015d6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7efc1538c0 0x7f7efc0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f015d6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7efc1538c0 0x7f7efc0ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51224/0 (socket says 172.21.15.146:51224) 2024-09-13T18:26:48.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f015d6700 1 -- 172.21.15.146:0/3336031663 learned_addr learned my addr 172.21.15.146:0/3336031663 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:48.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f01dd7700 1 --2- 172.21.15.146:0/3336031663 >> v2:172.21.15.154:3300/0 conn(0x7f7efc15a080 0x7f7efc0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f015d6700 1 -- 172.21.15.146:0/3336031663 >> v2:172.21.15.154:3300/0 conn(0x7f7efc15a080 msgr2=0x7f7efc0cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:48.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f015d6700 1 --2- 172.21.15.146:0/3336031663 >> v2:172.21.15.154:3300/0 conn(0x7f7efc15a080 0x7f7efc0cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f015d6700 1 -- 172.21.15.146:0/3336031663 >> v1:172.21.15.154:6789/0 conn(0x7f7efc0cc680 legacy=0x7f7efc0c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.853+0000 7f7f015d6700 1 -- 172.21.15.146:0/3336031663 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc0cb8c0 con 0x7f7efc1538c0 2024-09-13T18:26:48.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f015d6700 1 --2- 172.21.15.146:0/3336031663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7efc1538c0 0x7f7efc0ca240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009910 tx=0x7f7eec009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be06452223337395 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:48.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7ef3fff700 1 -- 172.21.15.146:0/3336031663 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec011830 con 0x7f7efc1538c0 2024-09-13T18:26:48.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7ef3fff700 1 -- 172.21.15.146:0/3336031663 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eec011990 con 0x7f7efc1538c0 2024-09-13T18:26:48.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7ef3fff700 1 -- 172.21.15.146:0/3336031663 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec011d40 con 0x7f7efc1538c0 2024-09-13T18:26:48.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 -- 172.21.15.146:0/3336031663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7efc1538c0 msgr2=0x7f7efc0ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 --2- 172.21.15.146:0/3336031663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7efc1538c0 0x7f7efc0ca240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009910 tx=0x7f7eec009c20 comp rx=0 tx=0).stop 2024-09-13T18:26:48.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 -- 172.21.15.146:0/3336031663 shutdown_connections 2024-09-13T18:26:48.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 --2- 172.21.15.146:0/3336031663 >> v2:172.21.15.154:3300/0 conn(0x7f7efc15a080 0x7f7efc0cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 --2- 172.21.15.146:0/3336031663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7efc1538c0 0x7f7efc0ca240 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 -- 172.21.15.146:0/3336031663 >> 172.21.15.146:0/3336031663 conn(0x7f7efc066270 msgr2=0x7f7efc066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:48.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 -- 172.21.15.146:0/3336031663 shutdown_connections 2024-09-13T18:26:48.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 -- 172.21.15.146:0/3336031663 wait complete. 2024-09-13T18:26:48.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 Processor -- start 2024-09-13T18:26:48.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 -- start start 2024-09-13T18:26:48.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7efc0cc680 0x7f7efc1e2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7efc1538c0 0x7f7efc15ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7efc15a080 0x7f7efc15b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc1e3210 con 0x7f7efc15a080 2024-09-13T18:26:48.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc1e64c0 con 0x7f7efc0cc680 2024-09-13T18:26:48.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f025d8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc15a770 con 0x7f7efc1538c0 2024-09-13T18:26:48.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f01dd7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7efc15a080 0x7f7efc15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f01dd7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7efc15a080 0x7f7efc15b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51236/0 (socket says 172.21.15.146:51236) 2024-09-13T18:26:48.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f01dd7700 1 -- 172.21.15.146:0/2710744920 learned_addr learned my addr 172.21.15.146:0/2710744920 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:48.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f015d6700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7efc0cc680 0x7f7efc1e2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.857+0000 7f7f00dd5700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7efc1538c0 0x7f7efc15ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7f01dd7700 1 -- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7efc1538c0 msgr2=0x7f7efc15ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7f01dd7700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7efc1538c0 0x7f7efc15ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7f01dd7700 1 -- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7efc0cc680 msgr2=0x7f7efc1e2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:48.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7f01dd7700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7efc0cc680 0x7f7efc1e2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:48.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7f01dd7700 1 -- 172.21.15.146:0/2710744920 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec009570 con 0x7f7efc15a080 2024-09-13T18:26:48.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7f015d6700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7efc0cc680 0x7f7efc1e2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:48.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7f01dd7700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7efc15a080 0x7f7efc15b140 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009d30 tx=0x7f7ef400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:48.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7ef1ffb700 1 -- 172.21.15.146:0/2710744920 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4009ba0 con 0x7f7efc15a080 2024-09-13T18:26:48.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7ef1ffb700 1 -- 172.21.15.146:0/2710744920 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef400f040 con 0x7f7efc15a080 2024-09-13T18:26:48.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7f025d8700 1 -- 172.21.15.146:0/2710744920 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc15d0f0 con 0x7f7efc15a080 2024-09-13T18:26:48.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7ef1ffb700 1 -- 172.21.15.146:0/2710744920 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4014480 con 0x7f7efc15a080 2024-09-13T18:26:48.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7f025d8700 1 -- 172.21.15.146:0/2710744920 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc14cfc0 con 0x7f7efc15a080 2024-09-13T18:26:48.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7f025d8700 1 -- 172.21.15.146:0/2710744920 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc0c99c0 con 0x7f7efc15a080 2024-09-13T18:26:48.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7ef1ffb700 1 -- 172.21.15.146:0/2710744920 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7ef40145e0 con 0x7f7efc15a080 2024-09-13T18:26:48.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7ef1ffb700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7ee0066730 0x7f7ee0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:48.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.861+0000 7f7ef1ffb700 1 -- 172.21.15.146:0/2710744920 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7ef40850d0 con 0x7f7efc15a080 2024-09-13T18:26:48.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.865+0000 7f7f015d6700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7ee0066730 0x7f7ee0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:48.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.865+0000 7f7f015d6700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7ee0066730 0x7f7ee0068bf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7eec000c00 tx=0x7f7eec008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:48.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:48.869+0000 7f7ef1ffb700 1 -- 172.21.15.146:0/2710744920 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7ef4052150 con 0x7f7efc15a080 2024-09-13T18:26:49.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.037+0000 7f7f025d8700 1 -- 172.21.15.146:0/2710744920 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7efc15a8b0 con 0x7f7efc15a080 2024-09-13T18:26:49.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.037+0000 7f7ef1ffb700 1 -- 172.21.15.146:0/2710744920 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7ef4055770 con 0x7f7efc15a080 2024-09-13T18:26:49.044 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:26:49.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.041+0000 7f7f025d8700 1 -- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7ee0066730 msgr2=0x7f7ee0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:49.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.041+0000 7f7f025d8700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7ee0066730 0x7f7ee0068bf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7eec000c00 tx=0x7f7eec008030 comp rx=0 tx=0).stop 2024-09-13T18:26:49.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.041+0000 7f7f025d8700 1 -- 172.21.15.146:0/2710744920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7efc15a080 msgr2=0x7f7efc15b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:49.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.041+0000 7f7f025d8700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7efc15a080 0x7f7efc15b140 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009d30 tx=0x7f7ef400c3a0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.041+0000 7f7f025d8700 1 -- 172.21.15.146:0/2710744920 shutdown_connections 2024-09-13T18:26:49.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.041+0000 7f7f025d8700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7ee0066730 0x7f7ee0068bf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.041+0000 7f7f025d8700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7efc15a080 0x7f7efc15b140 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.041+0000 7f7f025d8700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7efc1538c0 0x7f7efc15ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.041+0000 7f7f025d8700 1 --2- 172.21.15.146:0/2710744920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7efc0cc680 0x7f7efc1e2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.041+0000 7f7f025d8700 1 -- 172.21.15.146:0/2710744920 >> 172.21.15.146:0/2710744920 conn(0x7f7efc066270 msgr2=0x7f7efc066650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:49.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.041+0000 7f7f025d8700 1 -- 172.21.15.146:0/2710744920 shutdown_connections 2024-09-13T18:26:49.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.041+0000 7f7f025d8700 1 -- 172.21.15.146:0/2710744920 wait complete. 2024-09-13T18:26:49.058 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:26:49.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.161+0000 7f92a4ae1700 1 Processor -- start 2024-09-13T18:26:49.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.161+0000 7f92a4ae1700 1 -- start start 2024-09-13T18:26:49.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f92a4ae1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f92a0150760 0x7f92a0071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:49.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f92a4ae1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92a00cc900 0x7f92a01565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:49.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f92a4ae1700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a006e680 con 0x7f92a00cc520 2024-09-13T18:26:49.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f92a4ae1700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a0071b30 con 0x7f92a0150760 2024-09-13T18:26:49.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f92a4ae1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a006e960 con 0x7f92a00cc900 2024-09-13T18:26:49.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f929effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f92a0150760 0x7f92a0071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:49.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f929effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f92a0150760 0x7f92a0071480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51244/0 (socket says 172.21.15.146:51244) 2024-09-13T18:26:49.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f929effd700 1 -- 172.21.15.146:0/2099979358 learned_addr learned my addr 172.21.15.146:0/2099979358 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:49.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f929ffff700 1 --2- 172.21.15.146:0/2099979358 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92a00cc900 0x7f92a01565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:49.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f929effd700 1 -- 172.21.15.146:0/2099979358 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92a00cc900 msgr2=0x7f92a01565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:49.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f929effd700 1 --2- 172.21.15.146:0/2099979358 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92a00cc900 0x7f92a01565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f929effd700 1 -- 172.21.15.146:0/2099979358 >> v1:172.21.15.154:6789/0 conn(0x7f92a00cc520 legacy=0x7f92a0150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:49.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f929effd700 1 -- 172.21.15.146:0/2099979358 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a006ec40 con 0x7f92a0150760 2024-09-13T18:26:49.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f929effd700 1 --2- 172.21.15.146:0/2099979358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a0150760 0x7f92a0071480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9288009910 tx=0x7f9288009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a049b19a0ef20 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:49.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f929e7fc700 1 -- 172.21.15.146:0/2099979358 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928800ba60 con 0x7f92a0150760 2024-09-13T18:26:49.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f929e7fc700 1 -- 172.21.15.146:0/2099979358 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928800bbc0 con 0x7f92a0150760 2024-09-13T18:26:49.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f929e7fc700 1 -- 172.21.15.146:0/2099979358 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9288009e60 con 0x7f92a0150760 2024-09-13T18:26:49.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f92a4ae1700 1 -- 172.21.15.146:0/2099979358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a0150760 msgr2=0x7f92a0071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:49.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f92a4ae1700 1 --2- 172.21.15.146:0/2099979358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a0150760 0x7f92a0071480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9288009910 tx=0x7f9288009c20 comp rx=0 tx=0).stop 2024-09-13T18:26:49.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f92a4ae1700 1 -- 172.21.15.146:0/2099979358 shutdown_connections 2024-09-13T18:26:49.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f92a4ae1700 1 --2- 172.21.15.146:0/2099979358 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92a00cc900 0x7f92a01565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f92a4ae1700 1 --2- 172.21.15.146:0/2099979358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a0150760 0x7f92a0071480 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f92a4ae1700 1 -- 172.21.15.146:0/2099979358 >> 172.21.15.146:0/2099979358 conn(0x7f92a0071ed0 msgr2=0x7f92a00722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:49.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f92a4ae1700 1 -- 172.21.15.146:0/2099979358 shutdown_connections 2024-09-13T18:26:49.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.165+0000 7f92a4ae1700 1 -- 172.21.15.146:0/2099979358 wait complete. 2024-09-13T18:26:49.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f92a4ae1700 1 Processor -- start 2024-09-13T18:26:49.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f92a4ae1700 1 -- start start 2024-09-13T18:26:49.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f92a4ae1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00cc520 0x7f92a0152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:49.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f92a4ae1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a00cc900 0x7f92a0149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:49.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f92a4ae1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92a00b0b00 0x7f92a00b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:49.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f92a4ae1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a00b14d0 con 0x7f92a00cc900 2024-09-13T18:26:49.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f92a4ae1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a014d910 con 0x7f92a00cc520 2024-09-13T18:26:49.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f92a4ae1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a01ef6e0 con 0x7f92a00b0b00 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f929effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a00cc900 0x7f92a0149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f929effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a00cc900 0x7f92a0149650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51250/0 (socket says 172.21.15.146:51250) 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f929effd700 1 -- 172.21.15.146:0/1773895618 learned_addr learned my addr 172.21.15.146:0/1773895618 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f929ffff700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92a00b0b00 0x7f92a00b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f929f7fe700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00cc520 0x7f92a0152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f929effd700 1 -- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92a00b0b00 msgr2=0x7f92a00b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f929effd700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92a00b0b00 0x7f92a00b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f929effd700 1 -- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00cc520 msgr2=0x7f92a0152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f929effd700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00cc520 0x7f92a0152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f929effd700 1 -- 172.21.15.146:0/1773895618 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9288009570 con 0x7f92a00cc900 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f929f7fe700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00cc520 0x7f92a0152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f929effd700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a00cc900 0x7f92a0149650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9288014040 tx=0x7f9288004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f9297fff700 1 -- 172.21.15.146:0/1773895618 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9288004610 con 0x7f92a00cc900 2024-09-13T18:26:49.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f9297fff700 1 -- 172.21.15.146:0/1773895618 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928801e070 con 0x7f92a00cc900 2024-09-13T18:26:49.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f92a4ae1700 1 -- 172.21.15.146:0/1773895618 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a0152870 con 0x7f92a00cc900 2024-09-13T18:26:49.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f9297fff700 1 -- 172.21.15.146:0/1773895618 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9288003900 con 0x7f92a00cc900 2024-09-13T18:26:49.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.169+0000 7f92a4ae1700 1 -- 172.21.15.146:0/1773895618 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a01532f0 con 0x7f92a00cc900 2024-09-13T18:26:49.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.173+0000 7f92a4ae1700 1 -- 172.21.15.146:0/1773895618 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a006d5c0 con 0x7f92a00cc900 2024-09-13T18:26:49.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.173+0000 7f9297fff700 1 -- 172.21.15.146:0/1773895618 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9288003a60 con 0x7f92a00cc900 2024-09-13T18:26:49.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.173+0000 7f9297fff700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9280066730 0x7f9280068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:49.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.173+0000 7f9297fff700 1 -- 172.21.15.146:0/1773895618 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9288020070 con 0x7f92a00cc900 2024-09-13T18:26:49.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.173+0000 7f929f7fe700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9280066730 0x7f9280068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:49.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.173+0000 7f929f7fe700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9280066730 0x7f9280068bf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9290000f60 tx=0x7f9290009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:49.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.177+0000 7f9297fff700 1 -- 172.21.15.146:0/1773895618 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f92a006d5c0 con 0x7f92a00cc900 2024-09-13T18:26:49.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.361+0000 7f92a4ae1700 1 -- 172.21.15.146:0/1773895618 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f92a006d5c0 con 0x7f92a00cc900 2024-09-13T18:26:49.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.361+0000 7f9297fff700 1 -- 172.21.15.146:0/1773895618 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f92a006d5c0 con 0x7f92a00cc900 2024-09-13T18:26:49.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.365+0000 7f92a4ae1700 1 -- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9280066730 msgr2=0x7f9280068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:49.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.365+0000 7f92a4ae1700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9280066730 0x7f9280068bf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9290000f60 tx=0x7f9290009040 comp rx=0 tx=0).stop 2024-09-13T18:26:49.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.365+0000 7f92a4ae1700 1 -- 172.21.15.146:0/1773895618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a00cc900 msgr2=0x7f92a0149650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:49.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.365+0000 7f92a4ae1700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a00cc900 0x7f92a0149650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9288014040 tx=0x7f9288004770 comp rx=0 tx=0).stop 2024-09-13T18:26:49.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.365+0000 7f92a4ae1700 1 -- 172.21.15.146:0/1773895618 shutdown_connections 2024-09-13T18:26:49.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.365+0000 7f92a4ae1700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9280066730 0x7f9280068bf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.365+0000 7f92a4ae1700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92a00b0b00 0x7f92a00b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.365+0000 7f92a4ae1700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a00cc900 0x7f92a0149650 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.365+0000 7f92a4ae1700 1 --2- 172.21.15.146:0/1773895618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00cc520 0x7f92a0152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:49.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.365+0000 7f92a4ae1700 1 -- 172.21.15.146:0/1773895618 >> 172.21.15.146:0/1773895618 conn(0x7f92a0071ed0 msgr2=0x7f92a014fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:49.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.365+0000 7f92a4ae1700 1 -- 172.21.15.146:0/1773895618 shutdown_connections 2024-09-13T18:26:49.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:49.365+0000 7f92a4ae1700 1 -- 172.21.15.146:0/1773895618 wait complete. 2024-09-13T18:26:49.372 INFO:teuthology.orchestra.run.smithi146.stderr:listed 0 entries 2024-09-13T18:26:49.383 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-13T18:26:49.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:49.383 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:26:49.395 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi146.stderr:ceph-fuse[31113]: fuse finished with error 0 and tester_r 0 2024-09-13T18:26:49.397 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:26:55.498 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-13T18:26:55.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:26:55.498 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-13T18:26:55.515 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:26:55.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a1a9c0700 1 Processor -- start 2024-09-13T18:26:55.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a1a9c0700 1 -- start start 2024-09-13T18:26:55.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a1a9c0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6a14154a40 0x7f6a140721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:55.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a1a9c0700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a140c8c70 con 0x7f6a1406f7d0 2024-09-13T18:26:55.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a1a9c0700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a140c8f50 con 0x7f6a140cac60 2024-09-13T18:26:55.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a1a9c0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a14072810 con 0x7f6a14154a40 2024-09-13T18:26:55.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a191bd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6a1406f7d0 0x7f6a14157d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:47376/0 (socket says 172.21.15.146:47376) 2024-09-13T18:26:55.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a191bd700 1 -- 172.21.15.146:0/2164813069 learned_addr learned my addr 172.21.15.146:0/2164813069 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:55.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a199be700 1 --2- 172.21.15.146:0/2164813069 >> v2:172.21.15.154:3300/0 conn(0x7f6a14154a40 0x7f6a140721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:55.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a189bc700 1 -- 172.21.15.146:0/2164813069 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663071298 0 0) 0x7f6a140c8c70 con 0x7f6a1406f7d0 2024-09-13T18:26:55.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a189bc700 1 -- 172.21.15.146:0/2164813069 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a000035d0 con 0x7f6a1406f7d0 2024-09-13T18:26:55.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a199be700 1 -- 172.21.15.146:0/2164813069 >> v1:172.21.15.154:6789/0 conn(0x7f6a140cac60 legacy=0x7f6a14161c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:55.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a199be700 1 -- 172.21.15.146:0/2164813069 >> v1:172.21.15.146:6789/0 conn(0x7f6a1406f7d0 legacy=0x7f6a14157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:55.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a199be700 1 -- 172.21.15.146:0/2164813069 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a14157530 con 0x7f6a14154a40 2024-09-13T18:26:55.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a199be700 1 --2- 172.21.15.146:0/2164813069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a14154a40 0x7f6a140721f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009850 tx=0x7f6a04009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b04385ad827434 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:55.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.673+0000 7f6a189bc700 1 -- 172.21.15.146:0/2164813069 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04011830 con 0x7f6a14154a40 2024-09-13T18:26:55.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a189bc700 1 -- 172.21.15.146:0/2164813069 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a04011990 con 0x7f6a14154a40 2024-09-13T18:26:55.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a189bc700 1 -- 172.21.15.146:0/2164813069 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04011d40 con 0x7f6a14154a40 2024-09-13T18:26:55.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2164813069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a14154a40 msgr2=0x7f6a140721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:55.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 --2- 172.21.15.146:0/2164813069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a14154a40 0x7f6a140721f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009850 tx=0x7f6a04009b60 comp rx=0 tx=0).stop 2024-09-13T18:26:55.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2164813069 shutdown_connections 2024-09-13T18:26:55.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 --2- 172.21.15.146:0/2164813069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a14154a40 0x7f6a140721f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:55.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2164813069 >> 172.21.15.146:0/2164813069 conn(0x7f6a14066090 msgr2=0x7f6a14066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:55.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2164813069 shutdown_connections 2024-09-13T18:26:55.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2164813069 wait complete. 2024-09-13T18:26:55.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 Processor -- start 2024-09-13T18:26:55.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 -- start start 2024-09-13T18:26:55.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a1406f7d0 0x7f6a140aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:55.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a140cac60 0x7f6a1406c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:55.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a14154a40 0x7f6a141e2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:55.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14157910 con 0x7f6a14154a40 2024-09-13T18:26:55.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140afc70 con 0x7f6a140cac60 2024-09-13T18:26:55.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140ae1a0 con 0x7f6a1406f7d0 2024-09-13T18:26:55.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a1bf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a14154a40 0x7f6a141e2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:55.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a1bf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a14154a40 0x7f6a141e2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39374/0 (socket says 172.21.15.146:39374) 2024-09-13T18:26:55.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a1bf700 1 -- 172.21.15.146:0/2013717926 learned_addr learned my addr 172.21.15.146:0/2013717926 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a191bd700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a140cac60 0x7f6a1406c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a199be700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a1406f7d0 0x7f6a140aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a1bf700 1 -- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a1406f7d0 msgr2=0x7f6a140aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a1bf700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a1406f7d0 0x7f6a140aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a1bf700 1 -- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a140cac60 msgr2=0x7f6a1406c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a1bf700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a140cac60 0x7f6a1406c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a1bf700 1 -- 172.21.15.146:0/2013717926 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a040094b0 con 0x7f6a14154a40 2024-09-13T18:26:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a1bf700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a14154a40 0x7f6a141e2b90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00cb50 tx=0x7f6a0c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a0a7fc700 1 -- 172.21.15.146:0/2013717926 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c003c20 con 0x7f6a14154a40 2024-09-13T18:26:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a0a7fc700 1 -- 172.21.15.146:0/2013717926 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c003d80 con 0x7f6a14154a40 2024-09-13T18:26:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.677+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2013717926 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a140ae450 con 0x7f6a14154a40 2024-09-13T18:26:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.681+0000 7f6a0a7fc700 1 -- 172.21.15.146:0/2013717926 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00aa20 con 0x7f6a14154a40 2024-09-13T18:26:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.681+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2013717926 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a140adca0 con 0x7f6a14154a40 2024-09-13T18:26:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.681+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2013717926 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a140adec0 con 0x7f6a14154a40 2024-09-13T18:26:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.681+0000 7f6a0a7fc700 1 -- 172.21.15.146:0/2013717926 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6a0c00ab80 con 0x7f6a14154a40 2024-09-13T18:26:55.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.681+0000 7f6a0a7fc700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69f8066730 0x7f69f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:55.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.681+0000 7f6a0a7fc700 1 -- 172.21.15.146:0/2013717926 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6a0c084c80 con 0x7f6a14154a40 2024-09-13T18:26:55.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.685+0000 7f6a199be700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69f8066730 0x7f69f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:55.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.685+0000 7f6a199be700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69f8066730 0x7f69f8068bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a040090b0 tx=0x7f6a04008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:55.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.689+0000 7f6a0a7fc700 1 -- 172.21.15.146:0/2013717926 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6a0c051de0 con 0x7f6a14154a40 2024-09-13T18:26:55.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.861+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2013717926 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a140adec0 con 0x7f6a14154a40 2024-09-13T18:26:55.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.865+0000 7f6a0a7fc700 1 -- 172.21.15.146:0/2013717926 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f6a140adec0 con 0x7f6a14154a40 2024-09-13T18:26:55.872 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:55.872 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:26:55.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.869+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69f8066730 msgr2=0x7f69f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:55.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.869+0000 7f6a1a9c0700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69f8066730 0x7f69f8068bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a040090b0 tx=0x7f6a04008030 comp rx=0 tx=0).stop 2024-09-13T18:26:55.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.869+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2013717926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a14154a40 msgr2=0x7f6a141e2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:55.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.869+0000 7f6a1a9c0700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a14154a40 0x7f6a141e2b90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00cb50 tx=0x7f6a0c00ce60 comp rx=0 tx=0).stop 2024-09-13T18:26:55.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.869+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2013717926 shutdown_connections 2024-09-13T18:26:55.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.869+0000 7f6a1a9c0700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69f8066730 0x7f69f8068bf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:55.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.869+0000 7f6a1a9c0700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a14154a40 0x7f6a141e2b90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:55.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.869+0000 7f6a1a9c0700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a140cac60 0x7f6a1406c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:55.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.869+0000 7f6a1a9c0700 1 --2- 172.21.15.146:0/2013717926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a1406f7d0 0x7f6a140aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:55.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.869+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2013717926 >> 172.21.15.146:0/2013717926 conn(0x7f6a14066090 msgr2=0x7f6a141565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:55.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.869+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2013717926 shutdown_connections 2024-09-13T18:26:55.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.869+0000 7f6a1a9c0700 1 -- 172.21.15.146:0/2013717926 wait complete. 2024-09-13T18:26:55.887 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:26:56.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.993+0000 7f6c55fc2700 1 Processor -- start 2024-09-13T18:26:56.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.993+0000 7f6c55fc2700 1 -- start start 2024-09-13T18:26:56.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c55fc2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6c5014fb50 0x7f6c500c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:56.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c55fc2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6c500c7b00 0x7f6c500c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:56.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c55fc2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c50156810 con 0x7f6c500c6060 2024-09-13T18:26:56.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c55fc2700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c501559b0 con 0x7f6c500c7b00 2024-09-13T18:26:56.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c55fc2700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c50155c90 con 0x7f6c5014fb50 2024-09-13T18:26:56.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c4ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6c500c7b00 0x7f6c500c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:56.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c4ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6c500c7b00 0x7f6c500c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39388/0 (socket says 172.21.15.146:39388) 2024-09-13T18:26:56.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c4ffff700 1 -- 172.21.15.146:0/3276164894 learned_addr learned my addr 172.21.15.146:0/3276164894 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:56.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c54fc0700 1 --2- 172.21.15.146:0/3276164894 >> v2:172.21.15.154:3300/0 conn(0x7f6c5014fb50 0x7f6c500c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:56.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c4ffff700 1 -- 172.21.15.146:0/3276164894 >> v2:172.21.15.154:3300/0 conn(0x7f6c5014fb50 msgr2=0x7f6c500c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:56.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c4ffff700 1 --2- 172.21.15.146:0/3276164894 >> v2:172.21.15.154:3300/0 conn(0x7f6c5014fb50 0x7f6c500c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c4ffff700 1 -- 172.21.15.146:0/3276164894 >> v1:172.21.15.154:6789/0 conn(0x7f6c500c6060 legacy=0x7f6c50159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:56.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c4ffff700 1 -- 172.21.15.146:0/3276164894 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c50155f70 con 0x7f6c500c7b00 2024-09-13T18:26:56.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c54fc0700 1 --2- 172.21.15.146:0/3276164894 >> v2:172.21.15.154:3300/0 conn(0x7f6c5014fb50 0x7f6c500c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:56.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c4ffff700 1 --2- 172.21.15.146:0/3276164894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c500c7b00 0x7f6c500c8510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c006fd0 tx=0x7f6c3c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24dcc424202fc784 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:56.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c4f7fe700 1 -- 172.21.15.146:0/3276164894 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c00dc00 con 0x7f6c500c7b00 2024-09-13T18:26:56.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c4f7fe700 1 -- 172.21.15.146:0/3276164894 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c00dd60 con 0x7f6c500c7b00 2024-09-13T18:26:56.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c4f7fe700 1 -- 172.21.15.146:0/3276164894 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c004250 con 0x7f6c500c7b00 2024-09-13T18:26:56.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c55fc2700 1 -- 172.21.15.146:0/3276164894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c500c7b00 msgr2=0x7f6c500c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:56.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c55fc2700 1 --2- 172.21.15.146:0/3276164894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c500c7b00 0x7f6c500c8510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c006fd0 tx=0x7f6c3c00bea0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c55fc2700 1 -- 172.21.15.146:0/3276164894 shutdown_connections 2024-09-13T18:26:56.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c55fc2700 1 --2- 172.21.15.146:0/3276164894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c500c7b00 0x7f6c500c8510 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c55fc2700 1 --2- 172.21.15.146:0/3276164894 >> v2:172.21.15.154:3300/0 conn(0x7f6c5014fb50 0x7f6c500c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c55fc2700 1 -- 172.21.15.146:0/3276164894 >> 172.21.15.146:0/3276164894 conn(0x7f6c50066b00 msgr2=0x7f6c5006aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:56.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c55fc2700 1 -- 172.21.15.146:0/3276164894 shutdown_connections 2024-09-13T18:26:56.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:55.997+0000 7f6c55fc2700 1 -- 172.21.15.146:0/3276164894 wait complete. 2024-09-13T18:26:56.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c55fc2700 1 Processor -- start 2024-09-13T18:26:56.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c55fc2700 1 -- start start 2024-09-13T18:26:56.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c55fc2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c500c6060 0x7f6c500c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:56.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c55fc2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c500c7b00 0x7f6c500c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:56.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c55fc2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c5014fb50 0x7f6c50157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:56.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c55fc2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c500c3c20 con 0x7f6c500c6060 2024-09-13T18:26:56.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c55fc2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c501eba10 con 0x7f6c500c7b00 2024-09-13T18:26:56.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c55fc2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c50141300 con 0x7f6c5014fb50 2024-09-13T18:26:56.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c54fc0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c500c6060 0x7f6c500c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:56.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c54fc0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c500c6060 0x7f6c500c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39400/0 (socket says 172.21.15.146:39400) 2024-09-13T18:26:56.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c54fc0700 1 -- 172.21.15.146:0/2063403335 learned_addr learned my addr 172.21.15.146:0/2063403335 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:56.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c4ffff700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c500c7b00 0x7f6c500c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:56.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c557c1700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c5014fb50 0x7f6c50157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:56.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c54fc0700 1 -- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c5014fb50 msgr2=0x7f6c50157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:56.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c54fc0700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c5014fb50 0x7f6c50157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c54fc0700 1 -- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c500c7b00 msgr2=0x7f6c500c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:56.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c54fc0700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c500c7b00 0x7f6c500c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c54fc0700 1 -- 172.21.15.146:0/2063403335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c00b820 con 0x7f6c500c6060 2024-09-13T18:26:56.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c54fc0700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c500c6060 0x7f6c500c48b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400e810 tx=0x7f6c4400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:56.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c4d7fa700 1 -- 172.21.15.146:0/2063403335 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c44004230 con 0x7f6c500c6060 2024-09-13T18:26:56.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c4d7fa700 1 -- 172.21.15.146:0/2063403335 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c44004390 con 0x7f6c500c6060 2024-09-13T18:26:56.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c4d7fa700 1 -- 172.21.15.146:0/2063403335 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440036c0 con 0x7f6c500c6060 2024-09-13T18:26:56.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c55fc2700 1 -- 172.21.15.146:0/2063403335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c501419f0 con 0x7f6c500c6060 2024-09-13T18:26:56.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.001+0000 7f6c55fc2700 1 -- 172.21.15.146:0/2063403335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c50141710 con 0x7f6c500c6060 2024-09-13T18:26:56.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.005+0000 7f6c55fc2700 1 -- 172.21.15.146:0/2063403335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c500c8e80 con 0x7f6c500c6060 2024-09-13T18:26:56.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.005+0000 7f6c4d7fa700 1 -- 172.21.15.146:0/2063403335 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6c44003840 con 0x7f6c500c6060 2024-09-13T18:26:56.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.005+0000 7f6c4d7fa700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c34066730 0x7f6c34068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:56.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.005+0000 7f6c4d7fa700 1 -- 172.21.15.146:0/2063403335 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6c44083d00 con 0x7f6c500c6060 2024-09-13T18:26:56.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.005+0000 7f6c4ffff700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c34066730 0x7f6c34068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:56.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.005+0000 7f6c4ffff700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c34066730 0x7f6c34068bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c019040 tx=0x7f6c3c014000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:56.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.013+0000 7f6c4d7fa700 1 -- 172.21.15.146:0/2063403335 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6c44050e10 con 0x7f6c500c6060 2024-09-13T18:26:56.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.185+0000 7f6c55fc2700 1 -- 172.21.15.146:0/2063403335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c500c3410 con 0x7f6c500c6060 2024-09-13T18:26:56.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.185+0000 7f6c4d7fa700 1 -- 172.21.15.146:0/2063403335 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f6c44020090 con 0x7f6c500c6060 2024-09-13T18:26:56.193 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:26:56.193 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:26:56.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.189+0000 7f6c55fc2700 1 -- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c34066730 msgr2=0x7f6c34068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:56.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.189+0000 7f6c55fc2700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c34066730 0x7f6c34068bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c019040 tx=0x7f6c3c014000 comp rx=0 tx=0).stop 2024-09-13T18:26:56.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.189+0000 7f6c55fc2700 1 -- 172.21.15.146:0/2063403335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c500c6060 msgr2=0x7f6c500c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:56.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.189+0000 7f6c55fc2700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c500c6060 0x7f6c500c48b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400e810 tx=0x7f6c4400eb20 comp rx=0 tx=0).stop 2024-09-13T18:26:56.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.189+0000 7f6c55fc2700 1 -- 172.21.15.146:0/2063403335 shutdown_connections 2024-09-13T18:26:56.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.189+0000 7f6c55fc2700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c34066730 0x7f6c34068bf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.189+0000 7f6c55fc2700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c5014fb50 0x7f6c50157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.189+0000 7f6c55fc2700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c500c7b00 0x7f6c500c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.189+0000 7f6c55fc2700 1 --2- 172.21.15.146:0/2063403335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c500c6060 0x7f6c500c48b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.189+0000 7f6c55fc2700 1 -- 172.21.15.146:0/2063403335 >> 172.21.15.146:0/2063403335 conn(0x7f6c50066b00 msgr2=0x7f6c5014d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:56.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.189+0000 7f6c55fc2700 1 -- 172.21.15.146:0/2063403335 shutdown_connections 2024-09-13T18:26:56.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.189+0000 7f6c55fc2700 1 -- 172.21.15.146:0/2063403335 wait complete. 2024-09-13T18:26:56.208 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T18:26:56.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.317+0000 7f5da7a3b700 1 Processor -- start 2024-09-13T18:26:56.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.317+0000 7f5da7a3b700 1 -- start start 2024-09-13T18:26:56.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da7a3b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da00ca020 0x7f5da00ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:56.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da7a3b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5da00cad90 0x7f5da00cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:56.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da7a3b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da00c5bd0 con 0x7f5da00c57f0 2024-09-13T18:26:56.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da7a3b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da015a160 con 0x7f5da00cad90 2024-09-13T18:26:56.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da7a3b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0159830 con 0x7f5da00ca020 2024-09-13T18:26:56.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da5fd8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5da00cad90 0x7f5da00cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:56.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da5fd8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5da00cad90 0x7f5da00cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39408/0 (socket says 172.21.15.146:39408) 2024-09-13T18:26:56.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da5fd8700 1 -- 172.21.15.146:0/2182636220 learned_addr learned my addr 172.21.15.146:0/2182636220 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:56.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da4fd6700 1 --2- 172.21.15.146:0/2182636220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da00ca020 0x7f5da00ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:56.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da5fd8700 1 -- 172.21.15.146:0/2182636220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da00ca020 msgr2=0x7f5da00ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:56.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da5fd8700 1 --2- 172.21.15.146:0/2182636220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da00ca020 0x7f5da00ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da5fd8700 1 -- 172.21.15.146:0/2182636220 >> v1:172.21.15.154:6789/0 conn(0x7f5da00c57f0 legacy=0x7f5da00cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:56.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da5fd8700 1 -- 172.21.15.146:0/2182636220 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da00c9bb0 con 0x7f5da00cad90 2024-09-13T18:26:56.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da5fd8700 1 --2- 172.21.15.146:0/2182636220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5da00cad90 0x7f5da00cf450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009e50 tx=0x7f5d9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3ca622be8d62b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:56.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5d97fff700 1 -- 172.21.15.146:0/2182636220 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9800e070 con 0x7f5da00cad90 2024-09-13T18:26:56.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5d97fff700 1 -- 172.21.15.146:0/2182636220 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9800b910 con 0x7f5da00cad90 2024-09-13T18:26:56.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5d97fff700 1 -- 172.21.15.146:0/2182636220 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9800bcc0 con 0x7f5da00cad90 2024-09-13T18:26:56.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da7a3b700 1 -- 172.21.15.146:0/2182636220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5da00cad90 msgr2=0x7f5da00cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:56.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da7a3b700 1 --2- 172.21.15.146:0/2182636220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5da00cad90 0x7f5da00cf450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009e50 tx=0x7f5d9800b330 comp rx=0 tx=0).stop 2024-09-13T18:26:56.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da7a3b700 1 -- 172.21.15.146:0/2182636220 shutdown_connections 2024-09-13T18:26:56.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da7a3b700 1 --2- 172.21.15.146:0/2182636220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5da00cad90 0x7f5da00cf450 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da7a3b700 1 --2- 172.21.15.146:0/2182636220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da00ca020 0x7f5da00ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da7a3b700 1 -- 172.21.15.146:0/2182636220 >> 172.21.15.146:0/2182636220 conn(0x7f5da0071d30 msgr2=0x7f5da0072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:56.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da7a3b700 1 -- 172.21.15.146:0/2182636220 shutdown_connections 2024-09-13T18:26:56.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.321+0000 7f5da7a3b700 1 -- 172.21.15.146:0/2182636220 wait complete. 2024-09-13T18:26:56.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da7a3b700 1 Processor -- start 2024-09-13T18:26:56.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da7a3b700 1 -- start start 2024-09-13T18:26:56.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da7a3b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5da00c57f0 0x7f5da00cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:56.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da7a3b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da00ca020 0x7f5da00ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:56.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da7a3b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da00cad90 0x7f5da00ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:56.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da7a3b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00ad030 con 0x7f5da00c57f0 2024-09-13T18:26:56.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da7a3b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01ef310 con 0x7f5da00ca020 2024-09-13T18:26:56.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da7a3b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01ef5f0 con 0x7f5da00cad90 2024-09-13T18:26:56.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da57d7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5da00c57f0 0x7f5da00cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:56.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da57d7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5da00c57f0 0x7f5da00cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39410/0 (socket says 172.21.15.146:39410) 2024-09-13T18:26:56.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da57d7700 1 -- 172.21.15.146:0/1437042198 learned_addr learned my addr 172.21.15.146:0/1437042198 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:56.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da4fd6700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da00ca020 0x7f5da00ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:56.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da5fd8700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da00cad90 0x7f5da00ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da57d7700 1 -- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da00cad90 msgr2=0x7f5da00ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da57d7700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da00cad90 0x7f5da00ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da57d7700 1 -- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da00ca020 msgr2=0x7f5da00ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da57d7700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da00ca020 0x7f5da00ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da57d7700 1 -- 172.21.15.146:0/1437042198 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d98009b30 con 0x7f5da00c57f0 2024-09-13T18:26:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da57d7700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5da00c57f0 0x7f5da00cf6c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c500 tx=0x7f5d9000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5d95ffb700 1 -- 172.21.15.146:0/1437042198 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900052d0 con 0x7f5da00c57f0 2024-09-13T18:26:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5d95ffb700 1 -- 172.21.15.146:0/1437042198 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d90005430 con 0x7f5da00c57f0 2024-09-13T18:26:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da7a3b700 1 -- 172.21.15.146:0/1437042198 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da01ef8d0 con 0x7f5da00c57f0 2024-09-13T18:26:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5d95ffb700 1 -- 172.21.15.146:0/1437042198 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900055a0 con 0x7f5da00c57f0 2024-09-13T18:26:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.325+0000 7f5da7a3b700 1 -- 172.21.15.146:0/1437042198 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da01eefe0 con 0x7f5da00c57f0 2024-09-13T18:26:56.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.329+0000 7f5da7a3b700 1 -- 172.21.15.146:0/1437042198 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da00ad2a0 con 0x7f5da00c57f0 2024-09-13T18:26:56.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.329+0000 7f5d95ffb700 1 -- 172.21.15.146:0/1437042198 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5d900038e0 con 0x7f5da00c57f0 2024-09-13T18:26:56.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.329+0000 7f5d95ffb700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d84066760 0x7f5d84068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:56.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.329+0000 7f5d95ffb700 1 -- 172.21.15.146:0/1437042198 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5d900837b0 con 0x7f5da00c57f0 2024-09-13T18:26:56.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.329+0000 7f5da4fd6700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d84066760 0x7f5d84068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:56.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.333+0000 7f5da4fd6700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d84066760 0x7f5d84068c20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5da01f3610 tx=0x7f5d9c009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:56.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.341+0000 7f5d95ffb700 1 -- 172.21.15.146:0/1437042198 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5d90050840 con 0x7f5da00c57f0 2024-09-13T18:26:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:56.545+0000 7f5da7a3b700 1 -- 172.21.15.146:0/1437042198 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5da00c9090 con 0x7f5da00c57f0 2024-09-13T18:26:57.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.449+0000 7f5d95ffb700 1 -- 172.21.15.146:0/1437042198 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5d90053e60 con 0x7f5da00c57f0 2024-09-13T18:26:57.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.457+0000 7f5da7a3b700 1 -- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d84066760 msgr2=0x7f5d84068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:57.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.457+0000 7f5da7a3b700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d84066760 0x7f5d84068c20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5da01f3610 tx=0x7f5d9c009240 comp rx=0 tx=0).stop 2024-09-13T18:26:57.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.457+0000 7f5da7a3b700 1 -- 172.21.15.146:0/1437042198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5da00c57f0 msgr2=0x7f5da00cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:57.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.457+0000 7f5da7a3b700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5da00c57f0 0x7f5da00cf6c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c500 tx=0x7f5d9000c810 comp rx=0 tx=0).stop 2024-09-13T18:26:57.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.457+0000 7f5da7a3b700 1 -- 172.21.15.146:0/1437042198 shutdown_connections 2024-09-13T18:26:57.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.457+0000 7f5da7a3b700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d84066760 0x7f5d84068c20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:57.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.457+0000 7f5da7a3b700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da00cad90 0x7f5da00ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:57.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.457+0000 7f5da7a3b700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da00ca020 0x7f5da00ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:57.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.457+0000 7f5da7a3b700 1 --2- 172.21.15.146:0/1437042198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5da00c57f0 0x7f5da00cf6c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:57.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.457+0000 7f5da7a3b700 1 -- 172.21.15.146:0/1437042198 >> 172.21.15.146:0/1437042198 conn(0x7f5da0071d30 msgr2=0x7f5da00c6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:57.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.457+0000 7f5da7a3b700 1 -- 172.21.15.146:0/1437042198 shutdown_connections 2024-09-13T18:26:57.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.457+0000 7f5da7a3b700 1 -- 172.21.15.146:0/1437042198 wait complete. 2024-09-13T18:26:57.478 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300} 2024-09-13T18:26:57.478 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-13T18:26:57.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29de58a700 1 Processor -- start 2024-09-13T18:26:57.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29de58a700 1 -- start start 2024-09-13T18:26:57.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29de58a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29d80cf790 0x7f29d8157980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:57.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29de58a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f29d80cfb70 0x7f29d8157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:57.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29de58a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d81541b0 con 0x7f29d80c8ba0 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29de58a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d815cb00 con 0x7f29d80cfb70 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29de58a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d815bba0 con 0x7f29d80cf790 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29d77fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f29d80cfb70 0x7f29d8157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29d77fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f29d80cfb70 0x7f29d8157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39424/0 (socket says 172.21.15.146:39424) 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29dcb27700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f29d80c8ba0 0x7f29d80c9030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:47386/0 (socket says 172.21.15.146:47386) 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29d77fe700 1 -- 172.21.15.146:0/303881142 learned_addr learned my addr 172.21.15.146:0/303881142 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29d7fff700 1 --2- 172.21.15.146:0/303881142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29d80cf790 0x7f29d8157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29d77fe700 1 -- 172.21.15.146:0/303881142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29d80cf790 msgr2=0x7f29d8157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29d77fe700 1 --2- 172.21.15.146:0/303881142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29d80cf790 0x7f29d8157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29d77fe700 1 -- 172.21.15.146:0/303881142 >> v1:172.21.15.146:6789/0 conn(0x7f29d80c8ba0 legacy=0x7f29d80c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29d77fe700 1 -- 172.21.15.146:0/303881142 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d815be80 con 0x7f29d80cfb70 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29d6ffd700 1 -- 172.21.15.146:0/303881142 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009271957 0 0) 0x7f29d81541b0 con 0x7f29d80c8ba0 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.601+0000 7f29d77fe700 1 --2- 172.21.15.146:0/303881142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29d80cfb70 0x7f29d8157ec0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009910 tx=0x7f29cc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faf848eb99242f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29d6ffd700 1 -- 172.21.15.146:0/303881142 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc011830 con 0x7f29d80cfb70 2024-09-13T18:26:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29d6ffd700 1 -- 172.21.15.146:0/303881142 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc011990 con 0x7f29d80cfb70 2024-09-13T18:26:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29d6ffd700 1 -- 172.21.15.146:0/303881142 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc011d40 con 0x7f29d80cfb70 2024-09-13T18:26:57.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 -- 172.21.15.146:0/303881142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29d80cfb70 msgr2=0x7f29d8157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:57.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 --2- 172.21.15.146:0/303881142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29d80cfb70 0x7f29d8157ec0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009910 tx=0x7f29cc009c20 comp rx=0 tx=0).stop 2024-09-13T18:26:57.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 -- 172.21.15.146:0/303881142 shutdown_connections 2024-09-13T18:26:57.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 --2- 172.21.15.146:0/303881142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29d80cfb70 0x7f29d8157ec0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:57.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 --2- 172.21.15.146:0/303881142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29d80cf790 0x7f29d8157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:57.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 -- 172.21.15.146:0/303881142 >> 172.21.15.146:0/303881142 conn(0x7f29d8071d90 msgr2=0x7f29d80721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:57.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 -- 172.21.15.146:0/303881142 shutdown_connections 2024-09-13T18:26:57.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 -- 172.21.15.146:0/303881142 wait complete. 2024-09-13T18:26:57.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 Processor -- start 2024-09-13T18:26:57.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 -- start start 2024-09-13T18:26:57.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29d80c8ba0 0x7f29d8158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:57.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29d80cf790 0x7f29d81f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:57.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29d80cfb70 0x7f29d80c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:57.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d81f3280 con 0x7f29d80cf790 2024-09-13T18:26:57.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d80cd760 con 0x7f29d80c8ba0 2024-09-13T18:26:57.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29de58a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d80725b0 con 0x7f29d80cfb70 2024-09-13T18:26:57.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29d77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29d80cf790 0x7f29d81f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:57.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29d77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29d80cf790 0x7f29d81f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39436/0 (socket says 172.21.15.146:39436) 2024-09-13T18:26:57.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29d77fe700 1 -- 172.21.15.146:0/831152326 learned_addr learned my addr 172.21.15.146:0/831152326 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:57.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29d7fff700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29d80c8ba0 0x7f29d8158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:57.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29dcb27700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29d80cfb70 0x7f29d80c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:57.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29d77fe700 1 -- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29d80cfb70 msgr2=0x7f29d80c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:57.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29d77fe700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29d80cfb70 0x7f29d80c5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:57.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29d77fe700 1 -- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29d80c8ba0 msgr2=0x7f29d8158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:57.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29d77fe700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29d80c8ba0 0x7f29d8158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:57.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.605+0000 7f29d77fe700 1 -- 172.21.15.146:0/831152326 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc009570 con 0x7f29d80cf790 2024-09-13T18:26:57.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.609+0000 7f29d77fe700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29d80cf790 0x7f29d81f2690 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009f70 tx=0x7f29cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:57.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.609+0000 7f29d4ff9700 1 -- 172.21.15.146:0/831152326 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc01a900 con 0x7f29d80cf790 2024-09-13T18:26:57.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.609+0000 7f29d4ff9700 1 -- 172.21.15.146:0/831152326 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc01f070 con 0x7f29d80cf790 2024-09-13T18:26:57.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.609+0000 7f29de58a700 1 -- 172.21.15.146:0/831152326 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d81f2fe0 con 0x7f29d80cf790 2024-09-13T18:26:57.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.609+0000 7f29d4ff9700 1 -- 172.21.15.146:0/831152326 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc01acc0 con 0x7f29d80cf790 2024-09-13T18:26:57.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.609+0000 7f29de58a700 1 -- 172.21.15.146:0/831152326 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d8158500 con 0x7f29d80cf790 2024-09-13T18:26:57.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.609+0000 7f29de58a700 1 -- 172.21.15.146:0/831152326 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d8158080 con 0x7f29d80cf790 2024-09-13T18:26:57.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.609+0000 7f29d4ff9700 1 -- 172.21.15.146:0/831152326 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f29cc01b430 con 0x7f29d80cf790 2024-09-13T18:26:57.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.609+0000 7f29d4ff9700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f29c4066760 0x7f29c4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:57.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.609+0000 7f29d4ff9700 1 -- 172.21.15.146:0/831152326 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f29cc021070 con 0x7f29d80cf790 2024-09-13T18:26:57.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.613+0000 7f29d7fff700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f29c4066760 0x7f29c4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:57.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.613+0000 7f29d7fff700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f29c4066760 0x7f29c4068c20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009bf0 tx=0x7f29c8009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:57.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.617+0000 7f29d4ff9700 1 -- 172.21.15.146:0/831152326 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f29d8158080 con 0x7f29d80cf790 2024-09-13T18:26:57.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:57.825+0000 7f29de58a700 1 -- 172.21.15.146:0/831152326 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f29d8158080 con 0x7f29d80cf790 2024-09-13T18:26:58.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.453+0000 7f29d4ff9700 1 -- 172.21.15.146:0/831152326 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v9) v1 ==== 105+0+0 (secure 0 0 0) 0x7f29d8158080 con 0x7f29d80cf790 2024-09-13T18:26:58.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.457+0000 7f29de58a700 1 -- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f29c4066760 msgr2=0x7f29c4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:58.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.457+0000 7f29de58a700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f29c4066760 0x7f29c4068c20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009bf0 tx=0x7f29c8009240 comp rx=0 tx=0).stop 2024-09-13T18:26:58.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.457+0000 7f29de58a700 1 -- 172.21.15.146:0/831152326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29d80cf790 msgr2=0x7f29d81f2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:58.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.457+0000 7f29de58a700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29d80cf790 0x7f29d81f2690 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009f70 tx=0x7f29cc004770 comp rx=0 tx=0).stop 2024-09-13T18:26:58.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.457+0000 7f29de58a700 1 -- 172.21.15.146:0/831152326 shutdown_connections 2024-09-13T18:26:58.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.457+0000 7f29de58a700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f29c4066760 0x7f29c4068c20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:58.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.457+0000 7f29de58a700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29d80cfb70 0x7f29d80c5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:58.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.457+0000 7f29de58a700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29d80cf790 0x7f29d81f2690 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:58.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.457+0000 7f29de58a700 1 --2- 172.21.15.146:0/831152326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29d80c8ba0 0x7f29d8158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:58.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.457+0000 7f29de58a700 1 -- 172.21.15.146:0/831152326 >> 172.21.15.146:0/831152326 conn(0x7f29d8071d90 msgr2=0x7f29d80c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:58.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.457+0000 7f29de58a700 1 -- 172.21.15.146:0/831152326 shutdown_connections 2024-09-13T18:26:58.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.457+0000 7f29de58a700 1 -- 172.21.15.146:0/831152326 wait complete. 2024-09-13T18:26:58.483 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T18:26:58.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.597+0000 7f5c7bfff700 1 Processor -- start 2024-09-13T18:26:58.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.597+0000 7f5c7bfff700 1 -- start start 2024-09-13T18:26:58.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7c0caed0 0x7f5c7c1610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:58.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c7c0aa4d0 0x7f5c7c15e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:58.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7bfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7c161fe0 con 0x7f5c7c0cb2b0 2024-09-13T18:26:58.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7bfff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7c0a9930 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c161780 con 0x7f5c7c0caed0 2024-09-13T18:26:58.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7a7fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5c7c0cb2b0 0x7f5c7c0a9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:47402/0 (socket says 172.21.15.146:47402) 2024-09-13T18:26:58.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7a7fc700 1 -- 172.21.15.146:0/2219455263 learned_addr learned my addr 172.21.15.146:0/2219455263 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:26:58.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7b7fe700 1 --2- 172.21.15.146:0/2219455263 >> v2:172.21.15.154:3300/0 conn(0x7f5c7c0aa4d0 0x7f5c7c15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:58.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7affd700 1 --2- 172.21.15.146:0/2219455263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7c0caed0 0x7f5c7c1610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:58.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c79ffb700 1 -- 172.21.15.146:0/2219455263 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997705963 0 0) 0x7f5c7c161fe0 con 0x7f5c7c0cb2b0 2024-09-13T18:26:58.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c79ffb700 1 -- 172.21.15.146:0/2219455263 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c640035d0 con 0x7f5c7c0cb2b0 2024-09-13T18:26:58.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7b7fe700 1 -- 172.21.15.146:0/2219455263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7c0caed0 msgr2=0x7f5c7c1610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:58.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7b7fe700 1 --2- 172.21.15.146:0/2219455263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7c0caed0 0x7f5c7c1610f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:58.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7b7fe700 1 -- 172.21.15.146:0/2219455263 >> v1:172.21.15.146:6789/0 conn(0x7f5c7c0cb2b0 legacy=0x7f5c7c0a9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:58.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c79ffb700 1 -- 172.21.15.146:0/2219455263 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 512297669 0 0) 0x7f5c640035d0 con 0x7f5c7c0cb2b0 2024-09-13T18:26:58.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7b7fe700 1 -- 172.21.15.146:0/2219455263 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c161a20 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7affd700 1 --2- 172.21.15.146:0/2219455263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7c0caed0 0x7f5c7c1610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:58.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7b7fe700 1 --2- 172.21.15.146:0/2219455263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7c0aa4d0 0x7f5c7c15e190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009850 tx=0x7f5c74009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=147302eac3bd6f8f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:58.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c79ffb700 1 -- 172.21.15.146:0/2219455263 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0019f0 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c79ffb700 1 -- 172.21.15.146:0/2219455263 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c74004030 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c79ffb700 1 -- 172.21.15.146:0/2219455263 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c740043a0 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7bfff700 1 -- 172.21.15.146:0/2219455263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7c0aa4d0 msgr2=0x7f5c7c15e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:58.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7bfff700 1 --2- 172.21.15.146:0/2219455263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7c0aa4d0 0x7f5c7c15e190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009850 tx=0x7f5c74009b60 comp rx=0 tx=0).stop 2024-09-13T18:26:58.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7bfff700 1 -- 172.21.15.146:0/2219455263 shutdown_connections 2024-09-13T18:26:58.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7bfff700 1 --2- 172.21.15.146:0/2219455263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7c0aa4d0 0x7f5c7c15e190 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:58.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7bfff700 1 --2- 172.21.15.146:0/2219455263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7c0caed0 0x7f5c7c1610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:58.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7bfff700 1 -- 172.21.15.146:0/2219455263 >> 172.21.15.146:0/2219455263 conn(0x7f5c7c066090 msgr2=0x7f5c7c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:58.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7bfff700 1 -- 172.21.15.146:0/2219455263 shutdown_connections 2024-09-13T18:26:58.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.601+0000 7f5c7bfff700 1 -- 172.21.15.146:0/2219455263 wait complete. 2024-09-13T18:26:58.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7bfff700 1 Processor -- start 2024-09-13T18:26:58.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7bfff700 1 -- start start 2024-09-13T18:26:58.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7c0aa4d0 0x7f5c7c1f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:58.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7c0caed0 0x7f5c7c15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:58.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7c0cb2b0 0x7f5c7c1f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:58.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7bfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1f4400 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7bfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1f6a40 con 0x7f5c7c0cb2b0 2024-09-13T18:26:58.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c0c3940 con 0x7f5c7c0caed0 2024-09-13T18:26:58.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7c0aa4d0 0x7f5c7c1f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:58.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7c0aa4d0 0x7f5c7c1f3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39444/0 (socket says 172.21.15.146:39444) 2024-09-13T18:26:58.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7affd700 1 -- 172.21.15.146:0/3881233010 learned_addr learned my addr 172.21.15.146:0/3881233010 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:58.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7a7fc700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7c0caed0 0x7f5c7c15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:58.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7b7fe700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7c0cb2b0 0x7f5c7c1f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7affd700 1 -- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7c0caed0 msgr2=0x7f5c7c15e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7affd700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7c0caed0 0x7f5c7c15e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7affd700 1 -- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7c0cb2b0 msgr2=0x7f5c7c1f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7affd700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7c0cb2b0 0x7f5c7c1f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7affd700 1 -- 172.21.15.146:0/3881233010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c700094b0 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7affd700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7c0aa4d0 0x7f5c7c1f3ec0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000ebe0 tx=0x7f5c7000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c637fe700 1 -- 172.21.15.146:0/3881233010 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000c850 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c637fe700 1 -- 172.21.15.146:0/3881233010 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7000c9b0 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7bfff700 1 -- 172.21.15.146:0/3881233010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c740094b0 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c637fe700 1 -- 172.21.15.146:0/3881233010 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000cd40 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.605+0000 7f5c7bfff700 1 -- 172.21.15.146:0/3881233010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c0c42b0 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.609+0000 7f5c637fe700 1 -- 172.21.15.146:0/3881233010 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5c70013470 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.609+0000 7f5c7bfff700 1 -- 172.21.15.146:0/3881233010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c072460 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.609+0000 7f5c637fe700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c68066760 0x7f5c68068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:58.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.609+0000 7f5c637fe700 1 -- 172.21.15.146:0/3881233010 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5c70084520 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.609+0000 7f5c7a7fc700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c68066760 0x7f5c68068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:58.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.609+0000 7f5c7a7fc700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c68066760 0x7f5c68068c20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c06e7a0 tx=0x7f5c6c0021c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:58.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.621+0000 7f5c637fe700 1 -- 172.21.15.146:0/3881233010 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5c70051510 con 0x7f5c7c0aa4d0 2024-09-13T18:26:58.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:58.825+0000 7f5c7bfff700 1 -- 172.21.15.146:0/3881233010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5c7c0c7310 con 0x7f5c7c0aa4d0 2024-09-13T18:26:59.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.469+0000 7f5c637fe700 1 -- 172.21.15.146:0/3881233010 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v11) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5c7000cea0 con 0x7f5c7c0aa4d0 2024-09-13T18:26:59.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.473+0000 7f5c7bfff700 1 -- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c68066760 msgr2=0x7f5c68068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:59.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.473+0000 7f5c7bfff700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c68066760 0x7f5c68068c20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c06e7a0 tx=0x7f5c6c0021c0 comp rx=0 tx=0).stop 2024-09-13T18:26:59.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.473+0000 7f5c7bfff700 1 -- 172.21.15.146:0/3881233010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7c0aa4d0 msgr2=0x7f5c7c1f3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:59.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.473+0000 7f5c7bfff700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7c0aa4d0 0x7f5c7c1f3ec0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000ebe0 tx=0x7f5c7000eef0 comp rx=0 tx=0).stop 2024-09-13T18:26:59.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.473+0000 7f5c7bfff700 1 -- 172.21.15.146:0/3881233010 shutdown_connections 2024-09-13T18:26:59.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.473+0000 7f5c7bfff700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c68066760 0x7f5c68068c20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:59.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.473+0000 7f5c7bfff700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7c0cb2b0 0x7f5c7c1f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:59.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.473+0000 7f5c7bfff700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7c0caed0 0x7f5c7c15e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:59.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.473+0000 7f5c7bfff700 1 --2- 172.21.15.146:0/3881233010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7c0aa4d0 0x7f5c7c1f3ec0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:59.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.473+0000 7f5c7bfff700 1 -- 172.21.15.146:0/3881233010 >> 172.21.15.146:0/3881233010 conn(0x7f5c7c066090 msgr2=0x7f5c7c06c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:59.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.473+0000 7f5c7bfff700 1 -- 172.21.15.146:0/3881233010 shutdown_connections 2024-09-13T18:26:59.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.473+0000 7f5c7bfff700 1 -- 172.21.15.146:0/3881233010 wait complete. 2024-09-13T18:26:59.497 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-09-13T18:26:59.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.621+0000 7f4c36bda700 1 Processor -- start 2024-09-13T18:26:59.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.621+0000 7f4c36bda700 1 -- start start 2024-09-13T18:26:59.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c36bda700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c3015d960 0x7f4c300c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:59.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c36bda700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4c300ca500 0x7f4c300cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:59.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c36bda700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c300c5e50 con 0x7f4c301539e0 2024-09-13T18:26:59.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c36bda700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c300cc3a0 con 0x7f4c300ca500 2024-09-13T18:26:59.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c36bda700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c300cb480 con 0x7f4c3015d960 2024-09-13T18:26:59.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c363d9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4c300ca500 0x7f4c300cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:59.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c363d9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4c300ca500 0x7f4c300cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39458/0 (socket says 172.21.15.146:39458) 2024-09-13T18:26:59.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c363d9700 1 -- 172.21.15.146:0/776013243 learned_addr learned my addr 172.21.15.146:0/776013243 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:59.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c353d7700 1 --2- 172.21.15.146:0/776013243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c3015d960 0x7f4c300c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:59.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c363d9700 1 -- 172.21.15.146:0/776013243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c3015d960 msgr2=0x7f4c300c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:59.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c363d9700 1 --2- 172.21.15.146:0/776013243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c3015d960 0x7f4c300c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:59.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c363d9700 1 -- 172.21.15.146:0/776013243 >> v1:172.21.15.154:6789/0 conn(0x7f4c301539e0 legacy=0x7f4c300b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:59.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c363d9700 1 -- 172.21.15.146:0/776013243 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c300cb720 con 0x7f4c300ca500 2024-09-13T18:26:59.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c363d9700 1 --2- 172.21.15.146:0/776013243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c300ca500 0x7f4c300cf580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009e50 tx=0x7f4c2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179b02c6253ababf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:59.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c34bd6700 1 -- 172.21.15.146:0/776013243 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2800e070 con 0x7f4c300ca500 2024-09-13T18:26:59.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c34bd6700 1 -- 172.21.15.146:0/776013243 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2800b9c0 con 0x7f4c300ca500 2024-09-13T18:26:59.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c34bd6700 1 -- 172.21.15.146:0/776013243 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2800bd70 con 0x7f4c300ca500 2024-09-13T18:26:59.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c36bda700 1 -- 172.21.15.146:0/776013243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c300ca500 msgr2=0x7f4c300cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:59.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c36bda700 1 --2- 172.21.15.146:0/776013243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c300ca500 0x7f4c300cf580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009e50 tx=0x7f4c2800b330 comp rx=0 tx=0).stop 2024-09-13T18:26:59.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c36bda700 1 -- 172.21.15.146:0/776013243 shutdown_connections 2024-09-13T18:26:59.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c36bda700 1 --2- 172.21.15.146:0/776013243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c300ca500 0x7f4c300cf580 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:59.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c36bda700 1 --2- 172.21.15.146:0/776013243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c3015d960 0x7f4c300c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:59.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c36bda700 1 -- 172.21.15.146:0/776013243 >> 172.21.15.146:0/776013243 conn(0x7f4c30071ec0 msgr2=0x7f4c300722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:59.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c36bda700 1 -- 172.21.15.146:0/776013243 shutdown_connections 2024-09-13T18:26:59.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.625+0000 7f4c36bda700 1 -- 172.21.15.146:0/776013243 wait complete. 2024-09-13T18:26:59.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c36bda700 1 Processor -- start 2024-09-13T18:26:59.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c36bda700 1 -- start start 2024-09-13T18:26:59.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c36bda700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c300ca500 0x7f4c3014beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:59.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c36bda700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c301539e0 0x7f4c30148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:59.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c36bda700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c3015d960 0x7f4c30149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:59.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c36bda700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3014a5c0 con 0x7f4c3015d960 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c36bda700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3014b7e0 con 0x7f4c300ca500 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c36bda700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3014a7d0 con 0x7f4c301539e0 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c363d9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c3015d960 0x7f4c30149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c363d9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c3015d960 0x7f4c30149250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39460/0 (socket says 172.21.15.146:39460) 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c363d9700 1 -- 172.21.15.146:0/3707847427 learned_addr learned my addr 172.21.15.146:0/3707847427 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c35bd8700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c300ca500 0x7f4c3014beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c353d7700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c301539e0 0x7f4c30148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c363d9700 1 -- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c301539e0 msgr2=0x7f4c30148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c363d9700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c301539e0 0x7f4c30148d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c363d9700 1 -- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c300ca500 msgr2=0x7f4c3014beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c363d9700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c300ca500 0x7f4c3014beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c363d9700 1 -- 172.21.15.146:0/3707847427 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c28009b30 con 0x7f4c3015d960 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c353d7700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c301539e0 0x7f4c30148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:59.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c363d9700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c3015d960 0x7f4c30149250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4c30067de0 tx=0x7f4c28004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:59.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c267fc700 1 -- 172.21.15.146:0/3707847427 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c280048b0 con 0x7f4c3015d960 2024-09-13T18:26:59.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.629+0000 7f4c267fc700 1 -- 172.21.15.146:0/3707847427 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c280092e0 con 0x7f4c3015d960 2024-09-13T18:26:59.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.633+0000 7f4c36bda700 1 -- 172.21.15.146:0/3707847427 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3014aab0 con 0x7f4c3015d960 2024-09-13T18:26:59.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.633+0000 7f4c267fc700 1 -- 172.21.15.146:0/3707847427 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2801c660 con 0x7f4c3015d960 2024-09-13T18:26:59.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.633+0000 7f4c36bda700 1 -- 172.21.15.146:0/3707847427 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3014c3f0 con 0x7f4c3015d960 2024-09-13T18:26:59.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.633+0000 7f4c267fc700 1 -- 172.21.15.146:0/3707847427 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4c2801c7c0 con 0x7f4c3015d960 2024-09-13T18:26:59.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.633+0000 7f4c36bda700 1 -- 172.21.15.146:0/3707847427 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c301effc0 con 0x7f4c3015d960 2024-09-13T18:26:59.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.633+0000 7f4c267fc700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c14066730 0x7f4c14068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:59.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.633+0000 7f4c267fc700 1 -- 172.21.15.146:0/3707847427 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4c28085fe0 con 0x7f4c3015d960 2024-09-13T18:26:59.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.633+0000 7f4c35bd8700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c14066730 0x7f4c14068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:59.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.637+0000 7f4c35bd8700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c14066730 0x7f4c14068bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4c20000f60 tx=0x7f4c20009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:59.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.641+0000 7f4c267fc700 1 -- 172.21.15.146:0/3707847427 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4c301effc0 con 0x7f4c3015d960 2024-09-13T18:26:59.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:26:59.849+0000 7f4c36bda700 1 -- 172.21.15.146:0/3707847427 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f4c301effc0 con 0x7f4c3015d960 2024-09-13T18:27:00.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.473+0000 7f4c267fc700 1 -- 172.21.15.146:0/3707847427 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v12) v1 ==== 115+0+0 (secure 0 0 0) 0x7f4c301effc0 con 0x7f4c3015d960 2024-09-13T18:27:00.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.477+0000 7f4c36bda700 1 -- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c14066730 msgr2=0x7f4c14068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:00.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.477+0000 7f4c36bda700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c14066730 0x7f4c14068bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4c20000f60 tx=0x7f4c20009040 comp rx=0 tx=0).stop 2024-09-13T18:27:00.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.477+0000 7f4c36bda700 1 -- 172.21.15.146:0/3707847427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c3015d960 msgr2=0x7f4c30149250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:00.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.477+0000 7f4c36bda700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c3015d960 0x7f4c30149250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4c30067de0 tx=0x7f4c28004af0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.477+0000 7f4c36bda700 1 -- 172.21.15.146:0/3707847427 shutdown_connections 2024-09-13T18:27:00.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.477+0000 7f4c36bda700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c14066730 0x7f4c14068bf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.477+0000 7f4c36bda700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c3015d960 0x7f4c30149250 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.477+0000 7f4c36bda700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c301539e0 0x7f4c30148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.477+0000 7f4c36bda700 1 --2- 172.21.15.146:0/3707847427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c300ca500 0x7f4c3014beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.477+0000 7f4c36bda700 1 -- 172.21.15.146:0/3707847427 >> 172.21.15.146:0/3707847427 conn(0x7f4c30071ec0 msgr2=0x7f4c300cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:00.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.477+0000 7f4c36bda700 1 -- 172.21.15.146:0/3707847427 shutdown_connections 2024-09-13T18:27:00.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.477+0000 7f4c36bda700 1 -- 172.21.15.146:0/3707847427 wait complete. 2024-09-13T18:27:00.497 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T18:27:00.498 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:27:00.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf359e700 1 Processor -- start 2024-09-13T18:27:00.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf359e700 1 -- start start 2024-09-13T18:27:00.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facf414ff70 0x7facf406ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:00.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7facf40c2810 0x7facf40c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:00.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7facf415caf0 0x7facf415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:00.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf406b2e0 con 0x7facf415caf0 2024-09-13T18:27:00.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf4071700 con 0x7facf40c2810 2024-09-13T18:27:00.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf4070850 con 0x7facf414ff70 2024-09-13T18:27:00.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf2d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7facf415caf0 0x7facf415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:00.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf2d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7facf415caf0 0x7facf415cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39476/0 (socket says 172.21.15.146:39476) 2024-09-13T18:27:00.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf2d9d700 1 -- 172.21.15.146:0/2262399609 learned_addr learned my addr 172.21.15.146:0/2262399609 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:00.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf1d9b700 1 --2- 172.21.15.146:0/2262399609 >> v2:172.21.15.154:3300/0 conn(0x7facf40c2810 0x7facf40c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:00.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf259c700 1 --2- 172.21.15.146:0/2262399609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facf414ff70 0x7facf406ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:00.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf2d9d700 1 -- 172.21.15.146:0/2262399609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facf414ff70 msgr2=0x7facf406ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:00.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf2d9d700 1 --2- 172.21.15.146:0/2262399609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facf414ff70 0x7facf406ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf2d9d700 1 -- 172.21.15.146:0/2262399609 >> v2:172.21.15.154:3300/0 conn(0x7facf40c2810 msgr2=0x7facf40c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:00.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf2d9d700 1 --2- 172.21.15.146:0/2262399609 >> v2:172.21.15.154:3300/0 conn(0x7facf40c2810 0x7facf40c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf2d9d700 1 -- 172.21.15.146:0/2262399609 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf4070af0 con 0x7facf415caf0 2024-09-13T18:27:00.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.605+0000 7facf1d9b700 1 --2- 172.21.15.146:0/2262399609 >> v2:172.21.15.154:3300/0 conn(0x7facf40c2810 0x7facf40c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:00.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.609+0000 7facf2d9d700 1 --2- 172.21.15.146:0/2262399609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facf415caf0 0x7facf415cf80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7facf4067070 tx=0x7facec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709acb5fc2d2a580 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:00.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.609+0000 7facf159a700 1 -- 172.21.15.146:0/2262399609 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec00e070 con 0x7facf415caf0 2024-09-13T18:27:00.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.609+0000 7facf159a700 1 -- 172.21.15.146:0/2262399609 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec00b9c0 con 0x7facf415caf0 2024-09-13T18:27:00.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.609+0000 7facf159a700 1 -- 172.21.15.146:0/2262399609 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec00bd70 con 0x7facf415caf0 2024-09-13T18:27:00.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.609+0000 7facf359e700 1 -- 172.21.15.146:0/2262399609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facf415caf0 msgr2=0x7facf415cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:00.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.609+0000 7facf359e700 1 --2- 172.21.15.146:0/2262399609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facf415caf0 0x7facf415cf80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7facf4067070 tx=0x7facec00b330 comp rx=0 tx=0).stop 2024-09-13T18:27:00.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.609+0000 7facf359e700 1 -- 172.21.15.146:0/2262399609 shutdown_connections 2024-09-13T18:27:00.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.609+0000 7facf359e700 1 --2- 172.21.15.146:0/2262399609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facf415caf0 0x7facf415cf80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.609+0000 7facf359e700 1 --2- 172.21.15.146:0/2262399609 >> v2:172.21.15.154:3300/0 conn(0x7facf40c2810 0x7facf40c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.609+0000 7facf359e700 1 --2- 172.21.15.146:0/2262399609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facf414ff70 0x7facf406ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.609+0000 7facf359e700 1 -- 172.21.15.146:0/2262399609 >> 172.21.15.146:0/2262399609 conn(0x7facf406f5c0 msgr2=0x7facf406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:00.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.609+0000 7facf359e700 1 -- 172.21.15.146:0/2262399609 shutdown_connections 2024-09-13T18:27:00.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.609+0000 7facf359e700 1 -- 172.21.15.146:0/2262399609 wait complete. 2024-09-13T18:27:00.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.613+0000 7facf359e700 1 Processor -- start 2024-09-13T18:27:00.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.613+0000 7facf359e700 1 -- start start 2024-09-13T18:27:00.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.613+0000 7facf359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facf40c2810 0x7facf40c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:00.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.613+0000 7facf359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facf414ff70 0x7facf40c5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:00.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.613+0000 7facf359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf415caf0 0x7facf41437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:00.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.613+0000 7facf359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf40c7520 con 0x7facf40c2810 2024-09-13T18:27:00.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.613+0000 7facf359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf40cc660 con 0x7facf415caf0 2024-09-13T18:27:00.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.613+0000 7facf359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf40c6870 con 0x7facf414ff70 2024-09-13T18:27:00.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.613+0000 7facf259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facf40c2810 0x7facf40c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:00.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.613+0000 7facf259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facf40c2810 0x7facf40c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39480/0 (socket says 172.21.15.146:39480) 2024-09-13T18:27:00.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.613+0000 7facf259c700 1 -- 172.21.15.146:0/2631840486 learned_addr learned my addr 172.21.15.146:0/2631840486 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:00.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.613+0000 7facf1d9b700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facf414ff70 0x7facf40c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:00.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.613+0000 7facf2d9d700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf415caf0 0x7facf41437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:00.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7facf259c700 1 -- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facf414ff70 msgr2=0x7facf40c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:00.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7facf259c700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facf414ff70 0x7facf40c5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7facf259c700 1 -- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf415caf0 msgr2=0x7facf41437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:00.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7facf259c700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf415caf0 0x7facf41437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7facf259c700 1 -- 172.21.15.146:0/2631840486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec009b30 con 0x7facf40c2810 2024-09-13T18:27:00.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7facf259c700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facf40c2810 0x7facf40c6fe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7face800e790 tx=0x7face800eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:00.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7face2ffd700 1 -- 172.21.15.146:0/2631840486 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face800c070 con 0x7facf40c2810 2024-09-13T18:27:00.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7face2ffd700 1 -- 172.21.15.146:0/2631840486 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face8013610 con 0x7facf40c2810 2024-09-13T18:27:00.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7facf359e700 1 -- 172.21.15.146:0/2631840486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf40cc220 con 0x7facf40c2810 2024-09-13T18:27:00.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7face2ffd700 1 -- 172.21.15.146:0/2631840486 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8013980 con 0x7facf40c2810 2024-09-13T18:27:00.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7facf359e700 1 -- 172.21.15.146:0/2631840486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf40c49f0 con 0x7facf40c2810 2024-09-13T18:27:00.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7face2ffd700 1 -- 172.21.15.146:0/2631840486 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7face8013ae0 con 0x7facf40c2810 2024-09-13T18:27:00.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7facf359e700 1 -- 172.21.15.146:0/2631840486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf4150350 con 0x7facf40c2810 2024-09-13T18:27:00.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7face2ffd700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7facd4066730 0x7facd4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:00.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7face2ffd700 1 -- 172.21.15.146:0/2631840486 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7face8084200 con 0x7facf40c2810 2024-09-13T18:27:00.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.617+0000 7facf1d9b700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7facd4066730 0x7facd4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:00.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.621+0000 7facf1d9b700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7facd4066730 0x7facd4068bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7facf4149c00 tx=0x7face4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:00.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.621+0000 7face2ffd700 1 -- 172.21.15.146:0/2631840486 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7face80511f0 con 0x7facf40c2810 2024-09-13T18:27:00.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.829+0000 7facf359e700 1 -- 172.21.15.146:0/2631840486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facf40caa90 con 0x7facf40c2810 2024-09-13T18:27:00.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.829+0000 7face2ffd700 1 -- 172.21.15.146:0/2631840486 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6271 (secure 0 0 0) 0x7face8054810 con 0x7facf40c2810 2024-09-13T18:27:00.837 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:00.837 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:27:00.466838+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":4290,"mds_1":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:creating","state_seq":1,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:27:00.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.833+0000 7facf359e700 1 -- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7facd4066730 msgr2=0x7facd4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:00.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.833+0000 7facf359e700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7facd4066730 0x7facd4068bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7facf4149c00 tx=0x7face4008040 comp rx=0 tx=0).stop 2024-09-13T18:27:00.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.833+0000 7facf359e700 1 -- 172.21.15.146:0/2631840486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facf40c2810 msgr2=0x7facf40c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:00.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.833+0000 7facf359e700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facf40c2810 0x7facf40c6fe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7face800e790 tx=0x7face800eaa0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.833+0000 7facf359e700 1 -- 172.21.15.146:0/2631840486 shutdown_connections 2024-09-13T18:27:00.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.833+0000 7facf359e700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7facd4066730 0x7facd4068bf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.833+0000 7facf359e700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf415caf0 0x7facf41437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.833+0000 7facf359e700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facf414ff70 0x7facf40c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.833+0000 7facf359e700 1 --2- 172.21.15.146:0/2631840486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facf40c2810 0x7facf40c6fe0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.833+0000 7facf359e700 1 -- 172.21.15.146:0/2631840486 >> 172.21.15.146:0/2631840486 conn(0x7facf406f5c0 msgr2=0x7facf414d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:00.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.833+0000 7facf359e700 1 -- 172.21.15.146:0/2631840486 shutdown_connections 2024-09-13T18:27:00.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.833+0000 7facf359e700 1 -- 172.21.15.146:0/2631840486 wait complete. 2024-09-13T18:27:00.841 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 12 2024-09-13T18:27:00.852 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:27:00.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 Processor -- start 2024-09-13T18:27:00.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 -- start start 2024-09-13T18:27:00.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3d58157f30 0x7f3d5815db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:00.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3d580ca130 0x7f3d580c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:00.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d58158310 con 0x7f3d58161d60 2024-09-13T18:27:00.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5815eb10 con 0x7f3d58157f30 2024-09-13T18:27:00.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5815e070 con 0x7f3d580ca130 2024-09-13T18:27:00.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d56ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3d58157f30 0x7f3d5815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:00.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d56ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3d58157f30 0x7f3d5815db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39486/0 (socket says 172.21.15.146:39486) 2024-09-13T18:27:00.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d56ffd700 1 -- 172.21.15.146:0/3791049457 learned_addr learned my addr 172.21.15.146:0/3791049457 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:00.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d577fe700 1 --2- 172.21.15.146:0/3791049457 >> v2:172.21.15.154:3300/0 conn(0x7f3d580ca130 0x7f3d580c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:00.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d56ffd700 1 -- 172.21.15.146:0/3791049457 >> v2:172.21.15.154:3300/0 conn(0x7f3d580ca130 msgr2=0x7f3d580c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:00.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d56ffd700 1 --2- 172.21.15.146:0/3791049457 >> v2:172.21.15.154:3300/0 conn(0x7f3d580ca130 0x7f3d580c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d56ffd700 1 -- 172.21.15.146:0/3791049457 >> v1:172.21.15.154:6789/0 conn(0x7f3d58161d60 legacy=0x7f3d580cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:00.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d56ffd700 1 -- 172.21.15.146:0/3791049457 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d581618d0 con 0x7f3d58157f30 2024-09-13T18:27:00.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d55ffb700 1 -- 172.21.15.146:0/3791049457 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393216682 0 0) 0x7f3d58158310 con 0x7f3d58161d60 2024-09-13T18:27:00.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d56ffd700 1 --2- 172.21.15.146:0/3791049457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d58157f30 0x7f3d5815db30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3d58071800 tx=0x7f3d40009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82e51f071548b22 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:00.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d55ffb700 1 -- 172.21.15.146:0/3791049457 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d40011830 con 0x7f3d58157f30 2024-09-13T18:27:00.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d55ffb700 1 -- 172.21.15.146:0/3791049457 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d40011990 con 0x7f3d58157f30 2024-09-13T18:27:00.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d55ffb700 1 -- 172.21.15.146:0/3791049457 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d40011d20 con 0x7f3d58157f30 2024-09-13T18:27:00.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 -- 172.21.15.146:0/3791049457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d58157f30 msgr2=0x7f3d5815db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:00.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 --2- 172.21.15.146:0/3791049457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d58157f30 0x7f3d5815db30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3d58071800 tx=0x7f3d40009bf0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 -- 172.21.15.146:0/3791049457 shutdown_connections 2024-09-13T18:27:00.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 --2- 172.21.15.146:0/3791049457 >> v2:172.21.15.154:3300/0 conn(0x7f3d580ca130 0x7f3d580c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 --2- 172.21.15.146:0/3791049457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d58157f30 0x7f3d5815db30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 -- 172.21.15.146:0/3791049457 >> 172.21.15.146:0/3791049457 conn(0x7f3d58071d30 msgr2=0x7f3d58072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:00.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 -- 172.21.15.146:0/3791049457 shutdown_connections 2024-09-13T18:27:00.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.957+0000 7f3d5d544700 1 -- 172.21.15.146:0/3791049457 wait complete. 2024-09-13T18:27:00.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d5d544700 1 Processor -- start 2024-09-13T18:27:00.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d5d544700 1 -- start start 2024-09-13T18:27:00.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d5d544700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d580ca130 0x7f3d581608d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:00.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d5d544700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d58157f30 0x7f3d581e8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:00.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d5d544700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d58161d60 0x7f3d581f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:00.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d5d544700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d581e76f0 con 0x7f3d580ca130 2024-09-13T18:27:00.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d5d544700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d581603a0 con 0x7f3d58157f30 2024-09-13T18:27:00.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d5d544700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d581f3530 con 0x7f3d58161d60 2024-09-13T18:27:00.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d56ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d580ca130 0x7f3d581608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:00.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d56ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d580ca130 0x7f3d581608d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39494/0 (socket says 172.21.15.146:39494) 2024-09-13T18:27:00.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d56ffd700 1 -- 172.21.15.146:0/2414960276 learned_addr learned my addr 172.21.15.146:0/2414960276 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:00.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d567fc700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d58157f30 0x7f3d581e8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:00.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d577fe700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d58161d60 0x7f3d581f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:00.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d56ffd700 1 -- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d58161d60 msgr2=0x7f3d581f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:00.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d56ffd700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d58161d60 0x7f3d581f3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d56ffd700 1 -- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d58157f30 msgr2=0x7f3d581e8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:00.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d56ffd700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d58157f30 0x7f3d581e8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:00.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d56ffd700 1 -- 172.21.15.146:0/2414960276 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d40009570 con 0x7f3d580ca130 2024-09-13T18:27:00.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.961+0000 7f3d56ffd700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d580ca130 0x7f3d581608d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3d58072af0 tx=0x7f3d40004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:00.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.965+0000 7f3d4f7fe700 1 -- 172.21.15.146:0/2414960276 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d400044d0 con 0x7f3d580ca130 2024-09-13T18:27:00.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.965+0000 7f3d4f7fe700 1 -- 172.21.15.146:0/2414960276 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4001f070 con 0x7f3d580ca130 2024-09-13T18:27:00.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.965+0000 7f3d5d544700 1 -- 172.21.15.146:0/2414960276 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d581e9530 con 0x7f3d580ca130 2024-09-13T18:27:00.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.965+0000 7f3d4f7fe700 1 -- 172.21.15.146:0/2414960276 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d40003900 con 0x7f3d580ca130 2024-09-13T18:27:00.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.965+0000 7f3d5d544700 1 -- 172.21.15.146:0/2414960276 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d58160eb0 con 0x7f3d580ca130 2024-09-13T18:27:00.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.965+0000 7f3d5d544700 1 -- 172.21.15.146:0/2414960276 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d580cd3d0 con 0x7f3d580ca130 2024-09-13T18:27:00.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.965+0000 7f3d4f7fe700 1 -- 172.21.15.146:0/2414960276 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3d40003a60 con 0x7f3d580ca130 2024-09-13T18:27:00.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.965+0000 7f3d4f7fe700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d38066730 0x7f3d38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:00.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.969+0000 7f3d4f7fe700 1 -- 172.21.15.146:0/2414960276 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3d40021070 con 0x7f3d580ca130 2024-09-13T18:27:00.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.969+0000 7f3d567fc700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d38066730 0x7f3d38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:00.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.969+0000 7f3d567fc700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d38066730 0x7f3d38068bf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3d500012e0 tx=0x7f3d50009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:00.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:00.973+0000 7f3d4f7fe700 1 -- 172.21.15.146:0/2414960276 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3d40052970 con 0x7f3d580ca130 2024-09-13T18:27:01.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.149+0000 7f3d5d544700 1 -- 172.21.15.146:0/2414960276 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d5815e830 con 0x7f3d580ca130 2024-09-13T18:27:01.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.149+0000 7f3d4f7fe700 1 -- 172.21.15.146:0/2414960276 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f3d40055f90 con 0x7f3d580ca130 2024-09-13T18:27:01.156 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:01.156 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:27:01.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.153+0000 7f3d5d544700 1 -- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d38066730 msgr2=0x7f3d38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.153+0000 7f3d5d544700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d38066730 0x7f3d38068bf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3d500012e0 tx=0x7f3d50009040 comp rx=0 tx=0).stop 2024-09-13T18:27:01.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.153+0000 7f3d5d544700 1 -- 172.21.15.146:0/2414960276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d580ca130 msgr2=0x7f3d581608d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.153+0000 7f3d5d544700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d580ca130 0x7f3d581608d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3d58072af0 tx=0x7f3d40004770 comp rx=0 tx=0).stop 2024-09-13T18:27:01.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.153+0000 7f3d5d544700 1 -- 172.21.15.146:0/2414960276 shutdown_connections 2024-09-13T18:27:01.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.153+0000 7f3d5d544700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d38066730 0x7f3d38068bf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.153+0000 7f3d5d544700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d58161d60 0x7f3d581f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.153+0000 7f3d5d544700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d58157f30 0x7f3d581e8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.153+0000 7f3d5d544700 1 --2- 172.21.15.146:0/2414960276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d580ca130 0x7f3d581608d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.153+0000 7f3d5d544700 1 -- 172.21.15.146:0/2414960276 >> 172.21.15.146:0/2414960276 conn(0x7f3d58071d30 msgr2=0x7f3d580c8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:01.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.153+0000 7f3d5d544700 1 -- 172.21.15.146:0/2414960276 shutdown_connections 2024-09-13T18:27:01.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.153+0000 7f3d5d544700 1 -- 172.21.15.146:0/2414960276 wait complete. 2024-09-13T18:27:01.174 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T18:27:01.175 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:27:01.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.281+0000 7f9616644700 1 Processor -- start 2024-09-13T18:27:01.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.281+0000 7f9616644700 1 -- start start 2024-09-13T18:27:01.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9616644700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f96100ca210 0x7f96100b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9616644700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9610071390 0x7f96100cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9616644700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96100cc670 con 0x7f96100cbd50 2024-09-13T18:27:01.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9616644700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96100caa30 con 0x7f96100ca210 2024-09-13T18:27:01.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9616644700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96100cb810 con 0x7f9610071390 2024-09-13T18:27:01.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9615642700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f96100cbd50 0x7f96100701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35408/0 (socket says 172.21.15.146:35408) 2024-09-13T18:27:01.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9615642700 1 -- 172.21.15.146:0/3943110235 learned_addr learned my addr 172.21.15.146:0/3943110235 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:01.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9615e43700 1 --2- 172.21.15.146:0/3943110235 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9610071390 0x7f96100cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9614e41700 1 --2- 172.21.15.146:0/3943110235 >> v2:172.21.15.154:3300/0 conn(0x7f96100ca210 0x7f96100b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9607fff700 1 -- 172.21.15.146:0/3943110235 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70486968 0 0) 0x7f960c001110 con 0x7f96100cbd50 2024-09-13T18:27:01.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9607fff700 1 -- 172.21.15.146:0/3943110235 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95fc0035d0 con 0x7f96100cbd50 2024-09-13T18:27:01.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9615e43700 1 -- 172.21.15.146:0/3943110235 >> v2:172.21.15.154:3300/0 conn(0x7f96100ca210 msgr2=0x7f96100b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:01.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9615e43700 1 --2- 172.21.15.146:0/3943110235 >> v2:172.21.15.154:3300/0 conn(0x7f96100ca210 0x7f96100b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9615e43700 1 -- 172.21.15.146:0/3943110235 >> v1:172.21.15.146:6789/0 conn(0x7f96100cbd50 legacy=0x7f96100701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9615e43700 1 -- 172.21.15.146:0/3943110235 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961015da20 con 0x7f9610071390 2024-09-13T18:27:01.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9615e43700 1 --2- 172.21.15.146:0/3943110235 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9610071390 0x7f96100cc130 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9608009d90 tx=0x7f960800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de46d86a013c4121 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:01.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9607fff700 1 -- 172.21.15.146:0/3943110235 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960800e070 con 0x7f9610071390 2024-09-13T18:27:01.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9607fff700 1 -- 172.21.15.146:0/3943110235 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960800bb20 con 0x7f9610071390 2024-09-13T18:27:01.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9607fff700 1 -- 172.21.15.146:0/3943110235 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608012430 con 0x7f9610071390 2024-09-13T18:27:01.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9616644700 1 -- 172.21.15.146:0/3943110235 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9610071390 msgr2=0x7f96100cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9616644700 1 --2- 172.21.15.146:0/3943110235 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9610071390 0x7f96100cc130 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9608009d90 tx=0x7f960800b330 comp rx=0 tx=0).stop 2024-09-13T18:27:01.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9616644700 1 -- 172.21.15.146:0/3943110235 shutdown_connections 2024-09-13T18:27:01.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9616644700 1 --2- 172.21.15.146:0/3943110235 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9610071390 0x7f96100cc130 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9616644700 1 --2- 172.21.15.146:0/3943110235 >> v2:172.21.15.154:3300/0 conn(0x7f96100ca210 0x7f96100b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9616644700 1 -- 172.21.15.146:0/3943110235 >> 172.21.15.146:0/3943110235 conn(0x7f9610066270 msgr2=0x7f9610066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:01.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9616644700 1 -- 172.21.15.146:0/3943110235 shutdown_connections 2024-09-13T18:27:01.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.285+0000 7f9616644700 1 -- 172.21.15.146:0/3943110235 wait complete. 2024-09-13T18:27:01.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9616644700 1 Processor -- start 2024-09-13T18:27:01.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9616644700 1 -- start start 2024-09-13T18:27:01.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9616644700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9610071390 0x7f96101f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9616644700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96100ca210 0x7f961014ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9616644700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96100cbd50 0x7f96101e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9616644700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96101e31a0 con 0x7f9610071390 2024-09-13T18:27:01.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9616644700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96101f3b40 con 0x7f96100ca210 2024-09-13T18:27:01.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9616644700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961014f600 con 0x7f96100cbd50 2024-09-13T18:27:01.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9615642700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9610071390 0x7f96101f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9615642700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9610071390 0x7f96101f3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54350/0 (socket says 172.21.15.146:54350) 2024-09-13T18:27:01.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9615642700 1 -- 172.21.15.146:0/2392936004 learned_addr learned my addr 172.21.15.146:0/2392936004 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:01.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9614e41700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96100ca210 0x7f961014ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9615e43700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96100cbd50 0x7f96101e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9615642700 1 -- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96100cbd50 msgr2=0x7f96101e35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9615642700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96100cbd50 0x7f96101e35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9615642700 1 -- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96100ca210 msgr2=0x7f961014ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9615642700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96100ca210 0x7f961014ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9615642700 1 -- 172.21.15.146:0/2392936004 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9608009a70 con 0x7f9610071390 2024-09-13T18:27:01.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9615642700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9610071390 0x7f96101f3600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f960c001270 tx=0x7f960c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:01.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9605ffb700 1 -- 172.21.15.146:0/2392936004 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c005d90 con 0x7f9610071390 2024-09-13T18:27:01.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9605ffb700 1 -- 172.21.15.146:0/2392936004 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c0070d0 con 0x7f9610071390 2024-09-13T18:27:01.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9605ffb700 1 -- 172.21.15.146:0/2392936004 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c007480 con 0x7f9610071390 2024-09-13T18:27:01.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9616644700 1 -- 172.21.15.146:0/2392936004 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96101ef9e0 con 0x7f9610071390 2024-09-13T18:27:01.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.289+0000 7f9616644700 1 -- 172.21.15.146:0/2392936004 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96101effa0 con 0x7f9610071390 2024-09-13T18:27:01.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.293+0000 7f9616644700 1 -- 172.21.15.146:0/2392936004 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961015d500 con 0x7f9610071390 2024-09-13T18:27:01.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.293+0000 7f9605ffb700 1 -- 172.21.15.146:0/2392936004 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f960c0075e0 con 0x7f9610071390 2024-09-13T18:27:01.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.293+0000 7f9605ffb700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95f4066730 0x7f95f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.293+0000 7f9605ffb700 1 -- 172.21.15.146:0/2392936004 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f960c083660 con 0x7f9610071390 2024-09-13T18:27:01.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.293+0000 7f9614e41700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95f4066730 0x7f95f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.293+0000 7f9614e41700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95f4066730 0x7f95f4068bf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f96101e4c10 tx=0x7f9600009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:01.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.301+0000 7f9605ffb700 1 -- 172.21.15.146:0/2392936004 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f960c050770 con 0x7f9610071390 2024-09-13T18:27:01.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.469+0000 7f9616644700 1 -- 172.21.15.146:0/2392936004 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f961015cf40 con 0x7f95f4066730 2024-09-13T18:27:01.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.489+0000 7f9605ffb700 1 -- 172.21.15.146:0/2392936004 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345276 (secure 0 0 0) 0x7f96000118c0 con 0x7f95f4066730 2024-09-13T18:27:01.499 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:01.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.493+0000 7f9616644700 1 -- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95f4066730 msgr2=0x7f95f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.493+0000 7f9616644700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95f4066730 0x7f95f4068bf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f96101e4c10 tx=0x7f9600009240 comp rx=0 tx=0).stop 2024-09-13T18:27:01.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.493+0000 7f9616644700 1 -- 172.21.15.146:0/2392936004 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9610071390 msgr2=0x7f96101f3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.493+0000 7f9616644700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9610071390 0x7f96101f3600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f960c001270 tx=0x7f960c00c7d0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.493+0000 7f9616644700 1 -- 172.21.15.146:0/2392936004 shutdown_connections 2024-09-13T18:27:01.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.493+0000 7f9616644700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95f4066730 0x7f95f4068bf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.493+0000 7f9616644700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96100cbd50 0x7f96101e35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.493+0000 7f9616644700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96100ca210 0x7f961014ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.493+0000 7f9616644700 1 --2- 172.21.15.146:0/2392936004 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9610071390 0x7f96101f3600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.493+0000 7f9616644700 1 -- 172.21.15.146:0/2392936004 >> 172.21.15.146:0/2392936004 conn(0x7f9610066270 msgr2=0x7f96101487c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:01.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.493+0000 7f9616644700 1 -- 172.21.15.146:0/2392936004 shutdown_connections 2024-09-13T18:27:01.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.493+0000 7f9616644700 1 -- 172.21.15.146:0/2392936004 wait complete. 2024-09-13T18:27:01.504 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:27:01.517 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-13T18:27:00.827152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222476,"kb_used_data":2208,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754752244,"statfs":{"total":773094113280,"available":772866297856,"internally_reserved":0,"allocated":2260992,"data_stored":2015306,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":225431099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005224"},"pg_stats":[{"pgid":"2.1f","version":"25'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.345647+0000","last_change":"2024-09-13T18:26:31.972797+0000","last_active":"2024-09-13T18:26:37.345647+0000","last_peered":"2024-09-13T18:26:37.345647+0000","last_clean":"2024-09-13T18:26:37.345647+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:26:37.345647+0000","last_undegraded":"2024-09-13T18:26:37.345647+0000","last_fullsized":"2024-09-13T18:26:37.345647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:50.511747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384385+0000","last_change":"2024-09-13T18:26:32.349901+0000","last_active":"2024-09-13T18:26:36.384385+0000","last_peered":"2024-09-13T18:26:36.384385+0000","last_clean":"2024-09-13T18:26:36.384385+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:26:36.384385+0000","last_undegraded":"2024-09-13T18:26:36.384385+0000","last_fullsized":"2024-09-13T18:26:36.384385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:24.978813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768058+0000","last_change":"2024-09-13T18:26:36.407579+0000","last_active":"2024-09-13T18:26:36.768058+0000","last_peered":"2024-09-13T18:26:36.768058+0000","last_clean":"2024-09-13T18:26:36.768058+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:26:36.768058+0000","last_undegraded":"2024-09-13T18:26:36.768058+0000","last_fullsized":"2024-09-13T18:26:36.768058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:53:56.370013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769520+0000","last_change":"2024-09-13T18:26:31.980716+0000","last_active":"2024-09-13T18:26:36.769520+0000","last_peered":"2024-09-13T18:26:36.769520+0000","last_clean":"2024-09-13T18:26:36.769520+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:26:36.769520+0000","last_undegraded":"2024-09-13T18:26:36.769520+0000","last_fullsized":"2024-09-13T18:26:36.769520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:20.020903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388711+0000","last_change":"2024-09-13T18:26:32.342690+0000","last_active":"2024-09-13T18:26:36.388711+0000","last_peered":"2024-09-13T18:26:36.388711+0000","last_clean":"2024-09-13T18:26:36.388711+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:26:36.388711+0000","last_undegraded":"2024-09-13T18:26:36.388711+0000","last_fullsized":"2024-09-13T18:26:36.388711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.212930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.398481+0000","last_change":"2024-09-13T18:26:36.396545+0000","last_active":"2024-09-13T18:26:36.398481+0000","last_peered":"2024-09-13T18:26:36.398481+0000","last_clean":"2024-09-13T18:26:36.398481+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:26:36.398481+0000","last_undegraded":"2024-09-13T18:26:36.398481+0000","last_fullsized":"2024-09-13T18:26:36.398481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:38.760215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.349299+0000","last_change":"2024-09-13T18:26:31.977823+0000","last_active":"2024-09-13T18:26:37.349299+0000","last_peered":"2024-09-13T18:26:37.349299+0000","last_clean":"2024-09-13T18:26:37.349299+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:26:37.349299+0000","last_undegraded":"2024-09-13T18:26:37.349299+0000","last_fullsized":"2024-09-13T18:26:37.349299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:46.956527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394225+0000","last_change":"2024-09-13T18:26:32.361288+0000","last_active":"2024-09-13T18:26:36.394225+0000","last_peered":"2024-09-13T18:26:36.394225+0000","last_clean":"2024-09-13T18:26:36.394225+0000","last_became_active":"2024-09-13T18:26:32.360918+0000","last_became_peered":"2024-09-13T18:26:32.360918+0000","last_unstale":"2024-09-13T18:26:36.394225+0000","last_undegraded":"2024-09-13T18:26:36.394225+0000","last_fullsized":"2024-09-13T18:26:36.394225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:30.143436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408364+0000","last_change":"2024-09-13T18:26:36.406675+0000","last_active":"2024-09-13T18:26:36.408364+0000","last_peered":"2024-09-13T18:26:36.408364+0000","last_clean":"2024-09-13T18:26:36.408364+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:26:36.408364+0000","last_undegraded":"2024-09-13T18:26:36.408364+0000","last_fullsized":"2024-09-13T18:26:36.408364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:36.896027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.347551+0000","last_change":"2024-09-13T18:26:31.979860+0000","last_active":"2024-09-13T18:26:37.347551+0000","last_peered":"2024-09-13T18:26:37.347551+0000","last_clean":"2024-09-13T18:26:37.347551+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:26:37.347551+0000","last_undegraded":"2024-09-13T18:26:37.347551+0000","last_fullsized":"2024-09-13T18:26:37.347551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:45.323631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393203+0000","last_change":"2024-09-13T18:26:32.351494+0000","last_active":"2024-09-13T18:26:36.393203+0000","last_peered":"2024-09-13T18:26:36.393203+0000","last_clean":"2024-09-13T18:26:36.393203+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:26:36.393203+0000","last_undegraded":"2024-09-13T18:26:36.393203+0000","last_fullsized":"2024-09-13T18:26:36.393203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:52.804526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400955+0000","last_change":"2024-09-13T18:26:36.398763+0000","last_active":"2024-09-13T18:26:36.400955+0000","last_peered":"2024-09-13T18:26:36.400955+0000","last_clean":"2024-09-13T18:26:36.400955+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:26:36.400955+0000","last_undegraded":"2024-09-13T18:26:36.400955+0000","last_fullsized":"2024-09-13T18:26:36.400955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:35.417438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897834+0000","last_change":"2024-09-13T18:26:32.360976+0000","last_active":"2024-09-13T18:26:36.897834+0000","last_peered":"2024-09-13T18:26:36.897834+0000","last_clean":"2024-09-13T18:26:36.897834+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:26:36.897834+0000","last_undegraded":"2024-09-13T18:26:36.897834+0000","last_fullsized":"2024-09-13T18:26:36.897834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:53:52.373515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.348911+0000","last_change":"2024-09-13T18:26:31.977891+0000","last_active":"2024-09-13T18:26:37.348911+0000","last_peered":"2024-09-13T18:26:37.348911+0000","last_clean":"2024-09-13T18:26:37.348911+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:26:37.348911+0000","last_undegraded":"2024-09-13T18:26:37.348911+0000","last_fullsized":"2024-09-13T18:26:37.348911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:01.453094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400563+0000","last_change":"2024-09-13T18:26:36.397484+0000","last_active":"2024-09-13T18:26:36.400563+0000","last_peered":"2024-09-13T18:26:36.400563+0000","last_clean":"2024-09-13T18:26:36.400563+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:26:36.400563+0000","last_undegraded":"2024-09-13T18:26:36.400563+0000","last_fullsized":"2024-09-13T18:26:36.400563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:44.686244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386958+0000","last_change":"2024-09-13T18:26:32.345564+0000","last_active":"2024-09-13T18:26:36.386958+0000","last_peered":"2024-09-13T18:26:36.386958+0000","last_clean":"2024-09-13T18:26:36.386958+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:26:36.386958+0000","last_undegraded":"2024-09-13T18:26:36.386958+0000","last_fullsized":"2024-09-13T18:26:36.386958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:54.331806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386241+0000","last_change":"2024-09-13T18:26:31.974107+0000","last_active":"2024-09-13T18:26:36.386241+0000","last_peered":"2024-09-13T18:26:36.386241+0000","last_clean":"2024-09-13T18:26:36.386241+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:26:36.386241+0000","last_undegraded":"2024-09-13T18:26:36.386241+0000","last_fullsized":"2024-09-13T18:26:36.386241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:37.547480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408253+0000","last_change":"2024-09-13T18:26:36.405986+0000","last_active":"2024-09-13T18:26:36.408253+0000","last_peered":"2024-09-13T18:26:36.408253+0000","last_clean":"2024-09-13T18:26:36.408253+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:26:36.408253+0000","last_undegraded":"2024-09-13T18:26:36.408253+0000","last_fullsized":"2024-09-13T18:26:36.408253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:30.052430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383772+0000","last_change":"2024-09-13T18:26:32.340616+0000","last_active":"2024-09-13T18:26:36.383772+0000","last_peered":"2024-09-13T18:26:36.383772+0000","last_clean":"2024-09-13T18:26:36.383772+0000","last_became_active":"2024-09-13T18:26:32.339405+0000","last_became_peered":"2024-09-13T18:26:32.339405+0000","last_unstale":"2024-09-13T18:26:36.383772+0000","last_undegraded":"2024-09-13T18:26:36.383772+0000","last_fullsized":"2024-09-13T18:26:36.383772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:26.081896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.385156+0000","last_change":"2024-09-13T18:26:31.968895+0000","last_active":"2024-09-13T18:26:36.385156+0000","last_peered":"2024-09-13T18:26:36.385156+0000","last_clean":"2024-09-13T18:26:36.385156+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:26:36.385156+0000","last_undegraded":"2024-09-13T18:26:36.385156+0000","last_fullsized":"2024-09-13T18:26:36.385156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:15.525097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767834+0000","last_change":"2024-09-13T18:26:36.406484+0000","last_active":"2024-09-13T18:26:36.767834+0000","last_peered":"2024-09-13T18:26:36.767834+0000","last_clean":"2024-09-13T18:26:36.767834+0000","last_became_active":"2024-09-13T18:26:36.406161+0000","last_became_peered":"2024-09-13T18:26:36.406161+0000","last_unstale":"2024-09-13T18:26:36.767834+0000","last_undegraded":"2024-09-13T18:26:36.767834+0000","last_fullsized":"2024-09-13T18:26:36.767834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:53.330919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391435+0000","last_change":"2024-09-13T18:26:32.360406+0000","last_active":"2024-09-13T18:26:36.391435+0000","last_peered":"2024-09-13T18:26:36.391435+0000","last_clean":"2024-09-13T18:26:36.391435+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:26:36.391435+0000","last_undegraded":"2024-09-13T18:26:36.391435+0000","last_fullsized":"2024-09-13T18:26:36.391435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:37.417707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404700+0000","last_change":"2024-09-13T18:26:31.976872+0000","last_active":"2024-09-13T18:26:36.404700+0000","last_peered":"2024-09-13T18:26:36.404700+0000","last_clean":"2024-09-13T18:26:36.404700+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:26:36.404700+0000","last_undegraded":"2024-09-13T18:26:36.404700+0000","last_fullsized":"2024-09-13T18:26:36.404700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:26.163029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408244+0000","last_change":"2024-09-13T18:26:36.405722+0000","last_active":"2024-09-13T18:26:36.408244+0000","last_peered":"2024-09-13T18:26:36.408244+0000","last_clean":"2024-09-13T18:26:36.408244+0000","last_became_active":"2024-09-13T18:26:36.405310+0000","last_became_peered":"2024-09-13T18:26:36.405310+0000","last_unstale":"2024-09-13T18:26:36.408244+0000","last_undegraded":"2024-09-13T18:26:36.408244+0000","last_fullsized":"2024-09-13T18:26:36.408244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:09.815655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.389080+0000","last_change":"2024-09-13T18:26:32.343915+0000","last_active":"2024-09-13T18:26:36.389080+0000","last_peered":"2024-09-13T18:26:36.389080+0000","last_clean":"2024-09-13T18:26:36.389080+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:26:36.389080+0000","last_undegraded":"2024-09-13T18:26:36.389080+0000","last_fullsized":"2024-09-13T18:26:36.389080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:19.357243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.349375+0000","last_change":"2024-09-13T18:26:31.979944+0000","last_active":"2024-09-13T18:26:37.349375+0000","last_peered":"2024-09-13T18:26:37.349375+0000","last_clean":"2024-09-13T18:26:37.349375+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:26:37.349375+0000","last_undegraded":"2024-09-13T18:26:37.349375+0000","last_fullsized":"2024-09-13T18:26:37.349375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:21.721921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.411958+0000","last_change":"2024-09-13T18:26:36.410028+0000","last_active":"2024-09-13T18:26:36.411958+0000","last_peered":"2024-09-13T18:26:36.411958+0000","last_clean":"2024-09-13T18:26:36.411958+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:26:36.411958+0000","last_undegraded":"2024-09-13T18:26:36.411958+0000","last_fullsized":"2024-09-13T18:26:36.411958+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:32.264341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387875+0000","last_change":"2024-09-13T18:26:32.358980+0000","last_active":"2024-09-13T18:26:36.387875+0000","last_peered":"2024-09-13T18:26:36.387875+0000","last_clean":"2024-09-13T18:26:36.387875+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:26:36.387875+0000","last_undegraded":"2024-09-13T18:26:36.387875+0000","last_fullsized":"2024-09-13T18:26:36.387875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:08.748139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769462+0000","last_change":"2024-09-13T18:26:31.968972+0000","last_active":"2024-09-13T18:26:36.769462+0000","last_peered":"2024-09-13T18:26:36.769462+0000","last_clean":"2024-09-13T18:26:36.769462+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:26:36.769462+0000","last_undegraded":"2024-09-13T18:26:36.769462+0000","last_fullsized":"2024-09-13T18:26:36.769462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:56.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768011+0000","last_change":"2024-09-13T18:26:36.407034+0000","last_active":"2024-09-13T18:26:36.768011+0000","last_peered":"2024-09-13T18:26:36.768011+0000","last_clean":"2024-09-13T18:26:36.768011+0000","last_became_active":"2024-09-13T18:26:36.406473+0000","last_became_peered":"2024-09-13T18:26:36.406473+0000","last_unstale":"2024-09-13T18:26:36.768011+0000","last_undegraded":"2024-09-13T18:26:36.768011+0000","last_fullsized":"2024-09-13T18:26:36.768011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:53.815901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386420+0000","last_change":"2024-09-13T18:26:32.342823+0000","last_active":"2024-09-13T18:26:36.386420+0000","last_peered":"2024-09-13T18:26:36.386420+0000","last_clean":"2024-09-13T18:26:36.386420+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:26:36.386420+0000","last_undegraded":"2024-09-13T18:26:36.386420+0000","last_fullsized":"2024-09-13T18:26:36.386420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:28.051252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407752+0000","last_change":"2024-09-13T18:26:31.978976+0000","last_active":"2024-09-13T18:26:36.407752+0000","last_peered":"2024-09-13T18:26:36.407752+0000","last_clean":"2024-09-13T18:26:36.407752+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:26:36.407752+0000","last_undegraded":"2024-09-13T18:26:36.407752+0000","last_fullsized":"2024-09-13T18:26:36.407752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:11.279557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103455+0000","last_change":"2024-09-13T18:26:36.408844+0000","last_active":"2024-09-13T18:26:37.103455+0000","last_peered":"2024-09-13T18:26:37.103455+0000","last_clean":"2024-09-13T18:26:37.103455+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:26:37.103455+0000","last_undegraded":"2024-09-13T18:26:37.103455+0000","last_fullsized":"2024-09-13T18:26:37.103455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:06.969342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393043+0000","last_change":"2024-09-13T18:26:32.355038+0000","last_active":"2024-09-13T18:26:36.393043+0000","last_peered":"2024-09-13T18:26:36.393043+0000","last_clean":"2024-09-13T18:26:36.393043+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:26:36.393043+0000","last_undegraded":"2024-09-13T18:26:36.393043+0000","last_fullsized":"2024-09-13T18:26:36.393043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.159231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386562+0000","last_change":"2024-09-13T18:26:31.972931+0000","last_active":"2024-09-13T18:26:36.386562+0000","last_peered":"2024-09-13T18:26:36.386562+0000","last_clean":"2024-09-13T18:26:36.386562+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:26:36.386562+0000","last_undegraded":"2024-09-13T18:26:36.386562+0000","last_fullsized":"2024-09-13T18:26:36.386562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:05.646924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766310+0000","last_change":"2024-09-13T18:26:36.398876+0000","last_active":"2024-09-13T18:26:36.766310+0000","last_peered":"2024-09-13T18:26:36.766310+0000","last_clean":"2024-09-13T18:26:36.766310+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:26:36.766310+0000","last_undegraded":"2024-09-13T18:26:36.766310+0000","last_fullsized":"2024-09-13T18:26:36.766310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:56.918239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392625+0000","last_change":"2024-09-13T18:26:32.353111+0000","last_active":"2024-09-13T18:26:36.392625+0000","last_peered":"2024-09-13T18:26:36.392625+0000","last_clean":"2024-09-13T18:26:36.392625+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:26:36.392625+0000","last_undegraded":"2024-09-13T18:26:36.392625+0000","last_fullsized":"2024-09-13T18:26:36.392625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:43.927936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391864+0000","last_change":"2024-09-13T18:26:31.976075+0000","last_active":"2024-09-13T18:26:36.391864+0000","last_peered":"2024-09-13T18:26:36.391864+0000","last_clean":"2024-09-13T18:26:36.391864+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:26:36.391864+0000","last_undegraded":"2024-09-13T18:26:36.391864+0000","last_fullsized":"2024-09-13T18:26:36.391864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:03.693518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766877+0000","last_change":"2024-09-13T18:26:36.401777+0000","last_active":"2024-09-13T18:26:36.766877+0000","last_peered":"2024-09-13T18:26:36.766877+0000","last_clean":"2024-09-13T18:26:36.766877+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:26:36.766877+0000","last_undegraded":"2024-09-13T18:26:36.766877+0000","last_fullsized":"2024-09-13T18:26:36.766877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:43.440284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384991+0000","last_change":"2024-09-13T18:26:32.343791+0000","last_active":"2024-09-13T18:26:36.384991+0000","last_peered":"2024-09-13T18:26:36.384991+0000","last_clean":"2024-09-13T18:26:36.384991+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:26:36.384991+0000","last_undegraded":"2024-09-13T18:26:36.384991+0000","last_fullsized":"2024-09-13T18:26:36.384991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:36.296571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.375229+0000","last_change":"2024-09-13T18:26:31.970786+0000","last_active":"2024-09-13T18:26:36.375229+0000","last_peered":"2024-09-13T18:26:36.375229+0000","last_clean":"2024-09-13T18:26:36.375229+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:26:36.375229+0000","last_undegraded":"2024-09-13T18:26:36.375229+0000","last_fullsized":"2024-09-13T18:26:36.375229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:54.584009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767929+0000","last_change":"2024-09-13T18:26:36.401529+0000","last_active":"2024-09-13T18:26:36.767929+0000","last_peered":"2024-09-13T18:26:36.767929+0000","last_clean":"2024-09-13T18:26:36.767929+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:26:36.767929+0000","last_undegraded":"2024-09-13T18:26:36.767929+0000","last_fullsized":"2024-09-13T18:26:36.767929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:47.318347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.382662+0000","last_change":"2024-09-13T18:26:32.350470+0000","last_active":"2024-09-13T18:26:36.382662+0000","last_peered":"2024-09-13T18:26:36.382662+0000","last_clean":"2024-09-13T18:26:36.382662+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:26:36.382662+0000","last_undegraded":"2024-09-13T18:26:36.382662+0000","last_fullsized":"2024-09-13T18:26:36.382662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:55.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404763+0000","last_change":"2024-09-13T18:26:31.972587+0000","last_active":"2024-09-13T18:26:36.404763+0000","last_peered":"2024-09-13T18:26:36.404763+0000","last_clean":"2024-09-13T18:26:36.404763+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:26:36.404763+0000","last_undegraded":"2024-09-13T18:26:36.404763+0000","last_fullsized":"2024-09-13T18:26:36.404763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:27.547763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103735+0000","last_change":"2024-09-13T18:26:36.405086+0000","last_active":"2024-09-13T18:26:37.103735+0000","last_peered":"2024-09-13T18:26:37.103735+0000","last_clean":"2024-09-13T18:26:37.103735+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:26:37.103735+0000","last_undegraded":"2024-09-13T18:26:37.103735+0000","last_fullsized":"2024-09-13T18:26:37.103735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:02:51.694634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388036+0000","last_change":"2024-09-13T18:26:32.350510+0000","last_active":"2024-09-13T18:26:36.388036+0000","last_peered":"2024-09-13T18:26:36.388036+0000","last_clean":"2024-09-13T18:26:36.388036+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:26:36.388036+0000","last_undegraded":"2024-09-13T18:26:36.388036+0000","last_fullsized":"2024-09-13T18:26:36.388036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:40.850255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391579+0000","last_change":"2024-09-13T18:26:31.978355+0000","last_active":"2024-09-13T18:26:36.391579+0000","last_peered":"2024-09-13T18:26:36.391579+0000","last_clean":"2024-09-13T18:26:36.391579+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:26:36.391579+0000","last_undegraded":"2024-09-13T18:26:36.391579+0000","last_fullsized":"2024-09-13T18:26:36.391579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:12.946518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404778+0000","last_change":"2024-09-13T18:26:26.931403+0000","last_active":"2024-09-13T18:26:36.404778+0000","last_peered":"2024-09-13T18:26:36.404778+0000","last_clean":"2024-09-13T18:26:36.404778+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:26:36.404778+0000","last_undegraded":"2024-09-13T18:26:36.404778+0000","last_fullsized":"2024-09-13T18:26:36.404778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_clean_scrub_stamp":"2024-09-13T18:26:24.880028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:30.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897926+0000","last_change":"2024-09-13T18:26:36.409827+0000","last_active":"2024-09-13T18:26:36.897926+0000","last_peered":"2024-09-13T18:26:36.897926+0000","last_clean":"2024-09-13T18:26:36.897926+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:26:36.897926+0000","last_undegraded":"2024-09-13T18:26:36.897926+0000","last_fullsized":"2024-09-13T18:26:36.897926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:50.482400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405746+0000","last_change":"2024-09-13T18:26:32.356850+0000","last_active":"2024-09-13T18:26:36.405746+0000","last_peered":"2024-09-13T18:26:36.405746+0000","last_clean":"2024-09-13T18:26:36.405746+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:26:36.405746+0000","last_undegraded":"2024-09-13T18:26:36.405746+0000","last_fullsized":"2024-09-13T18:26:36.405746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:38.710828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.376787+0000","last_change":"2024-09-13T18:26:31.975942+0000","last_active":"2024-09-13T18:26:36.376787+0000","last_peered":"2024-09-13T18:26:36.376787+0000","last_clean":"2024-09-13T18:26:36.376787+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:26:36.376787+0000","last_undegraded":"2024-09-13T18:26:36.376787+0000","last_fullsized":"2024-09-13T18:26:36.376787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:05.840582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897805+0000","last_change":"2024-09-13T18:26:36.409802+0000","last_active":"2024-09-13T18:26:36.897805+0000","last_peered":"2024-09-13T18:26:36.897805+0000","last_clean":"2024-09-13T18:26:36.897805+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:26:36.897805+0000","last_undegraded":"2024-09-13T18:26:36.897805+0000","last_fullsized":"2024-09-13T18:26:36.897805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:38.178151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394309+0000","last_change":"2024-09-13T18:26:32.355181+0000","last_active":"2024-09-13T18:26:36.394309+0000","last_peered":"2024-09-13T18:26:36.394309+0000","last_clean":"2024-09-13T18:26:36.394309+0000","last_became_active":"2024-09-13T18:26:32.353043+0000","last_became_peered":"2024-09-13T18:26:32.353043+0000","last_unstale":"2024-09-13T18:26:36.394309+0000","last_undegraded":"2024-09-13T18:26:36.394309+0000","last_fullsized":"2024-09-13T18:26:36.394309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:04:17.000915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767488+0000","last_change":"2024-09-13T18:26:31.966618+0000","last_active":"2024-09-13T18:26:36.767488+0000","last_peered":"2024-09-13T18:26:36.767488+0000","last_clean":"2024-09-13T18:26:36.767488+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:26:36.767488+0000","last_undegraded":"2024-09-13T18:26:36.767488+0000","last_fullsized":"2024-09-13T18:26:36.767488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:54:26.422611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103645+0000","last_change":"2024-09-13T18:26:36.394344+0000","last_active":"2024-09-13T18:26:37.103645+0000","last_peered":"2024-09-13T18:26:37.103645+0000","last_clean":"2024-09-13T18:26:37.103645+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:26:37.103645+0000","last_undegraded":"2024-09-13T18:26:37.103645+0000","last_fullsized":"2024-09-13T18:26:37.103645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:18.425816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405784+0000","last_change":"2024-09-13T18:26:32.365655+0000","last_active":"2024-09-13T18:26:36.405784+0000","last_peered":"2024-09-13T18:26:36.405784+0000","last_clean":"2024-09-13T18:26:36.405784+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:26:36.405784+0000","last_undegraded":"2024-09-13T18:26:36.405784+0000","last_fullsized":"2024-09-13T18:26:36.405784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:40.288128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343912+0000","last_change":"2024-09-13T18:26:31.970652+0000","last_active":"2024-09-13T18:26:37.343912+0000","last_peered":"2024-09-13T18:26:37.343912+0000","last_clean":"2024-09-13T18:26:37.343912+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:26:37.343912+0000","last_undegraded":"2024-09-13T18:26:37.343912+0000","last_fullsized":"2024-09-13T18:26:37.343912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:11.169661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.770631+0000","last_change":"2024-09-13T18:26:36.410280+0000","last_active":"2024-09-13T18:26:36.770631+0000","last_peered":"2024-09-13T18:26:36.770631+0000","last_clean":"2024-09-13T18:26:36.770631+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:26:36.770631+0000","last_undegraded":"2024-09-13T18:26:36.770631+0000","last_fullsized":"2024-09-13T18:26:36.770631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:01.395290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405642+0000","last_change":"2024-09-13T18:26:32.361575+0000","last_active":"2024-09-13T18:26:36.405642+0000","last_peered":"2024-09-13T18:26:36.405642+0000","last_clean":"2024-09-13T18:26:36.405642+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:26:36.405642+0000","last_undegraded":"2024-09-13T18:26:36.405642+0000","last_fullsized":"2024-09-13T18:26:36.405642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:08.818877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.239195+0000","last_change":"2024-09-13T18:26:31.977029+0000","last_active":"2024-09-13T18:26:37.239195+0000","last_peered":"2024-09-13T18:26:37.239195+0000","last_clean":"2024-09-13T18:26:37.239195+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:26:37.239195+0000","last_undegraded":"2024-09-13T18:26:37.239195+0000","last_fullsized":"2024-09-13T18:26:37.239195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:13.701689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407565+0000","last_change":"2024-09-13T18:26:36.403479+0000","last_active":"2024-09-13T18:26:36.407565+0000","last_peered":"2024-09-13T18:26:36.407565+0000","last_clean":"2024-09-13T18:26:36.407565+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:26:36.407565+0000","last_undegraded":"2024-09-13T18:26:36.407565+0000","last_fullsized":"2024-09-13T18:26:36.407565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:51.095852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404976+0000","last_change":"2024-09-13T18:26:32.365143+0000","last_active":"2024-09-13T18:26:36.404976+0000","last_peered":"2024-09-13T18:26:36.404976+0000","last_clean":"2024-09-13T18:26:36.404976+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:26:36.404976+0000","last_undegraded":"2024-09-13T18:26:36.404976+0000","last_fullsized":"2024-09-13T18:26:36.404976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:46.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.342638+0000","last_change":"2024-09-13T18:26:31.965802+0000","last_active":"2024-09-13T18:26:37.342638+0000","last_peered":"2024-09-13T18:26:37.342638+0000","last_clean":"2024-09-13T18:26:37.342638+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:26:37.342638+0000","last_undegraded":"2024-09-13T18:26:37.342638+0000","last_fullsized":"2024-09-13T18:26:37.342638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:12.759181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103768+0000","last_change":"2024-09-13T18:26:36.399235+0000","last_active":"2024-09-13T18:26:37.103768+0000","last_peered":"2024-09-13T18:26:37.103768+0000","last_clean":"2024-09-13T18:26:37.103768+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:26:37.103768+0000","last_undegraded":"2024-09-13T18:26:37.103768+0000","last_fullsized":"2024-09-13T18:26:37.103768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:13:54.711083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897760+0000","last_change":"2024-09-13T18:26:32.357403+0000","last_active":"2024-09-13T18:26:36.897760+0000","last_peered":"2024-09-13T18:26:36.897760+0000","last_clean":"2024-09-13T18:26:36.897760+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:26:36.897760+0000","last_undegraded":"2024-09-13T18:26:36.897760+0000","last_fullsized":"2024-09-13T18:26:36.897760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:12.600397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383205+0000","last_change":"2024-09-13T18:26:31.976148+0000","last_active":"2024-09-13T18:26:36.383205+0000","last_peered":"2024-09-13T18:26:36.383205+0000","last_clean":"2024-09-13T18:26:36.383205+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:26:36.383205+0000","last_undegraded":"2024-09-13T18:26:36.383205+0000","last_fullsized":"2024-09-13T18:26:36.383205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:38.023636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769582+0000","last_change":"2024-09-13T18:26:36.406177+0000","last_active":"2024-09-13T18:26:36.769582+0000","last_peered":"2024-09-13T18:26:36.769582+0000","last_clean":"2024-09-13T18:26:36.769582+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:26:36.769582+0000","last_undegraded":"2024-09-13T18:26:36.769582+0000","last_fullsized":"2024-09-13T18:26:36.769582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:00.309995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405459+0000","last_change":"2024-09-13T18:26:32.361726+0000","last_active":"2024-09-13T18:26:36.405459+0000","last_peered":"2024-09-13T18:26:36.405459+0000","last_clean":"2024-09-13T18:26:36.405459+0000","last_became_active":"2024-09-13T18:26:32.361432+0000","last_became_peered":"2024-09-13T18:26:32.361432+0000","last_unstale":"2024-09-13T18:26:36.405459+0000","last_undegraded":"2024-09-13T18:26:36.405459+0000","last_fullsized":"2024-09-13T18:26:36.405459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:42.021515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391337+0000","last_change":"2024-09-13T18:26:31.982114+0000","last_active":"2024-09-13T18:26:36.391337+0000","last_peered":"2024-09-13T18:26:36.391337+0000","last_clean":"2024-09-13T18:26:36.391337+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:26:36.391337+0000","last_undegraded":"2024-09-13T18:26:36.391337+0000","last_fullsized":"2024-09-13T18:26:36.391337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.266528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769049+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:26:36.769049+0000","last_peered":"2024-09-13T18:26:36.769049+0000","last_clean":"2024-09-13T18:26:36.769049+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:26:36.769049+0000","last_undegraded":"2024-09-13T18:26:36.769049+0000","last_fullsized":"2024-09-13T18:26:36.769049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:58.853971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388651+0000","last_change":"2024-09-13T18:26:32.351145+0000","last_active":"2024-09-13T18:26:36.388651+0000","last_peered":"2024-09-13T18:26:36.388651+0000","last_clean":"2024-09-13T18:26:36.388651+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:26:36.388651+0000","last_undegraded":"2024-09-13T18:26:36.388651+0000","last_fullsized":"2024-09-13T18:26:36.388651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:57.002435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.344611+0000","last_change":"2024-09-13T18:26:31.961202+0000","last_active":"2024-09-13T18:26:37.344611+0000","last_peered":"2024-09-13T18:26:37.344611+0000","last_clean":"2024-09-13T18:26:37.344611+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:26:37.344611+0000","last_undegraded":"2024-09-13T18:26:37.344611+0000","last_fullsized":"2024-09-13T18:26:37.344611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:27.340127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.772209+0000","last_change":"2024-09-13T18:26:36.410014+0000","last_active":"2024-09-13T18:26:36.772209+0000","last_peered":"2024-09-13T18:26:36.772209+0000","last_clean":"2024-09-13T18:26:36.772209+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:26:36.772209+0000","last_undegraded":"2024-09-13T18:26:36.772209+0000","last_fullsized":"2024-09-13T18:26:36.772209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:24.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405669+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:26:36.405669+0000","last_peered":"2024-09-13T18:26:36.405669+0000","last_clean":"2024-09-13T18:26:36.405669+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:26:36.405669+0000","last_undegraded":"2024-09-13T18:26:36.405669+0000","last_fullsized":"2024-09-13T18:26:36.405669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:40.102815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:45.381784+0000","last_change":"2024-09-13T18:26:31.974110+0000","last_active":"2024-09-13T18:26:45.381784+0000","last_peered":"2024-09-13T18:26:45.381784+0000","last_clean":"2024-09-13T18:26:45.381784+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:26:45.381784+0000","last_undegraded":"2024-09-13T18:26:45.381784+0000","last_fullsized":"2024-09-13T18:26:45.381784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:56.058561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766576+0000","last_change":"2024-09-13T18:26:36.394080+0000","last_active":"2024-09-13T18:26:36.766576+0000","last_peered":"2024-09-13T18:26:36.766576+0000","last_clean":"2024-09-13T18:26:36.766576+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:26:36.766576+0000","last_undegraded":"2024-09-13T18:26:36.766576+0000","last_fullsized":"2024-09-13T18:26:36.766576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:47.121814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392195+0000","last_change":"2024-09-13T18:26:32.362873+0000","last_active":"2024-09-13T18:26:36.392195+0000","last_peered":"2024-09-13T18:26:36.392195+0000","last_clean":"2024-09-13T18:26:36.392195+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:26:36.392195+0000","last_undegraded":"2024-09-13T18:26:36.392195+0000","last_fullsized":"2024-09-13T18:26:36.392195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:37.598189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.344576+0000","last_change":"2024-09-13T18:26:31.976401+0000","last_active":"2024-09-13T18:26:37.344576+0000","last_peered":"2024-09-13T18:26:37.344576+0000","last_clean":"2024-09-13T18:26:37.344576+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:26:37.344576+0000","last_undegraded":"2024-09-13T18:26:37.344576+0000","last_fullsized":"2024-09-13T18:26:37.344576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:26.434292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408317+0000","last_change":"2024-09-13T18:26:36.406228+0000","last_active":"2024-09-13T18:26:36.408317+0000","last_peered":"2024-09-13T18:26:36.408317+0000","last_clean":"2024-09-13T18:26:36.408317+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:26:36.408317+0000","last_undegraded":"2024-09-13T18:26:36.408317+0000","last_fullsized":"2024-09-13T18:26:36.408317+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:49.198397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388733+0000","last_change":"2024-09-13T18:26:32.350841+0000","last_active":"2024-09-13T18:26:36.388733+0000","last_peered":"2024-09-13T18:26:36.388733+0000","last_clean":"2024-09-13T18:26:36.388733+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:26:36.388733+0000","last_undegraded":"2024-09-13T18:26:36.388733+0000","last_fullsized":"2024-09-13T18:26:36.388733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:05.137882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391761+0000","last_change":"2024-09-13T18:26:31.978214+0000","last_active":"2024-09-13T18:26:36.391761+0000","last_peered":"2024-09-13T18:26:36.391761+0000","last_clean":"2024-09-13T18:26:36.391761+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:26:36.391761+0000","last_undegraded":"2024-09-13T18:26:36.391761+0000","last_fullsized":"2024-09-13T18:26:36.391761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:17.381375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768498+0000","last_change":"2024-09-13T18:26:36.406090+0000","last_active":"2024-09-13T18:26:36.768498+0000","last_peered":"2024-09-13T18:26:36.768498+0000","last_clean":"2024-09-13T18:26:36.768498+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:26:36.768498+0000","last_undegraded":"2024-09-13T18:26:36.768498+0000","last_fullsized":"2024-09-13T18:26:36.768498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:09.865147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391759+0000","last_change":"2024-09-13T18:26:32.357701+0000","last_active":"2024-09-13T18:26:36.391759+0000","last_peered":"2024-09-13T18:26:36.391759+0000","last_clean":"2024-09-13T18:26:36.391759+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:26:36.391759+0000","last_undegraded":"2024-09-13T18:26:36.391759+0000","last_fullsized":"2024-09-13T18:26:36.391759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:47.587683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384903+0000","last_change":"2024-09-13T18:26:31.980133+0000","last_active":"2024-09-13T18:26:36.384903+0000","last_peered":"2024-09-13T18:26:36.384903+0000","last_clean":"2024-09-13T18:26:36.384903+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:26:36.384903+0000","last_undegraded":"2024-09-13T18:26:36.384903+0000","last_fullsized":"2024-09-13T18:26:36.384903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:05.354335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768120+0000","last_change":"2024-09-13T18:26:36.406553+0000","last_active":"2024-09-13T18:26:36.768120+0000","last_peered":"2024-09-13T18:26:36.768120+0000","last_clean":"2024-09-13T18:26:36.768120+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:26:36.768120+0000","last_undegraded":"2024-09-13T18:26:36.768120+0000","last_fullsized":"2024-09-13T18:26:36.768120+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:02.980826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103481+0000","last_change":"2024-09-13T18:26:36.406986+0000","last_active":"2024-09-13T18:26:37.103481+0000","last_peered":"2024-09-13T18:26:37.103481+0000","last_clean":"2024-09-13T18:26:37.103481+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:26:37.103481+0000","last_undegraded":"2024-09-13T18:26:37.103481+0000","last_fullsized":"2024-09-13T18:26:37.103481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:53.984231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384974+0000","last_change":"2024-09-13T18:26:32.349081+0000","last_active":"2024-09-13T18:26:36.384974+0000","last_peered":"2024-09-13T18:26:36.384974+0000","last_clean":"2024-09-13T18:26:36.384974+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:26:36.384974+0000","last_undegraded":"2024-09-13T18:26:36.384974+0000","last_fullsized":"2024-09-13T18:26:36.384974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:47.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391124+0000","last_change":"2024-09-13T18:26:31.968944+0000","last_active":"2024-09-13T18:26:36.391124+0000","last_peered":"2024-09-13T18:26:36.391124+0000","last_clean":"2024-09-13T18:26:36.391124+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:26:36.391124+0000","last_undegraded":"2024-09-13T18:26:36.391124+0000","last_fullsized":"2024-09-13T18:26:36.391124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:29.363312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.898021+0000","last_change":"2024-09-13T18:26:36.396418+0000","last_active":"2024-09-13T18:26:36.898021+0000","last_peered":"2024-09-13T18:26:36.898021+0000","last_clean":"2024-09-13T18:26:36.898021+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:26:36.898021+0000","last_undegraded":"2024-09-13T18:26:36.898021+0000","last_fullsized":"2024-09-13T18:26:36.898021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:03.252773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383753+0000","last_change":"2024-09-13T18:26:32.351369+0000","last_active":"2024-09-13T18:26:36.383753+0000","last_peered":"2024-09-13T18:26:36.383753+0000","last_clean":"2024-09-13T18:26:36.383753+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:26:36.383753+0000","last_undegraded":"2024-09-13T18:26:36.383753+0000","last_fullsized":"2024-09-13T18:26:36.383753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:10.892347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387713+0000","last_change":"2024-09-13T18:26:31.972048+0000","last_active":"2024-09-13T18:26:36.387713+0000","last_peered":"2024-09-13T18:26:36.387713+0000","last_clean":"2024-09-13T18:26:36.387713+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:26:36.387713+0000","last_undegraded":"2024-09-13T18:26:36.387713+0000","last_fullsized":"2024-09-13T18:26:36.387713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:28.430119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343265+0000","last_change":"2024-09-13T18:26:31.975310+0000","last_active":"2024-09-13T18:26:37.343265+0000","last_peered":"2024-09-13T18:26:37.343265+0000","last_clean":"2024-09-13T18:26:37.343265+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:26:37.343265+0000","last_undegraded":"2024-09-13T18:26:37.343265+0000","last_fullsized":"2024-09-13T18:26:37.343265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:52.915310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388817+0000","last_change":"2024-09-13T18:26:32.345825+0000","last_active":"2024-09-13T18:26:36.388817+0000","last_peered":"2024-09-13T18:26:36.388817+0000","last_clean":"2024-09-13T18:26:36.388817+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:26:36.388817+0000","last_undegraded":"2024-09-13T18:26:36.388817+0000","last_fullsized":"2024-09-13T18:26:36.388817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:25.200489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.395253+0000","last_change":"2024-09-13T18:26:36.391520+0000","last_active":"2024-09-13T18:26:36.395253+0000","last_peered":"2024-09-13T18:26:36.395253+0000","last_clean":"2024-09-13T18:26:36.395253+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:26:36.395253+0000","last_undegraded":"2024-09-13T18:26:36.395253+0000","last_fullsized":"2024-09-13T18:26:36.395253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:42.638713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768901+0000","last_change":"2024-09-13T18:26:31.976603+0000","last_active":"2024-09-13T18:26:36.768901+0000","last_peered":"2024-09-13T18:26:36.768901+0000","last_clean":"2024-09-13T18:26:36.768901+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:26:36.768901+0000","last_undegraded":"2024-09-13T18:26:36.768901+0000","last_fullsized":"2024-09-13T18:26:36.768901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:03.965376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387960+0000","last_change":"2024-09-13T18:26:32.352982+0000","last_active":"2024-09-13T18:26:36.387960+0000","last_peered":"2024-09-13T18:26:36.387960+0000","last_clean":"2024-09-13T18:26:36.387960+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:26:36.387960+0000","last_undegraded":"2024-09-13T18:26:36.387960+0000","last_fullsized":"2024-09-13T18:26:36.387960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:55.670360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769131+0000","last_change":"2024-09-13T18:26:36.403817+0000","last_active":"2024-09-13T18:26:36.769131+0000","last_peered":"2024-09-13T18:26:36.769131+0000","last_clean":"2024-09-13T18:26:36.769131+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:26:36.769131+0000","last_undegraded":"2024-09-13T18:26:36.769131+0000","last_fullsized":"2024-09-13T18:26:36.769131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:34.892901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897728+0000","last_change":"2024-09-13T18:26:32.357856+0000","last_active":"2024-09-13T18:26:36.897728+0000","last_peered":"2024-09-13T18:26:36.897728+0000","last_clean":"2024-09-13T18:26:36.897728+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:26:36.897728+0000","last_undegraded":"2024-09-13T18:26:36.897728+0000","last_fullsized":"2024-09-13T18:26:36.897728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:26.316214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.375208+0000","last_change":"2024-09-13T18:26:32.350966+0000","last_active":"2024-09-13T18:26:36.375208+0000","last_peered":"2024-09-13T18:26:36.375208+0000","last_clean":"2024-09-13T18:26:36.375208+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:26:36.375208+0000","last_undegraded":"2024-09-13T18:26:36.375208+0000","last_fullsized":"2024-09-13T18:26:36.375208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:52.632112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391246+0000","last_change":"2024-09-13T18:26:32.355806+0000","last_active":"2024-09-13T18:26:36.391246+0000","last_peered":"2024-09-13T18:26:36.391246+0000","last_clean":"2024-09-13T18:26:36.391246+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:26:36.391246+0000","last_undegraded":"2024-09-13T18:26:36.391246+0000","last_fullsized":"2024-09-13T18:26:36.391246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.988231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388740+0000","last_change":"2024-09-13T18:26:32.340965+0000","last_active":"2024-09-13T18:26:36.388740+0000","last_peered":"2024-09-13T18:26:36.388740+0000","last_clean":"2024-09-13T18:26:36.388740+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:26:36.388740+0000","last_undegraded":"2024-09-13T18:26:36.388740+0000","last_fullsized":"2024-09-13T18:26:36.388740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:01:04.716919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384429+0000","last_change":"2024-09-13T18:26:32.349625+0000","last_active":"2024-09-13T18:26:36.384429+0000","last_peered":"2024-09-13T18:26:36.384429+0000","last_clean":"2024-09-13T18:26:36.384429+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:26:36.384429+0000","last_undegraded":"2024-09-13T18:26:36.384429+0000","last_fullsized":"2024-09-13T18:26:36.384429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.519754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386397+0000","last_change":"2024-09-13T18:26:32.349170+0000","last_active":"2024-09-13T18:26:36.386397+0000","last_peered":"2024-09-13T18:26:36.386397+0000","last_clean":"2024-09-13T18:26:36.386397+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:26:36.386397+0000","last_undegraded":"2024-09-13T18:26:36.386397+0000","last_fullsized":"2024-09-13T18:26:36.386397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:43.942732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897998+0000","last_change":"2024-09-13T18:26:32.364786+0000","last_active":"2024-09-13T18:26:36.897998+0000","last_peered":"2024-09-13T18:26:36.897998+0000","last_clean":"2024-09-13T18:26:36.897998+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:26:36.897998+0000","last_undegraded":"2024-09-13T18:26:36.897998+0000","last_fullsized":"2024-09-13T18:26:36.897998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:22.949875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387927+0000","last_change":"2024-09-13T18:26:32.347541+0000","last_active":"2024-09-13T18:26:36.387927+0000","last_peered":"2024-09-13T18:26:36.387927+0000","last_clean":"2024-09-13T18:26:36.387927+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:26:36.387927+0000","last_undegraded":"2024-09-13T18:26:36.387927+0000","last_fullsized":"2024-09-13T18:26:36.387927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:58.144128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390892+0000","last_change":"2024-09-13T18:26:32.353359+0000","last_active":"2024-09-13T18:26:36.390892+0000","last_peered":"2024-09-13T18:26:36.390892+0000","last_clean":"2024-09-13T18:26:36.390892+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:26:36.390892+0000","last_undegraded":"2024-09-13T18:26:36.390892+0000","last_fullsized":"2024-09-13T18:26:36.390892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:09.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.380537+0000","last_change":"2024-09-13T18:26:32.349566+0000","last_active":"2024-09-13T18:26:36.380537+0000","last_peered":"2024-09-13T18:26:36.380537+0000","last_clean":"2024-09-13T18:26:36.380537+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:26:36.380537+0000","last_undegraded":"2024-09-13T18:26:36.380537+0000","last_fullsized":"2024-09-13T18:26:36.380537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:22.341899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391168+0000","last_change":"2024-09-13T18:26:32.358640+0000","last_active":"2024-09-13T18:26:36.391168+0000","last_peered":"2024-09-13T18:26:36.391168+0000","last_clean":"2024-09-13T18:26:36.391168+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:26:36.391168+0000","last_undegraded":"2024-09-13T18:26:36.391168+0000","last_fullsized":"2024-09-13T18:26:36.391168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:40.174754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388633+0000","last_change":"2024-09-13T18:26:32.353675+0000","last_active":"2024-09-13T18:26:36.388633+0000","last_peered":"2024-09-13T18:26:36.388633+0000","last_clean":"2024-09-13T18:26:36.388633+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:26:36.388633+0000","last_undegraded":"2024-09-13T18:26:36.388633+0000","last_fullsized":"2024-09-13T18:26:36.388633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:58.501843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897877+0000","last_change":"2024-09-13T18:26:32.353223+0000","last_active":"2024-09-13T18:26:36.897877+0000","last_peered":"2024-09-13T18:26:36.897877+0000","last_clean":"2024-09-13T18:26:36.897877+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:26:36.897877+0000","last_undegraded":"2024-09-13T18:26:36.897877+0000","last_fullsized":"2024-09-13T18:26:36.897877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:45.898213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407736+0000","last_change":"2024-09-13T18:26:32.357354+0000","last_active":"2024-09-13T18:26:36.407736+0000","last_peered":"2024-09-13T18:26:36.407736+0000","last_clean":"2024-09-13T18:26:36.407736+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:26:36.407736+0000","last_undegraded":"2024-09-13T18:26:36.407736+0000","last_fullsized":"2024-09-13T18:26:36.407736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:19.185008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.376351+0000","last_change":"2024-09-13T18:26:32.348832+0000","last_active":"2024-09-13T18:26:36.376351+0000","last_peered":"2024-09-13T18:26:36.376351+0000","last_clean":"2024-09-13T18:26:36.376351+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:26:36.376351+0000","last_undegraded":"2024-09-13T18:26:36.376351+0000","last_fullsized":"2024-09-13T18:26:36.376351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:11:59.874789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393723+0000","last_change":"2024-09-13T18:26:32.356902+0000","last_active":"2024-09-13T18:26:36.393723+0000","last_peered":"2024-09-13T18:26:36.393723+0000","last_clean":"2024-09-13T18:26:36.393723+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:26:36.393723+0000","last_undegraded":"2024-09-13T18:26:36.393723+0000","last_fullsized":"2024-09-13T18:26:36.393723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:18.362772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390031+0000","last_change":"2024-09-13T18:26:32.347047+0000","last_active":"2024-09-13T18:26:36.390031+0000","last_peered":"2024-09-13T18:26:36.390031+0000","last_clean":"2024-09-13T18:26:36.390031+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:26:36.390031+0000","last_undegraded":"2024-09-13T18:26:36.390031+0000","last_fullsized":"2024-09-13T18:26:36.390031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:40.882789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.385066+0000","last_change":"2024-09-13T18:26:32.349651+0000","last_active":"2024-09-13T18:26:36.385066+0000","last_peered":"2024-09-13T18:26:36.385066+0000","last_clean":"2024-09-13T18:26:36.385066+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:26:36.385066+0000","last_undegraded":"2024-09-13T18:26:36.385066+0000","last_fullsized":"2024-09-13T18:26:36.385066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:33.356563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388522+0000","last_change":"2024-09-13T18:26:32.344607+0000","last_active":"2024-09-13T18:26:36.388522+0000","last_peered":"2024-09-13T18:26:36.388522+0000","last_clean":"2024-09-13T18:26:36.388522+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:26:36.388522+0000","last_undegraded":"2024-09-13T18:26:36.388522+0000","last_fullsized":"2024-09-13T18:26:36.388522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:31.658172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387737+0000","last_change":"2024-09-13T18:26:32.347717+0000","last_active":"2024-09-13T18:26:36.387737+0000","last_peered":"2024-09-13T18:26:36.387737+0000","last_clean":"2024-09-13T18:26:36.387737+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:26:36.387737+0000","last_undegraded":"2024-09-13T18:26:36.387737+0000","last_fullsized":"2024-09-13T18:26:36.387737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:47.187837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392104+0000","last_change":"2024-09-13T18:26:32.359506+0000","last_active":"2024-09-13T18:26:36.392104+0000","last_peered":"2024-09-13T18:26:36.392104+0000","last_clean":"2024-09-13T18:26:36.392104+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:26:36.392104+0000","last_undegraded":"2024-09-13T18:26:36.392104+0000","last_fullsized":"2024-09-13T18:26:36.392104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:39:57.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897951+0000","last_change":"2024-09-13T18:26:32.357729+0000","last_active":"2024-09-13T18:26:36.897951+0000","last_peered":"2024-09-13T18:26:36.897951+0000","last_clean":"2024-09-13T18:26:36.897951+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:26:36.897951+0000","last_undegraded":"2024-09-13T18:26:36.897951+0000","last_fullsized":"2024-09-13T18:26:36.897951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:05.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897974+0000","last_change":"2024-09-13T18:26:32.357980+0000","last_active":"2024-09-13T18:26:36.897974+0000","last_peered":"2024-09-13T18:26:36.897974+0000","last_clean":"2024-09-13T18:26:36.897974+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:26:36.897974+0000","last_undegraded":"2024-09-13T18:26:36.897974+0000","last_fullsized":"2024-09-13T18:26:36.897974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:00.858702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405699+0000","last_change":"2024-09-13T18:26:32.365638+0000","last_active":"2024-09-13T18:26:36.405699+0000","last_peered":"2024-09-13T18:26:36.405699+0000","last_clean":"2024-09-13T18:26:36.405699+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:26:36.405699+0000","last_undegraded":"2024-09-13T18:26:36.405699+0000","last_fullsized":"2024-09-13T18:26:36.405699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.065853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383235+0000","last_change":"2024-09-13T18:26:32.345496+0000","last_active":"2024-09-13T18:26:36.383235+0000","last_peered":"2024-09-13T18:26:36.383235+0000","last_clean":"2024-09-13T18:26:36.383235+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:26:36.383235+0000","last_undegraded":"2024-09-13T18:26:36.383235+0000","last_fullsized":"2024-09-13T18:26:36.383235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:22.447315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404895+0000","last_change":"2024-09-13T18:26:32.356901+0000","last_active":"2024-09-13T18:26:36.404895+0000","last_peered":"2024-09-13T18:26:36.404895+0000","last_clean":"2024-09-13T18:26:36.404895+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:26:36.404895+0000","last_undegraded":"2024-09-13T18:26:36.404895+0000","last_fullsized":"2024-09-13T18:26:36.404895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:03.311603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391082+0000","last_change":"2024-09-13T18:26:32.355203+0000","last_active":"2024-09-13T18:26:36.391082+0000","last_peered":"2024-09-13T18:26:36.391082+0000","last_clean":"2024-09-13T18:26:36.391082+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:26:36.391082+0000","last_undegraded":"2024-09-13T18:26:36.391082+0000","last_fullsized":"2024-09-13T18:26:36.391082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:52.769708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405315+0000","last_change":"2024-09-13T18:26:32.358190+0000","last_active":"2024-09-13T18:26:36.405315+0000","last_peered":"2024-09-13T18:26:36.405315+0000","last_clean":"2024-09-13T18:26:36.405315+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:26:36.405315+0000","last_undegraded":"2024-09-13T18:26:36.405315+0000","last_fullsized":"2024-09-13T18:26:36.405315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:35.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393258+0000","last_change":"2024-09-13T18:26:32.355121+0000","last_active":"2024-09-13T18:26:36.393258+0000","last_peered":"2024-09-13T18:26:36.393258+0000","last_clean":"2024-09-13T18:26:36.393258+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:26:36.393258+0000","last_undegraded":"2024-09-13T18:26:36.393258+0000","last_fullsized":"2024-09-13T18:26:36.393258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:48.427332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388810+0000","last_change":"2024-09-13T18:26:32.341497+0000","last_active":"2024-09-13T18:26:36.388810+0000","last_peered":"2024-09-13T18:26:36.388810+0000","last_clean":"2024-09-13T18:26:36.388810+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:26:36.388810+0000","last_undegraded":"2024-09-13T18:26:36.388810+0000","last_fullsized":"2024-09-13T18:26:36.388810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:15.510048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386627+0000","last_change":"2024-09-13T18:26:32.350106+0000","last_active":"2024-09-13T18:26:36.386627+0000","last_peered":"2024-09-13T18:26:36.386627+0000","last_clean":"2024-09-13T18:26:36.386627+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:26:36.386627+0000","last_undegraded":"2024-09-13T18:26:36.386627+0000","last_fullsized":"2024-09-13T18:26:36.386627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:12.387451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405597+0000","last_change":"2024-09-13T18:26:32.362196+0000","last_active":"2024-09-13T18:26:36.405597+0000","last_peered":"2024-09-13T18:26:36.405597+0000","last_clean":"2024-09-13T18:26:36.405597+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:26:36.405597+0000","last_undegraded":"2024-09-13T18:26:36.405597+0000","last_fullsized":"2024-09-13T18:26:36.405597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:14.946904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":694123,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":103299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":103173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":262144,"data_stored":103263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":103333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4239999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":694179,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.655}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:27:01.523 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:27:01.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.633+0000 7f5f188d7700 1 Processor -- start 2024-09-13T18:27:01.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.633+0000 7f5f188d7700 1 -- start start 2024-09-13T18:27:01.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f188d7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f140a9cf0 0x7f5f140cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f188d7700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f14159f80 con 0x7f5f140c9350 2024-09-13T18:27:01.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f188d7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f140aa120 con 0x7f5f140c8f70 2024-09-13T18:27:01.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f188d7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f140ca1d0 con 0x7f5f140a9cf0 2024-09-13T18:27:01.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f1259c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5f140c9350 0x7f5f140c7530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35412/0 (socket says 172.21.15.146:35412) 2024-09-13T18:27:01.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f1259c700 1 -- 172.21.15.146:0/2296395552 learned_addr learned my addr 172.21.15.146:0/2296395552 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:01.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f1359e700 1 --2- 172.21.15.146:0/2296395552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f140a9cf0 0x7f5f140cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f11d9b700 1 -- 172.21.15.146:0/2296395552 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468805935 0 0) 0x7f5f14159f80 con 0x7f5f140c9350 2024-09-13T18:27:01.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f11d9b700 1 -- 172.21.15.146:0/2296395552 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f000035d0 con 0x7f5f140c9350 2024-09-13T18:27:01.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f11d9b700 1 -- 172.21.15.146:0/2296395552 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4259463205 0 0) 0x7f5f140aa120 con 0x7f5f140c8f70 2024-09-13T18:27:01.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f11d9b700 1 -- 172.21.15.146:0/2296395552 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f14159f80 con 0x7f5f140c8f70 2024-09-13T18:27:01.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f1359e700 1 -- 172.21.15.146:0/2296395552 >> v1:172.21.15.154:6789/0 conn(0x7f5f140c8f70 legacy=0x7f5f140723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f1359e700 1 -- 172.21.15.146:0/2296395552 >> v1:172.21.15.146:6789/0 conn(0x7f5f140c9350 legacy=0x7f5f140c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f1359e700 1 -- 172.21.15.146:0/2296395552 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f14159670 con 0x7f5f140a9cf0 2024-09-13T18:27:01.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f1359e700 1 --2- 172.21.15.146:0/2296395552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f140a9cf0 0x7f5f140cac60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009880 tx=0x7f5f0c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e09e09324c66f6fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:01.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f11d9b700 1 -- 172.21.15.146:0/2296395552 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c011830 con 0x7f5f140a9cf0 2024-09-13T18:27:01.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f11d9b700 1 -- 172.21.15.146:0/2296395552 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c011990 con 0x7f5f140a9cf0 2024-09-13T18:27:01.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f11d9b700 1 -- 172.21.15.146:0/2296395552 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c011d20 con 0x7f5f140a9cf0 2024-09-13T18:27:01.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f188d7700 1 -- 172.21.15.146:0/2296395552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f140a9cf0 msgr2=0x7f5f140cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f188d7700 1 --2- 172.21.15.146:0/2296395552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f140a9cf0 0x7f5f140cac60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009880 tx=0x7f5f0c009b90 comp rx=0 tx=0).stop 2024-09-13T18:27:01.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f188d7700 1 -- 172.21.15.146:0/2296395552 shutdown_connections 2024-09-13T18:27:01.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f188d7700 1 --2- 172.21.15.146:0/2296395552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f140a9cf0 0x7f5f140cac60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f188d7700 1 -- 172.21.15.146:0/2296395552 >> 172.21.15.146:0/2296395552 conn(0x7f5f14066090 msgr2=0x7f5f14066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:01.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f188d7700 1 -- 172.21.15.146:0/2296395552 shutdown_connections 2024-09-13T18:27:01.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.637+0000 7f5f188d7700 1 -- 172.21.15.146:0/2296395552 wait complete. 2024-09-13T18:27:01.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f188d7700 1 Processor -- start 2024-09-13T18:27:01.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f188d7700 1 -- start start 2024-09-13T18:27:01.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f188d7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f140a9cf0 0x7f5f141f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f188d7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f140c8f70 0x7f5f141ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f188d7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f140c9350 0x7f5f1414fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f188d7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f141f2cf0 con 0x7f5f140a9cf0 2024-09-13T18:27:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f188d7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f141515c0 con 0x7f5f140c8f70 2024-09-13T18:27:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f188d7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f14151970 con 0x7f5f140c9350 2024-09-13T18:27:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f12d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f140a9cf0 0x7f5f141f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f12d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f140a9cf0 0x7f5f141f3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54358/0 (socket says 172.21.15.146:54358) 2024-09-13T18:27:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f12d9d700 1 -- 172.21.15.146:0/1969158799 learned_addr learned my addr 172.21.15.146:0/1969158799 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f1259c700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f140c8f70 0x7f5f141ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f1359e700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f140c9350 0x7f5f1414fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f12d9d700 1 -- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f140c9350 msgr2=0x7f5f1414fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f12d9d700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f140c9350 0x7f5f1414fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f12d9d700 1 -- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f140c8f70 msgr2=0x7f5f141ef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f12d9d700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f140c8f70 0x7f5f141ef0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f12d9d700 1 -- 172.21.15.146:0/1969158799 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c0094b0 con 0x7f5f140a9cf0 2024-09-13T18:27:01.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f1259c700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f140c8f70 0x7f5f141ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:01.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f1359e700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f140c9350 0x7f5f1414fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:01.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f12d9d700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f140a9cf0 0x7f5f141f3110 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00c760 tx=0x7f5efc00cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:01.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f0b7fe700 1 -- 172.21.15.146:0/1969158799 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0052d0 con 0x7f5f140a9cf0 2024-09-13T18:27:01.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f0b7fe700 1 -- 172.21.15.146:0/1969158799 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5efc005430 con 0x7f5f140a9cf0 2024-09-13T18:27:01.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f188d7700 1 -- 172.21.15.146:0/1969158799 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f14151c30 con 0x7f5f140a9cf0 2024-09-13T18:27:01.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f0b7fe700 1 -- 172.21.15.146:0/1969158799 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0055a0 con 0x7f5f140a9cf0 2024-09-13T18:27:01.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f188d7700 1 -- 172.21.15.146:0/1969158799 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f141521f0 con 0x7f5f140a9cf0 2024-09-13T18:27:01.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.641+0000 7f5f188d7700 1 -- 172.21.15.146:0/1969158799 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f141f2f60 con 0x7f5f140a9cf0 2024-09-13T18:27:01.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.645+0000 7f5f0b7fe700 1 -- 172.21.15.146:0/1969158799 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5efc0038e0 con 0x7f5f140a9cf0 2024-09-13T18:27:01.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.645+0000 7f5f0b7fe700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5ef4066730 0x7f5ef4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.645+0000 7f5f0b7fe700 1 -- 172.21.15.146:0/1969158799 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5efc083a10 con 0x7f5f140a9cf0 2024-09-13T18:27:01.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.645+0000 7f5f1259c700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5ef4066730 0x7f5ef4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.645+0000 7f5f1259c700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5ef4066730 0x7f5ef4068bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5f14155290 tx=0x7f5f04009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:01.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.653+0000 7f5f0b7fe700 1 -- 172.21.15.146:0/1969158799 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5efc0509f0 con 0x7f5f140a9cf0 2024-09-13T18:27:01.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.821+0000 7f5f188d7700 1 -- 172.21.15.146:0/1969158799 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f140adb50 con 0x7f5ef4066730 2024-09-13T18:27:01.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.841+0000 7f5f0b7fe700 1 -- 172.21.15.146:0/1969158799 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345276 (secure 0 0 0) 0x7f5f140adb50 con 0x7f5ef4066730 2024-09-13T18:27:01.851 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:01.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.849+0000 7f5f188d7700 1 -- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5ef4066730 msgr2=0x7f5ef4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.849+0000 7f5f188d7700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5ef4066730 0x7f5ef4068bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5f14155290 tx=0x7f5f04009040 comp rx=0 tx=0).stop 2024-09-13T18:27:01.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.849+0000 7f5f188d7700 1 -- 172.21.15.146:0/1969158799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f140a9cf0 msgr2=0x7f5f141f3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.849+0000 7f5f188d7700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f140a9cf0 0x7f5f141f3110 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00c760 tx=0x7f5efc00cb20 comp rx=0 tx=0).stop 2024-09-13T18:27:01.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.849+0000 7f5f188d7700 1 -- 172.21.15.146:0/1969158799 shutdown_connections 2024-09-13T18:27:01.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.849+0000 7f5f188d7700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5ef4066730 0x7f5ef4068bf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.849+0000 7f5f188d7700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f140c9350 0x7f5f1414fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.849+0000 7f5f188d7700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f140c8f70 0x7f5f141ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.849+0000 7f5f188d7700 1 --2- 172.21.15.146:0/1969158799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f140a9cf0 0x7f5f141f3110 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.849+0000 7f5f188d7700 1 -- 172.21.15.146:0/1969158799 >> 172.21.15.146:0/1969158799 conn(0x7f5f14066090 msgr2=0x7f5f14158c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:01.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.849+0000 7f5f188d7700 1 -- 172.21.15.146:0/1969158799 shutdown_connections 2024-09-13T18:27:01.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.849+0000 7f5f188d7700 1 -- 172.21.15.146:0/1969158799 wait complete. 2024-09-13T18:27:01.855 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:27:01.870 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-13T18:27:00.827152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222476,"kb_used_data":2208,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754752244,"statfs":{"total":773094113280,"available":772866297856,"internally_reserved":0,"allocated":2260992,"data_stored":2015306,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":225431099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005224"},"pg_stats":[{"pgid":"2.1f","version":"25'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.345647+0000","last_change":"2024-09-13T18:26:31.972797+0000","last_active":"2024-09-13T18:26:37.345647+0000","last_peered":"2024-09-13T18:26:37.345647+0000","last_clean":"2024-09-13T18:26:37.345647+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:26:37.345647+0000","last_undegraded":"2024-09-13T18:26:37.345647+0000","last_fullsized":"2024-09-13T18:26:37.345647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:50.511747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384385+0000","last_change":"2024-09-13T18:26:32.349901+0000","last_active":"2024-09-13T18:26:36.384385+0000","last_peered":"2024-09-13T18:26:36.384385+0000","last_clean":"2024-09-13T18:26:36.384385+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:26:36.384385+0000","last_undegraded":"2024-09-13T18:26:36.384385+0000","last_fullsized":"2024-09-13T18:26:36.384385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:24.978813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768058+0000","last_change":"2024-09-13T18:26:36.407579+0000","last_active":"2024-09-13T18:26:36.768058+0000","last_peered":"2024-09-13T18:26:36.768058+0000","last_clean":"2024-09-13T18:26:36.768058+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:26:36.768058+0000","last_undegraded":"2024-09-13T18:26:36.768058+0000","last_fullsized":"2024-09-13T18:26:36.768058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:53:56.370013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769520+0000","last_change":"2024-09-13T18:26:31.980716+0000","last_active":"2024-09-13T18:26:36.769520+0000","last_peered":"2024-09-13T18:26:36.769520+0000","last_clean":"2024-09-13T18:26:36.769520+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:26:36.769520+0000","last_undegraded":"2024-09-13T18:26:36.769520+0000","last_fullsized":"2024-09-13T18:26:36.769520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:20.020903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388711+0000","last_change":"2024-09-13T18:26:32.342690+0000","last_active":"2024-09-13T18:26:36.388711+0000","last_peered":"2024-09-13T18:26:36.388711+0000","last_clean":"2024-09-13T18:26:36.388711+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:26:36.388711+0000","last_undegraded":"2024-09-13T18:26:36.388711+0000","last_fullsized":"2024-09-13T18:26:36.388711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.212930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.398481+0000","last_change":"2024-09-13T18:26:36.396545+0000","last_active":"2024-09-13T18:26:36.398481+0000","last_peered":"2024-09-13T18:26:36.398481+0000","last_clean":"2024-09-13T18:26:36.398481+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:26:36.398481+0000","last_undegraded":"2024-09-13T18:26:36.398481+0000","last_fullsized":"2024-09-13T18:26:36.398481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:38.760215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.349299+0000","last_change":"2024-09-13T18:26:31.977823+0000","last_active":"2024-09-13T18:26:37.349299+0000","last_peered":"2024-09-13T18:26:37.349299+0000","last_clean":"2024-09-13T18:26:37.349299+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:26:37.349299+0000","last_undegraded":"2024-09-13T18:26:37.349299+0000","last_fullsized":"2024-09-13T18:26:37.349299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:46.956527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394225+0000","last_change":"2024-09-13T18:26:32.361288+0000","last_active":"2024-09-13T18:26:36.394225+0000","last_peered":"2024-09-13T18:26:36.394225+0000","last_clean":"2024-09-13T18:26:36.394225+0000","last_became_active":"2024-09-13T18:26:32.360918+0000","last_became_peered":"2024-09-13T18:26:32.360918+0000","last_unstale":"2024-09-13T18:26:36.394225+0000","last_undegraded":"2024-09-13T18:26:36.394225+0000","last_fullsized":"2024-09-13T18:26:36.394225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:30.143436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408364+0000","last_change":"2024-09-13T18:26:36.406675+0000","last_active":"2024-09-13T18:26:36.408364+0000","last_peered":"2024-09-13T18:26:36.408364+0000","last_clean":"2024-09-13T18:26:36.408364+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:26:36.408364+0000","last_undegraded":"2024-09-13T18:26:36.408364+0000","last_fullsized":"2024-09-13T18:26:36.408364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:36.896027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.347551+0000","last_change":"2024-09-13T18:26:31.979860+0000","last_active":"2024-09-13T18:26:37.347551+0000","last_peered":"2024-09-13T18:26:37.347551+0000","last_clean":"2024-09-13T18:26:37.347551+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:26:37.347551+0000","last_undegraded":"2024-09-13T18:26:37.347551+0000","last_fullsized":"2024-09-13T18:26:37.347551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:45.323631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393203+0000","last_change":"2024-09-13T18:26:32.351494+0000","last_active":"2024-09-13T18:26:36.393203+0000","last_peered":"2024-09-13T18:26:36.393203+0000","last_clean":"2024-09-13T18:26:36.393203+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:26:36.393203+0000","last_undegraded":"2024-09-13T18:26:36.393203+0000","last_fullsized":"2024-09-13T18:26:36.393203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:52.804526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400955+0000","last_change":"2024-09-13T18:26:36.398763+0000","last_active":"2024-09-13T18:26:36.400955+0000","last_peered":"2024-09-13T18:26:36.400955+0000","last_clean":"2024-09-13T18:26:36.400955+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:26:36.400955+0000","last_undegraded":"2024-09-13T18:26:36.400955+0000","last_fullsized":"2024-09-13T18:26:36.400955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:35.417438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897834+0000","last_change":"2024-09-13T18:26:32.360976+0000","last_active":"2024-09-13T18:26:36.897834+0000","last_peered":"2024-09-13T18:26:36.897834+0000","last_clean":"2024-09-13T18:26:36.897834+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:26:36.897834+0000","last_undegraded":"2024-09-13T18:26:36.897834+0000","last_fullsized":"2024-09-13T18:26:36.897834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:53:52.373515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.348911+0000","last_change":"2024-09-13T18:26:31.977891+0000","last_active":"2024-09-13T18:26:37.348911+0000","last_peered":"2024-09-13T18:26:37.348911+0000","last_clean":"2024-09-13T18:26:37.348911+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:26:37.348911+0000","last_undegraded":"2024-09-13T18:26:37.348911+0000","last_fullsized":"2024-09-13T18:26:37.348911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:01.453094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400563+0000","last_change":"2024-09-13T18:26:36.397484+0000","last_active":"2024-09-13T18:26:36.400563+0000","last_peered":"2024-09-13T18:26:36.400563+0000","last_clean":"2024-09-13T18:26:36.400563+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:26:36.400563+0000","last_undegraded":"2024-09-13T18:26:36.400563+0000","last_fullsized":"2024-09-13T18:26:36.400563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:44.686244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386958+0000","last_change":"2024-09-13T18:26:32.345564+0000","last_active":"2024-09-13T18:26:36.386958+0000","last_peered":"2024-09-13T18:26:36.386958+0000","last_clean":"2024-09-13T18:26:36.386958+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:26:36.386958+0000","last_undegraded":"2024-09-13T18:26:36.386958+0000","last_fullsized":"2024-09-13T18:26:36.386958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:54.331806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386241+0000","last_change":"2024-09-13T18:26:31.974107+0000","last_active":"2024-09-13T18:26:36.386241+0000","last_peered":"2024-09-13T18:26:36.386241+0000","last_clean":"2024-09-13T18:26:36.386241+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:26:36.386241+0000","last_undegraded":"2024-09-13T18:26:36.386241+0000","last_fullsized":"2024-09-13T18:26:36.386241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:37.547480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408253+0000","last_change":"2024-09-13T18:26:36.405986+0000","last_active":"2024-09-13T18:26:36.408253+0000","last_peered":"2024-09-13T18:26:36.408253+0000","last_clean":"2024-09-13T18:26:36.408253+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:26:36.408253+0000","last_undegraded":"2024-09-13T18:26:36.408253+0000","last_fullsized":"2024-09-13T18:26:36.408253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:30.052430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383772+0000","last_change":"2024-09-13T18:26:32.340616+0000","last_active":"2024-09-13T18:26:36.383772+0000","last_peered":"2024-09-13T18:26:36.383772+0000","last_clean":"2024-09-13T18:26:36.383772+0000","last_became_active":"2024-09-13T18:26:32.339405+0000","last_became_peered":"2024-09-13T18:26:32.339405+0000","last_unstale":"2024-09-13T18:26:36.383772+0000","last_undegraded":"2024-09-13T18:26:36.383772+0000","last_fullsized":"2024-09-13T18:26:36.383772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:26.081896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.385156+0000","last_change":"2024-09-13T18:26:31.968895+0000","last_active":"2024-09-13T18:26:36.385156+0000","last_peered":"2024-09-13T18:26:36.385156+0000","last_clean":"2024-09-13T18:26:36.385156+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:26:36.385156+0000","last_undegraded":"2024-09-13T18:26:36.385156+0000","last_fullsized":"2024-09-13T18:26:36.385156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:15.525097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767834+0000","last_change":"2024-09-13T18:26:36.406484+0000","last_active":"2024-09-13T18:26:36.767834+0000","last_peered":"2024-09-13T18:26:36.767834+0000","last_clean":"2024-09-13T18:26:36.767834+0000","last_became_active":"2024-09-13T18:26:36.406161+0000","last_became_peered":"2024-09-13T18:26:36.406161+0000","last_unstale":"2024-09-13T18:26:36.767834+0000","last_undegraded":"2024-09-13T18:26:36.767834+0000","last_fullsized":"2024-09-13T18:26:36.767834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:53.330919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391435+0000","last_change":"2024-09-13T18:26:32.360406+0000","last_active":"2024-09-13T18:26:36.391435+0000","last_peered":"2024-09-13T18:26:36.391435+0000","last_clean":"2024-09-13T18:26:36.391435+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:26:36.391435+0000","last_undegraded":"2024-09-13T18:26:36.391435+0000","last_fullsized":"2024-09-13T18:26:36.391435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:37.417707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404700+0000","last_change":"2024-09-13T18:26:31.976872+0000","last_active":"2024-09-13T18:26:36.404700+0000","last_peered":"2024-09-13T18:26:36.404700+0000","last_clean":"2024-09-13T18:26:36.404700+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:26:36.404700+0000","last_undegraded":"2024-09-13T18:26:36.404700+0000","last_fullsized":"2024-09-13T18:26:36.404700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:26.163029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408244+0000","last_change":"2024-09-13T18:26:36.405722+0000","last_active":"2024-09-13T18:26:36.408244+0000","last_peered":"2024-09-13T18:26:36.408244+0000","last_clean":"2024-09-13T18:26:36.408244+0000","last_became_active":"2024-09-13T18:26:36.405310+0000","last_became_peered":"2024-09-13T18:26:36.405310+0000","last_unstale":"2024-09-13T18:26:36.408244+0000","last_undegraded":"2024-09-13T18:26:36.408244+0000","last_fullsized":"2024-09-13T18:26:36.408244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:09.815655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.389080+0000","last_change":"2024-09-13T18:26:32.343915+0000","last_active":"2024-09-13T18:26:36.389080+0000","last_peered":"2024-09-13T18:26:36.389080+0000","last_clean":"2024-09-13T18:26:36.389080+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:26:36.389080+0000","last_undegraded":"2024-09-13T18:26:36.389080+0000","last_fullsized":"2024-09-13T18:26:36.389080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:19.357243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.349375+0000","last_change":"2024-09-13T18:26:31.979944+0000","last_active":"2024-09-13T18:26:37.349375+0000","last_peered":"2024-09-13T18:26:37.349375+0000","last_clean":"2024-09-13T18:26:37.349375+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:26:37.349375+0000","last_undegraded":"2024-09-13T18:26:37.349375+0000","last_fullsized":"2024-09-13T18:26:37.349375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:21.721921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.411958+0000","last_change":"2024-09-13T18:26:36.410028+0000","last_active":"2024-09-13T18:26:36.411958+0000","last_peered":"2024-09-13T18:26:36.411958+0000","last_clean":"2024-09-13T18:26:36.411958+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:26:36.411958+0000","last_undegraded":"2024-09-13T18:26:36.411958+0000","last_fullsized":"2024-09-13T18:26:36.411958+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:32.264341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387875+0000","last_change":"2024-09-13T18:26:32.358980+0000","last_active":"2024-09-13T18:26:36.387875+0000","last_peered":"2024-09-13T18:26:36.387875+0000","last_clean":"2024-09-13T18:26:36.387875+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:26:36.387875+0000","last_undegraded":"2024-09-13T18:26:36.387875+0000","last_fullsized":"2024-09-13T18:26:36.387875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:08.748139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769462+0000","last_change":"2024-09-13T18:26:31.968972+0000","last_active":"2024-09-13T18:26:36.769462+0000","last_peered":"2024-09-13T18:26:36.769462+0000","last_clean":"2024-09-13T18:26:36.769462+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:26:36.769462+0000","last_undegraded":"2024-09-13T18:26:36.769462+0000","last_fullsized":"2024-09-13T18:26:36.769462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:56.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768011+0000","last_change":"2024-09-13T18:26:36.407034+0000","last_active":"2024-09-13T18:26:36.768011+0000","last_peered":"2024-09-13T18:26:36.768011+0000","last_clean":"2024-09-13T18:26:36.768011+0000","last_became_active":"2024-09-13T18:26:36.406473+0000","last_became_peered":"2024-09-13T18:26:36.406473+0000","last_unstale":"2024-09-13T18:26:36.768011+0000","last_undegraded":"2024-09-13T18:26:36.768011+0000","last_fullsized":"2024-09-13T18:26:36.768011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:53.815901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386420+0000","last_change":"2024-09-13T18:26:32.342823+0000","last_active":"2024-09-13T18:26:36.386420+0000","last_peered":"2024-09-13T18:26:36.386420+0000","last_clean":"2024-09-13T18:26:36.386420+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:26:36.386420+0000","last_undegraded":"2024-09-13T18:26:36.386420+0000","last_fullsized":"2024-09-13T18:26:36.386420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:28.051252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407752+0000","last_change":"2024-09-13T18:26:31.978976+0000","last_active":"2024-09-13T18:26:36.407752+0000","last_peered":"2024-09-13T18:26:36.407752+0000","last_clean":"2024-09-13T18:26:36.407752+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:26:36.407752+0000","last_undegraded":"2024-09-13T18:26:36.407752+0000","last_fullsized":"2024-09-13T18:26:36.407752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:11.279557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103455+0000","last_change":"2024-09-13T18:26:36.408844+0000","last_active":"2024-09-13T18:26:37.103455+0000","last_peered":"2024-09-13T18:26:37.103455+0000","last_clean":"2024-09-13T18:26:37.103455+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:26:37.103455+0000","last_undegraded":"2024-09-13T18:26:37.103455+0000","last_fullsized":"2024-09-13T18:26:37.103455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:06.969342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393043+0000","last_change":"2024-09-13T18:26:32.355038+0000","last_active":"2024-09-13T18:26:36.393043+0000","last_peered":"2024-09-13T18:26:36.393043+0000","last_clean":"2024-09-13T18:26:36.393043+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:26:36.393043+0000","last_undegraded":"2024-09-13T18:26:36.393043+0000","last_fullsized":"2024-09-13T18:26:36.393043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.159231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386562+0000","last_change":"2024-09-13T18:26:31.972931+0000","last_active":"2024-09-13T18:26:36.386562+0000","last_peered":"2024-09-13T18:26:36.386562+0000","last_clean":"2024-09-13T18:26:36.386562+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:26:36.386562+0000","last_undegraded":"2024-09-13T18:26:36.386562+0000","last_fullsized":"2024-09-13T18:26:36.386562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:05.646924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766310+0000","last_change":"2024-09-13T18:26:36.398876+0000","last_active":"2024-09-13T18:26:36.766310+0000","last_peered":"2024-09-13T18:26:36.766310+0000","last_clean":"2024-09-13T18:26:36.766310+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:26:36.766310+0000","last_undegraded":"2024-09-13T18:26:36.766310+0000","last_fullsized":"2024-09-13T18:26:36.766310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:56.918239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392625+0000","last_change":"2024-09-13T18:26:32.353111+0000","last_active":"2024-09-13T18:26:36.392625+0000","last_peered":"2024-09-13T18:26:36.392625+0000","last_clean":"2024-09-13T18:26:36.392625+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:26:36.392625+0000","last_undegraded":"2024-09-13T18:26:36.392625+0000","last_fullsized":"2024-09-13T18:26:36.392625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:43.927936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391864+0000","last_change":"2024-09-13T18:26:31.976075+0000","last_active":"2024-09-13T18:26:36.391864+0000","last_peered":"2024-09-13T18:26:36.391864+0000","last_clean":"2024-09-13T18:26:36.391864+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:26:36.391864+0000","last_undegraded":"2024-09-13T18:26:36.391864+0000","last_fullsized":"2024-09-13T18:26:36.391864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:03.693518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766877+0000","last_change":"2024-09-13T18:26:36.401777+0000","last_active":"2024-09-13T18:26:36.766877+0000","last_peered":"2024-09-13T18:26:36.766877+0000","last_clean":"2024-09-13T18:26:36.766877+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:26:36.766877+0000","last_undegraded":"2024-09-13T18:26:36.766877+0000","last_fullsized":"2024-09-13T18:26:36.766877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:43.440284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384991+0000","last_change":"2024-09-13T18:26:32.343791+0000","last_active":"2024-09-13T18:26:36.384991+0000","last_peered":"2024-09-13T18:26:36.384991+0000","last_clean":"2024-09-13T18:26:36.384991+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:26:36.384991+0000","last_undegraded":"2024-09-13T18:26:36.384991+0000","last_fullsized":"2024-09-13T18:26:36.384991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:36.296571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.375229+0000","last_change":"2024-09-13T18:26:31.970786+0000","last_active":"2024-09-13T18:26:36.375229+0000","last_peered":"2024-09-13T18:26:36.375229+0000","last_clean":"2024-09-13T18:26:36.375229+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:26:36.375229+0000","last_undegraded":"2024-09-13T18:26:36.375229+0000","last_fullsized":"2024-09-13T18:26:36.375229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:54.584009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767929+0000","last_change":"2024-09-13T18:26:36.401529+0000","last_active":"2024-09-13T18:26:36.767929+0000","last_peered":"2024-09-13T18:26:36.767929+0000","last_clean":"2024-09-13T18:26:36.767929+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:26:36.767929+0000","last_undegraded":"2024-09-13T18:26:36.767929+0000","last_fullsized":"2024-09-13T18:26:36.767929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:47.318347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.382662+0000","last_change":"2024-09-13T18:26:32.350470+0000","last_active":"2024-09-13T18:26:36.382662+0000","last_peered":"2024-09-13T18:26:36.382662+0000","last_clean":"2024-09-13T18:26:36.382662+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:26:36.382662+0000","last_undegraded":"2024-09-13T18:26:36.382662+0000","last_fullsized":"2024-09-13T18:26:36.382662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:55.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404763+0000","last_change":"2024-09-13T18:26:31.972587+0000","last_active":"2024-09-13T18:26:36.404763+0000","last_peered":"2024-09-13T18:26:36.404763+0000","last_clean":"2024-09-13T18:26:36.404763+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:26:36.404763+0000","last_undegraded":"2024-09-13T18:26:36.404763+0000","last_fullsized":"2024-09-13T18:26:36.404763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:27.547763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103735+0000","last_change":"2024-09-13T18:26:36.405086+0000","last_active":"2024-09-13T18:26:37.103735+0000","last_peered":"2024-09-13T18:26:37.103735+0000","last_clean":"2024-09-13T18:26:37.103735+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:26:37.103735+0000","last_undegraded":"2024-09-13T18:26:37.103735+0000","last_fullsized":"2024-09-13T18:26:37.103735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:02:51.694634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388036+0000","last_change":"2024-09-13T18:26:32.350510+0000","last_active":"2024-09-13T18:26:36.388036+0000","last_peered":"2024-09-13T18:26:36.388036+0000","last_clean":"2024-09-13T18:26:36.388036+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:26:36.388036+0000","last_undegraded":"2024-09-13T18:26:36.388036+0000","last_fullsized":"2024-09-13T18:26:36.388036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:40.850255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391579+0000","last_change":"2024-09-13T18:26:31.978355+0000","last_active":"2024-09-13T18:26:36.391579+0000","last_peered":"2024-09-13T18:26:36.391579+0000","last_clean":"2024-09-13T18:26:36.391579+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:26:36.391579+0000","last_undegraded":"2024-09-13T18:26:36.391579+0000","last_fullsized":"2024-09-13T18:26:36.391579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:12.946518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404778+0000","last_change":"2024-09-13T18:26:26.931403+0000","last_active":"2024-09-13T18:26:36.404778+0000","last_peered":"2024-09-13T18:26:36.404778+0000","last_clean":"2024-09-13T18:26:36.404778+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:26:36.404778+0000","last_undegraded":"2024-09-13T18:26:36.404778+0000","last_fullsized":"2024-09-13T18:26:36.404778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_clean_scrub_stamp":"2024-09-13T18:26:24.880028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:30.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897926+0000","last_change":"2024-09-13T18:26:36.409827+0000","last_active":"2024-09-13T18:26:36.897926+0000","last_peered":"2024-09-13T18:26:36.897926+0000","last_clean":"2024-09-13T18:26:36.897926+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:26:36.897926+0000","last_undegraded":"2024-09-13T18:26:36.897926+0000","last_fullsized":"2024-09-13T18:26:36.897926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:50.482400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405746+0000","last_change":"2024-09-13T18:26:32.356850+0000","last_active":"2024-09-13T18:26:36.405746+0000","last_peered":"2024-09-13T18:26:36.405746+0000","last_clean":"2024-09-13T18:26:36.405746+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:26:36.405746+0000","last_undegraded":"2024-09-13T18:26:36.405746+0000","last_fullsized":"2024-09-13T18:26:36.405746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:38.710828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.376787+0000","last_change":"2024-09-13T18:26:31.975942+0000","last_active":"2024-09-13T18:26:36.376787+0000","last_peered":"2024-09-13T18:26:36.376787+0000","last_clean":"2024-09-13T18:26:36.376787+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:26:36.376787+0000","last_undegraded":"2024-09-13T18:26:36.376787+0000","last_fullsized":"2024-09-13T18:26:36.376787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:05.840582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897805+0000","last_change":"2024-09-13T18:26:36.409802+0000","last_active":"2024-09-13T18:26:36.897805+0000","last_peered":"2024-09-13T18:26:36.897805+0000","last_clean":"2024-09-13T18:26:36.897805+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:26:36.897805+0000","last_undegraded":"2024-09-13T18:26:36.897805+0000","last_fullsized":"2024-09-13T18:26:36.897805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:38.178151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394309+0000","last_change":"2024-09-13T18:26:32.355181+0000","last_active":"2024-09-13T18:26:36.394309+0000","last_peered":"2024-09-13T18:26:36.394309+0000","last_clean":"2024-09-13T18:26:36.394309+0000","last_became_active":"2024-09-13T18:26:32.353043+0000","last_became_peered":"2024-09-13T18:26:32.353043+0000","last_unstale":"2024-09-13T18:26:36.394309+0000","last_undegraded":"2024-09-13T18:26:36.394309+0000","last_fullsized":"2024-09-13T18:26:36.394309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:04:17.000915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767488+0000","last_change":"2024-09-13T18:26:31.966618+0000","last_active":"2024-09-13T18:26:36.767488+0000","last_peered":"2024-09-13T18:26:36.767488+0000","last_clean":"2024-09-13T18:26:36.767488+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:26:36.767488+0000","last_undegraded":"2024-09-13T18:26:36.767488+0000","last_fullsized":"2024-09-13T18:26:36.767488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:54:26.422611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103645+0000","last_change":"2024-09-13T18:26:36.394344+0000","last_active":"2024-09-13T18:26:37.103645+0000","last_peered":"2024-09-13T18:26:37.103645+0000","last_clean":"2024-09-13T18:26:37.103645+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:26:37.103645+0000","last_undegraded":"2024-09-13T18:26:37.103645+0000","last_fullsized":"2024-09-13T18:26:37.103645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:18.425816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405784+0000","last_change":"2024-09-13T18:26:32.365655+0000","last_active":"2024-09-13T18:26:36.405784+0000","last_peered":"2024-09-13T18:26:36.405784+0000","last_clean":"2024-09-13T18:26:36.405784+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:26:36.405784+0000","last_undegraded":"2024-09-13T18:26:36.405784+0000","last_fullsized":"2024-09-13T18:26:36.405784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:40.288128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343912+0000","last_change":"2024-09-13T18:26:31.970652+0000","last_active":"2024-09-13T18:26:37.343912+0000","last_peered":"2024-09-13T18:26:37.343912+0000","last_clean":"2024-09-13T18:26:37.343912+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:26:37.343912+0000","last_undegraded":"2024-09-13T18:26:37.343912+0000","last_fullsized":"2024-09-13T18:26:37.343912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:11.169661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.770631+0000","last_change":"2024-09-13T18:26:36.410280+0000","last_active":"2024-09-13T18:26:36.770631+0000","last_peered":"2024-09-13T18:26:36.770631+0000","last_clean":"2024-09-13T18:26:36.770631+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:26:36.770631+0000","last_undegraded":"2024-09-13T18:26:36.770631+0000","last_fullsized":"2024-09-13T18:26:36.770631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:01.395290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405642+0000","last_change":"2024-09-13T18:26:32.361575+0000","last_active":"2024-09-13T18:26:36.405642+0000","last_peered":"2024-09-13T18:26:36.405642+0000","last_clean":"2024-09-13T18:26:36.405642+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:26:36.405642+0000","last_undegraded":"2024-09-13T18:26:36.405642+0000","last_fullsized":"2024-09-13T18:26:36.405642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:08.818877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.239195+0000","last_change":"2024-09-13T18:26:31.977029+0000","last_active":"2024-09-13T18:26:37.239195+0000","last_peered":"2024-09-13T18:26:37.239195+0000","last_clean":"2024-09-13T18:26:37.239195+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:26:37.239195+0000","last_undegraded":"2024-09-13T18:26:37.239195+0000","last_fullsized":"2024-09-13T18:26:37.239195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:13.701689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407565+0000","last_change":"2024-09-13T18:26:36.403479+0000","last_active":"2024-09-13T18:26:36.407565+0000","last_peered":"2024-09-13T18:26:36.407565+0000","last_clean":"2024-09-13T18:26:36.407565+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:26:36.407565+0000","last_undegraded":"2024-09-13T18:26:36.407565+0000","last_fullsized":"2024-09-13T18:26:36.407565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:51.095852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404976+0000","last_change":"2024-09-13T18:26:32.365143+0000","last_active":"2024-09-13T18:26:36.404976+0000","last_peered":"2024-09-13T18:26:36.404976+0000","last_clean":"2024-09-13T18:26:36.404976+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:26:36.404976+0000","last_undegraded":"2024-09-13T18:26:36.404976+0000","last_fullsized":"2024-09-13T18:26:36.404976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:46.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.342638+0000","last_change":"2024-09-13T18:26:31.965802+0000","last_active":"2024-09-13T18:26:37.342638+0000","last_peered":"2024-09-13T18:26:37.342638+0000","last_clean":"2024-09-13T18:26:37.342638+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:26:37.342638+0000","last_undegraded":"2024-09-13T18:26:37.342638+0000","last_fullsized":"2024-09-13T18:26:37.342638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:12.759181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103768+0000","last_change":"2024-09-13T18:26:36.399235+0000","last_active":"2024-09-13T18:26:37.103768+0000","last_peered":"2024-09-13T18:26:37.103768+0000","last_clean":"2024-09-13T18:26:37.103768+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:26:37.103768+0000","last_undegraded":"2024-09-13T18:26:37.103768+0000","last_fullsized":"2024-09-13T18:26:37.103768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:13:54.711083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897760+0000","last_change":"2024-09-13T18:26:32.357403+0000","last_active":"2024-09-13T18:26:36.897760+0000","last_peered":"2024-09-13T18:26:36.897760+0000","last_clean":"2024-09-13T18:26:36.897760+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:26:36.897760+0000","last_undegraded":"2024-09-13T18:26:36.897760+0000","last_fullsized":"2024-09-13T18:26:36.897760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:12.600397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383205+0000","last_change":"2024-09-13T18:26:31.976148+0000","last_active":"2024-09-13T18:26:36.383205+0000","last_peered":"2024-09-13T18:26:36.383205+0000","last_clean":"2024-09-13T18:26:36.383205+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:26:36.383205+0000","last_undegraded":"2024-09-13T18:26:36.383205+0000","last_fullsized":"2024-09-13T18:26:36.383205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:38.023636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769582+0000","last_change":"2024-09-13T18:26:36.406177+0000","last_active":"2024-09-13T18:26:36.769582+0000","last_peered":"2024-09-13T18:26:36.769582+0000","last_clean":"2024-09-13T18:26:36.769582+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:26:36.769582+0000","last_undegraded":"2024-09-13T18:26:36.769582+0000","last_fullsized":"2024-09-13T18:26:36.769582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:00.309995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405459+0000","last_change":"2024-09-13T18:26:32.361726+0000","last_active":"2024-09-13T18:26:36.405459+0000","last_peered":"2024-09-13T18:26:36.405459+0000","last_clean":"2024-09-13T18:26:36.405459+0000","last_became_active":"2024-09-13T18:26:32.361432+0000","last_became_peered":"2024-09-13T18:26:32.361432+0000","last_unstale":"2024-09-13T18:26:36.405459+0000","last_undegraded":"2024-09-13T18:26:36.405459+0000","last_fullsized":"2024-09-13T18:26:36.405459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:42.021515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391337+0000","last_change":"2024-09-13T18:26:31.982114+0000","last_active":"2024-09-13T18:26:36.391337+0000","last_peered":"2024-09-13T18:26:36.391337+0000","last_clean":"2024-09-13T18:26:36.391337+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:26:36.391337+0000","last_undegraded":"2024-09-13T18:26:36.391337+0000","last_fullsized":"2024-09-13T18:26:36.391337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.266528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769049+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:26:36.769049+0000","last_peered":"2024-09-13T18:26:36.769049+0000","last_clean":"2024-09-13T18:26:36.769049+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:26:36.769049+0000","last_undegraded":"2024-09-13T18:26:36.769049+0000","last_fullsized":"2024-09-13T18:26:36.769049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:58.853971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388651+0000","last_change":"2024-09-13T18:26:32.351145+0000","last_active":"2024-09-13T18:26:36.388651+0000","last_peered":"2024-09-13T18:26:36.388651+0000","last_clean":"2024-09-13T18:26:36.388651+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:26:36.388651+0000","last_undegraded":"2024-09-13T18:26:36.388651+0000","last_fullsized":"2024-09-13T18:26:36.388651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:57.002435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.344611+0000","last_change":"2024-09-13T18:26:31.961202+0000","last_active":"2024-09-13T18:26:37.344611+0000","last_peered":"2024-09-13T18:26:37.344611+0000","last_clean":"2024-09-13T18:26:37.344611+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:26:37.344611+0000","last_undegraded":"2024-09-13T18:26:37.344611+0000","last_fullsized":"2024-09-13T18:26:37.344611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:27.340127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.772209+0000","last_change":"2024-09-13T18:26:36.410014+0000","last_active":"2024-09-13T18:26:36.772209+0000","last_peered":"2024-09-13T18:26:36.772209+0000","last_clean":"2024-09-13T18:26:36.772209+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:26:36.772209+0000","last_undegraded":"2024-09-13T18:26:36.772209+0000","last_fullsized":"2024-09-13T18:26:36.772209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:24.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405669+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:26:36.405669+0000","last_peered":"2024-09-13T18:26:36.405669+0000","last_clean":"2024-09-13T18:26:36.405669+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:26:36.405669+0000","last_undegraded":"2024-09-13T18:26:36.405669+0000","last_fullsized":"2024-09-13T18:26:36.405669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:40.102815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:45.381784+0000","last_change":"2024-09-13T18:26:31.974110+0000","last_active":"2024-09-13T18:26:45.381784+0000","last_peered":"2024-09-13T18:26:45.381784+0000","last_clean":"2024-09-13T18:26:45.381784+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:26:45.381784+0000","last_undegraded":"2024-09-13T18:26:45.381784+0000","last_fullsized":"2024-09-13T18:26:45.381784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:56.058561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766576+0000","last_change":"2024-09-13T18:26:36.394080+0000","last_active":"2024-09-13T18:26:36.766576+0000","last_peered":"2024-09-13T18:26:36.766576+0000","last_clean":"2024-09-13T18:26:36.766576+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:26:36.766576+0000","last_undegraded":"2024-09-13T18:26:36.766576+0000","last_fullsized":"2024-09-13T18:26:36.766576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:47.121814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392195+0000","last_change":"2024-09-13T18:26:32.362873+0000","last_active":"2024-09-13T18:26:36.392195+0000","last_peered":"2024-09-13T18:26:36.392195+0000","last_clean":"2024-09-13T18:26:36.392195+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:26:36.392195+0000","last_undegraded":"2024-09-13T18:26:36.392195+0000","last_fullsized":"2024-09-13T18:26:36.392195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:37.598189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.344576+0000","last_change":"2024-09-13T18:26:31.976401+0000","last_active":"2024-09-13T18:26:37.344576+0000","last_peered":"2024-09-13T18:26:37.344576+0000","last_clean":"2024-09-13T18:26:37.344576+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:26:37.344576+0000","last_undegraded":"2024-09-13T18:26:37.344576+0000","last_fullsized":"2024-09-13T18:26:37.344576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:26.434292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408317+0000","last_change":"2024-09-13T18:26:36.406228+0000","last_active":"2024-09-13T18:26:36.408317+0000","last_peered":"2024-09-13T18:26:36.408317+0000","last_clean":"2024-09-13T18:26:36.408317+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:26:36.408317+0000","last_undegraded":"2024-09-13T18:26:36.408317+0000","last_fullsized":"2024-09-13T18:26:36.408317+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:49.198397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388733+0000","last_change":"2024-09-13T18:26:32.350841+0000","last_active":"2024-09-13T18:26:36.388733+0000","last_peered":"2024-09-13T18:26:36.388733+0000","last_clean":"2024-09-13T18:26:36.388733+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:26:36.388733+0000","last_undegraded":"2024-09-13T18:26:36.388733+0000","last_fullsized":"2024-09-13T18:26:36.388733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:05.137882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391761+0000","last_change":"2024-09-13T18:26:31.978214+0000","last_active":"2024-09-13T18:26:36.391761+0000","last_peered":"2024-09-13T18:26:36.391761+0000","last_clean":"2024-09-13T18:26:36.391761+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:26:36.391761+0000","last_undegraded":"2024-09-13T18:26:36.391761+0000","last_fullsized":"2024-09-13T18:26:36.391761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:17.381375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768498+0000","last_change":"2024-09-13T18:26:36.406090+0000","last_active":"2024-09-13T18:26:36.768498+0000","last_peered":"2024-09-13T18:26:36.768498+0000","last_clean":"2024-09-13T18:26:36.768498+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:26:36.768498+0000","last_undegraded":"2024-09-13T18:26:36.768498+0000","last_fullsized":"2024-09-13T18:26:36.768498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:09.865147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391759+0000","last_change":"2024-09-13T18:26:32.357701+0000","last_active":"2024-09-13T18:26:36.391759+0000","last_peered":"2024-09-13T18:26:36.391759+0000","last_clean":"2024-09-13T18:26:36.391759+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:26:36.391759+0000","last_undegraded":"2024-09-13T18:26:36.391759+0000","last_fullsized":"2024-09-13T18:26:36.391759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:47.587683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384903+0000","last_change":"2024-09-13T18:26:31.980133+0000","last_active":"2024-09-13T18:26:36.384903+0000","last_peered":"2024-09-13T18:26:36.384903+0000","last_clean":"2024-09-13T18:26:36.384903+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:26:36.384903+0000","last_undegraded":"2024-09-13T18:26:36.384903+0000","last_fullsized":"2024-09-13T18:26:36.384903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:05.354335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768120+0000","last_change":"2024-09-13T18:26:36.406553+0000","last_active":"2024-09-13T18:26:36.768120+0000","last_peered":"2024-09-13T18:26:36.768120+0000","last_clean":"2024-09-13T18:26:36.768120+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:26:36.768120+0000","last_undegraded":"2024-09-13T18:26:36.768120+0000","last_fullsized":"2024-09-13T18:26:36.768120+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:02.980826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103481+0000","last_change":"2024-09-13T18:26:36.406986+0000","last_active":"2024-09-13T18:26:37.103481+0000","last_peered":"2024-09-13T18:26:37.103481+0000","last_clean":"2024-09-13T18:26:37.103481+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:26:37.103481+0000","last_undegraded":"2024-09-13T18:26:37.103481+0000","last_fullsized":"2024-09-13T18:26:37.103481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:53.984231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384974+0000","last_change":"2024-09-13T18:26:32.349081+0000","last_active":"2024-09-13T18:26:36.384974+0000","last_peered":"2024-09-13T18:26:36.384974+0000","last_clean":"2024-09-13T18:26:36.384974+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:26:36.384974+0000","last_undegraded":"2024-09-13T18:26:36.384974+0000","last_fullsized":"2024-09-13T18:26:36.384974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:47.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391124+0000","last_change":"2024-09-13T18:26:31.968944+0000","last_active":"2024-09-13T18:26:36.391124+0000","last_peered":"2024-09-13T18:26:36.391124+0000","last_clean":"2024-09-13T18:26:36.391124+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:26:36.391124+0000","last_undegraded":"2024-09-13T18:26:36.391124+0000","last_fullsized":"2024-09-13T18:26:36.391124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:29.363312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.898021+0000","last_change":"2024-09-13T18:26:36.396418+0000","last_active":"2024-09-13T18:26:36.898021+0000","last_peered":"2024-09-13T18:26:36.898021+0000","last_clean":"2024-09-13T18:26:36.898021+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:26:36.898021+0000","last_undegraded":"2024-09-13T18:26:36.898021+0000","last_fullsized":"2024-09-13T18:26:36.898021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:03.252773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383753+0000","last_change":"2024-09-13T18:26:32.351369+0000","last_active":"2024-09-13T18:26:36.383753+0000","last_peered":"2024-09-13T18:26:36.383753+0000","last_clean":"2024-09-13T18:26:36.383753+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:26:36.383753+0000","last_undegraded":"2024-09-13T18:26:36.383753+0000","last_fullsized":"2024-09-13T18:26:36.383753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:10.892347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387713+0000","last_change":"2024-09-13T18:26:31.972048+0000","last_active":"2024-09-13T18:26:36.387713+0000","last_peered":"2024-09-13T18:26:36.387713+0000","last_clean":"2024-09-13T18:26:36.387713+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:26:36.387713+0000","last_undegraded":"2024-09-13T18:26:36.387713+0000","last_fullsized":"2024-09-13T18:26:36.387713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:28.430119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343265+0000","last_change":"2024-09-13T18:26:31.975310+0000","last_active":"2024-09-13T18:26:37.343265+0000","last_peered":"2024-09-13T18:26:37.343265+0000","last_clean":"2024-09-13T18:26:37.343265+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:26:37.343265+0000","last_undegraded":"2024-09-13T18:26:37.343265+0000","last_fullsized":"2024-09-13T18:26:37.343265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:52.915310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388817+0000","last_change":"2024-09-13T18:26:32.345825+0000","last_active":"2024-09-13T18:26:36.388817+0000","last_peered":"2024-09-13T18:26:36.388817+0000","last_clean":"2024-09-13T18:26:36.388817+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:26:36.388817+0000","last_undegraded":"2024-09-13T18:26:36.388817+0000","last_fullsized":"2024-09-13T18:26:36.388817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:25.200489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.395253+0000","last_change":"2024-09-13T18:26:36.391520+0000","last_active":"2024-09-13T18:26:36.395253+0000","last_peered":"2024-09-13T18:26:36.395253+0000","last_clean":"2024-09-13T18:26:36.395253+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:26:36.395253+0000","last_undegraded":"2024-09-13T18:26:36.395253+0000","last_fullsized":"2024-09-13T18:26:36.395253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:42.638713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768901+0000","last_change":"2024-09-13T18:26:31.976603+0000","last_active":"2024-09-13T18:26:36.768901+0000","last_peered":"2024-09-13T18:26:36.768901+0000","last_clean":"2024-09-13T18:26:36.768901+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:26:36.768901+0000","last_undegraded":"2024-09-13T18:26:36.768901+0000","last_fullsized":"2024-09-13T18:26:36.768901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:03.965376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387960+0000","last_change":"2024-09-13T18:26:32.352982+0000","last_active":"2024-09-13T18:26:36.387960+0000","last_peered":"2024-09-13T18:26:36.387960+0000","last_clean":"2024-09-13T18:26:36.387960+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:26:36.387960+0000","last_undegraded":"2024-09-13T18:26:36.387960+0000","last_fullsized":"2024-09-13T18:26:36.387960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:55.670360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769131+0000","last_change":"2024-09-13T18:26:36.403817+0000","last_active":"2024-09-13T18:26:36.769131+0000","last_peered":"2024-09-13T18:26:36.769131+0000","last_clean":"2024-09-13T18:26:36.769131+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:26:36.769131+0000","last_undegraded":"2024-09-13T18:26:36.769131+0000","last_fullsized":"2024-09-13T18:26:36.769131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:34.892901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897728+0000","last_change":"2024-09-13T18:26:32.357856+0000","last_active":"2024-09-13T18:26:36.897728+0000","last_peered":"2024-09-13T18:26:36.897728+0000","last_clean":"2024-09-13T18:26:36.897728+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:26:36.897728+0000","last_undegraded":"2024-09-13T18:26:36.897728+0000","last_fullsized":"2024-09-13T18:26:36.897728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:26.316214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.375208+0000","last_change":"2024-09-13T18:26:32.350966+0000","last_active":"2024-09-13T18:26:36.375208+0000","last_peered":"2024-09-13T18:26:36.375208+0000","last_clean":"2024-09-13T18:26:36.375208+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:26:36.375208+0000","last_undegraded":"2024-09-13T18:26:36.375208+0000","last_fullsized":"2024-09-13T18:26:36.375208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:52.632112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391246+0000","last_change":"2024-09-13T18:26:32.355806+0000","last_active":"2024-09-13T18:26:36.391246+0000","last_peered":"2024-09-13T18:26:36.391246+0000","last_clean":"2024-09-13T18:26:36.391246+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:26:36.391246+0000","last_undegraded":"2024-09-13T18:26:36.391246+0000","last_fullsized":"2024-09-13T18:26:36.391246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.988231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388740+0000","last_change":"2024-09-13T18:26:32.340965+0000","last_active":"2024-09-13T18:26:36.388740+0000","last_peered":"2024-09-13T18:26:36.388740+0000","last_clean":"2024-09-13T18:26:36.388740+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:26:36.388740+0000","last_undegraded":"2024-09-13T18:26:36.388740+0000","last_fullsized":"2024-09-13T18:26:36.388740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:01:04.716919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384429+0000","last_change":"2024-09-13T18:26:32.349625+0000","last_active":"2024-09-13T18:26:36.384429+0000","last_peered":"2024-09-13T18:26:36.384429+0000","last_clean":"2024-09-13T18:26:36.384429+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:26:36.384429+0000","last_undegraded":"2024-09-13T18:26:36.384429+0000","last_fullsized":"2024-09-13T18:26:36.384429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.519754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386397+0000","last_change":"2024-09-13T18:26:32.349170+0000","last_active":"2024-09-13T18:26:36.386397+0000","last_peered":"2024-09-13T18:26:36.386397+0000","last_clean":"2024-09-13T18:26:36.386397+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:26:36.386397+0000","last_undegraded":"2024-09-13T18:26:36.386397+0000","last_fullsized":"2024-09-13T18:26:36.386397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:43.942732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897998+0000","last_change":"2024-09-13T18:26:32.364786+0000","last_active":"2024-09-13T18:26:36.897998+0000","last_peered":"2024-09-13T18:26:36.897998+0000","last_clean":"2024-09-13T18:26:36.897998+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:26:36.897998+0000","last_undegraded":"2024-09-13T18:26:36.897998+0000","last_fullsized":"2024-09-13T18:26:36.897998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:22.949875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387927+0000","last_change":"2024-09-13T18:26:32.347541+0000","last_active":"2024-09-13T18:26:36.387927+0000","last_peered":"2024-09-13T18:26:36.387927+0000","last_clean":"2024-09-13T18:26:36.387927+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:26:36.387927+0000","last_undegraded":"2024-09-13T18:26:36.387927+0000","last_fullsized":"2024-09-13T18:26:36.387927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:58.144128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390892+0000","last_change":"2024-09-13T18:26:32.353359+0000","last_active":"2024-09-13T18:26:36.390892+0000","last_peered":"2024-09-13T18:26:36.390892+0000","last_clean":"2024-09-13T18:26:36.390892+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:26:36.390892+0000","last_undegraded":"2024-09-13T18:26:36.390892+0000","last_fullsized":"2024-09-13T18:26:36.390892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:09.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.380537+0000","last_change":"2024-09-13T18:26:32.349566+0000","last_active":"2024-09-13T18:26:36.380537+0000","last_peered":"2024-09-13T18:26:36.380537+0000","last_clean":"2024-09-13T18:26:36.380537+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:26:36.380537+0000","last_undegraded":"2024-09-13T18:26:36.380537+0000","last_fullsized":"2024-09-13T18:26:36.380537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:22.341899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391168+0000","last_change":"2024-09-13T18:26:32.358640+0000","last_active":"2024-09-13T18:26:36.391168+0000","last_peered":"2024-09-13T18:26:36.391168+0000","last_clean":"2024-09-13T18:26:36.391168+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:26:36.391168+0000","last_undegraded":"2024-09-13T18:26:36.391168+0000","last_fullsized":"2024-09-13T18:26:36.391168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:40.174754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388633+0000","last_change":"2024-09-13T18:26:32.353675+0000","last_active":"2024-09-13T18:26:36.388633+0000","last_peered":"2024-09-13T18:26:36.388633+0000","last_clean":"2024-09-13T18:26:36.388633+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:26:36.388633+0000","last_undegraded":"2024-09-13T18:26:36.388633+0000","last_fullsized":"2024-09-13T18:26:36.388633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:58.501843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897877+0000","last_change":"2024-09-13T18:26:32.353223+0000","last_active":"2024-09-13T18:26:36.897877+0000","last_peered":"2024-09-13T18:26:36.897877+0000","last_clean":"2024-09-13T18:26:36.897877+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:26:36.897877+0000","last_undegraded":"2024-09-13T18:26:36.897877+0000","last_fullsized":"2024-09-13T18:26:36.897877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:45.898213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407736+0000","last_change":"2024-09-13T18:26:32.357354+0000","last_active":"2024-09-13T18:26:36.407736+0000","last_peered":"2024-09-13T18:26:36.407736+0000","last_clean":"2024-09-13T18:26:36.407736+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:26:36.407736+0000","last_undegraded":"2024-09-13T18:26:36.407736+0000","last_fullsized":"2024-09-13T18:26:36.407736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:19.185008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.376351+0000","last_change":"2024-09-13T18:26:32.348832+0000","last_active":"2024-09-13T18:26:36.376351+0000","last_peered":"2024-09-13T18:26:36.376351+0000","last_clean":"2024-09-13T18:26:36.376351+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:26:36.376351+0000","last_undegraded":"2024-09-13T18:26:36.376351+0000","last_fullsized":"2024-09-13T18:26:36.376351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:11:59.874789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393723+0000","last_change":"2024-09-13T18:26:32.356902+0000","last_active":"2024-09-13T18:26:36.393723+0000","last_peered":"2024-09-13T18:26:36.393723+0000","last_clean":"2024-09-13T18:26:36.393723+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:26:36.393723+0000","last_undegraded":"2024-09-13T18:26:36.393723+0000","last_fullsized":"2024-09-13T18:26:36.393723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:18.362772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390031+0000","last_change":"2024-09-13T18:26:32.347047+0000","last_active":"2024-09-13T18:26:36.390031+0000","last_peered":"2024-09-13T18:26:36.390031+0000","last_clean":"2024-09-13T18:26:36.390031+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:26:36.390031+0000","last_undegraded":"2024-09-13T18:26:36.390031+0000","last_fullsized":"2024-09-13T18:26:36.390031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:40.882789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.385066+0000","last_change":"2024-09-13T18:26:32.349651+0000","last_active":"2024-09-13T18:26:36.385066+0000","last_peered":"2024-09-13T18:26:36.385066+0000","last_clean":"2024-09-13T18:26:36.385066+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:26:36.385066+0000","last_undegraded":"2024-09-13T18:26:36.385066+0000","last_fullsized":"2024-09-13T18:26:36.385066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:33.356563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388522+0000","last_change":"2024-09-13T18:26:32.344607+0000","last_active":"2024-09-13T18:26:36.388522+0000","last_peered":"2024-09-13T18:26:36.388522+0000","last_clean":"2024-09-13T18:26:36.388522+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:26:36.388522+0000","last_undegraded":"2024-09-13T18:26:36.388522+0000","last_fullsized":"2024-09-13T18:26:36.388522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:31.658172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387737+0000","last_change":"2024-09-13T18:26:32.347717+0000","last_active":"2024-09-13T18:26:36.387737+0000","last_peered":"2024-09-13T18:26:36.387737+0000","last_clean":"2024-09-13T18:26:36.387737+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:26:36.387737+0000","last_undegraded":"2024-09-13T18:26:36.387737+0000","last_fullsized":"2024-09-13T18:26:36.387737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:47.187837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392104+0000","last_change":"2024-09-13T18:26:32.359506+0000","last_active":"2024-09-13T18:26:36.392104+0000","last_peered":"2024-09-13T18:26:36.392104+0000","last_clean":"2024-09-13T18:26:36.392104+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:26:36.392104+0000","last_undegraded":"2024-09-13T18:26:36.392104+0000","last_fullsized":"2024-09-13T18:26:36.392104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:39:57.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897951+0000","last_change":"2024-09-13T18:26:32.357729+0000","last_active":"2024-09-13T18:26:36.897951+0000","last_peered":"2024-09-13T18:26:36.897951+0000","last_clean":"2024-09-13T18:26:36.897951+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:26:36.897951+0000","last_undegraded":"2024-09-13T18:26:36.897951+0000","last_fullsized":"2024-09-13T18:26:36.897951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:05.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897974+0000","last_change":"2024-09-13T18:26:32.357980+0000","last_active":"2024-09-13T18:26:36.897974+0000","last_peered":"2024-09-13T18:26:36.897974+0000","last_clean":"2024-09-13T18:26:36.897974+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:26:36.897974+0000","last_undegraded":"2024-09-13T18:26:36.897974+0000","last_fullsized":"2024-09-13T18:26:36.897974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:00.858702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405699+0000","last_change":"2024-09-13T18:26:32.365638+0000","last_active":"2024-09-13T18:26:36.405699+0000","last_peered":"2024-09-13T18:26:36.405699+0000","last_clean":"2024-09-13T18:26:36.405699+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:26:36.405699+0000","last_undegraded":"2024-09-13T18:26:36.405699+0000","last_fullsized":"2024-09-13T18:26:36.405699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.065853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383235+0000","last_change":"2024-09-13T18:26:32.345496+0000","last_active":"2024-09-13T18:26:36.383235+0000","last_peered":"2024-09-13T18:26:36.383235+0000","last_clean":"2024-09-13T18:26:36.383235+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:26:36.383235+0000","last_undegraded":"2024-09-13T18:26:36.383235+0000","last_fullsized":"2024-09-13T18:26:36.383235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:22.447315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404895+0000","last_change":"2024-09-13T18:26:32.356901+0000","last_active":"2024-09-13T18:26:36.404895+0000","last_peered":"2024-09-13T18:26:36.404895+0000","last_clean":"2024-09-13T18:26:36.404895+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:26:36.404895+0000","last_undegraded":"2024-09-13T18:26:36.404895+0000","last_fullsized":"2024-09-13T18:26:36.404895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:03.311603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391082+0000","last_change":"2024-09-13T18:26:32.355203+0000","last_active":"2024-09-13T18:26:36.391082+0000","last_peered":"2024-09-13T18:26:36.391082+0000","last_clean":"2024-09-13T18:26:36.391082+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:26:36.391082+0000","last_undegraded":"2024-09-13T18:26:36.391082+0000","last_fullsized":"2024-09-13T18:26:36.391082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:52.769708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405315+0000","last_change":"2024-09-13T18:26:32.358190+0000","last_active":"2024-09-13T18:26:36.405315+0000","last_peered":"2024-09-13T18:26:36.405315+0000","last_clean":"2024-09-13T18:26:36.405315+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:26:36.405315+0000","last_undegraded":"2024-09-13T18:26:36.405315+0000","last_fullsized":"2024-09-13T18:26:36.405315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:35.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393258+0000","last_change":"2024-09-13T18:26:32.355121+0000","last_active":"2024-09-13T18:26:36.393258+0000","last_peered":"2024-09-13T18:26:36.393258+0000","last_clean":"2024-09-13T18:26:36.393258+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:26:36.393258+0000","last_undegraded":"2024-09-13T18:26:36.393258+0000","last_fullsized":"2024-09-13T18:26:36.393258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:48.427332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388810+0000","last_change":"2024-09-13T18:26:32.341497+0000","last_active":"2024-09-13T18:26:36.388810+0000","last_peered":"2024-09-13T18:26:36.388810+0000","last_clean":"2024-09-13T18:26:36.388810+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:26:36.388810+0000","last_undegraded":"2024-09-13T18:26:36.388810+0000","last_fullsized":"2024-09-13T18:26:36.388810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:15.510048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386627+0000","last_change":"2024-09-13T18:26:32.350106+0000","last_active":"2024-09-13T18:26:36.386627+0000","last_peered":"2024-09-13T18:26:36.386627+0000","last_clean":"2024-09-13T18:26:36.386627+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:26:36.386627+0000","last_undegraded":"2024-09-13T18:26:36.386627+0000","last_fullsized":"2024-09-13T18:26:36.386627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:12.387451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405597+0000","last_change":"2024-09-13T18:26:32.362196+0000","last_active":"2024-09-13T18:26:36.405597+0000","last_peered":"2024-09-13T18:26:36.405597+0000","last_clean":"2024-09-13T18:26:36.405597+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:26:36.405597+0000","last_undegraded":"2024-09-13T18:26:36.405597+0000","last_fullsized":"2024-09-13T18:26:36.405597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:14.946904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":694123,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":103299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":103173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":262144,"data_stored":103263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":103333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4239999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":694179,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.655}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:27:01.875 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T18:27:01.875 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T18:27:01.875 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T18:27:01.876 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T18:27:01.876 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T18:27:01.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b287c700 1 Processor -- start 2024-09-13T18:27:01.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b287c700 1 -- start start 2024-09-13T18:27:01.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b287c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7ac0c5fc0 0x7fa7ac0c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b287c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa7ac0c63a0 0x7fa7ac0c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b287c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7ac0cb380 con 0x7fa7ac06f5c0 2024-09-13T18:27:01.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b287c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ac0c9bd0 con 0x7fa7ac0c63a0 2024-09-13T18:27:01.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b287c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac0c8c70 con 0x7fa7ac0c5fc0 2024-09-13T18:27:01.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b207b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa7ac06f5c0 0x7fa7ac0cac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35416/0 (socket says 172.21.15.146:35416) 2024-09-13T18:27:01.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b207b700 1 -- 172.21.15.146:0/666405553 learned_addr learned my addr 172.21.15.146:0/666405553 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:01.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b1079700 1 --2- 172.21.15.146:0/666405553 >> v2:172.21.15.146:3300/0 conn(0x7fa7ac0c63a0 0x7fa7ac0c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b187a700 1 --2- 172.21.15.146:0/666405553 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7ac0c5fc0 0x7fa7ac0c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b0878700 1 -- 172.21.15.146:0/666405553 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567617856 0 0) 0x7fa7ac0cb380 con 0x7fa7ac06f5c0 2024-09-13T18:27:01.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b0878700 1 -- 172.21.15.146:0/666405553 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7980035d0 con 0x7fa7ac06f5c0 2024-09-13T18:27:01.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b1079700 1 -- 172.21.15.146:0/666405553 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7ac0c5fc0 msgr2=0x7fa7ac0c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:01.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b1079700 1 --2- 172.21.15.146:0/666405553 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7ac0c5fc0 0x7fa7ac0c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b1079700 1 -- 172.21.15.146:0/666405553 >> v1:172.21.15.146:6789/0 conn(0x7fa7ac06f5c0 legacy=0x7fa7ac0cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b1079700 1 -- 172.21.15.146:0/666405553 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac0c8f50 con 0x7fa7ac0c63a0 2024-09-13T18:27:01.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.989+0000 7fa7b1079700 1 --2- 172.21.15.146:0/666405553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac0c63a0 0x7fa7ac0c7140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009850 tx=0x7fa7a8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273c0fe6938037ac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:01.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b0878700 1 -- 172.21.15.146:0/666405553 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8011830 con 0x7fa7ac0c63a0 2024-09-13T18:27:01.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b0878700 1 -- 172.21.15.146:0/666405553 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a8011990 con 0x7fa7ac0c63a0 2024-09-13T18:27:01.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b0878700 1 -- 172.21.15.146:0/666405553 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8011d40 con 0x7fa7ac0c63a0 2024-09-13T18:27:01.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 -- 172.21.15.146:0/666405553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac0c63a0 msgr2=0x7fa7ac0c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 --2- 172.21.15.146:0/666405553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac0c63a0 0x7fa7ac0c7140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009850 tx=0x7fa7a8009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:01.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 -- 172.21.15.146:0/666405553 shutdown_connections 2024-09-13T18:27:01.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 --2- 172.21.15.146:0/666405553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac0c63a0 0x7fa7ac0c7140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 --2- 172.21.15.146:0/666405553 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7ac0c5fc0 0x7fa7ac0c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 -- 172.21.15.146:0/666405553 >> 172.21.15.146:0/666405553 conn(0x7fa7ac066090 msgr2=0x7fa7ac066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:01.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 -- 172.21.15.146:0/666405553 shutdown_connections 2024-09-13T18:27:01.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 -- 172.21.15.146:0/666405553 wait complete. 2024-09-13T18:27:02.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 Processor -- start 2024-09-13T18:27:02.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 -- start start 2024-09-13T18:27:02.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac06f5c0 0x7fa7ac154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7ac0c5fc0 0x7fa7ac1ef390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7ac0c63a0 0x7fa7ac1ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac0aeda0 con 0x7fa7ac06f5c0 2024-09-13T18:27:02.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac153c30 con 0x7fa7ac0c5fc0 2024-09-13T18:27:02.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b287c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac14c3e0 con 0x7fa7ac0c63a0 2024-09-13T18:27:02.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b187a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac06f5c0 0x7fa7ac154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b187a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac06f5c0 0x7fa7ac154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54370/0 (socket says 172.21.15.146:54370) 2024-09-13T18:27:02.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b187a700 1 -- 172.21.15.146:0/3733546912 learned_addr learned my addr 172.21.15.146:0/3733546912 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:02.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b1079700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7ac0c5fc0 0x7fa7ac1ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.993+0000 7fa7b207b700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7ac0c63a0 0x7fa7ac1ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7b187a700 1 -- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7ac0c63a0 msgr2=0x7fa7ac1ef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7b187a700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7ac0c63a0 0x7fa7ac1ef8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7b187a700 1 -- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7ac0c5fc0 msgr2=0x7fa7ac1ef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7b187a700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7ac0c5fc0 0x7fa7ac1ef390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7b187a700 1 -- 172.21.15.146:0/3733546912 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a80094b0 con 0x7fa7ac06f5c0 2024-09-13T18:27:02.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7b187a700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac06f5c0 0x7fa7ac154990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00e9d0 tx=0x7fa79c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:02.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7a27fc700 1 -- 172.21.15.146:0/3733546912 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c009720 con 0x7fa7ac06f5c0 2024-09-13T18:27:02.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7a27fc700 1 -- 172.21.15.146:0/3733546912 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c00c850 con 0x7fa7ac06f5c0 2024-09-13T18:27:02.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7a27fc700 1 -- 172.21.15.146:0/3733546912 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c00cbc0 con 0x7fa7ac06f5c0 2024-09-13T18:27:02.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7b287c700 1 -- 172.21.15.146:0/3733546912 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ac153950 con 0x7fa7ac06f5c0 2024-09-13T18:27:02.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7b287c700 1 -- 172.21.15.146:0/3733546912 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac1eee70 con 0x7fa7ac06f5c0 2024-09-13T18:27:02.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7b287c700 1 -- 172.21.15.146:0/3733546912 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ac0cb790 con 0x7fa7ac06f5c0 2024-09-13T18:27:02.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7a27fc700 1 -- 172.21.15.146:0/3733546912 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa79c00cd20 con 0x7fa7ac06f5c0 2024-09-13T18:27:02.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:01.997+0000 7fa7a27fc700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa790066730 0x7fa790068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.001+0000 7fa7a27fc700 1 -- 172.21.15.146:0/3733546912 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa79c084cf0 con 0x7fa7ac06f5c0 2024-09-13T18:27:02.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.001+0000 7fa7b1079700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa790066730 0x7fa790068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.001+0000 7fa7b1079700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa790066730 0x7fa790068bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009850 tx=0x7fa7a8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:02.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.001+0000 7fa7a27fc700 1 -- 172.21.15.146:0/3733546912 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa79c051de0 con 0x7fa7ac06f5c0 2024-09-13T18:27:02.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.209+0000 7fa7b287c700 1 -- 172.21.15.146:0/3733546912 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa7ac1eecd0 con 0x7fa7ac06f5c0 2024-09-13T18:27:02.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.213+0000 7fa7a27fc700 1 -- 172.21.15.146:0/3733546912 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164906 (secure 0 0 0) 0x7fa7ac1eecd0 con 0x7fa7ac06f5c0 2024-09-13T18:27:02.223 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:02.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.217+0000 7fa7b287c700 1 -- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa790066730 msgr2=0x7fa790068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.217+0000 7fa7b287c700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa790066730 0x7fa790068bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009850 tx=0x7fa7a8008030 comp rx=0 tx=0).stop 2024-09-13T18:27:02.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.217+0000 7fa7b287c700 1 -- 172.21.15.146:0/3733546912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac06f5c0 msgr2=0x7fa7ac154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.217+0000 7fa7b287c700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac06f5c0 0x7fa7ac154990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00e9d0 tx=0x7fa79c00ed90 comp rx=0 tx=0).stop 2024-09-13T18:27:02.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.217+0000 7fa7b287c700 1 -- 172.21.15.146:0/3733546912 shutdown_connections 2024-09-13T18:27:02.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.217+0000 7fa7b287c700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa790066730 0x7fa790068bf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.217+0000 7fa7b287c700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7ac0c63a0 0x7fa7ac1ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.217+0000 7fa7b287c700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7ac0c5fc0 0x7fa7ac1ef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.217+0000 7fa7b287c700 1 --2- 172.21.15.146:0/3733546912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac06f5c0 0x7fa7ac154990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.217+0000 7fa7b287c700 1 -- 172.21.15.146:0/3733546912 >> 172.21.15.146:0/3733546912 conn(0x7fa7ac066090 msgr2=0x7fa7ac158900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:02.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.217+0000 7fa7b287c700 1 -- 172.21.15.146:0/3733546912 shutdown_connections 2024-09-13T18:27:02.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.217+0000 7fa7b287c700 1 -- 172.21.15.146:0/3733546912 wait complete. 2024-09-13T18:27:02.238 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1235826889},{"type":"v1","addr":"172.21.15.154:6833","nonce":1235826889}]},"active_addr":"172.21.15.154:6833/1235826889","active_change":"2024-09-13T18:26:20.800362+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4131,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":3141255848}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":1331218366}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":2468691464}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":671471055}]}]} 2024-09-13T18:27:02.242 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T18:27:02.242 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T18:27:02.243 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:27:02.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b8958c700 1 Processor -- start 2024-09-13T18:27:02.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b8958c700 1 -- start start 2024-09-13T18:27:02.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b8958c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4b84154a40 0x7f4b840721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b8958c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b840cac60 0x7f4b840cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b8958c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b840c8c70 con 0x7f4b8406f7d0 2024-09-13T18:27:02.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b8958c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b84072810 con 0x7f4b84154a40 2024-09-13T18:27:02.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b8958c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b84157910 con 0x7f4b840cac60 2024-09-13T18:27:02.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b837fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4b8406f7d0 0x7f4b84157d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35426/0 (socket says 172.21.15.146:35426) 2024-09-13T18:27:02.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b837fe700 1 -- 172.21.15.146:0/3834791651 learned_addr learned my addr 172.21.15.146:0/3834791651 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:02.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b88d8b700 1 --2- 172.21.15.146:0/3834791651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b840cac60 0x7f4b840cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b83fff700 1 --2- 172.21.15.146:0/3834791651 >> v2:172.21.15.154:3300/0 conn(0x7f4b84154a40 0x7f4b840721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b82ffd700 1 -- 172.21.15.146:0/3834791651 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504329826 0 0) 0x7f4b840c8c70 con 0x7f4b8406f7d0 2024-09-13T18:27:02.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b82ffd700 1 -- 172.21.15.146:0/3834791651 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b640035d0 con 0x7f4b8406f7d0 2024-09-13T18:27:02.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b88d8b700 1 -- 172.21.15.146:0/3834791651 >> v2:172.21.15.154:3300/0 conn(0x7f4b84154a40 msgr2=0x7f4b840721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:02.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b88d8b700 1 --2- 172.21.15.146:0/3834791651 >> v2:172.21.15.154:3300/0 conn(0x7f4b84154a40 0x7f4b840721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b88d8b700 1 -- 172.21.15.146:0/3834791651 >> v1:172.21.15.146:6789/0 conn(0x7f4b8406f7d0 legacy=0x7f4b84157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b88d8b700 1 -- 172.21.15.146:0/3834791651 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840c8f50 con 0x7f4b840cac60 2024-09-13T18:27:02.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b88d8b700 1 --2- 172.21.15.146:0/3834791651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b840cac60 0x7f4b840cb0f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009d90 tx=0x7f4b7c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0f5b10f40a39439 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:02.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.353+0000 7f4b82ffd700 1 -- 172.21.15.146:0/3834791651 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00e070 con 0x7f4b840cac60 2024-09-13T18:27:02.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b82ffd700 1 -- 172.21.15.146:0/3834791651 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c00bb20 con 0x7f4b840cac60 2024-09-13T18:27:02.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b82ffd700 1 -- 172.21.15.146:0/3834791651 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c012430 con 0x7f4b840cac60 2024-09-13T18:27:02.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 -- 172.21.15.146:0/3834791651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b840cac60 msgr2=0x7f4b840cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 --2- 172.21.15.146:0/3834791651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b840cac60 0x7f4b840cb0f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009d90 tx=0x7f4b7c00b330 comp rx=0 tx=0).stop 2024-09-13T18:27:02.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 -- 172.21.15.146:0/3834791651 shutdown_connections 2024-09-13T18:27:02.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 --2- 172.21.15.146:0/3834791651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b840cac60 0x7f4b840cb0f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 --2- 172.21.15.146:0/3834791651 >> v2:172.21.15.154:3300/0 conn(0x7f4b84154a40 0x7f4b840721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 -- 172.21.15.146:0/3834791651 >> 172.21.15.146:0/3834791651 conn(0x7f4b84066090 msgr2=0x7f4b84066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:02.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 -- 172.21.15.146:0/3834791651 shutdown_connections 2024-09-13T18:27:02.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 -- 172.21.15.146:0/3834791651 wait complete. 2024-09-13T18:27:02.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 Processor -- start 2024-09-13T18:27:02.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 -- start start 2024-09-13T18:27:02.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b8406f7d0 0x7f4b840c0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b840cac60 0x7f4b84156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b84154a40 0x7f4b8414b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b840c0ed0 con 0x7f4b84154a40 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8406ddc0 con 0x7f4b840cac60 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b8958c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8406d7b0 con 0x7f4b8406f7d0 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b88d8b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b84154a40 0x7f4b8414b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b88d8b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b84154a40 0x7f4b8414b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54384/0 (socket says 172.21.15.146:54384) 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b88d8b700 1 -- 172.21.15.146:0/2630991124 learned_addr learned my addr 172.21.15.146:0/2630991124 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b83fff700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b8406f7d0 0x7f4b840c0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b837fe700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b840cac60 0x7f4b84156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b88d8b700 1 -- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b8406f7d0 msgr2=0x7f4b840c0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b88d8b700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b8406f7d0 0x7f4b840c0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b88d8b700 1 -- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b840cac60 msgr2=0x7f4b84156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b88d8b700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b840cac60 0x7f4b84156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b88d8b700 1 -- 172.21.15.146:0/2630991124 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c009a70 con 0x7f4b84154a40 2024-09-13T18:27:02.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.357+0000 7f4b88d8b700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b84154a40 0x7f4b8414b510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c014040 tx=0x7f4b7c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:02.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.361+0000 7f4b80ff9700 1 -- 172.21.15.146:0/2630991124 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c012430 con 0x7f4b84154a40 2024-09-13T18:27:02.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.361+0000 7f4b80ff9700 1 -- 172.21.15.146:0/2630991124 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c020070 con 0x7f4b84154a40 2024-09-13T18:27:02.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.361+0000 7f4b8958c700 1 -- 172.21.15.146:0/2630991124 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8406c940 con 0x7f4b84154a40 2024-09-13T18:27:02.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.361+0000 7f4b80ff9700 1 -- 172.21.15.146:0/2630991124 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c004280 con 0x7f4b84154a40 2024-09-13T18:27:02.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.361+0000 7f4b8958c700 1 -- 172.21.15.146:0/2630991124 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8406cf00 con 0x7f4b84154a40 2024-09-13T18:27:02.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.361+0000 7f4b8958c700 1 -- 172.21.15.146:0/2630991124 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b840c9e00 con 0x7f4b84154a40 2024-09-13T18:27:02.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.361+0000 7f4b80ff9700 1 -- 172.21.15.146:0/2630991124 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4b7c0043e0 con 0x7f4b84154a40 2024-09-13T18:27:02.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.361+0000 7f4b80ff9700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4b68066730 0x7f4b68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.361+0000 7f4b80ff9700 1 -- 172.21.15.146:0/2630991124 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4b7c022070 con 0x7f4b84154a40 2024-09-13T18:27:02.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.361+0000 7f4b83fff700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4b68066730 0x7f4b68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.365+0000 7f4b83fff700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4b68066730 0x7f4b68068bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009bf0 tx=0x7f4b74009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:02.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.373+0000 7f4b80ff9700 1 -- 172.21.15.146:0/2630991124 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4b7c0530b0 con 0x7f4b84154a40 2024-09-13T18:27:02.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.545+0000 7f4b8958c700 1 -- 172.21.15.146:0/2630991124 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b8406df30 con 0x7f4b84154a40 2024-09-13T18:27:02.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.549+0000 7f4b80ff9700 1 -- 172.21.15.146:0/2630991124 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f4b7c0566d0 con 0x7f4b84154a40 2024-09-13T18:27:02.555 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:02.556 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:27:02.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.553+0000 7f4b8958c700 1 -- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4b68066730 msgr2=0x7f4b68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.553+0000 7f4b8958c700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4b68066730 0x7f4b68068bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009bf0 tx=0x7f4b74009240 comp rx=0 tx=0).stop 2024-09-13T18:27:02.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.553+0000 7f4b8958c700 1 -- 172.21.15.146:0/2630991124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b84154a40 msgr2=0x7f4b8414b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.553+0000 7f4b8958c700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b84154a40 0x7f4b8414b510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c014040 tx=0x7f4b7c004af0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.553+0000 7f4b8958c700 1 -- 172.21.15.146:0/2630991124 shutdown_connections 2024-09-13T18:27:02.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.553+0000 7f4b8958c700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4b68066730 0x7f4b68068bf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.553+0000 7f4b8958c700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b84154a40 0x7f4b8414b510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.553+0000 7f4b8958c700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b840cac60 0x7f4b84156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.553+0000 7f4b8958c700 1 --2- 172.21.15.146:0/2630991124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b8406f7d0 0x7f4b840c0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.553+0000 7f4b8958c700 1 -- 172.21.15.146:0/2630991124 >> 172.21.15.146:0/2630991124 conn(0x7f4b84066090 msgr2=0x7f4b84153450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:02.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.553+0000 7f4b8958c700 1 -- 172.21.15.146:0/2630991124 shutdown_connections 2024-09-13T18:27:02.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.553+0000 7f4b8958c700 1 -- 172.21.15.146:0/2630991124 wait complete. 2024-09-13T18:27:02.571 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T18:27:02.571 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:27:02.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d7a9a6700 1 Processor -- start 2024-09-13T18:27:02.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d7a9a6700 1 -- start start 2024-09-13T18:27:02.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d7a9a6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1d740c5fc0 0x7f1d740c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d7a9a6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1d740c63a0 0x7f1d740c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d7a9a6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d7406f5c0 0x7f1d740cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d7a9a6700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d740c9bd0 con 0x7f1d740c63a0 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d7a9a6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d740c8c70 con 0x7f1d740c5fc0 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d7a9a6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740c8f50 con 0x7f1d7406f5c0 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d791a3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1d740c63a0 0x7f1d740c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d791a3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1d740c63a0 0x7f1d740c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54390/0 (socket says 172.21.15.146:54390) 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d791a3700 1 -- 172.21.15.146:0/3014047136 learned_addr learned my addr 172.21.15.146:0/3014047136 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d7a1a5700 1 --2- 172.21.15.146:0/3014047136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d7406f5c0 0x7f1d740cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d791a3700 1 -- 172.21.15.146:0/3014047136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d7406f5c0 msgr2=0x7f1d740cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d791a3700 1 --2- 172.21.15.146:0/3014047136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d7406f5c0 0x7f1d740cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d791a3700 1 -- 172.21.15.146:0/3014047136 >> v2:172.21.15.154:3300/0 conn(0x7f1d740c5fc0 msgr2=0x7f1d740c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d799a4700 1 --2- 172.21.15.146:0/3014047136 >> v2:172.21.15.154:3300/0 conn(0x7f1d740c5fc0 0x7f1d740c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d791a3700 1 --2- 172.21.15.146:0/3014047136 >> v2:172.21.15.154:3300/0 conn(0x7f1d740c5fc0 0x7f1d740c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d791a3700 1 -- 172.21.15.146:0/3014047136 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d740c9230 con 0x7f1d740c63a0 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d799a4700 1 --2- 172.21.15.146:0/3014047136 >> v2:172.21.15.154:3300/0 conn(0x7f1d740c5fc0 0x7f1d740c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d791a3700 1 --2- 172.21.15.146:0/3014047136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d740c63a0 0x7f1d740c7140 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1d740729c0 tx=0x7f1d6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7c09bf54da8df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:02.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d789a2700 1 -- 172.21.15.146:0/3014047136 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400e070 con 0x7f1d740c63a0 2024-09-13T18:27:02.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d789a2700 1 -- 172.21.15.146:0/3014047136 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6400b9c0 con 0x7f1d740c63a0 2024-09-13T18:27:02.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d789a2700 1 -- 172.21.15.146:0/3014047136 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400bd70 con 0x7f1d740c63a0 2024-09-13T18:27:02.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/3014047136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d740c63a0 msgr2=0x7f1d740c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.677+0000 7f1d7a9a6700 1 --2- 172.21.15.146:0/3014047136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d740c63a0 0x7f1d740c7140 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1d740729c0 tx=0x7f1d6400b330 comp rx=0 tx=0).stop 2024-09-13T18:27:02.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.681+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/3014047136 shutdown_connections 2024-09-13T18:27:02.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.681+0000 7f1d7a9a6700 1 --2- 172.21.15.146:0/3014047136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d7406f5c0 0x7f1d740cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.681+0000 7f1d7a9a6700 1 --2- 172.21.15.146:0/3014047136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d740c63a0 0x7f1d740c7140 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.681+0000 7f1d7a9a6700 1 --2- 172.21.15.146:0/3014047136 >> v2:172.21.15.154:3300/0 conn(0x7f1d740c5fc0 0x7f1d740c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.681+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/3014047136 >> 172.21.15.146:0/3014047136 conn(0x7f1d74066090 msgr2=0x7f1d74066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:02.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.681+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/3014047136 shutdown_connections 2024-09-13T18:27:02.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.681+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/3014047136 wait complete. 2024-09-13T18:27:02.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.681+0000 7f1d7a9a6700 1 Processor -- start 2024-09-13T18:27:02.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.681+0000 7f1d7a9a6700 1 -- start start 2024-09-13T18:27:02.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d7a9a6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d7406f5c0 0x7f1d7415b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d7a9a6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d740c5fc0 0x7f1d740ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d7a9a6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d740c63a0 0x7f1d74159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d7a9a6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740ad340 con 0x7f1d740c5fc0 2024-09-13T18:27:02.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d7a9a6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740abf30 con 0x7f1d740c63a0 2024-09-13T18:27:02.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d7a9a6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740ab640 con 0x7f1d7406f5c0 2024-09-13T18:27:02.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d791a3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d740c5fc0 0x7f1d740ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d791a3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d740c5fc0 0x7f1d740ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54394/0 (socket says 172.21.15.146:54394) 2024-09-13T18:27:02.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d791a3700 1 -- 172.21.15.146:0/4239599372 learned_addr learned my addr 172.21.15.146:0/4239599372 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:02.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d799a4700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d7406f5c0 0x7f1d7415b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d7a1a5700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d740c63a0 0x7f1d74159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d791a3700 1 -- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d7406f5c0 msgr2=0x7f1d7415b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d791a3700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d7406f5c0 0x7f1d7415b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d791a3700 1 -- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d740c63a0 msgr2=0x7f1d74159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d791a3700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d740c63a0 0x7f1d74159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d791a3700 1 -- 172.21.15.146:0/4239599372 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d64009b30 con 0x7f1d740c5fc0 2024-09-13T18:27:02.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d7a1a5700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d740c63a0 0x7f1d74159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:02.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d791a3700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d740c5fc0 0x7f1d740ae360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1d64012000 tx=0x7f1d64004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:02.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d727fc700 1 -- 172.21.15.146:0/4239599372 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64004250 con 0x7f1d740c5fc0 2024-09-13T18:27:02.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d727fc700 1 -- 172.21.15.146:0/4239599372 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d640043b0 con 0x7f1d740c5fc0 2024-09-13T18:27:02.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/4239599372 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d740ad590 con 0x7f1d740c5fc0 2024-09-13T18:27:02.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d727fc700 1 -- 172.21.15.146:0/4239599372 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64004740 con 0x7f1d740c5fc0 2024-09-13T18:27:02.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/4239599372 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d740abbc0 con 0x7f1d740c5fc0 2024-09-13T18:27:02.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.685+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/4239599372 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d740cb790 con 0x7f1d740c5fc0 2024-09-13T18:27:02.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.689+0000 7f1d727fc700 1 -- 172.21.15.146:0/4239599372 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d6401c440 con 0x7f1d740c5fc0 2024-09-13T18:27:02.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.689+0000 7f1d727fc700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1d60066730 0x7f1d60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.689+0000 7f1d727fc700 1 -- 172.21.15.146:0/4239599372 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1d640859d0 con 0x7f1d740c5fc0 2024-09-13T18:27:02.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.689+0000 7f1d799a4700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1d60066730 0x7f1d60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.689+0000 7f1d799a4700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1d60066730 0x7f1d60068bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800ab60 tx=0x7f1d68008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:02.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.701+0000 7f1d727fc700 1 -- 172.21.15.146:0/4239599372 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1d64056620 con 0x7f1d740c5fc0 2024-09-13T18:27:02.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.873+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/4239599372 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d7406e700 con 0x7f1d740c5fc0 2024-09-13T18:27:02.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.873+0000 7f1d727fc700 1 -- 172.21.15.146:0/4239599372 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f1d640048a0 con 0x7f1d740c5fc0 2024-09-13T18:27:02.881 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:02.881 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:27:02.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.877+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1d60066730 msgr2=0x7f1d60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.877+0000 7f1d7a9a6700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1d60066730 0x7f1d60068bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800ab60 tx=0x7f1d68008040 comp rx=0 tx=0).stop 2024-09-13T18:27:02.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.877+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/4239599372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d740c5fc0 msgr2=0x7f1d740ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.877+0000 7f1d7a9a6700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d740c5fc0 0x7f1d740ae360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1d64012000 tx=0x7f1d64004af0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.877+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/4239599372 shutdown_connections 2024-09-13T18:27:02.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.877+0000 7f1d7a9a6700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1d60066730 0x7f1d60068bf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.877+0000 7f1d7a9a6700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d740c63a0 0x7f1d74159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.877+0000 7f1d7a9a6700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d740c5fc0 0x7f1d740ae360 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.877+0000 7f1d7a9a6700 1 --2- 172.21.15.146:0/4239599372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d7406f5c0 0x7f1d7415b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.877+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/4239599372 >> 172.21.15.146:0/4239599372 conn(0x7f1d74066090 msgr2=0x7f1d740ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:02.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.877+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/4239599372 shutdown_connections 2024-09-13T18:27:02.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:02.877+0000 7f1d7a9a6700 1 -- 172.21.15.146:0/4239599372 wait complete. 2024-09-13T18:27:02.897 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T18:27:02.897 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T18:27:02.898 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T18:27:02.898 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T18:27:02.899 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T18:27:02.899 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T18:27:02.899 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T18:27:02.900 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T18:27:03.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f1130c8f700 1 Processor -- start 2024-09-13T18:27:03.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f1130c8f700 1 -- start start 2024-09-13T18:27:03.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f1130c8f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f112c06fcd0 0x7f112c06eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f1130c8f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112c14fb30 con 0x7f112c14f750 2024-09-13T18:27:03.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f1130c8f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112c155580 con 0x7f112c0a9490 2024-09-13T18:27:03.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f1130c8f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c155860 con 0x7f112c06fcd0 2024-09-13T18:27:03.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f112b7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f112c14f750 0x7f112c0cb120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35432/0 (socket says 172.21.15.146:35432) 2024-09-13T18:27:03.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f112b7fe700 1 -- 172.21.15.146:0/3285165485 learned_addr learned my addr 172.21.15.146:0/3285165485 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:03.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f722248b700 1 Processor -- start 2024-09-13T18:27:03.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f112bfff700 1 --2- 172.21.15.146:0/3285165485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f112c06fcd0 0x7f112c06eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f621baca700 1 Processor -- start 2024-09-13T18:27:03.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f621baca700 1 -- start start 2024-09-13T18:27:03.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f722248b700 1 -- start start 2024-09-13T18:27:03.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f564bf33700 1 Processor -- start 2024-09-13T18:27:03.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f564bf33700 1 -- start start 2024-09-13T18:27:03.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f112bfff700 1 -- 172.21.15.146:0/3285165485 >> v1:172.21.15.154:6789/0 conn(0x7f112c0a9490 legacy=0x7f112c06f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f112bfff700 1 -- 172.21.15.146:0/3285165485 >> v1:172.21.15.146:6789/0 conn(0x7f112c14f750 legacy=0x7f112c0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f112bfff700 1 -- 172.21.15.146:0/3285165485 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c155b40 con 0x7f112c06fcd0 2024-09-13T18:27:03.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f112bfff700 1 --2- 172.21.15.146:0/3285165485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f112c06fcd0 0x7f112c06eea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f111c009880 tx=0x7f111c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c02c6f4fc9d13117 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f621baca700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62140c57f0 0x7f62140cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f621baca700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f62140cf450 0x7f62140cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f621baca700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62140c5bd0 con 0x7f62140ca020 2024-09-13T18:27:03.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f621baca700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f621415a160 con 0x7f62140cf450 2024-09-13T18:27:03.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f621baca700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6214159830 con 0x7f62140c57f0 2024-09-13T18:27:03.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f564bf33700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56440c7280 0x7f56441559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f564bf33700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f56440c7660 0x7f5644155f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f564bf33700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f56440c01f0 0x7f5644070b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f564bf33700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564406ce90 con 0x7f56440c01f0 2024-09-13T18:27:03.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f6219065700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f62140ca020 0x7f62140ca850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35438/0 (socket says 172.21.15.146:35438) 2024-09-13T18:27:03.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f6219065700 1 -- 172.21.15.146:0/3186039627 learned_addr learned my addr 172.21.15.146:0/3186039627 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:03.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f722248b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f721c0ca340 0x7f721c0a9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f722248b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f721c0aa2a0 0x7f721c0cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f722248b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f721c0cb130 0x7f721c06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f722248b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c0c9930 con 0x7f721c0aa2a0 2024-09-13T18:27:03.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f722248b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c0cb5a0 con 0x7f721c0ca340 2024-09-13T18:27:03.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f722248b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c06f5c0 con 0x7f721c0cb130 2024-09-13T18:27:03.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f7220c88700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f721c0aa2a0 0x7f721c0cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f564bf33700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5644158f30 con 0x7f56440c7660 2024-09-13T18:27:03.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f564bf33700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56440c6180 con 0x7f56440c7280 2024-09-13T18:27:03.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f56494ce700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f56440c7660 0x7f5644155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f112a7fc700 1 -- 172.21.15.146:0/3285165485 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c011830 con 0x7f112c06fcd0 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f56494ce700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f56440c7660 0x7f5644155f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:46446/0 (socket says 172.21.15.146:46446) 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f56494ce700 1 -- 172.21.15.146:0/1460589264 learned_addr learned my addr 172.21.15.146:0/1460589264 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f7220c88700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f721c0aa2a0 0x7f721c0cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54410/0 (socket says 172.21.15.146:54410) 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f7220c88700 1 -- 172.21.15.146:0/4101029165 learned_addr learned my addr 172.21.15.146:0/4101029165 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f5649ccf700 1 --2- 172.21.15.146:0/1460589264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56440c7280 0x7f56441559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f112a7fc700 1 -- 172.21.15.146:0/3285165485 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c011990 con 0x7f112c06fcd0 2024-09-13T18:27:03.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f112a7fc700 1 -- 172.21.15.146:0/3285165485 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c011d20 con 0x7f112c06fcd0 2024-09-13T18:27:03.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f7221489700 1 --2- 172.21.15.146:0/4101029165 >> v2:172.21.15.154:3300/0 conn(0x7f721c0ca340 0x7f721c0a9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f621a067700 1 --2- 172.21.15.146:0/3186039627 >> v2:172.21.15.146:3300/0 conn(0x7f62140cf450 0x7f62140cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f6219866700 1 --2- 172.21.15.146:0/3186039627 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62140c57f0 0x7f62140cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f7220c88700 1 -- 172.21.15.146:0/4101029165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f721c0cb130 msgr2=0x7f721c06ffd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:03.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f7220c88700 1 --2- 172.21.15.146:0/4101029165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f721c0cb130 0x7f721c06ffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f7220c88700 1 -- 172.21.15.146:0/4101029165 >> v2:172.21.15.154:3300/0 conn(0x7f721c0ca340 msgr2=0x7f721c0a9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f7220c88700 1 --2- 172.21.15.146:0/4101029165 >> v2:172.21.15.154:3300/0 conn(0x7f721c0ca340 0x7f721c0a9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f7220c88700 1 -- 172.21.15.146:0/4101029165 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c06fb40 con 0x7f721c0aa2a0 2024-09-13T18:27:03.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7fc79978d700 1 Processor -- start 2024-09-13T18:27:03.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.045+0000 7f7221489700 1 --2- 172.21.15.146:0/4101029165 >> v2:172.21.15.154:3300/0 conn(0x7f721c0ca340 0x7f721c0a9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:03.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f621a067700 1 -- 172.21.15.146:0/3186039627 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62140c57f0 msgr2=0x7f62140cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f564a4d0700 1 --2- 172.21.15.146:0/1460589264 >> v2:172.21.15.146:3300/0 conn(0x7f56440c01f0 0x7f5644070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f5649ccf700 1 -- 172.21.15.146:0/1460589264 >> v2:172.21.15.154:3300/0 conn(0x7f56440c7660 msgr2=0x7f5644155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f5649ccf700 1 --2- 172.21.15.146:0/1460589264 >> v2:172.21.15.154:3300/0 conn(0x7f56440c7660 0x7f5644155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f5649ccf700 1 -- 172.21.15.146:0/1460589264 >> v2:172.21.15.146:3300/0 conn(0x7f56440c01f0 msgr2=0x7f5644070b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f621a067700 1 --2- 172.21.15.146:0/3186039627 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62140c57f0 0x7f62140cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f621a067700 1 -- 172.21.15.146:0/3186039627 >> v1:172.21.15.146:6789/0 conn(0x7f62140ca020 legacy=0x7f62140ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc79978d700 1 -- start start 2024-09-13T18:27:03.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1bcc9ac700 1 Processor -- start 2024-09-13T18:27:03.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f5649ccf700 1 --2- 172.21.15.146:0/1460589264 >> v2:172.21.15.146:3300/0 conn(0x7f56440c01f0 0x7f5644070b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f5649ccf700 1 -- 172.21.15.146:0/1460589264 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5644159140 con 0x7f56440c7280 2024-09-13T18:27:03.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f564a4d0700 1 --2- 172.21.15.146:0/1460589264 >> v2:172.21.15.146:3300/0 conn(0x7f56440c01f0 0x7f5644070b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:03.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f621a067700 1 -- 172.21.15.146:0/3186039627 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62140c9bb0 con 0x7f62140cf450 2024-09-13T18:27:03.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1bcc9ac700 1 -- start start 2024-09-13T18:27:03.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1130c8f700 1 -- 172.21.15.146:0/3285165485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f112c06fcd0 msgr2=0x7f112c06eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1130c8f700 1 --2- 172.21.15.146:0/3285165485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f112c06fcd0 0x7f112c06eea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f111c009880 tx=0x7f111c009b90 comp rx=0 tx=0).stop 2024-09-13T18:27:03.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1130c8f700 1 -- 172.21.15.146:0/3285165485 shutdown_connections 2024-09-13T18:27:03.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1130c8f700 1 --2- 172.21.15.146:0/3285165485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f112c06fcd0 0x7f112c06eea0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1130c8f700 1 -- 172.21.15.146:0/3285165485 >> 172.21.15.146:0/3285165485 conn(0x7f112c066090 msgr2=0x7f112c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f5649ccf700 1 --2- 172.21.15.146:0/1460589264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56440c7280 0x7f56441559c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5644071d40 tx=0x7f5634009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d05dd6824004232 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f621a067700 1 --2- 172.21.15.146:0/3186039627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62140cf450 0x7f62140cf900 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6204009850 tx=0x7f6204009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb04a58d503d984 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f5648ccd700 1 -- 172.21.15.146:0/1460589264 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634011830 con 0x7f56440c7280 2024-09-13T18:27:03.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f7220c88700 1 --2- 172.21.15.146:0/4101029165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f721c0aa2a0 0x7f721c0cabf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f721c0c87b0 tx=0x7f721400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3c0eeff901820a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f7213fff700 1 -- 172.21.15.146:0/4101029165 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721400e070 con 0x7f721c0aa2a0 2024-09-13T18:27:03.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f7213fff700 1 -- 172.21.15.146:0/4101029165 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721400b9c0 con 0x7f721c0aa2a0 2024-09-13T18:27:03.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f7213fff700 1 -- 172.21.15.146:0/4101029165 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721400bd70 con 0x7f721c0aa2a0 2024-09-13T18:27:03.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f6218864700 1 -- 172.21.15.146:0/3186039627 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204004230 con 0x7f62140cf450 2024-09-13T18:27:03.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f5648ccd700 1 -- 172.21.15.146:0/1460589264 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5634011990 con 0x7f56440c7280 2024-09-13T18:27:03.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f5648ccd700 1 -- 172.21.15.146:0/1460589264 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634011d20 con 0x7f56440c7280 2024-09-13T18:27:03.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f6218864700 1 -- 172.21.15.146:0/3186039627 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6204004390 con 0x7f62140cf450 2024-09-13T18:27:03.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f6218864700 1 -- 172.21.15.146:0/3186039627 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62040036c0 con 0x7f62140cf450 2024-09-13T18:27:03.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc79978d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc7940c9500 0x7fc79414f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc79978d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7940cacd0 0x7fc7940cb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc79978d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7940c98e0 con 0x7fc7940a9490 2024-09-13T18:27:03.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc79978d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79406f5c0 con 0x7fc7940c9500 2024-09-13T18:27:03.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc79978d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc794155420 con 0x7fc7940cacd0 2024-09-13T18:27:03.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc793fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc7940c9500 0x7fc79414f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc798f8c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7940cacd0 0x7fc7940cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc793fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc7940c9500 0x7fc79414f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:46468/0 (socket says 172.21.15.146:46468) 2024-09-13T18:27:03.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc793fff700 1 -- 172.21.15.146:0/2381890954 learned_addr learned my addr 172.21.15.146:0/2381890954 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc793fff700 1 -- 172.21.15.146:0/2381890954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7940cacd0 msgr2=0x7fc7940cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc793fff700 1 --2- 172.21.15.146:0/2381890954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7940cacd0 0x7fc7940cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc793fff700 1 -- 172.21.15.146:0/2381890954 >> v1:172.21.15.154:6789/0 conn(0x7fc7940a9490 legacy=0x7fc794155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc793fff700 1 -- 172.21.15.146:0/2381890954 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79414fb70 con 0x7fc7940c9500 2024-09-13T18:27:03.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f621baca700 1 -- 172.21.15.146:0/3186039627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62140cf450 msgr2=0x7f62140cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f621baca700 1 --2- 172.21.15.146:0/3186039627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62140cf450 0x7f62140cf900 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6204009850 tx=0x7f6204009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:03.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f722248b700 1 -- 172.21.15.146:0/4101029165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f721c0aa2a0 msgr2=0x7f721c0cabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f722248b700 1 --2- 172.21.15.146:0/4101029165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f721c0aa2a0 0x7f721c0cabf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f721c0c87b0 tx=0x7f721400b330 comp rx=0 tx=0).stop 2024-09-13T18:27:03.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f722248b700 1 -- 172.21.15.146:0/4101029165 shutdown_connections 2024-09-13T18:27:03.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f722248b700 1 --2- 172.21.15.146:0/4101029165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f721c0cb130 0x7f721c06ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f722248b700 1 --2- 172.21.15.146:0/4101029165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f721c0aa2a0 0x7f721c0cabf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f722248b700 1 --2- 172.21.15.146:0/4101029165 >> v2:172.21.15.154:3300/0 conn(0x7f721c0ca340 0x7f721c0a9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f722248b700 1 -- 172.21.15.146:0/4101029165 >> 172.21.15.146:0/4101029165 conn(0x7f721c066090 msgr2=0x7f721c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f722248b700 1 -- 172.21.15.146:0/4101029165 shutdown_connections 2024-09-13T18:27:03.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f564bf33700 1 -- 172.21.15.146:0/1460589264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56440c7280 msgr2=0x7f56441559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f564bf33700 1 --2- 172.21.15.146:0/1460589264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56440c7280 0x7f56441559c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5644071d40 tx=0x7f5634009b30 comp rx=0 tx=0).stop 2024-09-13T18:27:03.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f722248b700 1 -- 172.21.15.146:0/4101029165 wait complete. 2024-09-13T18:27:03.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f564bf33700 1 -- 172.21.15.146:0/1460589264 shutdown_connections 2024-09-13T18:27:03.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f564bf33700 1 --2- 172.21.15.146:0/1460589264 >> v2:172.21.15.146:3300/0 conn(0x7f56440c01f0 0x7f5644070b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f564bf33700 1 --2- 172.21.15.146:0/1460589264 >> v2:172.21.15.154:3300/0 conn(0x7f56440c7660 0x7f5644155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f564bf33700 1 --2- 172.21.15.146:0/1460589264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56440c7280 0x7f56441559c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f564bf33700 1 -- 172.21.15.146:0/1460589264 >> 172.21.15.146:0/1460589264 conn(0x7f564406f5c0 msgr2=0x7f564406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f621baca700 1 -- 172.21.15.146:0/3186039627 shutdown_connections 2024-09-13T18:27:03.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f621baca700 1 --2- 172.21.15.146:0/3186039627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62140cf450 0x7f62140cf900 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f621baca700 1 --2- 172.21.15.146:0/3186039627 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62140c57f0 0x7f62140cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f621baca700 1 -- 172.21.15.146:0/3186039627 >> 172.21.15.146:0/3186039627 conn(0x7f6214071d30 msgr2=0x7f6214072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc793fff700 1 --2- 172.21.15.146:0/2381890954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7940c9500 0x7fc79414f630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009e50 tx=0x7fc78c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac65e18b77e052da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1130c8f700 1 -- 172.21.15.146:0/3285165485 shutdown_connections 2024-09-13T18:27:03.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1130c8f700 1 -- 172.21.15.146:0/3285165485 wait complete. 2024-09-13T18:27:03.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f621baca700 1 -- 172.21.15.146:0/3186039627 shutdown_connections 2024-09-13T18:27:03.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f564bf33700 1 -- 172.21.15.146:0/1460589264 shutdown_connections 2024-09-13T18:27:03.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc792ffd700 1 -- 172.21.15.146:0/2381890954 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c00e070 con 0x7fc7940c9500 2024-09-13T18:27:03.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc792ffd700 1 -- 172.21.15.146:0/2381890954 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c00b910 con 0x7fc7940c9500 2024-09-13T18:27:03.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7fc792ffd700 1 -- 172.21.15.146:0/2381890954 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c00bcc0 con 0x7fc7940c9500 2024-09-13T18:27:03.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f564bf33700 1 -- 172.21.15.146:0/1460589264 wait complete. 2024-09-13T18:27:03.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f621baca700 1 -- 172.21.15.146:0/3186039627 wait complete. 2024-09-13T18:27:03.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1bcc9ac700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1bc814f750 0x7f1bc806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1bcc9ac700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1bc80a9490 0x7f1bc80cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1bcc9ac700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc814fb30 con 0x7f1bc80cb660 2024-09-13T18:27:03.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1bcc9ac700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc8155580 con 0x7f1bc814f750 2024-09-13T18:27:03.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1bcc9ac700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc8155860 con 0x7f1bc80a9490 2024-09-13T18:27:03.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f722248b700 1 Processor -- start 2024-09-13T18:27:03.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564bf33700 1 Processor -- start 2024-09-13T18:27:03.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1130c8f700 1 Processor -- start 2024-09-13T18:27:03.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1130c8f700 1 -- start start 2024-09-13T18:27:03.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1bc7fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1bc80cb660 0x7f1bc806f5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35444/0 (socket says 172.21.15.146:35444) 2024-09-13T18:27:03.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f1bc7fff700 1 -- 172.21.15.146:0/1292881567 learned_addr learned my addr 172.21.15.146:0/1292881567 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:03.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050484579 0 0) 0x7f1bc814fb30 con 0x7f1bc80cb660 2024-09-13T18:27:03.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bb00046e0 con 0x7f1bc80cb660 2024-09-13T18:27:03.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 569608763 0 0) 0x7f1bb00046e0 con 0x7f1bc80cb660 2024-09-13T18:27:03.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1bc80a9490 msgr2=0x7f1bc80cb120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:03.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 --2- 172.21.15.146:0/1292881567 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1bc80a9490 0x7f1bc80cb120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 >> v2:172.21.15.146:3300/0 conn(0x7f1bc814f750 msgr2=0x7f1bc806eea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:03.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 --2- 172.21.15.146:0/1292881567 >> v2:172.21.15.146:3300/0 conn(0x7f1bc814f750 0x7f1bc806eea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc8155b40 con 0x7f1bc80cb660 2024-09-13T18:27:03.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f1bc00019f0 con 0x7f1bc80cb660 2024-09-13T18:27:03.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 >> v1:172.21.15.146:6789/0 conn(0x7f1bc80cb660 legacy=0x7f1bc806f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 --2- 172.21.15.146:0/1292881567 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1bb0004ba0 0x7f1bb0004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 --2- 172.21.15.146:0/1292881567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bb0005980 0x7f1bb0005e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 --2- 172.21.15.146:0/1292881567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb0006440 0x7f1bb000cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc8155580 con 0x7f1bb0005980 2024-09-13T18:27:03.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc8155860 con 0x7f1bb0006440 2024-09-13T18:27:03.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb000d3d0 con 0x7f1bb0004ba0 2024-09-13T18:27:03.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc6ffd700 1 --2- 172.21.15.146:0/1292881567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bb0005980 0x7f1bb0005e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc7fff700 1 --2- 172.21.15.146:0/1292881567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb0006440 0x7f1bb000cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc77fe700 1 --2- 172.21.15.146:0/1292881567 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1bb0004ba0 0x7f1bb0004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f74c207a700 1 Processor -- start 2024-09-13T18:27:03.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.049+0000 7f74c207a700 1 -- start start 2024-09-13T18:27:03.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c207a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74bc1539c0 0x7f74bc0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c207a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f74bc06ffd0 0x7f74bc0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c207a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74bc0aa010 con 0x7f74bc15a570 2024-09-13T18:27:03.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c207a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74bc0aa5d0 con 0x7f74bc06ffd0 2024-09-13T18:27:03.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c207a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc0aa2f0 con 0x7f74bc1539c0 2024-09-13T18:27:03.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621baca700 1 Processor -- start 2024-09-13T18:27:03.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564bf33700 1 -- start start 2024-09-13T18:27:03.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f722248b700 1 -- start start 2024-09-13T18:27:03.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1130c8f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f112c06fcd0 0x7f112c1eec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1130c8f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f112c0a9490 0x7f112c0c1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1130c8f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f112c14f750 0x7f112c14b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1130c8f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c150c40 con 0x7f112c14f750 2024-09-13T18:27:03.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1130c8f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c1ef150 con 0x7f112c0a9490 2024-09-13T18:27:03.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1130c8f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c1503e0 con 0x7f112c06fcd0 2024-09-13T18:27:03.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c1078700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74bc1539c0 0x7f74bc0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c1879700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f74bc06ffd0 0x7f74bc0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c1879700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f74bc06ffd0 0x7f74bc0caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:46482/0 (socket says 172.21.15.146:46482) 2024-09-13T18:27:03.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c1879700 1 -- 172.21.15.146:0/2950248243 learned_addr learned my addr 172.21.15.146:0/2950248243 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc6ffd700 1 -- 172.21.15.146:0/1292881567 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1bb0004ba0 msgr2=0x7f1bb0004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc6ffd700 1 --2- 172.21.15.146:0/1292881567 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1bb0004ba0 0x7f1bb0004fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc6ffd700 1 -- 172.21.15.146:0/1292881567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb0006440 msgr2=0x7f1bb000cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc6ffd700 1 --2- 172.21.15.146:0/1292881567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb0006440 0x7f1bb000cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc6ffd700 1 -- 172.21.15.146:0/1292881567 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb00046e0 con 0x7f1bb0005980 2024-09-13T18:27:03.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f112bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f112c14f750 0x7f112c14b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f112bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f112c14f750 0x7f112c14b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54448/0 (socket says 172.21.15.146:54448) 2024-09-13T18:27:03.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f112bfff700 1 -- 172.21.15.146:0/3201064102 learned_addr learned my addr 172.21.15.146:0/3201064102 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f112b7fe700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f112c06fcd0 0x7f112c1eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc7fff700 1 --2- 172.21.15.146:0/1292881567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb0006440 0x7f1bb000cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:03.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc6ffd700 1 --2- 172.21.15.146:0/1292881567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bb0005980 0x7f1bb0005e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc006fd0 tx=0x7f1bb00069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b3553e98790621 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c1879700 1 -- 172.21.15.146:0/2950248243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74bc1539c0 msgr2=0x7f74bc0c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f112affd700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f112c0a9490 0x7f112c0c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f112bfff700 1 -- 172.21.15.146:0/3201064102 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f112c06fcd0 msgr2=0x7f112c1eec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f112bfff700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f112c06fcd0 0x7f112c1eec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f112bfff700 1 -- 172.21.15.146:0/3201064102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f112c0a9490 msgr2=0x7f112c0c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f112bfff700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f112c0a9490 0x7f112c0c1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f112bfff700 1 -- 172.21.15.146:0/3201064102 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c0094b0 con 0x7f112c14f750 2024-09-13T18:27:03.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c1879700 1 --2- 172.21.15.146:0/2950248243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74bc1539c0 0x7f74bc0c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f722248b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f721c0aa2a0 0x7f721c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f722248b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f721c0ca340 0x7f721c0bfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f722248b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f721c0cb130 0x7f721c06d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f722248b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c15d6f0 con 0x7f721c0cb130 2024-09-13T18:27:03.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f722248b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c15b9a0 con 0x7f721c0aa2a0 2024-09-13T18:27:03.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f722248b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c15d920 con 0x7f721c0ca340 2024-09-13T18:27:03.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c1879700 1 -- 172.21.15.146:0/2950248243 >> v1:172.21.15.154:6789/0 conn(0x7f74bc15a570 legacy=0x7f74bc0c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7fc79978d700 1 -- 172.21.15.146:0/2381890954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7940c9500 msgr2=0x7fc79414f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7fc79978d700 1 --2- 172.21.15.146:0/2381890954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7940c9500 0x7fc79414f630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009e50 tx=0x7fc78c00b330 comp rx=0 tx=0).stop 2024-09-13T18:27:03.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7fc79978d700 1 -- 172.21.15.146:0/2381890954 shutdown_connections 2024-09-13T18:27:03.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7fc79978d700 1 --2- 172.21.15.146:0/2381890954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7940cacd0 0x7fc7940cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7fc79978d700 1 --2- 172.21.15.146:0/2381890954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7940c9500 0x7fc79414f630 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7fc79978d700 1 -- 172.21.15.146:0/2381890954 >> 172.21.15.146:0/2381890954 conn(0x7fc794066090 msgr2=0x7fc794066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621baca700 1 -- start start 2024-09-13T18:27:03.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621baca700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62140c57f0 0x7f62141e35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621baca700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62140ca020 0x7f6214157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621baca700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62140cf450 0x7f6214157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621baca700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62141e3b00 con 0x7f62140cf450 2024-09-13T18:27:03.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621baca700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62141df840 con 0x7f62140c57f0 2024-09-13T18:27:03.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621baca700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62141efb00 con 0x7f62140ca020 2024-09-13T18:27:03.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f112bfff700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f112c14f750 0x7f112c14b510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f111c009bd0 tx=0x7f111c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0092f0 con 0x7f1bb0005980 2024-09-13T18:27:03.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7fc79978d700 1 -- 172.21.15.146:0/2381890954 shutdown_connections 2024-09-13T18:27:03.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f7221489700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f721c0aa2a0 0x7f721c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f7221489700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f721c0aa2a0 0x7f721c15e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:46506/0 (socket says 172.21.15.146:46506) 2024-09-13T18:27:03.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f7221489700 1 -- 172.21.15.146:0/3186413088 learned_addr learned my addr 172.21.15.146:0/3186413088 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc00b620 con 0x7f1bb0005980 2024-09-13T18:27:03.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c1879700 1 -- 172.21.15.146:0/2950248243 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc159a90 con 0x7f74bc06ffd0 2024-09-13T18:27:03.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f74c1879700 1 --2- 172.21.15.146:0/2950248243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74bc06ffd0 0x7f74bc0caa30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f74b800bbb0 tx=0x7f74b800bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6746dd361db045ca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621a067700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62140cf450 0x7f6214157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621a067700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62140cf450 0x7f6214157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54456/0 (socket says 172.21.15.146:54456) 2024-09-13T18:27:03.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621a067700 1 -- 172.21.15.146:0/3427901947 learned_addr learned my addr 172.21.15.146:0/3427901947 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f6219866700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62140c57f0 0x7f62141e35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f6219065700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62140ca020 0x7f6214157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f7220c88700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f721c0ca340 0x7f721c0bfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621a067700 1 -- 172.21.15.146:0/3427901947 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62140ca020 msgr2=0x7f6214157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621a067700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62140ca020 0x7f6214157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621a067700 1 -- 172.21.15.146:0/3427901947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62140c57f0 msgr2=0x7f62141e35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621a067700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62140c57f0 0x7f62141e35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621a067700 1 -- 172.21.15.146:0/3427901947 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62040094b0 con 0x7f62140cf450 2024-09-13T18:27:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f7221489700 1 -- 172.21.15.146:0/3186413088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f721c0ca340 msgr2=0x7f721c0bfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f7221489700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f721c0ca340 0x7f721c0bfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f7221489700 1 -- 172.21.15.146:0/3186413088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f721c0cb130 msgr2=0x7f721c06d680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:27:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f7221c8a700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f721c0cb130 0x7f721c06d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f7221489700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f721c0cb130 0x7f721c06d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564bf33700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56440c01f0 0x7f5644155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564bf33700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56440c7280 0x7f56440c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564bf33700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56440c7660 0x7f564414b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564bf33700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5644163360 con 0x7f56440c7660 2024-09-13T18:27:03.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564bf33700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56440cbcc0 con 0x7f56440c7280 2024-09-13T18:27:03.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f7221489700 1 -- 172.21.15.146:0/3186413088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7214009b30 con 0x7f721c0aa2a0 2024-09-13T18:27:03.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564bf33700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5644155140 con 0x7f56440c01f0 2024-09-13T18:27:03.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f5649ccf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56440c01f0 0x7f5644155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f5649ccf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56440c01f0 0x7f5644155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41140/0 (socket says 172.21.15.146:41140) 2024-09-13T18:27:03.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f5649ccf700 1 -- 172.21.15.146:0/4001348793 learned_addr learned my addr 172.21.15.146:0/4001348793 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f56494ce700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56440c7280 0x7f56440c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564a4d0700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56440c7660 0x7f564414b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564a4d0700 1 -- 172.21.15.146:0/4001348793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56440c01f0 msgr2=0x7f5644155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564a4d0700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56440c01f0 0x7f5644155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564a4d0700 1 -- 172.21.15.146:0/4001348793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56440c7280 msgr2=0x7f56440c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564a4d0700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56440c7280 0x7f56440c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f564a4d0700 1 -- 172.21.15.146:0/4001348793 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56340094b0 con 0x7f56440c7660 2024-09-13T18:27:03.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f5649ccf700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56440c01f0 0x7f5644155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:03.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f7221c8a700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f721c0cb130 0x7f721c06d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:03.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f621a067700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62140cf450 0x7f6214157850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6204004000 tx=0x7f6204004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/1292881567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bb0005980 msgr2=0x7f1bb0005e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f307bfff700 1 Processor -- start 2024-09-13T18:27:03.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f564a4d0700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56440c7660 0x7f564414b660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f564000afd0 tx=0x7f564001d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f7221489700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f721c0aa2a0 0x7f721c15e480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f721800c770 tx=0x7f721800ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f620a7fc700 1 -- 172.21.15.146:0/3427901947 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204003bf0 con 0x7f62140cf450 2024-09-13T18:27:03.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f307bfff700 1 -- start start 2024-09-13T18:27:03.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f307bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f307c072460 0x7f307c1686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f7211ffb700 1 -- 172.21.15.146:0/3186413088 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218011800 con 0x7f721c0aa2a0 2024-09-13T18:27:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f7211ffb700 1 -- 172.21.15.146:0/3186413088 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7218011960 con 0x7f721c0aa2a0 2024-09-13T18:27:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f7211ffb700 1 -- 172.21.15.146:0/3186413088 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218011d10 con 0x7f721c0aa2a0 2024-09-13T18:27:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1113fff700 1 -- 172.21.15.146:0/3201064102 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c011800 con 0x7f112c14f750 2024-09-13T18:27:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1130c8f700 1 -- 172.21.15.146:0/3201064102 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c150680 con 0x7f112c14f750 2024-09-13T18:27:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1130c8f700 1 -- 172.21.15.146:0/3201064102 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112c1ee410 con 0x7f112c14f750 2024-09-13T18:27:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1113fff700 1 -- 172.21.15.146:0/3201064102 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c01f070 con 0x7f112c14f750 2024-09-13T18:27:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1113fff700 1 -- 172.21.15.146:0/3201064102 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c0038c0 con 0x7f112c14f750 2024-09-13T18:27:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f307bfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307c0a9490 con 0x7f307c0c8d40 2024-09-13T18:27:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f307bfff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307c0ca400 con 0x7f307c0c9120 2024-09-13T18:27:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7f307bfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c0c7ec0 con 0x7f307c072460 2024-09-13T18:27:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307affd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f307c0c8d40 0x7f307c0c76a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35446/0 (socket says 172.21.15.146:35446) 2024-09-13T18:27:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307affd700 1 -- 172.21.15.146:0/2669896294 learned_addr learned my addr 172.21.15.146:0/2669896294 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307b7fe700 1 --2- 172.21.15.146:0/2669896294 >> v2:172.21.15.146:3300/0 conn(0x7f307c072460 0x7f307c1686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542890971 0 0) 0x7f307c0ca400 con 0x7f307c0c9120 2024-09-13T18:27:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30640035d0 con 0x7f307c0c9120 2024-09-13T18:27:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f621baca700 1 -- 172.21.15.146:0/3427901947 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62141dfae0 con 0x7f62140cf450 2024-09-13T18:27:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f621baca700 1 -- 172.21.15.146:0/3427901947 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62140cda30 con 0x7f62140cf450 2024-09-13T18:27:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f620a7fc700 1 -- 172.21.15.146:0/3427901947 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620401f070 con 0x7f62140cf450 2024-09-13T18:27:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f620a7fc700 1 -- 172.21.15.146:0/3427901947 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204011630 con 0x7f62140cf450 2024-09-13T18:27:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f722248b700 1 -- 172.21.15.146:0/3186413088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c15c4b0 con 0x7f721c0aa2a0 2024-09-13T18:27:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f722248b700 1 -- 172.21.15.146:0/3186413088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c15b490 con 0x7f721c0aa2a0 2024-09-13T18:27:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f621baca700 1 -- 172.21.15.146:0/3427901947 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f61fc000fc0 con 0x7f62140cf450 2024-09-13T18:27:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.053+0000 7fc79978d700 1 -- 172.21.15.146:0/2381890954 wait complete. 2024-09-13T18:27:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc79978d700 1 Processor -- start 2024-09-13T18:27:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc79978d700 1 -- start start 2024-09-13T18:27:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752297707 0 0) 0x7f307c0a9490 con 0x7f307c0c8d40 2024-09-13T18:27:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f307c0ca400 con 0x7f307c0c8d40 2024-09-13T18:27:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4179965375 0 0) 0x7f30640035d0 con 0x7f307c0c9120 2024-09-13T18:27:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 >> v2:172.21.15.146:3300/0 conn(0x7f307c072460 msgr2=0x7f307c1686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 --2- 172.21.15.146:0/2669896294 >> v2:172.21.15.146:3300/0 conn(0x7f307c072460 0x7f307c1686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 >> v1:172.21.15.146:6789/0 conn(0x7f307c0c8d40 legacy=0x7f307c0c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c0c81a0 con 0x7f307c0c9120 2024-09-13T18:27:03.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f3070001670 con 0x7f307c0c9120 2024-09-13T18:27:03.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 >> v1:172.21.15.154:6789/0 conn(0x7f307c0c9120 legacy=0x7f307c0a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 --2- 172.21.15.146:0/2669896294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3064003c20 0x7f3064004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 --2- 172.21.15.146:0/2669896294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3064004870 0x7f3064004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 --2- 172.21.15.146:0/2669896294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3064005230 0x7f306400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0c7ec0 con 0x7f3064005230 2024-09-13T18:27:03.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306400c130 con 0x7f3064003c20 2024-09-13T18:27:03.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306400c2f0 con 0x7f3064004870 2024-09-13T18:27:03.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307affd700 1 --2- 172.21.15.146:0/2669896294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3064003c20 0x7f3064004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307b7fe700 1 --2- 172.21.15.146:0/2669896294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3064005230 0x7f306400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f74b3fff700 1 -- 172.21.15.146:0/2950248243 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b800dd30 con 0x7f74bc06ffd0 2024-09-13T18:27:03.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f74b3fff700 1 -- 172.21.15.146:0/2950248243 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b800de90 con 0x7f74bc06ffd0 2024-09-13T18:27:03.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1bc67fc700 1 -- 172.21.15.146:0/1292881567 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0092f0 con 0x7f1bb0005980 2024-09-13T18:27:03.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1bcc9ac700 1 --2- 172.21.15.146:0/1292881567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bb0005980 0x7f1bb0005e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc006fd0 tx=0x7f1bb00069b0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f722248b700 1 -- 172.21.15.146:0/3186413088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f721c0aa780 con 0x7f721c0aa2a0 2024-09-13T18:27:03.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1bc7fff700 1 -- 172.21.15.146:0/1292881567 reap_dead start 2024-09-13T18:27:03.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/1292881567 shutdown_connections 2024-09-13T18:27:03.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/1292881567 >> 172.21.15.146:0/1292881567 conn(0x7f1bc8066090 msgr2=0x7f1bc8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1130c8f700 1 -- 172.21.15.146:0/3201064102 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f110c000fc0 con 0x7f112c14f750 2024-09-13T18:27:03.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307b7fe700 1 -- 172.21.15.146:0/2669896294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3064004870 msgr2=0x7f3064004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307a7fc700 1 --2- 172.21.15.146:0/2669896294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3064004870 0x7f3064004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307b7fe700 1 --2- 172.21.15.146:0/2669896294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3064004870 0x7f3064004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307b7fe700 1 -- 172.21.15.146:0/2669896294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3064003c20 msgr2=0x7f3064004020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307b7fe700 1 --2- 172.21.15.146:0/2669896294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3064003c20 0x7f3064004020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307b7fe700 1 -- 172.21.15.146:0/2669896294 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c0c81a0 con 0x7f3064005230 2024-09-13T18:27:03.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307a7fc700 1 --2- 172.21.15.146:0/2669896294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3064004870 0x7f3064004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:03.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307b7fe700 1 -- 172.21.15.146:0/2669896294 reap_dead start 2024-09-13T18:27:03.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f563a7fc700 1 -- 172.21.15.146:0/4001348793 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640022610 con 0x7f56440c7660 2024-09-13T18:27:03.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc79978d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7940a9490 0x7fc7941f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc79978d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940c9500 0x7fc7940cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc79978d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7940cacd0 0x7fc794151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc79978d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc794151a80 con 0x7fc7940c9500 2024-09-13T18:27:03.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc79978d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc794150990 con 0x7fc7940cacd0 2024-09-13T18:27:03.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc79978d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc794150d60 con 0x7fc7940a9490 2024-09-13T18:27:03.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940c9500 0x7fc7940cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc793fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7940a9490 0x7fc7941f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940c9500 0x7fc7940cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54500/0 (socket says 172.21.15.146:54500) 2024-09-13T18:27:03.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc7937fe700 1 -- 172.21.15.146:0/851531462 learned_addr learned my addr 172.21.15.146:0/851531462 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f74b3fff700 1 -- 172.21.15.146:0/2950248243 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8010690 con 0x7f74bc06ffd0 2024-09-13T18:27:03.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc7937fe700 1 -- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7940a9490 msgr2=0x7fc7941f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc7937fe700 1 --2- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7940a9490 0x7fc7941f2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc7937fe700 1 -- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7940cacd0 msgr2=0x7fc794151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc7937fe700 1 --2- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7940cacd0 0x7fc794151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc7937fe700 1 -- 172.21.15.146:0/851531462 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c009b30 con 0x7fc7940c9500 2024-09-13T18:27:03.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f620a7fc700 1 -- 172.21.15.146:0/3427901947 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6204021030 con 0x7f62140cf450 2024-09-13T18:27:03.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f563a7fc700 1 -- 172.21.15.146:0/4001348793 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5640022790 con 0x7f56440c7660 2024-09-13T18:27:03.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f564bf33700 1 -- 172.21.15.146:0/4001348793 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564414b1d0 con 0x7f56440c7660 2024-09-13T18:27:03.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f564bf33700 1 -- 172.21.15.146:0/4001348793 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5644163080 con 0x7f56440c7660 2024-09-13T18:27:03.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f74c207a700 1 -- 172.21.15.146:0/2950248243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74bc06ffd0 msgr2=0x7f74bc0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f74c207a700 1 --2- 172.21.15.146:0/2950248243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74bc06ffd0 0x7f74bc0caa30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f74b800bbb0 tx=0x7f74b800bec0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f307b7fe700 1 --2- 172.21.15.146:0/2669896294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3064005230 0x7f306400ba30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3074009fa0 tx=0x7f30640058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c41f4784b402d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074003a40 con 0x7f3064005230 2024-09-13T18:27:03.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1113fff700 1 -- 172.21.15.146:0/3201064102 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f111c003a20 con 0x7f112c14f750 2024-09-13T18:27:03.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1113fff700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1118066730 0x7f1118068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1113fff700 1 -- 172.21.15.146:0/3201064102 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f111c021070 con 0x7f112c14f750 2024-09-13T18:27:03.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1113fff700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] conn(0x7f111806fc70 0x7f11180720b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7f1113fff700 1 -- 172.21.15.146:0/3201064102 --> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1118072780 con 0x7f111806fc70 2024-09-13T18:27:03.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.057+0000 7fc7937fe700 1 --2- 172.21.15.146:0/851531462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940c9500 0x7fc7940cb3d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc78800c530 tx=0x7fc78800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f620a7fc700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61f8066760 0x7f61f8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f74c207a700 1 -- 172.21.15.146:0/2950248243 shutdown_connections 2024-09-13T18:27:03.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f74c207a700 1 --2- 172.21.15.146:0/2950248243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74bc06ffd0 0x7f74bc0caa30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f74c207a700 1 --2- 172.21.15.146:0/2950248243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74bc1539c0 0x7f74bc0c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f74c207a700 1 -- 172.21.15.146:0/2950248243 >> 172.21.15.146:0/2950248243 conn(0x7f74bc066090 msgr2=0x7f74bc066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f563a7fc700 1 -- 172.21.15.146:0/4001348793 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640022b40 con 0x7f56440c7660 2024-09-13T18:27:03.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307400f040 con 0x7f3064005230 2024-09-13T18:27:03.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f74c207a700 1 -- 172.21.15.146:0/2950248243 shutdown_connections 2024-09-13T18:27:03.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f6219866700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61f8066760 0x7f61f8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f620a7fc700 1 -- 172.21.15.146:0/3427901947 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6204085db0 con 0x7f62140cf450 2024-09-13T18:27:03.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f620a7fc700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] conn(0x7f61f806fd20 0x7f61f8072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f7211ffb700 1 -- 172.21.15.146:0/3186413088 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f72180233e0 con 0x7f721c0aa2a0 2024-09-13T18:27:03.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f620a7fc700 1 -- 172.21.15.146:0/3427901947 --> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61f8072830 con 0x7f61f806fd20 2024-09-13T18:27:03.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f112affd700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] conn(0x7f111806fc70 0x7f11180720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f1113fff700 1 -- 172.21.15.146:0/3201064102 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f111c052970 con 0x7f112c14f750 2024-09-13T18:27:03.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f112b7fe700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1118066730 0x7f1118068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f112affd700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] conn(0x7f111806fc70 0x7f11180720b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f620a7fc700 1 -- 172.21.15.146:0/3427901947 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6204052f40 con 0x7f62140cf450 2024-09-13T18:27:03.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7fc790ff9700 1 -- 172.21.15.146:0/851531462 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7880052d0 con 0x7fc7940c9500 2024-09-13T18:27:03.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7fc790ff9700 1 -- 172.21.15.146:0/851531462 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788005450 con 0x7fc7940c9500 2024-09-13T18:27:03.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7fc790ff9700 1 -- 172.21.15.146:0/851531462 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788006ac0 con 0x7fc7940c9500 2024-09-13T18:27:03.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f6219065700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] conn(0x7f61f806fd20 0x7f61f8072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f6219866700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61f8066760 0x7f61f8068c20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6210009bf0 tx=0x7f6210009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f3079ffb700 1 -- 172.21.15.146:0/2669896294 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074003dc0 con 0x7f3064005230 2024-09-13T18:27:03.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f112b7fe700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1118066730 0x7f1118068bf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f11240012e0 tx=0x7f1124001a30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f564bf33700 1 -- 172.21.15.146:0/4001348793 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f56441599e0 con 0x7f56440c7660 2024-09-13T18:27:03.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f307bfff700 1 -- 172.21.15.146:0/2669896294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3064005230 msgr2=0x7f306400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f307bfff700 1 --2- 172.21.15.146:0/2669896294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3064005230 0x7f306400ba30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3074009fa0 tx=0x7f30640058a0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f74c207a700 1 -- 172.21.15.146:0/2950248243 wait complete. 2024-09-13T18:27:03.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/1292881567 shutdown_connections 2024-09-13T18:27:03.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f6219065700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] conn(0x7f61f806fd20 0x7f61f8072160 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7fc79978d700 1 -- 172.21.15.146:0/851531462 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7941f31f0 con 0x7fc7940c9500 2024-09-13T18:27:03.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7fc79978d700 1 -- 172.21.15.146:0/851531462 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7940ad530 con 0x7fc7940c9500 2024-09-13T18:27:03.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f7211ffb700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f72000667b0 0x7f7200068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f7211ffb700 1 -- 172.21.15.146:0/3186413088 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f72180849d0 con 0x7f721c0aa2a0 2024-09-13T18:27:03.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f7211ffb700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] conn(0x7f720006fdf0 0x7f7200072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f7211ffb700 1 -- 172.21.15.146:0/3186413088 --> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7200072900 con 0x7f720006fdf0 2024-09-13T18:27:03.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f7211ffb700 1 -- 172.21.15.146:0/3186413088 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7218084e20 con 0x7f721c0aa2a0 2024-09-13T18:27:03.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f7220c88700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f72000667b0 0x7f7200068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f7221c8a700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] conn(0x7f720006fdf0 0x7f7200072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f563a7fc700 1 -- 172.21.15.146:0/4001348793 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5640022ca0 con 0x7f56440c7660 2024-09-13T18:27:03.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/1292881567 wait complete. 2024-09-13T18:27:03.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f620a7fc700 1 -- 172.21.15.146:0/3427901947 <== osd.1 v2:172.21.15.146:6804/3907005797 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f61f8072830 con 0x7f61f806fd20 2024-09-13T18:27:03.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f563a7fc700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5628066780 0x7f5628068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f1113fff700 1 -- 172.21.15.146:0/3201064102 <== osd.3 v2:172.21.15.146:6824/3480660705 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f1118072780 con 0x7f111806fc70 2024-09-13T18:27:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f307bfff700 1 -- 172.21.15.146:0/2669896294 shutdown_connections 2024-09-13T18:27:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f307bfff700 1 --2- 172.21.15.146:0/2669896294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3064005230 0x7f306400ba30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f307bfff700 1 -- 172.21.15.146:0/2669896294 >> 172.21.15.146:0/2669896294 conn(0x7f307c066090 msgr2=0x7f307c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f7221c8a700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] conn(0x7f720006fdf0 0x7f7200072230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f5649ccf700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5628066780 0x7f5628068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f307bfff700 1 -- 172.21.15.146:0/2669896294 shutdown_connections 2024-09-13T18:27:03.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f307bfff700 1 -- 172.21.15.146:0/2669896294 wait complete. 2024-09-13T18:27:03.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7fc79978d700 1 -- 172.21.15.146:0/851531462 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc7940c8130 con 0x7fc7940c9500 2024-09-13T18:27:03.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f1bcc9ac700 1 Processor -- start 2024-09-13T18:27:03.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f7220c88700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f72000667b0 0x7f7200068c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f721400b370 tx=0x7f7214009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f1bcc9ac700 1 -- start start 2024-09-13T18:27:03.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f7211ffb700 1 -- 172.21.15.146:0/3186413088 <== osd.0 v2:172.21.15.146:6816/2811789208 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f7200072900 con 0x7f720006fdf0 2024-09-13T18:27:03.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bcc9ac700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1bc80c6b70 0x7f1bc806d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bcc9ac700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bc80aba80 0x7f1bc81456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bcc9ac700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bc80c6f50 0x7f1bc8145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bcc9ac700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc8146160 con 0x7f1bc80c6f50 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bcc9ac700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc80aaf50 con 0x7f1bc80aba80 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bcc9ac700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc806dd30 con 0x7f1bc80c6b70 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bc6ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bc80aba80 0x7f1bc81456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bc6ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bc80aba80 0x7f1bc81456e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:46546/0 (socket says 172.21.15.146:46546) 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bc6ffd700 1 -- 172.21.15.146:0/938452976 learned_addr learned my addr 172.21.15.146:0/938452976 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bc77fe700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1bc80c6b70 0x7f1bc806d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bc7fff700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bc80c6f50 0x7f1bc8145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bc7fff700 1 -- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1bc80c6b70 msgr2=0x7f1bc806d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bc7fff700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1bc80c6b70 0x7f1bc806d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bc7fff700 1 -- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bc80aba80 msgr2=0x7f1bc81456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bc7fff700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bc80aba80 0x7f1bc81456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bc7fff700 1 -- 172.21.15.146:0/938452976 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc806e6b0 con 0x7f1bc80c6f50 2024-09-13T18:27:03.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bc6ffd700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bc80aba80 0x7f1bc81456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:03.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f1bc7fff700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bc80c6f50 0x7f1bc8145c20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8071a10 tx=0x7f1bc000af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307bfff700 1 Processor -- start 2024-09-13T18:27:03.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307bfff700 1 -- start start 2024-09-13T18:27:03.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.061+0000 7f74c207a700 1 Processor -- start 2024-09-13T18:27:03.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c207a700 1 -- start start 2024-09-13T18:27:03.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c207a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74bc06ffd0 0x7f74bc1f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c207a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74bc1539c0 0x7f74bc150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c207a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74bc15a570 0x7f74bc1514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3064005230 0x7f307c0c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f307c13b030 0x7f307c1259e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f307c1275a0 0x7f307c127a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307bfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0c3c20 con 0x7f3064005230 2024-09-13T18:27:03.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307bfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0c2b30 con 0x7f307c1275a0 2024-09-13T18:27:03.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c13fba0 con 0x7f307c13b030 2024-09-13T18:27:03.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c207a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc1f7580 con 0x7f74bc1539c0 2024-09-13T18:27:03.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c207a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc0c4af0 con 0x7f74bc15a570 2024-09-13T18:27:03.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c207a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc154570 con 0x7f74bc06ffd0 2024-09-13T18:27:03.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c0877700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74bc1539c0 0x7f74bc150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3064005230 0x7f307c0c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3064005230 0x7f307c0c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54534/0 (socket says 172.21.15.146:54534) 2024-09-13T18:27:03.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307affd700 1 -- 172.21.15.146:0/2503875537 learned_addr learned my addr 172.21.15.146:0/2503875537 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307a7fc700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f307c13b030 0x7f307c1259e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307b7fe700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f307c1275a0 0x7f307c127a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c0877700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74bc1539c0 0x7f74bc150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54516/0 (socket says 172.21.15.146:54516) 2024-09-13T18:27:03.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c0877700 1 -- 172.21.15.146:0/1350490566 learned_addr learned my addr 172.21.15.146:0/1350490566 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c1078700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74bc06ffd0 0x7f74bc1f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c1879700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74bc15a570 0x7f74bc1514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307affd700 1 -- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f307c13b030 msgr2=0x7f307c1259e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307affd700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f307c13b030 0x7f307c1259e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307affd700 1 -- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f307c1275a0 msgr2=0x7f307c127a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307affd700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f307c1275a0 0x7f307c127a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307affd700 1 -- 172.21.15.146:0/2503875537 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c13fdd0 con 0x7f3064005230 2024-09-13T18:27:03.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c0877700 1 -- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74bc06ffd0 msgr2=0x7f74bc1f68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c0877700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74bc06ffd0 0x7f74bc1f68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c0877700 1 -- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74bc15a570 msgr2=0x7f74bc1514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c0877700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74bc15a570 0x7f74bc1514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307b7fe700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f307c1275a0 0x7f307c127a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:03.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f307affd700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3064005230 0x7f307c0c36e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f306c004fc0 tx=0x7f306c007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c0877700 1 -- 172.21.15.146:0/1350490566 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b800b810 con 0x7f74bc1539c0 2024-09-13T18:27:03.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.065+0000 7f74c0877700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74bc1539c0 0x7f74bc150f90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f74b400cc20 tx=0x7f74b400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f563a7fc700 1 -- 172.21.15.146:0/4001348793 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5640094f60 con 0x7f56440c7660 2024-09-13T18:27:03.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f5649ccf700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5628066780 0x7f5628068c40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5634009b70 tx=0x7f5634008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f563a7fc700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] conn(0x7f562806fc50 0x7f5628072090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f563a7fc700 1 -- 172.21.15.146:0/4001348793 --> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5628072760 con 0x7f562806fc50 2024-09-13T18:27:03.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f563a7fc700 1 -- 172.21.15.146:0/4001348793 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5640097610 con 0x7f56440c7660 2024-09-13T18:27:03.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f56494ce700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] conn(0x7f562806fc50 0x7f5628072090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f74b1ffb700 1 -- 172.21.15.146:0/1350490566 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4003c20 con 0x7f74bc1539c0 2024-09-13T18:27:03.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f1baffff700 1 -- 172.21.15.146:0/938452976 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0007750 con 0x7f1bc80c6f50 2024-09-13T18:27:03.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f1baffff700 1 -- 172.21.15.146:0/938452976 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc00078b0 con 0x7f1bc80c6f50 2024-09-13T18:27:03.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/938452976 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc806ec50 con 0x7f1bc80c6f50 2024-09-13T18:27:03.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f56494ce700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] conn(0x7f562806fc50 0x7f5628072090 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f3063fff700 1 -- 172.21.15.146:0/2503875537 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c017070 con 0x7f3064005230 2024-09-13T18:27:03.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f3063fff700 1 -- 172.21.15.146:0/2503875537 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c011580 con 0x7f3064005230 2024-09-13T18:27:03.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f3063fff700 1 -- 172.21.15.146:0/2503875537 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c011730 con 0x7f3064005230 2024-09-13T18:27:03.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f74b1ffb700 1 -- 172.21.15.146:0/1350490566 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b4003d80 con 0x7f74bc1539c0 2024-09-13T18:27:03.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f74b1ffb700 1 -- 172.21.15.146:0/1350490566 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400aae0 con 0x7f74bc1539c0 2024-09-13T18:27:03.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f74c207a700 1 -- 172.21.15.146:0/1350490566 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74bc1542b0 con 0x7f74bc1539c0 2024-09-13T18:27:03.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f74c207a700 1 -- 172.21.15.146:0/1350490566 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74bc0c4780 con 0x7f74bc1539c0 2024-09-13T18:27:03.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7fc790ff9700 1 -- 172.21.15.146:0/851531462 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc7880036a0 con 0x7fc7940c9500 2024-09-13T18:27:03.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7fc790ff9700 1 --2- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc7800667b0 0x7fc780068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f563a7fc700 1 -- 172.21.15.146:0/4001348793 <== osd.2 v2:172.21.15.146:6800/3571336781 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f5628072760 con 0x7f562806fc50 2024-09-13T18:27:03.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7fc790ff9700 1 -- 172.21.15.146:0/851531462 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc788084000 con 0x7fc7940c9500 2024-09-13T18:27:03.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7fc790ff9700 1 --2- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] conn(0x7fc78006fdf0 0x7fc780072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/938452976 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc806cd10 con 0x7f1bc80c6f50 2024-09-13T18:27:03.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f1baffff700 1 -- 172.21.15.146:0/938452976 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00133f0 con 0x7f1bc80c6f50 2024-09-13T18:27:03.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7f1baffff700 1 -- 172.21.15.146:0/938452976 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1bc0013550 con 0x7f1bc80c6f50 2024-09-13T18:27:03.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7f1baffff700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1bb0068320 0x7f1bb006a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7f1baffff700 1 -- 172.21.15.146:0/938452976 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1bc0084ac0 con 0x7f1bc80c6f50 2024-09-13T18:27:03.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.077+0000 7f1bcc9ac700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] conn(0x7f1bc81ee110 0x7f1bc8153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.077+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/938452976 --> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1bc80c9890 con 0x7f1bc81ee110 2024-09-13T18:27:03.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.077+0000 7f1bc6ffd700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] conn(0x7f1bc81ee110 0x7f1bc8153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.077+0000 7f1bc6ffd700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] conn(0x7f1bc81ee110 0x7f1bc8153ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.077+0000 7f1baffff700 1 -- 172.21.15.146:0/938452976 <== osd.4 v2:172.21.15.154:6808/2872450676 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f1bbc00b950 con 0x7f1bc81ee110 2024-09-13T18:27:03.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7fc790ff9700 1 -- 172.21.15.146:0/851531462 --> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc780072900 con 0x7fc78006fdf0 2024-09-13T18:27:03.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7fc790ff9700 1 -- 172.21.15.146:0/851531462 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7880844a0 con 0x7fc7940c9500 2024-09-13T18:27:03.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7fc793fff700 1 --2- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc7800667b0 0x7fc780068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7fc798f8c700 1 --2- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] conn(0x7fc78006fdf0 0x7fc780072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7fc793fff700 1 --2- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc7800667b0 0x7fc780068c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc78c000f80 tx=0x7fc78c009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7fc798f8c700 1 --2- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] conn(0x7fc78006fdf0 0x7fc780072230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f307bfff700 1 -- 172.21.15.146:0/2503875537 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c13b5c0 con 0x7f3064005230 2024-09-13T18:27:03.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.069+0000 7f307bfff700 1 -- 172.21.15.146:0/2503875537 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c13bb80 con 0x7f3064005230 2024-09-13T18:27:03.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7f3063fff700 1 -- 172.21.15.146:0/2503875537 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f306c011890 con 0x7f3064005230 2024-09-13T18:27:03.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7f3063fff700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3064068260 0x7f306406a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7f3063fff700 1 -- 172.21.15.146:0/2503875537 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f306c083b40 con 0x7f3064005230 2024-09-13T18:27:03.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7f307a7fc700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3064068260 0x7f306406a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7f307a7fc700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3064068260 0x7f306406a720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f307c14df80 tx=0x7f30700078f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.081+0000 7f307bfff700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] conn(0x7f307c13e860 0x7f307c13ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.081+0000 7f307bfff700 1 -- 172.21.15.146:0/2503875537 --> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f307c0a96e0 con 0x7f307c13e860 2024-09-13T18:27:03.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7f74b1ffb700 1 -- 172.21.15.146:0/1350490566 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f74b400ac40 con 0x7f74bc1539c0 2024-09-13T18:27:03.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7f74b1ffb700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f74a0066730 0x7f74a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7f74b1ffb700 1 -- 172.21.15.146:0/1350490566 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f74b4084530 con 0x7f74bc1539c0 2024-09-13T18:27:03.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7f74c1078700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f74a0066730 0x7f74a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.073+0000 7f74c1078700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f74a0066730 0x7f74a0068bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009bf0 tx=0x7f74ac009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.077+0000 7f74c207a700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] conn(0x7f749c0015e0 0x7f749c003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.077+0000 7f74c207a700 1 -- 172.21.15.146:0/1350490566 --> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f749c006bf0 con 0x7f749c0015e0 2024-09-13T18:27:03.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.077+0000 7f74c1879700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] conn(0x7f749c0015e0 0x7f749c003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.077+0000 7f74c1879700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] conn(0x7f749c0015e0 0x7f749c003aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.077+0000 7f74b1ffb700 1 -- 172.21.15.146:0/1350490566 <== osd.7 v2:172.21.15.154:6817/3339245911 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f749c006bf0 con 0x7f749c0015e0 2024-09-13T18:27:03.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.081+0000 7f1bc77fe700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1bb0068320 0x7f1bb006a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.081+0000 7f1bc77fe700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1bb0068320 0x7f1bb006a7e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800aac0 tx=0x7f1bb8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.081+0000 7f1130c8f700 1 -- 172.21.15.146:0/3201064102 --> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f110c002d60 con 0x7f111806fc70 2024-09-13T18:27:03.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.081+0000 7f1113fff700 1 -- 172.21.15.146:0/3201064102 <== osd.3 v2:172.21.15.146:6824/3480660705 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f110c002d60 con 0x7f111806fc70 2024-09-13T18:27:03.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f621baca700 1 -- 172.21.15.146:0/3427901947 --> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f61fc002d60 con 0x7f61f806fd20 2024-09-13T18:27:03.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.081+0000 7fc790ff9700 1 -- 172.21.15.146:0/851531462 <== osd.5 v2:172.21.15.154:6800/363322763 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc780072900 con 0x7fc78006fdf0 2024-09-13T18:27:03.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f620a7fc700 1 -- 172.21.15.146:0/3427901947 <== osd.1 v2:172.21.15.146:6804/3907005797 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f61fc002d60 con 0x7f61f806fd20 2024-09-13T18:27:03.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f621baca700 1 -- 172.21.15.146:0/3427901947 >> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] conn(0x7f61f806fd20 msgr2=0x7f61f8072160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f621baca700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] conn(0x7f61f806fd20 0x7f61f8072160 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f621baca700 1 -- 172.21.15.146:0/3427901947 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61f8066760 msgr2=0x7f61f8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f621baca700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61f8066760 0x7f61f8068c20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6210009bf0 tx=0x7f6210009240 comp rx=0 tx=0).stop 2024-09-13T18:27:03.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f621baca700 1 -- 172.21.15.146:0/3427901947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62140cf450 msgr2=0x7f6214157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f621baca700 1 --2- 172.21.15.146:0/3427901947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62140cf450 0x7f6214157850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6204004000 tx=0x7f6204004770 comp rx=0 tx=0).stop 2024-09-13T18:27:03.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f621a067700 1 -- 172.21.15.146:0/3427901947 reap_dead start 2024-09-13T18:27:03.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f621baca700 1 -- 172.21.15.146:0/3427901947 shutdown_connections 2024-09-13T18:27:03.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f621baca700 1 -- 172.21.15.146:0/3427901947 >> 172.21.15.146:0/3427901947 conn(0x7f6214071d30 msgr2=0x7f62140c7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f621baca700 1 -- 172.21.15.146:0/3427901947 shutdown_connections 2024-09-13T18:27:03.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f621baca700 1 -- 172.21.15.146:0/3427901947 wait complete. 2024-09-13T18:27:03.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f307b7fe700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] conn(0x7f307c13e860 0x7f307c13ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f307b7fe700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] conn(0x7f307c13e860 0x7f307c13ec40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.089+0000 7f3063fff700 1 -- 172.21.15.146:0/2503875537 <== osd.6 v2:172.21.15.154:6816/2272517392 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f3074003a40 con 0x7f307c13e860 2024-09-13T18:27:03.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f1130c8f700 1 -- 172.21.15.146:0/3201064102 >> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] conn(0x7f111806fc70 msgr2=0x7f11180720b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f1130c8f700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] conn(0x7f111806fc70 0x7f11180720b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f1130c8f700 1 -- 172.21.15.146:0/3201064102 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1118066730 msgr2=0x7f1118068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f1130c8f700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1118066730 0x7f1118068bf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f11240012e0 tx=0x7f1124001a30 comp rx=0 tx=0).stop 2024-09-13T18:27:03.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f1130c8f700 1 -- 172.21.15.146:0/3201064102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f112c14f750 msgr2=0x7f112c14b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f1130c8f700 1 --2- 172.21.15.146:0/3201064102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f112c14f750 0x7f112c14b510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f111c009bd0 tx=0x7f111c004770 comp rx=0 tx=0).stop 2024-09-13T18:27:03.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f112bfff700 1 -- 172.21.15.146:0/3201064102 reap_dead start 2024-09-13T18:27:03.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f1130c8f700 1 -- 172.21.15.146:0/3201064102 shutdown_connections 2024-09-13T18:27:03.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f1130c8f700 1 -- 172.21.15.146:0/3201064102 >> 172.21.15.146:0/3201064102 conn(0x7f112c066090 msgr2=0x7f112c0adb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f1130c8f700 1 -- 172.21.15.146:0/3201064102 shutdown_connections 2024-09-13T18:27:03.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.085+0000 7f1130c8f700 1 -- 172.21.15.146:0/3201064102 wait complete. 2024-09-13T18:27:03.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f722248b700 1 -- 172.21.15.146:0/3186413088 --> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f721c0a98b0 con 0x7f720006fdf0 2024-09-13T18:27:03.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f7211ffb700 1 -- 172.21.15.146:0/3186413088 <== osd.0 v2:172.21.15.146:6816/2811789208 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f720c00a950 con 0x7f720006fdf0 2024-09-13T18:27:03.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f564bf33700 1 -- 172.21.15.146:0/4001348793 --> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5644159700 con 0x7f562806fc50 2024-09-13T18:27:03.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f722248b700 1 -- 172.21.15.146:0/3186413088 >> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] conn(0x7f720006fdf0 msgr2=0x7f7200072230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f722248b700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] conn(0x7f720006fdf0 0x7f7200072230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f722248b700 1 -- 172.21.15.146:0/3186413088 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f72000667b0 msgr2=0x7f7200068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f722248b700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f72000667b0 0x7f7200068c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f721400b370 tx=0x7f7214009000 comp rx=0 tx=0).stop 2024-09-13T18:27:03.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f722248b700 1 -- 172.21.15.146:0/3186413088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f721c0aa2a0 msgr2=0x7f721c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f722248b700 1 --2- 172.21.15.146:0/3186413088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f721c0aa2a0 0x7f721c15e480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f721800c770 tx=0x7f721800ca80 comp rx=0 tx=0).stop 2024-09-13T18:27:03.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f7221c8a700 1 -- 172.21.15.146:0/3186413088 reap_dead start 2024-09-13T18:27:03.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f563a7fc700 1 -- 172.21.15.146:0/4001348793 <== osd.2 v2:172.21.15.146:6800/3571336781 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f563c00f040 con 0x7f562806fc50 2024-09-13T18:27:03.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f722248b700 1 -- 172.21.15.146:0/3186413088 shutdown_connections 2024-09-13T18:27:03.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f722248b700 1 -- 172.21.15.146:0/3186413088 >> 172.21.15.146:0/3186413088 conn(0x7f721c066090 msgr2=0x7f721c15a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f722248b700 1 -- 172.21.15.146:0/3186413088 shutdown_connections 2024-09-13T18:27:03.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f722248b700 1 -- 172.21.15.146:0/3186413088 wait complete. 2024-09-13T18:27:03.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f5627fff700 1 -- 172.21.15.146:0/4001348793 >> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] conn(0x7f562806fc50 msgr2=0x7f5628072090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f5627fff700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] conn(0x7f562806fc50 0x7f5628072090 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f5627fff700 1 -- 172.21.15.146:0/4001348793 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5628066780 msgr2=0x7f5628068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f5627fff700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5628066780 0x7f5628068c40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5634009b70 tx=0x7f5634008030 comp rx=0 tx=0).stop 2024-09-13T18:27:03.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f5627fff700 1 -- 172.21.15.146:0/4001348793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56440c7660 msgr2=0x7f564414b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f5627fff700 1 --2- 172.21.15.146:0/4001348793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56440c7660 0x7f564414b660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f564000afd0 tx=0x7f564001d930 comp rx=0 tx=0).stop 2024-09-13T18:27:03.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f564a4d0700 1 -- 172.21.15.146:0/4001348793 reap_dead start 2024-09-13T18:27:03.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f5627fff700 1 -- 172.21.15.146:0/4001348793 shutdown_connections 2024-09-13T18:27:03.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f5627fff700 1 -- 172.21.15.146:0/4001348793 >> 172.21.15.146:0/4001348793 conn(0x7f564406f5c0 msgr2=0x7f56440b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f5627fff700 1 -- 172.21.15.146:0/4001348793 shutdown_connections 2024-09-13T18:27:03.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.097+0000 7f5627fff700 1 -- 172.21.15.146:0/4001348793 wait complete. 2024-09-13T18:27:03.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.105+0000 7f74c207a700 1 -- 172.21.15.146:0/1350490566 --> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f749c005c70 con 0x7f749c0015e0 2024-09-13T18:27:03.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.105+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/938452976 --> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1bc80c94e0 con 0x7f1bc81ee110 2024-09-13T18:27:03.150 INFO:teuthology.orchestra.run.smithi146.stdout:55834574857 2024-09-13T18:27:03.150 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:27:03.151 INFO:teuthology.orchestra.run.smithi146.stdout:60129542154 2024-09-13T18:27:03.151 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:27:03.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.105+0000 7f1baffff700 1 -- 172.21.15.146:0/938452976 <== osd.4 v2:172.21.15.154:6808/2872450676 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1bbc00bac0 con 0x7f1bc81ee110 2024-09-13T18:27:03.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f74b1ffb700 1 -- 172.21.15.146:0/1350490566 <== osd.7 v2:172.21.15.154:6817/3339245911 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f749c005c70 con 0x7f749c0015e0 2024-09-13T18:27:03.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] conn(0x7f1bc81ee110 msgr2=0x7f1bc8153ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f1bcc9ac700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] conn(0x7f1bc81ee110 0x7f1bc8153ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1bb0068320 msgr2=0x7f1bb006a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f1bcc9ac700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1bb0068320 0x7f1bb006a7e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800aac0 tx=0x7f1bb8008040 comp rx=0 tx=0).stop 2024-09-13T18:27:03.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/938452976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bc80c6f50 msgr2=0x7f1bc8145c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f1bcc9ac700 1 --2- 172.21.15.146:0/938452976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bc80c6f50 0x7f1bc8145c20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8071a10 tx=0x7f1bc000af10 comp rx=0 tx=0).stop 2024-09-13T18:27:03.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f1bc7fff700 1 -- 172.21.15.146:0/938452976 reap_dead start 2024-09-13T18:27:03.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/938452976 shutdown_connections 2024-09-13T18:27:03.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/938452976 >> 172.21.15.146:0/938452976 conn(0x7f1bc8066090 msgr2=0x7f1bc8154cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/938452976 shutdown_connections 2024-09-13T18:27:03.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f1bcc9ac700 1 -- 172.21.15.146:0/938452976 wait complete. 2024-09-13T18:27:03.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7fc79978d700 1 -- 172.21.15.146:0/851531462 --> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc7940c7e50 con 0x7fc78006fdf0 2024-09-13T18:27:03.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f307bfff700 1 -- 172.21.15.146:0/2503875537 --> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f307c0cb530 con 0x7f307c13e860 2024-09-13T18:27:03.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7fc790ff9700 1 -- 172.21.15.146:0/851531462 <== osd.5 v2:172.21.15.154:6800/363322763 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc7840096d0 con 0x7fc78006fdf0 2024-09-13T18:27:03.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f3063fff700 1 -- 172.21.15.146:0/2503875537 <== osd.6 v2:172.21.15.154:6816/2272517392 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f307400ea00 con 0x7f307c13e860 2024-09-13T18:27:03.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f307bfff700 1 -- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] conn(0x7f307c13e860 msgr2=0x7f307c13ec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.109+0000 7f307bfff700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] conn(0x7f307c13e860 0x7f307c13ec40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f307bfff700 1 -- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3064068260 msgr2=0x7f306406a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f307bfff700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3064068260 0x7f306406a720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f307c14df80 tx=0x7f30700078f0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f307bfff700 1 -- 172.21.15.146:0/2503875537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3064005230 msgr2=0x7f307c0c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f307bfff700 1 --2- 172.21.15.146:0/2503875537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3064005230 0x7f307c0c36e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f306c004fc0 tx=0x7f306c007340 comp rx=0 tx=0).stop 2024-09-13T18:27:03.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f307b7fe700 1 -- 172.21.15.146:0/2503875537 reap_dead start 2024-09-13T18:27:03.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f74c207a700 1 -- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] conn(0x7f749c0015e0 msgr2=0x7f749c003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f74c207a700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] conn(0x7f749c0015e0 0x7f749c003aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f307bfff700 1 -- 172.21.15.146:0/2503875537 shutdown_connections 2024-09-13T18:27:03.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f307bfff700 1 -- 172.21.15.146:0/2503875537 >> 172.21.15.146:0/2503875537 conn(0x7f307c066090 msgr2=0x7f307c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f74c207a700 1 -- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f74a0066730 msgr2=0x7f74a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f74c207a700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f74a0066730 0x7f74a0068bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009bf0 tx=0x7f74ac009240 comp rx=0 tx=0).stop 2024-09-13T18:27:03.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f74c207a700 1 -- 172.21.15.146:0/1350490566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74bc1539c0 msgr2=0x7f74bc150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f74c207a700 1 --2- 172.21.15.146:0/1350490566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74bc1539c0 0x7f74bc150f90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f74b400cc20 tx=0x7f74b400cf30 comp rx=0 tx=0).stop 2024-09-13T18:27:03.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f74c1879700 1 -- 172.21.15.146:0/1350490566 reap_dead start 2024-09-13T18:27:03.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f307bfff700 1 -- 172.21.15.146:0/2503875537 shutdown_connections 2024-09-13T18:27:03.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f307bfff700 1 -- 172.21.15.146:0/2503875537 wait complete. 2024-09-13T18:27:03.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f74c207a700 1 -- 172.21.15.146:0/1350490566 shutdown_connections 2024-09-13T18:27:03.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f74c207a700 1 -- 172.21.15.146:0/1350490566 >> 172.21.15.146:0/1350490566 conn(0x7f74bc066090 msgr2=0x7f74bc066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f74c207a700 1 -- 172.21.15.146:0/1350490566 shutdown_connections 2024-09-13T18:27:03.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.113+0000 7f74c207a700 1 -- 172.21.15.146:0/1350490566 wait complete. 2024-09-13T18:27:03.159 INFO:teuthology.orchestra.run.smithi146.stdout:55834574857 2024-09-13T18:27:03.159 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:27:03.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.117+0000 7fc79978d700 1 -- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] conn(0x7fc78006fdf0 msgr2=0x7fc780072230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.117+0000 7fc79978d700 1 --2- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] conn(0x7fc78006fdf0 0x7fc780072230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.117+0000 7fc79978d700 1 -- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc7800667b0 msgr2=0x7fc780068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.117+0000 7fc79978d700 1 --2- 172.21.15.146:0/851531462 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc7800667b0 0x7fc780068c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc78c000f80 tx=0x7fc78c009000 comp rx=0 tx=0).stop 2024-09-13T18:27:03.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.117+0000 7fc79978d700 1 -- 172.21.15.146:0/851531462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940c9500 msgr2=0x7fc7940cb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.117+0000 7fc79978d700 1 --2- 172.21.15.146:0/851531462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940c9500 0x7fc7940cb3d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc78800c530 tx=0x7fc78800c8f0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.117+0000 7fc798f8c700 1 -- 172.21.15.146:0/851531462 reap_dead start 2024-09-13T18:27:03.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.117+0000 7fc79978d700 1 -- 172.21.15.146:0/851531462 shutdown_connections 2024-09-13T18:27:03.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.117+0000 7fc79978d700 1 -- 172.21.15.146:0/851531462 >> 172.21.15.146:0/851531462 conn(0x7fc794066090 msgr2=0x7fc794066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.117+0000 7fc79978d700 1 -- 172.21.15.146:0/851531462 shutdown_connections 2024-09-13T18:27:03.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.117+0000 7fc79978d700 1 -- 172.21.15.146:0/851531462 wait complete. 2024-09-13T18:27:03.162 INFO:teuthology.orchestra.run.smithi146.stdout:60129542155 2024-09-13T18:27:03.162 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:27:03.163 INFO:teuthology.orchestra.run.smithi146.stdout:64424509449 2024-09-13T18:27:03.163 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T18:27:03.164 INFO:teuthology.orchestra.run.smithi146.stdout:64424509450 2024-09-13T18:27:03.164 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T18:27:03.165 INFO:teuthology.orchestra.run.smithi146.stdout:64424509449 2024-09-13T18:27:03.165 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T18:27:03.166 INFO:teuthology.orchestra.run.smithi146.stdout:55834574858 2024-09-13T18:27:03.166 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T18:27:03.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.337+0000 7f1590a58700 1 Processor -- start 2024-09-13T18:27:03.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.337+0000 7f1590a58700 1 -- start start 2024-09-13T18:27:03.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.337+0000 7f3f6d8e1700 1 Processor -- start 2024-09-13T18:27:03.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.337+0000 7fea46cba700 1 Processor -- start 2024-09-13T18:27:03.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.337+0000 7f3f6d8e1700 1 -- start start 2024-09-13T18:27:03.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.337+0000 7fea46cba700 1 -- start start 2024-09-13T18:27:03.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.337+0000 7f7a8dc64700 1 Processor -- start 2024-09-13T18:27:03.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.337+0000 7f7a8dc64700 1 -- start start 2024-09-13T18:27:03.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f1590a58700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f158c0a9490 0x7f158c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f1590a58700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c0c8650 con 0x7f158c0c8f70 2024-09-13T18:27:03.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f1590a58700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c0aa1a0 con 0x7f158c0c9350 2024-09-13T18:27:03.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f1590a58700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c0aa480 con 0x7f158c0a9490 2024-09-13T18:27:03.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158a59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f158c0c8f70 0x7f158c0cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35460/0 (socket says 172.21.15.146:35460) 2024-09-13T18:27:03.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158a59c700 1 -- 172.21.15.146:0/786902053 learned_addr learned my addr 172.21.15.146:0/786902053 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:03.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.337+0000 7f5447fff700 1 Processor -- start 2024-09-13T18:27:03.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.337+0000 7f5447fff700 1 -- start start 2024-09-13T18:27:03.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea46cba700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fea400a9490 0x7fea4006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea46cba700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fea4014f750 0x7fea4006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea46cba700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea4006fb00 0x7fea400c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea46cba700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4014fb90 con 0x7fea400a9490 2024-09-13T18:27:03.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea46cba700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4006ff90 con 0x7fea4014f750 2024-09-13T18:27:03.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea46cba700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea40155580 con 0x7fea4006fb00 2024-09-13T18:27:03.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158ad9d700 1 --2- 172.21.15.146:0/786902053 >> v2:172.21.15.154:3300/0 conn(0x7f158c0a9490 0x7f158c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158959a700 1 -- 172.21.15.146:0/786902053 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693930677 0 0) 0x7f158c0c8650 con 0x7f158c0c8f70 2024-09-13T18:27:03.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158959a700 1 -- 172.21.15.146:0/786902053 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15740035d0 con 0x7f158c0c8f70 2024-09-13T18:27:03.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158959a700 1 -- 172.21.15.146:0/786902053 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758477040 0 0) 0x7f158c0aa1a0 con 0x7f158c0c9350 2024-09-13T18:27:03.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158959a700 1 -- 172.21.15.146:0/786902053 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f158c0c8650 con 0x7f158c0c9350 2024-09-13T18:27:03.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea45257700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea4006fb00 0x7fea400c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea45257700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea4006fb00 0x7fea400c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41218/0 (socket says 172.21.15.146:41218) 2024-09-13T18:27:03.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea45257700 1 -- 172.21.15.146:0/2511067453 learned_addr learned my addr 172.21.15.146:0/2511067453 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea44a56700 1 --2- 172.21.15.146:0/2511067453 >> v2:172.21.15.146:3300/0 conn(0x7fea400a9490 0x7fea4006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f6d8e1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3f680c7360 0x7f3f681612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f6d8e1700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f68158fa0 con 0x7f3f680c0510 2024-09-13T18:27:03.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f667fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f3f680c0510 0x7f3f68155a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35466/0 (socket says 172.21.15.146:35466) 2024-09-13T18:27:03.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f667fc700 1 -- 172.21.15.146:0/2846945149 learned_addr learned my addr 172.21.15.146:0/2846945149 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:03.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f6d8e1700 1 -- 172.21.15.146:0/2846945149 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f680c83f0 con 0x7f3f680700b0 2024-09-13T18:27:03.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f6d8e1700 1 -- 172.21.15.146:0/2846945149 --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f68156140 con 0x7f3f680c7360 2024-09-13T18:27:03.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f677fe700 1 --2- 172.21.15.146:0/2846945149 >> v2:172.21.15.146:3300/0 conn(0x7f3f680c7360 0x7f3f681612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158ad9d700 1 -- 172.21.15.146:0/786902053 >> v1:172.21.15.154:6789/0 conn(0x7f158c0c9350 legacy=0x7f158c162010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158ad9d700 1 -- 172.21.15.146:0/786902053 >> v1:172.21.15.146:6789/0 conn(0x7f158c0c8f70 legacy=0x7f158c0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f5447fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806eec0 0x7f54480caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f5447fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f544814f650 0x7f544814fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f5447fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544806fef0 con 0x7f544806c500 2024-09-13T18:27:03.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f5447fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54480ca1e0 con 0x7f544814f650 2024-09-13T18:27:03.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f5447fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544806f2c0 con 0x7f544806eec0 2024-09-13T18:27:03.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f65ffb700 1 -- 172.21.15.146:0/2846945149 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516688424 0 0) 0x7f3f68158fa0 con 0x7f3f680c0510 2024-09-13T18:27:03.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158ad9d700 1 -- 172.21.15.146:0/786902053 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c159890 con 0x7f158c0a9490 2024-09-13T18:27:03.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea3ffff700 1 --2- 172.21.15.146:0/2511067453 >> v2:172.21.15.154:3300/0 conn(0x7fea4014f750 0x7fea4006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f65ffb700 1 -- 172.21.15.146:0/2846945149 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f500035d0 con 0x7f3f680c0510 2024-09-13T18:27:03.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7ff0ad858700 1 Processor -- start 2024-09-13T18:27:03.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f5446ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f544806c500 0x7f5448072460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35482/0 (socket says 172.21.15.146:35482) 2024-09-13T18:27:03.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f5446ffd700 1 -- 172.21.15.146:0/1463971973 learned_addr learned my addr 172.21.15.146:0/1463971973 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:03.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f54477fe700 1 --2- 172.21.15.146:0/1463971973 >> v2:172.21.15.154:3300/0 conn(0x7f544814f650 0x7f544814fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7ff0ad858700 1 -- start start 2024-09-13T18:27:03.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea44a56700 1 -- 172.21.15.146:0/2511067453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea4006fb00 msgr2=0x7fea400c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea44a56700 1 --2- 172.21.15.146:0/2511067453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea4006fb00 0x7fea400c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea44a56700 1 -- 172.21.15.146:0/2511067453 >> v2:172.21.15.154:3300/0 conn(0x7fea4014f750 msgr2=0x7fea4006f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea44a56700 1 --2- 172.21.15.146:0/2511067453 >> v2:172.21.15.154:3300/0 conn(0x7fea4014f750 0x7fea4006f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea44a56700 1 -- 172.21.15.146:0/2511067453 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea40155860 con 0x7fea400a9490 2024-09-13T18:27:03.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f677fe700 1 -- 172.21.15.146:0/2846945149 >> v1:172.21.15.154:6789/0 conn(0x7f3f680700b0 legacy=0x7f3f68152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f677fe700 1 -- 172.21.15.146:0/2846945149 >> v1:172.21.15.146:6789/0 conn(0x7f3f680c0510 legacy=0x7f3f68155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f677fe700 1 -- 172.21.15.146:0/2846945149 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f680c6230 con 0x7f3f680c7360 2024-09-13T18:27:03.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f65ffb700 1 -- 172.21.15.146:0/2846945149 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629470864 0 0) 0x7f3f680c83f0 con 0x7f3f680700b0 2024-09-13T18:27:03.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158ad9d700 1 --2- 172.21.15.146:0/786902053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f158c0a9490 0x7f158c15a370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f157c009850 tx=0x7f157c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2753bd363807ff3f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f54467fc700 1 --2- 172.21.15.146:0/1463971973 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806eec0 0x7f54480caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f677fe700 1 --2- 172.21.15.146:0/2846945149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f680c7360 0x7f3f681612e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000bfa0 tx=0x7f3f6000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282b3c6dbc6c2ca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158959a700 1 -- 172.21.15.146:0/786902053 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c011830 con 0x7f158c0a9490 2024-09-13T18:27:03.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea44a56700 1 --2- 172.21.15.146:0/2511067453 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea400a9490 0x7fea4006eea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fea400cbc30 tx=0x7fea34009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1ee42b8db2435c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a8dc64700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a8815b380 0x7f7a88157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f5445ffb700 1 -- 172.21.15.146:0/1463971973 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495775620 0 0) 0x7f544806fef0 con 0x7f544806c500 2024-09-13T18:27:03.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f54477fe700 1 -- 172.21.15.146:0/1463971973 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806eec0 msgr2=0x7f54480caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f54477fe700 1 --2- 172.21.15.146:0/1463971973 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806eec0 0x7f54480caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f54477fe700 1 -- 172.21.15.146:0/1463971973 >> v1:172.21.15.146:6789/0 conn(0x7f544806c500 legacy=0x7f5448072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f54477fe700 1 -- 172.21.15.146:0/1463971973 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5448070790 con 0x7f544814f650 2024-09-13T18:27:03.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a8dc64700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7a88158030 0x7f7a881544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158959a700 1 -- 172.21.15.146:0/786902053 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c011990 con 0x7f158c0a9490 2024-09-13T18:27:03.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f158959a700 1 -- 172.21.15.146:0/786902053 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c011d00 con 0x7f158c0a9490 2024-09-13T18:27:03.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a877fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7a880cab20 0x7f7a880caf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:54934/0 (socket says 172.21.15.146:54934) 2024-09-13T18:27:03.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a877fe700 1 -- 172.21.15.146:0/2388378724 learned_addr learned my addr 172.21.15.146:0/2388378724 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:03.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a86ffd700 1 --2- 172.21.15.146:0/2388378724 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a8815b380 0x7f7a88157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a8dc64700 1 -- 172.21.15.146:0/2388378724 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a880a97e0 con 0x7f7a880cab20 2024-09-13T18:27:03.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea3f7fe700 1 -- 172.21.15.146:0/2511067453 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34011830 con 0x7fea400a9490 2024-09-13T18:27:03.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a8dc64700 1 -- 172.21.15.146:0/2388378724 --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a880c99a0 con 0x7f7a88158030 2024-09-13T18:27:03.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a8dc64700 1 -- 172.21.15.146:0/2388378724 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880723d0 con 0x7f7a8815b380 2024-09-13T18:27:03.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f54477fe700 1 --2- 172.21.15.146:0/1463971973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544814f650 0x7f544814fb00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5440009d90 tx=0x7f544000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bef615f1206626a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a87fff700 1 --2- 172.21.15.146:0/2388378724 >> v2:172.21.15.146:3300/0 conn(0x7f7a88158030 0x7f7a881544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f65ffb700 1 -- 172.21.15.146:0/2846945149 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60010070 con 0x7f3f680c7360 2024-09-13T18:27:03.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7ff0ad858700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff0a8159b70 0x7ff0a80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7ff0ad858700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff0a8070500 0x7ff0a80c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7ff0ad858700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a80c94a0 con 0x7ff0a80aa360 2024-09-13T18:27:03.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f65ffb700 1 -- 172.21.15.146:0/2846945149 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6000da00 con 0x7f3f680c7360 2024-09-13T18:27:03.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f3f65ffb700 1 -- 172.21.15.146:0/2846945149 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000dd90 con 0x7f3f680c7360 2024-09-13T18:27:03.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7ff0ad858700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a80c7df0 con 0x7ff0a8070500 2024-09-13T18:27:03.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7ff0ad858700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a80c8090 con 0x7ff0a8159b70 2024-09-13T18:27:03.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a87fff700 1 -- 172.21.15.146:0/2388378724 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a8815b380 msgr2=0x7f7a88157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a87fff700 1 --2- 172.21.15.146:0/2388378724 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a8815b380 0x7f7a88157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a87fff700 1 -- 172.21.15.146:0/2388378724 >> v1:172.21.15.154:6789/0 conn(0x7f7a880cab20 legacy=0x7f7a880caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a87fff700 1 -- 172.21.15.146:0/2388378724 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a880c9370 con 0x7f7a88158030 2024-09-13T18:27:03.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a86ffd700 1 --2- 172.21.15.146:0/2388378724 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a8815b380 0x7f7a88157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:03.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7ff0ad057700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff0a80aa360 0x7ff0a80a9620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35492/0 (socket says 172.21.15.146:35492) 2024-09-13T18:27:03.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7ff0ad057700 1 -- 172.21.15.146:0/2659410884 learned_addr learned my addr 172.21.15.146:0/2659410884 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:03.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7ff0ac856700 1 --2- 172.21.15.146:0/2659410884 >> v2:172.21.15.154:3300/0 conn(0x7ff0a8159b70 0x7ff0a80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f7a87fff700 1 --2- 172.21.15.146:0/2388378724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a88158030 0x7f7a881544a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009850 tx=0x7f7a78009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d112d644931a643f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f1590a58700 1 -- 172.21.15.146:0/786902053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f158c0a9490 msgr2=0x7f158c15a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7ff0a7fff700 1 --2- 172.21.15.146:0/2659410884 >> v2:172.21.15.146:3300/0 conn(0x7ff0a8070500 0x7ff0a80c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f1590a58700 1 --2- 172.21.15.146:0/786902053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f158c0a9490 0x7f158c15a370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f157c009850 tx=0x7f157c009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:03.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f5445ffb700 1 -- 172.21.15.146:0/1463971973 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544000e070 con 0x7f544814f650 2024-09-13T18:27:03.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7f5445ffb700 1 -- 172.21.15.146:0/1463971973 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544000bbd0 con 0x7f544814f650 2024-09-13T18:27:03.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea3f7fe700 1 -- 172.21.15.146:0/2511067453 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea34011990 con 0x7fea400a9490 2024-09-13T18:27:03.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea3f7fe700 1 -- 172.21.15.146:0/2511067453 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34011d20 con 0x7fea400a9490 2024-09-13T18:27:03.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea46cba700 1 -- 172.21.15.146:0/2511067453 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea400a9490 msgr2=0x7fea4006eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.341+0000 7fea46cba700 1 --2- 172.21.15.146:0/2511067453 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea400a9490 0x7fea4006eea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fea400cbc30 tx=0x7fea34009b30 comp rx=0 tx=0).stop 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 -- 172.21.15.146:0/2511067453 shutdown_connections 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 --2- 172.21.15.146:0/2511067453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea4006fb00 0x7fea400c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 --2- 172.21.15.146:0/2511067453 >> v2:172.21.15.154:3300/0 conn(0x7fea4014f750 0x7fea4006f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 --2- 172.21.15.146:0/2511067453 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea400a9490 0x7fea4006eea0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 -- 172.21.15.146:0/2511067453 >> 172.21.15.146:0/2511067453 conn(0x7fea40066090 msgr2=0x7fea40066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0a7fff700 1 -- 172.21.15.146:0/2659410884 >> v2:172.21.15.154:3300/0 conn(0x7ff0a8159b70 msgr2=0x7ff0a80c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0a7fff700 1 --2- 172.21.15.146:0/2659410884 >> v2:172.21.15.154:3300/0 conn(0x7ff0a8159b70 0x7ff0a80c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0a7fff700 1 -- 172.21.15.146:0/2659410884 >> v1:172.21.15.146:6789/0 conn(0x7ff0a80aa360 legacy=0x7ff0a80a9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f7a867fc700 1 -- 172.21.15.146:0/2388378724 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78011830 con 0x7f7a88158030 2024-09-13T18:27:03.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f7a867fc700 1 -- 172.21.15.146:0/2388378724 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a78011990 con 0x7f7a88158030 2024-09-13T18:27:03.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f7a867fc700 1 -- 172.21.15.146:0/2388378724 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78011d20 con 0x7f7a88158030 2024-09-13T18:27:03.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0a7fff700 1 -- 172.21.15.146:0/2659410884 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a80c8b70 con 0x7ff0a8070500 2024-09-13T18:27:03.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f3f6d8e1700 1 -- 172.21.15.146:0/2846945149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f680c7360 msgr2=0x7f3f681612e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f3f6d8e1700 1 --2- 172.21.15.146:0/2846945149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f680c7360 0x7f3f681612e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000bfa0 tx=0x7f3f6000d330 comp rx=0 tx=0).stop 2024-09-13T18:27:03.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0a7fff700 1 --2- 172.21.15.146:0/2659410884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0a8070500 0x7ff0a80c5d70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8072a50 tx=0x7ff09c00acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bd4481d6c5c496 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f3f6d8e1700 1 -- 172.21.15.146:0/2846945149 shutdown_connections 2024-09-13T18:27:03.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f5447fff700 1 -- 172.21.15.146:0/1463971973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544814f650 msgr2=0x7f544814fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f5447fff700 1 --2- 172.21.15.146:0/1463971973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544814f650 0x7f544814fb00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5440009d90 tx=0x7f544000b330 comp rx=0 tx=0).stop 2024-09-13T18:27:03.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0a77fe700 1 -- 172.21.15.146:0/2659410884 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00c970 con 0x7ff0a8070500 2024-09-13T18:27:03.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f3f6d8e1700 1 --2- 172.21.15.146:0/2846945149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f680c7360 0x7f3f681612e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 -- 172.21.15.146:0/2511067453 shutdown_connections 2024-09-13T18:27:03.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f3f6d8e1700 1 -- 172.21.15.146:0/2846945149 >> 172.21.15.146:0/2846945149 conn(0x7f3f6806f5c0 msgr2=0x7f3f6806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f1590a58700 1 -- 172.21.15.146:0/786902053 shutdown_connections 2024-09-13T18:27:03.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f1590a58700 1 --2- 172.21.15.146:0/786902053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f158c0a9490 0x7f158c15a370 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f1590a58700 1 -- 172.21.15.146:0/786902053 >> 172.21.15.146:0/786902053 conn(0x7f158c066090 msgr2=0x7f158c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f5447fff700 1 -- 172.21.15.146:0/1463971973 shutdown_connections 2024-09-13T18:27:03.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f5447fff700 1 --2- 172.21.15.146:0/1463971973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544814f650 0x7f544814fb00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f5447fff700 1 --2- 172.21.15.146:0/1463971973 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806eec0 0x7f54480caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f5447fff700 1 -- 172.21.15.146:0/1463971973 >> 172.21.15.146:0/1463971973 conn(0x7f5448066090 msgr2=0x7f5448066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 -- 172.21.15.146:0/2511067453 wait complete. 2024-09-13T18:27:03.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0a77fe700 1 -- 172.21.15.146:0/2659410884 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c00cad0 con 0x7ff0a8070500 2024-09-13T18:27:03.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0a77fe700 1 -- 172.21.15.146:0/2659410884 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00ce60 con 0x7ff0a8070500 2024-09-13T18:27:03.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f7a8dc64700 1 -- 172.21.15.146:0/2388378724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a88158030 msgr2=0x7f7a881544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f7a8dc64700 1 --2- 172.21.15.146:0/2388378724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a88158030 0x7f7a881544a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009850 tx=0x7f7a78009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:03.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f7a8dc64700 1 -- 172.21.15.146:0/2388378724 shutdown_connections 2024-09-13T18:27:03.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f7a8dc64700 1 --2- 172.21.15.146:0/2388378724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a88158030 0x7f7a881544a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f7a8dc64700 1 --2- 172.21.15.146:0/2388378724 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a8815b380 0x7f7a88157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f7a8dc64700 1 -- 172.21.15.146:0/2388378724 >> 172.21.15.146:0/2388378724 conn(0x7f7a88066090 msgr2=0x7f7a88066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f7a8dc64700 1 -- 172.21.15.146:0/2388378724 shutdown_connections 2024-09-13T18:27:03.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f7a8dc64700 1 -- 172.21.15.146:0/2388378724 wait complete. 2024-09-13T18:27:03.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0ad858700 1 -- 172.21.15.146:0/2659410884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0a8070500 msgr2=0x7ff0a80c5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0ad858700 1 --2- 172.21.15.146:0/2659410884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0a8070500 0x7ff0a80c5d70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8072a50 tx=0x7ff09c00acd0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0ad858700 1 -- 172.21.15.146:0/2659410884 shutdown_connections 2024-09-13T18:27:03.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0ad858700 1 --2- 172.21.15.146:0/2659410884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0a8070500 0x7ff0a80c5d70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0ad858700 1 --2- 172.21.15.146:0/2659410884 >> v2:172.21.15.154:3300/0 conn(0x7ff0a8159b70 0x7ff0a80c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0ad858700 1 -- 172.21.15.146:0/2659410884 >> 172.21.15.146:0/2659410884 conn(0x7ff0a8066090 msgr2=0x7ff0a8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0ad858700 1 -- 172.21.15.146:0/2659410884 shutdown_connections 2024-09-13T18:27:03.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0ad858700 1 -- 172.21.15.146:0/2659410884 wait complete. 2024-09-13T18:27:03.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f5447fff700 1 -- 172.21.15.146:0/1463971973 shutdown_connections 2024-09-13T18:27:03.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f869f116700 1 Processor -- start 2024-09-13T18:27:03.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f3f6d8e1700 1 -- 172.21.15.146:0/2846945149 shutdown_connections 2024-09-13T18:27:03.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f3f6d8e1700 1 -- 172.21.15.146:0/2846945149 wait complete. 2024-09-13T18:27:03.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f869f116700 1 -- start start 2024-09-13T18:27:03.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 Processor -- start 2024-09-13T18:27:03.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f1590a58700 1 -- 172.21.15.146:0/786902053 shutdown_connections 2024-09-13T18:27:03.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 -- start start 2024-09-13T18:27:03.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f7a8dc64700 1 Processor -- start 2024-09-13T18:27:03.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f7a8dc64700 1 -- start start 2024-09-13T18:27:03.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea4006fb00 0x7fea401526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea400a9490 0x7fea400abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea4014f750 0x7fea400ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4014da30 con 0x7fea4006fb00 2024-09-13T18:27:03.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea400ab810 con 0x7fea400a9490 2024-09-13T18:27:03.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea46cba700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea400aab90 con 0x7fea4014f750 2024-09-13T18:27:03.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0ad858700 1 Processor -- start 2024-09-13T18:27:03.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f5447fff700 1 -- 172.21.15.146:0/1463971973 wait complete. 2024-09-13T18:27:03.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f869f116700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f86980c6960 0x7f86980c0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f869f116700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8698071640 0x7f86980c14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f869f116700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86980cc1a0 con 0x7f8698157de0 2024-09-13T18:27:03.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f869f116700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869806f240 con 0x7f8698071640 2024-09-13T18:27:03.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f869f116700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869806f5c0 con 0x7f86980c6960 2024-09-13T18:27:03.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7f1590a58700 1 -- 172.21.15.146:0/786902053 wait complete. 2024-09-13T18:27:03.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea45257700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea4014f750 0x7fea400ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea45257700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea4014f750 0x7fea400ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41240/0 (socket says 172.21.15.146:41240) 2024-09-13T18:27:03.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7fea45257700 1 -- 172.21.15.146:0/1263341439 learned_addr learned my addr 172.21.15.146:0/1263341439 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a8dc64700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a880cab20 0x7f7a880add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a8dc64700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a88158030 0x7f7a880c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a8dc64700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a8815b380 0x7f7a880c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a8dc64700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880c17f0 con 0x7f7a8815b380 2024-09-13T18:27:03.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a8dc64700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a881567b0 con 0x7f7a880cab20 2024-09-13T18:27:03.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a8dc64700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880af7a0 con 0x7f7a88158030 2024-09-13T18:27:03.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f869e915700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8698071640 0x7f86980c14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f869e915700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8698071640 0x7f86980c14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:46574/0 (socket says 172.21.15.146:46574) 2024-09-13T18:27:03.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f869e915700 1 -- 172.21.15.146:0/1988395395 learned_addr learned my addr 172.21.15.146:0/1988395395 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.345+0000 7ff0ad858700 1 -- start start 2024-09-13T18:27:03.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f6d8e1700 1 Processor -- start 2024-09-13T18:27:03.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f869d913700 1 --2- 172.21.15.146:0/1988395395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f86980c6960 0x7f86980c0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a877fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a880cab20 0x7f7a880add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7fea3ffff700 1 --2- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea400a9490 0x7fea400abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f6d8e1700 1 -- start start 2024-09-13T18:27:03.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7fea45257700 1 -- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea400a9490 msgr2=0x7fea400abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7fea45257700 1 --2- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea400a9490 0x7fea400abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a87fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a8815b380 0x7f7a880c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a87fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a8815b380 0x7f7a880c10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54596/0 (socket says 172.21.15.146:54596) 2024-09-13T18:27:03.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a87fff700 1 -- 172.21.15.146:0/2902988619 learned_addr learned my addr 172.21.15.146:0/2902988619 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7fea45257700 1 -- 172.21.15.146:0/1263341439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea4006fb00 msgr2=0x7fea401526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7fea45257700 1 --2- 172.21.15.146:0/1263341439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea4006fb00 0x7fea401526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7fea45257700 1 -- 172.21.15.146:0/1263341439 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea340094b0 con 0x7fea4014f750 2024-09-13T18:27:03.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f5447fff700 1 Processor -- start 2024-09-13T18:27:03.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f869e915700 1 -- 172.21.15.146:0/1988395395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f86980c6960 msgr2=0x7f86980c0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f869e915700 1 --2- 172.21.15.146:0/1988395395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f86980c6960 0x7f86980c0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f869e915700 1 -- 172.21.15.146:0/1988395395 >> v1:172.21.15.146:6789/0 conn(0x7f8698157de0 legacy=0x7f869815e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f869e915700 1 -- 172.21.15.146:0/1988395395 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869815d900 con 0x7f8698071640 2024-09-13T18:27:03.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a86ffd700 1 --2- 172.21.15.146:0/2902988619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a88158030 0x7f7a880c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a87fff700 1 -- 172.21.15.146:0/2902988619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a88158030 msgr2=0x7f7a880c0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a87fff700 1 --2- 172.21.15.146:0/2902988619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a88158030 0x7f7a880c0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a87fff700 1 -- 172.21.15.146:0/2902988619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a880cab20 msgr2=0x7f7a880add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a87fff700 1 --2- 172.21.15.146:0/2902988619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a880cab20 0x7f7a880add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a87fff700 1 -- 172.21.15.146:0/2902988619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c0094b0 con 0x7f7a8815b380 2024-09-13T18:27:03.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f5447fff700 1 -- start start 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0ad858700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a8070500 0x7ff0a8152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0ad858700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0a80aa360 0x7ff0a81465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0ad858700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a8159b70 0x7ff0a8146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0ad858700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a80c1ef0 con 0x7ff0a80aa360 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0ad858700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a80c1c50 con 0x7ff0a8070500 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0ad858700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a81eec90 con 0x7ff0a8159b70 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0a7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0a80aa360 0x7ff0a81465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0a7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0a80aa360 0x7ff0a81465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54604/0 (socket says 172.21.15.146:54604) 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0a7fff700 1 -- 172.21.15.146:0/2766515186 learned_addr learned my addr 172.21.15.146:0/2766515186 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0ac856700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a8070500 0x7ff0a8152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0ad057700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a8159b70 0x7ff0a8146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7fea45257700 1 --2- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea4014f750 0x7fea400ac170 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fea3000e9e0 tx=0x7fea3000eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f1590a58700 1 Processor -- start 2024-09-13T18:27:03.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f1590a58700 1 -- start start 2024-09-13T18:27:03.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a87fff700 1 --2- 172.21.15.146:0/2902988619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a8815b380 0x7f7a880c10b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7a88070a80 tx=0x7f7a78004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7fea3d7fa700 1 -- 172.21.15.146:0/1263341439 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3000dc50 con 0x7fea4014f750 2024-09-13T18:27:03.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7fea3d7fa700 1 -- 172.21.15.146:0/1263341439 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3000ddb0 con 0x7fea4014f750 2024-09-13T18:27:03.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7fea3d7fa700 1 -- 172.21.15.146:0/1263341439 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300196c0 con 0x7fea4014f750 2024-09-13T18:27:03.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0a7fff700 1 -- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a8159b70 msgr2=0x7ff0a8146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0a7fff700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a8159b70 0x7ff0a8146b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0a7fff700 1 -- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a8070500 msgr2=0x7ff0a8152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0a7fff700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a8070500 0x7ff0a8152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0a7fff700 1 -- 172.21.15.146:0/2766515186 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c00a650 con 0x7ff0a80aa360 2024-09-13T18:27:03.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a6ffff700 1 -- 172.21.15.146:0/2902988619 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78006ea0 con 0x7f7a8815b380 2024-09-13T18:27:03.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7fea46cba700 1 -- 172.21.15.146:0/1263341439 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea400aae70 con 0x7fea4014f750 2024-09-13T18:27:03.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7fea46cba700 1 -- 172.21.15.146:0/1263341439 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea400ab430 con 0x7fea4014f750 2024-09-13T18:27:03.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a6ffff700 1 -- 172.21.15.146:0/2902988619 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a780036c0 con 0x7f7a8815b380 2024-09-13T18:27:03.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a6ffff700 1 -- 172.21.15.146:0/2902988619 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78003a70 con 0x7f7a8815b380 2024-09-13T18:27:03.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a8dc64700 1 -- 172.21.15.146:0/2902988619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a780094b0 con 0x7f7a8815b380 2024-09-13T18:27:03.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a8dc64700 1 -- 172.21.15.146:0/2902988619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a880cb320 con 0x7f7a8815b380 2024-09-13T18:27:03.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f7a8dc64700 1 -- 172.21.15.146:0/2902988619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a880c1960 con 0x7f7a8815b380 2024-09-13T18:27:03.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f7a6ffff700 1 -- 172.21.15.146:0/2902988619 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a78003bd0 con 0x7f7a8815b380 2024-09-13T18:27:03.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f7a6ffff700 1 --2- 172.21.15.146:0/2902988619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7a74066730 0x7f7a74068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f7a6ffff700 1 -- 172.21.15.146:0/2902988619 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7a78021030 con 0x7f7a8815b380 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f5447fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806c500 0x7f54481ed1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f5447fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544806eec0 0x7f544815e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f5447fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f544814f650 0x7f544814cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f5447fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54481ec7a0 con 0x7f544814f650 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f5447fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5448146190 con 0x7f544806eec0 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f5447fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54481ecd20 con 0x7f544806c500 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f54477fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f544814f650 0x7f544814cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f54477fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f544814f650 0x7f544814cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54640/0 (socket says 172.21.15.146:54640) 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f54477fe700 1 -- 172.21.15.146:0/1165472849 learned_addr learned my addr 172.21.15.146:0/1165472849 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f5446ffd700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806c500 0x7f54481ed1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f54467fc700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544806eec0 0x7f544815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f54477fe700 1 -- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806c500 msgr2=0x7f54481ed1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f54477fe700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806c500 0x7f54481ed1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f54477fe700 1 -- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544806eec0 msgr2=0x7f544815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f54477fe700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544806eec0 0x7f544815e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f54477fe700 1 -- 172.21.15.146:0/1165472849 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5440009a70 con 0x7f544814f650 2024-09-13T18:27:03.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f54467fc700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544806eec0 0x7f544815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:03.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f54477fe700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f544814f650 0x7f544814cbc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5440014040 tx=0x7f5440004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0a7fff700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0a80aa360 0x7ff0a81465c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff09c019040 tx=0x7ff09c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0a57fa700 1 -- 172.21.15.146:0/2766515186 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c004050 con 0x7ff0a80aa360 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0ad858700 1 -- 172.21.15.146:0/2766515186 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a814bfe0 con 0x7ff0a80aa360 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0ad858700 1 -- 172.21.15.146:0/2766515186 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a814c5a0 con 0x7ff0a80aa360 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0a57fa700 1 -- 172.21.15.146:0/2766515186 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c0041b0 con 0x7ff0a80aa360 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7ff0a57fa700 1 -- 172.21.15.146:0/2766515186 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c004560 con 0x7ff0a80aa360 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7ff0a57fa700 1 -- 172.21.15.146:0/2766515186 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff09c0036c0 con 0x7ff0a80aa360 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f1590a58700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c0a9490 0x7f158c1f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f1590a58700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f158c0c8f70 0x7f158c0c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f1590a58700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f158c0c9350 0x7f158c0ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f1590a58700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0adc30 con 0x7f158c0a9490 2024-09-13T18:27:03.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f1590a58700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c1f7760 con 0x7f158c0c8f70 2024-09-13T18:27:03.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f1590a58700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c1f6c00 con 0x7f158c0c9350 2024-09-13T18:27:03.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f1589d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f158c0c8f70 0x7f158c0c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f1589d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f158c0c8f70 0x7f158c0c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:46614/0 (socket says 172.21.15.146:46614) 2024-09-13T18:27:03.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f1589d9b700 1 -- 172.21.15.146:0/3730060752 learned_addr learned my addr 172.21.15.146:0/3730060752 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f158ad9d700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f158c0c9350 0x7f158c0ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f158a59c700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c0a9490 0x7f158c1f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f158a59c700 1 -- 172.21.15.146:0/3730060752 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f158c0c9350 msgr2=0x7f158c0ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f158a59c700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f158c0c9350 0x7f158c0ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f158a59c700 1 -- 172.21.15.146:0/3730060752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f158c0c8f70 msgr2=0x7f158c0c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f158a59c700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f158c0c8f70 0x7f158c0c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f158a59c700 1 -- 172.21.15.146:0/3730060752 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c0094b0 con 0x7f158c0a9490 2024-09-13T18:27:03.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f158a59c700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c0a9490 0x7f158c1f3230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f158400cbf0 tx=0x7f158400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7fea46cba700 1 -- 172.21.15.146:0/1263341439 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea400c94e0 con 0x7fea4014f750 2024-09-13T18:27:03.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f869e915700 1 --2- 172.21.15.146:0/1988395395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8698071640 0x7f86980c14c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8694009910 tx=0x7f8694009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1d2c672c27e006e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f869d112700 1 -- 172.21.15.146:0/1988395395 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694004230 con 0x7f8698071640 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f869d112700 1 -- 172.21.15.146:0/1988395395 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8694004390 con 0x7f8698071640 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f869d112700 1 -- 172.21.15.146:0/1988395395 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86940036c0 con 0x7f8698071640 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f869f116700 1 -- 172.21.15.146:0/1988395395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8698071640 msgr2=0x7f86980c14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f869f116700 1 --2- 172.21.15.146:0/1988395395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8698071640 0x7f86980c14c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8694009910 tx=0x7f8694009c20 comp rx=0 tx=0).stop 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f869f116700 1 -- 172.21.15.146:0/1988395395 shutdown_connections 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f869f116700 1 --2- 172.21.15.146:0/1988395395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8698071640 0x7f86980c14c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f869f116700 1 --2- 172.21.15.146:0/1988395395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f86980c6960 0x7f86980c0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f869f116700 1 -- 172.21.15.146:0/1988395395 >> 172.21.15.146:0/1988395395 conn(0x7f8698071ed0 msgr2=0x7f86980722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f869f116700 1 -- 172.21.15.146:0/1988395395 shutdown_connections 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f869f116700 1 -- 172.21.15.146:0/1988395395 wait complete. 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f6d8e1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680700b0 0x7f3f6814ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f6d8e1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f680c0510 0x7f3f681ea530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f6d8e1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f680c7360 0x7f3f681486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f667fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f680c0510 0x7f3f681ea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f6d8e1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f68155680 con 0x7f3f680c0510 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f6d8e1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680c0f40 con 0x7f3f680c7360 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f6d8e1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680c1200 con 0x7f3f680700b0 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f667fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f680c0510 0x7f3f681ea530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54618/0 (socket says 172.21.15.146:54618) 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f667fc700 1 -- 172.21.15.146:0/3348594006 learned_addr learned my addr 172.21.15.146:0/3348594006 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f66ffd700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680700b0 0x7f3f6814ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f677fe700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f680c7360 0x7f3f681486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f667fc700 1 -- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680700b0 msgr2=0x7f3f6814ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f667fc700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680700b0 0x7f3f6814ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f667fc700 1 -- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f680c7360 msgr2=0x7f3f681486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f667fc700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f680c7360 0x7f3f681486c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f667fc700 1 -- 172.21.15.146:0/3348594006 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6000bc80 con 0x7f3f680c0510 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f667fc700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f680c0510 0x7f3f681ea530 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00c700 tx=0x7f3f5c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.349+0000 7f3f4ffff700 1 -- 172.21.15.146:0/3348594006 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c005d90 con 0x7f3f680c0510 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f3f4ffff700 1 -- 172.21.15.146:0/3348594006 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c0070d0 con 0x7f3f680c0510 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f3f4ffff700 1 -- 172.21.15.146:0/3348594006 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c007480 con 0x7f3f680c0510 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f3f6d8e1700 1 -- 172.21.15.146:0/3348594006 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f680c14e0 con 0x7f3f680c0510 2024-09-13T18:27:03.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f3f6d8e1700 1 -- 172.21.15.146:0/3348594006 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f68155310 con 0x7f3f680c0510 2024-09-13T18:27:03.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7ff0a57fa700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff094066780 0x7ff094068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7ff0a57fa700 1 -- 172.21.15.146:0/2766515186 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff09c013070 con 0x7ff0a80aa360 2024-09-13T18:27:03.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7fea3d7fa700 1 -- 172.21.15.146:0/1263341439 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fea30019820 con 0x7fea4014f750 2024-09-13T18:27:03.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7ff0ad858700 1 -- 172.21.15.146:0/2766515186 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff088005310 con 0x7ff0a80aa360 2024-09-13T18:27:03.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7fea3d7fa700 1 --2- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea2c066730 0x7fea2c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7fea3d7fa700 1 -- 172.21.15.146:0/1263341439 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fea30014070 con 0x7fea4014f750 2024-09-13T18:27:03.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f7a877fe700 1 --2- 172.21.15.146:0/2902988619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7a74066730 0x7f7a74068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7fea44a56700 1 --2- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea2c066730 0x7fea2c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f7a877fe700 1 --2- 172.21.15.146:0/2902988619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7a74066730 0x7f7a74068bf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a80005cb0 tx=0x7f7a80009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f869f116700 1 Processor -- start 2024-09-13T18:27:03.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.353+0000 7f3f4ffff700 1 -- 172.21.15.146:0/3348594006 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3f5c0075e0 con 0x7f3f680c0510 2024-09-13T18:27:03.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7ff0ac856700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff094066780 0x7ff094068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f3f4ffff700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f5406ac40 0x7f3f5406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fea44a56700 1 --2- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea2c066730 0x7fea2c068bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea340090b0 tx=0x7fea34010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f3f4ffff700 1 -- 172.21.15.146:0/3348594006 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3f5c083730 con 0x7f3f680c0510 2024-09-13T18:27:03.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f3f66ffd700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f5406ac40 0x7f3f5406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869f116700 1 -- start start 2024-09-13T18:27:03.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f3f6d8e1700 1 -- 172.21.15.146:0/3348594006 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f48005310 con 0x7f3f680c0510 2024-09-13T18:27:03.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7ff0ac856700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff094066780 0x7ff094068c40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff098009b50 tx=0x7ff098009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f3f66ffd700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f5406ac40 0x7f3f5406d100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3f58007fd0 tx=0x7f3f58009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869f116700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8698071640 0x7f86980c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869f116700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86980c6960 0x7f86980c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869f116700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698157de0 0x7f869814d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869f116700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8698152030 con 0x7f8698157de0 2024-09-13T18:27:03.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869f116700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86980c89d0 con 0x7f86980c6960 2024-09-13T18:27:03.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869f116700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86980c8150 con 0x7f8698071640 2024-09-13T18:27:03.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869e915700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698157de0 0x7f869814d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869e915700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698157de0 0x7f869814d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54642/0 (socket says 172.21.15.146:54642) 2024-09-13T18:27:03.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869e915700 1 -- 172.21.15.146:0/3819163985 learned_addr learned my addr 172.21.15.146:0/3819163985 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869e114700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8698071640 0x7f86980c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd2e743700 1 Processor -- start 2024-09-13T18:27:03.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869d913700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86980c6960 0x7f86980c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd2e743700 1 -- start start 2024-09-13T18:27:03.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869e915700 1 -- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8698071640 msgr2=0x7f86980c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869e915700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8698071640 0x7f86980c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869e915700 1 -- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86980c6960 msgr2=0x7f86980c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869e915700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86980c6960 0x7f86980c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869e915700 1 -- 172.21.15.146:0/3819163985 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8694009570 con 0x7f8698157de0 2024-09-13T18:27:03.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869e915700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698157de0 0x7f869814d0f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f86980ccdb0 tx=0x7f8694004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd2e743700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdd28153dc0 0x7fdd280c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd2e743700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd28159e90 con 0x7fdd2815ce10 2024-09-13T18:27:03.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd2e743700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd28159860 con 0x7fdd281539e0 2024-09-13T18:27:03.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd2e743700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd280c7d90 con 0x7fdd28153dc0 2024-09-13T18:27:03.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd2cce0700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fdd2815ce10 0x7fdd281641a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35516/0 (socket says 172.21.15.146:35516) 2024-09-13T18:27:03.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd2cce0700 1 -- 172.21.15.146:0/2232011870 learned_addr learned my addr 172.21.15.146:0/2232011870 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:03.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd277fe700 1 --2- 172.21.15.146:0/2232011870 >> v2:172.21.15.154:3300/0 conn(0x7fdd28153dc0 0x7fdd280c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd277fe700 1 -- 172.21.15.146:0/2232011870 >> v1:172.21.15.154:6789/0 conn(0x7fdd281539e0 legacy=0x7fdd280c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd277fe700 1 -- 172.21.15.146:0/2232011870 >> v1:172.21.15.146:6789/0 conn(0x7fdd2815ce10 legacy=0x7fdd281641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd277fe700 1 -- 172.21.15.146:0/2232011870 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd280c8070 con 0x7fdd28153dc0 2024-09-13T18:27:03.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd277fe700 1 --2- 172.21.15.146:0/2232011870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd28153dc0 0x7fdd280c76e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009850 tx=0x7fdd1c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47706395a9739997 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f542f7fe700 1 -- 172.21.15.146:0/1165472849 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440012dd0 con 0x7f544814f650 2024-09-13T18:27:03.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f5447fff700 1 -- 172.21.15.146:0/1165472849 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5448146f90 con 0x7f544814f650 2024-09-13T18:27:03.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f157affd700 1 -- 172.21.15.146:0/3730060752 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1584007480 con 0x7f158c0a9490 2024-09-13T18:27:03.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f7a6ffff700 1 -- 172.21.15.146:0/2902988619 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7a880c1960 con 0x7f7a8815b380 2024-09-13T18:27:03.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f5447fff700 1 -- 172.21.15.146:0/1165472849 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54481458f0 con 0x7f544814f650 2024-09-13T18:27:03.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f868effd700 1 -- 172.21.15.146:0/3819163985 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86940036c0 con 0x7f8698157de0 2024-09-13T18:27:03.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f1590a58700 1 -- 172.21.15.146:0/3730060752 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158c1f6ec0 con 0x7f158c0a9490 2024-09-13T18:27:03.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f1590a58700 1 -- 172.21.15.146:0/3730060752 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158c1f7480 con 0x7f158c0a9490 2024-09-13T18:27:03.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869f116700 1 -- 172.21.15.146:0/3819163985 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86980c8c40 con 0x7f8698157de0 2024-09-13T18:27:03.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7f869f116700 1 -- 172.21.15.146:0/3819163985 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8698156390 con 0x7f8698157de0 2024-09-13T18:27:03.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd26ffd700 1 -- 172.21.15.146:0/2232011870 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c011830 con 0x7fdd28153dc0 2024-09-13T18:27:03.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd26ffd700 1 -- 172.21.15.146:0/2232011870 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c011990 con 0x7fdd28153dc0 2024-09-13T18:27:03.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.357+0000 7fdd26ffd700 1 -- 172.21.15.146:0/2232011870 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c011830 con 0x7fdd28153dc0 2024-09-13T18:27:03.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.361+0000 7fdd2e743700 1 -- 172.21.15.146:0/2232011870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd28153dc0 msgr2=0x7fdd280c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.361+0000 7fdd2e743700 1 --2- 172.21.15.146:0/2232011870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd28153dc0 0x7fdd280c76e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009850 tx=0x7fdd1c009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:03.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.361+0000 7f1590a58700 1 -- 172.21.15.146:0/3730060752 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158c0a9d60 con 0x7f158c0a9490 2024-09-13T18:27:03.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.361+0000 7fdd2e743700 1 -- 172.21.15.146:0/2232011870 shutdown_connections 2024-09-13T18:27:03.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.361+0000 7fdd2e743700 1 --2- 172.21.15.146:0/2232011870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd28153dc0 0x7fdd280c76e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.361+0000 7fdd2e743700 1 -- 172.21.15.146:0/2232011870 >> 172.21.15.146:0/2232011870 conn(0x7fdd2806f5c0 msgr2=0x7fdd2806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.361+0000 7f5447fff700 1 -- 172.21.15.146:0/1165472849 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5448070420 con 0x7f544814f650 2024-09-13T18:27:03.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.361+0000 7fdd2e743700 1 -- 172.21.15.146:0/2232011870 shutdown_connections 2024-09-13T18:27:03.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.361+0000 7fdd2e743700 1 -- 172.21.15.146:0/2232011870 wait complete. 2024-09-13T18:27:03.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.361+0000 7f869f116700 1 -- 172.21.15.146:0/3819163985 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86980c6650 con 0x7f8698157de0 2024-09-13T18:27:03.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.361+0000 7fdd2e743700 1 Processor -- start 2024-09-13T18:27:03.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.361+0000 7fdd2e743700 1 -- start start 2024-09-13T18:27:03.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2e743700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd281539e0 0x7fdd2815a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2e743700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd28153dc0 0x7fdd28147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2e743700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2815ce10 0x7fdd28148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2e743700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2814acf0 con 0x7fdd2815ce10 2024-09-13T18:27:03.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2e743700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2815c260 con 0x7fdd28153dc0 2024-09-13T18:27:03.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2e743700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2815c6c0 con 0x7fdd281539e0 2024-09-13T18:27:03.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7ff0a57fa700 1 -- 172.21.15.146:0/2766515186 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff09c053190 con 0x7ff0a80aa360 2024-09-13T18:27:03.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2cce0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2815ce10 0x7fdd28148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd27fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd281539e0 0x7fdd2815a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd277fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd28153dc0 0x7fdd28147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2cce0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2815ce10 0x7fdd28148330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54648/0 (socket says 172.21.15.146:54648) 2024-09-13T18:27:03.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2cce0700 1 -- 172.21.15.146:0/3427414200 learned_addr learned my addr 172.21.15.146:0/3427414200 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:03.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2cce0700 1 -- 172.21.15.146:0/3427414200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd281539e0 msgr2=0x7fdd2815a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2cce0700 1 --2- 172.21.15.146:0/3427414200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd281539e0 0x7fdd2815a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2cce0700 1 -- 172.21.15.146:0/3427414200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd28153dc0 msgr2=0x7fdd28147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2cce0700 1 --2- 172.21.15.146:0/3427414200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd28153dc0 0x7fdd28147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2cce0700 1 -- 172.21.15.146:0/3427414200 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c0094b0 con 0x7fdd2815ce10 2024-09-13T18:27:03.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fdd2cce0700 1 --2- 172.21.15.146:0/3427414200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2815ce10 0x7fdd28148330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000ca80 tx=0x7fdd2000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.365+0000 7fea3d7fa700 1 -- 172.21.15.146:0/1263341439 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fea30051da0 con 0x7fea4014f750 2024-09-13T18:27:03.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.369+0000 7f157affd700 1 -- 172.21.15.146:0/3730060752 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15840075e0 con 0x7f158c0a9490 2024-09-13T18:27:03.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.369+0000 7f157affd700 1 -- 172.21.15.146:0/3730060752 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1584007950 con 0x7f158c0a9490 2024-09-13T18:27:03.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.369+0000 7f3f4ffff700 1 -- 172.21.15.146:0/3348594006 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3f5c0508d0 con 0x7f3f680c0510 2024-09-13T18:27:03.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.369+0000 7f542f7fe700 1 -- 172.21.15.146:0/1165472849 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544000bd40 con 0x7f544814f650 2024-09-13T18:27:03.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.369+0000 7f542f7fe700 1 -- 172.21.15.146:0/1165472849 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440012430 con 0x7f544814f650 2024-09-13T18:27:03.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.369+0000 7f868effd700 1 -- 172.21.15.146:0/3819163985 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8694011630 con 0x7f8698157de0 2024-09-13T18:27:03.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.369+0000 7f868effd700 1 -- 172.21.15.146:0/3819163985 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86940119a0 con 0x7f8698157de0 2024-09-13T18:27:03.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.369+0000 7fdd24ff9700 1 -- 172.21.15.146:0/3427414200 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20007480 con 0x7fdd2815ce10 2024-09-13T18:27:03.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.373+0000 7fdd2e743700 1 -- 172.21.15.146:0/3427414200 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2815c9a0 con 0x7fdd2815ce10 2024-09-13T18:27:03.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.373+0000 7fdd2e743700 1 -- 172.21.15.146:0/3427414200 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2814b1b0 con 0x7fdd2815ce10 2024-09-13T18:27:03.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.373+0000 7fdd24ff9700 1 -- 172.21.15.146:0/3427414200 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd20001b60 con 0x7fdd2815ce10 2024-09-13T18:27:03.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.373+0000 7fdd24ff9700 1 -- 172.21.15.146:0/3427414200 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd200077e0 con 0x7fdd2815ce10 2024-09-13T18:27:03.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.373+0000 7f542f7fe700 1 -- 172.21.15.146:0/1165472849 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5440024610 con 0x7f544814f650 2024-09-13T18:27:03.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.373+0000 7f542f7fe700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5434066730 0x7f5434068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.373+0000 7fdd24ff9700 1 -- 172.21.15.146:0/3427414200 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdd20007940 con 0x7fdd2815ce10 2024-09-13T18:27:03.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.373+0000 7f542f7fe700 1 -- 172.21.15.146:0/1165472849 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5440086020 con 0x7f544814f650 2024-09-13T18:27:03.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.373+0000 7f5446ffd700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5434066730 0x7f5434068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.373+0000 7f157affd700 1 -- 172.21.15.146:0/3730060752 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f158401b3b0 con 0x7f158c0a9490 2024-09-13T18:27:03.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.377+0000 7fdd24ff9700 1 --2- 172.21.15.146:0/3427414200 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdd14066730 0x7fdd14068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.377+0000 7f5446ffd700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5434066730 0x7f5434068bf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5438000f60 tx=0x7f5438007e00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.373+0000 7f868effd700 1 -- 172.21.15.146:0/3819163985 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8694011b00 con 0x7f8698157de0 2024-09-13T18:27:03.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.377+0000 7f868effd700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f867c066730 0x7f867c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.377+0000 7f868effd700 1 -- 172.21.15.146:0/3819163985 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f86940861a0 con 0x7f8698157de0 2024-09-13T18:27:03.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.377+0000 7f869e114700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f867c066730 0x7f867c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.377+0000 7fdd27fff700 1 --2- 172.21.15.146:0/3427414200 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdd14066730 0x7fdd14068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.377+0000 7f157affd700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f156c066730 0x7f156c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.377+0000 7fdd2e743700 1 -- 172.21.15.146:0/3427414200 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd08005310 con 0x7fdd2815ce10 2024-09-13T18:27:03.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.377+0000 7f1589d9b700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f156c066730 0x7f156c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.377+0000 7f869e114700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f867c066730 0x7f867c068bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f869000f300 tx=0x7f86900042e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.377+0000 7f157affd700 1 -- 172.21.15.146:0/3730060752 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1584084290 con 0x7f158c0a9490 2024-09-13T18:27:03.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.377+0000 7f1589d9b700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f156c066730 0x7f156c068bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f158c0c2140 tx=0x7f1580009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.377+0000 7fdd27fff700 1 --2- 172.21.15.146:0/3427414200 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdd14066730 0x7fdd14068bf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdd18000f60 tx=0x7fdd18009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.381+0000 7f157affd700 1 -- 172.21.15.146:0/3730060752 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1584051290 con 0x7f158c0a9490 2024-09-13T18:27:03.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.389+0000 7f542f7fe700 1 -- 172.21.15.146:0/1165472849 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f54400532e0 con 0x7f544814f650 2024-09-13T18:27:03.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.401+0000 7f868effd700 1 -- 172.21.15.146:0/3819163985 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8694053290 con 0x7f8698157de0 2024-09-13T18:27:03.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.401+0000 7fdd24ff9700 1 -- 172.21.15.146:0/3427414200 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdd20054940 con 0x7fdd2815ce10 2024-09-13T18:27:03.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.409+0000 7fdd24ff9700 1 -- 172.21.15.146:0/3427414200 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdd20050c80 con 0x7fdd2815ce10 2024-09-13T18:27:03.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.677+0000 7f7a8dc64700 1 -- 172.21.15.146:0/2902988619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7a880c1930 con 0x7f7a8815b380 2024-09-13T18:27:03.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.681+0000 7f7a6ffff700 1 -- 172.21.15.146:0/2902988619 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a7802b090 con 0x7f7a8815b380 2024-09-13T18:27:03.688 INFO:teuthology.orchestra.run.smithi146.stdout:55834574856 2024-09-13T18:27:03.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.685+0000 7f7a6dffb700 1 -- 172.21.15.146:0/2902988619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7a74066730 msgr2=0x7f7a74068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.685+0000 7f7a6dffb700 1 --2- 172.21.15.146:0/2902988619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7a74066730 0x7f7a74068bf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a80005cb0 tx=0x7f7a80009040 comp rx=0 tx=0).stop 2024-09-13T18:27:03.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.685+0000 7f7a6dffb700 1 -- 172.21.15.146:0/2902988619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a8815b380 msgr2=0x7f7a880c10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.685+0000 7f7a6dffb700 1 --2- 172.21.15.146:0/2902988619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a8815b380 0x7f7a880c10b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7a88070a80 tx=0x7f7a78004770 comp rx=0 tx=0).stop 2024-09-13T18:27:03.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.685+0000 7f7a6dffb700 1 -- 172.21.15.146:0/2902988619 shutdown_connections 2024-09-13T18:27:03.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.685+0000 7f7a6dffb700 1 --2- 172.21.15.146:0/2902988619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7a74066730 0x7f7a74068bf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.685+0000 7f7a6dffb700 1 --2- 172.21.15.146:0/2902988619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a8815b380 0x7f7a880c10b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.685+0000 7f7a6dffb700 1 --2- 172.21.15.146:0/2902988619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a88158030 0x7f7a880c0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.685+0000 7f7a6dffb700 1 --2- 172.21.15.146:0/2902988619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a880cab20 0x7f7a880add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.685+0000 7f7a6dffb700 1 -- 172.21.15.146:0/2902988619 >> 172.21.15.146:0/2902988619 conn(0x7f7a88066090 msgr2=0x7f7a88150a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.685+0000 7f7a6dffb700 1 -- 172.21.15.146:0/2902988619 shutdown_connections 2024-09-13T18:27:03.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.685+0000 7f7a6dffb700 1 -- 172.21.15.146:0/2902988619 wait complete. 2024-09-13T18:27:03.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.693+0000 7ff0ad858700 1 -- 172.21.15.146:0/2766515186 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff088005890 con 0x7ff0a80aa360 2024-09-13T18:27:03.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.697+0000 7ff0a57fa700 1 -- 172.21.15.146:0/2766515186 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff09c056770 con 0x7ff0a80aa360 2024-09-13T18:27:03.703 INFO:teuthology.orchestra.run.smithi146.stdout:60129542154 2024-09-13T18:27:03.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.697+0000 7fea46cba700 1 -- 172.21.15.146:0/1263341439 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fea400ab950 con 0x7fea4014f750 2024-09-13T18:27:03.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.697+0000 7fea3d7fa700 1 -- 172.21.15.146:0/1263341439 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fea400ab950 con 0x7fea4014f750 2024-09-13T18:27:03.705 INFO:teuthology.orchestra.run.smithi146.stdout:64424509449 2024-09-13T18:27:03.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7ff0ad858700 1 -- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff094066780 msgr2=0x7ff094068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7ff0ad858700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff094066780 0x7ff094068c40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff098009b50 tx=0x7ff098009240 comp rx=0 tx=0).stop 2024-09-13T18:27:03.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7ff0ad858700 1 -- 172.21.15.146:0/2766515186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0a80aa360 msgr2=0x7ff0a81465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7ff0ad858700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0a80aa360 0x7ff0a81465c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff09c019040 tx=0x7ff09c004770 comp rx=0 tx=0).stop 2024-09-13T18:27:03.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7ff0ad858700 1 -- 172.21.15.146:0/2766515186 shutdown_connections 2024-09-13T18:27:03.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7ff0ad858700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff094066780 0x7ff094068c40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7ff0ad858700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a8159b70 0x7ff0a8146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7ff0ad858700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0a80aa360 0x7ff0a81465c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7ff0ad858700 1 --2- 172.21.15.146:0/2766515186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a8070500 0x7ff0a8152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7ff0ad858700 1 -- 172.21.15.146:0/2766515186 >> 172.21.15.146:0/2766515186 conn(0x7ff0a8066090 msgr2=0x7ff0a806f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7ff0ad858700 1 -- 172.21.15.146:0/2766515186 shutdown_connections 2024-09-13T18:27:03.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7ff0ad858700 1 -- 172.21.15.146:0/2766515186 wait complete. 2024-09-13T18:27:03.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7fea46cba700 1 -- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea2c066730 msgr2=0x7fea2c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7fea46cba700 1 --2- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea2c066730 0x7fea2c068bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea340090b0 tx=0x7fea34010000 comp rx=0 tx=0).stop 2024-09-13T18:27:03.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7fea46cba700 1 -- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea4014f750 msgr2=0x7fea400ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7fea46cba700 1 --2- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea4014f750 0x7fea400ac170 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fea3000e9e0 tx=0x7fea3000eda0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7fea46cba700 1 -- 172.21.15.146:0/1263341439 shutdown_connections 2024-09-13T18:27:03.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7fea46cba700 1 --2- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea2c066730 0x7fea2c068bf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7fea46cba700 1 --2- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea4014f750 0x7fea400ac170 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7fea46cba700 1 --2- 172.21.15.146:0/1263341439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea400a9490 0x7fea400abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7fea46cba700 1 --2- 172.21.15.146:0/1263341439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea4006fb00 0x7fea401526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7fea46cba700 1 -- 172.21.15.146:0/1263341439 >> 172.21.15.146:0/1263341439 conn(0x7fea40066090 msgr2=0x7fea400ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7fea46cba700 1 -- 172.21.15.146:0/1263341439 shutdown_connections 2024-09-13T18:27:03.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7fea46cba700 1 -- 172.21.15.146:0/1263341439 wait complete. 2024-09-13T18:27:03.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2024-09-13T18:27:03.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7f3f6d8e1700 1 -- 172.21.15.146:0/3348594006 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3f48005180 con 0x7f3f680c0510 2024-09-13T18:27:03.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.701+0000 7f1590a58700 1 -- 172.21.15.146:0/3730060752 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f158c0c0600 con 0x7f158c0a9490 2024-09-13T18:27:03.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.705+0000 7f3f4ffff700 1 -- 172.21.15.146:0/3348594006 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3f5c017070 con 0x7f3f680c0510 2024-09-13T18:27:03.710 INFO:teuthology.orchestra.run.smithi146.stdout:55834574857 2024-09-13T18:27:03.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.705+0000 7f157affd700 1 -- 172.21.15.146:0/3730060752 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1584017090 con 0x7f158c0a9490 2024-09-13T18:27:03.711 INFO:teuthology.orchestra.run.smithi146.stdout:64424509448 2024-09-13T18:27:03.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f3f4dffb700 1 -- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f5406ac40 msgr2=0x7f3f5406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f3f4dffb700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f5406ac40 0x7f3f5406d100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3f58007fd0 tx=0x7f3f58009040 comp rx=0 tx=0).stop 2024-09-13T18:27:03.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f3f4dffb700 1 -- 172.21.15.146:0/3348594006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f680c0510 msgr2=0x7f3f681ea530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f3f4dffb700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f680c0510 0x7f3f681ea530 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00c700 tx=0x7f3f5c00ca10 comp rx=0 tx=0).stop 2024-09-13T18:27:03.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f3f4dffb700 1 -- 172.21.15.146:0/3348594006 shutdown_connections 2024-09-13T18:27:03.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f3f4dffb700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f5406ac40 0x7f3f5406d100 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f3f4dffb700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f680c7360 0x7f3f681486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f3f4dffb700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f680c0510 0x7f3f681ea530 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f3f4dffb700 1 --2- 172.21.15.146:0/3348594006 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680700b0 0x7f3f6814ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f3f4dffb700 1 -- 172.21.15.146:0/3348594006 >> 172.21.15.146:0/3348594006 conn(0x7f3f6806f5c0 msgr2=0x7f3f6806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f3f4dffb700 1 -- 172.21.15.146:0/3348594006 shutdown_connections 2024-09-13T18:27:03.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f3f4dffb700 1 -- 172.21.15.146:0/3348594006 wait complete. 2024-09-13T18:27:03.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f1590a58700 1 -- 172.21.15.146:0/3730060752 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f156c066730 msgr2=0x7f156c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f1590a58700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f156c066730 0x7f156c068bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f158c0c2140 tx=0x7f1580009040 comp rx=0 tx=0).stop 2024-09-13T18:27:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f1590a58700 1 -- 172.21.15.146:0/3730060752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c0a9490 msgr2=0x7f158c1f3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f1590a58700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c0a9490 0x7f158c1f3230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f158400cbf0 tx=0x7f158400cc20 comp rx=0 tx=0).stop 2024-09-13T18:27:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f1590a58700 1 -- 172.21.15.146:0/3730060752 shutdown_connections 2024-09-13T18:27:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f1590a58700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f156c066730 0x7f156c068bf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f1590a58700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f158c0c9350 0x7f158c0ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f1590a58700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f158c0c8f70 0x7f158c0c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f1590a58700 1 --2- 172.21.15.146:0/3730060752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c0a9490 0x7f158c1f3230 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f1590a58700 1 -- 172.21.15.146:0/3730060752 >> 172.21.15.146:0/3730060752 conn(0x7f158c066090 msgr2=0x7f158c0c0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f1590a58700 1 -- 172.21.15.146:0/3730060752 shutdown_connections 2024-09-13T18:27:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.709+0000 7f1590a58700 1 -- 172.21.15.146:0/3730060752 wait complete. 2024-09-13T18:27:03.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.6 2024-09-13T18:27:03.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.2 2024-09-13T18:27:03.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.4 2024-09-13T18:27:03.729 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-09-13T18:27:03.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.725+0000 7f869f116700 1 -- 172.21.15.146:0/3819163985 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f86980c6650 con 0x7f8698157de0 2024-09-13T18:27:03.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.725+0000 7f868effd700 1 -- 172.21.15.146:0/3819163985 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f86940568b0 con 0x7f8698157de0 2024-09-13T18:27:03.732 INFO:teuthology.orchestra.run.smithi146.stdout:64424509448 2024-09-13T18:27:03.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.729+0000 7f869f116700 1 -- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f867c066730 msgr2=0x7f867c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.729+0000 7f869f116700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f867c066730 0x7f867c068bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f869000f300 tx=0x7f86900042e0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.729+0000 7f869f116700 1 -- 172.21.15.146:0/3819163985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698157de0 msgr2=0x7f869814d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.729+0000 7f869f116700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698157de0 0x7f869814d0f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f86980ccdb0 tx=0x7f8694004770 comp rx=0 tx=0).stop 2024-09-13T18:27:03.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.729+0000 7f869f116700 1 -- 172.21.15.146:0/3819163985 shutdown_connections 2024-09-13T18:27:03.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.729+0000 7f869f116700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f867c066730 0x7f867c068bf0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.729+0000 7f869f116700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698157de0 0x7f869814d0f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.729+0000 7f869f116700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86980c6960 0x7f86980c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.729+0000 7f869f116700 1 --2- 172.21.15.146:0/3819163985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8698071640 0x7f86980c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.729+0000 7f869f116700 1 -- 172.21.15.146:0/3819163985 >> 172.21.15.146:0/3819163985 conn(0x7f8698071ed0 msgr2=0x7f869815bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.729+0000 7f869f116700 1 -- 172.21.15.146:0/3819163985 shutdown_connections 2024-09-13T18:27:03.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.729+0000 7f869f116700 1 -- 172.21.15.146:0/3819163985 wait complete. 2024-09-13T18:27:03.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.733+0000 7f5447fff700 1 -- 172.21.15.146:0/1165472849 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f54481ece60 con 0x7f544814f650 2024-09-13T18:27:03.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.733+0000 7f542f7fe700 1 -- 172.21.15.146:0/1165472849 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5440056900 con 0x7f544814f650 2024-09-13T18:27:03.740 INFO:teuthology.orchestra.run.smithi146.stdout:60129542153 2024-09-13T18:27:03.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.733+0000 7fdd2e743700 1 -- 172.21.15.146:0/3427414200 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdd080055c0 con 0x7fdd2815ce10 2024-09-13T18:27:03.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.733+0000 7fdd24ff9700 1 -- 172.21.15.146:0/3427414200 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdd20017030 con 0x7fdd2815ce10 2024-09-13T18:27:03.741 INFO:teuthology.orchestra.run.smithi146.stdout:55834574856 2024-09-13T18:27:03.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7fdd2e743700 1 -- 172.21.15.146:0/3427414200 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdd14066730 msgr2=0x7fdd14068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7fdd2e743700 1 --2- 172.21.15.146:0/3427414200 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdd14066730 0x7fdd14068bf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdd18000f60 tx=0x7fdd18009040 comp rx=0 tx=0).stop 2024-09-13T18:27:03.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7fdd2e743700 1 -- 172.21.15.146:0/3427414200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2815ce10 msgr2=0x7fdd28148330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7fdd2e743700 1 --2- 172.21.15.146:0/3427414200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2815ce10 0x7fdd28148330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000ca80 tx=0x7fdd2000cab0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7fdd2e743700 1 -- 172.21.15.146:0/3427414200 shutdown_connections 2024-09-13T18:27:03.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7fdd2e743700 1 --2- 172.21.15.146:0/3427414200 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdd14066730 0x7fdd14068bf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7fdd2e743700 1 --2- 172.21.15.146:0/3427414200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2815ce10 0x7fdd28148330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7fdd2e743700 1 --2- 172.21.15.146:0/3427414200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd28153dc0 0x7fdd28147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7fdd2e743700 1 --2- 172.21.15.146:0/3427414200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd281539e0 0x7fdd2815a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7fdd2e743700 1 -- 172.21.15.146:0/3427414200 >> 172.21.15.146:0/3427414200 conn(0x7fdd2806f5c0 msgr2=0x7fdd28158ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7fdd2e743700 1 -- 172.21.15.146:0/3427414200 shutdown_connections 2024-09-13T18:27:03.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7fdd2e743700 1 -- 172.21.15.146:0/3427414200 wait complete. 2024-09-13T18:27:03.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7f5447fff700 1 -- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5434066730 msgr2=0x7f5434068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7f5447fff700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5434066730 0x7f5434068bf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5438000f60 tx=0x7f5438007e00 comp rx=0 tx=0).stop 2024-09-13T18:27:03.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7f5447fff700 1 -- 172.21.15.146:0/1165472849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f544814f650 msgr2=0x7f544814cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7f5447fff700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f544814f650 0x7f544814cbc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5440014040 tx=0x7f5440004af0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7f5447fff700 1 -- 172.21.15.146:0/1165472849 shutdown_connections 2024-09-13T18:27:03.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7f5447fff700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5434066730 0x7f5434068bf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7f5447fff700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f544814f650 0x7f544814cbc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7f5447fff700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544806eec0 0x7f544815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7f5447fff700 1 --2- 172.21.15.146:0/1165472849 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806c500 0x7f54481ed1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7f5447fff700 1 -- 172.21.15.146:0/1165472849 >> 172.21.15.146:0/1165472849 conn(0x7f5448066090 msgr2=0x7f54480c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7f5447fff700 1 -- 172.21.15.146:0/1165472849 shutdown_connections 2024-09-13T18:27:03.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:03.737+0000 7f5447fff700 1 -- 172.21.15.146:0/1165472849 wait complete. 2024-09-13T18:27:03.751 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.7 2024-09-13T18:27:03.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2024-09-13T18:27:03.757 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.3 2024-09-13T18:27:04.710 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:27:04.724 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T18:27:04.728 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:27:04.728 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T18:27:04.731 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T18:27:04.752 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T18:27:04.753 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:27:04.759 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:27:04.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbfa0ebf700 1 Processor -- start 2024-09-13T18:27:04.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbfa0ebf700 1 -- start start 2024-09-13T18:27:04.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbfa0ebf700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbf9c14f750 0x7fbf9c06eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbfa0ebf700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbf9c0a9490 0x7fbf9c0cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbfa0ebf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf9c0cb660 0x7fbf9c06f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbfa0ebf700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c14fb30 con 0x7fbf9c14f750 2024-09-13T18:27:04.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbfa0ebf700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c155580 con 0x7fbf9c0a9490 2024-09-13T18:27:04.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbfa0ebf700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c155860 con 0x7fbf9c0cb660 2024-09-13T18:27:04.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbf9b7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbf9c14f750 0x7fbf9c06eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbf9b7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbf9c14f750 0x7fbf9c06eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54654/0 (socket says 172.21.15.146:54654) 2024-09-13T18:27:04.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbf9b7fe700 1 -- 172.21.15.146:0/2382481604 learned_addr learned my addr 172.21.15.146:0/2382481604 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:04.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbf9bfff700 1 --2- 172.21.15.146:0/2382481604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf9c0cb660 0x7fbf9c06f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbf9affd700 1 --2- 172.21.15.146:0/2382481604 >> v2:172.21.15.154:3300/0 conn(0x7fbf9c0a9490 0x7fbf9c0cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbf9b7fe700 1 -- 172.21.15.146:0/2382481604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf9c0cb660 msgr2=0x7fbf9c06f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbf9b7fe700 1 --2- 172.21.15.146:0/2382481604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf9c0cb660 0x7fbf9c06f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbf9b7fe700 1 -- 172.21.15.146:0/2382481604 >> v2:172.21.15.154:3300/0 conn(0x7fbf9c0a9490 msgr2=0x7fbf9c0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbf9b7fe700 1 --2- 172.21.15.146:0/2382481604 >> v2:172.21.15.154:3300/0 conn(0x7fbf9c0a9490 0x7fbf9c0cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbf9b7fe700 1 -- 172.21.15.146:0/2382481604 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c155b40 con 0x7fbf9c14f750 2024-09-13T18:27:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbf9affd700 1 --2- 172.21.15.146:0/2382481604 >> v2:172.21.15.154:3300/0 conn(0x7fbf9c0a9490 0x7fbf9c0cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.857+0000 7fbf9b7fe700 1 --2- 172.21.15.146:0/2382481604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf9c14f750 0x7fbf9c06eea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0c9fc0 tx=0x7fbf8c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9958d8c8cd3443 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7fbf9a7fc700 1 -- 172.21.15.146:0/2382481604 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c011830 con 0x7fbf9c14f750 2024-09-13T18:27:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7fbf9a7fc700 1 -- 172.21.15.146:0/2382481604 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8c011990 con 0x7fbf9c14f750 2024-09-13T18:27:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7fbf9a7fc700 1 -- 172.21.15.146:0/2382481604 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c011d20 con 0x7fbf9c14f750 2024-09-13T18:27:04.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7fbfa0ebf700 1 -- 172.21.15.146:0/2382481604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf9c14f750 msgr2=0x7fbf9c06eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7fbfa0ebf700 1 --2- 172.21.15.146:0/2382481604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf9c14f750 0x7fbf9c06eea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0c9fc0 tx=0x7fbf8c009f90 comp rx=0 tx=0).stop 2024-09-13T18:27:04.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7fbfa0ebf700 1 -- 172.21.15.146:0/2382481604 shutdown_connections 2024-09-13T18:27:04.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7fbfa0ebf700 1 --2- 172.21.15.146:0/2382481604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf9c0cb660 0x7fbf9c06f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7fbfa0ebf700 1 --2- 172.21.15.146:0/2382481604 >> v2:172.21.15.154:3300/0 conn(0x7fbf9c0a9490 0x7fbf9c0cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7fbfa0ebf700 1 --2- 172.21.15.146:0/2382481604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf9c14f750 0x7fbf9c06eea0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7fbfa0ebf700 1 -- 172.21.15.146:0/2382481604 >> 172.21.15.146:0/2382481604 conn(0x7fbf9c066090 msgr2=0x7fbf9c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4941226700 1 Processor -- start 2024-09-13T18:27:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4941226700 1 -- start start 2024-09-13T18:27:04.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4941226700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f493c0caed0 0x7f493c0a9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4941226700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f493c0ca340 0x7f493c0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4941226700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f493c0cb2b0 0x7f493c0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4941226700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493c0c9930 con 0x7f493c0cb2b0 2024-09-13T18:27:04.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4941226700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493c06f5c0 con 0x7f493c0caed0 2024-09-13T18:27:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4941226700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c06fb40 con 0x7f493c0ca340 2024-09-13T18:27:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4940a25700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f493c0cb2b0 0x7f493c0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4940a25700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f493c0cb2b0 0x7f493c0c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54668/0 (socket says 172.21.15.146:54668) 2024-09-13T18:27:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4940a25700 1 -- 172.21.15.146:0/2239829851 learned_addr learned my addr 172.21.15.146:0/2239829851 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f493bfff700 1 --2- 172.21.15.146:0/2239829851 >> v2:172.21.15.154:3300/0 conn(0x7f493c0caed0 0x7f493c0a9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f493b7fe700 1 --2- 172.21.15.146:0/2239829851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f493c0ca340 0x7f493c0c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4940a25700 1 -- 172.21.15.146:0/2239829851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f493c0ca340 msgr2=0x7f493c0c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4940a25700 1 --2- 172.21.15.146:0/2239829851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f493c0ca340 0x7f493c0c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4940a25700 1 -- 172.21.15.146:0/2239829851 >> v2:172.21.15.154:3300/0 conn(0x7f493c0caed0 msgr2=0x7f493c0a9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4940a25700 1 --2- 172.21.15.146:0/2239829851 >> v2:172.21.15.154:3300/0 conn(0x7f493c0caed0 0x7f493c0a9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.861+0000 7f4940a25700 1 -- 172.21.15.146:0/2239829851 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c072500 con 0x7f493c0cb2b0 2024-09-13T18:27:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7f4940a25700 1 --2- 172.21.15.146:0/2239829851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb2b0 0x7f493c0c7c00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f493c0cb760 tx=0x7f492c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d18d579fe6e863 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbfa0ebf700 1 -- 172.21.15.146:0/2382481604 shutdown_connections 2024-09-13T18:27:04.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbfa0ebf700 1 -- 172.21.15.146:0/2382481604 wait complete. 2024-09-13T18:27:04.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbfa0ebf700 1 Processor -- start 2024-09-13T18:27:04.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbfa0ebf700 1 -- start start 2024-09-13T18:27:04.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbfa0ebf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf9c0a9490 0x7fbf9c156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbfa0ebf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf9c0cb660 0x7fbf9c06d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7f493affd700 1 -- 172.21.15.146:0/2239829851 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c011830 con 0x7f493c0cb2b0 2024-09-13T18:27:04.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbfa0ebf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf9c1ee810 0x7fbf9c1eec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbfa0ebf700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c1ef1d0 con 0x7fbf9c1ee810 2024-09-13T18:27:04.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbfa0ebf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0acd70 con 0x7fbf9c0a9490 2024-09-13T18:27:04.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbfa0ebf700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0ad8d0 con 0x7fbf9c0cb660 2024-09-13T18:27:04.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7f493affd700 1 -- 172.21.15.146:0/2239829851 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c011990 con 0x7f493c0cb2b0 2024-09-13T18:27:04.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7f493affd700 1 -- 172.21.15.146:0/2239829851 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c011d20 con 0x7f493c0cb2b0 2024-09-13T18:27:04.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7f4941226700 1 -- 172.21.15.146:0/2239829851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb2b0 msgr2=0x7f493c0c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7f4941226700 1 --2- 172.21.15.146:0/2239829851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb2b0 0x7f493c0c7c00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f493c0cb760 tx=0x7f492c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbf9bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf9c1ee810 0x7fbf9c1eec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbf9bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf9c1ee810 0x7fbf9c1eec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54670/0 (socket says 172.21.15.146:54670) 2024-09-13T18:27:04.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbf9bfff700 1 -- 172.21.15.146:0/983775892 learned_addr learned my addr 172.21.15.146:0/983775892 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:04.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbf9affd700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf9c0cb660 0x7fbf9c06d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbf9b7fe700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf9c0a9490 0x7fbf9c156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbf9bfff700 1 -- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf9c0cb660 msgr2=0x7fbf9c06d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbf9bfff700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf9c0cb660 0x7fbf9c06d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbf9bfff700 1 -- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf9c0a9490 msgr2=0x7fbf9c156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbf9bfff700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf9c0a9490 0x7fbf9c156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbf9bfff700 1 -- 172.21.15.146:0/983775892 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c009e00 con 0x7fbf9c1ee810 2024-09-13T18:27:04.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.865+0000 7fbf9bfff700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf9c1ee810 0x7fbf9c1eec90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400bc50 tx=0x7fbf94010420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.869+0000 7fbf83fff700 1 -- 172.21.15.146:0/983775892 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94010d10 con 0x7fbf9c1ee810 2024-09-13T18:27:04.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.869+0000 7fbf83fff700 1 -- 172.21.15.146:0/983775892 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf94011040 con 0x7fbf9c1ee810 2024-09-13T18:27:04.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.869+0000 7fbfa0ebf700 1 -- 172.21.15.146:0/983775892 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c0ad030 con 0x7fbf9c1ee810 2024-09-13T18:27:04.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.869+0000 7fbf83fff700 1 -- 172.21.15.146:0/983775892 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400b260 con 0x7fbf9c1ee810 2024-09-13T18:27:04.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.869+0000 7fbfa0ebf700 1 -- 172.21.15.146:0/983775892 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c0ad5f0 con 0x7fbf9c1ee810 2024-09-13T18:27:04.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7f4941226700 1 -- 172.21.15.146:0/2239829851 shutdown_connections 2024-09-13T18:27:04.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7fbf83fff700 1 -- 172.21.15.146:0/983775892 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf9400a530 con 0x7fbf9c1ee810 2024-09-13T18:27:04.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7fbf83fff700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbf88066780 0x7fbf88068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7fbf83fff700 1 -- 172.21.15.146:0/983775892 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbf94084c60 con 0x7fbf9c1ee810 2024-09-13T18:27:04.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7f4941226700 1 --2- 172.21.15.146:0/2239829851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f493c0ca340 0x7f493c0c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7f4941226700 1 --2- 172.21.15.146:0/2239829851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb2b0 0x7f493c0c7c00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7f4941226700 1 --2- 172.21.15.146:0/2239829851 >> v2:172.21.15.154:3300/0 conn(0x7f493c0caed0 0x7f493c0a9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7f4941226700 1 -- 172.21.15.146:0/2239829851 >> 172.21.15.146:0/2239829851 conn(0x7f493c066090 msgr2=0x7f493c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7fbfa0ebf700 1 -- 172.21.15.146:0/983775892 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c06eea0 con 0x7fbf9c1ee810 2024-09-13T18:27:04.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7fbf9b7fe700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbf88066780 0x7fbf88068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7f4941226700 1 -- 172.21.15.146:0/2239829851 shutdown_connections 2024-09-13T18:27:04.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7f4941226700 1 -- 172.21.15.146:0/2239829851 wait complete. 2024-09-13T18:27:04.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7fbf9b7fe700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbf88066780 0x7fbf88068c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c071bb0 tx=0x7fbf8c020040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7f4941226700 1 Processor -- start 2024-09-13T18:27:04.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.873+0000 7f4941226700 1 -- start start 2024-09-13T18:27:04.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4941226700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f493c0ca340 0x7f493c0af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4941226700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0caed0 0x7f493c0c2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4941226700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb2b0 0x7f493c1780b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4941226700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c169120 con 0x7f493c0cb2b0 2024-09-13T18:27:04.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4941226700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0c3390 con 0x7f493c0caed0 2024-09-13T18:27:04.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4941226700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0c0ec0 con 0x7f493c0ca340 2024-09-13T18:27:04.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4940a25700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb2b0 0x7f493c1780b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4940a25700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb2b0 0x7f493c1780b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54682/0 (socket says 172.21.15.146:54682) 2024-09-13T18:27:04.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4940a25700 1 -- 172.21.15.146:0/3289599827 learned_addr learned my addr 172.21.15.146:0/3289599827 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:04.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d619a700 1 Processor -- start 2024-09-13T18:27:04.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f493bfff700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f493c0ca340 0x7f493c0af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f493b7fe700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0caed0 0x7f493c0c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4940a25700 1 -- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f493c0ca340 msgr2=0x7f493c0af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4940a25700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f493c0ca340 0x7f493c0af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4940a25700 1 -- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0caed0 msgr2=0x7f493c0c2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4940a25700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0caed0 0x7f493c0c2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4940a25700 1 -- 172.21.15.146:0/3289599827 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c009570 con 0x7f493c0cb2b0 2024-09-13T18:27:04.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d619a700 1 -- start start 2024-09-13T18:27:04.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d2939f700 1 Processor -- start 2024-09-13T18:27:04.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f4940a25700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb2b0 0x7f493c1780b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f492c014040 tx=0x7f492c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d2939f700 1 -- start start 2024-09-13T18:27:04.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d619a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f25d00c9350 0x7f25d015a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d619a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f25d00a9490 0x7f25d00c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d619a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d00c8650 con 0x7f25d00c8f70 2024-09-13T18:27:04.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d619a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d00aa1a0 con 0x7f25d00a9490 2024-09-13T18:27:04.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d619a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d00aa480 con 0x7f25d00c9350 2024-09-13T18:27:04.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d2939f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9d240c9350 0x7f9d2415a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d2939f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9d240a9490 0x7f9d240c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d2939f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d240c8650 con 0x7f9d240c8f70 2024-09-13T18:27:04.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d2939f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d240aa1a0 con 0x7f9d240a9490 2024-09-13T18:27:04.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d2939f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d240aa480 con 0x7f9d240c9350 2024-09-13T18:27:04.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d23fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9d240c8f70 0x7f9d240cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35518/0 (socket says 172.21.15.146:35518) 2024-09-13T18:27:04.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d23fff700 1 -- 172.21.15.146:0/3219367492 learned_addr learned my addr 172.21.15.146:0/3219367492 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:04.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d5999700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f25d00a9490 0x7f25d00c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d237fe700 1 --2- 172.21.15.146:0/3219367492 >> v2:172.21.15.154:3300/0 conn(0x7f9d240c9350 0x7f9d2415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d28b9e700 1 --2- 172.21.15.146:0/3219367492 >> v2:172.21.15.146:3300/0 conn(0x7f9d240a9490 0x7f9d240c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d5999700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f25d00a9490 0x7f25d00c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54706/0 (socket says 172.21.15.146:54706) 2024-09-13T18:27:04.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d5198700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f25d00c8f70 0x7f25d00cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:54946/0 (socket says 172.21.15.146:54946) 2024-09-13T18:27:04.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d5999700 1 -- 172.21.15.146:0/1766380947 learned_addr learned my addr 172.21.15.146:0/1766380947 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:04.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d4997700 1 --2- 172.21.15.146:0/1766380947 >> v2:172.21.15.154:3300/0 conn(0x7f25d00c9350 0x7f25d015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d4997700 1 -- 172.21.15.146:0/1766380947 >> v2:172.21.15.146:3300/0 conn(0x7f25d00a9490 msgr2=0x7f25d00c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d4997700 1 --2- 172.21.15.146:0/1766380947 >> v2:172.21.15.146:3300/0 conn(0x7f25d00a9490 0x7f25d00c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d4997700 1 -- 172.21.15.146:0/1766380947 >> v1:172.21.15.154:6789/0 conn(0x7f25d00c8f70 legacy=0x7f25d00cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f25d4997700 1 -- 172.21.15.146:0/1766380947 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d0159890 con 0x7f25d00c9350 2024-09-13T18:27:04.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d28b9e700 1 -- 172.21.15.146:0/3219367492 >> v2:172.21.15.154:3300/0 conn(0x7f9d240c9350 msgr2=0x7f9d2415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d28b9e700 1 --2- 172.21.15.146:0/3219367492 >> v2:172.21.15.154:3300/0 conn(0x7f9d240c9350 0x7f9d2415a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d28b9e700 1 -- 172.21.15.146:0/3219367492 >> v1:172.21.15.146:6789/0 conn(0x7f9d240c8f70 legacy=0x7f9d240cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.877+0000 7f9d28b9e700 1 -- 172.21.15.146:0/3219367492 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d24159890 con 0x7f9d240a9490 2024-09-13T18:27:04.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f9d28b9e700 1 --2- 172.21.15.146:0/3219367492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d240a9490 0x7f9d240c76c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9d240c7300 tx=0x7f9d1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb401c3a6efdec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f25d4997700 1 --2- 172.21.15.146:0/1766380947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d00c9350 0x7f25d015a370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f25d00c7300 tx=0x7f25cc009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c7b0c5732e2b6a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f25c7fff700 1 -- 172.21.15.146:0/1766380947 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc011830 con 0x7f25d00c9350 2024-09-13T18:27:04.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f25c7fff700 1 -- 172.21.15.146:0/1766380947 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc011990 con 0x7f25d00c9350 2024-09-13T18:27:04.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f25c7fff700 1 -- 172.21.15.146:0/1766380947 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc011d20 con 0x7f25d00c9350 2024-09-13T18:27:04.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f25d619a700 1 -- 172.21.15.146:0/1766380947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d00c9350 msgr2=0x7f25d015a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f25d619a700 1 --2- 172.21.15.146:0/1766380947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d00c9350 0x7f25d015a370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f25d00c7300 tx=0x7f25cc009b30 comp rx=0 tx=0).stop 2024-09-13T18:27:04.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7efee594d700 1 Processor -- start 2024-09-13T18:27:04.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f4938ff9700 1 -- 172.21.15.146:0/3289599827 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c004450 con 0x7f493c0cb2b0 2024-09-13T18:27:04.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f4938ff9700 1 -- 172.21.15.146:0/3289599827 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c01e070 con 0x7f493c0cb2b0 2024-09-13T18:27:04.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f4938ff9700 1 -- 172.21.15.146:0/3289599827 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c003900 con 0x7f493c0cb2b0 2024-09-13T18:27:04.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f9d22ffd700 1 -- 172.21.15.146:0/3219367492 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c00e070 con 0x7f9d240a9490 2024-09-13T18:27:04.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f4941226700 1 -- 172.21.15.146:0/3289599827 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493c0c11a0 con 0x7f493c0cb2b0 2024-09-13T18:27:04.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7efee594d700 1 -- start start 2024-09-13T18:27:04.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7fbf83fff700 1 -- 172.21.15.146:0/983775892 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbf94051e70 con 0x7fbf9c1ee810 2024-09-13T18:27:04.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f4941226700 1 -- 172.21.15.146:0/3289599827 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c0c1760 con 0x7f493c0cb2b0 2024-09-13T18:27:04.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f25d619a700 1 -- 172.21.15.146:0/1766380947 shutdown_connections 2024-09-13T18:27:04.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f25d619a700 1 --2- 172.21.15.146:0/1766380947 >> v2:172.21.15.146:3300/0 conn(0x7f25d00a9490 0x7f25d00c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f25d619a700 1 --2- 172.21.15.146:0/1766380947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d00c9350 0x7f25d015a370 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f25d619a700 1 -- 172.21.15.146:0/1766380947 >> 172.21.15.146:0/1766380947 conn(0x7f25d0066090 msgr2=0x7f25d0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7efee594d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efee00cb2b0 0x7efee01610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7efee594d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efee00aa4d0 0x7efee015e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7efee594d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee0161fe0 con 0x7efee00caed0 2024-09-13T18:27:04.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7efee594d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee00a9930 con 0x7efee00cb2b0 2024-09-13T18:27:04.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7efee594d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee0161780 con 0x7efee00aa4d0 2024-09-13T18:27:04.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7f9d22ffd700 1 -- 172.21.15.146:0/3219367492 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c00bb20 con 0x7f9d240a9490 2024-09-13T18:27:04.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7efedffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efee00cb2b0 0x7efee01610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7efedffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efee00cb2b0 0x7efee01610f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54712/0 (socket says 172.21.15.146:54712) 2024-09-13T18:27:04.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.881+0000 7efedffff700 1 -- 172.21.15.146:0/3277375727 learned_addr learned my addr 172.21.15.146:0/3277375727 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:04.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7efee514c700 1 --2- 172.21.15.146:0/3277375727 >> v2:172.21.15.154:3300/0 conn(0x7efee00aa4d0 0x7efee015e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7efedffff700 1 -- 172.21.15.146:0/3277375727 >> v2:172.21.15.154:3300/0 conn(0x7efee00aa4d0 msgr2=0x7efee015e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7efedffff700 1 --2- 172.21.15.146:0/3277375727 >> v2:172.21.15.154:3300/0 conn(0x7efee00aa4d0 0x7efee015e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7efedffff700 1 -- 172.21.15.146:0/3277375727 >> v1:172.21.15.154:6789/0 conn(0x7efee00caed0 legacy=0x7efee00a9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7efedffff700 1 -- 172.21.15.146:0/3277375727 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee0161a20 con 0x7efee00cb2b0 2024-09-13T18:27:04.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f9d2939f700 1 -- 172.21.15.146:0/3219367492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d240a9490 msgr2=0x7f9d240c76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f9d2939f700 1 --2- 172.21.15.146:0/3219367492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d240a9490 0x7f9d240c76c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9d240c7300 tx=0x7f9d1c00b330 comp rx=0 tx=0).stop 2024-09-13T18:27:04.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7efee514c700 1 --2- 172.21.15.146:0/3277375727 >> v2:172.21.15.154:3300/0 conn(0x7efee00aa4d0 0x7efee015e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:04.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7efedffff700 1 --2- 172.21.15.146:0/3277375727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efee00cb2b0 0x7efee01610f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efee00723f0 tx=0x7efed4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df61a7cb0b1b18f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f4938ff9700 1 -- 172.21.15.146:0/3289599827 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f492c003a60 con 0x7f493c0cb2b0 2024-09-13T18:27:04.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f4938ff9700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4920066780 0x7f4920068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f4938ff9700 1 -- 172.21.15.146:0/3289599827 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f492c085390 con 0x7f493c0cb2b0 2024-09-13T18:27:04.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f25d619a700 1 -- 172.21.15.146:0/1766380947 shutdown_connections 2024-09-13T18:27:04.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f493bfff700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4920066780 0x7f4920068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f25d619a700 1 -- 172.21.15.146:0/1766380947 wait complete. 2024-09-13T18:27:04.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f493bfff700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4920066780 0x7f4920068c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4930009bf0 tx=0x7f4930009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f4941226700 1 -- 172.21.15.146:0/3289599827 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c0c6f50 con 0x7f493c0cb2b0 2024-09-13T18:27:04.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7efedf7fe700 1 -- 172.21.15.146:0/3277375727 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400b8a0 con 0x7efee00cb2b0 2024-09-13T18:27:04.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7efedf7fe700 1 -- 172.21.15.146:0/3277375727 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed400ba00 con 0x7efee00cb2b0 2024-09-13T18:27:04.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f9d2939f700 1 -- 172.21.15.146:0/3219367492 shutdown_connections 2024-09-13T18:27:04.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f9d2939f700 1 --2- 172.21.15.146:0/3219367492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d240a9490 0x7f9d240c76c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f9d2939f700 1 --2- 172.21.15.146:0/3219367492 >> v2:172.21.15.154:3300/0 conn(0x7f9d240c9350 0x7f9d2415a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.885+0000 7f9d2939f700 1 -- 172.21.15.146:0/3219367492 >> 172.21.15.146:0/3219367492 conn(0x7f9d24066090 msgr2=0x7f9d24066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7efedf7fe700 1 -- 172.21.15.146:0/3277375727 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400bd90 con 0x7efee00cb2b0 2024-09-13T18:27:04.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7efee594d700 1 -- 172.21.15.146:0/3277375727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efee00cb2b0 msgr2=0x7efee01610f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7efee594d700 1 --2- 172.21.15.146:0/3277375727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efee00cb2b0 0x7efee01610f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efee00723f0 tx=0x7efed4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7efee594d700 1 -- 172.21.15.146:0/3277375727 shutdown_connections 2024-09-13T18:27:04.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7efee594d700 1 --2- 172.21.15.146:0/3277375727 >> v2:172.21.15.154:3300/0 conn(0x7efee00aa4d0 0x7efee015e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7efee594d700 1 --2- 172.21.15.146:0/3277375727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efee00cb2b0 0x7efee01610f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7efee594d700 1 -- 172.21.15.146:0/3277375727 >> 172.21.15.146:0/3277375727 conn(0x7efee0066090 msgr2=0x7efee0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f9d2939f700 1 -- 172.21.15.146:0/3219367492 shutdown_connections 2024-09-13T18:27:04.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7efee594d700 1 -- 172.21.15.146:0/3277375727 shutdown_connections 2024-09-13T18:27:04.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f9d2939f700 1 -- 172.21.15.146:0/3219367492 wait complete. 2024-09-13T18:27:04.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f9d2939f700 1 Processor -- start 2024-09-13T18:27:04.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d619a700 1 Processor -- start 2024-09-13T18:27:04.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d619a700 1 -- start start 2024-09-13T18:27:04.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d619a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d00a9490 0x7f25d006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d619a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25d00c8f70 0x7f25d01579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d619a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25d00c9350 0x7f25d00aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d619a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d01dfc10 con 0x7f25d00c8f70 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d619a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d01e0ab0 con 0x7f25d00a9490 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d619a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00ac290 con 0x7f25d00c9350 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d4997700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25d00c8f70 0x7f25d01579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d4997700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25d00c8f70 0x7f25d01579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54728/0 (socket says 172.21.15.146:54728) 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d4997700 1 -- 172.21.15.146:0/2288986217 learned_addr learned my addr 172.21.15.146:0/2288986217 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d5198700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d00a9490 0x7f25d006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d5999700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25d00c9350 0x7f25d00aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d4997700 1 -- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25d00c9350 msgr2=0x7f25d00aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d4997700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25d00c9350 0x7f25d00aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d4997700 1 -- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d00a9490 msgr2=0x7f25d006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d4997700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d00a9490 0x7f25d006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d4997700 1 -- 172.21.15.146:0/2288986217 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc0094b0 con 0x7f25d00c8f70 2024-09-13T18:27:04.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f25d4997700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25d00c8f70 0x7f25d01579e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009b70 tx=0x7f25cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7f9d2939f700 1 -- start start 2024-09-13T18:27:04.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.889+0000 7efee594d700 1 -- 172.21.15.146:0/3277375727 wait complete. 2024-09-13T18:27:04.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d2939f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d240a9490 0x7f9d240c2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d2939f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d240c8f70 0x7f9d240c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d2939f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d240c9350 0x7f9d24157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d2939f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2406eb50 con 0x7f9d240c9350 2024-09-13T18:27:04.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d2939f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d241360a0 con 0x7f9d240c8f70 2024-09-13T18:27:04.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d2939f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d240c2170 con 0x7f9d240a9490 2024-09-13T18:27:04.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee594d700 1 Processor -- start 2024-09-13T18:27:04.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee594d700 1 -- start start 2024-09-13T18:27:04.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d28b9e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d240c9350 0x7f9d24157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d28b9e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d240c9350 0x7f9d24157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54734/0 (socket says 172.21.15.146:54734) 2024-09-13T18:27:04.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d28b9e700 1 -- 172.21.15.146:0/1552193070 learned_addr learned my addr 172.21.15.146:0/1552193070 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:04.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d237fe700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d240c8f70 0x7f9d240c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f77f5215700 1 Processor -- start 2024-09-13T18:27:04.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d23fff700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d240a9490 0x7f9d240c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f77f5215700 1 -- start start 2024-09-13T18:27:04.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d28b9e700 1 -- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d240a9490 msgr2=0x7f9d240c2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d28b9e700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d240a9490 0x7f9d240c2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d28b9e700 1 -- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d240c8f70 msgr2=0x7f9d240c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d28b9e700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d240c8f70 0x7f9d240c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d28b9e700 1 -- 172.21.15.146:0/1552193070 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c009a70 con 0x7f9d240c9350 2024-09-13T18:27:04.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d28b9e700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d240c9350 0x7f9d24157200 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9d240cac30 tx=0x7f9d1c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee594d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efee00aa4d0 0x7efee015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee594d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00caed0 0x7efee00ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee594d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efee00ad510 0x7efee00aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee594d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01f7b30 con 0x7efee00aa4d0 2024-09-13T18:27:04.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee594d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01f75b0 con 0x7efee00ad510 2024-09-13T18:27:04.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee594d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee01f7850 con 0x7efee00caed0 2024-09-13T18:27:04.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee494b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efee00aa4d0 0x7efee015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee494b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efee00aa4d0 0x7efee015d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54742/0 (socket says 172.21.15.146:54742) 2024-09-13T18:27:04.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee494b700 1 -- 172.21.15.146:0/657385626 learned_addr learned my addr 172.21.15.146:0/657385626 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:04.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee514c700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efee00ad510 0x7efee00aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efedffff700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00caed0 0x7efee00ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee494b700 1 -- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00caed0 msgr2=0x7efee00ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee494b700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00caed0 0x7efee00ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee494b700 1 -- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efee00ad510 msgr2=0x7efee00aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee494b700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efee00ad510 0x7efee00aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee494b700 1 -- 172.21.15.146:0/657385626 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed4009570 con 0x7efee00aa4d0 2024-09-13T18:27:04.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f77f5215700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f77f00aa4d0 0x7f77f01610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f77f5215700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f0161fe0 con 0x7f77f00cb2b0 2024-09-13T18:27:04.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f77f5215700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f00a9930 con 0x7f77f00caed0 2024-09-13T18:27:04.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f77f5215700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f0161780 con 0x7f77f00aa4d0 2024-09-13T18:27:04.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f25c5ffb700 1 -- 172.21.15.146:0/2288986217 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0045d0 con 0x7f25d00c8f70 2024-09-13T18:27:04.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f25d619a700 1 -- 172.21.15.146:0/2288986217 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d006d610 con 0x7f25d00c8f70 2024-09-13T18:27:04.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f25d619a700 1 -- 172.21.15.146:0/2288986217 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d00ac810 con 0x7f25d00c8f70 2024-09-13T18:27:04.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d20ff9700 1 -- 172.21.15.146:0/1552193070 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c01c810 con 0x7f9d240c9350 2024-09-13T18:27:04.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f77effff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f77f00caed0 0x7f77f00a9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:54970/0 (socket says 172.21.15.146:54970) 2024-09-13T18:27:04.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f77effff700 1 -- 172.21.15.146:0/2471455339 learned_addr learned my addr 172.21.15.146:0/2471455339 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:04.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f77f4a14700 1 --2- 172.21.15.146:0/2471455339 >> v2:172.21.15.146:3300/0 conn(0x7f77f00aa4d0 0x7f77f01610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7efee494b700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efee00aa4d0 0x7efee015d380 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efed000c500 tx=0x7efed000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f4938ff9700 1 -- 172.21.15.146:0/3289599827 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f492c052610 con 0x7f493c0cb2b0 2024-09-13T18:27:04.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d2939f700 1 -- 172.21.15.146:0/1552193070 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2406e870 con 0x7f9d240c9350 2024-09-13T18:27:04.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f9d2939f700 1 -- 172.21.15.146:0/1552193070 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d240afa20 con 0x7f9d240c9350 2024-09-13T18:27:04.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.893+0000 7f25c5ffb700 1 -- 172.21.15.146:0/2288986217 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc01f070 con 0x7f25d00c8f70 2024-09-13T18:27:04.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f25c5ffb700 1 -- 172.21.15.146:0/2288986217 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0038c0 con 0x7f25d00c8f70 2024-09-13T18:27:04.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f9d2939f700 1 -- 172.21.15.146:0/1552193070 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d240c9da0 con 0x7f9d240c9350 2024-09-13T18:27:04.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f25d619a700 1 -- 172.21.15.146:0/2288986217 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b0005310 con 0x7f25d00c8f70 2024-09-13T18:27:04.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f25c5ffb700 1 -- 172.21.15.146:0/2288986217 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f25cc003a20 con 0x7f25d00c8f70 2024-09-13T18:27:04.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f25c5ffb700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f25b4066730 0x7f25b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f25c5ffb700 1 -- 172.21.15.146:0/2288986217 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f25cc021070 con 0x7f25d00c8f70 2024-09-13T18:27:04.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f25d5198700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f25b4066730 0x7f25b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f25d5198700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f25b4066730 0x7f25b4068bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f25c0000f60 tx=0x7f25c0007f30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f563b105700 1 Processor -- start 2024-09-13T18:27:04.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.901+0000 7f9d20ff9700 1 -- 172.21.15.146:0/1552193070 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c020070 con 0x7f9d240c9350 2024-09-13T18:27:04.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.901+0000 7f9d20ff9700 1 -- 172.21.15.146:0/1552193070 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c012430 con 0x7f9d240c9350 2024-09-13T18:27:04.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f77f4a14700 1 -- 172.21.15.146:0/2471455339 >> v1:172.21.15.154:6789/0 conn(0x7f77f00caed0 legacy=0x7f77f00a9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f77f4a14700 1 -- 172.21.15.146:0/2471455339 >> v1:172.21.15.146:6789/0 conn(0x7f77f00cb2b0 legacy=0x7f77f015e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f77eeffd700 1 -- 172.21.15.146:0/2471455339 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941338530 0 0) 0x7f77f00a9930 con 0x7f77f00caed0 2024-09-13T18:27:04.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f77f4a14700 1 -- 172.21.15.146:0/2471455339 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f0161a20 con 0x7f77f00aa4d0 2024-09-13T18:27:04.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f77f4a14700 1 --2- 172.21.15.146:0/2471455339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f00aa4d0 0x7f77f01610f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009850 tx=0x7f77e0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae721c2ad1a0ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f77eeffd700 1 -- 172.21.15.146:0/2471455339 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0004230 con 0x7f77f00aa4d0 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f77eeffd700 1 -- 172.21.15.146:0/2471455339 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e0004390 con 0x7f77f00aa4d0 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f77eeffd700 1 -- 172.21.15.146:0/2471455339 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e00036c0 con 0x7f77f00aa4d0 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f77f5215700 1 -- 172.21.15.146:0/2471455339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f00aa4d0 msgr2=0x7f77f01610f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f77f5215700 1 --2- 172.21.15.146:0/2471455339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f00aa4d0 0x7f77f01610f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009850 tx=0x7f77e0009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f77f5215700 1 -- 172.21.15.146:0/2471455339 shutdown_connections 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f77f5215700 1 --2- 172.21.15.146:0/2471455339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f00aa4d0 0x7f77f01610f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f77f5215700 1 -- 172.21.15.146:0/2471455339 >> 172.21.15.146:0/2471455339 conn(0x7f77f0066090 msgr2=0x7f77f0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f563b105700 1 -- start start 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f563b105700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f563406eec0 0x7f56340a9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f563b105700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563406fef0 con 0x7f56340caed0 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f563b105700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5634070790 con 0x7f56340cb2b0 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f563b105700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56340aa770 con 0x7f563406eec0 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f563a103700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f56340caed0 0x7f5634072460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35530/0 (socket says 172.21.15.146:35530) 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f563a103700 1 -- 172.21.15.146:0/564249526 learned_addr learned my addr 172.21.15.146:0/564249526 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f563a904700 1 --2- 172.21.15.146:0/564249526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f563406eec0 0x7f56340a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f563a904700 1 -- 172.21.15.146:0/564249526 >> v1:172.21.15.154:6789/0 conn(0x7f56340cb2b0 legacy=0x7f563414f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f563a904700 1 -- 172.21.15.146:0/564249526 >> v1:172.21.15.146:6789/0 conn(0x7f56340caed0 legacy=0x7f5634072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.897+0000 7f563a904700 1 -- 172.21.15.146:0/564249526 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563406f5c0 con 0x7f563406eec0 2024-09-13T18:27:04.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.901+0000 7f563a904700 1 --2- 172.21.15.146:0/564249526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f563406eec0 0x7f56340a9500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5624009800 tx=0x7f5624009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76d1c21c2ef873dd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.901+0000 7efedd7fa700 1 -- 172.21.15.146:0/657385626 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00052d0 con 0x7efee00aa4d0 2024-09-13T18:27:04.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.901+0000 7efedd7fa700 1 -- 172.21.15.146:0/657385626 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed0005430 con 0x7efee00aa4d0 2024-09-13T18:27:04.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.901+0000 7efedd7fa700 1 -- 172.21.15.146:0/657385626 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00055a0 con 0x7efee00aa4d0 2024-09-13T18:27:04.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.901+0000 7efee594d700 1 -- 172.21.15.146:0/657385626 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee0123f80 con 0x7efee00aa4d0 2024-09-13T18:27:04.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.901+0000 7efee594d700 1 -- 172.21.15.146:0/657385626 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee0124540 con 0x7efee00aa4d0 2024-09-13T18:27:04.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77f5215700 1 -- 172.21.15.146:0/2471455339 shutdown_connections 2024-09-13T18:27:04.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f5639101700 1 -- 172.21.15.146:0/564249526 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624004230 con 0x7f563406eec0 2024-09-13T18:27:04.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f5639101700 1 -- 172.21.15.146:0/564249526 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5624004390 con 0x7f563406eec0 2024-09-13T18:27:04.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f5639101700 1 -- 172.21.15.146:0/564249526 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56240036c0 con 0x7f563406eec0 2024-09-13T18:27:04.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77f5215700 1 -- 172.21.15.146:0/2471455339 wait complete. 2024-09-13T18:27:04.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77f5215700 1 Processor -- start 2024-09-13T18:27:04.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77f5215700 1 -- start start 2024-09-13T18:27:04.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f25c5ffb700 1 -- 172.21.15.146:0/2288986217 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f25cc052cd0 con 0x7f25d00c8f70 2024-09-13T18:27:04.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77f5215700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77f00aa4d0 0x7f77f01251f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77f5215700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77f00caed0 0x7f77f00c0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f563b105700 1 -- 172.21.15.146:0/564249526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f563406eec0 msgr2=0x7f56340a9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f563b105700 1 --2- 172.21.15.146:0/564249526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f563406eec0 0x7f56340a9500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5624009800 tx=0x7f5624009b10 comp rx=0 tx=0).stop 2024-09-13T18:27:04.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f563b105700 1 -- 172.21.15.146:0/564249526 shutdown_connections 2024-09-13T18:27:04.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f563b105700 1 --2- 172.21.15.146:0/564249526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f563406eec0 0x7f56340a9500 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f563b105700 1 -- 172.21.15.146:0/564249526 >> 172.21.15.146:0/564249526 conn(0x7f5634066090 msgr2=0x7f5634066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77f5215700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f00cb2b0 0x7f77f01f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77f5215700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f015d8e0 con 0x7f77f00cb2b0 2024-09-13T18:27:04.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77f5215700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00c0bd0 con 0x7f77f00caed0 2024-09-13T18:27:04.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77f5215700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f01f6bb0 con 0x7f77f00aa4d0 2024-09-13T18:27:04.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f563b105700 1 -- 172.21.15.146:0/564249526 shutdown_connections 2024-09-13T18:27:04.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77ef7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77f00caed0 0x7f77f00c0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77ef7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77f00caed0 0x7f77f00c0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:46738/0 (socket says 172.21.15.146:46738) 2024-09-13T18:27:04.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77ef7fe700 1 -- 172.21.15.146:0/819587651 learned_addr learned my addr 172.21.15.146:0/819587651 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:04.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77effff700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77f00aa4d0 0x7f77f01251f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77f4a14700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f00cb2b0 0x7f77f01f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77ef7fe700 1 -- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77f00aa4d0 msgr2=0x7f77f01251f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77ef7fe700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77f00aa4d0 0x7f77f01251f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77ef7fe700 1 -- 172.21.15.146:0/819587651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f00cb2b0 msgr2=0x7f77f01f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77ef7fe700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f00cb2b0 0x7f77f01f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.905+0000 7f77ef7fe700 1 -- 172.21.15.146:0/819587651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e00094b0 con 0x7f77f00caed0 2024-09-13T18:27:04.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7f77ef7fe700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77f00caed0 0x7f77f00c0690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f77e400c500 tx=0x7f77e400c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7efee594d700 1 -- 172.21.15.146:0/657385626 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee0072460 con 0x7efee00aa4d0 2024-09-13T18:27:04.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7f77ecff9700 1 -- 172.21.15.146:0/819587651 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e40054e0 con 0x7f77f00caed0 2024-09-13T18:27:04.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7f77f5215700 1 -- 172.21.15.146:0/819587651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f01f39e0 con 0x7f77f00caed0 2024-09-13T18:27:04.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7f77ecff9700 1 -- 172.21.15.146:0/819587651 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e40036a0 con 0x7f77f00caed0 2024-09-13T18:27:04.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7f77ecff9700 1 -- 172.21.15.146:0/819587651 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4003a50 con 0x7f77f00caed0 2024-09-13T18:27:04.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7f77f5215700 1 -- 172.21.15.146:0/819587651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f015d600 con 0x7f77f00caed0 2024-09-13T18:27:04.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7f9d20ff9700 1 -- 172.21.15.146:0/1552193070 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d1c01c970 con 0x7f9d240c9350 2024-09-13T18:27:04.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7f9d20ff9700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d10066730 0x7f9d10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7f9d20ff9700 1 -- 172.21.15.146:0/1552193070 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9d1c086770 con 0x7f9d240c9350 2024-09-13T18:27:04.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7f9d23fff700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d10066730 0x7f9d10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7f563b105700 1 -- 172.21.15.146:0/564249526 wait complete. 2024-09-13T18:27:04.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7efedd7fa700 1 -- 172.21.15.146:0/657385626 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efed00038e0 con 0x7efee00aa4d0 2024-09-13T18:27:04.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7efedd7fa700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efecc066730 0x7efecc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7efedd7fa700 1 -- 172.21.15.146:0/657385626 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efed0083950 con 0x7efee00aa4d0 2024-09-13T18:27:04.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7efedffff700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efecc066730 0x7efecc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7efedffff700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efecc066730 0x7efecc068bf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efee00cb6c0 tx=0x7efed4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.909+0000 7f9d23fff700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d10066730 0x7f9d10068bf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d14000f60 tx=0x7f9d14007f30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f563b105700 1 Processor -- start 2024-09-13T18:27:04.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f563b105700 1 -- start start 2024-09-13T18:27:04.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f563b105700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563406eec0 0x7f5634150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f9d20ff9700 1 -- 172.21.15.146:0/1552193070 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9d1c0537f0 con 0x7f9d240c9350 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f563b105700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56340caed0 0x7f56341f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f563b105700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56340cb2b0 0x7f563413d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f563b105700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56341513a0 con 0x7f563406eec0 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f563b105700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56341f22e0 con 0x7f56340cb2b0 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f563b105700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56341f25c0 con 0x7f56340caed0 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f563a103700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563406eec0 0x7f5634150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f563a103700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563406eec0 0x7f5634150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54780/0 (socket says 172.21.15.146:54780) 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f563a103700 1 -- 172.21.15.146:0/2295135153 learned_addr learned my addr 172.21.15.146:0/2295135153 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:04.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f77ecff9700 1 -- 172.21.15.146:0/819587651 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f77e4003bb0 con 0x7f77f00caed0 2024-09-13T18:27:04.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f5639902700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56340caed0 0x7f56341f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f563a904700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56340cb2b0 0x7f563413d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f77ecff9700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f77dc066730 0x7f77dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f77ecff9700 1 -- 172.21.15.146:0/819587651 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f77e4083880 con 0x7f77f00caed0 2024-09-13T18:27:04.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f5639902700 1 -- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56340cb2b0 msgr2=0x7f563413d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f5639902700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56340cb2b0 0x7f563413d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f5639902700 1 -- 172.21.15.146:0/2295135153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563406eec0 msgr2=0x7f5634150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f5639902700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563406eec0 0x7f5634150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f5639902700 1 -- 172.21.15.146:0/2295135153 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56240094b0 con 0x7f56340caed0 2024-09-13T18:27:04.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f77effff700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f77dc066730 0x7f77dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.913+0000 7f5639902700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56340caed0 0x7f56341f2a50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f562c001f60 tx=0x7f562c00a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.917+0000 7f77effff700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f77dc066730 0x7f77dc068bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f77e8002230 tx=0x7f77e8001dd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.917+0000 7f77f5215700 1 -- 172.21.15.146:0/819587651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d0005310 con 0x7f77f00caed0 2024-09-13T18:27:04.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.921+0000 7efedd7fa700 1 -- 172.21.15.146:0/657385626 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efed00509d0 con 0x7efee00aa4d0 2024-09-13T18:27:04.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.925+0000 7f562affd700 1 -- 172.21.15.146:0/2295135153 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c003e50 con 0x7f56340caed0 2024-09-13T18:27:04.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.925+0000 7f563b105700 1 -- 172.21.15.146:0/2295135153 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563414f420 con 0x7f56340caed0 2024-09-13T18:27:04.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.925+0000 7f563b105700 1 -- 172.21.15.146:0/2295135153 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563414f9c0 con 0x7f56340caed0 2024-09-13T18:27:04.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.925+0000 7f562affd700 1 -- 172.21.15.146:0/2295135153 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c0020b0 con 0x7f56340caed0 2024-09-13T18:27:04.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.925+0000 7f562affd700 1 -- 172.21.15.146:0/2295135153 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c00ab90 con 0x7f56340caed0 2024-09-13T18:27:04.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.925+0000 7f562affd700 1 -- 172.21.15.146:0/2295135153 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f562c00acf0 con 0x7f56340caed0 2024-09-13T18:27:04.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.925+0000 7f77ecff9700 1 -- 172.21.15.146:0/819587651 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f77e4050860 con 0x7f77f00caed0 2024-09-13T18:27:04.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.925+0000 7f562affd700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5618066830 0x7f5618068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.925+0000 7f562affd700 1 -- 172.21.15.146:0/2295135153 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f562c084000 con 0x7f56340caed0 2024-09-13T18:27:04.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0903fff700 1 Processor -- start 2024-09-13T18:27:04.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f563b105700 1 -- 172.21.15.146:0/2295135153 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561c005310 con 0x7f56340caed0 2024-09-13T18:27:04.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0903fff700 1 -- start start 2024-09-13T18:27:04.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0903fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09040c9350 0x7f090415a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f563a103700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5618066830 0x7f5618068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f563a103700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5618066830 0x7f5618068cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56300017c0 tx=0x7f5630009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0903fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09040c8650 con 0x7f09040a9490 2024-09-13T18:27:04.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0903fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09040aa1a0 con 0x7f09040c8f70 2024-09-13T18:27:04.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0903fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09040aa480 con 0x7f09040c9350 2024-09-13T18:27:04.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0902ffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f09040c8f70 0x7f09040cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:54982/0 (socket says 172.21.15.146:54982) 2024-09-13T18:27:04.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0902ffd700 1 -- 172.21.15.146:0/2226990432 learned_addr learned my addr 172.21.15.146:0/2226990432 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:04.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f09027fc700 1 --2- 172.21.15.146:0/2226990432 >> v2:172.21.15.154:3300/0 conn(0x7f09040c9350 0x7f090415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0901ffb700 1 -- 172.21.15.146:0/2226990432 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393130191 0 0) 0x7f09040aa1a0 con 0x7f09040c8f70 2024-09-13T18:27:04.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0901ffb700 1 -- 172.21.15.146:0/2226990432 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08ec0035d0 con 0x7f09040c8f70 2024-09-13T18:27:04.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0901ffb700 1 -- 172.21.15.146:0/2226990432 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10297096 0 0) 0x7f09040c8650 con 0x7f09040a9490 2024-09-13T18:27:04.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0901ffb700 1 -- 172.21.15.146:0/2226990432 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09040aa1a0 con 0x7f09040a9490 2024-09-13T18:27:04.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f09027fc700 1 -- 172.21.15.146:0/2226990432 >> v1:172.21.15.154:6789/0 conn(0x7f09040c8f70 legacy=0x7f09040cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f09027fc700 1 -- 172.21.15.146:0/2226990432 >> v1:172.21.15.146:6789/0 conn(0x7f09040a9490 legacy=0x7f0904164020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f09027fc700 1 -- 172.21.15.146:0/2226990432 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0904159890 con 0x7f09040c9350 2024-09-13T18:27:04.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f09027fc700 1 --2- 172.21.15.146:0/2226990432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09040c9350 0x7f090415a370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009d90 tx=0x7f08fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a079514e77b336c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0901ffb700 1 -- 172.21.15.146:0/2226990432 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00e070 con 0x7f09040c9350 2024-09-13T18:27:04.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0901ffb700 1 -- 172.21.15.146:0/2226990432 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc00bb20 con 0x7f09040c9350 2024-09-13T18:27:04.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.933+0000 7f0901ffb700 1 -- 172.21.15.146:0/2226990432 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00be90 con 0x7f09040c9350 2024-09-13T18:27:04.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.937+0000 7f0903fff700 1 -- 172.21.15.146:0/2226990432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09040c9350 msgr2=0x7f090415a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.937+0000 7f0903fff700 1 --2- 172.21.15.146:0/2226990432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09040c9350 0x7f090415a370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009d90 tx=0x7f08fc00b330 comp rx=0 tx=0).stop 2024-09-13T18:27:04.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.937+0000 7f562affd700 1 -- 172.21.15.146:0/2295135153 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f562c051060 con 0x7f56340caed0 2024-09-13T18:27:04.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f0903fff700 1 -- 172.21.15.146:0/2226990432 shutdown_connections 2024-09-13T18:27:04.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f0903fff700 1 --2- 172.21.15.146:0/2226990432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09040c9350 0x7f090415a370 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f0903fff700 1 -- 172.21.15.146:0/2226990432 >> 172.21.15.146:0/2226990432 conn(0x7f0904066090 msgr2=0x7f0904066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f0903fff700 1 -- 172.21.15.146:0/2226990432 shutdown_connections 2024-09-13T18:27:04.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f0903fff700 1 -- 172.21.15.146:0/2226990432 wait complete. 2024-09-13T18:27:04.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f0903fff700 1 Processor -- start 2024-09-13T18:27:04.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f0903fff700 1 -- start start 2024-09-13T18:27:04.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f0903fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09040a9490 0x7f09040c0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f0903fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040c8f70 0x7f09041f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f0903fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09040c9350 0x7f0904156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f0903fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040c0970 con 0x7f09040c9350 2024-09-13T18:27:04.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f0903fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09041f2620 con 0x7f09040a9490 2024-09-13T18:27:04.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f0903fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09040c2c40 con 0x7f09040c8f70 2024-09-13T18:27:04.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f09027fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040c8f70 0x7f09041f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f09027fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040c8f70 0x7f09041f1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41442/0 (socket says 172.21.15.146:41442) 2024-09-13T18:27:04.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.941+0000 7f09027fc700 1 -- 172.21.15.146:0/447674611 learned_addr learned my addr 172.21.15.146:0/447674611 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:04.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.945+0000 7f0902ffd700 1 --2- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09040a9490 0x7f09040c0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.945+0000 7f09027fc700 1 -- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09040a9490 msgr2=0x7f09040c0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.945+0000 7f09027fc700 1 --2- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09040a9490 0x7f09040c0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.945+0000 7f09027fc700 1 -- 172.21.15.146:0/447674611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09040c9350 msgr2=0x7f0904156a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:27:04.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.945+0000 7f09027fc700 1 --2- 172.21.15.146:0/447674611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09040c9350 0x7f0904156a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.945+0000 7f09027fc700 1 -- 172.21.15.146:0/447674611 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc009a70 con 0x7f09040c8f70 2024-09-13T18:27:04.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.945+0000 7f09027fc700 1 --2- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040c8f70 0x7f09041f1bd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009470 tx=0x7f08fc004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.953+0000 7f08f3fff700 1 -- 172.21.15.146:0/447674611 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc012840 con 0x7f09040c8f70 2024-09-13T18:27:04.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.953+0000 7f08f3fff700 1 -- 172.21.15.146:0/447674611 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc0129a0 con 0x7f09040c8f70 2024-09-13T18:27:04.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.953+0000 7f08f3fff700 1 -- 172.21.15.146:0/447674611 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc012d50 con 0x7f09040c8f70 2024-09-13T18:27:04.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.953+0000 7f0903fff700 1 -- 172.21.15.146:0/447674611 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09040c2f20 con 0x7f09040c8f70 2024-09-13T18:27:04.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.953+0000 7f0903fff700 1 -- 172.21.15.146:0/447674611 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09040af980 con 0x7f09040c8f70 2024-09-13T18:27:04.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.953+0000 7f0903fff700 1 -- 172.21.15.146:0/447674611 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e8005310 con 0x7f09040c8f70 2024-09-13T18:27:04.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.953+0000 7f08f3fff700 1 -- 172.21.15.146:0/447674611 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f08fc026410 con 0x7f09040c8f70 2024-09-13T18:27:04.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.953+0000 7f08f3fff700 1 --2- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f08ec066d30 0x7f08ec0691f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.953+0000 7f08f3fff700 1 -- 172.21.15.146:0/447674611 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f08fc086dc0 con 0x7f09040c8f70 2024-09-13T18:27:04.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.953+0000 7f0902ffd700 1 --2- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f08ec066d30 0x7f08ec0691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.957+0000 7f0902ffd700 1 --2- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f08ec066d30 0x7f08ec0691f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f08f4000f60 tx=0x7f08f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:04.961+0000 7f08f3fff700 1 -- 172.21.15.146:0/447674611 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f08fc054040 con 0x7f09040c8f70 2024-09-13T18:27:05.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.177+0000 7fbfa0ebf700 1 -- 172.21.15.146:0/983775892 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbf9c0aa690 con 0x7fbf9c1ee810 2024-09-13T18:27:05.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.177+0000 7fbf83fff700 1 -- 172.21.15.146:0/983775892 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbf94055490 con 0x7fbf9c1ee810 2024-09-13T18:27:05.184 INFO:teuthology.orchestra.run.smithi146.stdout:64424509448 2024-09-13T18:27:05.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.181+0000 7fbf81ffb700 1 -- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbf88066780 msgr2=0x7fbf88068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.181+0000 7fbf81ffb700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbf88066780 0x7fbf88068c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c071bb0 tx=0x7fbf8c020040 comp rx=0 tx=0).stop 2024-09-13T18:27:05.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.181+0000 7fbf81ffb700 1 -- 172.21.15.146:0/983775892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf9c1ee810 msgr2=0x7fbf9c1eec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.181+0000 7fbf81ffb700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf9c1ee810 0x7fbf9c1eec90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400bc50 tx=0x7fbf94010420 comp rx=0 tx=0).stop 2024-09-13T18:27:05.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.181+0000 7fbf81ffb700 1 -- 172.21.15.146:0/983775892 shutdown_connections 2024-09-13T18:27:05.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.181+0000 7fbf81ffb700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbf88066780 0x7fbf88068c40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.181+0000 7fbf81ffb700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf9c1ee810 0x7fbf9c1eec90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.181+0000 7fbf81ffb700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf9c0cb660 0x7fbf9c06d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.181+0000 7fbf81ffb700 1 --2- 172.21.15.146:0/983775892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf9c0a9490 0x7fbf9c156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.181+0000 7fbf81ffb700 1 -- 172.21.15.146:0/983775892 >> 172.21.15.146:0/983775892 conn(0x7fbf9c066090 msgr2=0x7fbf9c154720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.181+0000 7fbf81ffb700 1 -- 172.21.15.146:0/983775892 shutdown_connections 2024-09-13T18:27:05.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.181+0000 7fbf81ffb700 1 -- 172.21.15.146:0/983775892 wait complete. 2024-09-13T18:27:05.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.4 2024-09-13T18:27:05.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.205+0000 7f4941226700 1 -- 172.21.15.146:0/3289599827 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f493c0709e0 con 0x7f493c0cb2b0 2024-09-13T18:27:05.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.209+0000 7f4938ff9700 1 -- 172.21.15.146:0/3289599827 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f492c015090 con 0x7f493c0cb2b0 2024-09-13T18:27:05.214 INFO:teuthology.orchestra.run.smithi146.stdout:55834574856 2024-09-13T18:27:05.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.209+0000 7f4941226700 1 -- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4920066780 msgr2=0x7f4920068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.209+0000 7f4941226700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4920066780 0x7f4920068c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4930009bf0 tx=0x7f4930009240 comp rx=0 tx=0).stop 2024-09-13T18:27:05.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.209+0000 7f4941226700 1 -- 172.21.15.146:0/3289599827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb2b0 msgr2=0x7f493c1780b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.209+0000 7f4941226700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb2b0 0x7f493c1780b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f492c014040 tx=0x7f492c004770 comp rx=0 tx=0).stop 2024-09-13T18:27:05.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.209+0000 7f4941226700 1 -- 172.21.15.146:0/3289599827 shutdown_connections 2024-09-13T18:27:05.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.213+0000 7f4941226700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4920066780 0x7f4920068c40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.213+0000 7f4941226700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb2b0 0x7f493c1780b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.213+0000 7f4941226700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0caed0 0x7f493c0c2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.213+0000 7f4941226700 1 --2- 172.21.15.146:0/3289599827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f493c0ca340 0x7f493c0af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.213+0000 7f4941226700 1 -- 172.21.15.146:0/3289599827 >> 172.21.15.146:0/3289599827 conn(0x7f493c066090 msgr2=0x7f493c0a9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.213+0000 7f4941226700 1 -- 172.21.15.146:0/3289599827 shutdown_connections 2024-09-13T18:27:05.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.213+0000 7f4941226700 1 -- 172.21.15.146:0/3289599827 wait complete. 2024-09-13T18:27:05.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.213+0000 7efee594d700 1 -- 172.21.15.146:0/657385626 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efee00c7310 con 0x7efee00aa4d0 2024-09-13T18:27:05.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.217+0000 7efedd7fa700 1 -- 172.21.15.146:0/657385626 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efed0017020 con 0x7efee00aa4d0 2024-09-13T18:27:05.222 INFO:teuthology.orchestra.run.smithi146.stdout:60129542154 2024-09-13T18:27:05.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.217+0000 7efee594d700 1 -- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efecc066730 msgr2=0x7efecc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.217+0000 7efee594d700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efecc066730 0x7efecc068bf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efee00cb6c0 tx=0x7efed4008030 comp rx=0 tx=0).stop 2024-09-13T18:27:05.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.217+0000 7efee594d700 1 -- 172.21.15.146:0/657385626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efee00aa4d0 msgr2=0x7efee015d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.217+0000 7efee594d700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efee00aa4d0 0x7efee015d380 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efed000c500 tx=0x7efed000c8c0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.217+0000 7efee594d700 1 -- 172.21.15.146:0/657385626 shutdown_connections 2024-09-13T18:27:05.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.217+0000 7efee594d700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efecc066730 0x7efecc068bf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.217+0000 7efee594d700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efee00ad510 0x7efee00aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.217+0000 7efee594d700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00caed0 0x7efee00ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.217+0000 7efee594d700 1 --2- 172.21.15.146:0/657385626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efee00aa4d0 0x7efee015d380 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.217+0000 7efee594d700 1 -- 172.21.15.146:0/657385626 >> 172.21.15.146:0/657385626 conn(0x7efee0066090 msgr2=0x7efee00adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.221+0000 7efee594d700 1 -- 172.21.15.146:0/657385626 shutdown_connections 2024-09-13T18:27:05.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.221+0000 7efee594d700 1 -- 172.21.15.146:0/657385626 wait complete. 2024-09-13T18:27:05.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.225+0000 7f9d2939f700 1 -- 172.21.15.146:0/1552193070 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f9d240c1f40 con 0x7f9d240c9350 2024-09-13T18:27:05.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.225+0000 7f9d20ff9700 1 -- 172.21.15.146:0/1552193070 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9d240c1f40 con 0x7f9d240c9350 2024-09-13T18:27:05.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2024-09-13T18:27:05.232 INFO:teuthology.orchestra.run.smithi146.stdout:55834574857 2024-09-13T18:27:05.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f25d619a700 1 -- 172.21.15.146:0/2288986217 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f25b0005890 con 0x7f25d00c8f70 2024-09-13T18:27:05.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f25c5ffb700 1 -- 172.21.15.146:0/2288986217 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f25cc015020 con 0x7f25d00c8f70 2024-09-13T18:27:05.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f9d2939f700 1 -- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d10066730 msgr2=0x7f9d10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f9d2939f700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d10066730 0x7f9d10068bf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d14000f60 tx=0x7f9d14007f30 comp rx=0 tx=0).stop 2024-09-13T18:27:05.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f9d2939f700 1 -- 172.21.15.146:0/1552193070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d240c9350 msgr2=0x7f9d24157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f9d2939f700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d240c9350 0x7f9d24157200 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9d240cac30 tx=0x7f9d1c004af0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.235 INFO:teuthology.orchestra.run.smithi146.stdout:64424509449 2024-09-13T18:27:05.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f9d2939f700 1 -- 172.21.15.146:0/1552193070 shutdown_connections 2024-09-13T18:27:05.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f9d2939f700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d10066730 0x7f9d10068bf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f9d2939f700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d240c9350 0x7f9d24157200 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f9d2939f700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d240c8f70 0x7f9d240c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f9d2939f700 1 --2- 172.21.15.146:0/1552193070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d240a9490 0x7f9d240c2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f9d2939f700 1 -- 172.21.15.146:0/1552193070 >> 172.21.15.146:0/1552193070 conn(0x7f9d24066090 msgr2=0x7f9d2406ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f9d2939f700 1 -- 172.21.15.146:0/1552193070 shutdown_connections 2024-09-13T18:27:05.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.229+0000 7f9d2939f700 1 -- 172.21.15.146:0/1552193070 wait complete. 2024-09-13T18:27:05.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f563b105700 1 -- 172.21.15.146:0/2295135153 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f561c0055c0 con 0x7f56340caed0 2024-09-13T18:27:05.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f25d619a700 1 -- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f25b4066730 msgr2=0x7f25b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f25d619a700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f25b4066730 0x7f25b4068bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f25c0000f60 tx=0x7f25c0007f30 comp rx=0 tx=0).stop 2024-09-13T18:27:05.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f25d619a700 1 -- 172.21.15.146:0/2288986217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25d00c8f70 msgr2=0x7f25d01579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f25d619a700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25d00c8f70 0x7f25d01579e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009b70 tx=0x7f25cc004770 comp rx=0 tx=0).stop 2024-09-13T18:27:05.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f25d619a700 1 -- 172.21.15.146:0/2288986217 shutdown_connections 2024-09-13T18:27:05.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f25d619a700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f25b4066730 0x7f25b4068bf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f25d619a700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25d00c9350 0x7f25d00aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f25d619a700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25d00c8f70 0x7f25d01579e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f25d619a700 1 --2- 172.21.15.146:0/2288986217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d00a9490 0x7f25d006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f25d619a700 1 -- 172.21.15.146:0/2288986217 >> 172.21.15.146:0/2288986217 conn(0x7f25d0066090 msgr2=0x7f25d00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f25d619a700 1 -- 172.21.15.146:0/2288986217 shutdown_connections 2024-09-13T18:27:05.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f25d619a700 1 -- 172.21.15.146:0/2288986217 wait complete. 2024-09-13T18:27:05.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.233+0000 7f562affd700 1 -- 172.21.15.146:0/2295135153 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f562c054680 con 0x7f56340caed0 2024-09-13T18:27:05.239 INFO:teuthology.orchestra.run.smithi146.stdout:64424509448 2024-09-13T18:27:05.242 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.2 2024-09-13T18:27:05.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.237+0000 7f563b105700 1 -- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5618066830 msgr2=0x7f5618068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.237+0000 7f563b105700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5618066830 0x7f5618068cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56300017c0 tx=0x7f5630009040 comp rx=0 tx=0).stop 2024-09-13T18:27:05.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.237+0000 7f563b105700 1 -- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56340caed0 msgr2=0x7f56341f2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.237+0000 7f563b105700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56340caed0 0x7f56341f2a50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f562c001f60 tx=0x7f562c00a340 comp rx=0 tx=0).stop 2024-09-13T18:27:05.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.237+0000 7f563b105700 1 -- 172.21.15.146:0/2295135153 shutdown_connections 2024-09-13T18:27:05.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.237+0000 7f563b105700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5618066830 0x7f5618068cf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.237+0000 7f563b105700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56340cb2b0 0x7f563413d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.237+0000 7f563b105700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56340caed0 0x7f56341f2a50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.237+0000 7f563b105700 1 --2- 172.21.15.146:0/2295135153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563406eec0 0x7f5634150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.237+0000 7f563b105700 1 -- 172.21.15.146:0/2295135153 >> 172.21.15.146:0/2295135153 conn(0x7f5634066090 msgr2=0x7f563414e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.237+0000 7f563b105700 1 -- 172.21.15.146:0/2295135153 shutdown_connections 2024-09-13T18:27:05.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.237+0000 7f563b105700 1 -- 172.21.15.146:0/2295135153 wait complete. 2024-09-13T18:27:05.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.241+0000 7f77f5215700 1 -- 172.21.15.146:0/819587651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f77d00055c0 con 0x7f77f00caed0 2024-09-13T18:27:05.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-09-13T18:27:05.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.241+0000 7f77ecff9700 1 -- 172.21.15.146:0/819587651 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f77e4017070 con 0x7f77f00caed0 2024-09-13T18:27:05.248 INFO:teuthology.orchestra.run.smithi146.stdout:55834574856 2024-09-13T18:27:05.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.245+0000 7f77f5215700 1 -- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f77dc066730 msgr2=0x7f77dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.245+0000 7f77f5215700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f77dc066730 0x7f77dc068bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f77e8002230 tx=0x7f77e8001dd0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.245+0000 7f77f5215700 1 -- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77f00caed0 msgr2=0x7f77f00c0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.245+0000 7f77f5215700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77f00caed0 0x7f77f00c0690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f77e400c500 tx=0x7f77e400c810 comp rx=0 tx=0).stop 2024-09-13T18:27:05.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.6 2024-09-13T18:27:05.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.245+0000 7f77f5215700 1 -- 172.21.15.146:0/819587651 shutdown_connections 2024-09-13T18:27:05.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.245+0000 7f77f5215700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f77dc066730 0x7f77dc068bf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.245+0000 7f77f5215700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f00cb2b0 0x7f77f01f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.245+0000 7f77f5215700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77f00caed0 0x7f77f00c0690 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.245+0000 7f77f5215700 1 --2- 172.21.15.146:0/819587651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77f00aa4d0 0x7f77f01251f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.245+0000 7f77f5215700 1 -- 172.21.15.146:0/819587651 >> 172.21.15.146:0/819587651 conn(0x7f77f0066090 msgr2=0x7f77f006c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.245+0000 7f77f5215700 1 -- 172.21.15.146:0/819587651 shutdown_connections 2024-09-13T18:27:05.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.245+0000 7f77f5215700 1 -- 172.21.15.146:0/819587651 wait complete. 2024-09-13T18:27:05.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.7 2024-09-13T18:27:05.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2024-09-13T18:27:05.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.261+0000 7f0903fff700 1 -- 172.21.15.146:0/447674611 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f08e8005890 con 0x7f09040c8f70 2024-09-13T18:27:05.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.261+0000 7f08f3fff700 1 -- 172.21.15.146:0/447674611 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f08fc057660 con 0x7f09040c8f70 2024-09-13T18:27:05.269 INFO:teuthology.orchestra.run.smithi146.stdout:60129542153 2024-09-13T18:27:05.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.265+0000 7f0903fff700 1 -- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f08ec066d30 msgr2=0x7f08ec0691f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.265+0000 7f0903fff700 1 --2- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f08ec066d30 0x7f08ec0691f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f08f4000f60 tx=0x7f08f4009040 comp rx=0 tx=0).stop 2024-09-13T18:27:05.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.265+0000 7f0903fff700 1 -- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040c8f70 msgr2=0x7f09041f1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.265+0000 7f0903fff700 1 --2- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040c8f70 0x7f09041f1bd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009470 tx=0x7f08fc004af0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.265+0000 7f0903fff700 1 -- 172.21.15.146:0/447674611 shutdown_connections 2024-09-13T18:27:05.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.265+0000 7f0903fff700 1 --2- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f08ec066d30 0x7f08ec0691f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.265+0000 7f0903fff700 1 --2- 172.21.15.146:0/447674611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09040c9350 0x7f0904156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.265+0000 7f0903fff700 1 --2- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040c8f70 0x7f09041f1bd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.265+0000 7f0903fff700 1 --2- 172.21.15.146:0/447674611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09040a9490 0x7f09040c0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.265+0000 7f0903fff700 1 -- 172.21.15.146:0/447674611 >> 172.21.15.146:0/447674611 conn(0x7f0904066090 msgr2=0x7f09041576e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.265+0000 7f0903fff700 1 -- 172.21.15.146:0/447674611 shutdown_connections 2024-09-13T18:27:05.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:05.265+0000 7f0903fff700 1 -- 172.21.15.146:0/447674611 wait complete. 2024-09-13T18:27:05.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.3 2024-09-13T18:27:06.202 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T18:27:06.233 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:27:06.243 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:27:06.249 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T18:27:06.252 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T18:27:06.258 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T18:27:06.260 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:27:06.288 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:27:06.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.341+0000 7f5d1ffff700 1 Processor -- start 2024-09-13T18:27:06.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1ffff700 1 -- start start 2024-09-13T18:27:06.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5d2006fcd0 0x7f5d2006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1ffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2014fb30 con 0x7f5d2014f750 2024-09-13T18:27:06.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1ffff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d20155580 con 0x7f5d200a9490 2024-09-13T18:27:06.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1ffff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d20155860 con 0x7f5d2006fcd0 2024-09-13T18:27:06.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1effd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5d2014f750 0x7f5d200cb120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35554/0 (socket says 172.21.15.146:35554) 2024-09-13T18:27:06.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1effd700 1 -- 172.21.15.146:0/2031760591 learned_addr learned my addr 172.21.15.146:0/2031760591 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:06.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1f7fe700 1 --2- 172.21.15.146:0/2031760591 >> v2:172.21.15.154:3300/0 conn(0x7f5d2006fcd0 0x7f5d2006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1dffb700 1 -- 172.21.15.146:0/2031760591 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180640509 0 0) 0x7f5d2014fb30 con 0x7f5d2014f750 2024-09-13T18:27:06.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1dffb700 1 -- 172.21.15.146:0/2031760591 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d080035d0 con 0x7f5d2014f750 2024-09-13T18:27:06.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1f7fe700 1 -- 172.21.15.146:0/2031760591 >> v1:172.21.15.154:6789/0 conn(0x7f5d200a9490 legacy=0x7f5d2006f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1f7fe700 1 -- 172.21.15.146:0/2031760591 >> v1:172.21.15.146:6789/0 conn(0x7f5d2014f750 legacy=0x7f5d200cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1f7fe700 1 -- 172.21.15.146:0/2031760591 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d20155b40 con 0x7f5d2006fcd0 2024-09-13T18:27:06.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1f7fe700 1 --2- 172.21.15.146:0/2031760591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2006fcd0 0x7f5d2006eea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800aea0 tx=0x7f5d1800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16b9473961015c15 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1dffb700 1 -- 172.21.15.146:0/2031760591 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1800f070 con 0x7f5d2006fcd0 2024-09-13T18:27:06.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.345+0000 7f5d1dffb700 1 -- 172.21.15.146:0/2031760591 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1800c910 con 0x7f5d2006fcd0 2024-09-13T18:27:06.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.349+0000 7f5d1ffff700 1 -- 172.21.15.146:0/2031760591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2006fcd0 msgr2=0x7f5d2006eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.349+0000 7f5d1ffff700 1 --2- 172.21.15.146:0/2031760591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2006fcd0 0x7f5d2006eea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800aea0 tx=0x7f5d1800c330 comp rx=0 tx=0).stop 2024-09-13T18:27:06.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.353+0000 7f5d1ffff700 1 -- 172.21.15.146:0/2031760591 shutdown_connections 2024-09-13T18:27:06.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.353+0000 7f5d1ffff700 1 --2- 172.21.15.146:0/2031760591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2006fcd0 0x7f5d2006eea0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.353+0000 7f5d1ffff700 1 -- 172.21.15.146:0/2031760591 >> 172.21.15.146:0/2031760591 conn(0x7f5d20066090 msgr2=0x7f5d20066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.353+0000 7f5d1ffff700 1 -- 172.21.15.146:0/2031760591 shutdown_connections 2024-09-13T18:27:06.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.353+0000 7f5d1ffff700 1 -- 172.21.15.146:0/2031760591 wait complete. 2024-09-13T18:27:06.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.353+0000 7f5d1ffff700 1 Processor -- start 2024-09-13T18:27:06.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.353+0000 7f5d1ffff700 1 -- start start 2024-09-13T18:27:06.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.353+0000 7f5d1ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d2006fcd0 0x7f5d201505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.353+0000 7f5d1ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d200a9490 0x7f5d20151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.353+0000 7f5d1ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2014f750 0x7f5d201523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.353+0000 7f5d1ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d20151450 con 0x7f5d200a9490 2024-09-13T18:27:06.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.353+0000 7f5d1ffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d20151a10 con 0x7f5d2014f750 2024-09-13T18:27:06.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.353+0000 7f5d1ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d20151070 con 0x7f5d2006fcd0 2024-09-13T18:27:06.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1effd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d2006fcd0 0x7f5d201505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1effd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d2006fcd0 0x7f5d201505d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41450/0 (socket says 172.21.15.146:41450) 2024-09-13T18:27:06.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1effd700 1 -- 172.21.15.146:0/970864903 learned_addr learned my addr 172.21.15.146:0/970864903 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1e7fc700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d200a9490 0x7f5d20151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1f7fe700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2014f750 0x7f5d201523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1e7fc700 1 -- 172.21.15.146:0/970864903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d2006fcd0 msgr2=0x7f5d201505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1e7fc700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d2006fcd0 0x7f5d201505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1e7fc700 1 -- 172.21.15.146:0/970864903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2014f750 msgr2=0x7f5d201523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1e7fc700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2014f750 0x7f5d201523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1e7fc700 1 -- 172.21.15.146:0/970864903 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1800ab80 con 0x7f5d200a9490 2024-09-13T18:27:06.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1e7fc700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d200a9490 0x7f5d20151ea0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400c440 tx=0x7f5d14001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d0f7fe700 1 -- 172.21.15.146:0/970864903 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d140052d0 con 0x7f5d200a9490 2024-09-13T18:27:06.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d0f7fe700 1 -- 172.21.15.146:0/970864903 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d14005430 con 0x7f5d200a9490 2024-09-13T18:27:06.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d0f7fe700 1 -- 172.21.15.146:0/970864903 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d140055a0 con 0x7f5d200a9490 2024-09-13T18:27:06.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1ffff700 1 -- 172.21.15.146:0/970864903 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d201f3c70 con 0x7f5d200a9490 2024-09-13T18:27:06.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1ffff700 1 -- 172.21.15.146:0/970864903 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d201516a0 con 0x7f5d200a9490 2024-09-13T18:27:06.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.357+0000 7f5d1ffff700 1 -- 172.21.15.146:0/970864903 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cfc005310 con 0x7f5d200a9490 2024-09-13T18:27:06.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.361+0000 7f5d0f7fe700 1 -- 172.21.15.146:0/970864903 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5d140038e0 con 0x7f5d200a9490 2024-09-13T18:27:06.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.361+0000 7f5d0f7fe700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d04066780 0x7f5d04068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.361+0000 7f5d0f7fe700 1 -- 172.21.15.146:0/970864903 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5d14083890 con 0x7f5d200a9490 2024-09-13T18:27:06.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.361+0000 7f5d1effd700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d04066780 0x7f5d04068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.361+0000 7f5d1effd700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d04066780 0x7f5d04068c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d10004c30 tx=0x7f5d1000a330 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.369+0000 7f5d0f7fe700 1 -- 172.21.15.146:0/970864903 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5d14050970 con 0x7f5d200a9490 2024-09-13T18:27:06.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd9477fe700 1 Processor -- start 2024-09-13T18:27:06.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd9477fe700 1 -- start start 2024-09-13T18:27:06.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd9477fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd94006fcd0 0x7fd94006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd9477fe700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94014fb30 con 0x7fd9400a9490 2024-09-13T18:27:06.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd9477fe700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd940155580 con 0x7fd94014f750 2024-09-13T18:27:06.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd9477fe700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd940155860 con 0x7fd94006fcd0 2024-09-13T18:27:06.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd94559a700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd94014f750 0x7fd9400cb120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:55010/0 (socket says 172.21.15.146:55010) 2024-09-13T18:27:06.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd94559a700 1 -- 172.21.15.146:0/3320078112 learned_addr learned my addr 172.21.15.146:0/3320078112 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:06.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd945d9b700 1 --2- 172.21.15.146:0/3320078112 >> v2:172.21.15.154:3300/0 conn(0x7fd94006fcd0 0x7fd94006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd937fff700 1 -- 172.21.15.146:0/3320078112 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491382183 0 0) 0x7fd940155580 con 0x7fd94014f750 2024-09-13T18:27:06.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd937fff700 1 -- 172.21.15.146:0/3320078112 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd92c0035d0 con 0x7fd94014f750 2024-09-13T18:27:06.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd945d9b700 1 -- 172.21.15.146:0/3320078112 >> v1:172.21.15.154:6789/0 conn(0x7fd94014f750 legacy=0x7fd9400cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd945d9b700 1 -- 172.21.15.146:0/3320078112 >> v1:172.21.15.146:6789/0 conn(0x7fd9400a9490 legacy=0x7fd94006f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.381+0000 7fd945d9b700 1 -- 172.21.15.146:0/3320078112 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd940155b40 con 0x7fd94006fcd0 2024-09-13T18:27:06.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.385+0000 7fd945d9b700 1 --2- 172.21.15.146:0/3320078112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd94006fcd0 0x7fd94006eea0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd938009c80 tx=0x7fd938009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdcdc678aa0e19dc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.385+0000 7fd937fff700 1 -- 172.21.15.146:0/3320078112 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93800baf0 con 0x7fd94006fcd0 2024-09-13T18:27:06.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.385+0000 7fd937fff700 1 -- 172.21.15.146:0/3320078112 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93800bc50 con 0x7fd94006fcd0 2024-09-13T18:27:06.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.385+0000 7fd937fff700 1 -- 172.21.15.146:0/3320078112 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd938010480 con 0x7fd94006fcd0 2024-09-13T18:27:06.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.385+0000 7fd9477fe700 1 -- 172.21.15.146:0/3320078112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd94006fcd0 msgr2=0x7fd94006eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.385+0000 7fd9477fe700 1 --2- 172.21.15.146:0/3320078112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd94006fcd0 0x7fd94006eea0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd938009c80 tx=0x7fd938009f90 comp rx=0 tx=0).stop 2024-09-13T18:27:06.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.385+0000 7fd9477fe700 1 -- 172.21.15.146:0/3320078112 shutdown_connections 2024-09-13T18:27:06.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.385+0000 7fd9477fe700 1 --2- 172.21.15.146:0/3320078112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd94006fcd0 0x7fd94006eea0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.385+0000 7fd9477fe700 1 -- 172.21.15.146:0/3320078112 >> 172.21.15.146:0/3320078112 conn(0x7fd940066090 msgr2=0x7fd940066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.385+0000 7fd9477fe700 1 -- 172.21.15.146:0/3320078112 shutdown_connections 2024-09-13T18:27:06.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.385+0000 7fd9477fe700 1 -- 172.21.15.146:0/3320078112 wait complete. 2024-09-13T18:27:06.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.385+0000 7fd9477fe700 1 Processor -- start 2024-09-13T18:27:06.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.385+0000 7fd9477fe700 1 -- start start 2024-09-13T18:27:06.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd9477fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd94006fcd0 0x7fd9401567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd9477fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9400a9490 0x7fd9400c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd9477fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd94014f750 0x7fd9400abd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd9477fe700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400c52d0 con 0x7fd94006fcd0 2024-09-13T18:27:06.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd9477fe700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd940151190 con 0x7fd9400a9490 2024-09-13T18:27:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd9477fe700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400c5030 con 0x7fd94014f750 2024-09-13T18:27:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd94559a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd94006fcd0 0x7fd9401567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd94559a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd94006fcd0 0x7fd9401567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54804/0 (socket says 172.21.15.146:54804) 2024-09-13T18:27:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd94559a700 1 -- 172.21.15.146:0/1570544972 learned_addr learned my addr 172.21.15.146:0/1570544972 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd945d9b700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd94014f750 0x7fd9400abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd944d99700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9400a9490 0x7fd9400c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd94559a700 1 -- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd94014f750 msgr2=0x7fd9400abd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd94559a700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd94014f750 0x7fd9400abd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd94559a700 1 -- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9400a9490 msgr2=0x7fd9400c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd94559a700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9400a9490 0x7fd9400c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd94559a700 1 -- 172.21.15.146:0/1570544972 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93800d040 con 0x7fd94006fcd0 2024-09-13T18:27:06.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.389+0000 7fd94559a700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd94006fcd0 0x7fd9401567a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd93000d960 tx=0x7fd93000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.393+0000 7fd935ffb700 1 -- 172.21.15.146:0/1570544972 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930008bb0 con 0x7fd94006fcd0 2024-09-13T18:27:06.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.393+0000 7fd9477fe700 1 -- 172.21.15.146:0/1570544972 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd940152a20 con 0x7fd94006fcd0 2024-09-13T18:27:06.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.393+0000 7fd9477fe700 1 -- 172.21.15.146:0/1570544972 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd940151f40 con 0x7fd94006fcd0 2024-09-13T18:27:06.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.393+0000 7fd935ffb700 1 -- 172.21.15.146:0/1570544972 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd930008d10 con 0x7fd94006fcd0 2024-09-13T18:27:06.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.393+0000 7fd935ffb700 1 -- 172.21.15.146:0/1570544972 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93000f660 con 0x7fd94006fcd0 2024-09-13T18:27:06.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.393+0000 7fd9477fe700 1 -- 172.21.15.146:0/1570544972 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd920005310 con 0x7fd94006fcd0 2024-09-13T18:27:06.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.393+0000 7fd935ffb700 1 -- 172.21.15.146:0/1570544972 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd93000f7c0 con 0x7fd94006fcd0 2024-09-13T18:27:06.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.393+0000 7fd935ffb700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd924066780 0x7fd924068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.393+0000 7fd935ffb700 1 -- 172.21.15.146:0/1570544972 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd930084a00 con 0x7fd94006fcd0 2024-09-13T18:27:06.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.393+0000 7fd944d99700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd924066780 0x7fd924068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.397+0000 7fd944d99700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd924066780 0x7fd924068c40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd940152c70 tx=0x7fd93c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.397+0000 7f69be516700 1 Processor -- start 2024-09-13T18:27:06.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.397+0000 7f69be516700 1 -- start start 2024-09-13T18:27:06.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.397+0000 7f53cbc1c700 1 Processor -- start 2024-09-13T18:27:06.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.397+0000 7f53cbc1c700 1 -- start start 2024-09-13T18:27:06.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.397+0000 7fd935ffb700 1 -- 172.21.15.146:0/1570544972 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd930051bf0 con 0x7fd94006fcd0 2024-09-13T18:27:06.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5fffff700 1 Processor -- start 2024-09-13T18:27:06.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5fffff700 1 -- start start 2024-09-13T18:27:06.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69be516700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69b00355d0 0x7f69b00dc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69be516700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f69b0044be0 0x7f69b00d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53cbc1c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f53c40cab20 0x7f53c415b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53cbc1c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c41544a0 0x7f53c41548d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53cbc1c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c40a97e0 con 0x7f53c40caf00 2024-09-13T18:27:06.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53cbc1c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c40c99a0 con 0x7f53c40cab20 2024-09-13T18:27:06.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53cbc1c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40723d0 con 0x7f53c41544a0 2024-09-13T18:27:06.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69be516700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b004b800 con 0x7f69b00351f0 2024-09-13T18:27:06.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69be516700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b004b2b0 con 0x7f69b0044be0 2024-09-13T18:27:06.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69be516700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b004a780 con 0x7f69b00355d0 2024-09-13T18:27:06.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69bdd15700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f69b0044be0 0x7f69b00d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53ca1b9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c41544a0 0x7f53c41548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53ca1b9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c41544a0 0x7f53c41548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41462/0 (socket says 172.21.15.146:41462) 2024-09-13T18:27:06.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53ca1b9700 1 -- 172.21.15.146:0/904497058 learned_addr learned my addr 172.21.15.146:0/904497058 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69bdd15700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f69b0044be0 0x7f69b00d2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:46810/0 (socket says 172.21.15.146:46810) 2024-09-13T18:27:06.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69bdd15700 1 -- 172.21.15.146:0/3223431484 learned_addr learned my addr 172.21.15.146:0/3223431484 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5fffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6000c9350 0x7fb60015a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5fffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb6000a9490 0x7fb6000c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5fffff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6000c8650 con 0x7fb6000c8f70 2024-09-13T18:27:06.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5fffff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6000aa1a0 con 0x7fb6000a9490 2024-09-13T18:27:06.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5fffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000aa480 con 0x7fb6000c9350 2024-09-13T18:27:06.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53c99b8700 1 --2- 172.21.15.146:0/904497058 >> v2:172.21.15.154:3300/0 conn(0x7f53c40cab20 0x7f53c415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53ca1b9700 1 -- 172.21.15.146:0/904497058 >> v2:172.21.15.154:3300/0 conn(0x7f53c40cab20 msgr2=0x7f53c415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:06.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53ca1b9700 1 --2- 172.21.15.146:0/904497058 >> v2:172.21.15.154:3300/0 conn(0x7f53c40cab20 0x7f53c415b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53ca1b9700 1 -- 172.21.15.146:0/904497058 >> v1:172.21.15.146:6789/0 conn(0x7f53c40caf00 legacy=0x7f53c4157af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:06.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53ca1b9700 1 -- 172.21.15.146:0/904497058 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c415b8c0 con 0x7f53c41544a0 2024-09-13T18:27:06.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5ff7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb6000a9490 0x7fb6000c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69bdd15700 1 -- 172.21.15.146:0/3223431484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69b00355d0 msgr2=0x7f69b00dc4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:06.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69bdd15700 1 --2- 172.21.15.146:0/3223431484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69b00355d0 0x7f69b00dc4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69bdd15700 1 -- 172.21.15.146:0/3223431484 >> v1:172.21.15.146:6789/0 conn(0x7f69b00351f0 legacy=0x7f69b004bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:06.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5ff7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb6000a9490 0x7fb6000c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:46832/0 (socket says 172.21.15.146:46832) 2024-09-13T18:27:06.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5ff7fe700 1 -- 172.21.15.146:0/5488872 learned_addr learned my addr 172.21.15.146:0/5488872 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5fe7fc700 1 --2- 172.21.15.146:0/5488872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6000c9350 0x7fb60015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53ca1b9700 1 --2- 172.21.15.146:0/904497058 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c41544a0 0x7f53c41548d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009e50 tx=0x7f53bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebc4620124ee9b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69bdd15700 1 -- 172.21.15.146:0/3223431484 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b00dcc00 con 0x7f69b0044be0 2024-09-13T18:27:06.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5ff7fe700 1 -- 172.21.15.146:0/5488872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6000c9350 msgr2=0x7fb60015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:06.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5ff7fe700 1 --2- 172.21.15.146:0/5488872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6000c9350 0x7fb60015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5ff7fe700 1 -- 172.21.15.146:0/5488872 >> v1:172.21.15.154:6789/0 conn(0x7fb6000c8f70 legacy=0x7fb6000cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5ff7fe700 1 -- 172.21.15.146:0/5488872 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb600159890 con 0x7fb6000a9490 2024-09-13T18:27:06.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69bdd15700 1 --2- 172.21.15.146:0/3223431484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69b0044be0 0x7f69b00d2a80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f69b80731e0 tx=0x7f69b8061ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9f390d543889eae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53c89b6700 1 -- 172.21.15.146:0/904497058 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc00e070 con 0x7f53c41544a0 2024-09-13T18:27:06.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53c89b6700 1 -- 172.21.15.146:0/904497058 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc003a40 con 0x7f53c41544a0 2024-09-13T18:27:06.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53c89b6700 1 -- 172.21.15.146:0/904497058 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc003df0 con 0x7f53c41544a0 2024-09-13T18:27:06.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69affff700 1 -- 172.21.15.146:0/3223431484 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b80739c0 con 0x7f69b0044be0 2024-09-13T18:27:06.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69affff700 1 -- 172.21.15.146:0/3223431484 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b8075a30 con 0x7f69b0044be0 2024-09-13T18:27:06.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69affff700 1 -- 172.21.15.146:0/3223431484 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8075dc0 con 0x7f69b0044be0 2024-09-13T18:27:06.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53cbc1c700 1 -- 172.21.15.146:0/904497058 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c41544a0 msgr2=0x7f53c41548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53cbc1c700 1 --2- 172.21.15.146:0/904497058 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c41544a0 0x7f53c41548d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009e50 tx=0x7f53bc00b330 comp rx=0 tx=0).stop 2024-09-13T18:27:06.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5ff7fe700 1 --2- 172.21.15.146:0/5488872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6000a9490 0x7fb6000c76c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009e50 tx=0x7fb5f800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fa265bb7f011f45 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5fdffb700 1 -- 172.21.15.146:0/5488872 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f800e070 con 0x7fb6000a9490 2024-09-13T18:27:06.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7fb5fdffb700 1 -- 172.21.15.146:0/5488872 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f800b910 con 0x7fb6000a9490 2024-09-13T18:27:06.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53cbc1c700 1 -- 172.21.15.146:0/904497058 shutdown_connections 2024-09-13T18:27:06.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53cbc1c700 1 --2- 172.21.15.146:0/904497058 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c41544a0 0x7f53c41548d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f53cbc1c700 1 --2- 172.21.15.146:0/904497058 >> v2:172.21.15.154:3300/0 conn(0x7f53c40cab20 0x7f53c415b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f53cbc1c700 1 -- 172.21.15.146:0/904497058 >> 172.21.15.146:0/904497058 conn(0x7f53c4066090 msgr2=0x7f53c4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69be516700 1 -- 172.21.15.146:0/3223431484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69b0044be0 msgr2=0x7f69b00d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.401+0000 7f69be516700 1 --2- 172.21.15.146:0/3223431484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69b0044be0 0x7f69b00d2a80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f69b80731e0 tx=0x7f69b8061ef0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fdffb700 1 -- 172.21.15.146:0/5488872 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f800bcc0 con 0x7fb6000a9490 2024-09-13T18:27:06.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 -- 172.21.15.146:0/5488872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6000a9490 msgr2=0x7fb6000c76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 --2- 172.21.15.146:0/5488872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6000a9490 0x7fb6000c76c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009e50 tx=0x7fb5f800b330 comp rx=0 tx=0).stop 2024-09-13T18:27:06.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f53cbc1c700 1 -- 172.21.15.146:0/904497058 shutdown_connections 2024-09-13T18:27:06.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 -- 172.21.15.146:0/5488872 shutdown_connections 2024-09-13T18:27:06.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 --2- 172.21.15.146:0/5488872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6000a9490 0x7fb6000c76c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 --2- 172.21.15.146:0/5488872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6000c9350 0x7fb60015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 -- 172.21.15.146:0/5488872 >> 172.21.15.146:0/5488872 conn(0x7fb600066090 msgr2=0x7fb600066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f53cbc1c700 1 -- 172.21.15.146:0/904497058 wait complete. 2024-09-13T18:27:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f69be516700 1 -- 172.21.15.146:0/3223431484 shutdown_connections 2024-09-13T18:27:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f69be516700 1 --2- 172.21.15.146:0/3223431484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69b0044be0 0x7f69b00d2a80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f69be516700 1 --2- 172.21.15.146:0/3223431484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69b00355d0 0x7f69b00dc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f69be516700 1 -- 172.21.15.146:0/3223431484 >> 172.21.15.146:0/3223431484 conn(0x7f69b004fd00 msgr2=0x7f69b0050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f69be516700 1 -- 172.21.15.146:0/3223431484 shutdown_connections 2024-09-13T18:27:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 -- 172.21.15.146:0/5488872 shutdown_connections 2024-09-13T18:27:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 -- 172.21.15.146:0/5488872 wait complete. 2024-09-13T18:27:06.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6969782700 1 Processor -- start 2024-09-13T18:27:06.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f53cbc1c700 1 Processor -- start 2024-09-13T18:27:06.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f53cbc1c700 1 -- start start 2024-09-13T18:27:06.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6969782700 1 -- start start 2024-09-13T18:27:06.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6969782700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f696414f750 0x7f696406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6969782700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f696406f5c0 0x7f696406fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6969782700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f696406ff80 con 0x7f69640a9490 2024-09-13T18:27:06.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6969782700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696414fb30 con 0x7f696406f5c0 2024-09-13T18:27:06.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6969782700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6964155580 con 0x7f696414f750 2024-09-13T18:27:06.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f69637fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f69640a9490 0x7f69640cb120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35582/0 (socket says 172.21.15.146:35582) 2024-09-13T18:27:06.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f69637fe700 1 -- 172.21.15.146:0/755176423 learned_addr learned my addr 172.21.15.146:0/755176423 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:06.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6963fff700 1 --2- 172.21.15.146:0/755176423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f696414f750 0x7f696406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6968f81700 1 --2- 172.21.15.146:0/755176423 >> v2:172.21.15.154:3300/0 conn(0x7f696406f5c0 0x7f696406fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 Processor -- start 2024-09-13T18:27:06.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 -- start start 2024-09-13T18:27:06.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f69be516700 1 -- 172.21.15.146:0/3223431484 wait complete. 2024-09-13T18:27:06.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f69be516700 1 Processor -- start 2024-09-13T18:27:06.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6962ffd700 1 -- 172.21.15.146:0/755176423 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 618462832 0 0) 0x7f696406ff80 con 0x7f69640a9490 2024-09-13T18:27:06.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6962ffd700 1 -- 172.21.15.146:0/755176423 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f694c0035d0 con 0x7f69640a9490 2024-09-13T18:27:06.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f69be516700 1 -- start start 2024-09-13T18:27:06.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6963fff700 1 -- 172.21.15.146:0/755176423 >> v2:172.21.15.154:3300/0 conn(0x7f696406f5c0 msgr2=0x7f696406fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:06.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6963fff700 1 --2- 172.21.15.146:0/755176423 >> v2:172.21.15.154:3300/0 conn(0x7f696406f5c0 0x7f696406fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6963fff700 1 -- 172.21.15.146:0/755176423 >> v1:172.21.15.146:6789/0 conn(0x7f69640a9490 legacy=0x7f69640cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7f6963fff700 1 -- 172.21.15.146:0/755176423 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6964155860 con 0x7f696414f750 2024-09-13T18:27:06.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6000a9490 0x7fb60015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6000c8f70 0x7fb6000af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6000c9350 0x7fb6001fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60015d0a0 con 0x7fb6000a9490 2024-09-13T18:27:06.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb600159620 con 0x7fb6000c9350 2024-09-13T18:27:06.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.405+0000 7fb5fffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000c3d20 con 0x7fb6000c8f70 2024-09-13T18:27:06.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53cbc1c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53c40cab20 0x7f53c40c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53cbc1c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c40caf00 0x7f53c4201690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5feffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6000a9490 0x7fb60015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53cbc1c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c41544a0 0x7f53c41fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53cbc1c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c41ff320 con 0x7f53c40cab20 2024-09-13T18:27:06.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53cbc1c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c41e2ff0 con 0x7f53c41544a0 2024-09-13T18:27:06.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53cbc1c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c41e32d0 con 0x7f53c40caf00 2024-09-13T18:27:06.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53c99b8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53c40cab20 0x7f53c40c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f6963fff700 1 --2- 172.21.15.146:0/755176423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f696414f750 0x7f696406eea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6954009260 tx=0x7f6954009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89bf538e391229e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5feffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6000a9490 0x7fb60015cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54818/0 (socket says 172.21.15.146:54818) 2024-09-13T18:27:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5feffd700 1 -- 172.21.15.146:0/301596844 learned_addr learned my addr 172.21.15.146:0/301596844 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5ff7fe700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6000c9350 0x7fb6001fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5fe7fc700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6000c8f70 0x7fb6000af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69be516700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69b00351f0 0x7f69b004e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69be516700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69b00355d0 0x7f69b00477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69be516700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69b0044be0 0x7f69b00cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69be516700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b00da650 con 0x7f69b0044be0 2024-09-13T18:27:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69be516700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b00da3b0 con 0x7f69b00351f0 2024-09-13T18:27:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69be516700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b00daa70 con 0x7f69b00355d0 2024-09-13T18:27:06.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53c99b8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53c40cab20 0x7f53c40c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54832/0 (socket says 172.21.15.146:54832) 2024-09-13T18:27:06.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53c99b8700 1 -- 172.21.15.146:0/2912149214 learned_addr learned my addr 172.21.15.146:0/2912149214 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53ca1b9700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c41544a0 0x7f53c41fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53c91b7700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c40caf00 0x7f53c4201690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5feffd700 1 -- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6000c8f70 msgr2=0x7fb6000af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5feffd700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6000c8f70 0x7fb6000af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5feffd700 1 -- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6000c9350 msgr2=0x7fb6001fbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5feffd700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6000c9350 0x7fb6001fbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5feffd700 1 -- 172.21.15.146:0/301596844 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f8009b30 con 0x7fb6000a9490 2024-09-13T18:27:06.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69bdd15700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69b0044be0 0x7f69b00cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69bdd15700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69b0044be0 0x7f69b00cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54846/0 (socket says 172.21.15.146:54846) 2024-09-13T18:27:06.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69bdd15700 1 -- 172.21.15.146:0/2188313344 learned_addr learned my addr 172.21.15.146:0/2188313344 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69bcd13700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69b00355d0 0x7f69b00477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69bd514700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69b00351f0 0x7f69b004e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdff516700 1 Processor -- start 2024-09-13T18:27:06.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5feffd700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6000a9490 0x7fb60015cb60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000c530 tx=0x7fb5f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdff516700 1 -- start start 2024-09-13T18:27:06.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53c99b8700 1 -- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c40caf00 msgr2=0x7f53c4201690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53c99b8700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c40caf00 0x7f53c4201690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53c99b8700 1 -- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c41544a0 msgr2=0x7f53c41fd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53c99b8700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c41544a0 0x7f53c41fd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53c99b8700 1 -- 172.21.15.146:0/2912149214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc009b30 con 0x7f53c40cab20 2024-09-13T18:27:06.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53c99b8700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53c40cab20 0x7f53c40c41b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f53c000d800 tx=0x7f53c000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53ba7fc700 1 -- 172.21.15.146:0/2912149214 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00125f0 con 0x7f53c40cab20 2024-09-13T18:27:06.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5e77fe700 1 -- 172.21.15.146:0/301596844 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00052d0 con 0x7fb6000a9490 2024-09-13T18:27:06.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5e77fe700 1 -- 172.21.15.146:0/301596844 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f0005430 con 0x7fb6000a9490 2024-09-13T18:27:06.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f6962ffd700 1 -- 172.21.15.146:0/755176423 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695400c880 con 0x7f696414f750 2024-09-13T18:27:06.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f6962ffd700 1 -- 172.21.15.146:0/755176423 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695400c9e0 con 0x7f696414f750 2024-09-13T18:27:06.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f6969782700 1 -- 172.21.15.146:0/755176423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f696414f750 msgr2=0x7f696406eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f6969782700 1 --2- 172.21.15.146:0/755176423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f696414f750 0x7f696406eea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6954009260 tx=0x7f6954009570 comp rx=0 tx=0).stop 2024-09-13T18:27:06.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f6969782700 1 -- 172.21.15.146:0/755176423 shutdown_connections 2024-09-13T18:27:06.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f6969782700 1 --2- 172.21.15.146:0/755176423 >> v2:172.21.15.154:3300/0 conn(0x7f696406f5c0 0x7f696406fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdff516700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcdf80cab20 0x7fcdf815b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdff516700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80caf00 0x7fcdf8157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdff516700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf80a97e0 con 0x7fcdf80cb300 2024-09-13T18:27:06.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdff516700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf80c99a0 con 0x7fcdf80cab20 2024-09-13T18:27:06.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdff516700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80723d0 con 0x7fcdf80caf00 2024-09-13T18:27:06.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f6969782700 1 --2- 172.21.15.146:0/755176423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f696414f750 0x7f696406eea0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f6969782700 1 -- 172.21.15.146:0/755176423 >> 172.21.15.146:0/755176423 conn(0x7f6964066090 msgr2=0x7f6964066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69bcd13700 1 -- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69b00351f0 msgr2=0x7f69b004e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69bcd13700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69b00351f0 0x7f69b004e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69bcd13700 1 -- 172.21.15.146:0/2188313344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69b0044be0 msgr2=0x7f69b00cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69bcd13700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69b0044be0 0x7f69b00cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69bcd13700 1 -- 172.21.15.146:0/2188313344 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b804f5c0 con 0x7f69b00355d0 2024-09-13T18:27:06.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69bcd13700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69b00355d0 0x7f69b00477b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69b0046040 tx=0x7f69a800b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69adffb700 1 -- 172.21.15.146:0/2188313344 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8011830 con 0x7f69b00355d0 2024-09-13T18:27:06.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69be516700 1 -- 172.21.15.146:0/2188313344 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b00dad50 con 0x7f69b00355d0 2024-09-13T18:27:06.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69be516700 1 -- 172.21.15.146:0/2188313344 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b00db310 con 0x7f69b00355d0 2024-09-13T18:27:06.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53cbc1c700 1 -- 172.21.15.146:0/2912149214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c40c3c90 con 0x7f53c40cab20 2024-09-13T18:27:06.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53cbc1c700 1 -- 172.21.15.146:0/2912149214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c40aa650 con 0x7f53c40cab20 2024-09-13T18:27:06.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f53ba7fc700 1 -- 172.21.15.146:0/2912149214 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c0012750 con 0x7f53c40cab20 2024-09-13T18:27:06.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f6969782700 1 -- 172.21.15.146:0/755176423 shutdown_connections 2024-09-13T18:27:06.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f6969782700 1 -- 172.21.15.146:0/755176423 wait complete. 2024-09-13T18:27:06.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69adffb700 1 -- 172.21.15.146:0/2188313344 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a8011990 con 0x7f69b00355d0 2024-09-13T18:27:06.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7f69adffb700 1 -- 172.21.15.146:0/2188313344 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8011d00 con 0x7f69b00355d0 2024-09-13T18:27:06.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5fffff700 1 -- 172.21.15.146:0/301596844 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60015d840 con 0x7fb6000a9490 2024-09-13T18:27:06.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fb5fffff700 1 -- 172.21.15.146:0/301596844 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6000c33f0 con 0x7fb6000a9490 2024-09-13T18:27:06.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f53ba7fc700 1 -- 172.21.15.146:0/2912149214 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0012ac0 con 0x7f53c40cab20 2024-09-13T18:27:06.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f53ba7fc700 1 -- 172.21.15.146:0/2912149214 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f53c0012c20 con 0x7f53c40cab20 2024-09-13T18:27:06.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdfe514700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcdf80cab20 0x7fcdf815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdfe514700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcdf80cab20 0x7fcdf815b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:46864/0 (socket says 172.21.15.146:46864) 2024-09-13T18:27:06.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdfe514700 1 -- 172.21.15.146:0/620584378 learned_addr learned my addr 172.21.15.146:0/620584378 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdfe514700 1 -- 172.21.15.146:0/620584378 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80caf00 msgr2=0x7fcdf8157af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:06.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdfe514700 1 --2- 172.21.15.146:0/620584378 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80caf00 0x7fcdf8157af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdfe514700 1 -- 172.21.15.146:0/620584378 >> v1:172.21.15.154:6789/0 conn(0x7fcdf80cb300 legacy=0x7fcdf81544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:06.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.409+0000 7fcdfe514700 1 -- 172.21.15.146:0/620584378 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf815b8c0 con 0x7fcdf80cab20 2024-09-13T18:27:06.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7fcdfe514700 1 --2- 172.21.15.146:0/620584378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80cab20 0x7fcdf815b380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400f9b0 tx=0x7fcdf400fcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ed19d68cc24eead server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6969782700 1 Processor -- start 2024-09-13T18:27:06.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6969782700 1 -- start start 2024-09-13T18:27:06.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f53ba7fc700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53a8066780 0x7f53a8068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7fb5e77fe700 1 -- 172.21.15.146:0/301596844 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00055a0 con 0x7fb6000a9490 2024-09-13T18:27:06.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f53c91b7700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53a8066780 0x7f53a8068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f53ba7fc700 1 -- 172.21.15.146:0/2912149214 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f53c00550a0 con 0x7f53c40cab20 2024-09-13T18:27:06.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7fb5e77fe700 1 -- 172.21.15.146:0/301596844 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb5f00036a0 con 0x7fb6000a9490 2024-09-13T18:27:06.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7fb5e77fe700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb5ec066880 0x7fb5ec068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6969782700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f696406f5c0 0x7f6964126a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6969782700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640a9490 0x7f69640c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6969782700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f696414f750 0x7f6964157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6969782700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6964126f70 con 0x7f69640a9490 2024-09-13T18:27:06.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6969782700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69640720e0 con 0x7f696414f750 2024-09-13T18:27:06.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6969782700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6964125e80 con 0x7f696406f5c0 2024-09-13T18:27:06.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f53c91b7700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53a8066780 0x7f53a8068c40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f53c40c5d20 tx=0x7f53b40090e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7fb5e77fe700 1 -- 172.21.15.146:0/301596844 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb5f00837e0 con 0x7fb6000a9490 2024-09-13T18:27:06.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f69adffb700 1 -- 172.21.15.146:0/2188313344 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69a801a400 con 0x7f69b00355d0 2024-09-13T18:27:06.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f69637fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640a9490 0x7f69640c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7fb5fe7fc700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb5ec066880 0x7fb5ec068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6963fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f696406f5c0 0x7f6964126a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f69637fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640a9490 0x7f69640c6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54852/0 (socket says 172.21.15.146:54852) 2024-09-13T18:27:06.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f69637fe700 1 -- 172.21.15.146:0/4040223709 learned_addr learned my addr 172.21.15.146:0/4040223709 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6968f81700 1 --2- 172.21.15.146:0/4040223709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f696414f750 0x7f6964157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f69637fe700 1 -- 172.21.15.146:0/4040223709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f696406f5c0 msgr2=0x7f6964126a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f69637fe700 1 --2- 172.21.15.146:0/4040223709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f696406f5c0 0x7f6964126a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f69637fe700 1 -- 172.21.15.146:0/4040223709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f696414f750 msgr2=0x7f6964157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f69637fe700 1 --2- 172.21.15.146:0/4040223709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f696414f750 0x7f6964157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f69637fe700 1 -- 172.21.15.146:0/4040223709 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6954009d40 con 0x7f69640a9490 2024-09-13T18:27:06.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f69637fe700 1 --2- 172.21.15.146:0/4040223709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640a9490 0x7f69640c6380 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6958005930 tx=0x7f695800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f69adffb700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f699c066800 0x7f699c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7fb5fe7fc700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb5ec066880 0x7fb5ec068d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb60012fa70 tx=0x7fb5f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6960ff9700 1 -- 172.21.15.146:0/4040223709 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958005d90 con 0x7f69640a9490 2024-09-13T18:27:06.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6960ff9700 1 -- 172.21.15.146:0/4040223709 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69580070d0 con 0x7f69640a9490 2024-09-13T18:27:06.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f69adffb700 1 -- 172.21.15.146:0/2188313344 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f69a8084440 con 0x7f69b00355d0 2024-09-13T18:27:06.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6969782700 1 -- 172.21.15.146:0/4040223709 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6964127fa0 con 0x7f69640a9490 2024-09-13T18:27:06.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f69bd514700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f699c066800 0x7f699c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6969782700 1 -- 172.21.15.146:0/4040223709 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6964071e00 con 0x7f69640a9490 2024-09-13T18:27:06.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.413+0000 7f6960ff9700 1 -- 172.21.15.146:0/4040223709 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958007480 con 0x7f69640a9490 2024-09-13T18:27:06.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.417+0000 7fcdfd512700 1 -- 172.21.15.146:0/620584378 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4009b80 con 0x7fcdf80cab20 2024-09-13T18:27:06.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.417+0000 7f53cbc1c700 1 -- 172.21.15.146:0/2912149214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ac005310 con 0x7f53c40cab20 2024-09-13T18:27:06.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.417+0000 7f69bd514700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f699c066800 0x7f699c068cc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f69b4000f60 tx=0x7f69b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.417+0000 7fcdfd512700 1 -- 172.21.15.146:0/620584378 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf400c070 con 0x7fcdf80cab20 2024-09-13T18:27:06.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.417+0000 7fcdff516700 1 -- 172.21.15.146:0/620584378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80cab20 msgr2=0x7fcdf815b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.417+0000 7fcdff516700 1 --2- 172.21.15.146:0/620584378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80cab20 0x7fcdf815b380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400f9b0 tx=0x7fcdf400fcc0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.417+0000 7fb5fffff700 1 -- 172.21.15.146:0/301596844 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6000c9da0 con 0x7fb6000a9490 2024-09-13T18:27:06.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.417+0000 7f6960ff9700 1 -- 172.21.15.146:0/4040223709 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69580075e0 con 0x7f69640a9490 2024-09-13T18:27:06.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.417+0000 7f6960ff9700 1 --2- 172.21.15.146:0/4040223709 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f694c066be0 0x7f694c0690a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.417+0000 7f6960ff9700 1 -- 172.21.15.146:0/4040223709 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f69580837a0 con 0x7f69640a9490 2024-09-13T18:27:06.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.417+0000 7f6963fff700 1 --2- 172.21.15.146:0/4040223709 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f694c066be0 0x7f694c0690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.417+0000 7f6963fff700 1 --2- 172.21.15.146:0/4040223709 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f694c066be0 0x7f694c0690a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f69540090b0 tx=0x7f6954008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.421+0000 7fb5e77fe700 1 -- 172.21.15.146:0/301596844 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb5f00508d0 con 0x7fb6000a9490 2024-09-13T18:27:06.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.421+0000 7f69be516700 1 -- 172.21.15.146:0/2188313344 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6998005310 con 0x7f69b00355d0 2024-09-13T18:27:06.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.421+0000 7f6969782700 1 -- 172.21.15.146:0/4040223709 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6944005310 con 0x7f69640a9490 2024-09-13T18:27:06.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.421+0000 7fcdff516700 1 -- 172.21.15.146:0/620584378 shutdown_connections 2024-09-13T18:27:06.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.421+0000 7fcdff516700 1 --2- 172.21.15.146:0/620584378 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80caf00 0x7fcdf8157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.421+0000 7fcdff516700 1 --2- 172.21.15.146:0/620584378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80cab20 0x7fcdf815b380 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.421+0000 7fcdff516700 1 -- 172.21.15.146:0/620584378 >> 172.21.15.146:0/620584378 conn(0x7fcdf8066090 msgr2=0x7fcdf8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.421+0000 7fcdff516700 1 -- 172.21.15.146:0/620584378 shutdown_connections 2024-09-13T18:27:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.421+0000 7fcdff516700 1 -- 172.21.15.146:0/620584378 wait complete. 2024-09-13T18:27:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.421+0000 7fcdff516700 1 Processor -- start 2024-09-13T18:27:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.421+0000 7fcdff516700 1 -- start start 2024-09-13T18:27:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdff516700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80cab20 0x7fcdf80c0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdff516700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80caf00 0x7fcdf81eb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdff516700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80cb300 0x7fcdf8152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdff516700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf81ebe90 con 0x7fcdf80cb300 2024-09-13T18:27:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdff516700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf81eaa20 con 0x7fcdf80caf00 2024-09-13T18:27:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdff516700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8159ff0 con 0x7fcdf80cab20 2024-09-13T18:27:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdfed15700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80cb300 0x7fcdf8152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdfed15700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80cb300 0x7fcdf8152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54862/0 (socket says 172.21.15.146:54862) 2024-09-13T18:27:06.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdfed15700 1 -- 172.21.15.146:0/2860106917 learned_addr learned my addr 172.21.15.146:0/2860106917 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdfdd13700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80caf00 0x7fcdf81eb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdfe514700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80cab20 0x7fcdf80c0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdfed15700 1 -- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80cab20 msgr2=0x7fcdf80c0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdfed15700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80cab20 0x7fcdf80c0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdfed15700 1 -- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80caf00 msgr2=0x7fcdf81eb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdfed15700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80caf00 0x7fcdf81eb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdfed15700 1 -- 172.21.15.146:0/2860106917 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf400f610 con 0x7fcdf80cb300 2024-09-13T18:27:06.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.425+0000 7fcdfed15700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80cb300 0x7fcdf8152d00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000b990 tx=0x7fcdf000bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.429+0000 7f53ba7fc700 1 -- 172.21.15.146:0/2912149214 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f53c0050ce0 con 0x7f53c40cab20 2024-09-13T18:27:06.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe25c3d700 1 Processor -- start 2024-09-13T18:27:06.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe25c3d700 1 -- start start 2024-09-13T18:27:06.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7f69adffb700 1 -- 172.21.15.146:0/2188313344 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f69a8051410 con 0x7f69b00355d0 2024-09-13T18:27:06.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe25c3d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efe200caed0 0x7efe20158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe25c3d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efe2006f7d0 0x7efe200c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe25c3d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe200c7d20 con 0x7efe200cb2b0 2024-09-13T18:27:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe25c3d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe20072970 con 0x7efe200caed0 2024-09-13T18:27:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe25c3d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe200c8ad0 con 0x7efe2006f7d0 2024-09-13T18:27:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe24c3b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efe200caed0 0x7efe20158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe24c3b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efe200caed0 0x7efe20158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54872/0 (socket says 172.21.15.146:54872) 2024-09-13T18:27:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe24c3b700 1 -- 172.21.15.146:0/3250147716 learned_addr learned my addr 172.21.15.146:0/3250147716 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe24c3b700 1 -- 172.21.15.146:0/3250147716 >> v2:172.21.15.154:3300/0 conn(0x7efe2006f7d0 msgr2=0x7efe200c81b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe24c3b700 1 --2- 172.21.15.146:0/3250147716 >> v2:172.21.15.154:3300/0 conn(0x7efe2006f7d0 0x7efe200c81b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe24c3b700 1 -- 172.21.15.146:0/3250147716 >> v1:172.21.15.154:6789/0 conn(0x7efe200cb2b0 legacy=0x7efe20072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe24c3b700 1 -- 172.21.15.146:0/3250147716 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe200c8d90 con 0x7efe200caed0 2024-09-13T18:27:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.433+0000 7efe24c3b700 1 --2- 172.21.15.146:0/3250147716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe200caed0 0x7efe20158110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efe200cb760 tx=0x7efe10009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a22d937c33c89c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.437+0000 7f6960ff9700 1 -- 172.21.15.146:0/4040223709 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f69580508f0 con 0x7f69640a9490 2024-09-13T18:27:06.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.437+0000 7efe1f7fe700 1 -- 172.21.15.146:0/3250147716 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10004230 con 0x7efe200caed0 2024-09-13T18:27:06.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.437+0000 7efe1f7fe700 1 -- 172.21.15.146:0/3250147716 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe10004390 con 0x7efe200caed0 2024-09-13T18:27:06.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.437+0000 7efe1f7fe700 1 -- 172.21.15.146:0/3250147716 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe100036c0 con 0x7efe200caed0 2024-09-13T18:27:06.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.437+0000 7efe25c3d700 1 -- 172.21.15.146:0/3250147716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe200caed0 msgr2=0x7efe20158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.437+0000 7efe25c3d700 1 --2- 172.21.15.146:0/3250147716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe200caed0 0x7efe20158110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efe200cb760 tx=0x7efe10009bf0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.437+0000 7efe25c3d700 1 -- 172.21.15.146:0/3250147716 shutdown_connections 2024-09-13T18:27:06.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.437+0000 7efe25c3d700 1 --2- 172.21.15.146:0/3250147716 >> v2:172.21.15.154:3300/0 conn(0x7efe2006f7d0 0x7efe200c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.437+0000 7efe25c3d700 1 --2- 172.21.15.146:0/3250147716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe200caed0 0x7efe20158110 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.437+0000 7efe25c3d700 1 -- 172.21.15.146:0/3250147716 >> 172.21.15.146:0/3250147716 conn(0x7efe20066090 msgr2=0x7efe20066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.441+0000 7fcdeeffd700 1 -- 172.21.15.146:0/2860106917 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf000c930 con 0x7fcdf80cb300 2024-09-13T18:27:06.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.441+0000 7fcdeeffd700 1 -- 172.21.15.146:0/2860106917 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf000ca90 con 0x7fcdf80cb300 2024-09-13T18:27:06.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.441+0000 7fcdff516700 1 -- 172.21.15.146:0/2860106917 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf815a2d0 con 0x7fcdf80cb300 2024-09-13T18:27:06.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.441+0000 7fcdeeffd700 1 -- 172.21.15.146:0/2860106917 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf000ce40 con 0x7fcdf80cb300 2024-09-13T18:27:06.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.441+0000 7efe25c3d700 1 -- 172.21.15.146:0/3250147716 shutdown_connections 2024-09-13T18:27:06.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.441+0000 7fcdff516700 1 -- 172.21.15.146:0/2860106917 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf81eace0 con 0x7fcdf80cb300 2024-09-13T18:27:06.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.441+0000 7efe25c3d700 1 -- 172.21.15.146:0/3250147716 wait complete. 2024-09-13T18:27:06.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7fcdeeffd700 1 -- 172.21.15.146:0/2860106917 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcdf0007370 con 0x7fcdf80cb300 2024-09-13T18:27:06.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7efe25c3d700 1 Processor -- start 2024-09-13T18:27:06.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7efe25c3d700 1 -- start start 2024-09-13T18:27:06.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7fcdeeffd700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcde4066900 0x7fcde4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7fcdeeffd700 1 -- 172.21.15.146:0/2860106917 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcdf0083d30 con 0x7fcdf80cb300 2024-09-13T18:27:06.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7fcdfe514700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcde4066900 0x7fcde4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7efe25c3d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe2006f7d0 0x7efe20152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7efe25c3d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe200caed0 0x7efe201f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7efe24c3b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe2006f7d0 0x7efe20152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7efe24c3b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe2006f7d0 0x7efe20152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54878/0 (socket says 172.21.15.146:54878) 2024-09-13T18:27:06.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7efe24c3b700 1 -- 172.21.15.146:0/3904066874 learned_addr learned my addr 172.21.15.146:0/3904066874 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7efe25c3d700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe200cb2b0 0x7efe201f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7efe25c3d700 1 -- 172.21.15.146:0/3904066874 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe201f39f0 con 0x7efe2006f7d0 2024-09-13T18:27:06.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7efe25c3d700 1 -- 172.21.15.146:0/3904066874 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe200c4ae0 con 0x7efe200cb2b0 2024-09-13T18:27:06.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7efe25c3d700 1 -- 172.21.15.146:0/3904066874 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe200c41f0 con 0x7efe200caed0 2024-09-13T18:27:06.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.445+0000 7fcdfe514700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcde4066900 0x7fcde4068dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400f5e0 tx=0x7fcdf4008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.449+0000 7efe1ffff700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe200caed0 0x7efe201f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.449+0000 7efe24c3b700 1 -- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe200caed0 msgr2=0x7efe201f2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.449+0000 7efe24c3b700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe200caed0 0x7efe201f2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.449+0000 7efe24c3b700 1 -- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe200cb2b0 msgr2=0x7efe201f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.449+0000 7efe24c3b700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe200cb2b0 0x7efe201f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.449+0000 7efe24c3b700 1 -- 172.21.15.146:0/3904066874 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe10009570 con 0x7efe2006f7d0 2024-09-13T18:27:06.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.449+0000 7efe24c3b700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe2006f7d0 0x7efe20152f60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efe10004000 tx=0x7efe10004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.449+0000 7fcdff516700 1 -- 172.21.15.146:0/2860106917 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde0005310 con 0x7fcdf80cb300 2024-09-13T18:27:06.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.453+0000 7efe1d7fa700 1 -- 172.21.15.146:0/3904066874 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1000ba70 con 0x7efe2006f7d0 2024-09-13T18:27:06.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.453+0000 7efe25c3d700 1 -- 172.21.15.146:0/3904066874 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe20150300 con 0x7efe2006f7d0 2024-09-13T18:27:06.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.453+0000 7efe25c3d700 1 -- 172.21.15.146:0/3904066874 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe200c4770 con 0x7efe2006f7d0 2024-09-13T18:27:06.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.453+0000 7efe1d7fa700 1 -- 172.21.15.146:0/3904066874 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1000bbd0 con 0x7efe2006f7d0 2024-09-13T18:27:06.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.453+0000 7efe1d7fa700 1 -- 172.21.15.146:0/3904066874 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10024470 con 0x7efe2006f7d0 2024-09-13T18:27:06.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.457+0000 7efe25c3d700 1 -- 172.21.15.146:0/3904066874 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe00005310 con 0x7efe2006f7d0 2024-09-13T18:27:06.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.457+0000 7fcdeeffd700 1 -- 172.21.15.146:0/2860106917 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcdf0050d20 con 0x7fcdf80cb300 2024-09-13T18:27:06.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.469+0000 7efe1d7fa700 1 -- 172.21.15.146:0/3904066874 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efe10021070 con 0x7efe2006f7d0 2024-09-13T18:27:06.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.469+0000 7efe1d7fa700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efe0c066780 0x7efe0c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.469+0000 7efe1d7fa700 1 -- 172.21.15.146:0/3904066874 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efe10086810 con 0x7efe2006f7d0 2024-09-13T18:27:06.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.469+0000 7efe1d7fa700 1 -- 172.21.15.146:0/3904066874 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efe10086c80 con 0x7efe2006f7d0 2024-09-13T18:27:06.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.469+0000 7efe1ffff700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efe0c066780 0x7efe0c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.469+0000 7efe1ffff700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efe0c066780 0x7efe0c068c40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe200c6e80 tx=0x7efe180094f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.673+0000 7f5d1ffff700 1 -- 172.21.15.146:0/970864903 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5cfc005180 con 0x7f5d200a9490 2024-09-13T18:27:06.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.673+0000 7f5d0f7fe700 1 -- 172.21.15.146:0/970864903 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5d14001120 con 0x7f5d200a9490 2024-09-13T18:27:06.681 INFO:teuthology.orchestra.run.smithi146.stdout:64424509449 2024-09-13T18:27:06.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.681+0000 7f5d0d7fa700 1 -- 172.21.15.146:0/970864903 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d04066780 msgr2=0x7f5d04068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.681+0000 7f5d0d7fa700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d04066780 0x7f5d04068c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d10004c30 tx=0x7f5d1000a330 comp rx=0 tx=0).stop 2024-09-13T18:27:06.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.681+0000 7f5d0d7fa700 1 -- 172.21.15.146:0/970864903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d200a9490 msgr2=0x7f5d20151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.681+0000 7f5d0d7fa700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d200a9490 0x7f5d20151ea0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400c440 tx=0x7f5d14001e70 comp rx=0 tx=0).stop 2024-09-13T18:27:06.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.681+0000 7f5d0d7fa700 1 -- 172.21.15.146:0/970864903 shutdown_connections 2024-09-13T18:27:06.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.681+0000 7f5d0d7fa700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d04066780 0x7f5d04068c40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.681+0000 7f5d0d7fa700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2014f750 0x7f5d201523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.681+0000 7f5d0d7fa700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d200a9490 0x7f5d20151ea0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.681+0000 7f5d0d7fa700 1 --2- 172.21.15.146:0/970864903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d2006fcd0 0x7f5d201505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.681+0000 7f5d0d7fa700 1 -- 172.21.15.146:0/970864903 >> 172.21.15.146:0/970864903 conn(0x7f5d20066090 msgr2=0x7f5d200aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.681+0000 7f5d0d7fa700 1 -- 172.21.15.146:0/970864903 shutdown_connections 2024-09-13T18:27:06.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.681+0000 7f5d0d7fa700 1 -- 172.21.15.146:0/970864903 wait complete. 2024-09-13T18:27:06.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.4 2024-09-13T18:27:06.705 DEBUG:teuthology.parallel:result is None 2024-09-13T18:27:06.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.705+0000 7fd9477fe700 1 -- 172.21.15.146:0/1570544972 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd920005890 con 0x7fd94006fcd0 2024-09-13T18:27:06.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.709+0000 7fd935ffb700 1 -- 172.21.15.146:0/1570544972 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd930055210 con 0x7fd94006fcd0 2024-09-13T18:27:06.713 INFO:teuthology.orchestra.run.smithi146.stdout:64424509450 2024-09-13T18:27:06.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.709+0000 7fd9477fe700 1 -- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd924066780 msgr2=0x7fd924068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.709+0000 7fd9477fe700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd924066780 0x7fd924068c40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd940152c70 tx=0x7fd93c009040 comp rx=0 tx=0).stop 2024-09-13T18:27:06.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.709+0000 7fd9477fe700 1 -- 172.21.15.146:0/1570544972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd94006fcd0 msgr2=0x7fd9401567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.709+0000 7fd9477fe700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd94006fcd0 0x7fd9401567a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd93000d960 tx=0x7fd93000dd20 comp rx=0 tx=0).stop 2024-09-13T18:27:06.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.709+0000 7fd9477fe700 1 -- 172.21.15.146:0/1570544972 shutdown_connections 2024-09-13T18:27:06.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.709+0000 7fd9477fe700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd924066780 0x7fd924068c40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.709+0000 7fd9477fe700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd94014f750 0x7fd9400abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.709+0000 7fd9477fe700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9400a9490 0x7fd9400c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.709+0000 7fd9477fe700 1 --2- 172.21.15.146:0/1570544972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd94006fcd0 0x7fd9401567a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.709+0000 7fd9477fe700 1 -- 172.21.15.146:0/1570544972 >> 172.21.15.146:0/1570544972 conn(0x7fd940066090 msgr2=0x7fd940066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.709+0000 7fd9477fe700 1 -- 172.21.15.146:0/1570544972 shutdown_connections 2024-09-13T18:27:06.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.709+0000 7fd9477fe700 1 -- 172.21.15.146:0/1570544972 wait complete. 2024-09-13T18:27:06.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.6 2024-09-13T18:27:06.725 DEBUG:teuthology.parallel:result is None 2024-09-13T18:27:06.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.725+0000 7fb5fffff700 1 -- 172.21.15.146:0/301596844 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb6000a9d60 con 0x7fb6000a9490 2024-09-13T18:27:06.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.725+0000 7fb5e77fe700 1 -- 172.21.15.146:0/301596844 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb5f0053ef0 con 0x7fb6000a9490 2024-09-13T18:27:06.730 INFO:teuthology.orchestra.run.smithi146.stdout:55834574858 2024-09-13T18:27:06.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.725+0000 7fb5fffff700 1 -- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb5ec066880 msgr2=0x7fb5ec068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.725+0000 7fb5fffff700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb5ec066880 0x7fb5ec068d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb60012fa70 tx=0x7fb5f4008040 comp rx=0 tx=0).stop 2024-09-13T18:27:06.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.725+0000 7fb5fffff700 1 -- 172.21.15.146:0/301596844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6000a9490 msgr2=0x7fb60015cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.725+0000 7fb5fffff700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6000a9490 0x7fb60015cb60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000c530 tx=0x7fb5f000c8f0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.725+0000 7fb5fffff700 1 -- 172.21.15.146:0/301596844 shutdown_connections 2024-09-13T18:27:06.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.725+0000 7fb5fffff700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb5ec066880 0x7fb5ec068d40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.725+0000 7fb5fffff700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6000c9350 0x7fb6001fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.725+0000 7fb5fffff700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6000c8f70 0x7fb6000af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.725+0000 7fb5fffff700 1 --2- 172.21.15.146:0/301596844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6000a9490 0x7fb60015cb60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.729+0000 7fb5fffff700 1 -- 172.21.15.146:0/301596844 >> 172.21.15.146:0/301596844 conn(0x7fb600066090 msgr2=0x7fb600130120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.729+0000 7fb5fffff700 1 -- 172.21.15.146:0/301596844 shutdown_connections 2024-09-13T18:27:06.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.729+0000 7fb5fffff700 1 -- 172.21.15.146:0/301596844 wait complete. 2024-09-13T18:27:06.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2024-09-13T18:27:06.747 DEBUG:teuthology.parallel:result is None 2024-09-13T18:27:06.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.745+0000 7f53cbc1c700 1 -- 172.21.15.146:0/2912149214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f53ac005890 con 0x7f53c40cab20 2024-09-13T18:27:06.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.745+0000 7f53ba7fc700 1 -- 172.21.15.146:0/2912149214 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f53c0054300 con 0x7f53c40cab20 2024-09-13T18:27:06.752 INFO:teuthology.orchestra.run.smithi146.stdout:55834574858 2024-09-13T18:27:06.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.745+0000 7f6969782700 1 -- 172.21.15.146:0/4040223709 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6944005890 con 0x7f69640a9490 2024-09-13T18:27:06.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.745+0000 7f6960ff9700 1 -- 172.21.15.146:0/4040223709 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6958017070 con 0x7f69640a9490 2024-09-13T18:27:06.753 INFO:teuthology.orchestra.run.smithi146.stdout:60129542155 2024-09-13T18:27:06.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69be516700 1 -- 172.21.15.146:0/2188313344 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f69980055c0 con 0x7f69b00355d0 2024-09-13T18:27:06.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69adffb700 1 -- 172.21.15.146:0/2188313344 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f69a8011e60 con 0x7f69b00355d0 2024-09-13T18:27:06.754 INFO:teuthology.orchestra.run.smithi146.stdout:64424509449 2024-09-13T18:27:06.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f53cbc1c700 1 -- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53a8066780 msgr2=0x7f53a8068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f53cbc1c700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53a8066780 0x7f53a8068c40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f53c40c5d20 tx=0x7f53b40090e0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f53cbc1c700 1 -- 172.21.15.146:0/2912149214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53c40cab20 msgr2=0x7f53c40c41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f53cbc1c700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53c40cab20 0x7f53c40c41b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f53c000d800 tx=0x7f53c000db10 comp rx=0 tx=0).stop 2024-09-13T18:27:06.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f53cbc1c700 1 -- 172.21.15.146:0/2912149214 shutdown_connections 2024-09-13T18:27:06.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f53cbc1c700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53a8066780 0x7f53a8068c40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f53cbc1c700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c41544a0 0x7f53c41fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f53cbc1c700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c40caf00 0x7f53c4201690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f53cbc1c700 1 --2- 172.21.15.146:0/2912149214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53c40cab20 0x7f53c40c41b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f53cbc1c700 1 -- 172.21.15.146:0/2912149214 >> 172.21.15.146:0/2912149214 conn(0x7f53c4066090 msgr2=0x7f53c40c6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f53cbc1c700 1 -- 172.21.15.146:0/2912149214 shutdown_connections 2024-09-13T18:27:06.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f53cbc1c700 1 -- 172.21.15.146:0/2912149214 wait complete. 2024-09-13T18:27:06.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f6969782700 1 -- 172.21.15.146:0/4040223709 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f694c066be0 msgr2=0x7f694c0690a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f6969782700 1 --2- 172.21.15.146:0/4040223709 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f694c066be0 0x7f694c0690a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f69540090b0 tx=0x7f6954008030 comp rx=0 tx=0).stop 2024-09-13T18:27:06.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f6969782700 1 -- 172.21.15.146:0/4040223709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640a9490 msgr2=0x7f69640c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f6969782700 1 --2- 172.21.15.146:0/4040223709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640a9490 0x7f69640c6380 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6958005930 tx=0x7f695800c9e0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f6969782700 1 -- 172.21.15.146:0/4040223709 shutdown_connections 2024-09-13T18:27:06.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f6969782700 1 --2- 172.21.15.146:0/4040223709 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f694c066be0 0x7f694c0690a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f6969782700 1 --2- 172.21.15.146:0/4040223709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f696414f750 0x7f6964157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f6969782700 1 --2- 172.21.15.146:0/4040223709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640a9490 0x7f69640c6380 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f6969782700 1 --2- 172.21.15.146:0/4040223709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f696406f5c0 0x7f6964126a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f6969782700 1 -- 172.21.15.146:0/4040223709 >> 172.21.15.146:0/4040223709 conn(0x7f6964066090 msgr2=0x7f69640c79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f6969782700 1 -- 172.21.15.146:0/4040223709 shutdown_connections 2024-09-13T18:27:06.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f6969782700 1 -- 172.21.15.146:0/4040223709 wait complete. 2024-09-13T18:27:06.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69be516700 1 -- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f699c066800 msgr2=0x7f699c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69be516700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f699c066800 0x7f699c068cc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f69b4000f60 tx=0x7f69b4009040 comp rx=0 tx=0).stop 2024-09-13T18:27:06.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69be516700 1 -- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69b00355d0 msgr2=0x7f69b00477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69be516700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69b00355d0 0x7f69b00477b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69b0046040 tx=0x7f69a800b750 comp rx=0 tx=0).stop 2024-09-13T18:27:06.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69be516700 1 -- 172.21.15.146:0/2188313344 shutdown_connections 2024-09-13T18:27:06.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69be516700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f699c066800 0x7f699c068cc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69be516700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69b0044be0 0x7f69b00cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69be516700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69b00355d0 0x7f69b00477b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69be516700 1 --2- 172.21.15.146:0/2188313344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69b00351f0 0x7f69b004e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69be516700 1 -- 172.21.15.146:0/2188313344 >> 172.21.15.146:0/2188313344 conn(0x7f69b004fd00 msgr2=0x7f69b00d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69be516700 1 -- 172.21.15.146:0/2188313344 shutdown_connections 2024-09-13T18:27:06.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.749+0000 7f69be516700 1 -- 172.21.15.146:0/2188313344 wait complete. 2024-09-13T18:27:06.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.7 2024-09-13T18:27:06.769 DEBUG:teuthology.parallel:result is None 2024-09-13T18:27:06.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.1 2024-09-13T18:27:06.772 DEBUG:teuthology.parallel:result is None 2024-09-13T18:27:06.772 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.2 2024-09-13T18:27:06.772 DEBUG:teuthology.parallel:result is None 2024-09-13T18:27:06.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.769+0000 7fcdff516700 1 -- 172.21.15.146:0/2860106917 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcde00055c0 con 0x7fcdf80cb300 2024-09-13T18:27:06.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.769+0000 7fcdeeffd700 1 -- 172.21.15.146:0/2860106917 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcdf0019020 con 0x7fcdf80cb300 2024-09-13T18:27:06.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.769+0000 7efe25c3d700 1 -- 172.21.15.146:0/3904066874 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efe00005890 con 0x7efe2006f7d0 2024-09-13T18:27:06.776 INFO:teuthology.orchestra.run.smithi146.stdout:55834574858 2024-09-13T18:27:06.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.769+0000 7efe1d7fa700 1 -- 172.21.15.146:0/3904066874 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efe10053860 con 0x7efe2006f7d0 2024-09-13T18:27:06.777 INFO:teuthology.orchestra.run.smithi146.stdout:60129542154 2024-09-13T18:27:06.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7efe25c3d700 1 -- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efe0c066780 msgr2=0x7efe0c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7efe25c3d700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efe0c066780 0x7efe0c068c40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe200c6e80 tx=0x7efe180094f0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7fcdff516700 1 -- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcde4066900 msgr2=0x7fcde4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7efe25c3d700 1 -- 172.21.15.146:0/3904066874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe2006f7d0 msgr2=0x7efe20152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7efe25c3d700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe2006f7d0 0x7efe20152f60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efe10004000 tx=0x7efe10004770 comp rx=0 tx=0).stop 2024-09-13T18:27:06.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7fcdff516700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcde4066900 0x7fcde4068dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400f5e0 tx=0x7fcdf4008070 comp rx=0 tx=0).stop 2024-09-13T18:27:06.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7fcdff516700 1 -- 172.21.15.146:0/2860106917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80cb300 msgr2=0x7fcdf8152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7fcdff516700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80cb300 0x7fcdf8152d00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000b990 tx=0x7fcdf000bca0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7efe25c3d700 1 -- 172.21.15.146:0/3904066874 shutdown_connections 2024-09-13T18:27:06.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7efe25c3d700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efe0c066780 0x7efe0c068c40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7efe25c3d700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe200cb2b0 0x7efe201f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7efe25c3d700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe200caed0 0x7efe201f2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7efe25c3d700 1 --2- 172.21.15.146:0/3904066874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe2006f7d0 0x7efe20152f60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7efe25c3d700 1 -- 172.21.15.146:0/3904066874 >> 172.21.15.146:0/3904066874 conn(0x7efe20066090 msgr2=0x7efe20066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7efe25c3d700 1 -- 172.21.15.146:0/3904066874 shutdown_connections 2024-09-13T18:27:06.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7efe25c3d700 1 -- 172.21.15.146:0/3904066874 wait complete. 2024-09-13T18:27:06.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7fcdff516700 1 -- 172.21.15.146:0/2860106917 shutdown_connections 2024-09-13T18:27:06.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7fcdff516700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcde4066900 0x7fcde4068dc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7fcdff516700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80cb300 0x7fcdf8152d00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7fcdff516700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80caf00 0x7fcdf81eb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7fcdff516700 1 --2- 172.21.15.146:0/2860106917 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80cab20 0x7fcdf80c0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7fcdff516700 1 -- 172.21.15.146:0/2860106917 >> 172.21.15.146:0/2860106917 conn(0x7fcdf8066090 msgr2=0x7fcdf8155160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7fcdff516700 1 -- 172.21.15.146:0/2860106917 shutdown_connections 2024-09-13T18:27:06.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.773+0000 7fcdff516700 1 -- 172.21.15.146:0/2860106917 wait complete. 2024-09-13T18:27:06.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.3 2024-09-13T18:27:06.789 DEBUG:teuthology.parallel:result is None 2024-09-13T18:27:06.791 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.0 2024-09-13T18:27:06.792 DEBUG:teuthology.parallel:result is None 2024-09-13T18:27:06.792 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T18:27:06.792 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:27:06.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.937+0000 7f5e309da700 1 Processor -- start 2024-09-13T18:27:06.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.937+0000 7f5e309da700 1 -- start start 2024-09-13T18:27:06.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.937+0000 7f5e309da700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5e2c153860 0x7f5e2c0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.937+0000 7f5e309da700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e2c159430 0x7f5e2c0c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.937+0000 7f5e309da700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c0c0f80 con 0x7f5e2c0b12a0 2024-09-13T18:27:06.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.937+0000 7f5e309da700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2c15cf30 con 0x7f5e2c153860 2024-09-13T18:27:06.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.937+0000 7f5e309da700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c15a420 con 0x7f5e2c159430 2024-09-13T18:27:06.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.937+0000 7f5e2b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e2c159430 0x7f5e2c0c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.937+0000 7f5e2ad9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5e2c153860 0x7f5e2c0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.937+0000 7f5e2259c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5e2c0b12a0 0x7f5e2c15d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:55046/0 (socket says 172.21.15.146:55046) 2024-09-13T18:27:06.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.937+0000 7f5e2259c700 1 -- 172.21.15.146:0/3239039902 learned_addr learned my addr 172.21.15.146:0/3239039902 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:06.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e2a59c700 1 -- 172.21.15.146:0/3239039902 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475898401 0 0) 0x7f5e2c0c0f80 con 0x7f5e2c0b12a0 2024-09-13T18:27:06.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e2a59c700 1 -- 172.21.15.146:0/3239039902 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e180035d0 con 0x7f5e2c0b12a0 2024-09-13T18:27:06.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e2b59e700 1 -- 172.21.15.146:0/3239039902 >> v2:172.21.15.154:3300/0 conn(0x7f5e2c153860 msgr2=0x7f5e2c0c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:06.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e2b59e700 1 --2- 172.21.15.146:0/3239039902 >> v2:172.21.15.154:3300/0 conn(0x7f5e2c153860 0x7f5e2c0c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e2b59e700 1 -- 172.21.15.146:0/3239039902 >> v1:172.21.15.154:6789/0 conn(0x7f5e2c0b12a0 legacy=0x7f5e2c15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e2b59e700 1 -- 172.21.15.146:0/3239039902 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c0c1790 con 0x7f5e2c159430 2024-09-13T18:27:06.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e2ad9d700 1 --2- 172.21.15.146:0/3239039902 >> v2:172.21.15.154:3300/0 conn(0x7f5e2c153860 0x7f5e2c0c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:06.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e2b59e700 1 --2- 172.21.15.146:0/3239039902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e2c159430 0x7f5e2c0c2d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009d90 tx=0x7f5e2400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0e032c35acc803a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e2a59c700 1 -- 172.21.15.146:0/3239039902 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400e070 con 0x7f5e2c159430 2024-09-13T18:27:06.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e2a59c700 1 -- 172.21.15.146:0/3239039902 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2400bb20 con 0x7f5e2c159430 2024-09-13T18:27:06.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e2a59c700 1 -- 172.21.15.146:0/3239039902 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400be90 con 0x7f5e2c159430 2024-09-13T18:27:06.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e309da700 1 -- 172.21.15.146:0/3239039902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e2c159430 msgr2=0x7f5e2c0c2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e309da700 1 --2- 172.21.15.146:0/3239039902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e2c159430 0x7f5e2c0c2d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009d90 tx=0x7f5e2400b330 comp rx=0 tx=0).stop 2024-09-13T18:27:06.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e309da700 1 -- 172.21.15.146:0/3239039902 shutdown_connections 2024-09-13T18:27:06.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e309da700 1 --2- 172.21.15.146:0/3239039902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e2c159430 0x7f5e2c0c2d00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e309da700 1 --2- 172.21.15.146:0/3239039902 >> v2:172.21.15.154:3300/0 conn(0x7f5e2c153860 0x7f5e2c0c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e309da700 1 -- 172.21.15.146:0/3239039902 >> 172.21.15.146:0/3239039902 conn(0x7f5e2c071ed0 msgr2=0x7f5e2c0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e309da700 1 -- 172.21.15.146:0/3239039902 shutdown_connections 2024-09-13T18:27:06.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.941+0000 7f5e309da700 1 -- 172.21.15.146:0/3239039902 wait complete. 2024-09-13T18:27:06.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e309da700 1 Processor -- start 2024-09-13T18:27:06.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e309da700 1 -- start start 2024-09-13T18:27:06.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e309da700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e2c0b12a0 0x7f5e2c157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e309da700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e2c153860 0x7f5e2c1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e309da700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c159430 0x7f5e2c0c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e309da700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c154e20 con 0x7f5e2c159430 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e309da700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0c2bd0 con 0x7f5e2c0b12a0 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e309da700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0c3e00 con 0x7f5e2c153860 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e2259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e2c153860 0x7f5e2c1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e2b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c159430 0x7f5e2c0c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e2259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e2c153860 0x7f5e2c1552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41558/0 (socket says 172.21.15.146:41558) 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e2259c700 1 -- 172.21.15.146:0/135541603 learned_addr learned my addr 172.21.15.146:0/135541603 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e2b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c159430 0x7f5e2c0c4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54882/0 (socket says 172.21.15.146:54882) 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e2ad9d700 1 --2- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e2c0b12a0 0x7f5e2c157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e2259c700 1 -- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e2c0b12a0 msgr2=0x7f5e2c157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e2259c700 1 --2- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e2c0b12a0 0x7f5e2c157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e2259c700 1 -- 172.21.15.146:0/135541603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c159430 msgr2=0x7f5e2c0c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e2259c700 1 --2- 172.21.15.146:0/135541603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c159430 0x7f5e2c0c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e2259c700 1 -- 172.21.15.146:0/135541603 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c0094b0 con 0x7f5e2c153860 2024-09-13T18:27:06.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.945+0000 7f5e2259c700 1 --2- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e2c153860 0x7f5e2c1552b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400c520 tx=0x7f5e1400c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.949+0000 7f5e23fff700 1 -- 172.21.15.146:0/135541603 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e14005d90 con 0x7f5e2c153860 2024-09-13T18:27:06.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.949+0000 7f5e23fff700 1 -- 172.21.15.146:0/135541603 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e140070d0 con 0x7f5e2c153860 2024-09-13T18:27:06.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.949+0000 7f5e23fff700 1 -- 172.21.15.146:0/135541603 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e14007440 con 0x7f5e2c153860 2024-09-13T18:27:06.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.949+0000 7f5e309da700 1 -- 172.21.15.146:0/135541603 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e24009a70 con 0x7f5e2c153860 2024-09-13T18:27:06.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.949+0000 7f5e309da700 1 -- 172.21.15.146:0/135541603 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c0c5330 con 0x7f5e2c153860 2024-09-13T18:27:06.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.949+0000 7f5e309da700 1 -- 172.21.15.146:0/135541603 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c154fd0 con 0x7f5e2c153860 2024-09-13T18:27:06.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.949+0000 7f5e23fff700 1 -- 172.21.15.146:0/135541603 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5e140075a0 con 0x7f5e2c153860 2024-09-13T18:27:06.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.949+0000 7f5e23fff700 1 --2- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e0c066730 0x7f5e0c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.949+0000 7f5e23fff700 1 -- 172.21.15.146:0/135541603 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5e14083790 con 0x7f5e2c153860 2024-09-13T18:27:06.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.949+0000 7f5e2ad9d700 1 --2- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e0c066730 0x7f5e0c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.953+0000 7f5e2ad9d700 1 --2- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e0c066730 0x7f5e0c068bf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009d80 tx=0x7f5e1c009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:06.961+0000 7f5e23fff700 1 -- 172.21.15.146:0/135541603 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5e2c154fd0 con 0x7f5e2c153860 2024-09-13T18:27:07.117 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:27:07.119 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:27:07.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.113+0000 7f5e309da700 1 -- 172.21.15.146:0/135541603 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e2c15cbc0 con 0x7f5e0c066730 2024-09-13T18:27:07.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.133+0000 7f5e23fff700 1 -- 172.21.15.146:0/135541603 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345422 (secure 0 0 0) 0x7f5e1c0099d0 con 0x7f5e0c066730 2024-09-13T18:27:07.143 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:07.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.141+0000 7f5e309da700 1 -- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e0c066730 msgr2=0x7f5e0c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.141+0000 7f5e309da700 1 --2- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e0c066730 0x7f5e0c068bf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009d80 tx=0x7f5e1c009240 comp rx=0 tx=0).stop 2024-09-13T18:27:07.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.141+0000 7f5e309da700 1 -- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e2c153860 msgr2=0x7f5e2c1552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.141+0000 7f5e309da700 1 --2- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e2c153860 0x7f5e2c1552b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400c520 tx=0x7f5e1400c830 comp rx=0 tx=0).stop 2024-09-13T18:27:07.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.141+0000 7f5e309da700 1 -- 172.21.15.146:0/135541603 shutdown_connections 2024-09-13T18:27:07.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.141+0000 7f5e309da700 1 --2- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e0c066730 0x7f5e0c068bf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.141+0000 7f5e309da700 1 --2- 172.21.15.146:0/135541603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c159430 0x7f5e2c0c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.141+0000 7f5e309da700 1 --2- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e2c153860 0x7f5e2c1552b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.141+0000 7f5e309da700 1 --2- 172.21.15.146:0/135541603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e2c0b12a0 0x7f5e2c157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.141+0000 7f5e309da700 1 -- 172.21.15.146:0/135541603 >> 172.21.15.146:0/135541603 conn(0x7f5e2c071ed0 msgr2=0x7f5e2c0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:07.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.141+0000 7f5e309da700 1 -- 172.21.15.146:0/135541603 shutdown_connections 2024-09-13T18:27:07.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.141+0000 7f5e309da700 1 -- 172.21.15.146:0/135541603 wait complete. 2024-09-13T18:27:07.147 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:27:07.160 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-13T18:27:06.830200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596229,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":236,"num_write_kb":2445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222476,"kb_used_data":2252,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754752244,"statfs":{"total":773094113280,"available":772866297856,"internally_reserved":0,"allocated":2306048,"data_stored":2017932,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":225431099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1268,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005630"},"pg_stats":[{"pgid":"2.1f","version":"25'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.345647+0000","last_change":"2024-09-13T18:26:31.972797+0000","last_active":"2024-09-13T18:26:37.345647+0000","last_peered":"2024-09-13T18:26:37.345647+0000","last_clean":"2024-09-13T18:26:37.345647+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:26:37.345647+0000","last_undegraded":"2024-09-13T18:26:37.345647+0000","last_fullsized":"2024-09-13T18:26:37.345647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:50.511747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384385+0000","last_change":"2024-09-13T18:26:32.349901+0000","last_active":"2024-09-13T18:26:36.384385+0000","last_peered":"2024-09-13T18:26:36.384385+0000","last_clean":"2024-09-13T18:26:36.384385+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:26:36.384385+0000","last_undegraded":"2024-09-13T18:26:36.384385+0000","last_fullsized":"2024-09-13T18:26:36.384385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:24.978813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768058+0000","last_change":"2024-09-13T18:26:36.407579+0000","last_active":"2024-09-13T18:26:36.768058+0000","last_peered":"2024-09-13T18:26:36.768058+0000","last_clean":"2024-09-13T18:26:36.768058+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:26:36.768058+0000","last_undegraded":"2024-09-13T18:26:36.768058+0000","last_fullsized":"2024-09-13T18:26:36.768058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:53:56.370013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769520+0000","last_change":"2024-09-13T18:26:31.980716+0000","last_active":"2024-09-13T18:26:36.769520+0000","last_peered":"2024-09-13T18:26:36.769520+0000","last_clean":"2024-09-13T18:26:36.769520+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:26:36.769520+0000","last_undegraded":"2024-09-13T18:26:36.769520+0000","last_fullsized":"2024-09-13T18:26:36.769520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:20.020903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388711+0000","last_change":"2024-09-13T18:26:32.342690+0000","last_active":"2024-09-13T18:26:36.388711+0000","last_peered":"2024-09-13T18:26:36.388711+0000","last_clean":"2024-09-13T18:26:36.388711+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:26:36.388711+0000","last_undegraded":"2024-09-13T18:26:36.388711+0000","last_fullsized":"2024-09-13T18:26:36.388711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.212930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.398481+0000","last_change":"2024-09-13T18:26:36.396545+0000","last_active":"2024-09-13T18:26:36.398481+0000","last_peered":"2024-09-13T18:26:36.398481+0000","last_clean":"2024-09-13T18:26:36.398481+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:26:36.398481+0000","last_undegraded":"2024-09-13T18:26:36.398481+0000","last_fullsized":"2024-09-13T18:26:36.398481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:38.760215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.349299+0000","last_change":"2024-09-13T18:26:31.977823+0000","last_active":"2024-09-13T18:26:37.349299+0000","last_peered":"2024-09-13T18:26:37.349299+0000","last_clean":"2024-09-13T18:26:37.349299+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:26:37.349299+0000","last_undegraded":"2024-09-13T18:26:37.349299+0000","last_fullsized":"2024-09-13T18:26:37.349299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:46.956527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394225+0000","last_change":"2024-09-13T18:26:32.361288+0000","last_active":"2024-09-13T18:26:36.394225+0000","last_peered":"2024-09-13T18:26:36.394225+0000","last_clean":"2024-09-13T18:26:36.394225+0000","last_became_active":"2024-09-13T18:26:32.360918+0000","last_became_peered":"2024-09-13T18:26:32.360918+0000","last_unstale":"2024-09-13T18:26:36.394225+0000","last_undegraded":"2024-09-13T18:26:36.394225+0000","last_fullsized":"2024-09-13T18:26:36.394225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:30.143436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408364+0000","last_change":"2024-09-13T18:26:36.406675+0000","last_active":"2024-09-13T18:26:36.408364+0000","last_peered":"2024-09-13T18:26:36.408364+0000","last_clean":"2024-09-13T18:26:36.408364+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:26:36.408364+0000","last_undegraded":"2024-09-13T18:26:36.408364+0000","last_fullsized":"2024-09-13T18:26:36.408364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:36.896027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.347551+0000","last_change":"2024-09-13T18:26:31.979860+0000","last_active":"2024-09-13T18:26:37.347551+0000","last_peered":"2024-09-13T18:26:37.347551+0000","last_clean":"2024-09-13T18:26:37.347551+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:26:37.347551+0000","last_undegraded":"2024-09-13T18:26:37.347551+0000","last_fullsized":"2024-09-13T18:26:37.347551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:45.323631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393203+0000","last_change":"2024-09-13T18:26:32.351494+0000","last_active":"2024-09-13T18:26:36.393203+0000","last_peered":"2024-09-13T18:26:36.393203+0000","last_clean":"2024-09-13T18:26:36.393203+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:26:36.393203+0000","last_undegraded":"2024-09-13T18:26:36.393203+0000","last_fullsized":"2024-09-13T18:26:36.393203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:52.804526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400955+0000","last_change":"2024-09-13T18:26:36.398763+0000","last_active":"2024-09-13T18:26:36.400955+0000","last_peered":"2024-09-13T18:26:36.400955+0000","last_clean":"2024-09-13T18:26:36.400955+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:26:36.400955+0000","last_undegraded":"2024-09-13T18:26:36.400955+0000","last_fullsized":"2024-09-13T18:26:36.400955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:35.417438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897834+0000","last_change":"2024-09-13T18:26:32.360976+0000","last_active":"2024-09-13T18:26:36.897834+0000","last_peered":"2024-09-13T18:26:36.897834+0000","last_clean":"2024-09-13T18:26:36.897834+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:26:36.897834+0000","last_undegraded":"2024-09-13T18:26:36.897834+0000","last_fullsized":"2024-09-13T18:26:36.897834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:53:52.373515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.477712+0000","last_change":"2024-09-13T18:26:31.977891+0000","last_active":"2024-09-13T18:27:03.477712+0000","last_peered":"2024-09-13T18:27:03.477712+0000","last_clean":"2024-09-13T18:27:03.477712+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:27:03.477712+0000","last_undegraded":"2024-09-13T18:27:03.477712+0000","last_fullsized":"2024-09-13T18:27:03.477712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:01.453094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400563+0000","last_change":"2024-09-13T18:26:36.397484+0000","last_active":"2024-09-13T18:26:36.400563+0000","last_peered":"2024-09-13T18:26:36.400563+0000","last_clean":"2024-09-13T18:26:36.400563+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:26:36.400563+0000","last_undegraded":"2024-09-13T18:26:36.400563+0000","last_fullsized":"2024-09-13T18:26:36.400563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:44.686244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386958+0000","last_change":"2024-09-13T18:26:32.345564+0000","last_active":"2024-09-13T18:26:36.386958+0000","last_peered":"2024-09-13T18:26:36.386958+0000","last_clean":"2024-09-13T18:26:36.386958+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:26:36.386958+0000","last_undegraded":"2024-09-13T18:26:36.386958+0000","last_fullsized":"2024-09-13T18:26:36.386958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:54.331806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386241+0000","last_change":"2024-09-13T18:26:31.974107+0000","last_active":"2024-09-13T18:26:36.386241+0000","last_peered":"2024-09-13T18:26:36.386241+0000","last_clean":"2024-09-13T18:26:36.386241+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:26:36.386241+0000","last_undegraded":"2024-09-13T18:26:36.386241+0000","last_fullsized":"2024-09-13T18:26:36.386241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:37.547480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408253+0000","last_change":"2024-09-13T18:26:36.405986+0000","last_active":"2024-09-13T18:26:36.408253+0000","last_peered":"2024-09-13T18:26:36.408253+0000","last_clean":"2024-09-13T18:26:36.408253+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:26:36.408253+0000","last_undegraded":"2024-09-13T18:26:36.408253+0000","last_fullsized":"2024-09-13T18:26:36.408253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:30.052430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383772+0000","last_change":"2024-09-13T18:26:32.340616+0000","last_active":"2024-09-13T18:26:36.383772+0000","last_peered":"2024-09-13T18:26:36.383772+0000","last_clean":"2024-09-13T18:26:36.383772+0000","last_became_active":"2024-09-13T18:26:32.339405+0000","last_became_peered":"2024-09-13T18:26:32.339405+0000","last_unstale":"2024-09-13T18:26:36.383772+0000","last_undegraded":"2024-09-13T18:26:36.383772+0000","last_fullsized":"2024-09-13T18:26:36.383772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:26.081896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.474579+0000","last_change":"2024-09-13T18:26:31.968895+0000","last_active":"2024-09-13T18:27:03.474579+0000","last_peered":"2024-09-13T18:27:03.474579+0000","last_clean":"2024-09-13T18:27:03.474579+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:27:03.474579+0000","last_undegraded":"2024-09-13T18:27:03.474579+0000","last_fullsized":"2024-09-13T18:27:03.474579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:15.525097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767834+0000","last_change":"2024-09-13T18:26:36.406484+0000","last_active":"2024-09-13T18:26:36.767834+0000","last_peered":"2024-09-13T18:26:36.767834+0000","last_clean":"2024-09-13T18:26:36.767834+0000","last_became_active":"2024-09-13T18:26:36.406161+0000","last_became_peered":"2024-09-13T18:26:36.406161+0000","last_unstale":"2024-09-13T18:26:36.767834+0000","last_undegraded":"2024-09-13T18:26:36.767834+0000","last_fullsized":"2024-09-13T18:26:36.767834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:53.330919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391435+0000","last_change":"2024-09-13T18:26:32.360406+0000","last_active":"2024-09-13T18:26:36.391435+0000","last_peered":"2024-09-13T18:26:36.391435+0000","last_clean":"2024-09-13T18:26:36.391435+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:26:36.391435+0000","last_undegraded":"2024-09-13T18:26:36.391435+0000","last_fullsized":"2024-09-13T18:26:36.391435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:37.417707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.476272+0000","last_change":"2024-09-13T18:26:31.976872+0000","last_active":"2024-09-13T18:27:03.476272+0000","last_peered":"2024-09-13T18:27:03.476272+0000","last_clean":"2024-09-13T18:27:03.476272+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:27:03.476272+0000","last_undegraded":"2024-09-13T18:27:03.476272+0000","last_fullsized":"2024-09-13T18:27:03.476272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:26.163029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408244+0000","last_change":"2024-09-13T18:26:36.405722+0000","last_active":"2024-09-13T18:26:36.408244+0000","last_peered":"2024-09-13T18:26:36.408244+0000","last_clean":"2024-09-13T18:26:36.408244+0000","last_became_active":"2024-09-13T18:26:36.405310+0000","last_became_peered":"2024-09-13T18:26:36.405310+0000","last_unstale":"2024-09-13T18:26:36.408244+0000","last_undegraded":"2024-09-13T18:26:36.408244+0000","last_fullsized":"2024-09-13T18:26:36.408244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:09.815655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.389080+0000","last_change":"2024-09-13T18:26:32.343915+0000","last_active":"2024-09-13T18:26:36.389080+0000","last_peered":"2024-09-13T18:26:36.389080+0000","last_clean":"2024-09-13T18:26:36.389080+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:26:36.389080+0000","last_undegraded":"2024-09-13T18:26:36.389080+0000","last_fullsized":"2024-09-13T18:26:36.389080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:19.357243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.349375+0000","last_change":"2024-09-13T18:26:31.979944+0000","last_active":"2024-09-13T18:26:37.349375+0000","last_peered":"2024-09-13T18:26:37.349375+0000","last_clean":"2024-09-13T18:26:37.349375+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:26:37.349375+0000","last_undegraded":"2024-09-13T18:26:37.349375+0000","last_fullsized":"2024-09-13T18:26:37.349375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:21.721921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.411958+0000","last_change":"2024-09-13T18:26:36.410028+0000","last_active":"2024-09-13T18:26:36.411958+0000","last_peered":"2024-09-13T18:26:36.411958+0000","last_clean":"2024-09-13T18:26:36.411958+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:26:36.411958+0000","last_undegraded":"2024-09-13T18:26:36.411958+0000","last_fullsized":"2024-09-13T18:26:36.411958+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:32.264341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387875+0000","last_change":"2024-09-13T18:26:32.358980+0000","last_active":"2024-09-13T18:26:36.387875+0000","last_peered":"2024-09-13T18:26:36.387875+0000","last_clean":"2024-09-13T18:26:36.387875+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:26:36.387875+0000","last_undegraded":"2024-09-13T18:26:36.387875+0000","last_fullsized":"2024-09-13T18:26:36.387875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:08.748139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769462+0000","last_change":"2024-09-13T18:26:31.968972+0000","last_active":"2024-09-13T18:26:36.769462+0000","last_peered":"2024-09-13T18:26:36.769462+0000","last_clean":"2024-09-13T18:26:36.769462+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:26:36.769462+0000","last_undegraded":"2024-09-13T18:26:36.769462+0000","last_fullsized":"2024-09-13T18:26:36.769462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:56.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768011+0000","last_change":"2024-09-13T18:26:36.407034+0000","last_active":"2024-09-13T18:26:36.768011+0000","last_peered":"2024-09-13T18:26:36.768011+0000","last_clean":"2024-09-13T18:26:36.768011+0000","last_became_active":"2024-09-13T18:26:36.406473+0000","last_became_peered":"2024-09-13T18:26:36.406473+0000","last_unstale":"2024-09-13T18:26:36.768011+0000","last_undegraded":"2024-09-13T18:26:36.768011+0000","last_fullsized":"2024-09-13T18:26:36.768011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:53.815901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386420+0000","last_change":"2024-09-13T18:26:32.342823+0000","last_active":"2024-09-13T18:26:36.386420+0000","last_peered":"2024-09-13T18:26:36.386420+0000","last_clean":"2024-09-13T18:26:36.386420+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:26:36.386420+0000","last_undegraded":"2024-09-13T18:26:36.386420+0000","last_fullsized":"2024-09-13T18:26:36.386420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:28.051252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.349179+0000","last_change":"2024-09-13T18:26:31.978976+0000","last_active":"2024-09-13T18:26:37.349179+0000","last_peered":"2024-09-13T18:26:37.349179+0000","last_clean":"2024-09-13T18:26:37.349179+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:26:37.349179+0000","last_undegraded":"2024-09-13T18:26:37.349179+0000","last_fullsized":"2024-09-13T18:26:37.349179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:11.279557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103455+0000","last_change":"2024-09-13T18:26:36.408844+0000","last_active":"2024-09-13T18:26:37.103455+0000","last_peered":"2024-09-13T18:26:37.103455+0000","last_clean":"2024-09-13T18:26:37.103455+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:26:37.103455+0000","last_undegraded":"2024-09-13T18:26:37.103455+0000","last_fullsized":"2024-09-13T18:26:37.103455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:06.969342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393043+0000","last_change":"2024-09-13T18:26:32.355038+0000","last_active":"2024-09-13T18:26:36.393043+0000","last_peered":"2024-09-13T18:26:36.393043+0000","last_clean":"2024-09-13T18:26:36.393043+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:26:36.393043+0000","last_undegraded":"2024-09-13T18:26:36.393043+0000","last_fullsized":"2024-09-13T18:26:36.393043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.159231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386562+0000","last_change":"2024-09-13T18:26:31.972931+0000","last_active":"2024-09-13T18:26:36.386562+0000","last_peered":"2024-09-13T18:26:36.386562+0000","last_clean":"2024-09-13T18:26:36.386562+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:26:36.386562+0000","last_undegraded":"2024-09-13T18:26:36.386562+0000","last_fullsized":"2024-09-13T18:26:36.386562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:05.646924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766310+0000","last_change":"2024-09-13T18:26:36.398876+0000","last_active":"2024-09-13T18:26:36.766310+0000","last_peered":"2024-09-13T18:26:36.766310+0000","last_clean":"2024-09-13T18:26:36.766310+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:26:36.766310+0000","last_undegraded":"2024-09-13T18:26:36.766310+0000","last_fullsized":"2024-09-13T18:26:36.766310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:56.918239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392625+0000","last_change":"2024-09-13T18:26:32.353111+0000","last_active":"2024-09-13T18:26:36.392625+0000","last_peered":"2024-09-13T18:26:36.392625+0000","last_clean":"2024-09-13T18:26:36.392625+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:26:36.392625+0000","last_undegraded":"2024-09-13T18:26:36.392625+0000","last_fullsized":"2024-09-13T18:26:36.392625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:43.927936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391864+0000","last_change":"2024-09-13T18:26:31.976075+0000","last_active":"2024-09-13T18:26:36.391864+0000","last_peered":"2024-09-13T18:26:36.391864+0000","last_clean":"2024-09-13T18:26:36.391864+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:26:36.391864+0000","last_undegraded":"2024-09-13T18:26:36.391864+0000","last_fullsized":"2024-09-13T18:26:36.391864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:03.693518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766877+0000","last_change":"2024-09-13T18:26:36.401777+0000","last_active":"2024-09-13T18:26:36.766877+0000","last_peered":"2024-09-13T18:26:36.766877+0000","last_clean":"2024-09-13T18:26:36.766877+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:26:36.766877+0000","last_undegraded":"2024-09-13T18:26:36.766877+0000","last_fullsized":"2024-09-13T18:26:36.766877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:43.440284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384991+0000","last_change":"2024-09-13T18:26:32.343791+0000","last_active":"2024-09-13T18:26:36.384991+0000","last_peered":"2024-09-13T18:26:36.384991+0000","last_clean":"2024-09-13T18:26:36.384991+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:26:36.384991+0000","last_undegraded":"2024-09-13T18:26:36.384991+0000","last_fullsized":"2024-09-13T18:26:36.384991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:36.296571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343727+0000","last_change":"2024-09-13T18:26:31.970786+0000","last_active":"2024-09-13T18:26:37.343727+0000","last_peered":"2024-09-13T18:26:37.343727+0000","last_clean":"2024-09-13T18:26:37.343727+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:26:37.343727+0000","last_undegraded":"2024-09-13T18:26:37.343727+0000","last_fullsized":"2024-09-13T18:26:37.343727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:54.584009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767929+0000","last_change":"2024-09-13T18:26:36.401529+0000","last_active":"2024-09-13T18:26:36.767929+0000","last_peered":"2024-09-13T18:26:36.767929+0000","last_clean":"2024-09-13T18:26:36.767929+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:26:36.767929+0000","last_undegraded":"2024-09-13T18:26:36.767929+0000","last_fullsized":"2024-09-13T18:26:36.767929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:47.318347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.382662+0000","last_change":"2024-09-13T18:26:32.350470+0000","last_active":"2024-09-13T18:26:36.382662+0000","last_peered":"2024-09-13T18:26:36.382662+0000","last_clean":"2024-09-13T18:26:36.382662+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:26:36.382662+0000","last_undegraded":"2024-09-13T18:26:36.382662+0000","last_fullsized":"2024-09-13T18:26:36.382662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:55.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.347577+0000","last_change":"2024-09-13T18:26:31.972587+0000","last_active":"2024-09-13T18:26:37.347577+0000","last_peered":"2024-09-13T18:26:37.347577+0000","last_clean":"2024-09-13T18:26:37.347577+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:26:37.347577+0000","last_undegraded":"2024-09-13T18:26:37.347577+0000","last_fullsized":"2024-09-13T18:26:37.347577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:27.547763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103735+0000","last_change":"2024-09-13T18:26:36.405086+0000","last_active":"2024-09-13T18:26:37.103735+0000","last_peered":"2024-09-13T18:26:37.103735+0000","last_clean":"2024-09-13T18:26:37.103735+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:26:37.103735+0000","last_undegraded":"2024-09-13T18:26:37.103735+0000","last_fullsized":"2024-09-13T18:26:37.103735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:02:51.694634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388036+0000","last_change":"2024-09-13T18:26:32.350510+0000","last_active":"2024-09-13T18:26:36.388036+0000","last_peered":"2024-09-13T18:26:36.388036+0000","last_clean":"2024-09-13T18:26:36.388036+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:26:36.388036+0000","last_undegraded":"2024-09-13T18:26:36.388036+0000","last_fullsized":"2024-09-13T18:26:36.388036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:40.850255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.479641+0000","last_change":"2024-09-13T18:26:31.978355+0000","last_active":"2024-09-13T18:27:03.479641+0000","last_peered":"2024-09-13T18:27:03.479641+0000","last_clean":"2024-09-13T18:27:03.479641+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:27:03.479641+0000","last_undegraded":"2024-09-13T18:27:03.479641+0000","last_fullsized":"2024-09-13T18:27:03.479641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:12.946518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404778+0000","last_change":"2024-09-13T18:26:26.931403+0000","last_active":"2024-09-13T18:26:36.404778+0000","last_peered":"2024-09-13T18:26:36.404778+0000","last_clean":"2024-09-13T18:26:36.404778+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:26:36.404778+0000","last_undegraded":"2024-09-13T18:26:36.404778+0000","last_fullsized":"2024-09-13T18:26:36.404778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_clean_scrub_stamp":"2024-09-13T18:26:24.880028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:30.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897926+0000","last_change":"2024-09-13T18:26:36.409827+0000","last_active":"2024-09-13T18:26:36.897926+0000","last_peered":"2024-09-13T18:26:36.897926+0000","last_clean":"2024-09-13T18:26:36.897926+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:26:36.897926+0000","last_undegraded":"2024-09-13T18:26:36.897926+0000","last_fullsized":"2024-09-13T18:26:36.897926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:50.482400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405746+0000","last_change":"2024-09-13T18:26:32.356850+0000","last_active":"2024-09-13T18:26:36.405746+0000","last_peered":"2024-09-13T18:26:36.405746+0000","last_clean":"2024-09-13T18:26:36.405746+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:26:36.405746+0000","last_undegraded":"2024-09-13T18:26:36.405746+0000","last_fullsized":"2024-09-13T18:26:36.405746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:38.710828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.376787+0000","last_change":"2024-09-13T18:26:31.975942+0000","last_active":"2024-09-13T18:26:36.376787+0000","last_peered":"2024-09-13T18:26:36.376787+0000","last_clean":"2024-09-13T18:26:36.376787+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:26:36.376787+0000","last_undegraded":"2024-09-13T18:26:36.376787+0000","last_fullsized":"2024-09-13T18:26:36.376787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:05.840582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897805+0000","last_change":"2024-09-13T18:26:36.409802+0000","last_active":"2024-09-13T18:26:36.897805+0000","last_peered":"2024-09-13T18:26:36.897805+0000","last_clean":"2024-09-13T18:26:36.897805+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:26:36.897805+0000","last_undegraded":"2024-09-13T18:26:36.897805+0000","last_fullsized":"2024-09-13T18:26:36.897805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:38.178151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394309+0000","last_change":"2024-09-13T18:26:32.355181+0000","last_active":"2024-09-13T18:26:36.394309+0000","last_peered":"2024-09-13T18:26:36.394309+0000","last_clean":"2024-09-13T18:26:36.394309+0000","last_became_active":"2024-09-13T18:26:32.353043+0000","last_became_peered":"2024-09-13T18:26:32.353043+0000","last_unstale":"2024-09-13T18:26:36.394309+0000","last_undegraded":"2024-09-13T18:26:36.394309+0000","last_fullsized":"2024-09-13T18:26:36.394309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:04:17.000915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767488+0000","last_change":"2024-09-13T18:26:31.966618+0000","last_active":"2024-09-13T18:26:36.767488+0000","last_peered":"2024-09-13T18:26:36.767488+0000","last_clean":"2024-09-13T18:26:36.767488+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:26:36.767488+0000","last_undegraded":"2024-09-13T18:26:36.767488+0000","last_fullsized":"2024-09-13T18:26:36.767488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:54:26.422611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103645+0000","last_change":"2024-09-13T18:26:36.394344+0000","last_active":"2024-09-13T18:26:37.103645+0000","last_peered":"2024-09-13T18:26:37.103645+0000","last_clean":"2024-09-13T18:26:37.103645+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:26:37.103645+0000","last_undegraded":"2024-09-13T18:26:37.103645+0000","last_fullsized":"2024-09-13T18:26:37.103645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:18.425816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405784+0000","last_change":"2024-09-13T18:26:32.365655+0000","last_active":"2024-09-13T18:26:36.405784+0000","last_peered":"2024-09-13T18:26:36.405784+0000","last_clean":"2024-09-13T18:26:36.405784+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:26:36.405784+0000","last_undegraded":"2024-09-13T18:26:36.405784+0000","last_fullsized":"2024-09-13T18:26:36.405784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:40.288128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343912+0000","last_change":"2024-09-13T18:26:31.970652+0000","last_active":"2024-09-13T18:26:37.343912+0000","last_peered":"2024-09-13T18:26:37.343912+0000","last_clean":"2024-09-13T18:26:37.343912+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:26:37.343912+0000","last_undegraded":"2024-09-13T18:26:37.343912+0000","last_fullsized":"2024-09-13T18:26:37.343912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:11.169661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.770631+0000","last_change":"2024-09-13T18:26:36.410280+0000","last_active":"2024-09-13T18:26:36.770631+0000","last_peered":"2024-09-13T18:26:36.770631+0000","last_clean":"2024-09-13T18:26:36.770631+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:26:36.770631+0000","last_undegraded":"2024-09-13T18:26:36.770631+0000","last_fullsized":"2024-09-13T18:26:36.770631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:01.395290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405642+0000","last_change":"2024-09-13T18:26:32.361575+0000","last_active":"2024-09-13T18:26:36.405642+0000","last_peered":"2024-09-13T18:26:36.405642+0000","last_clean":"2024-09-13T18:26:36.405642+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:26:36.405642+0000","last_undegraded":"2024-09-13T18:26:36.405642+0000","last_fullsized":"2024-09-13T18:26:36.405642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:08.818877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.239195+0000","last_change":"2024-09-13T18:26:31.977029+0000","last_active":"2024-09-13T18:26:37.239195+0000","last_peered":"2024-09-13T18:26:37.239195+0000","last_clean":"2024-09-13T18:26:37.239195+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:26:37.239195+0000","last_undegraded":"2024-09-13T18:26:37.239195+0000","last_fullsized":"2024-09-13T18:26:37.239195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:13.701689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407565+0000","last_change":"2024-09-13T18:26:36.403479+0000","last_active":"2024-09-13T18:26:36.407565+0000","last_peered":"2024-09-13T18:26:36.407565+0000","last_clean":"2024-09-13T18:26:36.407565+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:26:36.407565+0000","last_undegraded":"2024-09-13T18:26:36.407565+0000","last_fullsized":"2024-09-13T18:26:36.407565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:51.095852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404976+0000","last_change":"2024-09-13T18:26:32.365143+0000","last_active":"2024-09-13T18:26:36.404976+0000","last_peered":"2024-09-13T18:26:36.404976+0000","last_clean":"2024-09-13T18:26:36.404976+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:26:36.404976+0000","last_undegraded":"2024-09-13T18:26:36.404976+0000","last_fullsized":"2024-09-13T18:26:36.404976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:46.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'10","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.466996+0000","last_change":"2024-09-13T18:26:31.965802+0000","last_active":"2024-09-13T18:27:03.466996+0000","last_peered":"2024-09-13T18:27:03.466996+0000","last_clean":"2024-09-13T18:27:03.466996+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:27:03.466996+0000","last_undegraded":"2024-09-13T18:27:03.466996+0000","last_fullsized":"2024-09-13T18:27:03.466996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:12.759181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103768+0000","last_change":"2024-09-13T18:26:36.399235+0000","last_active":"2024-09-13T18:26:37.103768+0000","last_peered":"2024-09-13T18:26:37.103768+0000","last_clean":"2024-09-13T18:26:37.103768+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:26:37.103768+0000","last_undegraded":"2024-09-13T18:26:37.103768+0000","last_fullsized":"2024-09-13T18:26:37.103768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:13:54.711083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897760+0000","last_change":"2024-09-13T18:26:32.357403+0000","last_active":"2024-09-13T18:26:36.897760+0000","last_peered":"2024-09-13T18:26:36.897760+0000","last_clean":"2024-09-13T18:26:36.897760+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:26:36.897760+0000","last_undegraded":"2024-09-13T18:26:36.897760+0000","last_fullsized":"2024-09-13T18:26:36.897760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:12.600397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383205+0000","last_change":"2024-09-13T18:26:31.976148+0000","last_active":"2024-09-13T18:26:36.383205+0000","last_peered":"2024-09-13T18:26:36.383205+0000","last_clean":"2024-09-13T18:26:36.383205+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:26:36.383205+0000","last_undegraded":"2024-09-13T18:26:36.383205+0000","last_fullsized":"2024-09-13T18:26:36.383205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:38.023636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769582+0000","last_change":"2024-09-13T18:26:36.406177+0000","last_active":"2024-09-13T18:26:36.769582+0000","last_peered":"2024-09-13T18:26:36.769582+0000","last_clean":"2024-09-13T18:26:36.769582+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:26:36.769582+0000","last_undegraded":"2024-09-13T18:26:36.769582+0000","last_fullsized":"2024-09-13T18:26:36.769582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:00.309995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405459+0000","last_change":"2024-09-13T18:26:32.361726+0000","last_active":"2024-09-13T18:26:36.405459+0000","last_peered":"2024-09-13T18:26:36.405459+0000","last_clean":"2024-09-13T18:26:36.405459+0000","last_became_active":"2024-09-13T18:26:32.361432+0000","last_became_peered":"2024-09-13T18:26:32.361432+0000","last_unstale":"2024-09-13T18:26:36.405459+0000","last_undegraded":"2024-09-13T18:26:36.405459+0000","last_fullsized":"2024-09-13T18:26:36.405459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:42.021515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.480098+0000","last_change":"2024-09-13T18:26:31.982114+0000","last_active":"2024-09-13T18:27:03.480098+0000","last_peered":"2024-09-13T18:27:03.480098+0000","last_clean":"2024-09-13T18:27:03.480098+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:27:03.480098+0000","last_undegraded":"2024-09-13T18:27:03.480098+0000","last_fullsized":"2024-09-13T18:27:03.480098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.266528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769049+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:26:36.769049+0000","last_peered":"2024-09-13T18:26:36.769049+0000","last_clean":"2024-09-13T18:26:36.769049+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:26:36.769049+0000","last_undegraded":"2024-09-13T18:26:36.769049+0000","last_fullsized":"2024-09-13T18:26:36.769049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:58.853971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388651+0000","last_change":"2024-09-13T18:26:32.351145+0000","last_active":"2024-09-13T18:26:36.388651+0000","last_peered":"2024-09-13T18:26:36.388651+0000","last_clean":"2024-09-13T18:26:36.388651+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:26:36.388651+0000","last_undegraded":"2024-09-13T18:26:36.388651+0000","last_fullsized":"2024-09-13T18:26:36.388651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:57.002435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.471478+0000","last_change":"2024-09-13T18:26:31.961202+0000","last_active":"2024-09-13T18:27:03.471478+0000","last_peered":"2024-09-13T18:27:03.471478+0000","last_clean":"2024-09-13T18:27:03.471478+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:27:03.471478+0000","last_undegraded":"2024-09-13T18:27:03.471478+0000","last_fullsized":"2024-09-13T18:27:03.471478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:27.340127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.772209+0000","last_change":"2024-09-13T18:26:36.410014+0000","last_active":"2024-09-13T18:26:36.772209+0000","last_peered":"2024-09-13T18:26:36.772209+0000","last_clean":"2024-09-13T18:26:36.772209+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:26:36.772209+0000","last_undegraded":"2024-09-13T18:26:36.772209+0000","last_fullsized":"2024-09-13T18:26:36.772209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:24.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405669+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:26:36.405669+0000","last_peered":"2024-09-13T18:26:36.405669+0000","last_clean":"2024-09-13T18:26:36.405669+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:26:36.405669+0000","last_undegraded":"2024-09-13T18:26:36.405669+0000","last_fullsized":"2024-09-13T18:26:36.405669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:40.102815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:45.381784+0000","last_change":"2024-09-13T18:26:31.974110+0000","last_active":"2024-09-13T18:26:45.381784+0000","last_peered":"2024-09-13T18:26:45.381784+0000","last_clean":"2024-09-13T18:26:45.381784+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:26:45.381784+0000","last_undegraded":"2024-09-13T18:26:45.381784+0000","last_fullsized":"2024-09-13T18:26:45.381784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:56.058561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766576+0000","last_change":"2024-09-13T18:26:36.394080+0000","last_active":"2024-09-13T18:26:36.766576+0000","last_peered":"2024-09-13T18:26:36.766576+0000","last_clean":"2024-09-13T18:26:36.766576+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:26:36.766576+0000","last_undegraded":"2024-09-13T18:26:36.766576+0000","last_fullsized":"2024-09-13T18:26:36.766576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:47.121814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392195+0000","last_change":"2024-09-13T18:26:32.362873+0000","last_active":"2024-09-13T18:26:36.392195+0000","last_peered":"2024-09-13T18:26:36.392195+0000","last_clean":"2024-09-13T18:26:36.392195+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:26:36.392195+0000","last_undegraded":"2024-09-13T18:26:36.392195+0000","last_fullsized":"2024-09-13T18:26:36.392195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:37.598189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.344708+0000","last_change":"2024-09-13T18:26:31.976401+0000","last_active":"2024-09-13T18:26:37.344708+0000","last_peered":"2024-09-13T18:26:37.344708+0000","last_clean":"2024-09-13T18:26:37.344708+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:26:37.344708+0000","last_undegraded":"2024-09-13T18:26:37.344708+0000","last_fullsized":"2024-09-13T18:26:37.344708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:26.434292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408317+0000","last_change":"2024-09-13T18:26:36.406228+0000","last_active":"2024-09-13T18:26:36.408317+0000","last_peered":"2024-09-13T18:26:36.408317+0000","last_clean":"2024-09-13T18:26:36.408317+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:26:36.408317+0000","last_undegraded":"2024-09-13T18:26:36.408317+0000","last_fullsized":"2024-09-13T18:26:36.408317+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:49.198397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388733+0000","last_change":"2024-09-13T18:26:32.350841+0000","last_active":"2024-09-13T18:26:36.388733+0000","last_peered":"2024-09-13T18:26:36.388733+0000","last_clean":"2024-09-13T18:26:36.388733+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:26:36.388733+0000","last_undegraded":"2024-09-13T18:26:36.388733+0000","last_fullsized":"2024-09-13T18:26:36.388733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:05.137882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.476898+0000","last_change":"2024-09-13T18:26:31.978214+0000","last_active":"2024-09-13T18:27:03.476898+0000","last_peered":"2024-09-13T18:27:03.476898+0000","last_clean":"2024-09-13T18:27:03.476898+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:27:03.476898+0000","last_undegraded":"2024-09-13T18:27:03.476898+0000","last_fullsized":"2024-09-13T18:27:03.476898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:17.381375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768498+0000","last_change":"2024-09-13T18:26:36.406090+0000","last_active":"2024-09-13T18:26:36.768498+0000","last_peered":"2024-09-13T18:26:36.768498+0000","last_clean":"2024-09-13T18:26:36.768498+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:26:36.768498+0000","last_undegraded":"2024-09-13T18:26:36.768498+0000","last_fullsized":"2024-09-13T18:26:36.768498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:09.865147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391759+0000","last_change":"2024-09-13T18:26:32.357701+0000","last_active":"2024-09-13T18:26:36.391759+0000","last_peered":"2024-09-13T18:26:36.391759+0000","last_clean":"2024-09-13T18:26:36.391759+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:26:36.391759+0000","last_undegraded":"2024-09-13T18:26:36.391759+0000","last_fullsized":"2024-09-13T18:26:36.391759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:47.587683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384903+0000","last_change":"2024-09-13T18:26:31.980133+0000","last_active":"2024-09-13T18:26:36.384903+0000","last_peered":"2024-09-13T18:26:36.384903+0000","last_clean":"2024-09-13T18:26:36.384903+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:26:36.384903+0000","last_undegraded":"2024-09-13T18:26:36.384903+0000","last_fullsized":"2024-09-13T18:26:36.384903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:05.354335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768120+0000","last_change":"2024-09-13T18:26:36.406553+0000","last_active":"2024-09-13T18:26:36.768120+0000","last_peered":"2024-09-13T18:26:36.768120+0000","last_clean":"2024-09-13T18:26:36.768120+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:26:36.768120+0000","last_undegraded":"2024-09-13T18:26:36.768120+0000","last_fullsized":"2024-09-13T18:26:36.768120+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:02.980826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103481+0000","last_change":"2024-09-13T18:26:36.406986+0000","last_active":"2024-09-13T18:26:37.103481+0000","last_peered":"2024-09-13T18:26:37.103481+0000","last_clean":"2024-09-13T18:26:37.103481+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:26:37.103481+0000","last_undegraded":"2024-09-13T18:26:37.103481+0000","last_fullsized":"2024-09-13T18:26:37.103481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:53.984231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384974+0000","last_change":"2024-09-13T18:26:32.349081+0000","last_active":"2024-09-13T18:26:36.384974+0000","last_peered":"2024-09-13T18:26:36.384974+0000","last_clean":"2024-09-13T18:26:36.384974+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:26:36.384974+0000","last_undegraded":"2024-09-13T18:26:36.384974+0000","last_fullsized":"2024-09-13T18:26:36.384974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:47.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.771163+0000","last_change":"2024-09-13T18:26:31.968944+0000","last_active":"2024-09-13T18:26:36.771163+0000","last_peered":"2024-09-13T18:26:36.771163+0000","last_clean":"2024-09-13T18:26:36.771163+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:26:36.771163+0000","last_undegraded":"2024-09-13T18:26:36.771163+0000","last_fullsized":"2024-09-13T18:26:36.771163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:29.363312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.898021+0000","last_change":"2024-09-13T18:26:36.396418+0000","last_active":"2024-09-13T18:26:36.898021+0000","last_peered":"2024-09-13T18:26:36.898021+0000","last_clean":"2024-09-13T18:26:36.898021+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:26:36.898021+0000","last_undegraded":"2024-09-13T18:26:36.898021+0000","last_fullsized":"2024-09-13T18:26:36.898021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:03.252773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383753+0000","last_change":"2024-09-13T18:26:32.351369+0000","last_active":"2024-09-13T18:26:36.383753+0000","last_peered":"2024-09-13T18:26:36.383753+0000","last_clean":"2024-09-13T18:26:36.383753+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:26:36.383753+0000","last_undegraded":"2024-09-13T18:26:36.383753+0000","last_fullsized":"2024-09-13T18:26:36.383753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:10.892347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387713+0000","last_change":"2024-09-13T18:26:31.972048+0000","last_active":"2024-09-13T18:26:36.387713+0000","last_peered":"2024-09-13T18:26:36.387713+0000","last_clean":"2024-09-13T18:26:36.387713+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:26:36.387713+0000","last_undegraded":"2024-09-13T18:26:36.387713+0000","last_fullsized":"2024-09-13T18:26:36.387713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:28.430119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343265+0000","last_change":"2024-09-13T18:26:31.975310+0000","last_active":"2024-09-13T18:26:37.343265+0000","last_peered":"2024-09-13T18:26:37.343265+0000","last_clean":"2024-09-13T18:26:37.343265+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:26:37.343265+0000","last_undegraded":"2024-09-13T18:26:37.343265+0000","last_fullsized":"2024-09-13T18:26:37.343265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:52.915310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388817+0000","last_change":"2024-09-13T18:26:32.345825+0000","last_active":"2024-09-13T18:26:36.388817+0000","last_peered":"2024-09-13T18:26:36.388817+0000","last_clean":"2024-09-13T18:26:36.388817+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:26:36.388817+0000","last_undegraded":"2024-09-13T18:26:36.388817+0000","last_fullsized":"2024-09-13T18:26:36.388817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:25.200489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.395253+0000","last_change":"2024-09-13T18:26:36.391520+0000","last_active":"2024-09-13T18:26:36.395253+0000","last_peered":"2024-09-13T18:26:36.395253+0000","last_clean":"2024-09-13T18:26:36.395253+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:26:36.395253+0000","last_undegraded":"2024-09-13T18:26:36.395253+0000","last_fullsized":"2024-09-13T18:26:36.395253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:42.638713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.476885+0000","last_change":"2024-09-13T18:26:31.976603+0000","last_active":"2024-09-13T18:27:03.476885+0000","last_peered":"2024-09-13T18:27:03.476885+0000","last_clean":"2024-09-13T18:27:03.476885+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:27:03.476885+0000","last_undegraded":"2024-09-13T18:27:03.476885+0000","last_fullsized":"2024-09-13T18:27:03.476885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:03.965376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387960+0000","last_change":"2024-09-13T18:26:32.352982+0000","last_active":"2024-09-13T18:26:36.387960+0000","last_peered":"2024-09-13T18:26:36.387960+0000","last_clean":"2024-09-13T18:26:36.387960+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:26:36.387960+0000","last_undegraded":"2024-09-13T18:26:36.387960+0000","last_fullsized":"2024-09-13T18:26:36.387960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:55.670360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769131+0000","last_change":"2024-09-13T18:26:36.403817+0000","last_active":"2024-09-13T18:26:36.769131+0000","last_peered":"2024-09-13T18:26:36.769131+0000","last_clean":"2024-09-13T18:26:36.769131+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:26:36.769131+0000","last_undegraded":"2024-09-13T18:26:36.769131+0000","last_fullsized":"2024-09-13T18:26:36.769131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:34.892901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897728+0000","last_change":"2024-09-13T18:26:32.357856+0000","last_active":"2024-09-13T18:26:36.897728+0000","last_peered":"2024-09-13T18:26:36.897728+0000","last_clean":"2024-09-13T18:26:36.897728+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:26:36.897728+0000","last_undegraded":"2024-09-13T18:26:36.897728+0000","last_fullsized":"2024-09-13T18:26:36.897728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:26.316214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.375208+0000","last_change":"2024-09-13T18:26:32.350966+0000","last_active":"2024-09-13T18:26:36.375208+0000","last_peered":"2024-09-13T18:26:36.375208+0000","last_clean":"2024-09-13T18:26:36.375208+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:26:36.375208+0000","last_undegraded":"2024-09-13T18:26:36.375208+0000","last_fullsized":"2024-09-13T18:26:36.375208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:52.632112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391246+0000","last_change":"2024-09-13T18:26:32.355806+0000","last_active":"2024-09-13T18:26:36.391246+0000","last_peered":"2024-09-13T18:26:36.391246+0000","last_clean":"2024-09-13T18:26:36.391246+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:26:36.391246+0000","last_undegraded":"2024-09-13T18:26:36.391246+0000","last_fullsized":"2024-09-13T18:26:36.391246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.988231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388740+0000","last_change":"2024-09-13T18:26:32.340965+0000","last_active":"2024-09-13T18:26:36.388740+0000","last_peered":"2024-09-13T18:26:36.388740+0000","last_clean":"2024-09-13T18:26:36.388740+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:26:36.388740+0000","last_undegraded":"2024-09-13T18:26:36.388740+0000","last_fullsized":"2024-09-13T18:26:36.388740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:01:04.716919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384429+0000","last_change":"2024-09-13T18:26:32.349625+0000","last_active":"2024-09-13T18:26:36.384429+0000","last_peered":"2024-09-13T18:26:36.384429+0000","last_clean":"2024-09-13T18:26:36.384429+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:26:36.384429+0000","last_undegraded":"2024-09-13T18:26:36.384429+0000","last_fullsized":"2024-09-13T18:26:36.384429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.519754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386397+0000","last_change":"2024-09-13T18:26:32.349170+0000","last_active":"2024-09-13T18:26:36.386397+0000","last_peered":"2024-09-13T18:26:36.386397+0000","last_clean":"2024-09-13T18:26:36.386397+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:26:36.386397+0000","last_undegraded":"2024-09-13T18:26:36.386397+0000","last_fullsized":"2024-09-13T18:26:36.386397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:43.942732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897998+0000","last_change":"2024-09-13T18:26:32.364786+0000","last_active":"2024-09-13T18:26:36.897998+0000","last_peered":"2024-09-13T18:26:36.897998+0000","last_clean":"2024-09-13T18:26:36.897998+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:26:36.897998+0000","last_undegraded":"2024-09-13T18:26:36.897998+0000","last_fullsized":"2024-09-13T18:26:36.897998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:22.949875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387927+0000","last_change":"2024-09-13T18:26:32.347541+0000","last_active":"2024-09-13T18:26:36.387927+0000","last_peered":"2024-09-13T18:26:36.387927+0000","last_clean":"2024-09-13T18:26:36.387927+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:26:36.387927+0000","last_undegraded":"2024-09-13T18:26:36.387927+0000","last_fullsized":"2024-09-13T18:26:36.387927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:58.144128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390892+0000","last_change":"2024-09-13T18:26:32.353359+0000","last_active":"2024-09-13T18:26:36.390892+0000","last_peered":"2024-09-13T18:26:36.390892+0000","last_clean":"2024-09-13T18:26:36.390892+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:26:36.390892+0000","last_undegraded":"2024-09-13T18:26:36.390892+0000","last_fullsized":"2024-09-13T18:26:36.390892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:09.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.380537+0000","last_change":"2024-09-13T18:26:32.349566+0000","last_active":"2024-09-13T18:26:36.380537+0000","last_peered":"2024-09-13T18:26:36.380537+0000","last_clean":"2024-09-13T18:26:36.380537+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:26:36.380537+0000","last_undegraded":"2024-09-13T18:26:36.380537+0000","last_fullsized":"2024-09-13T18:26:36.380537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:22.341899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391168+0000","last_change":"2024-09-13T18:26:32.358640+0000","last_active":"2024-09-13T18:26:36.391168+0000","last_peered":"2024-09-13T18:26:36.391168+0000","last_clean":"2024-09-13T18:26:36.391168+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:26:36.391168+0000","last_undegraded":"2024-09-13T18:26:36.391168+0000","last_fullsized":"2024-09-13T18:26:36.391168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:40.174754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388633+0000","last_change":"2024-09-13T18:26:32.353675+0000","last_active":"2024-09-13T18:26:36.388633+0000","last_peered":"2024-09-13T18:26:36.388633+0000","last_clean":"2024-09-13T18:26:36.388633+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:26:36.388633+0000","last_undegraded":"2024-09-13T18:26:36.388633+0000","last_fullsized":"2024-09-13T18:26:36.388633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:58.501843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897877+0000","last_change":"2024-09-13T18:26:32.353223+0000","last_active":"2024-09-13T18:26:36.897877+0000","last_peered":"2024-09-13T18:26:36.897877+0000","last_clean":"2024-09-13T18:26:36.897877+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:26:36.897877+0000","last_undegraded":"2024-09-13T18:26:36.897877+0000","last_fullsized":"2024-09-13T18:26:36.897877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:45.898213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407736+0000","last_change":"2024-09-13T18:26:32.357354+0000","last_active":"2024-09-13T18:26:36.407736+0000","last_peered":"2024-09-13T18:26:36.407736+0000","last_clean":"2024-09-13T18:26:36.407736+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:26:36.407736+0000","last_undegraded":"2024-09-13T18:26:36.407736+0000","last_fullsized":"2024-09-13T18:26:36.407736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:19.185008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.376351+0000","last_change":"2024-09-13T18:26:32.348832+0000","last_active":"2024-09-13T18:26:36.376351+0000","last_peered":"2024-09-13T18:26:36.376351+0000","last_clean":"2024-09-13T18:26:36.376351+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:26:36.376351+0000","last_undegraded":"2024-09-13T18:26:36.376351+0000","last_fullsized":"2024-09-13T18:26:36.376351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:11:59.874789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393723+0000","last_change":"2024-09-13T18:26:32.356902+0000","last_active":"2024-09-13T18:26:36.393723+0000","last_peered":"2024-09-13T18:26:36.393723+0000","last_clean":"2024-09-13T18:26:36.393723+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:26:36.393723+0000","last_undegraded":"2024-09-13T18:26:36.393723+0000","last_fullsized":"2024-09-13T18:26:36.393723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:18.362772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390031+0000","last_change":"2024-09-13T18:26:32.347047+0000","last_active":"2024-09-13T18:26:36.390031+0000","last_peered":"2024-09-13T18:26:36.390031+0000","last_clean":"2024-09-13T18:26:36.390031+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:26:36.390031+0000","last_undegraded":"2024-09-13T18:26:36.390031+0000","last_fullsized":"2024-09-13T18:26:36.390031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:40.882789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.385066+0000","last_change":"2024-09-13T18:26:32.349651+0000","last_active":"2024-09-13T18:26:36.385066+0000","last_peered":"2024-09-13T18:26:36.385066+0000","last_clean":"2024-09-13T18:26:36.385066+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:26:36.385066+0000","last_undegraded":"2024-09-13T18:26:36.385066+0000","last_fullsized":"2024-09-13T18:26:36.385066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:33.356563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388522+0000","last_change":"2024-09-13T18:26:32.344607+0000","last_active":"2024-09-13T18:26:36.388522+0000","last_peered":"2024-09-13T18:26:36.388522+0000","last_clean":"2024-09-13T18:26:36.388522+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:26:36.388522+0000","last_undegraded":"2024-09-13T18:26:36.388522+0000","last_fullsized":"2024-09-13T18:26:36.388522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:31.658172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387737+0000","last_change":"2024-09-13T18:26:32.347717+0000","last_active":"2024-09-13T18:26:36.387737+0000","last_peered":"2024-09-13T18:26:36.387737+0000","last_clean":"2024-09-13T18:26:36.387737+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:26:36.387737+0000","last_undegraded":"2024-09-13T18:26:36.387737+0000","last_fullsized":"2024-09-13T18:26:36.387737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:47.187837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392104+0000","last_change":"2024-09-13T18:26:32.359506+0000","last_active":"2024-09-13T18:26:36.392104+0000","last_peered":"2024-09-13T18:26:36.392104+0000","last_clean":"2024-09-13T18:26:36.392104+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:26:36.392104+0000","last_undegraded":"2024-09-13T18:26:36.392104+0000","last_fullsized":"2024-09-13T18:26:36.392104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:39:57.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897951+0000","last_change":"2024-09-13T18:26:32.357729+0000","last_active":"2024-09-13T18:26:36.897951+0000","last_peered":"2024-09-13T18:26:36.897951+0000","last_clean":"2024-09-13T18:26:36.897951+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:26:36.897951+0000","last_undegraded":"2024-09-13T18:26:36.897951+0000","last_fullsized":"2024-09-13T18:26:36.897951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:05.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897974+0000","last_change":"2024-09-13T18:26:32.357980+0000","last_active":"2024-09-13T18:26:36.897974+0000","last_peered":"2024-09-13T18:26:36.897974+0000","last_clean":"2024-09-13T18:26:36.897974+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:26:36.897974+0000","last_undegraded":"2024-09-13T18:26:36.897974+0000","last_fullsized":"2024-09-13T18:26:36.897974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:00.858702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405699+0000","last_change":"2024-09-13T18:26:32.365638+0000","last_active":"2024-09-13T18:26:36.405699+0000","last_peered":"2024-09-13T18:26:36.405699+0000","last_clean":"2024-09-13T18:26:36.405699+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:26:36.405699+0000","last_undegraded":"2024-09-13T18:26:36.405699+0000","last_fullsized":"2024-09-13T18:26:36.405699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.065853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383235+0000","last_change":"2024-09-13T18:26:32.345496+0000","last_active":"2024-09-13T18:26:36.383235+0000","last_peered":"2024-09-13T18:26:36.383235+0000","last_clean":"2024-09-13T18:26:36.383235+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:26:36.383235+0000","last_undegraded":"2024-09-13T18:26:36.383235+0000","last_fullsized":"2024-09-13T18:26:36.383235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:22.447315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404895+0000","last_change":"2024-09-13T18:26:32.356901+0000","last_active":"2024-09-13T18:26:36.404895+0000","last_peered":"2024-09-13T18:26:36.404895+0000","last_clean":"2024-09-13T18:26:36.404895+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:26:36.404895+0000","last_undegraded":"2024-09-13T18:26:36.404895+0000","last_fullsized":"2024-09-13T18:26:36.404895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:03.311603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391082+0000","last_change":"2024-09-13T18:26:32.355203+0000","last_active":"2024-09-13T18:26:36.391082+0000","last_peered":"2024-09-13T18:26:36.391082+0000","last_clean":"2024-09-13T18:26:36.391082+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:26:36.391082+0000","last_undegraded":"2024-09-13T18:26:36.391082+0000","last_fullsized":"2024-09-13T18:26:36.391082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:52.769708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405315+0000","last_change":"2024-09-13T18:26:32.358190+0000","last_active":"2024-09-13T18:26:36.405315+0000","last_peered":"2024-09-13T18:26:36.405315+0000","last_clean":"2024-09-13T18:26:36.405315+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:26:36.405315+0000","last_undegraded":"2024-09-13T18:26:36.405315+0000","last_fullsized":"2024-09-13T18:26:36.405315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:35.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393258+0000","last_change":"2024-09-13T18:26:32.355121+0000","last_active":"2024-09-13T18:26:36.393258+0000","last_peered":"2024-09-13T18:26:36.393258+0000","last_clean":"2024-09-13T18:26:36.393258+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:26:36.393258+0000","last_undegraded":"2024-09-13T18:26:36.393258+0000","last_fullsized":"2024-09-13T18:26:36.393258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:48.427332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388810+0000","last_change":"2024-09-13T18:26:32.341497+0000","last_active":"2024-09-13T18:26:36.388810+0000","last_peered":"2024-09-13T18:26:36.388810+0000","last_clean":"2024-09-13T18:26:36.388810+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:26:36.388810+0000","last_undegraded":"2024-09-13T18:26:36.388810+0000","last_fullsized":"2024-09-13T18:26:36.388810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:15.510048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386627+0000","last_change":"2024-09-13T18:26:32.350106+0000","last_active":"2024-09-13T18:26:36.386627+0000","last_peered":"2024-09-13T18:26:36.386627+0000","last_clean":"2024-09-13T18:26:36.386627+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:26:36.386627+0000","last_undegraded":"2024-09-13T18:26:36.386627+0000","last_fullsized":"2024-09-13T18:26:36.386627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:12.387451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405597+0000","last_change":"2024-09-13T18:26:32.362196+0000","last_active":"2024-09-13T18:26:36.405597+0000","last_peered":"2024-09-13T18:26:36.405597+0000","last_clean":"2024-09-13T18:26:36.405597+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:26:36.405597+0000","last_undegraded":"2024-09-13T18:26:36.405597+0000","last_fullsized":"2024-09-13T18:26:36.405597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:14.946904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5861,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":11812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]}]},{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":344064,"data_stored":694809,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":103299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]}]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":107542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":258048,"data_stored":103173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":270336,"data_stored":103935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":104005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":339968,"data_stored":694201,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:27:07.166 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:27:07.240 INFO:tasks.ceph.mds.c.smithi146.stderr:2024-09-13T18:27:07.233+0000 7f6be472a700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-13T18:27:07.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 Processor -- start 2024-09-13T18:27:07.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 -- start start 2024-09-13T18:27:07.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3db40caed0 0x7f3db40723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3db40cb2b0 0x7f3db41619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db4161550 con 0x7f3db40a9840 2024-09-13T18:27:07.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db4072a30 con 0x7f3db40cb2b0 2024-09-13T18:27:07.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db40aa800 con 0x7f3db40caed0 2024-09-13T18:27:07.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db1d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3db40cb2b0 0x7f3db41619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db1d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3db40cb2b0 0x7f3db41619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54884/0 (socket says 172.21.15.146:54884) 2024-09-13T18:27:07.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db1d9b700 1 -- 172.21.15.146:0/3084193454 learned_addr learned my addr 172.21.15.146:0/3084193454 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:07.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db1d9b700 1 -- 172.21.15.146:0/3084193454 >> v2:172.21.15.154:3300/0 conn(0x7f3db40caed0 msgr2=0x7f3db40723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:07.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db259c700 1 --2- 172.21.15.146:0/3084193454 >> v2:172.21.15.154:3300/0 conn(0x7f3db40caed0 0x7f3db40723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db1d9b700 1 --2- 172.21.15.146:0/3084193454 >> v2:172.21.15.154:3300/0 conn(0x7f3db40caed0 0x7f3db40723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db1d9b700 1 -- 172.21.15.146:0/3084193454 >> v1:172.21.15.154:6789/0 conn(0x7f3db40a9840 legacy=0x7f3db415e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:07.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db1d9b700 1 -- 172.21.15.146:0/3084193454 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db4161220 con 0x7f3db40cb2b0 2024-09-13T18:27:07.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db259c700 1 --2- 172.21.15.146:0/3084193454 >> v2:172.21.15.154:3300/0 conn(0x7f3db40caed0 0x7f3db40723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:07.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db1d9b700 1 --2- 172.21.15.146:0/3084193454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3db40cb2b0 0x7f3db41619e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3db40c87e0 tx=0x7f3da8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee8db4a9789333a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db159a700 1 -- 172.21.15.146:0/3084193454 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8004230 con 0x7f3db40cb2b0 2024-09-13T18:27:07.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db159a700 1 -- 172.21.15.146:0/3084193454 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da8004390 con 0x7f3db40cb2b0 2024-09-13T18:27:07.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db159a700 1 -- 172.21.15.146:0/3084193454 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da80036c0 con 0x7f3db40cb2b0 2024-09-13T18:27:07.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 -- 172.21.15.146:0/3084193454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3db40cb2b0 msgr2=0x7f3db41619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 --2- 172.21.15.146:0/3084193454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3db40cb2b0 0x7f3db41619e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3db40c87e0 tx=0x7f3da8009bf0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 -- 172.21.15.146:0/3084193454 shutdown_connections 2024-09-13T18:27:07.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 --2- 172.21.15.146:0/3084193454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3db40cb2b0 0x7f3db41619e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 --2- 172.21.15.146:0/3084193454 >> v2:172.21.15.154:3300/0 conn(0x7f3db40caed0 0x7f3db40723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 -- 172.21.15.146:0/3084193454 >> 172.21.15.146:0/3084193454 conn(0x7f3db4066090 msgr2=0x7f3db4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:07.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 -- 172.21.15.146:0/3084193454 shutdown_connections 2024-09-13T18:27:07.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.309+0000 7f3db359e700 1 -- 172.21.15.146:0/3084193454 wait complete. 2024-09-13T18:27:07.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.313+0000 7f3db359e700 1 Processor -- start 2024-09-13T18:27:07.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.313+0000 7f3db359e700 1 -- start start 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.313+0000 7f3db359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db40a9840 0x7f3db41e8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.313+0000 7f3db359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db40caed0 0x7f3db40af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.313+0000 7f3db359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3db40cb2b0 0x7f3db40addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.313+0000 7f3db359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4157f10 con 0x7f3db40cb2b0 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.313+0000 7f3db359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db41e73e0 con 0x7f3db40caed0 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.313+0000 7f3db359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db41e93b0 con 0x7f3db40a9840 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db2d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3db40cb2b0 0x7f3db40addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db2d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3db40cb2b0 0x7f3db40addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54892/0 (socket says 172.21.15.146:54892) 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db2d9d700 1 -- 172.21.15.146:0/3219209206 learned_addr learned my addr 172.21.15.146:0/3219209206 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db259c700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db40a9840 0x7f3db41e8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db1d9b700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db40caed0 0x7f3db40af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db2d9d700 1 -- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db40a9840 msgr2=0x7f3db41e8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db2d9d700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db40a9840 0x7f3db41e8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db2d9d700 1 -- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db40caed0 msgr2=0x7f3db40af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db2d9d700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db40caed0 0x7f3db40af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db2d9d700 1 -- 172.21.15.146:0/3219209206 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da8009570 con 0x7f3db40cb2b0 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db2d9d700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3db40cb2b0 0x7f3db40addc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00b480 tx=0x7f3dac00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3da2ffd700 1 -- 172.21.15.146:0/3219209206 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac011620 con 0x7f3db40cb2b0 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3da2ffd700 1 -- 172.21.15.146:0/3219209206 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dac0117a0 con 0x7f3db40cb2b0 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db359e700 1 -- 172.21.15.146:0/3219209206 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db41e6e10 con 0x7f3db40cb2b0 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3da2ffd700 1 -- 172.21.15.146:0/3219209206 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac011b50 con 0x7f3db40cb2b0 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db359e700 1 -- 172.21.15.146:0/3219209206 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db41e9d60 con 0x7f3db40cb2b0 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3da2ffd700 1 -- 172.21.15.146:0/3219209206 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3dac011cb0 con 0x7f3db40cb2b0 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3da2ffd700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d94066760 0x7f3d94068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.317+0000 7f3db359e700 1 -- 172.21.15.146:0/3219209206 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db41e7590 con 0x7f3db40cb2b0 2024-09-13T18:27:07.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.321+0000 7f3da2ffd700 1 -- 172.21.15.146:0/3219209206 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3dac083f90 con 0x7f3db40cb2b0 2024-09-13T18:27:07.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.321+0000 7f3db259c700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d94066760 0x7f3d94068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.321+0000 7f3db259c700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d94066760 0x7f3d94068c20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3da400f500 tx=0x7f3da4007770 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.325+0000 7f3da2ffd700 1 -- 172.21.15.146:0/3219209206 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3db41e7590 con 0x7f3db40cb2b0 2024-09-13T18:27:07.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.481+0000 7f3db359e700 1 -- 172.21.15.146:0/3219209206 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3db41e9ef0 con 0x7f3d94066760 2024-09-13T18:27:07.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.497+0000 7f3da2ffd700 1 -- 172.21.15.146:0/3219209206 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345422 (secure 0 0 0) 0x7f3db41e9ef0 con 0x7f3d94066760 2024-09-13T18:27:07.508 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:07.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.505+0000 7f3db359e700 1 -- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d94066760 msgr2=0x7f3d94068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.505+0000 7f3db359e700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d94066760 0x7f3d94068c20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3da400f500 tx=0x7f3da4007770 comp rx=0 tx=0).stop 2024-09-13T18:27:07.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.505+0000 7f3db359e700 1 -- 172.21.15.146:0/3219209206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3db40cb2b0 msgr2=0x7f3db40addc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.505+0000 7f3db359e700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3db40cb2b0 0x7f3db40addc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00b480 tx=0x7f3dac00b790 comp rx=0 tx=0).stop 2024-09-13T18:27:07.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.505+0000 7f3db359e700 1 -- 172.21.15.146:0/3219209206 shutdown_connections 2024-09-13T18:27:07.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.505+0000 7f3db359e700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d94066760 0x7f3d94068c20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.505+0000 7f3db359e700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3db40cb2b0 0x7f3db40addc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.505+0000 7f3db359e700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db40caed0 0x7f3db40af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.505+0000 7f3db359e700 1 --2- 172.21.15.146:0/3219209206 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db40a9840 0x7f3db41e8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.505+0000 7f3db359e700 1 -- 172.21.15.146:0/3219209206 >> 172.21.15.146:0/3219209206 conn(0x7f3db4066090 msgr2=0x7f3db406f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:07.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.505+0000 7f3db359e700 1 -- 172.21.15.146:0/3219209206 shutdown_connections 2024-09-13T18:27:07.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.505+0000 7f3db359e700 1 -- 172.21.15.146:0/3219209206 wait complete. 2024-09-13T18:27:07.512 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:27:07.523 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-13T18:27:06.830200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596229,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":236,"num_write_kb":2445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222476,"kb_used_data":2252,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754752244,"statfs":{"total":773094113280,"available":772866297856,"internally_reserved":0,"allocated":2306048,"data_stored":2017932,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":225431099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1268,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005630"},"pg_stats":[{"pgid":"2.1f","version":"25'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.345647+0000","last_change":"2024-09-13T18:26:31.972797+0000","last_active":"2024-09-13T18:26:37.345647+0000","last_peered":"2024-09-13T18:26:37.345647+0000","last_clean":"2024-09-13T18:26:37.345647+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:26:37.345647+0000","last_undegraded":"2024-09-13T18:26:37.345647+0000","last_fullsized":"2024-09-13T18:26:37.345647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:50.511747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384385+0000","last_change":"2024-09-13T18:26:32.349901+0000","last_active":"2024-09-13T18:26:36.384385+0000","last_peered":"2024-09-13T18:26:36.384385+0000","last_clean":"2024-09-13T18:26:36.384385+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:26:36.384385+0000","last_undegraded":"2024-09-13T18:26:36.384385+0000","last_fullsized":"2024-09-13T18:26:36.384385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:24.978813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768058+0000","last_change":"2024-09-13T18:26:36.407579+0000","last_active":"2024-09-13T18:26:36.768058+0000","last_peered":"2024-09-13T18:26:36.768058+0000","last_clean":"2024-09-13T18:26:36.768058+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:26:36.768058+0000","last_undegraded":"2024-09-13T18:26:36.768058+0000","last_fullsized":"2024-09-13T18:26:36.768058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:53:56.370013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769520+0000","last_change":"2024-09-13T18:26:31.980716+0000","last_active":"2024-09-13T18:26:36.769520+0000","last_peered":"2024-09-13T18:26:36.769520+0000","last_clean":"2024-09-13T18:26:36.769520+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:26:36.769520+0000","last_undegraded":"2024-09-13T18:26:36.769520+0000","last_fullsized":"2024-09-13T18:26:36.769520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:20.020903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388711+0000","last_change":"2024-09-13T18:26:32.342690+0000","last_active":"2024-09-13T18:26:36.388711+0000","last_peered":"2024-09-13T18:26:36.388711+0000","last_clean":"2024-09-13T18:26:36.388711+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:26:36.388711+0000","last_undegraded":"2024-09-13T18:26:36.388711+0000","last_fullsized":"2024-09-13T18:26:36.388711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.212930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.398481+0000","last_change":"2024-09-13T18:26:36.396545+0000","last_active":"2024-09-13T18:26:36.398481+0000","last_peered":"2024-09-13T18:26:36.398481+0000","last_clean":"2024-09-13T18:26:36.398481+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:26:36.398481+0000","last_undegraded":"2024-09-13T18:26:36.398481+0000","last_fullsized":"2024-09-13T18:26:36.398481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:38.760215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.349299+0000","last_change":"2024-09-13T18:26:31.977823+0000","last_active":"2024-09-13T18:26:37.349299+0000","last_peered":"2024-09-13T18:26:37.349299+0000","last_clean":"2024-09-13T18:26:37.349299+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:26:37.349299+0000","last_undegraded":"2024-09-13T18:26:37.349299+0000","last_fullsized":"2024-09-13T18:26:37.349299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:46.956527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394225+0000","last_change":"2024-09-13T18:26:32.361288+0000","last_active":"2024-09-13T18:26:36.394225+0000","last_peered":"2024-09-13T18:26:36.394225+0000","last_clean":"2024-09-13T18:26:36.394225+0000","last_became_active":"2024-09-13T18:26:32.360918+0000","last_became_peered":"2024-09-13T18:26:32.360918+0000","last_unstale":"2024-09-13T18:26:36.394225+0000","last_undegraded":"2024-09-13T18:26:36.394225+0000","last_fullsized":"2024-09-13T18:26:36.394225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:30.143436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408364+0000","last_change":"2024-09-13T18:26:36.406675+0000","last_active":"2024-09-13T18:26:36.408364+0000","last_peered":"2024-09-13T18:26:36.408364+0000","last_clean":"2024-09-13T18:26:36.408364+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:26:36.408364+0000","last_undegraded":"2024-09-13T18:26:36.408364+0000","last_fullsized":"2024-09-13T18:26:36.408364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:36.896027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.347551+0000","last_change":"2024-09-13T18:26:31.979860+0000","last_active":"2024-09-13T18:26:37.347551+0000","last_peered":"2024-09-13T18:26:37.347551+0000","last_clean":"2024-09-13T18:26:37.347551+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:26:37.347551+0000","last_undegraded":"2024-09-13T18:26:37.347551+0000","last_fullsized":"2024-09-13T18:26:37.347551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:45.323631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393203+0000","last_change":"2024-09-13T18:26:32.351494+0000","last_active":"2024-09-13T18:26:36.393203+0000","last_peered":"2024-09-13T18:26:36.393203+0000","last_clean":"2024-09-13T18:26:36.393203+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:26:36.393203+0000","last_undegraded":"2024-09-13T18:26:36.393203+0000","last_fullsized":"2024-09-13T18:26:36.393203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:52.804526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400955+0000","last_change":"2024-09-13T18:26:36.398763+0000","last_active":"2024-09-13T18:26:36.400955+0000","last_peered":"2024-09-13T18:26:36.400955+0000","last_clean":"2024-09-13T18:26:36.400955+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:26:36.400955+0000","last_undegraded":"2024-09-13T18:26:36.400955+0000","last_fullsized":"2024-09-13T18:26:36.400955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:35.417438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897834+0000","last_change":"2024-09-13T18:26:32.360976+0000","last_active":"2024-09-13T18:26:36.897834+0000","last_peered":"2024-09-13T18:26:36.897834+0000","last_clean":"2024-09-13T18:26:36.897834+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:26:36.897834+0000","last_undegraded":"2024-09-13T18:26:36.897834+0000","last_fullsized":"2024-09-13T18:26:36.897834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:53:52.373515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.477712+0000","last_change":"2024-09-13T18:26:31.977891+0000","last_active":"2024-09-13T18:27:03.477712+0000","last_peered":"2024-09-13T18:27:03.477712+0000","last_clean":"2024-09-13T18:27:03.477712+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:27:03.477712+0000","last_undegraded":"2024-09-13T18:27:03.477712+0000","last_fullsized":"2024-09-13T18:27:03.477712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:01.453094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400563+0000","last_change":"2024-09-13T18:26:36.397484+0000","last_active":"2024-09-13T18:26:36.400563+0000","last_peered":"2024-09-13T18:26:36.400563+0000","last_clean":"2024-09-13T18:26:36.400563+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:26:36.400563+0000","last_undegraded":"2024-09-13T18:26:36.400563+0000","last_fullsized":"2024-09-13T18:26:36.400563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:44.686244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386958+0000","last_change":"2024-09-13T18:26:32.345564+0000","last_active":"2024-09-13T18:26:36.386958+0000","last_peered":"2024-09-13T18:26:36.386958+0000","last_clean":"2024-09-13T18:26:36.386958+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:26:36.386958+0000","last_undegraded":"2024-09-13T18:26:36.386958+0000","last_fullsized":"2024-09-13T18:26:36.386958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:54.331806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386241+0000","last_change":"2024-09-13T18:26:31.974107+0000","last_active":"2024-09-13T18:26:36.386241+0000","last_peered":"2024-09-13T18:26:36.386241+0000","last_clean":"2024-09-13T18:26:36.386241+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:26:36.386241+0000","last_undegraded":"2024-09-13T18:26:36.386241+0000","last_fullsized":"2024-09-13T18:26:36.386241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:37.547480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408253+0000","last_change":"2024-09-13T18:26:36.405986+0000","last_active":"2024-09-13T18:26:36.408253+0000","last_peered":"2024-09-13T18:26:36.408253+0000","last_clean":"2024-09-13T18:26:36.408253+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:26:36.408253+0000","last_undegraded":"2024-09-13T18:26:36.408253+0000","last_fullsized":"2024-09-13T18:26:36.408253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:30.052430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383772+0000","last_change":"2024-09-13T18:26:32.340616+0000","last_active":"2024-09-13T18:26:36.383772+0000","last_peered":"2024-09-13T18:26:36.383772+0000","last_clean":"2024-09-13T18:26:36.383772+0000","last_became_active":"2024-09-13T18:26:32.339405+0000","last_became_peered":"2024-09-13T18:26:32.339405+0000","last_unstale":"2024-09-13T18:26:36.383772+0000","last_undegraded":"2024-09-13T18:26:36.383772+0000","last_fullsized":"2024-09-13T18:26:36.383772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:26.081896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.474579+0000","last_change":"2024-09-13T18:26:31.968895+0000","last_active":"2024-09-13T18:27:03.474579+0000","last_peered":"2024-09-13T18:27:03.474579+0000","last_clean":"2024-09-13T18:27:03.474579+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:27:03.474579+0000","last_undegraded":"2024-09-13T18:27:03.474579+0000","last_fullsized":"2024-09-13T18:27:03.474579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:15.525097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767834+0000","last_change":"2024-09-13T18:26:36.406484+0000","last_active":"2024-09-13T18:26:36.767834+0000","last_peered":"2024-09-13T18:26:36.767834+0000","last_clean":"2024-09-13T18:26:36.767834+0000","last_became_active":"2024-09-13T18:26:36.406161+0000","last_became_peered":"2024-09-13T18:26:36.406161+0000","last_unstale":"2024-09-13T18:26:36.767834+0000","last_undegraded":"2024-09-13T18:26:36.767834+0000","last_fullsized":"2024-09-13T18:26:36.767834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:53.330919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391435+0000","last_change":"2024-09-13T18:26:32.360406+0000","last_active":"2024-09-13T18:26:36.391435+0000","last_peered":"2024-09-13T18:26:36.391435+0000","last_clean":"2024-09-13T18:26:36.391435+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:26:36.391435+0000","last_undegraded":"2024-09-13T18:26:36.391435+0000","last_fullsized":"2024-09-13T18:26:36.391435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:37.417707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.476272+0000","last_change":"2024-09-13T18:26:31.976872+0000","last_active":"2024-09-13T18:27:03.476272+0000","last_peered":"2024-09-13T18:27:03.476272+0000","last_clean":"2024-09-13T18:27:03.476272+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:27:03.476272+0000","last_undegraded":"2024-09-13T18:27:03.476272+0000","last_fullsized":"2024-09-13T18:27:03.476272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:26.163029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408244+0000","last_change":"2024-09-13T18:26:36.405722+0000","last_active":"2024-09-13T18:26:36.408244+0000","last_peered":"2024-09-13T18:26:36.408244+0000","last_clean":"2024-09-13T18:26:36.408244+0000","last_became_active":"2024-09-13T18:26:36.405310+0000","last_became_peered":"2024-09-13T18:26:36.405310+0000","last_unstale":"2024-09-13T18:26:36.408244+0000","last_undegraded":"2024-09-13T18:26:36.408244+0000","last_fullsized":"2024-09-13T18:26:36.408244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:09.815655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.389080+0000","last_change":"2024-09-13T18:26:32.343915+0000","last_active":"2024-09-13T18:26:36.389080+0000","last_peered":"2024-09-13T18:26:36.389080+0000","last_clean":"2024-09-13T18:26:36.389080+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:26:36.389080+0000","last_undegraded":"2024-09-13T18:26:36.389080+0000","last_fullsized":"2024-09-13T18:26:36.389080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:19.357243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.349375+0000","last_change":"2024-09-13T18:26:31.979944+0000","last_active":"2024-09-13T18:26:37.349375+0000","last_peered":"2024-09-13T18:26:37.349375+0000","last_clean":"2024-09-13T18:26:37.349375+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:26:37.349375+0000","last_undegraded":"2024-09-13T18:26:37.349375+0000","last_fullsized":"2024-09-13T18:26:37.349375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:21.721921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.411958+0000","last_change":"2024-09-13T18:26:36.410028+0000","last_active":"2024-09-13T18:26:36.411958+0000","last_peered":"2024-09-13T18:26:36.411958+0000","last_clean":"2024-09-13T18:26:36.411958+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:26:36.411958+0000","last_undegraded":"2024-09-13T18:26:36.411958+0000","last_fullsized":"2024-09-13T18:26:36.411958+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:32.264341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387875+0000","last_change":"2024-09-13T18:26:32.358980+0000","last_active":"2024-09-13T18:26:36.387875+0000","last_peered":"2024-09-13T18:26:36.387875+0000","last_clean":"2024-09-13T18:26:36.387875+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:26:36.387875+0000","last_undegraded":"2024-09-13T18:26:36.387875+0000","last_fullsized":"2024-09-13T18:26:36.387875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:08.748139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769462+0000","last_change":"2024-09-13T18:26:31.968972+0000","last_active":"2024-09-13T18:26:36.769462+0000","last_peered":"2024-09-13T18:26:36.769462+0000","last_clean":"2024-09-13T18:26:36.769462+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:26:36.769462+0000","last_undegraded":"2024-09-13T18:26:36.769462+0000","last_fullsized":"2024-09-13T18:26:36.769462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:56.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768011+0000","last_change":"2024-09-13T18:26:36.407034+0000","last_active":"2024-09-13T18:26:36.768011+0000","last_peered":"2024-09-13T18:26:36.768011+0000","last_clean":"2024-09-13T18:26:36.768011+0000","last_became_active":"2024-09-13T18:26:36.406473+0000","last_became_peered":"2024-09-13T18:26:36.406473+0000","last_unstale":"2024-09-13T18:26:36.768011+0000","last_undegraded":"2024-09-13T18:26:36.768011+0000","last_fullsized":"2024-09-13T18:26:36.768011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:53.815901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386420+0000","last_change":"2024-09-13T18:26:32.342823+0000","last_active":"2024-09-13T18:26:36.386420+0000","last_peered":"2024-09-13T18:26:36.386420+0000","last_clean":"2024-09-13T18:26:36.386420+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:26:36.386420+0000","last_undegraded":"2024-09-13T18:26:36.386420+0000","last_fullsized":"2024-09-13T18:26:36.386420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:28.051252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.349179+0000","last_change":"2024-09-13T18:26:31.978976+0000","last_active":"2024-09-13T18:26:37.349179+0000","last_peered":"2024-09-13T18:26:37.349179+0000","last_clean":"2024-09-13T18:26:37.349179+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:26:37.349179+0000","last_undegraded":"2024-09-13T18:26:37.349179+0000","last_fullsized":"2024-09-13T18:26:37.349179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:11.279557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103455+0000","last_change":"2024-09-13T18:26:36.408844+0000","last_active":"2024-09-13T18:26:37.103455+0000","last_peered":"2024-09-13T18:26:37.103455+0000","last_clean":"2024-09-13T18:26:37.103455+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:26:37.103455+0000","last_undegraded":"2024-09-13T18:26:37.103455+0000","last_fullsized":"2024-09-13T18:26:37.103455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:06.969342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393043+0000","last_change":"2024-09-13T18:26:32.355038+0000","last_active":"2024-09-13T18:26:36.393043+0000","last_peered":"2024-09-13T18:26:36.393043+0000","last_clean":"2024-09-13T18:26:36.393043+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:26:36.393043+0000","last_undegraded":"2024-09-13T18:26:36.393043+0000","last_fullsized":"2024-09-13T18:26:36.393043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.159231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386562+0000","last_change":"2024-09-13T18:26:31.972931+0000","last_active":"2024-09-13T18:26:36.386562+0000","last_peered":"2024-09-13T18:26:36.386562+0000","last_clean":"2024-09-13T18:26:36.386562+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:26:36.386562+0000","last_undegraded":"2024-09-13T18:26:36.386562+0000","last_fullsized":"2024-09-13T18:26:36.386562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:05.646924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766310+0000","last_change":"2024-09-13T18:26:36.398876+0000","last_active":"2024-09-13T18:26:36.766310+0000","last_peered":"2024-09-13T18:26:36.766310+0000","last_clean":"2024-09-13T18:26:36.766310+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:26:36.766310+0000","last_undegraded":"2024-09-13T18:26:36.766310+0000","last_fullsized":"2024-09-13T18:26:36.766310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:56.918239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392625+0000","last_change":"2024-09-13T18:26:32.353111+0000","last_active":"2024-09-13T18:26:36.392625+0000","last_peered":"2024-09-13T18:26:36.392625+0000","last_clean":"2024-09-13T18:26:36.392625+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:26:36.392625+0000","last_undegraded":"2024-09-13T18:26:36.392625+0000","last_fullsized":"2024-09-13T18:26:36.392625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:43.927936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391864+0000","last_change":"2024-09-13T18:26:31.976075+0000","last_active":"2024-09-13T18:26:36.391864+0000","last_peered":"2024-09-13T18:26:36.391864+0000","last_clean":"2024-09-13T18:26:36.391864+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:26:36.391864+0000","last_undegraded":"2024-09-13T18:26:36.391864+0000","last_fullsized":"2024-09-13T18:26:36.391864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:03.693518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766877+0000","last_change":"2024-09-13T18:26:36.401777+0000","last_active":"2024-09-13T18:26:36.766877+0000","last_peered":"2024-09-13T18:26:36.766877+0000","last_clean":"2024-09-13T18:26:36.766877+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:26:36.766877+0000","last_undegraded":"2024-09-13T18:26:36.766877+0000","last_fullsized":"2024-09-13T18:26:36.766877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:43.440284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384991+0000","last_change":"2024-09-13T18:26:32.343791+0000","last_active":"2024-09-13T18:26:36.384991+0000","last_peered":"2024-09-13T18:26:36.384991+0000","last_clean":"2024-09-13T18:26:36.384991+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:26:36.384991+0000","last_undegraded":"2024-09-13T18:26:36.384991+0000","last_fullsized":"2024-09-13T18:26:36.384991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:36.296571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343727+0000","last_change":"2024-09-13T18:26:31.970786+0000","last_active":"2024-09-13T18:26:37.343727+0000","last_peered":"2024-09-13T18:26:37.343727+0000","last_clean":"2024-09-13T18:26:37.343727+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:26:37.343727+0000","last_undegraded":"2024-09-13T18:26:37.343727+0000","last_fullsized":"2024-09-13T18:26:37.343727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:54.584009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767929+0000","last_change":"2024-09-13T18:26:36.401529+0000","last_active":"2024-09-13T18:26:36.767929+0000","last_peered":"2024-09-13T18:26:36.767929+0000","last_clean":"2024-09-13T18:26:36.767929+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:26:36.767929+0000","last_undegraded":"2024-09-13T18:26:36.767929+0000","last_fullsized":"2024-09-13T18:26:36.767929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:47.318347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.382662+0000","last_change":"2024-09-13T18:26:32.350470+0000","last_active":"2024-09-13T18:26:36.382662+0000","last_peered":"2024-09-13T18:26:36.382662+0000","last_clean":"2024-09-13T18:26:36.382662+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:26:36.382662+0000","last_undegraded":"2024-09-13T18:26:36.382662+0000","last_fullsized":"2024-09-13T18:26:36.382662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:55.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.347577+0000","last_change":"2024-09-13T18:26:31.972587+0000","last_active":"2024-09-13T18:26:37.347577+0000","last_peered":"2024-09-13T18:26:37.347577+0000","last_clean":"2024-09-13T18:26:37.347577+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:26:37.347577+0000","last_undegraded":"2024-09-13T18:26:37.347577+0000","last_fullsized":"2024-09-13T18:26:37.347577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:27.547763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103735+0000","last_change":"2024-09-13T18:26:36.405086+0000","last_active":"2024-09-13T18:26:37.103735+0000","last_peered":"2024-09-13T18:26:37.103735+0000","last_clean":"2024-09-13T18:26:37.103735+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:26:37.103735+0000","last_undegraded":"2024-09-13T18:26:37.103735+0000","last_fullsized":"2024-09-13T18:26:37.103735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:02:51.694634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388036+0000","last_change":"2024-09-13T18:26:32.350510+0000","last_active":"2024-09-13T18:26:36.388036+0000","last_peered":"2024-09-13T18:26:36.388036+0000","last_clean":"2024-09-13T18:26:36.388036+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:26:36.388036+0000","last_undegraded":"2024-09-13T18:26:36.388036+0000","last_fullsized":"2024-09-13T18:26:36.388036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:40.850255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.479641+0000","last_change":"2024-09-13T18:26:31.978355+0000","last_active":"2024-09-13T18:27:03.479641+0000","last_peered":"2024-09-13T18:27:03.479641+0000","last_clean":"2024-09-13T18:27:03.479641+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:27:03.479641+0000","last_undegraded":"2024-09-13T18:27:03.479641+0000","last_fullsized":"2024-09-13T18:27:03.479641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:12.946518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404778+0000","last_change":"2024-09-13T18:26:26.931403+0000","last_active":"2024-09-13T18:26:36.404778+0000","last_peered":"2024-09-13T18:26:36.404778+0000","last_clean":"2024-09-13T18:26:36.404778+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:26:36.404778+0000","last_undegraded":"2024-09-13T18:26:36.404778+0000","last_fullsized":"2024-09-13T18:26:36.404778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_clean_scrub_stamp":"2024-09-13T18:26:24.880028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:30.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897926+0000","last_change":"2024-09-13T18:26:36.409827+0000","last_active":"2024-09-13T18:26:36.897926+0000","last_peered":"2024-09-13T18:26:36.897926+0000","last_clean":"2024-09-13T18:26:36.897926+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:26:36.897926+0000","last_undegraded":"2024-09-13T18:26:36.897926+0000","last_fullsized":"2024-09-13T18:26:36.897926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:50.482400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405746+0000","last_change":"2024-09-13T18:26:32.356850+0000","last_active":"2024-09-13T18:26:36.405746+0000","last_peered":"2024-09-13T18:26:36.405746+0000","last_clean":"2024-09-13T18:26:36.405746+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:26:36.405746+0000","last_undegraded":"2024-09-13T18:26:36.405746+0000","last_fullsized":"2024-09-13T18:26:36.405746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:38.710828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.376787+0000","last_change":"2024-09-13T18:26:31.975942+0000","last_active":"2024-09-13T18:26:36.376787+0000","last_peered":"2024-09-13T18:26:36.376787+0000","last_clean":"2024-09-13T18:26:36.376787+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:26:36.376787+0000","last_undegraded":"2024-09-13T18:26:36.376787+0000","last_fullsized":"2024-09-13T18:26:36.376787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:05.840582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897805+0000","last_change":"2024-09-13T18:26:36.409802+0000","last_active":"2024-09-13T18:26:36.897805+0000","last_peered":"2024-09-13T18:26:36.897805+0000","last_clean":"2024-09-13T18:26:36.897805+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:26:36.897805+0000","last_undegraded":"2024-09-13T18:26:36.897805+0000","last_fullsized":"2024-09-13T18:26:36.897805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:38.178151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394309+0000","last_change":"2024-09-13T18:26:32.355181+0000","last_active":"2024-09-13T18:26:36.394309+0000","last_peered":"2024-09-13T18:26:36.394309+0000","last_clean":"2024-09-13T18:26:36.394309+0000","last_became_active":"2024-09-13T18:26:32.353043+0000","last_became_peered":"2024-09-13T18:26:32.353043+0000","last_unstale":"2024-09-13T18:26:36.394309+0000","last_undegraded":"2024-09-13T18:26:36.394309+0000","last_fullsized":"2024-09-13T18:26:36.394309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:04:17.000915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767488+0000","last_change":"2024-09-13T18:26:31.966618+0000","last_active":"2024-09-13T18:26:36.767488+0000","last_peered":"2024-09-13T18:26:36.767488+0000","last_clean":"2024-09-13T18:26:36.767488+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:26:36.767488+0000","last_undegraded":"2024-09-13T18:26:36.767488+0000","last_fullsized":"2024-09-13T18:26:36.767488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:54:26.422611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103645+0000","last_change":"2024-09-13T18:26:36.394344+0000","last_active":"2024-09-13T18:26:37.103645+0000","last_peered":"2024-09-13T18:26:37.103645+0000","last_clean":"2024-09-13T18:26:37.103645+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:26:37.103645+0000","last_undegraded":"2024-09-13T18:26:37.103645+0000","last_fullsized":"2024-09-13T18:26:37.103645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:18.425816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405784+0000","last_change":"2024-09-13T18:26:32.365655+0000","last_active":"2024-09-13T18:26:36.405784+0000","last_peered":"2024-09-13T18:26:36.405784+0000","last_clean":"2024-09-13T18:26:36.405784+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:26:36.405784+0000","last_undegraded":"2024-09-13T18:26:36.405784+0000","last_fullsized":"2024-09-13T18:26:36.405784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:40.288128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343912+0000","last_change":"2024-09-13T18:26:31.970652+0000","last_active":"2024-09-13T18:26:37.343912+0000","last_peered":"2024-09-13T18:26:37.343912+0000","last_clean":"2024-09-13T18:26:37.343912+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:26:37.343912+0000","last_undegraded":"2024-09-13T18:26:37.343912+0000","last_fullsized":"2024-09-13T18:26:37.343912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:11.169661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.770631+0000","last_change":"2024-09-13T18:26:36.410280+0000","last_active":"2024-09-13T18:26:36.770631+0000","last_peered":"2024-09-13T18:26:36.770631+0000","last_clean":"2024-09-13T18:26:36.770631+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:26:36.770631+0000","last_undegraded":"2024-09-13T18:26:36.770631+0000","last_fullsized":"2024-09-13T18:26:36.770631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:01.395290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405642+0000","last_change":"2024-09-13T18:26:32.361575+0000","last_active":"2024-09-13T18:26:36.405642+0000","last_peered":"2024-09-13T18:26:36.405642+0000","last_clean":"2024-09-13T18:26:36.405642+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:26:36.405642+0000","last_undegraded":"2024-09-13T18:26:36.405642+0000","last_fullsized":"2024-09-13T18:26:36.405642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:08.818877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.239195+0000","last_change":"2024-09-13T18:26:31.977029+0000","last_active":"2024-09-13T18:26:37.239195+0000","last_peered":"2024-09-13T18:26:37.239195+0000","last_clean":"2024-09-13T18:26:37.239195+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:26:37.239195+0000","last_undegraded":"2024-09-13T18:26:37.239195+0000","last_fullsized":"2024-09-13T18:26:37.239195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:13.701689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407565+0000","last_change":"2024-09-13T18:26:36.403479+0000","last_active":"2024-09-13T18:26:36.407565+0000","last_peered":"2024-09-13T18:26:36.407565+0000","last_clean":"2024-09-13T18:26:36.407565+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:26:36.407565+0000","last_undegraded":"2024-09-13T18:26:36.407565+0000","last_fullsized":"2024-09-13T18:26:36.407565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:51.095852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404976+0000","last_change":"2024-09-13T18:26:32.365143+0000","last_active":"2024-09-13T18:26:36.404976+0000","last_peered":"2024-09-13T18:26:36.404976+0000","last_clean":"2024-09-13T18:26:36.404976+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:26:36.404976+0000","last_undegraded":"2024-09-13T18:26:36.404976+0000","last_fullsized":"2024-09-13T18:26:36.404976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:46.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'10","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.466996+0000","last_change":"2024-09-13T18:26:31.965802+0000","last_active":"2024-09-13T18:27:03.466996+0000","last_peered":"2024-09-13T18:27:03.466996+0000","last_clean":"2024-09-13T18:27:03.466996+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:27:03.466996+0000","last_undegraded":"2024-09-13T18:27:03.466996+0000","last_fullsized":"2024-09-13T18:27:03.466996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:12.759181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103768+0000","last_change":"2024-09-13T18:26:36.399235+0000","last_active":"2024-09-13T18:26:37.103768+0000","last_peered":"2024-09-13T18:26:37.103768+0000","last_clean":"2024-09-13T18:26:37.103768+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:26:37.103768+0000","last_undegraded":"2024-09-13T18:26:37.103768+0000","last_fullsized":"2024-09-13T18:26:37.103768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:13:54.711083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897760+0000","last_change":"2024-09-13T18:26:32.357403+0000","last_active":"2024-09-13T18:26:36.897760+0000","last_peered":"2024-09-13T18:26:36.897760+0000","last_clean":"2024-09-13T18:26:36.897760+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:26:36.897760+0000","last_undegraded":"2024-09-13T18:26:36.897760+0000","last_fullsized":"2024-09-13T18:26:36.897760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:12.600397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383205+0000","last_change":"2024-09-13T18:26:31.976148+0000","last_active":"2024-09-13T18:26:36.383205+0000","last_peered":"2024-09-13T18:26:36.383205+0000","last_clean":"2024-09-13T18:26:36.383205+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:26:36.383205+0000","last_undegraded":"2024-09-13T18:26:36.383205+0000","last_fullsized":"2024-09-13T18:26:36.383205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:38.023636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769582+0000","last_change":"2024-09-13T18:26:36.406177+0000","last_active":"2024-09-13T18:26:36.769582+0000","last_peered":"2024-09-13T18:26:36.769582+0000","last_clean":"2024-09-13T18:26:36.769582+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:26:36.769582+0000","last_undegraded":"2024-09-13T18:26:36.769582+0000","last_fullsized":"2024-09-13T18:26:36.769582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:00.309995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405459+0000","last_change":"2024-09-13T18:26:32.361726+0000","last_active":"2024-09-13T18:26:36.405459+0000","last_peered":"2024-09-13T18:26:36.405459+0000","last_clean":"2024-09-13T18:26:36.405459+0000","last_became_active":"2024-09-13T18:26:32.361432+0000","last_became_peered":"2024-09-13T18:26:32.361432+0000","last_unstale":"2024-09-13T18:26:36.405459+0000","last_undegraded":"2024-09-13T18:26:36.405459+0000","last_fullsized":"2024-09-13T18:26:36.405459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:42.021515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.480098+0000","last_change":"2024-09-13T18:26:31.982114+0000","last_active":"2024-09-13T18:27:03.480098+0000","last_peered":"2024-09-13T18:27:03.480098+0000","last_clean":"2024-09-13T18:27:03.480098+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:27:03.480098+0000","last_undegraded":"2024-09-13T18:27:03.480098+0000","last_fullsized":"2024-09-13T18:27:03.480098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.266528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769049+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:26:36.769049+0000","last_peered":"2024-09-13T18:26:36.769049+0000","last_clean":"2024-09-13T18:26:36.769049+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:26:36.769049+0000","last_undegraded":"2024-09-13T18:26:36.769049+0000","last_fullsized":"2024-09-13T18:26:36.769049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:58.853971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388651+0000","last_change":"2024-09-13T18:26:32.351145+0000","last_active":"2024-09-13T18:26:36.388651+0000","last_peered":"2024-09-13T18:26:36.388651+0000","last_clean":"2024-09-13T18:26:36.388651+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:26:36.388651+0000","last_undegraded":"2024-09-13T18:26:36.388651+0000","last_fullsized":"2024-09-13T18:26:36.388651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:57.002435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.471478+0000","last_change":"2024-09-13T18:26:31.961202+0000","last_active":"2024-09-13T18:27:03.471478+0000","last_peered":"2024-09-13T18:27:03.471478+0000","last_clean":"2024-09-13T18:27:03.471478+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:27:03.471478+0000","last_undegraded":"2024-09-13T18:27:03.471478+0000","last_fullsized":"2024-09-13T18:27:03.471478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:27.340127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.772209+0000","last_change":"2024-09-13T18:26:36.410014+0000","last_active":"2024-09-13T18:26:36.772209+0000","last_peered":"2024-09-13T18:26:36.772209+0000","last_clean":"2024-09-13T18:26:36.772209+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:26:36.772209+0000","last_undegraded":"2024-09-13T18:26:36.772209+0000","last_fullsized":"2024-09-13T18:26:36.772209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:24.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405669+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:26:36.405669+0000","last_peered":"2024-09-13T18:26:36.405669+0000","last_clean":"2024-09-13T18:26:36.405669+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:26:36.405669+0000","last_undegraded":"2024-09-13T18:26:36.405669+0000","last_fullsized":"2024-09-13T18:26:36.405669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:40.102815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:45.381784+0000","last_change":"2024-09-13T18:26:31.974110+0000","last_active":"2024-09-13T18:26:45.381784+0000","last_peered":"2024-09-13T18:26:45.381784+0000","last_clean":"2024-09-13T18:26:45.381784+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:26:45.381784+0000","last_undegraded":"2024-09-13T18:26:45.381784+0000","last_fullsized":"2024-09-13T18:26:45.381784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:56.058561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766576+0000","last_change":"2024-09-13T18:26:36.394080+0000","last_active":"2024-09-13T18:26:36.766576+0000","last_peered":"2024-09-13T18:26:36.766576+0000","last_clean":"2024-09-13T18:26:36.766576+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:26:36.766576+0000","last_undegraded":"2024-09-13T18:26:36.766576+0000","last_fullsized":"2024-09-13T18:26:36.766576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:47.121814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392195+0000","last_change":"2024-09-13T18:26:32.362873+0000","last_active":"2024-09-13T18:26:36.392195+0000","last_peered":"2024-09-13T18:26:36.392195+0000","last_clean":"2024-09-13T18:26:36.392195+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:26:36.392195+0000","last_undegraded":"2024-09-13T18:26:36.392195+0000","last_fullsized":"2024-09-13T18:26:36.392195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:37.598189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.344708+0000","last_change":"2024-09-13T18:26:31.976401+0000","last_active":"2024-09-13T18:26:37.344708+0000","last_peered":"2024-09-13T18:26:37.344708+0000","last_clean":"2024-09-13T18:26:37.344708+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:26:37.344708+0000","last_undegraded":"2024-09-13T18:26:37.344708+0000","last_fullsized":"2024-09-13T18:26:37.344708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:26.434292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408317+0000","last_change":"2024-09-13T18:26:36.406228+0000","last_active":"2024-09-13T18:26:36.408317+0000","last_peered":"2024-09-13T18:26:36.408317+0000","last_clean":"2024-09-13T18:26:36.408317+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:26:36.408317+0000","last_undegraded":"2024-09-13T18:26:36.408317+0000","last_fullsized":"2024-09-13T18:26:36.408317+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:49.198397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388733+0000","last_change":"2024-09-13T18:26:32.350841+0000","last_active":"2024-09-13T18:26:36.388733+0000","last_peered":"2024-09-13T18:26:36.388733+0000","last_clean":"2024-09-13T18:26:36.388733+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:26:36.388733+0000","last_undegraded":"2024-09-13T18:26:36.388733+0000","last_fullsized":"2024-09-13T18:26:36.388733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:05.137882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.476898+0000","last_change":"2024-09-13T18:26:31.978214+0000","last_active":"2024-09-13T18:27:03.476898+0000","last_peered":"2024-09-13T18:27:03.476898+0000","last_clean":"2024-09-13T18:27:03.476898+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:27:03.476898+0000","last_undegraded":"2024-09-13T18:27:03.476898+0000","last_fullsized":"2024-09-13T18:27:03.476898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:17.381375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768498+0000","last_change":"2024-09-13T18:26:36.406090+0000","last_active":"2024-09-13T18:26:36.768498+0000","last_peered":"2024-09-13T18:26:36.768498+0000","last_clean":"2024-09-13T18:26:36.768498+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:26:36.768498+0000","last_undegraded":"2024-09-13T18:26:36.768498+0000","last_fullsized":"2024-09-13T18:26:36.768498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:09.865147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391759+0000","last_change":"2024-09-13T18:26:32.357701+0000","last_active":"2024-09-13T18:26:36.391759+0000","last_peered":"2024-09-13T18:26:36.391759+0000","last_clean":"2024-09-13T18:26:36.391759+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:26:36.391759+0000","last_undegraded":"2024-09-13T18:26:36.391759+0000","last_fullsized":"2024-09-13T18:26:36.391759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:47.587683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384903+0000","last_change":"2024-09-13T18:26:31.980133+0000","last_active":"2024-09-13T18:26:36.384903+0000","last_peered":"2024-09-13T18:26:36.384903+0000","last_clean":"2024-09-13T18:26:36.384903+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:26:36.384903+0000","last_undegraded":"2024-09-13T18:26:36.384903+0000","last_fullsized":"2024-09-13T18:26:36.384903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:05.354335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768120+0000","last_change":"2024-09-13T18:26:36.406553+0000","last_active":"2024-09-13T18:26:36.768120+0000","last_peered":"2024-09-13T18:26:36.768120+0000","last_clean":"2024-09-13T18:26:36.768120+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:26:36.768120+0000","last_undegraded":"2024-09-13T18:26:36.768120+0000","last_fullsized":"2024-09-13T18:26:36.768120+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:02.980826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103481+0000","last_change":"2024-09-13T18:26:36.406986+0000","last_active":"2024-09-13T18:26:37.103481+0000","last_peered":"2024-09-13T18:26:37.103481+0000","last_clean":"2024-09-13T18:26:37.103481+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:26:37.103481+0000","last_undegraded":"2024-09-13T18:26:37.103481+0000","last_fullsized":"2024-09-13T18:26:37.103481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:53.984231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384974+0000","last_change":"2024-09-13T18:26:32.349081+0000","last_active":"2024-09-13T18:26:36.384974+0000","last_peered":"2024-09-13T18:26:36.384974+0000","last_clean":"2024-09-13T18:26:36.384974+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:26:36.384974+0000","last_undegraded":"2024-09-13T18:26:36.384974+0000","last_fullsized":"2024-09-13T18:26:36.384974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:47.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.771163+0000","last_change":"2024-09-13T18:26:31.968944+0000","last_active":"2024-09-13T18:26:36.771163+0000","last_peered":"2024-09-13T18:26:36.771163+0000","last_clean":"2024-09-13T18:26:36.771163+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:26:36.771163+0000","last_undegraded":"2024-09-13T18:26:36.771163+0000","last_fullsized":"2024-09-13T18:26:36.771163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:29.363312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.898021+0000","last_change":"2024-09-13T18:26:36.396418+0000","last_active":"2024-09-13T18:26:36.898021+0000","last_peered":"2024-09-13T18:26:36.898021+0000","last_clean":"2024-09-13T18:26:36.898021+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:26:36.898021+0000","last_undegraded":"2024-09-13T18:26:36.898021+0000","last_fullsized":"2024-09-13T18:26:36.898021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:03.252773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383753+0000","last_change":"2024-09-13T18:26:32.351369+0000","last_active":"2024-09-13T18:26:36.383753+0000","last_peered":"2024-09-13T18:26:36.383753+0000","last_clean":"2024-09-13T18:26:36.383753+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:26:36.383753+0000","last_undegraded":"2024-09-13T18:26:36.383753+0000","last_fullsized":"2024-09-13T18:26:36.383753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:10.892347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387713+0000","last_change":"2024-09-13T18:26:31.972048+0000","last_active":"2024-09-13T18:26:36.387713+0000","last_peered":"2024-09-13T18:26:36.387713+0000","last_clean":"2024-09-13T18:26:36.387713+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:26:36.387713+0000","last_undegraded":"2024-09-13T18:26:36.387713+0000","last_fullsized":"2024-09-13T18:26:36.387713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:28.430119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343265+0000","last_change":"2024-09-13T18:26:31.975310+0000","last_active":"2024-09-13T18:26:37.343265+0000","last_peered":"2024-09-13T18:26:37.343265+0000","last_clean":"2024-09-13T18:26:37.343265+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:26:37.343265+0000","last_undegraded":"2024-09-13T18:26:37.343265+0000","last_fullsized":"2024-09-13T18:26:37.343265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:52.915310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388817+0000","last_change":"2024-09-13T18:26:32.345825+0000","last_active":"2024-09-13T18:26:36.388817+0000","last_peered":"2024-09-13T18:26:36.388817+0000","last_clean":"2024-09-13T18:26:36.388817+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:26:36.388817+0000","last_undegraded":"2024-09-13T18:26:36.388817+0000","last_fullsized":"2024-09-13T18:26:36.388817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:25.200489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.395253+0000","last_change":"2024-09-13T18:26:36.391520+0000","last_active":"2024-09-13T18:26:36.395253+0000","last_peered":"2024-09-13T18:26:36.395253+0000","last_clean":"2024-09-13T18:26:36.395253+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:26:36.395253+0000","last_undegraded":"2024-09-13T18:26:36.395253+0000","last_fullsized":"2024-09-13T18:26:36.395253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:42.638713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.476885+0000","last_change":"2024-09-13T18:26:31.976603+0000","last_active":"2024-09-13T18:27:03.476885+0000","last_peered":"2024-09-13T18:27:03.476885+0000","last_clean":"2024-09-13T18:27:03.476885+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:27:03.476885+0000","last_undegraded":"2024-09-13T18:27:03.476885+0000","last_fullsized":"2024-09-13T18:27:03.476885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:03.965376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387960+0000","last_change":"2024-09-13T18:26:32.352982+0000","last_active":"2024-09-13T18:26:36.387960+0000","last_peered":"2024-09-13T18:26:36.387960+0000","last_clean":"2024-09-13T18:26:36.387960+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:26:36.387960+0000","last_undegraded":"2024-09-13T18:26:36.387960+0000","last_fullsized":"2024-09-13T18:26:36.387960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:55.670360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769131+0000","last_change":"2024-09-13T18:26:36.403817+0000","last_active":"2024-09-13T18:26:36.769131+0000","last_peered":"2024-09-13T18:26:36.769131+0000","last_clean":"2024-09-13T18:26:36.769131+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:26:36.769131+0000","last_undegraded":"2024-09-13T18:26:36.769131+0000","last_fullsized":"2024-09-13T18:26:36.769131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:34.892901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897728+0000","last_change":"2024-09-13T18:26:32.357856+0000","last_active":"2024-09-13T18:26:36.897728+0000","last_peered":"2024-09-13T18:26:36.897728+0000","last_clean":"2024-09-13T18:26:36.897728+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:26:36.897728+0000","last_undegraded":"2024-09-13T18:26:36.897728+0000","last_fullsized":"2024-09-13T18:26:36.897728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:26.316214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.375208+0000","last_change":"2024-09-13T18:26:32.350966+0000","last_active":"2024-09-13T18:26:36.375208+0000","last_peered":"2024-09-13T18:26:36.375208+0000","last_clean":"2024-09-13T18:26:36.375208+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:26:36.375208+0000","last_undegraded":"2024-09-13T18:26:36.375208+0000","last_fullsized":"2024-09-13T18:26:36.375208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:52.632112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391246+0000","last_change":"2024-09-13T18:26:32.355806+0000","last_active":"2024-09-13T18:26:36.391246+0000","last_peered":"2024-09-13T18:26:36.391246+0000","last_clean":"2024-09-13T18:26:36.391246+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:26:36.391246+0000","last_undegraded":"2024-09-13T18:26:36.391246+0000","last_fullsized":"2024-09-13T18:26:36.391246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.988231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388740+0000","last_change":"2024-09-13T18:26:32.340965+0000","last_active":"2024-09-13T18:26:36.388740+0000","last_peered":"2024-09-13T18:26:36.388740+0000","last_clean":"2024-09-13T18:26:36.388740+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:26:36.388740+0000","last_undegraded":"2024-09-13T18:26:36.388740+0000","last_fullsized":"2024-09-13T18:26:36.388740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:01:04.716919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384429+0000","last_change":"2024-09-13T18:26:32.349625+0000","last_active":"2024-09-13T18:26:36.384429+0000","last_peered":"2024-09-13T18:26:36.384429+0000","last_clean":"2024-09-13T18:26:36.384429+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:26:36.384429+0000","last_undegraded":"2024-09-13T18:26:36.384429+0000","last_fullsized":"2024-09-13T18:26:36.384429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.519754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386397+0000","last_change":"2024-09-13T18:26:32.349170+0000","last_active":"2024-09-13T18:26:36.386397+0000","last_peered":"2024-09-13T18:26:36.386397+0000","last_clean":"2024-09-13T18:26:36.386397+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:26:36.386397+0000","last_undegraded":"2024-09-13T18:26:36.386397+0000","last_fullsized":"2024-09-13T18:26:36.386397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:43.942732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897998+0000","last_change":"2024-09-13T18:26:32.364786+0000","last_active":"2024-09-13T18:26:36.897998+0000","last_peered":"2024-09-13T18:26:36.897998+0000","last_clean":"2024-09-13T18:26:36.897998+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:26:36.897998+0000","last_undegraded":"2024-09-13T18:26:36.897998+0000","last_fullsized":"2024-09-13T18:26:36.897998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:22.949875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387927+0000","last_change":"2024-09-13T18:26:32.347541+0000","last_active":"2024-09-13T18:26:36.387927+0000","last_peered":"2024-09-13T18:26:36.387927+0000","last_clean":"2024-09-13T18:26:36.387927+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:26:36.387927+0000","last_undegraded":"2024-09-13T18:26:36.387927+0000","last_fullsized":"2024-09-13T18:26:36.387927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:58.144128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390892+0000","last_change":"2024-09-13T18:26:32.353359+0000","last_active":"2024-09-13T18:26:36.390892+0000","last_peered":"2024-09-13T18:26:36.390892+0000","last_clean":"2024-09-13T18:26:36.390892+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:26:36.390892+0000","last_undegraded":"2024-09-13T18:26:36.390892+0000","last_fullsized":"2024-09-13T18:26:36.390892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:09.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.380537+0000","last_change":"2024-09-13T18:26:32.349566+0000","last_active":"2024-09-13T18:26:36.380537+0000","last_peered":"2024-09-13T18:26:36.380537+0000","last_clean":"2024-09-13T18:26:36.380537+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:26:36.380537+0000","last_undegraded":"2024-09-13T18:26:36.380537+0000","last_fullsized":"2024-09-13T18:26:36.380537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:22.341899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391168+0000","last_change":"2024-09-13T18:26:32.358640+0000","last_active":"2024-09-13T18:26:36.391168+0000","last_peered":"2024-09-13T18:26:36.391168+0000","last_clean":"2024-09-13T18:26:36.391168+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:26:36.391168+0000","last_undegraded":"2024-09-13T18:26:36.391168+0000","last_fullsized":"2024-09-13T18:26:36.391168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:40.174754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388633+0000","last_change":"2024-09-13T18:26:32.353675+0000","last_active":"2024-09-13T18:26:36.388633+0000","last_peered":"2024-09-13T18:26:36.388633+0000","last_clean":"2024-09-13T18:26:36.388633+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:26:36.388633+0000","last_undegraded":"2024-09-13T18:26:36.388633+0000","last_fullsized":"2024-09-13T18:26:36.388633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:58.501843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897877+0000","last_change":"2024-09-13T18:26:32.353223+0000","last_active":"2024-09-13T18:26:36.897877+0000","last_peered":"2024-09-13T18:26:36.897877+0000","last_clean":"2024-09-13T18:26:36.897877+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:26:36.897877+0000","last_undegraded":"2024-09-13T18:26:36.897877+0000","last_fullsized":"2024-09-13T18:26:36.897877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:45.898213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407736+0000","last_change":"2024-09-13T18:26:32.357354+0000","last_active":"2024-09-13T18:26:36.407736+0000","last_peered":"2024-09-13T18:26:36.407736+0000","last_clean":"2024-09-13T18:26:36.407736+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:26:36.407736+0000","last_undegraded":"2024-09-13T18:26:36.407736+0000","last_fullsized":"2024-09-13T18:26:36.407736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:19.185008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.376351+0000","last_change":"2024-09-13T18:26:32.348832+0000","last_active":"2024-09-13T18:26:36.376351+0000","last_peered":"2024-09-13T18:26:36.376351+0000","last_clean":"2024-09-13T18:26:36.376351+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:26:36.376351+0000","last_undegraded":"2024-09-13T18:26:36.376351+0000","last_fullsized":"2024-09-13T18:26:36.376351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:11:59.874789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393723+0000","last_change":"2024-09-13T18:26:32.356902+0000","last_active":"2024-09-13T18:26:36.393723+0000","last_peered":"2024-09-13T18:26:36.393723+0000","last_clean":"2024-09-13T18:26:36.393723+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:26:36.393723+0000","last_undegraded":"2024-09-13T18:26:36.393723+0000","last_fullsized":"2024-09-13T18:26:36.393723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:18.362772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390031+0000","last_change":"2024-09-13T18:26:32.347047+0000","last_active":"2024-09-13T18:26:36.390031+0000","last_peered":"2024-09-13T18:26:36.390031+0000","last_clean":"2024-09-13T18:26:36.390031+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:26:36.390031+0000","last_undegraded":"2024-09-13T18:26:36.390031+0000","last_fullsized":"2024-09-13T18:26:36.390031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:40.882789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.385066+0000","last_change":"2024-09-13T18:26:32.349651+0000","last_active":"2024-09-13T18:26:36.385066+0000","last_peered":"2024-09-13T18:26:36.385066+0000","last_clean":"2024-09-13T18:26:36.385066+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:26:36.385066+0000","last_undegraded":"2024-09-13T18:26:36.385066+0000","last_fullsized":"2024-09-13T18:26:36.385066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:33.356563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388522+0000","last_change":"2024-09-13T18:26:32.344607+0000","last_active":"2024-09-13T18:26:36.388522+0000","last_peered":"2024-09-13T18:26:36.388522+0000","last_clean":"2024-09-13T18:26:36.388522+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:26:36.388522+0000","last_undegraded":"2024-09-13T18:26:36.388522+0000","last_fullsized":"2024-09-13T18:26:36.388522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:31.658172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387737+0000","last_change":"2024-09-13T18:26:32.347717+0000","last_active":"2024-09-13T18:26:36.387737+0000","last_peered":"2024-09-13T18:26:36.387737+0000","last_clean":"2024-09-13T18:26:36.387737+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:26:36.387737+0000","last_undegraded":"2024-09-13T18:26:36.387737+0000","last_fullsized":"2024-09-13T18:26:36.387737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:47.187837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392104+0000","last_change":"2024-09-13T18:26:32.359506+0000","last_active":"2024-09-13T18:26:36.392104+0000","last_peered":"2024-09-13T18:26:36.392104+0000","last_clean":"2024-09-13T18:26:36.392104+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:26:36.392104+0000","last_undegraded":"2024-09-13T18:26:36.392104+0000","last_fullsized":"2024-09-13T18:26:36.392104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:39:57.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897951+0000","last_change":"2024-09-13T18:26:32.357729+0000","last_active":"2024-09-13T18:26:36.897951+0000","last_peered":"2024-09-13T18:26:36.897951+0000","last_clean":"2024-09-13T18:26:36.897951+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:26:36.897951+0000","last_undegraded":"2024-09-13T18:26:36.897951+0000","last_fullsized":"2024-09-13T18:26:36.897951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:05.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897974+0000","last_change":"2024-09-13T18:26:32.357980+0000","last_active":"2024-09-13T18:26:36.897974+0000","last_peered":"2024-09-13T18:26:36.897974+0000","last_clean":"2024-09-13T18:26:36.897974+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:26:36.897974+0000","last_undegraded":"2024-09-13T18:26:36.897974+0000","last_fullsized":"2024-09-13T18:26:36.897974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:00.858702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405699+0000","last_change":"2024-09-13T18:26:32.365638+0000","last_active":"2024-09-13T18:26:36.405699+0000","last_peered":"2024-09-13T18:26:36.405699+0000","last_clean":"2024-09-13T18:26:36.405699+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:26:36.405699+0000","last_undegraded":"2024-09-13T18:26:36.405699+0000","last_fullsized":"2024-09-13T18:26:36.405699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.065853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383235+0000","last_change":"2024-09-13T18:26:32.345496+0000","last_active":"2024-09-13T18:26:36.383235+0000","last_peered":"2024-09-13T18:26:36.383235+0000","last_clean":"2024-09-13T18:26:36.383235+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:26:36.383235+0000","last_undegraded":"2024-09-13T18:26:36.383235+0000","last_fullsized":"2024-09-13T18:26:36.383235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:22.447315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404895+0000","last_change":"2024-09-13T18:26:32.356901+0000","last_active":"2024-09-13T18:26:36.404895+0000","last_peered":"2024-09-13T18:26:36.404895+0000","last_clean":"2024-09-13T18:26:36.404895+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:26:36.404895+0000","last_undegraded":"2024-09-13T18:26:36.404895+0000","last_fullsized":"2024-09-13T18:26:36.404895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:03.311603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391082+0000","last_change":"2024-09-13T18:26:32.355203+0000","last_active":"2024-09-13T18:26:36.391082+0000","last_peered":"2024-09-13T18:26:36.391082+0000","last_clean":"2024-09-13T18:26:36.391082+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:26:36.391082+0000","last_undegraded":"2024-09-13T18:26:36.391082+0000","last_fullsized":"2024-09-13T18:26:36.391082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:52.769708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405315+0000","last_change":"2024-09-13T18:26:32.358190+0000","last_active":"2024-09-13T18:26:36.405315+0000","last_peered":"2024-09-13T18:26:36.405315+0000","last_clean":"2024-09-13T18:26:36.405315+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:26:36.405315+0000","last_undegraded":"2024-09-13T18:26:36.405315+0000","last_fullsized":"2024-09-13T18:26:36.405315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:35.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393258+0000","last_change":"2024-09-13T18:26:32.355121+0000","last_active":"2024-09-13T18:26:36.393258+0000","last_peered":"2024-09-13T18:26:36.393258+0000","last_clean":"2024-09-13T18:26:36.393258+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:26:36.393258+0000","last_undegraded":"2024-09-13T18:26:36.393258+0000","last_fullsized":"2024-09-13T18:26:36.393258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:48.427332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388810+0000","last_change":"2024-09-13T18:26:32.341497+0000","last_active":"2024-09-13T18:26:36.388810+0000","last_peered":"2024-09-13T18:26:36.388810+0000","last_clean":"2024-09-13T18:26:36.388810+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:26:36.388810+0000","last_undegraded":"2024-09-13T18:26:36.388810+0000","last_fullsized":"2024-09-13T18:26:36.388810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:15.510048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386627+0000","last_change":"2024-09-13T18:26:32.350106+0000","last_active":"2024-09-13T18:26:36.386627+0000","last_peered":"2024-09-13T18:26:36.386627+0000","last_clean":"2024-09-13T18:26:36.386627+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:26:36.386627+0000","last_undegraded":"2024-09-13T18:26:36.386627+0000","last_fullsized":"2024-09-13T18:26:36.386627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:12.387451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405597+0000","last_change":"2024-09-13T18:26:32.362196+0000","last_active":"2024-09-13T18:26:36.405597+0000","last_peered":"2024-09-13T18:26:36.405597+0000","last_clean":"2024-09-13T18:26:36.405597+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:26:36.405597+0000","last_undegraded":"2024-09-13T18:26:36.405597+0000","last_fullsized":"2024-09-13T18:26:36.405597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:14.946904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5861,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":11812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]}]},{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":344064,"data_stored":694809,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":103299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]}]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":107542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":258048,"data_stored":103173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":270336,"data_stored":103935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":104005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":339968,"data_stored":694201,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:27:07.531 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T18:27:07.531 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T18:27:07.532 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T18:27:07.532 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T18:27:07.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.625+0000 7f3837fff700 1 Processor -- start 2024-09-13T18:27:07.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.625+0000 7f3837fff700 1 -- start start 2024-09-13T18:27:07.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3837fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3838153bc0 0x7f38380c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3837fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 0x7f38380c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3837fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3838153fa0 con 0x7f38380cac60 2024-09-13T18:27:07.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3837fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38380aa500 con 0x7f3838153bc0 2024-09-13T18:27:07.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3837fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383815a7e0 con 0x7f38380c6610 2024-09-13T18:27:07.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f38367fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 0x7f38380c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3836ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3838153bc0 0x7f38380c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f38377fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f38380cac60 0x7f38380c7740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:55058/0 (socket says 172.21.15.146:55058) 2024-09-13T18:27:07.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f38377fe700 1 -- 172.21.15.146:0/2159012950 learned_addr learned my addr 172.21.15.146:0/2159012950 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:07.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3836ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3838153bc0 0x7f38380c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:46942/0 (socket says 172.21.15.146:46942) 2024-09-13T18:27:07.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f38367fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 0x7f38380c6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41570/0 (socket says 172.21.15.146:41570) 2024-09-13T18:27:07.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3836ffd700 1 -- 172.21.15.146:0/2159012950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 msgr2=0x7f38380c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:07.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3836ffd700 1 --2- 172.21.15.146:0/2159012950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 0x7f38380c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3836ffd700 1 -- 172.21.15.146:0/2159012950 >> v1:172.21.15.154:6789/0 conn(0x7f38380cac60 legacy=0x7f38380c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3836ffd700 1 -- 172.21.15.146:0/2159012950 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380c9320 con 0x7f3838153bc0 2024-09-13T18:27:07.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3835ffb700 1 -- 172.21.15.146:0/2159012950 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032186963 0 0) 0x7f3838153fa0 con 0x7f38380cac60 2024-09-13T18:27:07.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f38367fc700 1 --2- 172.21.15.146:0/2159012950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 0x7f38380c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:07.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3836ffd700 1 --2- 172.21.15.146:0/2159012950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838153bc0 0x7f38380c8de0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3820009850 tx=0x7f3820009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f7401645441333e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3835ffb700 1 -- 172.21.15.146:0/2159012950 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3820011830 con 0x7f3838153bc0 2024-09-13T18:27:07.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3835ffb700 1 -- 172.21.15.146:0/2159012950 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3820011990 con 0x7f3838153bc0 2024-09-13T18:27:07.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3835ffb700 1 -- 172.21.15.146:0/2159012950 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3820011d40 con 0x7f3838153bc0 2024-09-13T18:27:07.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3837fff700 1 -- 172.21.15.146:0/2159012950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838153bc0 msgr2=0x7f38380c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.629+0000 7f3837fff700 1 --2- 172.21.15.146:0/2159012950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838153bc0 0x7f38380c8de0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3820009850 tx=0x7f3820009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:07.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.633+0000 7f3837fff700 1 -- 172.21.15.146:0/2159012950 shutdown_connections 2024-09-13T18:27:07.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.633+0000 7f3837fff700 1 --2- 172.21.15.146:0/2159012950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 0x7f38380c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.633+0000 7f3837fff700 1 --2- 172.21.15.146:0/2159012950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838153bc0 0x7f38380c8de0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.633+0000 7f3837fff700 1 -- 172.21.15.146:0/2159012950 >> 172.21.15.146:0/2159012950 conn(0x7f3838066090 msgr2=0x7f3838066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:07.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.633+0000 7f3837fff700 1 -- 172.21.15.146:0/2159012950 shutdown_connections 2024-09-13T18:27:07.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.633+0000 7f3837fff700 1 -- 172.21.15.146:0/2159012950 wait complete. 2024-09-13T18:27:07.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.633+0000 7f3837fff700 1 Processor -- start 2024-09-13T18:27:07.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.633+0000 7f3837fff700 1 -- start start 2024-09-13T18:27:07.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3837fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 0x7f38381ecdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3837fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38380cac60 0x7f38380c0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3837fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838153bc0 0x7f38381504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3837fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38381ec8e0 con 0x7f38380cac60 2024-09-13T18:27:07.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3837fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38381ef630 con 0x7f3838153bc0 2024-09-13T18:27:07.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3837fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38381ed330 con 0x7f38380c6610 2024-09-13T18:27:07.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3836ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 0x7f38381ecdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f38367fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38380cac60 0x7f38380c0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3836ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 0x7f38381ecdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41572/0 (socket says 172.21.15.146:41572) 2024-09-13T18:27:07.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3836ffd700 1 -- 172.21.15.146:0/2473785082 learned_addr learned my addr 172.21.15.146:0/2473785082 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:07.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f38377fe700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838153bc0 0x7f38381504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3836ffd700 1 -- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838153bc0 msgr2=0x7f38381504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3836ffd700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838153bc0 0x7f38381504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3836ffd700 1 -- 172.21.15.146:0/2473785082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38380cac60 msgr2=0x7f38380c0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3836ffd700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38380cac60 0x7f38380c0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3836ffd700 1 -- 172.21.15.146:0/2473785082 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38280094b0 con 0x7f38380c6610 2024-09-13T18:27:07.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f38367fc700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38380cac60 0x7f38380c0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:07.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3836ffd700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 0x7f38381ecdf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3820000c00 tx=0x7f3820004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f382ffff700 1 -- 172.21.15.146:0/2473785082 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382001b980 con 0x7f38380c6610 2024-09-13T18:27:07.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f382ffff700 1 -- 172.21.15.146:0/2473785082 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382001bae0 con 0x7f38380c6610 2024-09-13T18:27:07.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3837fff700 1 -- 172.21.15.146:0/2473785082 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38200094b0 con 0x7f38380c6610 2024-09-13T18:27:07.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f382ffff700 1 -- 172.21.15.146:0/2473785082 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382001be90 con 0x7f38380c6610 2024-09-13T18:27:07.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3837fff700 1 -- 172.21.15.146:0/2473785082 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38381ef020 con 0x7f38380c6610 2024-09-13T18:27:07.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.637+0000 7f3837fff700 1 -- 172.21.15.146:0/2473785082 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f383815a2c0 con 0x7f38380c6610 2024-09-13T18:27:07.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.641+0000 7f382ffff700 1 -- 172.21.15.146:0/2473785082 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3820023070 con 0x7f38380c6610 2024-09-13T18:27:07.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.641+0000 7f382ffff700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3818066760 0x7f3818068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.641+0000 7f382ffff700 1 -- 172.21.15.146:0/2473785082 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3820087230 con 0x7f38380c6610 2024-09-13T18:27:07.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.641+0000 7f38367fc700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3818066760 0x7f3818068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.641+0000 7f38367fc700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3818066760 0x7f3818068c20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f38380b0190 tx=0x7f3828009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.649+0000 7f382ffff700 1 -- 172.21.15.146:0/2473785082 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3820054340 con 0x7f38380c6610 2024-09-13T18:27:07.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.837+0000 7f3837fff700 1 -- 172.21.15.146:0/2473785082 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f383806f690 con 0x7f38380c6610 2024-09-13T18:27:07.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.837+0000 7f382ffff700 1 -- 172.21.15.146:0/2473785082 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3820057960 con 0x7f38380c6610 2024-09-13T18:27:07.844 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:07.845 INFO:teuthology.orchestra.run.smithi146.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T18:27:07.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.841+0000 7f3837fff700 1 -- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3818066760 msgr2=0x7f3818068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.841+0000 7f3837fff700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3818066760 0x7f3818068c20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f38380b0190 tx=0x7f3828009240 comp rx=0 tx=0).stop 2024-09-13T18:27:07.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.841+0000 7f3837fff700 1 -- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 msgr2=0x7f38381ecdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.841+0000 7f3837fff700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 0x7f38381ecdf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3820000c00 tx=0x7f3820004770 comp rx=0 tx=0).stop 2024-09-13T18:27:07.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.841+0000 7f3837fff700 1 -- 172.21.15.146:0/2473785082 shutdown_connections 2024-09-13T18:27:07.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.841+0000 7f3837fff700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3818066760 0x7f3818068c20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.841+0000 7f3837fff700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838153bc0 0x7f38381504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.841+0000 7f3837fff700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38380cac60 0x7f38380c0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.841+0000 7f3837fff700 1 --2- 172.21.15.146:0/2473785082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38380c6610 0x7f38381ecdf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.841+0000 7f3837fff700 1 -- 172.21.15.146:0/2473785082 >> 172.21.15.146:0/2473785082 conn(0x7f3838066090 msgr2=0x7f38380c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:07.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.841+0000 7f3837fff700 1 -- 172.21.15.146:0/2473785082 shutdown_connections 2024-09-13T18:27:07.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.841+0000 7f3837fff700 1 -- 172.21.15.146:0/2473785082 wait complete. 2024-09-13T18:27:07.858 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T18:27:07.859 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:27:07.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e355d6700 1 Processor -- start 2024-09-13T18:27:07.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e355d6700 1 -- start start 2024-09-13T18:27:07.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e355d6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300c48b0 0x7f5e3015e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e355d6700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e300ca260 con 0x7f5e300c9930 2024-09-13T18:27:07.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e355d6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e300ca540 con 0x7f5e300cbd30 2024-09-13T18:27:07.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e355d6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300c5b00 con 0x7f5e300c48b0 2024-09-13T18:27:07.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e34dd5700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5e300c9930 0x7f5e30168170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35594/0 (socket says 172.21.15.146:35594) 2024-09-13T18:27:07.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e34dd5700 1 -- 172.21.15.146:0/3511210980 learned_addr learned my addr 172.21.15.146:0/3511210980 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:07.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e2f7fe700 1 --2- 172.21.15.146:0/3511210980 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300c48b0 0x7f5e3015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e2f7fe700 1 -- 172.21.15.146:0/3511210980 >> v1:172.21.15.154:6789/0 conn(0x7f5e300cbd30 legacy=0x7f5e300c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e2f7fe700 1 -- 172.21.15.146:0/3511210980 >> v1:172.21.15.146:6789/0 conn(0x7f5e300c9930 legacy=0x7f5e30168170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e2f7fe700 1 -- 172.21.15.146:0/3511210980 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e300ca820 con 0x7f5e300c48b0 2024-09-13T18:27:07.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e2f7fe700 1 --2- 172.21.15.146:0/3511210980 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300c48b0 0x7f5e3015e290 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009dc0 tx=0x7f5e2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc35cbac6ec8f39c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e2effd700 1 -- 172.21.15.146:0/3511210980 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2000e070 con 0x7f5e300c48b0 2024-09-13T18:27:07.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e2effd700 1 -- 172.21.15.146:0/3511210980 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2000bbd0 con 0x7f5e300c48b0 2024-09-13T18:27:07.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e2effd700 1 -- 172.21.15.146:0/3511210980 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20012430 con 0x7f5e300c48b0 2024-09-13T18:27:07.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e355d6700 1 -- 172.21.15.146:0/3511210980 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300c48b0 msgr2=0x7f5e3015e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e355d6700 1 --2- 172.21.15.146:0/3511210980 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300c48b0 0x7f5e3015e290 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009dc0 tx=0x7f5e2000b330 comp rx=0 tx=0).stop 2024-09-13T18:27:07.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e355d6700 1 -- 172.21.15.146:0/3511210980 shutdown_connections 2024-09-13T18:27:07.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e355d6700 1 --2- 172.21.15.146:0/3511210980 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300c48b0 0x7f5e3015e290 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e355d6700 1 -- 172.21.15.146:0/3511210980 >> 172.21.15.146:0/3511210980 conn(0x7f5e30066270 msgr2=0x7f5e30066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:07.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e355d6700 1 -- 172.21.15.146:0/3511210980 shutdown_connections 2024-09-13T18:27:07.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.953+0000 7f5e355d6700 1 -- 172.21.15.146:0/3511210980 wait complete. 2024-09-13T18:27:07.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e355d6700 1 Processor -- start 2024-09-13T18:27:07.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e355d6700 1 -- start start 2024-09-13T18:27:07.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e355d6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300c48b0 0x7f5e301e9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e355d6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e300c9930 0x7f5e3015c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e355d6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e300cbd30 0x7f5e3015c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e355d6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300c7680 con 0x7f5e300cbd30 2024-09-13T18:27:07.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e355d6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300c7b70 con 0x7f5e300c9930 2024-09-13T18:27:07.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e355d6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300c8a70 con 0x7f5e300c48b0 2024-09-13T18:27:07.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e34dd5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e300cbd30 0x7f5e3015c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e34dd5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e300cbd30 0x7f5e3015c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54910/0 (socket says 172.21.15.146:54910) 2024-09-13T18:27:07.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e2ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300c48b0 0x7f5e301e9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e34dd5700 1 -- 172.21.15.146:0/702514983 learned_addr learned my addr 172.21.15.146:0/702514983 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:07.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e2f7fe700 1 --2- 172.21.15.146:0/702514983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e300c9930 0x7f5e3015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e34dd5700 1 -- 172.21.15.146:0/702514983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300c48b0 msgr2=0x7f5e301e9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e34dd5700 1 --2- 172.21.15.146:0/702514983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300c48b0 0x7f5e301e9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e34dd5700 1 -- 172.21.15.146:0/702514983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e300c9930 msgr2=0x7f5e3015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e34dd5700 1 --2- 172.21.15.146:0/702514983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e300c9930 0x7f5e3015c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e34dd5700 1 -- 172.21.15.146:0/702514983 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e20009a70 con 0x7f5e300cbd30 2024-09-13T18:27:07.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e34dd5700 1 --2- 172.21.15.146:0/702514983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e300cbd30 0x7f5e3015c5b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800c4d0 tx=0x7f5e2800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e2cff9700 1 -- 172.21.15.146:0/702514983 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28005d90 con 0x7f5e300cbd30 2024-09-13T18:27:07.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e2cff9700 1 -- 172.21.15.146:0/702514983 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e280070d0 con 0x7f5e300cbd30 2024-09-13T18:27:07.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e355d6700 1 -- 172.21.15.146:0/702514983 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e300c68e0 con 0x7f5e300cbd30 2024-09-13T18:27:07.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e2cff9700 1 -- 172.21.15.146:0/702514983 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28007440 con 0x7f5e300cbd30 2024-09-13T18:27:07.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.957+0000 7f5e355d6700 1 -- 172.21.15.146:0/702514983 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e301f49c0 con 0x7f5e300cbd30 2024-09-13T18:27:07.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.961+0000 7f5e355d6700 1 -- 172.21.15.146:0/702514983 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3015e290 con 0x7f5e300cbd30 2024-09-13T18:27:07.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.961+0000 7f5e2cff9700 1 -- 172.21.15.146:0/702514983 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5e280075a0 con 0x7f5e300cbd30 2024-09-13T18:27:07.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.961+0000 7f5e2cff9700 1 --2- 172.21.15.146:0/702514983 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e1c066760 0x7f5e1c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.961+0000 7f5e2cff9700 1 -- 172.21.15.146:0/702514983 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5e28083670 con 0x7f5e300cbd30 2024-09-13T18:27:07.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.961+0000 7f5e2ffff700 1 --2- 172.21.15.146:0/702514983 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e1c066760 0x7f5e1c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.965+0000 7f5e2ffff700 1 --2- 172.21.15.146:0/702514983 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e1c066760 0x7f5e1c068c20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5e24000f60 tx=0x7f5e24009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:07.969+0000 7f5e2cff9700 1 -- 172.21.15.146:0/702514983 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5e28050830 con 0x7f5e300cbd30 2024-09-13T18:27:08.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.133+0000 7f5e355d6700 1 -- 172.21.15.146:0/702514983 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e3015b0e0 con 0x7f5e300cbd30 2024-09-13T18:27:08.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.133+0000 7f5e2cff9700 1 -- 172.21.15.146:0/702514983 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f5e28053e50 con 0x7f5e300cbd30 2024-09-13T18:27:08.141 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:08.141 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:27:08.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.137+0000 7f5e355d6700 1 -- 172.21.15.146:0/702514983 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e1c066760 msgr2=0x7f5e1c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.137+0000 7f5e355d6700 1 --2- 172.21.15.146:0/702514983 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e1c066760 0x7f5e1c068c20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5e24000f60 tx=0x7f5e24009040 comp rx=0 tx=0).stop 2024-09-13T18:27:08.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.137+0000 7f5e355d6700 1 -- 172.21.15.146:0/702514983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e300cbd30 msgr2=0x7f5e3015c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.137+0000 7f5e355d6700 1 --2- 172.21.15.146:0/702514983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e300cbd30 0x7f5e3015c5b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800c4d0 tx=0x7f5e2800c7e0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.137+0000 7f5e355d6700 1 -- 172.21.15.146:0/702514983 shutdown_connections 2024-09-13T18:27:08.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.137+0000 7f5e355d6700 1 --2- 172.21.15.146:0/702514983 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e1c066760 0x7f5e1c068c20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.137+0000 7f5e355d6700 1 --2- 172.21.15.146:0/702514983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e300cbd30 0x7f5e3015c5b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.137+0000 7f5e355d6700 1 --2- 172.21.15.146:0/702514983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e300c9930 0x7f5e3015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.137+0000 7f5e355d6700 1 --2- 172.21.15.146:0/702514983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300c48b0 0x7f5e301e9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.137+0000 7f5e355d6700 1 -- 172.21.15.146:0/702514983 >> 172.21.15.146:0/702514983 conn(0x7f5e30066270 msgr2=0x7f5e300ccd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:08.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.137+0000 7f5e355d6700 1 -- 172.21.15.146:0/702514983 shutdown_connections 2024-09-13T18:27:08.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.137+0000 7f5e355d6700 1 -- 172.21.15.146:0/702514983 wait complete. 2024-09-13T18:27:08.154 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:26:24.867515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:26:30.782269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:26:31.323277+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:26:34.393386+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:27:08.155 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:27:08.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabffff700 1 Processor -- start 2024-09-13T18:27:08.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabffff700 1 -- start start 2024-09-13T18:27:08.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcac01539e0 0x7fcac00b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcac00cf580 0x7fcac00cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac00cff70 con 0x7fcac015d960 2024-09-13T18:27:08.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabffff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac00cc3a0 con 0x7fcac01539e0 2024-09-13T18:27:08.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00c5e50 con 0x7fcac00cf580 2024-09-13T18:27:08.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabe7fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fcac015d960 0x7fcac00c9fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35596/0 (socket says 172.21.15.146:35596) 2024-09-13T18:27:08.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabeffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcac01539e0 0x7fcac00b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabe7fc700 1 -- 172.21.15.146:0/909620277 learned_addr learned my addr 172.21.15.146:0/909620277 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:08.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabf7fe700 1 --2- 172.21.15.146:0/909620277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcac00cf580 0x7fcac00cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabdffb700 1 -- 172.21.15.146:0/909620277 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257607245 0 0) 0x7fcac00cff70 con 0x7fcac015d960 2024-09-13T18:27:08.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabdffb700 1 -- 172.21.15.146:0/909620277 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaa80035d0 con 0x7fcac015d960 2024-09-13T18:27:08.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabeffd700 1 -- 172.21.15.146:0/909620277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcac00cf580 msgr2=0x7fcac00cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:08.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabeffd700 1 --2- 172.21.15.146:0/909620277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcac00cf580 0x7fcac00cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabeffd700 1 -- 172.21.15.146:0/909620277 >> v1:172.21.15.146:6789/0 conn(0x7fcac015d960 legacy=0x7fcac00c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabeffd700 1 -- 172.21.15.146:0/909620277 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac00cb480 con 0x7fcac01539e0 2024-09-13T18:27:08.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.269+0000 7fcabeffd700 1 --2- 172.21.15.146:0/909620277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac01539e0 0x7fcac00b3370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009880 tx=0x7fcab4009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0fee5943951dff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:08.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.273+0000 7fcabdffb700 1 -- 172.21.15.146:0/909620277 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4011830 con 0x7fcac01539e0 2024-09-13T18:27:08.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.273+0000 7fcabdffb700 1 -- 172.21.15.146:0/909620277 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab4011990 con 0x7fcac01539e0 2024-09-13T18:27:08.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.273+0000 7fcabdffb700 1 -- 172.21.15.146:0/909620277 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4011d20 con 0x7fcac01539e0 2024-09-13T18:27:08.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.273+0000 7fcabffff700 1 -- 172.21.15.146:0/909620277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac01539e0 msgr2=0x7fcac00b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.273+0000 7fcabffff700 1 --2- 172.21.15.146:0/909620277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac01539e0 0x7fcac00b3370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009880 tx=0x7fcab4009b90 comp rx=0 tx=0).stop 2024-09-13T18:27:08.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.273+0000 7fcabffff700 1 -- 172.21.15.146:0/909620277 shutdown_connections 2024-09-13T18:27:08.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.273+0000 7fcabffff700 1 --2- 172.21.15.146:0/909620277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcac00cf580 0x7fcac00cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.273+0000 7fcabffff700 1 --2- 172.21.15.146:0/909620277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac01539e0 0x7fcac00b3370 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.273+0000 7fcabffff700 1 -- 172.21.15.146:0/909620277 >> 172.21.15.146:0/909620277 conn(0x7fcac0071ec0 msgr2=0x7fcac00722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:08.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.273+0000 7fcabffff700 1 -- 172.21.15.146:0/909620277 shutdown_connections 2024-09-13T18:27:08.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.273+0000 7fcabffff700 1 -- 172.21.15.146:0/909620277 wait complete. 2024-09-13T18:27:08.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.273+0000 7fcabffff700 1 Processor -- start 2024-09-13T18:27:08.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.273+0000 7fcabffff700 1 -- start start 2024-09-13T18:27:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcac00cf580 0x7fcac01ef270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac01539e0 0x7fcac014e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcac015d960 0x7fcac00cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01ef7b0 con 0x7fcac01539e0 2024-09-13T18:27:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00c8c30 con 0x7fcac00cf580 2024-09-13T18:27:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00c9440 con 0x7fcac015d960 2024-09-13T18:27:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabe7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac01539e0 0x7fcac014e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabe7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac01539e0 0x7fcac014e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54914/0 (socket says 172.21.15.146:54914) 2024-09-13T18:27:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabe7fc700 1 -- 172.21.15.146:0/2504098709 learned_addr learned my addr 172.21.15.146:0/2504098709 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabf7fe700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcac015d960 0x7fcac00cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabeffd700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcac00cf580 0x7fcac01ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabe7fc700 1 -- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcac015d960 msgr2=0x7fcac00cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabe7fc700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcac015d960 0x7fcac00cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabe7fc700 1 -- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcac00cf580 msgr2=0x7fcac01ef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabe7fc700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcac00cf580 0x7fcac01ef270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabe7fc700 1 -- 172.21.15.146:0/2504098709 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab40094b0 con 0x7fcac01539e0 2024-09-13T18:27:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabe7fc700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac01539e0 0x7fcac014e210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcab000c500 tx=0x7fcab000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcaa7fff700 1 -- 172.21.15.146:0/2504098709 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0005d90 con 0x7fcac01539e0 2024-09-13T18:27:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcaa7fff700 1 -- 172.21.15.146:0/2504098709 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab00070d0 con 0x7fcac01539e0 2024-09-13T18:27:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcaa7fff700 1 -- 172.21.15.146:0/2504098709 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0007480 con 0x7fcac01539e0 2024-09-13T18:27:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabffff700 1 -- 172.21.15.146:0/2504098709 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac00c78a0 con 0x7fcac01539e0 2024-09-13T18:27:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabffff700 1 -- 172.21.15.146:0/2504098709 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac00b4300 con 0x7fcac01539e0 2024-09-13T18:27:08.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.277+0000 7fcabffff700 1 -- 172.21.15.146:0/2504098709 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac0153dc0 con 0x7fcac01539e0 2024-09-13T18:27:08.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.281+0000 7fcaa7fff700 1 -- 172.21.15.146:0/2504098709 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcab00075e0 con 0x7fcac01539e0 2024-09-13T18:27:08.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.281+0000 7fcaa7fff700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcaac066730 0x7fcaac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.281+0000 7fcaa7fff700 1 -- 172.21.15.146:0/2504098709 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcab0083780 con 0x7fcac01539e0 2024-09-13T18:27:08.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.281+0000 7fcabeffd700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcaac066730 0x7fcaac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.281+0000 7fcabeffd700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcaac066730 0x7fcaac068bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009bd0 tx=0x7fcab4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:08.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.285+0000 7fcaa7fff700 1 -- 172.21.15.146:0/2504098709 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcab0050890 con 0x7fcac01539e0 2024-09-13T18:27:08.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.457+0000 7fcabffff700 1 -- 172.21.15.146:0/2504098709 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcac00c8e80 con 0x7fcac01539e0 2024-09-13T18:27:08.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.457+0000 7fcaa7fff700 1 -- 172.21.15.146:0/2504098709 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcab0053eb0 con 0x7fcac01539e0 2024-09-13T18:27:08.463 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-09-13T18:27:08.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.461+0000 7fcabffff700 1 -- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcaac066730 msgr2=0x7fcaac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.461+0000 7fcabffff700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcaac066730 0x7fcaac068bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009bd0 tx=0x7fcab4008030 comp rx=0 tx=0).stop 2024-09-13T18:27:08.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.461+0000 7fcabffff700 1 -- 172.21.15.146:0/2504098709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac01539e0 msgr2=0x7fcac014e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.461+0000 7fcabffff700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac01539e0 0x7fcac014e210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcab000c500 tx=0x7fcab000c810 comp rx=0 tx=0).stop 2024-09-13T18:27:08.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.461+0000 7fcabffff700 1 -- 172.21.15.146:0/2504098709 shutdown_connections 2024-09-13T18:27:08.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.461+0000 7fcabffff700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcaac066730 0x7fcaac068bf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.461+0000 7fcabffff700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcac015d960 0x7fcac00cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.461+0000 7fcabffff700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac01539e0 0x7fcac014e210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.461+0000 7fcabffff700 1 --2- 172.21.15.146:0/2504098709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcac00cf580 0x7fcac01ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.461+0000 7fcabffff700 1 -- 172.21.15.146:0/2504098709 >> 172.21.15.146:0/2504098709 conn(0x7fcac0071ec0 msgr2=0x7fcac00722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:08.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.461+0000 7fcabffff700 1 -- 172.21.15.146:0/2504098709 shutdown_connections 2024-09-13T18:27:08.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.461+0000 7fcabffff700 1 -- 172.21.15.146:0/2504098709 wait complete. 2024-09-13T18:27:08.477 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:27:08.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 Processor -- start 2024-09-13T18:27:08.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 -- start start 2024-09-13T18:27:08.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2fd40cab20 0x7f2fd415b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40caf00 0x7f2fd4157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd40a97e0 con 0x7f2fd40cb300 2024-09-13T18:27:08.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd40c99a0 con 0x7f2fd40cab20 2024-09-13T18:27:08.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40723d0 con 0x7f2fd40caf00 2024-09-13T18:27:08.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fd95a1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2fd40cab20 0x7f2fd415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fd9da2700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f2fd40cb300 0x7f2fd41544a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35604/0 (socket says 172.21.15.146:35604) 2024-09-13T18:27:08.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fd9da2700 1 -- 172.21.15.146:0/4153521421 learned_addr learned my addr 172.21.15.146:0/4153521421 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:08.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fd95a1700 1 -- 172.21.15.146:0/4153521421 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40caf00 msgr2=0x7f2fd4157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:08.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fd95a1700 1 --2- 172.21.15.146:0/4153521421 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40caf00 0x7f2fd4157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fd95a1700 1 -- 172.21.15.146:0/4153521421 >> v1:172.21.15.146:6789/0 conn(0x7f2fd40cb300 legacy=0x7f2fd41544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fd95a1700 1 -- 172.21.15.146:0/4153521421 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd415b8c0 con 0x7f2fd40cab20 2024-09-13T18:27:08.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fcbfff700 1 -- 172.21.15.146:0/4153521421 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396343192 0 0) 0x7f2fd40a97e0 con 0x7f2fd40cb300 2024-09-13T18:27:08.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fd95a1700 1 --2- 172.21.15.146:0/4153521421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd40cab20 0x7f2fd415b380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009d90 tx=0x7f2fcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378b0614b24d3d58 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:08.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fcbfff700 1 -- 172.21.15.146:0/4153521421 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc00e070 con 0x7f2fd40cab20 2024-09-13T18:27:08.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fcbfff700 1 -- 172.21.15.146:0/4153521421 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc00bbd0 con 0x7f2fd40cab20 2024-09-13T18:27:08.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fcbfff700 1 -- 172.21.15.146:0/4153521421 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc012430 con 0x7f2fd40cab20 2024-09-13T18:27:08.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 -- 172.21.15.146:0/4153521421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd40cab20 msgr2=0x7f2fd415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 --2- 172.21.15.146:0/4153521421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd40cab20 0x7f2fd415b380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009d90 tx=0x7f2fcc00b330 comp rx=0 tx=0).stop 2024-09-13T18:27:08.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 -- 172.21.15.146:0/4153521421 shutdown_connections 2024-09-13T18:27:08.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 --2- 172.21.15.146:0/4153521421 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40caf00 0x7f2fd4157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 --2- 172.21.15.146:0/4153521421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd40cab20 0x7f2fd415b380 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 -- 172.21.15.146:0/4153521421 >> 172.21.15.146:0/4153521421 conn(0x7f2fd4066090 msgr2=0x7f2fd4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.589+0000 7f2fda5a3700 1 -- 172.21.15.146:0/4153521421 shutdown_connections 2024-09-13T18:27:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fda5a3700 1 -- 172.21.15.146:0/4153521421 wait complete. 2024-09-13T18:27:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fda5a3700 1 Processor -- start 2024-09-13T18:27:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fda5a3700 1 -- start start 2024-09-13T18:27:08.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fda5a3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd40cab20 0x7f2fd40af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fda5a3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd40caf00 0x7f2fd41eff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fda5a3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40cb300 0x7f2fd40c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fda5a3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd41e0f30 con 0x7f2fd40caf00 2024-09-13T18:27:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fda5a3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd41e6510 con 0x7f2fd40cab20 2024-09-13T18:27:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fda5a3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd41f2110 con 0x7f2fd40cb300 2024-09-13T18:27:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fd8da0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd40caf00 0x7f2fd41eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fd8da0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd40caf00 0x7f2fd41eff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54934/0 (socket says 172.21.15.146:54934) 2024-09-13T18:27:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fd8da0700 1 -- 172.21.15.146:0/1501351409 learned_addr learned my addr 172.21.15.146:0/1501351409 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fd95a1700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd40cab20 0x7f2fd40af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fd9da2700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40cb300 0x7f2fd40c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fd8da0700 1 -- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40cb300 msgr2=0x7f2fd40c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fd8da0700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40cb300 0x7f2fd40c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fd8da0700 1 -- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd40cab20 msgr2=0x7f2fd40af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fd8da0700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd40cab20 0x7f2fd40af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fd8da0700 1 -- 172.21.15.146:0/1501351409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc009a70 con 0x7f2fd40caf00 2024-09-13T18:27:08.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fd95a1700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd40cab20 0x7f2fd40af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:08.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fd8da0700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd40caf00 0x7f2fd41eff70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400b550 tx=0x7f2fc400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:08.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fc9ffb700 1 -- 172.21.15.146:0/1501351409 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4011830 con 0x7f2fd40caf00 2024-09-13T18:27:08.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fc9ffb700 1 -- 172.21.15.146:0/1501351409 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc4011990 con 0x7f2fd40caf00 2024-09-13T18:27:08.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fda5a3700 1 -- 172.21.15.146:0/1501351409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd41f23f0 con 0x7f2fd40caf00 2024-09-13T18:27:08.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.593+0000 7f2fc9ffb700 1 -- 172.21.15.146:0/1501351409 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4011d40 con 0x7f2fd40caf00 2024-09-13T18:27:08.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.597+0000 7f2fda5a3700 1 -- 172.21.15.146:0/1501351409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd41f2780 con 0x7f2fd40caf00 2024-09-13T18:27:08.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.597+0000 7f2fc9ffb700 1 -- 172.21.15.146:0/1501351409 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2fc401a400 con 0x7f2fd40caf00 2024-09-13T18:27:08.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.597+0000 7f2fda5a3700 1 -- 172.21.15.146:0/1501351409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd41f2f90 con 0x7f2fd40caf00 2024-09-13T18:27:08.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.597+0000 7f2fc9ffb700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2fb8066730 0x7f2fb8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.597+0000 7f2fc9ffb700 1 -- 172.21.15.146:0/1501351409 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2fc4083750 con 0x7f2fd40caf00 2024-09-13T18:27:08.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.597+0000 7f2fd95a1700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2fb8066730 0x7f2fb8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.597+0000 7f2fd95a1700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2fb8066730 0x7f2fb8068bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00b370 tx=0x7f2fcc011000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:08.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.601+0000 7f2fc9ffb700 1 -- 172.21.15.146:0/1501351409 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2fc4050860 con 0x7f2fd40caf00 2024-09-13T18:27:08.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.769+0000 7f2fda5a3700 1 -- 172.21.15.146:0/1501351409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2fd41f2bc0 con 0x7f2fd40caf00 2024-09-13T18:27:08.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.769+0000 7f2fc9ffb700 1 -- 172.21.15.146:0/1501351409 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2fc4053e80 con 0x7f2fd40caf00 2024-09-13T18:27:08.777 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:27:08.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.773+0000 7f2fda5a3700 1 -- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2fb8066730 msgr2=0x7f2fb8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.773+0000 7f2fda5a3700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2fb8066730 0x7f2fb8068bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00b370 tx=0x7f2fcc011000 comp rx=0 tx=0).stop 2024-09-13T18:27:08.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.773+0000 7f2fda5a3700 1 -- 172.21.15.146:0/1501351409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd40caf00 msgr2=0x7f2fd41eff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.773+0000 7f2fda5a3700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd40caf00 0x7f2fd41eff70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400b550 tx=0x7f2fc400b910 comp rx=0 tx=0).stop 2024-09-13T18:27:08.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.773+0000 7f2fda5a3700 1 -- 172.21.15.146:0/1501351409 shutdown_connections 2024-09-13T18:27:08.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.773+0000 7f2fda5a3700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2fb8066730 0x7f2fb8068bf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.773+0000 7f2fda5a3700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40cb300 0x7f2fd40c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.773+0000 7f2fda5a3700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd40caf00 0x7f2fd41eff70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.773+0000 7f2fda5a3700 1 --2- 172.21.15.146:0/1501351409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd40cab20 0x7f2fd40af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.773+0000 7f2fda5a3700 1 -- 172.21.15.146:0/1501351409 >> 172.21.15.146:0/1501351409 conn(0x7f2fd4066090 msgr2=0x7f2fd4156700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:08.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.773+0000 7f2fda5a3700 1 -- 172.21.15.146:0/1501351409 shutdown_connections 2024-09-13T18:27:08.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.773+0000 7f2fda5a3700 1 -- 172.21.15.146:0/1501351409 wait complete. 2024-09-13T18:27:08.791 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:27:08.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.905+0000 7f90d08b5700 1 Processor -- start 2024-09-13T18:27:08.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.905+0000 7f90d08b5700 1 -- start start 2024-09-13T18:27:08.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90d08b5700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f90cc1539e0 0x7f90cc0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90d08b5700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90cc0c5e50 con 0x7f90cc15d960 2024-09-13T18:27:08.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90d08b5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90cc0cb480 con 0x7f90cc0cf580 2024-09-13T18:27:08.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90d08b5700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90cc0cc3a0 con 0x7f90cc1539e0 2024-09-13T18:27:08.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90caffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f90cc15d960 0x7f90cc0c9fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35610/0 (socket says 172.21.15.146:35610) 2024-09-13T18:27:08.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90caffd700 1 -- 172.21.15.146:0/1109597502 learned_addr learned my addr 172.21.15.146:0/1109597502 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:08.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90cb7fe700 1 --2- 172.21.15.146:0/1109597502 >> v2:172.21.15.146:3300/0 conn(0x7f90cc1539e0 0x7f90cc0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90ca7fc700 1 -- 172.21.15.146:0/1109597502 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038262639 0 0) 0x7f90bc001110 con 0x7f90cc15d960 2024-09-13T18:27:08.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90ca7fc700 1 -- 172.21.15.146:0/1109597502 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90b40035d0 con 0x7f90cc15d960 2024-09-13T18:27:08.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90ca7fc700 1 -- 172.21.15.146:0/1109597502 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403124023 0 0) 0x7f90cc0cb480 con 0x7f90cc0cf580 2024-09-13T18:27:08.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90cb7fe700 1 -- 172.21.15.146:0/1109597502 >> v1:172.21.15.154:6789/0 conn(0x7f90cc0cf580 legacy=0x7f90cc0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90cb7fe700 1 -- 172.21.15.146:0/1109597502 >> v1:172.21.15.146:6789/0 conn(0x7f90cc15d960 legacy=0x7f90cc0c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90cb7fe700 1 -- 172.21.15.146:0/1109597502 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc0cb720 con 0x7f90cc1539e0 2024-09-13T18:27:08.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90cb7fe700 1 --2- 172.21.15.146:0/1109597502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc1539e0 0x7f90cc0b3370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009880 tx=0x7f90c0009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6906aef74778ac5f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:08.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90ca7fc700 1 -- 172.21.15.146:0/1109597502 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c0011830 con 0x7f90cc1539e0 2024-09-13T18:27:08.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90ca7fc700 1 -- 172.21.15.146:0/1109597502 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c0011990 con 0x7f90cc1539e0 2024-09-13T18:27:08.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90ca7fc700 1 -- 172.21.15.146:0/1109597502 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c0011d20 con 0x7f90cc1539e0 2024-09-13T18:27:08.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90d08b5700 1 -- 172.21.15.146:0/1109597502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc1539e0 msgr2=0x7f90cc0b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90d08b5700 1 --2- 172.21.15.146:0/1109597502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc1539e0 0x7f90cc0b3370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009880 tx=0x7f90c0009b90 comp rx=0 tx=0).stop 2024-09-13T18:27:08.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90d08b5700 1 -- 172.21.15.146:0/1109597502 shutdown_connections 2024-09-13T18:27:08.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90d08b5700 1 --2- 172.21.15.146:0/1109597502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc1539e0 0x7f90cc0b3370 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90d08b5700 1 -- 172.21.15.146:0/1109597502 >> 172.21.15.146:0/1109597502 conn(0x7f90cc071ec0 msgr2=0x7f90cc0722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:08.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90d08b5700 1 -- 172.21.15.146:0/1109597502 shutdown_connections 2024-09-13T18:27:08.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.909+0000 7f90d08b5700 1 -- 172.21.15.146:0/1109597502 wait complete. 2024-09-13T18:27:08.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90d08b5700 1 Processor -- start 2024-09-13T18:27:08.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90d08b5700 1 -- start start 2024-09-13T18:27:08.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90d08b5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc0cf580 0x7f90cc154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90d08b5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90cc1539e0 0x7f90cc1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90d08b5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90cc15d960 0x7f90cc14ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90d08b5700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc1f2a90 con 0x7f90cc0cf580 2024-09-13T18:27:08.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90d08b5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc156a00 con 0x7f90cc15d960 2024-09-13T18:27:08.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90d08b5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc148600 con 0x7f90cc1539e0 2024-09-13T18:27:08.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90cb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc0cf580 0x7f90cc154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90cb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc0cf580 0x7f90cc154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54946/0 (socket says 172.21.15.146:54946) 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90cb7fe700 1 -- 172.21.15.146:0/1538192665 learned_addr learned my addr 172.21.15.146:0/1538192665 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90caffd700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90cc1539e0 0x7f90cc1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90cbfff700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90cc15d960 0x7f90cc14ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90cb7fe700 1 -- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90cc1539e0 msgr2=0x7f90cc1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90cb7fe700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90cc1539e0 0x7f90cc1564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90cb7fe700 1 -- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90cc15d960 msgr2=0x7f90cc14ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90cb7fe700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90cc15d960 0x7f90cc14ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90cb7fe700 1 -- 172.21.15.146:0/1538192665 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c00094b0 con 0x7f90cc0cf580 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90cbfff700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90cc15d960 0x7f90cc14ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.913+0000 7f90cb7fe700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc0cf580 0x7f90cc154080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f90c0000c00 tx=0x7f90c0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.917+0000 7f90b3fff700 1 -- 172.21.15.146:0/1538192665 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c001aac0 con 0x7f90cc0cf580 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.917+0000 7f90b3fff700 1 -- 172.21.15.146:0/1538192665 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c001f070 con 0x7f90cc0cf580 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.917+0000 7f90d08b5700 1 -- 172.21.15.146:0/1538192665 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90cc15c8b0 con 0x7f90cc0cf580 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.917+0000 7f90b3fff700 1 -- 172.21.15.146:0/1538192665 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c001ae60 con 0x7f90cc0cf580 2024-09-13T18:27:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.917+0000 7f90d08b5700 1 -- 172.21.15.146:0/1538192665 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc15c5d0 con 0x7f90cc0cf580 2024-09-13T18:27:08.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.917+0000 7f90d08b5700 1 -- 172.21.15.146:0/1538192665 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc1f2bd0 con 0x7f90cc0cf580 2024-09-13T18:27:08.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.917+0000 7f90b3fff700 1 -- 172.21.15.146:0/1538192665 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f90c002e450 con 0x7f90cc0cf580 2024-09-13T18:27:08.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.917+0000 7f90b3fff700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f90b8066730 0x7f90b8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:08.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.917+0000 7f90b3fff700 1 -- 172.21.15.146:0/1538192665 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f90c0085490 con 0x7f90cc0cf580 2024-09-13T18:27:08.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.917+0000 7f90caffd700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f90b8066730 0x7f90b8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:08.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.921+0000 7f90caffd700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f90b8066730 0x7f90b8068bf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0b4d10 tx=0x7f90bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:08.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:08.925+0000 7f90b3fff700 1 -- 172.21.15.146:0/1538192665 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f90cc1f2bd0 con 0x7f90cc0cf580 2024-09-13T18:27:09.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.093+0000 7f90d08b5700 1 -- 172.21.15.146:0/1538192665 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f90cc0b29f0 con 0x7f90cc0cf580 2024-09-13T18:27:09.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.093+0000 7f90b3fff700 1 -- 172.21.15.146:0/1538192665 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f90c0021020 con 0x7f90cc0cf580 2024-09-13T18:27:09.101 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-09-13T18:27:09.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.097+0000 7f90d08b5700 1 -- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f90b8066730 msgr2=0x7f90b8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.097+0000 7f90d08b5700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f90b8066730 0x7f90b8068bf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0b4d10 tx=0x7f90bc009040 comp rx=0 tx=0).stop 2024-09-13T18:27:09.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.097+0000 7f90d08b5700 1 -- 172.21.15.146:0/1538192665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc0cf580 msgr2=0x7f90cc154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.097+0000 7f90d08b5700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc0cf580 0x7f90cc154080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f90c0000c00 tx=0x7f90c0004770 comp rx=0 tx=0).stop 2024-09-13T18:27:09.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.097+0000 7f90d08b5700 1 -- 172.21.15.146:0/1538192665 shutdown_connections 2024-09-13T18:27:09.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.097+0000 7f90d08b5700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f90b8066730 0x7f90b8068bf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.097+0000 7f90d08b5700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90cc15d960 0x7f90cc14ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.097+0000 7f90d08b5700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90cc1539e0 0x7f90cc1564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.097+0000 7f90d08b5700 1 --2- 172.21.15.146:0/1538192665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc0cf580 0x7f90cc154080 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.097+0000 7f90d08b5700 1 -- 172.21.15.146:0/1538192665 >> 172.21.15.146:0/1538192665 conn(0x7f90cc071ec0 msgr2=0x7f90cc0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:09.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.097+0000 7f90d08b5700 1 -- 172.21.15.146:0/1538192665 shutdown_connections 2024-09-13T18:27:09.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.097+0000 7f90d08b5700 1 -- 172.21.15.146:0/1538192665 wait complete. 2024-09-13T18:27:09.115 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:27:09.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.229+0000 7f5c4517b700 1 Processor -- start 2024-09-13T18:27:09.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.229+0000 7f5c4517b700 1 -- start start 2024-09-13T18:27:09.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c4517b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c40150410 0x7f5c4015a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c4517b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400c7780 0x7f5c400c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c4517b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4015a650 con 0x7f5c400ae4c0 2024-09-13T18:27:09.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c4517b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c400ccd80 con 0x7f5c40150410 2024-09-13T18:27:09.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c4517b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4006f8b0 con 0x7f5c400c7780 2024-09-13T18:27:09.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3e59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5c400ae4c0 0x7f5c400ae8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35614/0 (socket says 172.21.15.146:35614) 2024-09-13T18:27:09.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3e59c700 1 -- 172.21.15.146:0/851145875 learned_addr learned my addr 172.21.15.146:0/851145875 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:09.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3ed9d700 1 --2- 172.21.15.146:0/851145875 >> v2:172.21.15.146:3300/0 conn(0x7f5c40150410 0x7f5c4015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3f59e700 1 --2- 172.21.15.146:0/851145875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400c7780 0x7f5c400c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3dd9b700 1 -- 172.21.15.146:0/851145875 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251784800 0 0) 0x7f5c4015a650 con 0x7f5c400ae4c0 2024-09-13T18:27:09.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3dd9b700 1 -- 172.21.15.146:0/851145875 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c280035d0 con 0x7f5c400ae4c0 2024-09-13T18:27:09.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3ed9d700 1 -- 172.21.15.146:0/851145875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400c7780 msgr2=0x7f5c400c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:09.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3ed9d700 1 --2- 172.21.15.146:0/851145875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400c7780 0x7f5c400c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3ed9d700 1 -- 172.21.15.146:0/851145875 >> v1:172.21.15.146:6789/0 conn(0x7f5c400ae4c0 legacy=0x7f5c400ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3ed9d700 1 -- 172.21.15.146:0/851145875 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4006fb90 con 0x7f5c40150410 2024-09-13T18:27:09.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3ed9d700 1 --2- 172.21.15.146:0/851145875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c40150410 0x7f5c4015a110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009850 tx=0x7f5c34009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c647daec15e575e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:09.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3dd9b700 1 -- 172.21.15.146:0/851145875 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34011830 con 0x7f5c40150410 2024-09-13T18:27:09.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3dd9b700 1 -- 172.21.15.146:0/851145875 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c34011990 con 0x7f5c40150410 2024-09-13T18:27:09.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c3dd9b700 1 -- 172.21.15.146:0/851145875 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34011d40 con 0x7f5c40150410 2024-09-13T18:27:09.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c4517b700 1 -- 172.21.15.146:0/851145875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c40150410 msgr2=0x7f5c4015a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c4517b700 1 --2- 172.21.15.146:0/851145875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c40150410 0x7f5c4015a110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009850 tx=0x7f5c34009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:09.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c4517b700 1 -- 172.21.15.146:0/851145875 shutdown_connections 2024-09-13T18:27:09.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c4517b700 1 --2- 172.21.15.146:0/851145875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400c7780 0x7f5c400c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c4517b700 1 --2- 172.21.15.146:0/851145875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c40150410 0x7f5c4015a110 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c4517b700 1 -- 172.21.15.146:0/851145875 >> 172.21.15.146:0/851145875 conn(0x7f5c400cc140 msgr2=0x7f5c400cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:09.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c4517b700 1 -- 172.21.15.146:0/851145875 shutdown_connections 2024-09-13T18:27:09.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.233+0000 7f5c4517b700 1 -- 172.21.15.146:0/851145875 wait complete. 2024-09-13T18:27:09.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c4517b700 1 Processor -- start 2024-09-13T18:27:09.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c4517b700 1 -- start start 2024-09-13T18:27:09.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c4517b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ae4c0 0x7f5c401e5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c4517b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c400c7780 0x7f5c400af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c4517b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c40150410 0x7f5c400af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c4517b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c401e2970 con 0x7f5c40150410 2024-09-13T18:27:09.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c4517b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c401e35b0 con 0x7f5c400c7780 2024-09-13T18:27:09.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c4517b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c400b0060 con 0x7f5c400ae4c0 2024-09-13T18:27:09.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c3f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c40150410 0x7f5c400af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c3f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c40150410 0x7f5c400af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54956/0 (socket says 172.21.15.146:54956) 2024-09-13T18:27:09.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c3f59e700 1 -- 172.21.15.146:0/770395064 learned_addr learned my addr 172.21.15.146:0/770395064 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:09.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c3ed9d700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ae4c0 0x7f5c401e5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c3e59c700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c400c7780 0x7f5c400af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c3f59e700 1 -- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ae4c0 msgr2=0x7f5c401e5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c3f59e700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ae4c0 0x7f5c401e5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c3f59e700 1 -- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c400c7780 msgr2=0x7f5c400af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c3f59e700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c400c7780 0x7f5c400af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c3f59e700 1 -- 172.21.15.146:0/770395064 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c340094b0 con 0x7f5c40150410 2024-09-13T18:27:09.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c3e59c700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c400c7780 0x7f5c400af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:09.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c3ed9d700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ae4c0 0x7f5c401e5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:09.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.237+0000 7f5c3f59e700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c40150410 0x7f5c400af8c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800efc0 tx=0x7f5c3800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:09.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.241+0000 7f5c277fe700 1 -- 172.21.15.146:0/770395064 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38009ba0 con 0x7f5c40150410 2024-09-13T18:27:09.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.241+0000 7f5c277fe700 1 -- 172.21.15.146:0/770395064 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3800f040 con 0x7f5c40150410 2024-09-13T18:27:09.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.241+0000 7f5c4517b700 1 -- 172.21.15.146:0/770395064 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c401e2e60 con 0x7f5c40150410 2024-09-13T18:27:09.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.241+0000 7f5c277fe700 1 -- 172.21.15.146:0/770395064 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38014480 con 0x7f5c40150410 2024-09-13T18:27:09.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.241+0000 7f5c4517b700 1 -- 172.21.15.146:0/770395064 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c401e3240 con 0x7f5c40150410 2024-09-13T18:27:09.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.241+0000 7f5c4517b700 1 -- 172.21.15.146:0/770395064 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4006f5c0 con 0x7f5c40150410 2024-09-13T18:27:09.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.241+0000 7f5c277fe700 1 -- 172.21.15.146:0/770395064 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5c380145e0 con 0x7f5c40150410 2024-09-13T18:27:09.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.245+0000 7f5c277fe700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c2c066760 0x7f5c2c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.245+0000 7f5c277fe700 1 -- 172.21.15.146:0/770395064 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5c38084c70 con 0x7f5c40150410 2024-09-13T18:27:09.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.245+0000 7f5c3ed9d700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c2c066760 0x7f5c2c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.245+0000 7f5c3ed9d700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c2c066760 0x7f5c2c068c20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5c340090b0 tx=0x7f5c34008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:09.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.249+0000 7f5c277fe700 1 -- 172.21.15.146:0/770395064 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5c38051ce0 con 0x7f5c40150410 2024-09-13T18:27:09.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.417+0000 7f5c4517b700 1 -- 172.21.15.146:0/770395064 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5c40072980 con 0x7f5c40150410 2024-09-13T18:27:09.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.417+0000 7f5c277fe700 1 -- 172.21.15.146:0/770395064 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5c38018020 con 0x7f5c40150410 2024-09-13T18:27:09.423 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:27:09.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.421+0000 7f5c4517b700 1 -- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c2c066760 msgr2=0x7f5c2c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.421+0000 7f5c4517b700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c2c066760 0x7f5c2c068c20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5c340090b0 tx=0x7f5c34008030 comp rx=0 tx=0).stop 2024-09-13T18:27:09.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.421+0000 7f5c4517b700 1 -- 172.21.15.146:0/770395064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c40150410 msgr2=0x7f5c400af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.421+0000 7f5c4517b700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c40150410 0x7f5c400af8c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800efc0 tx=0x7f5c3800c3a0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.421+0000 7f5c4517b700 1 -- 172.21.15.146:0/770395064 shutdown_connections 2024-09-13T18:27:09.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.421+0000 7f5c4517b700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c2c066760 0x7f5c2c068c20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.421+0000 7f5c4517b700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c40150410 0x7f5c400af8c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.421+0000 7f5c4517b700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c400c7780 0x7f5c400af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.421+0000 7f5c4517b700 1 --2- 172.21.15.146:0/770395064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ae4c0 0x7f5c401e5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.421+0000 7f5c4517b700 1 -- 172.21.15.146:0/770395064 >> 172.21.15.146:0/770395064 conn(0x7f5c400cc140 msgr2=0x7f5c40071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:09.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.421+0000 7f5c4517b700 1 -- 172.21.15.146:0/770395064 shutdown_connections 2024-09-13T18:27:09.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.421+0000 7f5c4517b700 1 -- 172.21.15.146:0/770395064 wait complete. 2024-09-13T18:27:09.438 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:27:09.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.541+0000 7f6585c08700 1 Processor -- start 2024-09-13T18:27:09.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.541+0000 7f6585c08700 1 -- start start 2024-09-13T18:27:09.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.541+0000 7f6585c08700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6578050380 0x7f65780de7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.541+0000 7f6585c08700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6578049270 0x7f65780ded20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.541+0000 7f6585c08700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6578050900 con 0x7f65780d8920 2024-09-13T18:27:09.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.541+0000 7f6585c08700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65780e2990 con 0x7f6578050380 2024-09-13T18:27:09.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.541+0000 7f6585c08700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65780e1d10 con 0x7f6578049270 2024-09-13T18:27:09.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.541+0000 7f6584c06700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f65780d8920 0x7f6578049a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:55092/0 (socket says 172.21.15.146:55092) 2024-09-13T18:27:09.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.541+0000 7f6584c06700 1 -- 172.21.15.146:0/349856736 learned_addr learned my addr 172.21.15.146:0/349856736 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:09.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.541+0000 7f6585407700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6578049270 0x7f65780ded20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.541+0000 7f657ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6578050380 0x7f65780de7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f657f7fe700 1 -- 172.21.15.146:0/349856736 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812182394 0 0) 0x7f6578050900 con 0x7f65780d8920 2024-09-13T18:27:09.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f657f7fe700 1 -- 172.21.15.146:0/349856736 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65700035d0 con 0x7f65780d8920 2024-09-13T18:27:09.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585407700 1 -- 172.21.15.146:0/349856736 >> v2:172.21.15.154:3300/0 conn(0x7f6578050380 msgr2=0x7f65780de7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:09.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585407700 1 --2- 172.21.15.146:0/349856736 >> v2:172.21.15.154:3300/0 conn(0x7f6578050380 0x7f65780de7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585407700 1 -- 172.21.15.146:0/349856736 >> v1:172.21.15.154:6789/0 conn(0x7f65780d8920 legacy=0x7f6578049a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585407700 1 -- 172.21.15.146:0/349856736 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65780e1ff0 con 0x7f6578049270 2024-09-13T18:27:09.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585407700 1 --2- 172.21.15.146:0/349856736 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6578049270 0x7f65780ded20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6574009850 tx=0x7f6574009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb033933d53785c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:09.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f657f7fe700 1 -- 172.21.15.146:0/349856736 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574011830 con 0x7f6578049270 2024-09-13T18:27:09.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f657f7fe700 1 -- 172.21.15.146:0/349856736 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6574011990 con 0x7f6578049270 2024-09-13T18:27:09.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f657f7fe700 1 -- 172.21.15.146:0/349856736 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574011d00 con 0x7f6578049270 2024-09-13T18:27:09.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 -- 172.21.15.146:0/349856736 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6578049270 msgr2=0x7f65780ded20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 --2- 172.21.15.146:0/349856736 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6578049270 0x7f65780ded20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6574009850 tx=0x7f6574009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:09.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 -- 172.21.15.146:0/349856736 shutdown_connections 2024-09-13T18:27:09.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 --2- 172.21.15.146:0/349856736 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6578049270 0x7f65780ded20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 --2- 172.21.15.146:0/349856736 >> v2:172.21.15.154:3300/0 conn(0x7f6578050380 0x7f65780de7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 -- 172.21.15.146:0/349856736 >> 172.21.15.146:0/349856736 conn(0x7f6578054500 msgr2=0x7f6578054910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:09.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 -- 172.21.15.146:0/349856736 shutdown_connections 2024-09-13T18:27:09.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 -- 172.21.15.146:0/349856736 wait complete. 2024-09-13T18:27:09.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 Processor -- start 2024-09-13T18:27:09.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 -- start start 2024-09-13T18:27:09.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6578049270 0x7f657804f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6578050380 0x7f6578053260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65780d8920 0x7f65780c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65780d00b0 con 0x7f6578049270 2024-09-13T18:27:09.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6578051f60 con 0x7f6578050380 2024-09-13T18:27:09.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585c08700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65780d08d0 con 0x7f65780d8920 2024-09-13T18:27:09.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6584c06700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6578049270 0x7f657804f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6584c06700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6578049270 0x7f657804f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54962/0 (socket says 172.21.15.146:54962) 2024-09-13T18:27:09.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6584c06700 1 -- 172.21.15.146:0/989059492 learned_addr learned my addr 172.21.15.146:0/989059492 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:09.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f6585407700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65780d8920 0x7f65780c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.545+0000 7f657ffff700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6578050380 0x7f6578053260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f6584c06700 1 -- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65780d8920 msgr2=0x7f65780c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f6584c06700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65780d8920 0x7f65780c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f6584c06700 1 -- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6578050380 msgr2=0x7f6578053260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f6584c06700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6578050380 0x7f6578053260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f6584c06700 1 -- 172.21.15.146:0/989059492 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65740094b0 con 0x7f6578049270 2024-09-13T18:27:09.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f6584c06700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6578049270 0x7f657804f850 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6580074d00 tx=0x7f658004f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:09.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f657d7fa700 1 -- 172.21.15.146:0/989059492 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658006f270 con 0x7f6578049270 2024-09-13T18:27:09.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f657d7fa700 1 -- 172.21.15.146:0/989059492 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6580067c50 con 0x7f6578049270 2024-09-13T18:27:09.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f6585c08700 1 -- 172.21.15.146:0/989059492 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6578052350 con 0x7f6578049270 2024-09-13T18:27:09.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f657d7fa700 1 -- 172.21.15.146:0/989059492 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580067fc0 con 0x7f6578049270 2024-09-13T18:27:09.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f6585c08700 1 -- 172.21.15.146:0/989059492 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65780d0300 con 0x7f6578049270 2024-09-13T18:27:09.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f6585c08700 1 -- 172.21.15.146:0/989059492 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6578052870 con 0x7f6578049270 2024-09-13T18:27:09.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f657d7fa700 1 -- 172.21.15.146:0/989059492 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f658006f5c0 con 0x7f6578049270 2024-09-13T18:27:09.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f657d7fa700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6564066730 0x7f6564068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f657d7fa700 1 -- 172.21.15.146:0/989059492 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f65800fed20 con 0x7f6578049270 2024-09-13T18:27:09.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.549+0000 7f657ffff700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6564066730 0x7f6564068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.553+0000 7f657ffff700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6564066730 0x7f6564068bf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6578006fc0 tx=0x7f656c009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:09.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.557+0000 7f657d7fa700 1 -- 172.21.15.146:0/989059492 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6578052870 con 0x7f6578049270 2024-09-13T18:27:09.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.765+0000 7f6585c08700 1 -- 172.21.15.146:0/989059492 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6578052870 con 0x7f6578049270 2024-09-13T18:27:09.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.765+0000 7f657d7fa700 1 -- 172.21.15.146:0/989059492 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6286 (secure 0 0 0) 0x7f6578052870 con 0x7f6578049270 2024-09-13T18:27:09.771 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:09.771 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:27:09.524480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"e","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:27:09.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.769+0000 7f6585c08700 1 -- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6564066730 msgr2=0x7f6564068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.769+0000 7f6585c08700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6564066730 0x7f6564068bf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6578006fc0 tx=0x7f656c009240 comp rx=0 tx=0).stop 2024-09-13T18:27:09.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.769+0000 7f6585c08700 1 -- 172.21.15.146:0/989059492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6578049270 msgr2=0x7f657804f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.769+0000 7f6585c08700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6578049270 0x7f657804f850 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6580074d00 tx=0x7f658004f750 comp rx=0 tx=0).stop 2024-09-13T18:27:09.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.769+0000 7f6585c08700 1 -- 172.21.15.146:0/989059492 shutdown_connections 2024-09-13T18:27:09.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.769+0000 7f6585c08700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6564066730 0x7f6564068bf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.769+0000 7f6585c08700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65780d8920 0x7f65780c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.769+0000 7f6585c08700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6578050380 0x7f6578053260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.769+0000 7f6585c08700 1 --2- 172.21.15.146:0/989059492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6578049270 0x7f657804f850 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.769+0000 7f6585c08700 1 -- 172.21.15.146:0/989059492 >> 172.21.15.146:0/989059492 conn(0x7f6578054500 msgr2=0x7f657804e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:09.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.769+0000 7f6585c08700 1 -- 172.21.15.146:0/989059492 shutdown_connections 2024-09-13T18:27:09.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.769+0000 7f6585c08700 1 -- 172.21.15.146:0/989059492 wait complete. 2024-09-13T18:27:09.775 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 15 2024-09-13T18:27:09.786 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:27:09.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.893+0000 7ff37359e700 1 Processor -- start 2024-09-13T18:27:09.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.893+0000 7ff37359e700 1 -- start start 2024-09-13T18:27:09.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff3740caed0 0x7ff3740723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3740cb2b0 0x7ff3741619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37359e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff374161550 con 0x7ff3740a9840 2024-09-13T18:27:09.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff374072a30 con 0x7ff3740caed0 2024-09-13T18:27:09.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3740aa800 con 0x7ff3740cb2b0 2024-09-13T18:27:09.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37259c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff3740caed0 0x7ff3740723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37259c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff3740caed0 0x7ff3740723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54970/0 (socket says 172.21.15.146:54970) 2024-09-13T18:27:09.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37259c700 1 -- 172.21.15.146:0/116837794 learned_addr learned my addr 172.21.15.146:0/116837794 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:09.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff371d9b700 1 --2- 172.21.15.146:0/116837794 >> v2:172.21.15.154:3300/0 conn(0x7ff3740cb2b0 0x7ff3741619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37259c700 1 -- 172.21.15.146:0/116837794 >> v2:172.21.15.154:3300/0 conn(0x7ff3740cb2b0 msgr2=0x7ff3741619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:09.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37259c700 1 --2- 172.21.15.146:0/116837794 >> v2:172.21.15.154:3300/0 conn(0x7ff3740cb2b0 0x7ff3741619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37259c700 1 -- 172.21.15.146:0/116837794 >> v1:172.21.15.154:6789/0 conn(0x7ff3740a9840 legacy=0x7ff37415e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37259c700 1 -- 172.21.15.146:0/116837794 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff374161220 con 0x7ff3740caed0 2024-09-13T18:27:09.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37259c700 1 --2- 172.21.15.146:0/116837794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740caed0 0x7ff3740723f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff374070b80 tx=0x7ff36c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cdf05a718942cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:09.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37159a700 1 -- 172.21.15.146:0/116837794 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00e070 con 0x7ff3740caed0 2024-09-13T18:27:09.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37159a700 1 -- 172.21.15.146:0/116837794 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c00b910 con 0x7ff3740caed0 2024-09-13T18:27:09.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37159a700 1 -- 172.21.15.146:0/116837794 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00bcc0 con 0x7ff3740caed0 2024-09-13T18:27:09.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37359e700 1 -- 172.21.15.146:0/116837794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740caed0 msgr2=0x7ff3740723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37359e700 1 --2- 172.21.15.146:0/116837794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740caed0 0x7ff3740723f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff374070b80 tx=0x7ff36c00b330 comp rx=0 tx=0).stop 2024-09-13T18:27:09.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37359e700 1 -- 172.21.15.146:0/116837794 shutdown_connections 2024-09-13T18:27:09.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37359e700 1 --2- 172.21.15.146:0/116837794 >> v2:172.21.15.154:3300/0 conn(0x7ff3740cb2b0 0x7ff3741619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37359e700 1 --2- 172.21.15.146:0/116837794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740caed0 0x7ff3740723f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37359e700 1 -- 172.21.15.146:0/116837794 >> 172.21.15.146:0/116837794 conn(0x7ff374066090 msgr2=0x7ff374066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:09.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37359e700 1 -- 172.21.15.146:0/116837794 shutdown_connections 2024-09-13T18:27:09.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.897+0000 7ff37359e700 1 -- 172.21.15.146:0/116837794 wait complete. 2024-09-13T18:27:09.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff37359e700 1 Processor -- start 2024-09-13T18:27:09.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff37359e700 1 -- start start 2024-09-13T18:27:09.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff37359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3740a9840 0x7ff3741eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff37359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740caed0 0x7ff374153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff37359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3740cb2b0 0x7ff3741541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff37359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3741f7c20 con 0x7ff3740caed0 2024-09-13T18:27:09.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff37359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3741e7630 con 0x7ff3740cb2b0 2024-09-13T18:27:09.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff37359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37406e8f0 con 0x7ff3740a9840 2024-09-13T18:27:09.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff371d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740caed0 0x7ff374153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff37259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3740a9840 0x7ff3741eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff371d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740caed0 0x7ff374153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54986/0 (socket says 172.21.15.146:54986) 2024-09-13T18:27:09.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff371d9b700 1 -- 172.21.15.146:0/2018220135 learned_addr learned my addr 172.21.15.146:0/2018220135 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:09.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff372d9d700 1 --2- 172.21.15.146:0/2018220135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3740cb2b0 0x7ff3741541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff371d9b700 1 -- 172.21.15.146:0/2018220135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3740a9840 msgr2=0x7ff3741eaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff371d9b700 1 --2- 172.21.15.146:0/2018220135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3740a9840 0x7ff3741eaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff371d9b700 1 -- 172.21.15.146:0/2018220135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3740cb2b0 msgr2=0x7ff3741541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:09.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff371d9b700 1 --2- 172.21.15.146:0/2018220135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3740cb2b0 0x7ff3741541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:09.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff371d9b700 1 -- 172.21.15.146:0/2018220135 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36c009b30 con 0x7ff3740caed0 2024-09-13T18:27:09.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff371d9b700 1 --2- 172.21.15.146:0/2018220135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740caed0 0x7ff374153ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00e960 tx=0x7ff35c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:09.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff36affd700 1 -- 172.21.15.146:0/2018220135 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c013680 con 0x7ff3740caed0 2024-09-13T18:27:09.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff36affd700 1 -- 172.21.15.146:0/2018220135 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35c0137e0 con 0x7ff3740caed0 2024-09-13T18:27:09.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff37359e700 1 -- 172.21.15.146:0/2018220135 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3741f6e20 con 0x7ff3740caed0 2024-09-13T18:27:09.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff36affd700 1 -- 172.21.15.146:0/2018220135 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c013b50 con 0x7ff3740caed0 2024-09-13T18:27:09.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.901+0000 7ff37359e700 1 -- 172.21.15.146:0/2018220135 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3741f7380 con 0x7ff3740caed0 2024-09-13T18:27:09.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.905+0000 7ff37359e700 1 -- 172.21.15.146:0/2018220135 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3740ca4f0 con 0x7ff3740caed0 2024-09-13T18:27:09.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.905+0000 7ff36affd700 1 -- 172.21.15.146:0/2018220135 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff35c013cb0 con 0x7ff3740caed0 2024-09-13T18:27:09.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.905+0000 7ff36affd700 1 --2- 172.21.15.146:0/2018220135 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff358066730 0x7ff358068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:09.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.905+0000 7ff36affd700 1 -- 172.21.15.146:0/2018220135 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff35c0843a0 con 0x7ff3740caed0 2024-09-13T18:27:09.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.905+0000 7ff37259c700 1 --2- 172.21.15.146:0/2018220135 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff358066730 0x7ff358068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:09.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.905+0000 7ff37259c700 1 --2- 172.21.15.146:0/2018220135 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff358066730 0x7ff358068bf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff36c000f80 tx=0x7ff36c009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:09.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:09.909+0000 7ff36affd700 1 -- 172.21.15.146:0/2018220135 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff35c051420 con 0x7ff3740caed0 2024-09-13T18:27:10.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.085+0000 7ff37359e700 1 -- 172.21.15.146:0/2018220135 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff37415db90 con 0x7ff3740caed0 2024-09-13T18:27:10.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.089+0000 7ff36affd700 1 -- 172.21.15.146:0/2018220135 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7ff35c054a40 con 0x7ff3740caed0 2024-09-13T18:27:10.095 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:10.096 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:27:10.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.093+0000 7ff37359e700 1 -- 172.21.15.146:0/2018220135 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff358066730 msgr2=0x7ff358068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.093+0000 7ff37359e700 1 --2- 172.21.15.146:0/2018220135 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff358066730 0x7ff358068bf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff36c000f80 tx=0x7ff36c009000 comp rx=0 tx=0).stop 2024-09-13T18:27:10.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.093+0000 7ff37359e700 1 -- 172.21.15.146:0/2018220135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740caed0 msgr2=0x7ff374153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.093+0000 7ff37359e700 1 --2- 172.21.15.146:0/2018220135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740caed0 0x7ff374153ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00e960 tx=0x7ff35c00ec70 comp rx=0 tx=0).stop 2024-09-13T18:27:10.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.093+0000 7ff37359e700 1 -- 172.21.15.146:0/2018220135 shutdown_connections 2024-09-13T18:27:10.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.093+0000 7ff37359e700 1 --2- 172.21.15.146:0/2018220135 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff358066730 0x7ff358068bf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.093+0000 7ff37359e700 1 --2- 172.21.15.146:0/2018220135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3740cb2b0 0x7ff3741541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.093+0000 7ff37359e700 1 --2- 172.21.15.146:0/2018220135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740caed0 0x7ff374153ca0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.093+0000 7ff37359e700 1 --2- 172.21.15.146:0/2018220135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3740a9840 0x7ff3741eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.093+0000 7ff37359e700 1 -- 172.21.15.146:0/2018220135 >> 172.21.15.146:0/2018220135 conn(0x7ff374066090 msgr2=0x7ff374070210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:10.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.093+0000 7ff37359e700 1 -- 172.21.15.146:0/2018220135 shutdown_connections 2024-09-13T18:27:10.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.093+0000 7ff37359e700 1 -- 172.21.15.146:0/2018220135 wait complete. 2024-09-13T18:27:10.110 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:26:24.867515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:26:30.782269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:26:31.323277+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:26:34.393386+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:27:10.111 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:27:10.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.225+0000 7f33b0dd3700 1 Processor -- start 2024-09-13T18:27:10.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.225+0000 7f33b0dd3700 1 -- start start 2024-09-13T18:27:10.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.225+0000 7f33b0dd3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f33ac0cab20 0x7f33ac15b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.225+0000 7f33b0dd3700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac0a97e0 con 0x7f33ac0caf00 2024-09-13T18:27:10.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.225+0000 7f33b0dd3700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac15b8c0 con 0x7f33ac1544a0 2024-09-13T18:27:10.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.225+0000 7f33b0dd3700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ac0c99a0 con 0x7f33ac0cab20 2024-09-13T18:27:10.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.225+0000 7f33aaffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f33ac0caf00 0x7f33ac157af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35616/0 (socket says 172.21.15.146:35616) 2024-09-13T18:27:10.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.225+0000 7f33ab7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f33ac0cab20 0x7f33ac15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.225+0000 7f33aaffd700 1 -- 172.21.15.146:0/892397332 learned_addr learned my addr 172.21.15.146:0/892397332 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:10.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.225+0000 7f33aa7fc700 1 -- 172.21.15.146:0/892397332 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401010664 0 0) 0x7f33ac0a97e0 con 0x7f33ac0caf00 2024-09-13T18:27:10.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.225+0000 7f33aa7fc700 1 -- 172.21.15.146:0/892397332 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f338c0035d0 con 0x7f33ac0caf00 2024-09-13T18:27:10.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33aa7fc700 1 -- 172.21.15.146:0/892397332 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504042678 0 0) 0x7f33a4001490 con 0x7f33ac1544a0 2024-09-13T18:27:10.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33aa7fc700 1 -- 172.21.15.146:0/892397332 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33ac0a97e0 con 0x7f33ac1544a0 2024-09-13T18:27:10.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33ab7fe700 1 -- 172.21.15.146:0/892397332 >> v1:172.21.15.154:6789/0 conn(0x7f33ac1544a0 legacy=0x7f33ac1548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33ab7fe700 1 -- 172.21.15.146:0/892397332 >> v1:172.21.15.146:6789/0 conn(0x7f33ac0caf00 legacy=0x7f33ac157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33ab7fe700 1 -- 172.21.15.146:0/892397332 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac0c9370 con 0x7f33ac0cab20 2024-09-13T18:27:10.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33ab7fe700 1 --2- 172.21.15.146:0/892397332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac0cab20 0x7f33ac15b380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3394009850 tx=0x7f3394009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7e9f8e7242c9d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:10.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33aa7fc700 1 -- 172.21.15.146:0/892397332 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394011830 con 0x7f33ac0cab20 2024-09-13T18:27:10.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33aa7fc700 1 -- 172.21.15.146:0/892397332 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3394011990 con 0x7f33ac0cab20 2024-09-13T18:27:10.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33aa7fc700 1 -- 172.21.15.146:0/892397332 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394011d00 con 0x7f33ac0cab20 2024-09-13T18:27:10.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33b0dd3700 1 -- 172.21.15.146:0/892397332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac0cab20 msgr2=0x7f33ac15b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33b0dd3700 1 --2- 172.21.15.146:0/892397332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac0cab20 0x7f33ac15b380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3394009850 tx=0x7f3394009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:10.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33b0dd3700 1 -- 172.21.15.146:0/892397332 shutdown_connections 2024-09-13T18:27:10.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33b0dd3700 1 --2- 172.21.15.146:0/892397332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac0cab20 0x7f33ac15b380 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33b0dd3700 1 -- 172.21.15.146:0/892397332 >> 172.21.15.146:0/892397332 conn(0x7f33ac066090 msgr2=0x7f33ac066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:10.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33b0dd3700 1 -- 172.21.15.146:0/892397332 shutdown_connections 2024-09-13T18:27:10.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33b0dd3700 1 -- 172.21.15.146:0/892397332 wait complete. 2024-09-13T18:27:10.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33b0dd3700 1 Processor -- start 2024-09-13T18:27:10.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.229+0000 7f33b0dd3700 1 -- start start 2024-09-13T18:27:10.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33b0dd3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac0cab20 0x7f33ac1f0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33b0dd3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33ac0caf00 0x7f33ac0c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33b0dd3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac1544a0 0x7f33ac0c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33b0dd3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0c58c0 con 0x7f33ac0cab20 2024-09-13T18:27:10.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33b0dd3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac1f1160 con 0x7f33ac0caf00 2024-09-13T18:27:10.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33b0dd3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0c55e0 con 0x7f33ac1544a0 2024-09-13T18:27:10.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33ab7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac0cab20 0x7f33ac1f0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33ab7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac0cab20 0x7f33ac1f0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55012/0 (socket says 172.21.15.146:55012) 2024-09-13T18:27:10.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33ab7fe700 1 -- 172.21.15.146:0/1704718534 learned_addr learned my addr 172.21.15.146:0/1704718534 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:10.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33aaffd700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33ac0caf00 0x7f33ac0c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33abfff700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac1544a0 0x7f33ac0c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33ab7fe700 1 -- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac1544a0 msgr2=0x7f33ac0c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33ab7fe700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac1544a0 0x7f33ac0c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33ab7fe700 1 -- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33ac0caf00 msgr2=0x7f33ac0c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33ab7fe700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33ac0caf00 0x7f33ac0c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33ab7fe700 1 -- 172.21.15.146:0/1704718534 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33940094b0 con 0x7f33ac0cab20 2024-09-13T18:27:10.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33ab7fe700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac0cab20 0x7f33ac1f0550 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f33940090b0 tx=0x7f3394004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:10.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33a3fff700 1 -- 172.21.15.146:0/1704718534 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394004380 con 0x7f33ac0cab20 2024-09-13T18:27:10.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33a3fff700 1 -- 172.21.15.146:0/1704718534 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339401f070 con 0x7f33ac0cab20 2024-09-13T18:27:10.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33a3fff700 1 -- 172.21.15.146:0/1704718534 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33940036c0 con 0x7f33ac0cab20 2024-09-13T18:27:10.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33b0dd3700 1 -- 172.21.15.146:0/1704718534 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ac1f0e80 con 0x7f33ac0cab20 2024-09-13T18:27:10.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33b0dd3700 1 -- 172.21.15.146:0/1704718534 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac0c4390 con 0x7f33ac0cab20 2024-09-13T18:27:10.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.233+0000 7f33b0dd3700 1 -- 172.21.15.146:0/1704718534 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ac15b280 con 0x7f33ac0cab20 2024-09-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.237+0000 7f33a3fff700 1 -- 172.21.15.146:0/1704718534 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3394021030 con 0x7f33ac0cab20 2024-09-13T18:27:10.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.237+0000 7f33a3fff700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3390066730 0x7f3390068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.237+0000 7f33a3fff700 1 -- 172.21.15.146:0/1704718534 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3394085270 con 0x7f33ac0cab20 2024-09-13T18:27:10.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.237+0000 7f33aaffd700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3390066730 0x7f3390068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.237+0000 7f33aaffd700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3390066730 0x7f3390068bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f33ac158660 tx=0x7f339c007e20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:10.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.241+0000 7f33a3fff700 1 -- 172.21.15.146:0/1704718534 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f33940521c0 con 0x7f33ac0cab20 2024-09-13T18:27:10.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.413+0000 7f33b0dd3700 1 -- 172.21.15.146:0/1704718534 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33ac0c8ad0 con 0x7f33ac0cab20 2024-09-13T18:27:10.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.413+0000 7f33a3fff700 1 -- 172.21.15.146:0/1704718534 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f33940557e0 con 0x7f33ac0cab20 2024-09-13T18:27:10.419 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-09-13T18:27:10.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.417+0000 7f33b0dd3700 1 -- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3390066730 msgr2=0x7f3390068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.417+0000 7f33b0dd3700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3390066730 0x7f3390068bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f33ac158660 tx=0x7f339c007e20 comp rx=0 tx=0).stop 2024-09-13T18:27:10.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.417+0000 7f33b0dd3700 1 -- 172.21.15.146:0/1704718534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac0cab20 msgr2=0x7f33ac1f0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.417+0000 7f33b0dd3700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac0cab20 0x7f33ac1f0550 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f33940090b0 tx=0x7f3394004770 comp rx=0 tx=0).stop 2024-09-13T18:27:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.417+0000 7f33b0dd3700 1 -- 172.21.15.146:0/1704718534 shutdown_connections 2024-09-13T18:27:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.417+0000 7f33b0dd3700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3390066730 0x7f3390068bf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.417+0000 7f33b0dd3700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac1544a0 0x7f33ac0c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.417+0000 7f33b0dd3700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33ac0caf00 0x7f33ac0c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.417+0000 7f33b0dd3700 1 --2- 172.21.15.146:0/1704718534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac0cab20 0x7f33ac1f0550 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.417+0000 7f33b0dd3700 1 -- 172.21.15.146:0/1704718534 >> 172.21.15.146:0/1704718534 conn(0x7f33ac066090 msgr2=0x7f33ac156830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.417+0000 7f33b0dd3700 1 -- 172.21.15.146:0/1704718534 shutdown_connections 2024-09-13T18:27:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.417+0000 7f33b0dd3700 1 -- 172.21.15.146:0/1704718534 wait complete. 2024-09-13T18:27:10.434 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:27:10.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.541+0000 7f5938b3f700 1 Processor -- start 2024-09-13T18:27:10.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.541+0000 7f5938b3f700 1 -- start start 2024-09-13T18:27:10.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5938b3f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f59340ca020 0x7f59340ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5938b3f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59340c5bd0 con 0x7f59340cad90 2024-09-13T18:27:10.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5938b3f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59340c9bb0 con 0x7f59340c57f0 2024-09-13T18:27:10.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5938b3f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f593415a160 con 0x7f59340ca020 2024-09-13T18:27:10.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5932d9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f59340cad90 0x7f59340cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35628/0 (socket says 172.21.15.146:35628) 2024-09-13T18:27:10.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5932d9d700 1 -- 172.21.15.146:0/120549601 learned_addr learned my addr 172.21.15.146:0/120549601 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:10.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5931d9b700 1 --2- 172.21.15.146:0/120549601 >> v2:172.21.15.146:3300/0 conn(0x7f59340ca020 0x7f59340ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f593159a700 1 -- 172.21.15.146:0/120549601 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072224235 0 0) 0x7f59340c5bd0 con 0x7f59340cad90 2024-09-13T18:27:10.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f593159a700 1 -- 172.21.15.146:0/120549601 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f591c0035d0 con 0x7f59340cad90 2024-09-13T18:27:10.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5931d9b700 1 -- 172.21.15.146:0/120549601 >> v1:172.21.15.154:6789/0 conn(0x7f59340c57f0 legacy=0x7f59340cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5931d9b700 1 -- 172.21.15.146:0/120549601 >> v1:172.21.15.146:6789/0 conn(0x7f59340cad90 legacy=0x7f59340cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5931d9b700 1 -- 172.21.15.146:0/120549601 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5934159830 con 0x7f59340ca020 2024-09-13T18:27:10.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5931d9b700 1 --2- 172.21.15.146:0/120549601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59340ca020 0x7f59340ca850 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5928009850 tx=0x7f5928009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4074e6b9f3dce6d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:10.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f593159a700 1 -- 172.21.15.146:0/120549601 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928011830 con 0x7f59340ca020 2024-09-13T18:27:10.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f593159a700 1 -- 172.21.15.146:0/120549601 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5928011990 con 0x7f59340ca020 2024-09-13T18:27:10.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f593159a700 1 -- 172.21.15.146:0/120549601 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928011d00 con 0x7f59340ca020 2024-09-13T18:27:10.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5938b3f700 1 -- 172.21.15.146:0/120549601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59340ca020 msgr2=0x7f59340ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5938b3f700 1 --2- 172.21.15.146:0/120549601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59340ca020 0x7f59340ca850 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5928009850 tx=0x7f5928009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:10.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5938b3f700 1 -- 172.21.15.146:0/120549601 shutdown_connections 2024-09-13T18:27:10.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5938b3f700 1 --2- 172.21.15.146:0/120549601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59340ca020 0x7f59340ca850 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5938b3f700 1 -- 172.21.15.146:0/120549601 >> 172.21.15.146:0/120549601 conn(0x7f5934071d30 msgr2=0x7f5934072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:10.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5938b3f700 1 -- 172.21.15.146:0/120549601 shutdown_connections 2024-09-13T18:27:10.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.545+0000 7f5938b3f700 1 -- 172.21.15.146:0/120549601 wait complete. 2024-09-13T18:27:10.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5938b3f700 1 Processor -- start 2024-09-13T18:27:10.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5938b3f700 1 -- start start 2024-09-13T18:27:10.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5938b3f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59340c57f0 0x7f59340cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5938b3f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59340ca020 0x7f59341eef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5938b3f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59340cad90 0x7f59341ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5938b3f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5934154e30 con 0x7f59340c57f0 2024-09-13T18:27:10.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5938b3f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5934154100 con 0x7f59340ca020 2024-09-13T18:27:10.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5938b3f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59341501d0 con 0x7f59340cad90 2024-09-13T18:27:10.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f593259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59340c57f0 0x7f59340cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f593259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59340c57f0 0x7f59340cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55034/0 (socket says 172.21.15.146:55034) 2024-09-13T18:27:10.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f593259c700 1 -- 172.21.15.146:0/2847694510 learned_addr learned my addr 172.21.15.146:0/2847694510 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:10.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5932d9d700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59340cad90 0x7f59341ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5931d9b700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59340ca020 0x7f59341eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f593259c700 1 -- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59340cad90 msgr2=0x7f59341ef4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f593259c700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59340cad90 0x7f59341ef4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f593259c700 1 -- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59340ca020 msgr2=0x7f59341eef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f593259c700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59340ca020 0x7f59341eef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f593259c700 1 -- 172.21.15.146:0/2847694510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59280094b0 con 0x7f59340c57f0 2024-09-13T18:27:10.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f593259c700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59340c57f0 0x7f59340cf6c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f592400c500 tx=0x7f592400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:10.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5922ffd700 1 -- 172.21.15.146:0/2847694510 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59240052d0 con 0x7f59340c57f0 2024-09-13T18:27:10.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5922ffd700 1 -- 172.21.15.146:0/2847694510 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5924005430 con 0x7f59340c57f0 2024-09-13T18:27:10.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5938b3f700 1 -- 172.21.15.146:0/2847694510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59341504b0 con 0x7f59340c57f0 2024-09-13T18:27:10.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5922ffd700 1 -- 172.21.15.146:0/2847694510 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59240055a0 con 0x7f59340c57f0 2024-09-13T18:27:10.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.549+0000 7f5938b3f700 1 -- 172.21.15.146:0/2847694510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5934154630 con 0x7f59340c57f0 2024-09-13T18:27:10.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.553+0000 7f5938b3f700 1 -- 172.21.15.146:0/2847694510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593415a580 con 0x7f59340c57f0 2024-09-13T18:27:10.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.553+0000 7f5922ffd700 1 -- 172.21.15.146:0/2847694510 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59240036a0 con 0x7f59340c57f0 2024-09-13T18:27:10.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.553+0000 7f5922ffd700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5914066730 0x7f5914068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.553+0000 7f5922ffd700 1 -- 172.21.15.146:0/2847694510 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5924083780 con 0x7f59340c57f0 2024-09-13T18:27:10.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.553+0000 7f5931d9b700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5914066730 0x7f5914068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.557+0000 7f5931d9b700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5914066730 0x7f5914068bf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5928009ba0 tx=0x7f5928008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:10.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.561+0000 7f5922ffd700 1 -- 172.21.15.146:0/2847694510 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5924050770 con 0x7f59340c57f0 2024-09-13T18:27:10.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.733+0000 7f5938b3f700 1 -- 172.21.15.146:0/2847694510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59340ccec0 con 0x7f59340c57f0 2024-09-13T18:27:10.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.733+0000 7f5922ffd700 1 -- 172.21.15.146:0/2847694510 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5924053d90 con 0x7f59340c57f0 2024-09-13T18:27:10.739 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:27:10.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.737+0000 7f5938b3f700 1 -- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5914066730 msgr2=0x7f5914068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.737+0000 7f5938b3f700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5914066730 0x7f5914068bf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5928009ba0 tx=0x7f5928008030 comp rx=0 tx=0).stop 2024-09-13T18:27:10.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.737+0000 7f5938b3f700 1 -- 172.21.15.146:0/2847694510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59340c57f0 msgr2=0x7f59340cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.737+0000 7f5938b3f700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59340c57f0 0x7f59340cf6c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f592400c500 tx=0x7f592400c8c0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.737+0000 7f5938b3f700 1 -- 172.21.15.146:0/2847694510 shutdown_connections 2024-09-13T18:27:10.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.737+0000 7f5938b3f700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5914066730 0x7f5914068bf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.737+0000 7f5938b3f700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59340cad90 0x7f59341ef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.737+0000 7f5938b3f700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59340ca020 0x7f59341eef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.737+0000 7f5938b3f700 1 --2- 172.21.15.146:0/2847694510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59340c57f0 0x7f59340cf6c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.737+0000 7f5938b3f700 1 -- 172.21.15.146:0/2847694510 >> 172.21.15.146:0/2847694510 conn(0x7f5934071d30 msgr2=0x7f5934072110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:10.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.737+0000 7f5938b3f700 1 -- 172.21.15.146:0/2847694510 shutdown_connections 2024-09-13T18:27:10.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.737+0000 7f5938b3f700 1 -- 172.21.15.146:0/2847694510 wait complete. 2024-09-13T18:27:10.757 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:27:10.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.873+0000 7f73851dc700 1 Processor -- start 2024-09-13T18:27:10.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.873+0000 7f73851dc700 1 -- start start 2024-09-13T18:27:10.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.873+0000 7f73851dc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f73801539e0 0x7f73800b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.873+0000 7f73851dc700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73800c5e50 con 0x7f738015d960 2024-09-13T18:27:10.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.873+0000 7f73851dc700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73800cb480 con 0x7f73800cf580 2024-09-13T18:27:10.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.873+0000 7f73851dc700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73800cc3a0 con 0x7f73801539e0 2024-09-13T18:27:10.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.873+0000 7f737ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f73801539e0 0x7f73800b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.873+0000 7f737f7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f738015d960 0x7f73800c9fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35638/0 (socket says 172.21.15.146:35638) 2024-09-13T18:27:10.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.873+0000 7f737f7fe700 1 -- 172.21.15.146:0/3935282057 learned_addr learned my addr 172.21.15.146:0/3935282057 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:10.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f737ffff700 1 -- 172.21.15.146:0/3935282057 >> v1:172.21.15.154:6789/0 conn(0x7f73800cf580 legacy=0x7f73800cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f737ffff700 1 -- 172.21.15.146:0/3935282057 >> v1:172.21.15.146:6789/0 conn(0x7f738015d960 legacy=0x7f73800c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f737effd700 1 -- 172.21.15.146:0/3935282057 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262216764 0 0) 0x7f7370001110 con 0x7f738015d960 2024-09-13T18:27:10.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f737ffff700 1 -- 172.21.15.146:0/3935282057 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73800cb720 con 0x7f73801539e0 2024-09-13T18:27:10.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f737ffff700 1 --2- 172.21.15.146:0/3935282057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73801539e0 0x7f73800b3370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7374009880 tx=0x7f7374009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfa187d1ad61bd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:10.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f737effd700 1 -- 172.21.15.146:0/3935282057 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374011830 con 0x7f73801539e0 2024-09-13T18:27:10.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f737effd700 1 -- 172.21.15.146:0/3935282057 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7374011990 con 0x7f73801539e0 2024-09-13T18:27:10.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f737effd700 1 -- 172.21.15.146:0/3935282057 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374011d20 con 0x7f73801539e0 2024-09-13T18:27:10.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f73851dc700 1 -- 172.21.15.146:0/3935282057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73801539e0 msgr2=0x7f73800b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f73851dc700 1 --2- 172.21.15.146:0/3935282057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73801539e0 0x7f73800b3370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7374009880 tx=0x7f7374009b90 comp rx=0 tx=0).stop 2024-09-13T18:27:10.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f73851dc700 1 -- 172.21.15.146:0/3935282057 shutdown_connections 2024-09-13T18:27:10.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f73851dc700 1 --2- 172.21.15.146:0/3935282057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73801539e0 0x7f73800b3370 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f73851dc700 1 -- 172.21.15.146:0/3935282057 >> 172.21.15.146:0/3935282057 conn(0x7f7380071ec0 msgr2=0x7f73800722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:10.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f73851dc700 1 -- 172.21.15.146:0/3935282057 shutdown_connections 2024-09-13T18:27:10.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f73851dc700 1 -- 172.21.15.146:0/3935282057 wait complete. 2024-09-13T18:27:10.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f73851dc700 1 Processor -- start 2024-09-13T18:27:10.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.877+0000 7f73851dc700 1 -- start start 2024-09-13T18:27:10.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f73851dc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800cf580 0x7f73800c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f73851dc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73801539e0 0x7f738014b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f73851dc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f738015d960 0x7f73801459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f73851dc700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7380148470 con 0x7f73800cf580 2024-09-13T18:27:10.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f73851dc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73801493f0 con 0x7f73801539e0 2024-09-13T18:27:10.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f73851dc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7380148680 con 0x7f738015d960 2024-09-13T18:27:10.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800cf580 0x7f73800c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800cf580 0x7f73800c7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55046/0 (socket says 172.21.15.146:55046) 2024-09-13T18:27:10.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737ffff700 1 -- 172.21.15.146:0/2354946622 learned_addr learned my addr 172.21.15.146:0/2354946622 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737f7fe700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73801539e0 0x7f738014b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f73849db700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f738015d960 0x7f73801459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737ffff700 1 -- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f738015d960 msgr2=0x7f73801459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737ffff700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f738015d960 0x7f73801459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737ffff700 1 -- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73801539e0 msgr2=0x7f738014b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737ffff700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73801539e0 0x7f738014b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737ffff700 1 -- 172.21.15.146:0/2354946622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73740094b0 con 0x7f73800cf580 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737f7fe700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73801539e0 0x7f738014b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f73849db700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f738015d960 0x7f73801459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737ffff700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800cf580 0x7f73800c7630 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f73740090b0 tx=0x7f7374004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737cff9700 1 -- 172.21.15.146:0/2354946622 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737401abb0 con 0x7f73800cf580 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737cff9700 1 -- 172.21.15.146:0/2354946622 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737401f070 con 0x7f73800cf580 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f73851dc700 1 -- 172.21.15.146:0/2354946622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73801eeff0 con 0x7f73800cf580 2024-09-13T18:27:10.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f737cff9700 1 -- 172.21.15.146:0/2354946622 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737401b430 con 0x7f73800cf580 2024-09-13T18:27:10.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f73851dc700 1 -- 172.21.15.146:0/2354946622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73801ef5b0 con 0x7f73800cf580 2024-09-13T18:27:10.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.881+0000 7f73851dc700 1 -- 172.21.15.146:0/2354946622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73801ef7d0 con 0x7f73800cf580 2024-09-13T18:27:10.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.885+0000 7f737cff9700 1 -- 172.21.15.146:0/2354946622 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f737401b590 con 0x7f73800cf580 2024-09-13T18:27:10.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.885+0000 7f737cff9700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f736c066730 0x7f736c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:10.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.885+0000 7f737cff9700 1 -- 172.21.15.146:0/2354946622 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7374085780 con 0x7f73800cf580 2024-09-13T18:27:10.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.885+0000 7f737f7fe700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f736c066730 0x7f736c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:10.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.885+0000 7f737f7fe700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f736c066730 0x7f736c068bf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f73800cd890 tx=0x7f7370007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:10.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:10.889+0000 7f737cff9700 1 -- 172.21.15.146:0/2354946622 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7374052800 con 0x7f73800cf580 2024-09-13T18:27:11.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.061+0000 7f73851dc700 1 -- 172.21.15.146:0/2354946622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73800cc7c0 con 0x7f73800cf580 2024-09-13T18:27:11.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.061+0000 7f737cff9700 1 -- 172.21.15.146:0/2354946622 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7374015020 con 0x7f73800cf580 2024-09-13T18:27:11.067 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-09-13T18:27:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.065+0000 7f73851dc700 1 -- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f736c066730 msgr2=0x7f736c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.065+0000 7f73851dc700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f736c066730 0x7f736c068bf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f73800cd890 tx=0x7f7370007d50 comp rx=0 tx=0).stop 2024-09-13T18:27:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.065+0000 7f73851dc700 1 -- 172.21.15.146:0/2354946622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800cf580 msgr2=0x7f73800c7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.065+0000 7f73851dc700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800cf580 0x7f73800c7630 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f73740090b0 tx=0x7f7374004770 comp rx=0 tx=0).stop 2024-09-13T18:27:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.065+0000 7f73851dc700 1 -- 172.21.15.146:0/2354946622 shutdown_connections 2024-09-13T18:27:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.065+0000 7f73851dc700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f736c066730 0x7f736c068bf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.065+0000 7f73851dc700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f738015d960 0x7f73801459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.065+0000 7f73851dc700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73801539e0 0x7f738014b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.065+0000 7f73851dc700 1 --2- 172.21.15.146:0/2354946622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800cf580 0x7f73800c7630 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.065+0000 7f73851dc700 1 -- 172.21.15.146:0/2354946622 >> 172.21.15.146:0/2354946622 conn(0x7f7380071ec0 msgr2=0x7f73800b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.065+0000 7f73851dc700 1 -- 172.21.15.146:0/2354946622 shutdown_connections 2024-09-13T18:27:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.065+0000 7f73851dc700 1 -- 172.21.15.146:0/2354946622 wait complete. 2024-09-13T18:27:11.082 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:27:11.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.193+0000 7f96a5e4c700 1 Processor -- start 2024-09-13T18:27:11.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.193+0000 7f96a5e4c700 1 -- start start 2024-09-13T18:27:11.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96a00cab20 0x7f96a015b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96a00caf00 0x7f96a0157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a00a97e0 con 0x7f96a00cb300 2024-09-13T18:27:11.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a00c99a0 con 0x7f96a00cab20 2024-09-13T18:27:11.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00723d0 con 0x7f96a00caf00 2024-09-13T18:27:11.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a4e4a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96a00cab20 0x7f96a015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a4e4a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96a00cab20 0x7f96a015b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52328/0 (socket says 172.21.15.146:52328) 2024-09-13T18:27:11.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a4e4a700 1 -- 172.21.15.146:0/1605243672 learned_addr learned my addr 172.21.15.146:0/1605243672 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:11.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f969ffff700 1 --2- 172.21.15.146:0/1605243672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96a00caf00 0x7f96a0157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a4e4a700 1 -- 172.21.15.146:0/1605243672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96a00caf00 msgr2=0x7f96a0157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:11.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a4e4a700 1 --2- 172.21.15.146:0/1605243672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96a00caf00 0x7f96a0157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a4e4a700 1 -- 172.21.15.146:0/1605243672 >> v1:172.21.15.154:6789/0 conn(0x7f96a00cb300 legacy=0x7f96a01544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a4e4a700 1 -- 172.21.15.146:0/1605243672 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a015b8c0 con 0x7f96a00cab20 2024-09-13T18:27:11.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a4e4a700 1 --2- 172.21.15.146:0/1605243672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a00cab20 0x7f96a015b380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9688009910 tx=0x7f9688009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37850a20a0a44b24 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:11.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f969f7fe700 1 -- 172.21.15.146:0/1605243672 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688011830 con 0x7f96a00cab20 2024-09-13T18:27:11.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f969f7fe700 1 -- 172.21.15.146:0/1605243672 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9688011990 con 0x7f96a00cab20 2024-09-13T18:27:11.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f969f7fe700 1 -- 172.21.15.146:0/1605243672 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688011d40 con 0x7f96a00cab20 2024-09-13T18:27:11.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1605243672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a00cab20 msgr2=0x7f96a015b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 --2- 172.21.15.146:0/1605243672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a00cab20 0x7f96a015b380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9688009910 tx=0x7f9688009c20 comp rx=0 tx=0).stop 2024-09-13T18:27:11.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1605243672 shutdown_connections 2024-09-13T18:27:11.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 --2- 172.21.15.146:0/1605243672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96a00caf00 0x7f96a0157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 --2- 172.21.15.146:0/1605243672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a00cab20 0x7f96a015b380 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1605243672 >> 172.21.15.146:0/1605243672 conn(0x7f96a0066090 msgr2=0x7f96a0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:11.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1605243672 shutdown_connections 2024-09-13T18:27:11.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1605243672 wait complete. 2024-09-13T18:27:11.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 Processor -- start 2024-09-13T18:27:11.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.197+0000 7f96a5e4c700 1 -- start start 2024-09-13T18:27:11.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a5e4c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a00caf00 0x7f96a00ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a5e4c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96a00cb300 0x7f96a0159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a5e4c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96a00c4660 0x7f96a0153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a5e4c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00c2030 con 0x7f96a00caf00 2024-09-13T18:27:11.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a5e4c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00ac0b0 con 0x7f96a00c4660 2024-09-13T18:27:11.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a5e4c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00aa0d0 con 0x7f96a00cb300 2024-09-13T18:27:11.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a4e4a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a00caf00 0x7f96a00ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a4e4a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a00caf00 0x7f96a00ab6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52338/0 (socket says 172.21.15.146:52338) 2024-09-13T18:27:11.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a4e4a700 1 -- 172.21.15.146:0/1013370900 learned_addr learned my addr 172.21.15.146:0/1013370900 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:11.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a564b700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96a00c4660 0x7f96a0153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a4e4a700 1 -- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96a00cb300 msgr2=0x7f96a0159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f969ffff700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96a00cb300 0x7f96a0159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a4e4a700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96a00cb300 0x7f96a0159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a4e4a700 1 -- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96a00c4660 msgr2=0x7f96a0153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a4e4a700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96a00c4660 0x7f96a0153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a4e4a700 1 -- 172.21.15.146:0/1013370900 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9688009570 con 0x7f96a00caf00 2024-09-13T18:27:11.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a564b700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96a00c4660 0x7f96a0153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:11.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f969ffff700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96a00cb300 0x7f96a0159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:11.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a4e4a700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a00caf00 0x7f96a00ab6b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9688000c00 tx=0x7f9688004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:11.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f969d7fa700 1 -- 172.21.15.146:0/1013370900 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688004550 con 0x7f96a00caf00 2024-09-13T18:27:11.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f969d7fa700 1 -- 172.21.15.146:0/1013370900 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96880036c0 con 0x7f96a00caf00 2024-09-13T18:27:11.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1013370900 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a00aa320 con 0x7f96a00caf00 2024-09-13T18:27:11.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f969d7fa700 1 -- 172.21.15.146:0/1013370900 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688003a70 con 0x7f96a00caf00 2024-09-13T18:27:11.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1013370900 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a00aa6b0 con 0x7f96a00caf00 2024-09-13T18:27:11.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.201+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1013370900 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a00c8db0 con 0x7f96a00caf00 2024-09-13T18:27:11.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.205+0000 7f969d7fa700 1 -- 172.21.15.146:0/1013370900 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9688003bd0 con 0x7f96a00caf00 2024-09-13T18:27:11.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.205+0000 7f969d7fa700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9680066760 0x7f9680068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.205+0000 7f969d7fa700 1 -- 172.21.15.146:0/1013370900 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9688021030 con 0x7f96a00caf00 2024-09-13T18:27:11.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.205+0000 7f969ffff700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9680066760 0x7f9680068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.205+0000 7f969ffff700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9680066760 0x7f9680068c20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f96a015a9d0 tx=0x7f9690008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:11.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.209+0000 7f969d7fa700 1 -- 172.21.15.146:0/1013370900 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9688052ea0 con 0x7f96a00caf00 2024-09-13T18:27:11.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.373+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1013370900 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f96a015b8c0 con 0x7f96a00caf00 2024-09-13T18:27:11.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.373+0000 7f969d7fa700 1 -- 172.21.15.146:0/1013370900 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9688015020 con 0x7f96a00caf00 2024-09-13T18:27:11.382 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:27:11.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.377+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9680066760 msgr2=0x7f9680068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.377+0000 7f96a5e4c700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9680066760 0x7f9680068c20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f96a015a9d0 tx=0x7f9690008040 comp rx=0 tx=0).stop 2024-09-13T18:27:11.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.377+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1013370900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a00caf00 msgr2=0x7f96a00ab6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.377+0000 7f96a5e4c700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a00caf00 0x7f96a00ab6b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9688000c00 tx=0x7f9688004770 comp rx=0 tx=0).stop 2024-09-13T18:27:11.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.377+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1013370900 shutdown_connections 2024-09-13T18:27:11.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.377+0000 7f96a5e4c700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9680066760 0x7f9680068c20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.377+0000 7f96a5e4c700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96a00c4660 0x7f96a0153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.377+0000 7f96a5e4c700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96a00cb300 0x7f96a0159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.377+0000 7f96a5e4c700 1 --2- 172.21.15.146:0/1013370900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a00caf00 0x7f96a00ab6b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.377+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1013370900 >> 172.21.15.146:0/1013370900 conn(0x7f96a0066090 msgr2=0x7f96a0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:11.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.377+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1013370900 shutdown_connections 2024-09-13T18:27:11.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.377+0000 7f96a5e4c700 1 -- 172.21.15.146:0/1013370900 wait complete. 2024-09-13T18:27:11.401 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:27:11.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 Processor -- start 2024-09-13T18:27:11.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 -- start start 2024-09-13T18:27:11.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3a880c6610 0x7f3a880cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3a880c6a80 0x7f3a880cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a88153fa0 con 0x7f3a88153bc0 2024-09-13T18:27:11.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a880aa500 con 0x7f3a880c6a80 2024-09-13T18:27:11.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8815a7e0 con 0x7f3a880c6610 2024-09-13T18:27:11.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8e67b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3a880c6a80 0x7f3a880cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8e67b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3a880c6a80 0x7f3a880cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52354/0 (socket says 172.21.15.146:52354) 2024-09-13T18:27:11.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8e67b700 1 -- 172.21.15.146:0/757717214 learned_addr learned my addr 172.21.15.146:0/757717214 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:11.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8d679700 1 --2- 172.21.15.146:0/757717214 >> v2:172.21.15.154:3300/0 conn(0x7f3a880c6610 0x7f3a880cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8e67b700 1 -- 172.21.15.146:0/757717214 >> v2:172.21.15.154:3300/0 conn(0x7f3a880c6610 msgr2=0x7f3a880cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:11.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8e67b700 1 --2- 172.21.15.146:0/757717214 >> v2:172.21.15.154:3300/0 conn(0x7f3a880c6610 0x7f3a880cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8e67b700 1 -- 172.21.15.146:0/757717214 >> v1:172.21.15.146:6789/0 conn(0x7f3a88153bc0 legacy=0x7f3a880c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8e67b700 1 -- 172.21.15.146:0/757717214 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a880aa220 con 0x7f3a880c6a80 2024-09-13T18:27:11.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8e67b700 1 --2- 172.21.15.146:0/757717214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a880c6a80 0x7f3a880cb1a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3a8806eee0 tx=0x7f3a8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14751a64565b6530 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:11.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ce78700 1 -- 172.21.15.146:0/757717214 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000e070 con 0x7f3a880c6a80 2024-09-13T18:27:11.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ce78700 1 -- 172.21.15.146:0/757717214 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8000b9c0 con 0x7f3a880c6a80 2024-09-13T18:27:11.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ce78700 1 -- 172.21.15.146:0/757717214 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000bd70 con 0x7f3a880c6a80 2024-09-13T18:27:11.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/757717214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a880c6a80 msgr2=0x7f3a880cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 --2- 172.21.15.146:0/757717214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a880c6a80 0x7f3a880cb1a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3a8806eee0 tx=0x7f3a8000b330 comp rx=0 tx=0).stop 2024-09-13T18:27:11.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/757717214 shutdown_connections 2024-09-13T18:27:11.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 --2- 172.21.15.146:0/757717214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a880c6a80 0x7f3a880cb1a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 --2- 172.21.15.146:0/757717214 >> v2:172.21.15.154:3300/0 conn(0x7f3a880c6610 0x7f3a880cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/757717214 >> 172.21.15.146:0/757717214 conn(0x7f3a88066090 msgr2=0x7f3a88066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:11.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/757717214 shutdown_connections 2024-09-13T18:27:11.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/757717214 wait complete. 2024-09-13T18:27:11.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 Processor -- start 2024-09-13T18:27:11.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.505+0000 7f3a8ee7c700 1 -- start start 2024-09-13T18:27:11.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8ee7c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a880c6610 0x7f3a88150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8ee7c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a880c6a80 0x7f3a8806d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8ee7c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a88153bc0 0x7f3a881f0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8ee7c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a881e4800 con 0x7f3a88153bc0 2024-09-13T18:27:11.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8ee7c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8806c500 con 0x7f3a880c6610 2024-09-13T18:27:11.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8ee7c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8806c7e0 con 0x7f3a880c6a80 2024-09-13T18:27:11.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8e67b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a88153bc0 0x7f3a881f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8e67b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a88153bc0 0x7f3a881f0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52370/0 (socket says 172.21.15.146:52370) 2024-09-13T18:27:11.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8e67b700 1 -- 172.21.15.146:0/1705797273 learned_addr learned my addr 172.21.15.146:0/1705797273 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:11.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8de7a700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a880c6610 0x7f3a88150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8d679700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a880c6a80 0x7f3a8806d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8e67b700 1 -- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a880c6a80 msgr2=0x7f3a8806d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8e67b700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a880c6a80 0x7f3a8806d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8e67b700 1 -- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a880c6610 msgr2=0x7f3a88150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8e67b700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a880c6610 0x7f3a88150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8e67b700 1 -- 172.21.15.146:0/1705797273 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a80009b30 con 0x7f3a88153bc0 2024-09-13T18:27:11.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8d679700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a880c6a80 0x7f3a8806d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:11.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8e67b700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a88153bc0 0x7f3a881f0660 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3a880cac30 tx=0x7f3a80004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:11.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a7e7fc700 1 -- 172.21.15.146:0/1705797273 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80004930 con 0x7f3a88153bc0 2024-09-13T18:27:11.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a7e7fc700 1 -- 172.21.15.146:0/1705797273 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a800099b0 con 0x7f3a88153bc0 2024-09-13T18:27:11.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/1705797273 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8806cac0 con 0x7f3a88153bc0 2024-09-13T18:27:11.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a7e7fc700 1 -- 172.21.15.146:0/1705797273 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8001c680 con 0x7f3a88153bc0 2024-09-13T18:27:11.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.509+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/1705797273 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8806d080 con 0x7f3a88153bc0 2024-09-13T18:27:11.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.513+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/1705797273 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a880a9de0 con 0x7f3a88153bc0 2024-09-13T18:27:11.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.513+0000 7f3a7e7fc700 1 -- 172.21.15.146:0/1705797273 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3a8001c7e0 con 0x7f3a88153bc0 2024-09-13T18:27:11.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.513+0000 7f3a7e7fc700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3a6c066760 0x7f3a6c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.513+0000 7f3a7e7fc700 1 -- 172.21.15.146:0/1705797273 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3a80085d30 con 0x7f3a88153bc0 2024-09-13T18:27:11.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.513+0000 7f3a8de7a700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3a6c066760 0x7f3a6c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.513+0000 7f3a8de7a700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3a6c066760 0x7f3a6c068c20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3a84000f60 tx=0x7f3a84009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:11.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.525+0000 7f3a7e7fc700 1 -- 172.21.15.146:0/1705797273 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3a80052e40 con 0x7f3a88153bc0 2024-09-13T18:27:11.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.737+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/1705797273 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a8806f690 con 0x7f3a88153bc0 2024-09-13T18:27:11.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.741+0000 7f3a7e7fc700 1 -- 172.21.15.146:0/1705797273 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6286 (secure 0 0 0) 0x7f3a80056460 con 0x7f3a88153bc0 2024-09-13T18:27:11.750 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:11.750 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:27:09.524480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"e","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:27:11.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.745+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3a6c066760 msgr2=0x7f3a6c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.745+0000 7f3a8ee7c700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3a6c066760 0x7f3a6c068c20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3a84000f60 tx=0x7f3a84009040 comp rx=0 tx=0).stop 2024-09-13T18:27:11.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.745+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/1705797273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a88153bc0 msgr2=0x7f3a881f0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.745+0000 7f3a8ee7c700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a88153bc0 0x7f3a881f0660 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3a880cac30 tx=0x7f3a80004af0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.745+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/1705797273 shutdown_connections 2024-09-13T18:27:11.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.745+0000 7f3a8ee7c700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3a6c066760 0x7f3a6c068c20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.745+0000 7f3a8ee7c700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a88153bc0 0x7f3a881f0660 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.745+0000 7f3a8ee7c700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a880c6a80 0x7f3a8806d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.745+0000 7f3a8ee7c700 1 --2- 172.21.15.146:0/1705797273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a880c6610 0x7f3a88150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.745+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/1705797273 >> 172.21.15.146:0/1705797273 conn(0x7f3a88066090 msgr2=0x7f3a880bfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:11.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.745+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/1705797273 shutdown_connections 2024-09-13T18:27:11.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.745+0000 7f3a8ee7c700 1 -- 172.21.15.146:0/1705797273 wait complete. 2024-09-13T18:27:11.756 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 15 2024-09-13T18:27:11.771 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:27:11.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f7c253700 1 Processor -- start 2024-09-13T18:27:11.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f7c253700 1 -- start start 2024-09-13T18:27:11.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f7c253700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6f74157e10 0x7f6f740c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f7c253700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f740c5bd0 0x7f6f740c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f7c253700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f740adfe0 con 0x7f6f7406fe60 2024-09-13T18:27:11.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f7c253700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7415e690 con 0x7f6f74157e10 2024-09-13T18:27:11.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f7c253700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740c7570 con 0x7f6f740c5bd0 2024-09-13T18:27:11.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f797ee700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6f7406fe60 0x7f6f74070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56676/0 (socket says 172.21.15.146:56676) 2024-09-13T18:27:11.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f797ee700 1 -- 172.21.15.146:0/1895749732 learned_addr learned my addr 172.21.15.146:0/1895749732 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:11.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f7a7f0700 1 --2- 172.21.15.146:0/1895749732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f740c5bd0 0x7f6f740c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f79fef700 1 --2- 172.21.15.146:0/1895749732 >> v2:172.21.15.154:3300/0 conn(0x7f6f74157e10 0x7f6f740c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058700997 0 0) 0x7f6f740adfe0 con 0x7f6f7406fe60 2024-09-13T18:27:11.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f600035d0 con 0x7f6f7406fe60 2024-09-13T18:27:11.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4032911324 0 0) 0x7f6f600035d0 con 0x7f6f7406fe60 2024-09-13T18:27:11.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f740c5bd0 msgr2=0x7f6f740c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:11.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 --2- 172.21.15.146:0/1895749732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f740c5bd0 0x7f6f740c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 >> v2:172.21.15.154:3300/0 conn(0x7f6f74157e10 msgr2=0x7f6f740c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:11.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 --2- 172.21.15.146:0/1895749732 >> v2:172.21.15.154:3300/0 conn(0x7f6f74157e10 0x7f6f740c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7415eb80 con 0x7f6f7406fe60 2024-09-13T18:27:11.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f6f64001670 con 0x7f6f7406fe60 2024-09-13T18:27:11.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 >> v1:172.21.15.146:6789/0 conn(0x7f6f7406fe60 legacy=0x7f6f74070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 --2- 172.21.15.146:0/1895749732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f740c5bd0 0x7f6f60003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 --2- 172.21.15.146:0/1895749732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f60003fa0 0x7f6f60004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 --2- 172.21.15.146:0/1895749732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f60004db0 0x7f6f60005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7415e690 con 0x7f6f60003fa0 2024-09-13T18:27:11.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740c7570 con 0x7f6f60004db0 2024-09-13T18:27:11.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6000ba30 con 0x7f6f740c5bd0 2024-09-13T18:27:11.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f797ee700 1 --2- 172.21.15.146:0/1895749732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f60003fa0 0x7f6f60004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f7a7f0700 1 --2- 172.21.15.146:0/1895749732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f60004db0 0x7f6f60005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f79fef700 1 --2- 172.21.15.146:0/1895749732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f740c5bd0 0x7f6f60003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f797ee700 1 -- 172.21.15.146:0/1895749732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f740c5bd0 msgr2=0x7f6f60003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f797ee700 1 --2- 172.21.15.146:0/1895749732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f740c5bd0 0x7f6f60003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f797ee700 1 -- 172.21.15.146:0/1895749732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f60004db0 msgr2=0x7f6f60005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f797ee700 1 --2- 172.21.15.146:0/1895749732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f60004db0 0x7f6f60005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f797ee700 1 -- 172.21.15.146:0/1895749732 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7415eb80 con 0x7f6f60003fa0 2024-09-13T18:27:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.873+0000 7f6f7a7f0700 1 --2- 172.21.15.146:0/1895749732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f60004db0 0x7f6f60005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.877+0000 7f6f797ee700 1 --2- 172.21.15.146:0/1895749732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f60003fa0 0x7f6f60004870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400c4e0 tx=0x7f6f600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dfbab183357802 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.877+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64005980 con 0x7f6f60003fa0 2024-09-13T18:27:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.877+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f64005b00 con 0x7f6f60003fa0 2024-09-13T18:27:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.877+0000 7f6f78fed700 1 -- 172.21.15.146:0/1895749732 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f640070d0 con 0x7f6f60003fa0 2024-09-13T18:27:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.877+0000 7f6f7c253700 1 -- 172.21.15.146:0/1895749732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f60003fa0 msgr2=0x7f6f60004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.877+0000 7f6f7c253700 1 --2- 172.21.15.146:0/1895749732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f60003fa0 0x7f6f60004870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400c4e0 tx=0x7f6f600058a0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.877+0000 7f6f7a7f0700 1 -- 172.21.15.146:0/1895749732 reap_dead start 2024-09-13T18:27:11.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.877+0000 7f6f7c253700 1 -- 172.21.15.146:0/1895749732 shutdown_connections 2024-09-13T18:27:11.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.877+0000 7f6f7c253700 1 -- 172.21.15.146:0/1895749732 >> 172.21.15.146:0/1895749732 conn(0x7f6f740cb1d0 msgr2=0x7f6f740cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:11.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.877+0000 7f6f7c253700 1 -- 172.21.15.146:0/1895749732 shutdown_connections 2024-09-13T18:27:11.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.877+0000 7f6f7c253700 1 -- 172.21.15.146:0/1895749732 wait complete. 2024-09-13T18:27:11.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.877+0000 7f6f7c253700 1 Processor -- start 2024-09-13T18:27:11.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.877+0000 7f6f7c253700 1 -- start start 2024-09-13T18:27:11.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7c253700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f740c4f00 0x7f6f7415edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7c253700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f740c52e0 0x7f6f74157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7c253700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f74157890 0x7f6f741f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7c253700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7415fb30 con 0x7f6f74157890 2024-09-13T18:27:11.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7c253700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f741605f0 con 0x7f6f740c4f00 2024-09-13T18:27:11.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7c253700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740c3990 con 0x7f6f740c52e0 2024-09-13T18:27:11.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7a7f0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f74157890 0x7f6f741f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7a7f0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f74157890 0x7f6f741f4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52398/0 (socket says 172.21.15.146:52398) 2024-09-13T18:27:11.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7a7f0700 1 -- 172.21.15.146:0/179870766 learned_addr learned my addr 172.21.15.146:0/179870766 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:11.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f797ee700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f740c52e0 0x7f6f74157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f79fef700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f740c4f00 0x7f6f7415edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7a7f0700 1 -- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f740c52e0 msgr2=0x7f6f74157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7a7f0700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f740c52e0 0x7f6f74157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7a7f0700 1 -- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f740c4f00 msgr2=0x7f6f7415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:11.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7a7f0700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f740c4f00 0x7f6f7415edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:11.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7a7f0700 1 -- 172.21.15.146:0/179870766 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f740c3be0 con 0x7f6f74157890 2024-09-13T18:27:11.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7a7f0700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f74157890 0x7f6f741f4230 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6f7406bbb0 tx=0x7f6f6c011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:11.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f727fc700 1 -- 172.21.15.146:0/179870766 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c01b070 con 0x7f6f74157890 2024-09-13T18:27:11.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f727fc700 1 -- 172.21.15.146:0/179870766 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c009e40 con 0x7f6f74157890 2024-09-13T18:27:11.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7c253700 1 -- 172.21.15.146:0/179870766 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f740c3ec0 con 0x7f6f74157890 2024-09-13T18:27:11.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f727fc700 1 -- 172.21.15.146:0/179870766 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c004020 con 0x7f6f74157890 2024-09-13T18:27:11.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7c253700 1 -- 172.21.15.146:0/179870766 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f740c4480 con 0x7f6f74157890 2024-09-13T18:27:11.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.881+0000 7f6f7c253700 1 -- 172.21.15.146:0/179870766 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7406fb80 con 0x7f6f74157890 2024-09-13T18:27:11.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.885+0000 7f6f727fc700 1 -- 172.21.15.146:0/179870766 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6f6c0041a0 con 0x7f6f74157890 2024-09-13T18:27:11.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.885+0000 7f6f727fc700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6f54066730 0x7f6f54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:11.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.885+0000 7f6f727fc700 1 -- 172.21.15.146:0/179870766 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6f6c083bc0 con 0x7f6f74157890 2024-09-13T18:27:11.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.885+0000 7f6f79fef700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6f54066730 0x7f6f54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:11.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.885+0000 7f6f79fef700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6f54066730 0x7f6f54068bf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c014510 tx=0x7f6f5c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:11.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:11.889+0000 7f6f727fc700 1 -- 172.21.15.146:0/179870766 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6f6c050bb0 con 0x7f6f74157890 2024-09-13T18:27:12.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.069+0000 7f6f7c253700 1 -- 172.21.15.146:0/179870766 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f740cbe80 con 0x7f6f74157890 2024-09-13T18:27:12.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.069+0000 7f6f727fc700 1 -- 172.21.15.146:0/179870766 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f6f6c000e60 con 0x7f6f74157890 2024-09-13T18:27:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:12.081 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:27:12.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.073+0000 7f6f7c253700 1 -- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6f54066730 msgr2=0x7f6f54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:12.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.073+0000 7f6f7c253700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6f54066730 0x7f6f54068bf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c014510 tx=0x7f6f5c00b040 comp rx=0 tx=0).stop 2024-09-13T18:27:12.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.073+0000 7f6f7c253700 1 -- 172.21.15.146:0/179870766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f74157890 msgr2=0x7f6f741f4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:12.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.073+0000 7f6f7c253700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f74157890 0x7f6f741f4230 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6f7406bbb0 tx=0x7f6f6c011eb0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.073+0000 7f6f7c253700 1 -- 172.21.15.146:0/179870766 shutdown_connections 2024-09-13T18:27:12.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.073+0000 7f6f7c253700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6f54066730 0x7f6f54068bf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.073+0000 7f6f7c253700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f74157890 0x7f6f741f4230 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.073+0000 7f6f7c253700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f740c52e0 0x7f6f74157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.073+0000 7f6f7c253700 1 --2- 172.21.15.146:0/179870766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f740c4f00 0x7f6f7415edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.073+0000 7f6f7c253700 1 -- 172.21.15.146:0/179870766 >> 172.21.15.146:0/179870766 conn(0x7f6f740cb1d0 msgr2=0x7f6f740caa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:12.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.073+0000 7f6f7c253700 1 -- 172.21.15.146:0/179870766 shutdown_connections 2024-09-13T18:27:12.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.073+0000 7f6f7c253700 1 -- 172.21.15.146:0/179870766 wait complete. 2024-09-13T18:27:12.101 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:27:12.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.205+0000 7fc4fa838700 1 Processor -- start 2024-09-13T18:27:12.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.205+0000 7fc4fa838700 1 -- start start 2024-09-13T18:27:12.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa838700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4f4157fc0 0x7fc4f406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:12.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa838700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc4f40c8380 0x7fc4f406c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:12.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa838700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f40c9890 con 0x7fc4f4151840 2024-09-13T18:27:12.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa838700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f40c36f0 con 0x7fc4f40c8380 2024-09-13T18:27:12.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa838700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f41572d0 con 0x7fc4f4157fc0 2024-09-13T18:27:12.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4f9035700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4f4157fc0 0x7fc4f406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:12.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4f9035700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4f4157fc0 0x7fc4f406c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:51938/0 (socket says 172.21.15.146:51938) 2024-09-13T18:27:12.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4f9035700 1 -- 172.21.15.146:0/2209156360 learned_addr learned my addr 172.21.15.146:0/2209156360 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:12.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa037700 1 --2- 172.21.15.146:0/2209156360 >> v2:172.21.15.154:3300/0 conn(0x7fc4f40c8380 0x7fc4f406c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:12.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4f9035700 1 -- 172.21.15.146:0/2209156360 >> v2:172.21.15.154:3300/0 conn(0x7fc4f40c8380 msgr2=0x7fc4f406c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:12.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4f9035700 1 --2- 172.21.15.146:0/2209156360 >> v2:172.21.15.154:3300/0 conn(0x7fc4f40c8380 0x7fc4f406c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4f9035700 1 -- 172.21.15.146:0/2209156360 >> v1:172.21.15.154:6789/0 conn(0x7fc4f4151840 legacy=0x7fc4f40c1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:12.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4f9035700 1 -- 172.21.15.146:0/2209156360 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f40c9510 con 0x7fc4f4157fc0 2024-09-13T18:27:12.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4f9035700 1 --2- 172.21.15.146:0/2209156360 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4f4157fc0 0x7fc4f406c070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009910 tx=0x7fc4e4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eca64ef7830d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:12.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4f8834700 1 -- 172.21.15.146:0/2209156360 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4011830 con 0x7fc4f4157fc0 2024-09-13T18:27:12.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4f8834700 1 -- 172.21.15.146:0/2209156360 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e4011990 con 0x7fc4f4157fc0 2024-09-13T18:27:12.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4f8834700 1 -- 172.21.15.146:0/2209156360 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4011d40 con 0x7fc4f4157fc0 2024-09-13T18:27:12.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa838700 1 -- 172.21.15.146:0/2209156360 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4f4157fc0 msgr2=0x7fc4f406c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:12.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa838700 1 --2- 172.21.15.146:0/2209156360 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4f4157fc0 0x7fc4f406c070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009910 tx=0x7fc4e4009c20 comp rx=0 tx=0).stop 2024-09-13T18:27:12.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa838700 1 -- 172.21.15.146:0/2209156360 shutdown_connections 2024-09-13T18:27:12.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa838700 1 --2- 172.21.15.146:0/2209156360 >> v2:172.21.15.154:3300/0 conn(0x7fc4f40c8380 0x7fc4f406c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa838700 1 --2- 172.21.15.146:0/2209156360 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4f4157fc0 0x7fc4f406c070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa838700 1 -- 172.21.15.146:0/2209156360 >> 172.21.15.146:0/2209156360 conn(0x7fc4f40ccf70 msgr2=0x7fc4f40cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:12.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa838700 1 -- 172.21.15.146:0/2209156360 shutdown_connections 2024-09-13T18:27:12.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.209+0000 7fc4fa838700 1 -- 172.21.15.146:0/2209156360 wait complete. 2024-09-13T18:27:12.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4fa838700 1 Processor -- start 2024-09-13T18:27:12.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4fa838700 1 -- start start 2024-09-13T18:27:12.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4fa838700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f40c8380 0x7fc4f41e2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:12.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4fa838700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4f4151840 0x7fc4f4150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:12.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4fa838700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4f4157fc0 0x7fc4f4151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:12.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4fa838700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f41ef070 con 0x7fc4f40c8380 2024-09-13T18:27:12.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4fa838700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f41f0000 con 0x7fc4f4151840 2024-09-13T18:27:12.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4fa838700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f41e2fa0 con 0x7fc4f4157fc0 2024-09-13T18:27:12.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4f9836700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f40c8380 0x7fc4f41e2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:12.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4f9035700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4f4151840 0x7fc4f4150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:12.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4f9836700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f40c8380 0x7fc4f41e2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52408/0 (socket says 172.21.15.146:52408) 2024-09-13T18:27:12.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4f9836700 1 -- 172.21.15.146:0/1381637038 learned_addr learned my addr 172.21.15.146:0/1381637038 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:12.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4fa037700 1 --2- 172.21.15.146:0/1381637038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4f4157fc0 0x7fc4f4151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:12.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4f9836700 1 -- 172.21.15.146:0/1381637038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4f4157fc0 msgr2=0x7fc4f4151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:12.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4f9836700 1 --2- 172.21.15.146:0/1381637038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4f4157fc0 0x7fc4f4151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4f9836700 1 -- 172.21.15.146:0/1381637038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4f4151840 msgr2=0x7fc4f4150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:12.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4f9836700 1 --2- 172.21.15.146:0/1381637038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4f4151840 0x7fc4f4150e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4f9836700 1 -- 172.21.15.146:0/1381637038 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e4009570 con 0x7fc4f40c8380 2024-09-13T18:27:12.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4f9836700 1 --2- 172.21.15.146:0/1381637038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f40c8380 0x7fc4f41e2560 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000c560 tx=0x7fc4f000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:12.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4ea7fc700 1 -- 172.21.15.146:0/1381637038 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f00052d0 con 0x7fc4f40c8380 2024-09-13T18:27:12.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4ea7fc700 1 -- 172.21.15.146:0/1381637038 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f000ce90 con 0x7fc4f40c8380 2024-09-13T18:27:12.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4fa838700 1 -- 172.21.15.146:0/1381637038 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f41ef280 con 0x7fc4f40c8380 2024-09-13T18:27:12.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4ea7fc700 1 -- 172.21.15.146:0/1381637038 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f00036a0 con 0x7fc4f40c8380 2024-09-13T18:27:12.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.213+0000 7fc4fa838700 1 -- 172.21.15.146:0/1381637038 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f41ef520 con 0x7fc4f40c8380 2024-09-13T18:27:12.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.217+0000 7fc4fa838700 1 -- 172.21.15.146:0/1381637038 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f4067a60 con 0x7fc4f40c8380 2024-09-13T18:27:12.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.217+0000 7fc4ea7fc700 1 -- 172.21.15.146:0/1381637038 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc4f0003800 con 0x7fc4f40c8380 2024-09-13T18:27:12.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.217+0000 7fc4ea7fc700 1 --2- 172.21.15.146:0/1381637038 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc4d8066730 0x7fc4d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:12.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.217+0000 7fc4ea7fc700 1 -- 172.21.15.146:0/1381637038 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc4f0083770 con 0x7fc4f40c8380 2024-09-13T18:27:12.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.217+0000 7fc4f9035700 1 --2- 172.21.15.146:0/1381637038 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc4d8066730 0x7fc4d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:12.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.217+0000 7fc4f9035700 1 --2- 172.21.15.146:0/1381637038 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc4d8066730 0x7fc4d8068bf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4000c00 tx=0x7fc4e4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:12.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.225+0000 7fc4ea7fc700 1 -- 172.21.15.146:0/1381637038 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc4f00507f0 con 0x7fc4f40c8380 2024-09-13T18:27:12.240 INFO:tasks.ceph.mds.c.smithi146.stderr:2024-09-13T18:27:12.229+0000 7f6be472a700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-13T18:27:12.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.433+0000 7fc4fa838700 1 -- 172.21.15.146:0/1381637038 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4f4157aa0 con 0x7fc4f40c8380 2024-09-13T18:27:12.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.433+0000 7fc4ea7fc700 1 -- 172.21.15.146:0/1381637038 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6286 (secure 0 0 0) 0x7fc4f0053e10 con 0x7fc4f40c8380 2024-09-13T18:27:12.445 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:12.445 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:27:09.524480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"e","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:27:12.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.437+0000 7fc4fa838700 1 -- 172.21.15.146:0/1381637038 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc4d8066730 msgr2=0x7fc4d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:12.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.437+0000 7fc4fa838700 1 --2- 172.21.15.146:0/1381637038 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc4d8066730 0x7fc4d8068bf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4000c00 tx=0x7fc4e4008030 comp rx=0 tx=0).stop 2024-09-13T18:27:12.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.437+0000 7fc4fa838700 1 -- 172.21.15.146:0/1381637038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f40c8380 msgr2=0x7fc4f41e2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:12.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.437+0000 7fc4fa838700 1 --2- 172.21.15.146:0/1381637038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f40c8380 0x7fc4f41e2560 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000c560 tx=0x7fc4f000c870 comp rx=0 tx=0).stop 2024-09-13T18:27:12.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.437+0000 7fc4fa838700 1 -- 172.21.15.146:0/1381637038 shutdown_connections 2024-09-13T18:27:12.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.437+0000 7fc4fa838700 1 --2- 172.21.15.146:0/1381637038 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc4d8066730 0x7fc4d8068bf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.437+0000 7fc4fa838700 1 --2- 172.21.15.146:0/1381637038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4f4157fc0 0x7fc4f4151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.437+0000 7fc4fa838700 1 --2- 172.21.15.146:0/1381637038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4f4151840 0x7fc4f4150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.437+0000 7fc4fa838700 1 --2- 172.21.15.146:0/1381637038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f40c8380 0x7fc4f41e2560 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:12.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.437+0000 7fc4fa838700 1 -- 172.21.15.146:0/1381637038 >> 172.21.15.146:0/1381637038 conn(0x7fc4f40ccf70 msgr2=0x7fc4f41e1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:12.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.437+0000 7fc4fa838700 1 -- 172.21.15.146:0/1381637038 shutdown_connections 2024-09-13T18:27:12.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:12.437+0000 7fc4fa838700 1 -- 172.21.15.146:0/1381637038 wait complete. 2024-09-13T18:27:12.450 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 15 2024-09-13T18:27:13.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:26:32.323712+0000', 'modified': '2024-09-13T18:27:09.524480+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4290, 'mds_1': 4281, 'mds_2': 4269}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4269': {'gid': 4269, 'name': 'e', 'rank': 2, 'incarnation': 14, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.146:6833/2835746745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 2835746745}, {'type': 'v1', 'addr': '172.21.15.146:6833', 'nonce': 2835746745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4281': {'gid': 4281, 'name': 'a', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.146:6835/1642500214', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 1642500214}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 1642500214}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4290': {'gid': 4290, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.146:6837/31041557', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 31041557}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 31041557}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:27:13.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-13T18:27:13.068 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-09-13T18:27:13.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc258e4b700 1 Processor -- start 2024-09-13T18:27:13.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc258e4b700 1 -- start start 2024-09-13T18:27:13.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc258e4b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc25406b1f0 0x7fc2540c2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:13.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc258e4b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc25406b5d0 0x7fc254156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:13.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc258e4b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2540c2f70 con 0x7fc2540c2a50 2024-09-13T18:27:13.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc258e4b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25415a2e0 con 0x7fc25406b1f0 2024-09-13T18:27:13.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc258e4b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25406d870 con 0x7fc25406b5d0 2024-09-13T18:27:13.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc2537fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc25406b1f0 0x7fc2540c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:13.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc2537fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc25406b1f0 0x7fc2540c2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52410/0 (socket says 172.21.15.146:52410) 2024-09-13T18:27:13.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc2537fe700 1 -- 172.21.15.146:0/3247716786 learned_addr learned my addr 172.21.15.146:0/3247716786 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:13.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc253fff700 1 --1- 172.21.15.146:0/3247716786 >> v1:172.21.15.146:6789/0 conn(0x7fc2540c2a50 0x7fc2541500e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56692/0 (socket says 172.21.15.146:56692) 2024-09-13T18:27:13.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc252ffd700 1 --2- 172.21.15.146:0/3247716786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc25406b5d0 0x7fc254156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:13.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc2537fe700 1 -- 172.21.15.146:0/3247716786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc25406b5d0 msgr2=0x7fc254156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:13.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc2537fe700 1 --2- 172.21.15.146:0/3247716786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc25406b5d0 0x7fc254156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:13.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc2537fe700 1 -- 172.21.15.146:0/3247716786 >> v1:172.21.15.146:6789/0 conn(0x7fc2540c2a50 legacy=0x7fc2541500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:13.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc2537fe700 1 -- 172.21.15.146:0/3247716786 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc254159940 con 0x7fc25406b1f0 2024-09-13T18:27:13.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc2527fc700 1 -- 172.21.15.146:0/3247716786 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188415695 0 0) 0x7fc2540c2f70 con 0x7fc2540c2a50 2024-09-13T18:27:13.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc2537fe700 1 --2- 172.21.15.146:0/3247716786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25406b1f0 0x7fc2540c2510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00d150 tx=0x7fc23c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21685cb40f9b8d99 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:13.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc2527fc700 1 -- 172.21.15.146:0/3247716786 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c010070 con 0x7fc25406b1f0 2024-09-13T18:27:13.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc2527fc700 1 -- 172.21.15.146:0/3247716786 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c00ddc0 con 0x7fc25406b1f0 2024-09-13T18:27:13.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.186+0000 7fc2527fc700 1 -- 172.21.15.146:0/3247716786 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c00b310 con 0x7fc25406b1f0 2024-09-13T18:27:13.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 -- 172.21.15.146:0/3247716786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25406b1f0 msgr2=0x7fc2540c2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:13.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 --2- 172.21.15.146:0/3247716786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25406b1f0 0x7fc2540c2510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00d150 tx=0x7fc23c00d460 comp rx=0 tx=0).stop 2024-09-13T18:27:13.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 -- 172.21.15.146:0/3247716786 shutdown_connections 2024-09-13T18:27:13.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 --2- 172.21.15.146:0/3247716786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc25406b5d0 0x7fc254156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:13.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 --2- 172.21.15.146:0/3247716786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25406b1f0 0x7fc2540c2510 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:13.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 -- 172.21.15.146:0/3247716786 >> 172.21.15.146:0/3247716786 conn(0x7fc2540cbf60 msgr2=0x7fc2540cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:13.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 -- 172.21.15.146:0/3247716786 shutdown_connections 2024-09-13T18:27:13.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 -- 172.21.15.146:0/3247716786 wait complete. 2024-09-13T18:27:13.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 Processor -- start 2024-09-13T18:27:13.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 -- start start 2024-09-13T18:27:13.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc25406b1f0 0x7fc2540caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:13.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25406b5d0 0x7fc25414ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:13.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2540c2a50 0x7fc25414bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:13.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540c8760 con 0x7fc25406b5d0 2024-09-13T18:27:13.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540c8c70 con 0x7fc2540c2a50 2024-09-13T18:27:13.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.190+0000 7fc258e4b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540c93a0 con 0x7fc25406b1f0 2024-09-13T18:27:13.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc252ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25406b5d0 0x7fc25414ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:13.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc252ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25406b5d0 0x7fc25414ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52412/0 (socket says 172.21.15.146:52412) 2024-09-13T18:27:13.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc252ffd700 1 -- 172.21.15.146:0/2613341171 learned_addr learned my addr 172.21.15.146:0/2613341171 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:13.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc2537fe700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc25406b1f0 0x7fc2540caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:13.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc253fff700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2540c2a50 0x7fc25414bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:13.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc252ffd700 1 -- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc25406b1f0 msgr2=0x7fc2540caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:13.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc252ffd700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc25406b1f0 0x7fc2540caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:13.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc252ffd700 1 -- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2540c2a50 msgr2=0x7fc25414bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:13.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc252ffd700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2540c2a50 0x7fc25414bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:13.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc252ffd700 1 -- 172.21.15.146:0/2613341171 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c00bd40 con 0x7fc25406b5d0 2024-09-13T18:27:13.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc253fff700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2540c2a50 0x7fc25414bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:13.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc252ffd700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25406b5d0 0x7fc25414ba10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc24400e840 tx=0x7fc24400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:13.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc24bfff700 1 -- 172.21.15.146:0/2613341171 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244013610 con 0x7fc25406b5d0 2024-09-13T18:27:13.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc24bfff700 1 -- 172.21.15.146:0/2613341171 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc244013790 con 0x7fc25406b5d0 2024-09-13T18:27:13.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc258e4b700 1 -- 172.21.15.146:0/2613341171 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2540c1b30 con 0x7fc25406b5d0 2024-09-13T18:27:13.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc24bfff700 1 -- 172.21.15.146:0/2613341171 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244013b40 con 0x7fc25406b5d0 2024-09-13T18:27:13.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc258e4b700 1 -- 172.21.15.146:0/2613341171 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2541ee6b0 con 0x7fc25406b5d0 2024-09-13T18:27:13.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc24bfff700 1 -- 172.21.15.146:0/2613341171 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc244013ca0 con 0x7fc25406b5d0 2024-09-13T18:27:13.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.194+0000 7fc24bfff700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc234066730 0x7fc234068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:13.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc24bfff700 1 -- 172.21.15.146:0/2613341171 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc244084100 con 0x7fc25406b5d0 2024-09-13T18:27:13.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc2537fe700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc234066730 0x7fc234068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:13.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc25d35e740 1 Processor -- start 2024-09-13T18:27:13.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc25d35e740 1 -- start start 2024-09-13T18:27:13.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc2537fe700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc234066730 0x7fc234068bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc23c000f80 tx=0x7fc23c00b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:13.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc25d35e740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223e9d0 0x223edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:13.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc25d35e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223f2f0 0x223f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:13.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc25d35e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 0x22680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:13.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc25d35e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x2189d80 con 0x223fc10 2024-09-13T18:27:13.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc25d35e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x2187510 con 0x223f2f0 2024-09-13T18:27:13.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc25d35e740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x2182e00 con 0x223e9d0 2024-09-13T18:27:13.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc253fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 0x22680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:13.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc253fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 0x22680c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52416/0 (socket says 172.21.15.146:52416) 2024-09-13T18:27:13.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc253fff700 1 -- 172.21.15.146:0/741938090 learned_addr learned my addr 172.21.15.146:0/741938090 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:13.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc252ffd700 1 --2- 172.21.15.146:0/741938090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223f2f0 0x223f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:13.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc2537fe700 1 --2- 172.21.15.146:0/741938090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223e9d0 0x223edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:13.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc253fff700 1 -- 172.21.15.146:0/741938090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223e9d0 msgr2=0x223edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:13.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc253fff700 1 --2- 172.21.15.146:0/741938090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223e9d0 0x223edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:13.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc253fff700 1 -- 172.21.15.146:0/741938090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223f2f0 msgr2=0x223f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:13.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc253fff700 1 --2- 172.21.15.146:0/741938090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223f2f0 0x223f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:13.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc253fff700 1 -- 172.21.15.146:0/741938090 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ff5960 con 0x223fc10 2024-09-13T18:27:13.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc253fff700 1 --2- 172.21.15.146:0/741938090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 0x22680c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0017b0 tx=0x7fc24c001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e94e8848a09b3a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:13.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc2497fa700 1 -- 172.21.15.146:0/741938090 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c004ca0 con 0x223fc10 2024-09-13T18:27:13.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc2497fa700 1 -- 172.21.15.146:0/741938090 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c004e00 con 0x223fc10 2024-09-13T18:27:13.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc2497fa700 1 -- 172.21.15.146:0/741938090 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0072d0 con 0x223fc10 2024-09-13T18:27:13.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc25d35e740 1 -- 172.21.15.146:0/741938090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 msgr2=0x22680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:13.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc25d35e740 1 --2- 172.21.15.146:0/741938090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 0x22680c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0017b0 tx=0x7fc24c001ac0 comp rx=0 tx=0).stop 2024-09-13T18:27:13.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc25d35e740 1 -- 172.21.15.146:0/741938090 shutdown_connections 2024-09-13T18:27:13.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc25d35e740 1 --2- 172.21.15.146:0/741938090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 0x22680c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:13.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.198+0000 7fc25d35e740 1 --2- 172.21.15.146:0/741938090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223f2f0 0x223f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:13.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc25d35e740 1 --2- 172.21.15.146:0/741938090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223e9d0 0x223edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:13.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc25d35e740 1 -- 172.21.15.146:0/741938090 >> 172.21.15.146:0/741938090 conn(0x1faa4d0 msgr2=0x2255ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:13.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc25d35e740 1 -- 172.21.15.146:0/741938090 shutdown_connections 2024-09-13T18:27:13.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc25d35e740 1 -- 172.21.15.146:0/741938090 wait complete. 2024-09-13T18:27:13.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc25d35e740 1 -- start start 2024-09-13T18:27:13.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc25d35e740 1 Processor -- start 2024-09-13T18:27:13.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc25d35e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223e9d0 0x226b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:13.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc25d35e740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223f2f0 0x226a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:13.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc25d35e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 0x226a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:13.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc25d35e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x2140730 con 0x223fc10 2024-09-13T18:27:13.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc25d35e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x21465b0 con 0x223e9d0 2024-09-13T18:27:13.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc25d35e740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x202eaa0 con 0x223f2f0 2024-09-13T18:27:13.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc25d35e740 10 client.0 _ll_register_callbacks cb 0x217c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:13.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc253fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 0x226a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:13.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc253fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 0x226a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52430/0 (socket says 172.21.15.146:52430) 2024-09-13T18:27:13.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc253fff700 1 -- 172.21.15.146:0/3838868059 learned_addr learned my addr 172.21.15.146:0/3838868059 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:13.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc2537fe700 1 --2- 172.21.15.146:0/3838868059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223e9d0 0x226b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:13.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc252ffd700 1 --2- 172.21.15.146:0/3838868059 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223f2f0 0x226a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:13.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc253fff700 1 -- 172.21.15.146:0/3838868059 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223f2f0 msgr2=0x226a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:13.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc253fff700 1 --2- 172.21.15.146:0/3838868059 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223f2f0 0x226a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:13.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc253fff700 1 -- 172.21.15.146:0/3838868059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223e9d0 msgr2=0x226b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:13.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc253fff700 1 --2- 172.21.15.146:0/3838868059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223e9d0 0x226b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:13.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc253fff700 1 -- 172.21.15.146:0/3838868059 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24c0013e0 con 0x223fc10 2024-09-13T18:27:13.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.202+0000 7fc253fff700 1 --2- 172.21.15.146:0/3838868059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 0x226a9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x217b9d0 tx=0x7fc24c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7645f886fe68ad96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:13.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc2327fc700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-13T18:27:13.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc2327fc700 1 -- 172.21.15.146:0/3838868059 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0072c0 con 0x223fc10 2024-09-13T18:27:13.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc2327fc700 1 -- 172.21.15.146:0/3838868059 <== mon.0 v2:172.21.15.146:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc24c007420 con 0x223fc10 2024-09-13T18:27:13.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc25d35e740 1 -- 172.21.15.146:0/3838868059 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1fb71e0 con 0x223fc10 2024-09-13T18:27:13.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc2327fc700 1 -- 172.21.15.146:0/3838868059 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc24c0075e0 con 0x223fc10 2024-09-13T18:27:13.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc25d35e740 10 client.4923 fetch_fsmap learned FSMap version 15 2024-09-13T18:27:13.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc25d35e740 1 -- 172.21.15.146:0/3838868059 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x2149220 con 0x223fc10 2024-09-13T18:27:13.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc2327fc700 1 -- 172.21.15.146:0/3838868059 <== mon.0 v2:172.21.15.146:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1fb71e0 con 0x223fc10 2024-09-13T18:27:13.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc2327fc700 1 -- 172.21.15.146:0/3838868059 <== mon.0 v2:172.21.15.146:3300/0 5 ==== fsmap(e 15) v1 ==== 4236+0+0 (secure 0 0 0) 0x7fc24c0283a0 con 0x223fc10 2024-09-13T18:27:13.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc25d35e740 10 client.4923 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T18:27:13.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc25d35e740 10 client.4923 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-13T18:27:13.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc25d35e740 20 client.4923 populate_metadata read hostname 'smithi146' 2024-09-13T18:27:13.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc25d35e740 1 --2- 172.21.15.146:0/3838868059 >> [v2:172.21.15.146:6832/2835746745,v1:172.21.15.146:6833/2835746745] conn(0x2244a50 0x2244e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:13.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc25d35e740 4 client.4923 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:13.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc25d35e740 1 -- 172.21.15.146:0/3838868059 --> [v2:172.21.15.146:6832/2835746745,v1:172.21.15.146:6833/2835746745] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2001460 con 0x2244a50 2024-09-13T18:27:13.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc2537fe700 1 --2- 172.21.15.146:0/3838868059 >> [v2:172.21.15.146:6832/2835746745,v1:172.21.15.146:6833/2835746745] conn(0x2244a50 0x2244e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:13.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc2537fe700 1 --2- 172.21.15.146:0/3838868059 >> [v2:172.21.15.146:6832/2835746745,v1:172.21.15.146:6833/2835746745] conn(0x2244a50 0x2244e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76605cb5874d6ea2 server_cookie=d6254d21fd441dfa in_seq=0 out_seq=0 2024-09-13T18:27:13.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:13.206+0000 7fc2327fc700 10 client.4923 ms_handle_connect on v2:172.21.15.146:6832/2835746745 2024-09-13T18:27:14.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc2327fc700 1 -- 172.21.15.146:0/3838868059 <== mds.2 v2:172.21.15.146:6832/2835746745 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x2001460 con 0x2244a50 2024-09-13T18:27:14.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc2327fc700 10 client.4923 handle_command_reply: tid=0 2024-09-13T18:27:14.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc25d35e740 1 client.4923 shutdown 2024-09-13T18:27:14.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc25d35e740 1 -- 172.21.15.146:0/3838868059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 msgr2=0x226a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc25d35e740 1 --2- 172.21.15.146:0/3838868059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 0x226a9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x217b9d0 tx=0x7fc24c000ed0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc25d35e740 1 -- 172.21.15.146:0/3838868059 shutdown_connections 2024-09-13T18:27:14.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc25d35e740 1 --2- 172.21.15.146:0/3838868059 >> [v2:172.21.15.146:6832/2835746745,v1:172.21.15.146:6833/2835746745] conn(0x2244a50 0x2244e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc25d35e740 1 --2- 172.21.15.146:0/3838868059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223fc10 0x226a9f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc2327fc700 0 client.4923 ms_handle_reset on v2:172.21.15.146:6832/2835746745 2024-09-13T18:27:14.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc25d35e740 1 --2- 172.21.15.146:0/3838868059 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223f2f0 0x226a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc25d35e740 1 --2- 172.21.15.146:0/3838868059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223e9d0 0x226b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc25d35e740 1 -- 172.21.15.146:0/3838868059 >> 172.21.15.146:0/3838868059 conn(0x1faa4d0 msgr2=0x223e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:14.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc25d35e740 1 -- 172.21.15.146:0/3838868059 shutdown_connections 2024-09-13T18:27:14.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc25d35e740 1 -- 172.21.15.146:0/3838868059 wait complete. 2024-09-13T18:27:14.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.534+0000 7fc25d35e740 20 client.4923 trim_cache size 0 max 16384 2024-09-13T18:27:14.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc25d35e740 1 Processor -- start 2024-09-13T18:27:14.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc25d35e740 1 -- start start 2024-09-13T18:27:14.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc25d35e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223e9d0 0x22681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc25d35e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223f2f0 0x2268700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc25d35e740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223fc10 0x2247e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc25d35e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x226a070 con 0x223f2f0 2024-09-13T18:27:14.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc25d35e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x214d4f0 con 0x223e9d0 2024-09-13T18:27:14.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc25d35e740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x21814d0 con 0x223fc10 2024-09-13T18:27:14.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc252ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223f2f0 0x2268700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc252ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223f2f0 0x2268700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52440/0 (socket says 172.21.15.146:52440) 2024-09-13T18:27:14.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc2537fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223e9d0 0x22681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc252ffd700 1 -- 172.21.15.146:0/586817698 learned_addr learned my addr 172.21.15.146:0/586817698 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:14.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc253fff700 1 --2- 172.21.15.146:0/586817698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223fc10 0x2247e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc252ffd700 1 -- 172.21.15.146:0/586817698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223fc10 msgr2=0x2247e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:14.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc252ffd700 1 --2- 172.21.15.146:0/586817698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223fc10 0x2247e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc252ffd700 1 -- 172.21.15.146:0/586817698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223e9d0 msgr2=0x22681c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:14.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc252ffd700 1 --2- 172.21.15.146:0/586817698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223e9d0 0x22681c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc252ffd700 1 -- 172.21.15.146:0/586817698 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x212f450 con 0x223f2f0 2024-09-13T18:27:14.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.554+0000 7fc253fff700 1 --2- 172.21.15.146:0/586817698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223fc10 0x2247e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:14.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc252ffd700 1 --2- 172.21.15.146:0/586817698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223f2f0 0x2268700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc244054b40 tx=0x7fc244015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e070b4248ceb65d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:14.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc249ffb700 1 -- 172.21.15.146:0/586817698 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244058c70 con 0x223f2f0 2024-09-13T18:27:14.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc249ffb700 1 -- 172.21.15.146:0/586817698 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24405d230 con 0x223f2f0 2024-09-13T18:27:14.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc249ffb700 1 -- 172.21.15.146:0/586817698 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244054330 con 0x223f2f0 2024-09-13T18:27:14.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 -- 172.21.15.146:0/586817698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223f2f0 msgr2=0x2268700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 --2- 172.21.15.146:0/586817698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223f2f0 0x2268700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc244054b40 tx=0x7fc244015000 comp rx=0 tx=0).stop 2024-09-13T18:27:14.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 -- 172.21.15.146:0/586817698 shutdown_connections 2024-09-13T18:27:14.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 --2- 172.21.15.146:0/586817698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223fc10 0x2247e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 --2- 172.21.15.146:0/586817698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223f2f0 0x2268700 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 --2- 172.21.15.146:0/586817698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223e9d0 0x22681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 -- 172.21.15.146:0/586817698 >> 172.21.15.146:0/586817698 conn(0x1faa4d0 msgr2=0x223e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:14.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 -- 172.21.15.146:0/586817698 shutdown_connections 2024-09-13T18:27:14.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 -- 172.21.15.146:0/586817698 wait complete. 2024-09-13T18:27:14.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 -- start start 2024-09-13T18:27:14.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 Processor -- start 2024-09-13T18:27:14.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223e9d0 0x22c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223f2f0 0x22700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223fc10 0x22bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x226b0b0 con 0x223e9d0 2024-09-13T18:27:14.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x223e780 con 0x223fc10 2024-09-13T18:27:14.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x226a290 con 0x223f2f0 2024-09-13T18:27:14.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc25d35e740 10 client.0 _ll_register_callbacks cb 0x7fc24c007e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:14.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc2537fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223e9d0 0x22c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc2537fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223e9d0 0x22c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52442/0 (socket says 172.21.15.146:52442) 2024-09-13T18:27:14.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc2537fe700 1 -- 172.21.15.146:0/2600569386 learned_addr learned my addr 172.21.15.146:0/2600569386 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:14.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc252ffd700 1 --2- 172.21.15.146:0/2600569386 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223f2f0 0x22700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.558+0000 7fc253fff700 1 --2- 172.21.15.146:0/2600569386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223fc10 0x22bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2537fe700 1 -- 172.21.15.146:0/2600569386 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223f2f0 msgr2=0x22700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2537fe700 1 --2- 172.21.15.146:0/2600569386 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223f2f0 0x22700f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2537fe700 1 -- 172.21.15.146:0/2600569386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223fc10 msgr2=0x22bd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2537fe700 1 --2- 172.21.15.146:0/2600569386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223fc10 0x22bd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2537fe700 1 -- 172.21.15.146:0/2600569386 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2541ee6b0 con 0x223e9d0 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2537fe700 1 --2- 172.21.15.146:0/2600569386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223e9d0 0x22c4320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc2540c6620 tx=0x7fc23c004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4fb11334a94da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2327fc700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2327fc700 1 -- 172.21.15.146:0/2600569386 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c02f8b0 con 0x223e9d0 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2327fc700 1 -- 172.21.15.146:0/2600569386 <== mon.0 v2:172.21.15.146:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc23c00a610 con 0x223e9d0 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 -- 172.21.15.146:0/2600569386 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc24c004e00 con 0x223e9d0 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2327fc700 1 -- 172.21.15.146:0/2600569386 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc23c00a7d0 con 0x223e9d0 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 10 client.4929 fetch_fsmap learned FSMap version 15 2024-09-13T18:27:14.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 -- 172.21.15.146:0/2600569386 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x2123720 con 0x223e9d0 2024-09-13T18:27:14.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2327fc700 1 -- 172.21.15.146:0/2600569386 <== mon.0 v2:172.21.15.146:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c004e00 con 0x223e9d0 2024-09-13T18:27:14.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2327fc700 1 -- 172.21.15.146:0/2600569386 <== mon.0 v2:172.21.15.146:3300/0 5 ==== fsmap(e 15) v1 ==== 4236+0+0 (secure 0 0 0) 0x7fc23c00a930 con 0x223e9d0 2024-09-13T18:27:14.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 10 client.4929 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T18:27:14.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 10 client.4929 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-13T18:27:14.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 20 client.4929 populate_metadata read hostname 'smithi146' 2024-09-13T18:27:14.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 --2- 172.21.15.146:0/2600569386 >> [v2:172.21.15.146:6832/2835746745,v1:172.21.15.146:6833/2835746745] conn(0x2244a50 0x2264cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 4 client.4929 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:27:14.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 -- 172.21.15.146:0/2600569386 --> [v2:172.21.15.146:6832/2835746745,v1:172.21.15.146:6833/2835746745] -- command(tid 0: {"prefix": "status"}) v1 -- 0x20261e0 con 0x2244a50 2024-09-13T18:27:14.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc252ffd700 1 --2- 172.21.15.146:0/2600569386 >> [v2:172.21.15.146:6832/2835746745,v1:172.21.15.146:6833/2835746745] conn(0x2244a50 0x2264cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2327fc700 10 client.4929 ms_handle_connect on v2:172.21.15.146:6832/2835746745 2024-09-13T18:27:14.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc252ffd700 1 --2- 172.21.15.146:0/2600569386 >> [v2:172.21.15.146:6832/2835746745,v1:172.21.15.146:6833/2835746745] conn(0x2244a50 0x2264cc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f4825f35648f167 server_cookie=7e24de6b547b1736 in_seq=0 out_seq=0 2024-09-13T18:27:14.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2327fc700 1 -- 172.21.15.146:0/2600569386 <== mds.2 v2:172.21.15.146:6832/2835746745 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x20261e0 con 0x2244a50 2024-09-13T18:27:14.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2327fc700 10 client.4929 handle_command_reply: tid=0 2024-09-13T18:27:14.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 client.4929 shutdown 2024-09-13T18:27:14.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 -- 172.21.15.146:0/2600569386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223e9d0 msgr2=0x22c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 --2- 172.21.15.146:0/2600569386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223e9d0 0x22c4320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc2540c6620 tx=0x7fc23c004b60 comp rx=0 tx=0).stop 2024-09-13T18:27:14.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 -- 172.21.15.146:0/2600569386 shutdown_connections 2024-09-13T18:27:14.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 --2- 172.21.15.146:0/2600569386 >> [v2:172.21.15.146:6832/2835746745,v1:172.21.15.146:6833/2835746745] conn(0x2244a50 0x2264cc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 --2- 172.21.15.146:0/2600569386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x223fc10 0x22bd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc2327fc700 0 client.4929 ms_handle_reset on v2:172.21.15.146:6832/2835746745 2024-09-13T18:27:14.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 --2- 172.21.15.146:0/2600569386 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x223f2f0 0x22700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 --2- 172.21.15.146:0/2600569386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x223e9d0 0x22c4320 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 -- 172.21.15.146:0/2600569386 >> 172.21.15.146:0/2600569386 conn(0x1faa4d0 msgr2=0x22688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:14.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 -- 172.21.15.146:0/2600569386 shutdown_connections 2024-09-13T18:27:14.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.562+0000 7fc25d35e740 1 -- 172.21.15.146:0/2600569386 wait complete. 2024-09-13T18:27:14.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.566+0000 7fc25d35e740 20 client.4929 trim_cache size 0 max 16384 2024-09-13T18:27:14.576 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:27:14.576 INFO:teuthology.orchestra.run.smithi146.stdout: "cluster_fsid": "509e851e-cdbc-4e57-a792-d3d1aeeeba86", 2024-09-13T18:27:14.576 INFO:teuthology.orchestra.run.smithi146.stdout: "whoami": 2, 2024-09-13T18:27:14.577 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4269, 2024-09-13T18:27:14.577 INFO:teuthology.orchestra.run.smithi146.stdout: "want_state": "up:active", 2024-09-13T18:27:14.577 INFO:teuthology.orchestra.run.smithi146.stdout: "state": "up:active", 2024-09-13T18:27:14.577 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs", 2024-09-13T18:27:14.577 INFO:teuthology.orchestra.run.smithi146.stdout: "rank_uptime": 11.062215585000001, 2024-09-13T18:27:14.577 INFO:teuthology.orchestra.run.smithi146.stdout: "mdsmap_epoch": 15, 2024-09-13T18:27:14.577 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_epoch": 25, 2024-09-13T18:27:14.577 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_epoch_barrier": 25, 2024-09-13T18:27:14.577 INFO:teuthology.orchestra.run.smithi146.stdout: "uptime": 48.360964623000001 2024-09-13T18:27:14.577 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:27:14.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.566+0000 7fc258e4b700 1 -- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc234066730 msgr2=0x7fc234068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.566+0000 7fc258e4b700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc234066730 0x7fc234068bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc23c000f80 tx=0x7fc23c00b000 comp rx=0 tx=0).stop 2024-09-13T18:27:14.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.566+0000 7fc258e4b700 1 -- 172.21.15.146:0/2613341171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25406b5d0 msgr2=0x7fc25414ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.566+0000 7fc258e4b700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25406b5d0 0x7fc25414ba10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc24400e840 tx=0x7fc24400ec00 comp rx=0 tx=0).stop 2024-09-13T18:27:14.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.566+0000 7fc258e4b700 1 -- 172.21.15.146:0/2613341171 shutdown_connections 2024-09-13T18:27:14.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.566+0000 7fc258e4b700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc234066730 0x7fc234068bf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.566+0000 7fc258e4b700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2540c2a50 0x7fc25414bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.566+0000 7fc258e4b700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25406b5d0 0x7fc25414ba10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.566+0000 7fc258e4b700 1 --2- 172.21.15.146:0/2613341171 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc25406b1f0 0x7fc2540caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.566+0000 7fc258e4b700 1 -- 172.21.15.146:0/2613341171 >> 172.21.15.146:0/2613341171 conn(0x7fc2540cbf60 msgr2=0x7fc2541488f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:14.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.566+0000 7fc258e4b700 1 -- 172.21.15.146:0/2613341171 shutdown_connections 2024-09-13T18:27:14.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.566+0000 7fc258e4b700 1 -- 172.21.15.146:0/2613341171 wait complete. 2024-09-13T18:27:14.591 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T18:27:14.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.698+0000 7fea4ec15700 1 Processor -- start 2024-09-13T18:27:14.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.698+0000 7fea4ec15700 1 -- start start 2024-09-13T18:27:14.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea4ec15700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fea480c7b20 0x7fea48066090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea4ec15700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4806e480 con 0x7fea480c2670 2024-09-13T18:27:14.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea4ec15700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4806e720 con 0x7fea48150900 2024-09-13T18:27:14.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea4ec15700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea48150ce0 con 0x7fea480c7b20 2024-09-13T18:27:14.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea4d1b2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fea480c7b20 0x7fea48066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea4d1b2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fea480c7b20 0x7fea48066090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52456/0 (socket says 172.21.15.146:52456) 2024-09-13T18:27:14.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea4d1b2700 1 -- 172.21.15.146:0/3411917487 learned_addr learned my addr 172.21.15.146:0/3411917487 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:14.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea47fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fea480c2670 0x7fea480c7410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56702/0 (socket says 172.21.15.146:56702) 2024-09-13T18:27:14.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea4c9b1700 1 --1- 172.21.15.146:0/3411917487 >> v1:172.21.15.154:6789/0 conn(0x7fea48150900 0x7fea480c8120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:59080/0 (socket says 172.21.15.146:59080) 2024-09-13T18:27:14.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea4d1b2700 1 -- 172.21.15.146:0/3411917487 >> v1:172.21.15.154:6789/0 conn(0x7fea48150900 legacy=0x7fea480c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea4d1b2700 1 -- 172.21.15.146:0/3411917487 >> v1:172.21.15.146:6789/0 conn(0x7fea480c2670 legacy=0x7fea480c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea4d1b2700 1 -- 172.21.15.146:0/3411917487 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea480c46c0 con 0x7fea480c7b20 2024-09-13T18:27:14.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea4d1b2700 1 --2- 172.21.15.146:0/3411917487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea480c7b20 0x7fea48066090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fea40009e50 tx=0x7fea4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6855502ed0b2ac34 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:14.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea477fe700 1 -- 172.21.15.146:0/3411917487 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4000e070 con 0x7fea480c7b20 2024-09-13T18:27:14.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea477fe700 1 -- 172.21.15.146:0/3411917487 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4000b9c0 con 0x7fea480c7b20 2024-09-13T18:27:14.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.702+0000 7fea477fe700 1 -- 172.21.15.146:0/3411917487 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4000bd70 con 0x7fea480c7b20 2024-09-13T18:27:14.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 -- 172.21.15.146:0/3411917487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea480c7b20 msgr2=0x7fea48066090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 --2- 172.21.15.146:0/3411917487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea480c7b20 0x7fea48066090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fea40009e50 tx=0x7fea4000b330 comp rx=0 tx=0).stop 2024-09-13T18:27:14.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 -- 172.21.15.146:0/3411917487 shutdown_connections 2024-09-13T18:27:14.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 --2- 172.21.15.146:0/3411917487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea480c7b20 0x7fea48066090 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 -- 172.21.15.146:0/3411917487 >> 172.21.15.146:0/3411917487 conn(0x7fea48066920 msgr2=0x7fea4806acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:14.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 -- 172.21.15.146:0/3411917487 shutdown_connections 2024-09-13T18:27:14.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 -- 172.21.15.146:0/3411917487 wait complete. 2024-09-13T18:27:14.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 Processor -- start 2024-09-13T18:27:14.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 -- start start 2024-09-13T18:27:14.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea480c2670 0x7fea481483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea480c7b20 0x7fea481edb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea48150900 0x7fea48142380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4814e120 con 0x7fea480c7b20 2024-09-13T18:27:14.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea481ed380 con 0x7fea480c2670 2024-09-13T18:27:14.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.706+0000 7fea4ec15700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4814e350 con 0x7fea48150900 2024-09-13T18:27:14.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea47fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea480c7b20 0x7fea481edb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea47fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea480c7b20 0x7fea481edb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52460/0 (socket says 172.21.15.146:52460) 2024-09-13T18:27:14.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea47fff700 1 -- 172.21.15.146:0/96812445 learned_addr learned my addr 172.21.15.146:0/96812445 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:14.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea480c2670 0x7fea481483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea4d1b2700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea48150900 0x7fea48142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea47fff700 1 -- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea48150900 msgr2=0x7fea48142380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea47fff700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea48150900 0x7fea48142380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea47fff700 1 -- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea480c2670 msgr2=0x7fea481483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea47fff700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea480c2670 0x7fea481483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea47fff700 1 -- 172.21.15.146:0/96812445 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea40009b30 con 0x7fea480c7b20 2024-09-13T18:27:14.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea47fff700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea480c7b20 0x7fea481edb10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c460 tx=0x7fea3c00c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:14.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea457fa700 1 -- 172.21.15.146:0/96812445 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c005cd0 con 0x7fea480c7b20 2024-09-13T18:27:14.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea457fa700 1 -- 172.21.15.146:0/96812445 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c005e30 con 0x7fea480c7b20 2024-09-13T18:27:14.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea4ec15700 1 -- 172.21.15.146:0/96812445 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4814e630 con 0x7fea480c7b20 2024-09-13T18:27:14.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea457fa700 1 -- 172.21.15.146:0/96812445 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0072d0 con 0x7fea480c7b20 2024-09-13T18:27:14.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.710+0000 7fea4ec15700 1 -- 172.21.15.146:0/96812445 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea481ed5f0 con 0x7fea480c7b20 2024-09-13T18:27:14.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.714+0000 7fea457fa700 1 -- 172.21.15.146:0/96812445 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fea3c007430 con 0x7fea480c7b20 2024-09-13T18:27:14.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.714+0000 7fea457fa700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea34066730 0x7fea34068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.714+0000 7fea457fa700 1 -- 172.21.15.146:0/96812445 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fea3c0837d0 con 0x7fea480c7b20 2024-09-13T18:27:14.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.714+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea34066730 0x7fea34068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.714+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea34066730 0x7fea34068bf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fea38000f60 tx=0x7fea38007ef0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:14.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.714+0000 7fea51ec6740 1 Processor -- start 2024-09-13T18:27:14.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.714+0000 7fea51ec6740 1 -- start start 2024-09-13T18:27:14.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 0x1a8c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8cd00 0x1a8d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8d620 0x1a8da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x19d7d80 con 0x1aa0ba0 2024-09-13T18:27:14.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x19d5510 con 0x1a8d620 2024-09-13T18:27:14.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x19d0e00 con 0x1a8cd00 2024-09-13T18:27:14.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea4c9b1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 0x1a8c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea4c9b1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 0x1a8c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52472/0 (socket says 172.21.15.146:52472) 2024-09-13T18:27:14.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea4c9b1700 1 -- 172.21.15.146:0/843484321 learned_addr learned my addr 172.21.15.146:0/843484321 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:14.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea47fff700 1 --2- 172.21.15.146:0/843484321 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8cd00 0x1a8d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea4d1b2700 1 --2- 172.21.15.146:0/843484321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8d620 0x1a8da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea4c9b1700 1 -- 172.21.15.146:0/843484321 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8cd00 msgr2=0x1a8d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:14.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/843484321 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8cd00 0x1a8d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea4c9b1700 1 -- 172.21.15.146:0/843484321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8d620 msgr2=0x1a8da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:14.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/843484321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8d620 0x1a8da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea4c9b1700 1 -- 172.21.15.146:0/843484321 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1843960 con 0x1aa0ba0 2024-09-13T18:27:14.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea4d1b2700 1 --2- 172.21.15.146:0/843484321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8d620 0x1a8da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:14.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/843484321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 0x1a8c7c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fea380015a0 tx=0x7fea38013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c11b889aea2d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:14.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea2e7fc700 1 -- 172.21.15.146:0/843484321 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38001c70 con 0x1aa0ba0 2024-09-13T18:27:14.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea2e7fc700 1 -- 172.21.15.146:0/843484321 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea380133f0 con 0x1aa0ba0 2024-09-13T18:27:14.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea2e7fc700 1 -- 172.21.15.146:0/843484321 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38013760 con 0x1aa0ba0 2024-09-13T18:27:14.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 -- 172.21.15.146:0/843484321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 msgr2=0x1a8c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 --2- 172.21.15.146:0/843484321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 0x1a8c7c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fea380015a0 tx=0x7fea38013000 comp rx=0 tx=0).stop 2024-09-13T18:27:14.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 -- 172.21.15.146:0/843484321 shutdown_connections 2024-09-13T18:27:14.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 --2- 172.21.15.146:0/843484321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8d620 0x1a8da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 --2- 172.21.15.146:0/843484321 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8cd00 0x1a8d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 --2- 172.21.15.146:0/843484321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 0x1a8c7c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 -- 172.21.15.146:0/843484321 >> 172.21.15.146:0/843484321 conn(0x17f84d0 msgr2=0x1aa3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:14.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 -- 172.21.15.146:0/843484321 shutdown_connections 2024-09-13T18:27:14.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 -- 172.21.15.146:0/843484321 wait complete. 2024-09-13T18:27:14.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 -- start start 2024-09-13T18:27:14.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.718+0000 7fea51ec6740 1 Processor -- start 2024-09-13T18:27:14.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8cd00 0x1aa3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8d620 0x1ab3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1aa0ba0 0x1ab3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x198e730 con 0x1a8cd00 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x19945b0 con 0x1aa0ba0 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x187caa0 con 0x1a8d620 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 10 client.0 _ll_register_callbacks cb 0x19ca610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea4c9b1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8cd00 0x1aa3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea4c9b1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8cd00 0x1aa3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52476/0 (socket says 172.21.15.146:52476) 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea4c9b1700 1 -- 172.21.15.146:0/3488486661 learned_addr learned my addr 172.21.15.146:0/3488486661 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea47fff700 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8d620 0x1ab3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea4d1b2700 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1aa0ba0 0x1ab3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea4c9b1700 1 -- 172.21.15.146:0/3488486661 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8d620 msgr2=0x1ab3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8d620 0x1ab3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea4c9b1700 1 -- 172.21.15.146:0/3488486661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1aa0ba0 msgr2=0x1ab3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1aa0ba0 0x1ab3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea4c9b1700 1 -- 172.21.15.146:0/3488486661 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4806e720 con 0x1a8cd00 2024-09-13T18:27:14.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea4d1b2700 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1aa0ba0 0x1ab3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8cd00 0x1aa3970 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fea380015a0 tx=0x7fea38000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ace29907ee10a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea277fe700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea277fe700 1 -- 172.21.15.146:0/3488486661 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38013e60 con 0x1a8cd00 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea277fe700 1 -- 172.21.15.146:0/3488486661 <== mon.0 v2:172.21.15.146:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fea3801b050 con 0x1a8cd00 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 1 -- 172.21.15.146:0/3488486661 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x18051e0 con 0x1a8cd00 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea277fe700 1 -- 172.21.15.146:0/3488486661 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea3802c420 con 0x1a8cd00 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 10 client.4944 fetch_fsmap learned FSMap version 15 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 1 -- 172.21.15.146:0/3488486661 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x1997220 con 0x1a8cd00 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea277fe700 1 -- 172.21.15.146:0/3488486661 <== mon.0 v2:172.21.15.146:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x18051e0 con 0x1a8cd00 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea277fe700 1 -- 172.21.15.146:0/3488486661 <== mon.0 v2:172.21.15.146:3300/0 5 ==== fsmap(e 15) v1 ==== 4236+0+0 (secure 0 0 0) 0x7fea3802c580 con 0x1a8cd00 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 10 client.4944 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 10 client.4944 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 20 client.4944 populate_metadata read hostname 'smithi146' 2024-09-13T18:27:14.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] conn(0x1ab5440 0x1ab5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 4 client.4944 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:14.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea51ec6740 1 -- 172.21.15.146:0/3488486661 --> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x184f460 con 0x1ab5440 2024-09-13T18:27:14.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea47fff700 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] conn(0x1ab5440 0x1ab5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea277fe700 10 client.4944 ms_handle_connect on v2:172.21.15.146:6834/1642500214 2024-09-13T18:27:14.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.722+0000 7fea47fff700 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] conn(0x1ab5440 0x1ab5820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca44917a0e46db3b server_cookie=9907764d0db401c5 in_seq=0 out_seq=0 2024-09-13T18:27:14.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea277fe700 1 -- 172.21.15.146:0/3488486661 <== mds.1 v2:172.21.15.146:6834/1642500214 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x184f460 con 0x1ab5440 2024-09-13T18:27:14.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea277fe700 10 client.4944 handle_command_reply: tid=0 2024-09-13T18:27:14.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea51ec6740 1 client.4944 shutdown 2024-09-13T18:27:14.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea51ec6740 1 -- 172.21.15.146:0/3488486661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8cd00 msgr2=0x1aa3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea51ec6740 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8cd00 0x1aa3970 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fea380015a0 tx=0x7fea38000ed0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea51ec6740 1 -- 172.21.15.146:0/3488486661 shutdown_connections 2024-09-13T18:27:14.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea51ec6740 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] conn(0x1ab5440 0x1ab5820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea51ec6740 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1aa0ba0 0x1ab3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea277fe700 0 client.4944 ms_handle_reset on v2:172.21.15.146:6834/1642500214 2024-09-13T18:27:14.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea51ec6740 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8d620 0x1ab3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea51ec6740 1 --2- 172.21.15.146:0/3488486661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8cd00 0x1aa3970 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea51ec6740 1 -- 172.21.15.146:0/3488486661 >> 172.21.15.146:0/3488486661 conn(0x17f84d0 msgr2=0x1aa0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:14.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea51ec6740 1 -- 172.21.15.146:0/3488486661 shutdown_connections 2024-09-13T18:27:14.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea51ec6740 1 -- 172.21.15.146:0/3488486661 wait complete. 2024-09-13T18:27:14.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.726+0000 7fea51ec6740 20 client.4944 trim_cache size 0 max 16384 2024-09-13T18:27:14.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.746+0000 7fea51ec6740 1 Processor -- start 2024-09-13T18:27:14.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.746+0000 7fea51ec6740 1 -- start start 2024-09-13T18:27:14.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8cd00 0x1a8e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8d620 0x1aa0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1aa0ba0 0x1a8c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x199b4f0 con 0x1a8d620 2024-09-13T18:27:14.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x19ffec0 con 0x1a8cd00 2024-09-13T18:27:14.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x19c9440 con 0x1aa0ba0 2024-09-13T18:27:14.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea47fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8d620 0x1aa0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea47fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8d620 0x1aa0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52488/0 (socket says 172.21.15.146:52488) 2024-09-13T18:27:14.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea47fff700 1 -- 172.21.15.146:0/362373693 learned_addr learned my addr 172.21.15.146:0/362373693 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:14.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea4d1b2700 1 --2- 172.21.15.146:0/362373693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1aa0ba0 0x1a8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/362373693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8cd00 0x1a8e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea47fff700 1 -- 172.21.15.146:0/362373693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1aa0ba0 msgr2=0x1a8c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:14.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea47fff700 1 --2- 172.21.15.146:0/362373693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1aa0ba0 0x1a8c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea47fff700 1 -- 172.21.15.146:0/362373693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8cd00 msgr2=0x1a8e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:14.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea47fff700 1 --2- 172.21.15.146:0/362373693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8cd00 0x1a8e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea47fff700 1 -- 172.21.15.146:0/362373693 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x197d450 con 0x1a8d620 2024-09-13T18:27:14.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/362373693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8cd00 0x1a8e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:14.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea47fff700 1 --2- 172.21.15.146:0/362373693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8d620 0x1aa0620 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0070a0 tx=0x7fea3c083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7310d3375ef432d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:14.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea2effd700 1 -- 172.21.15.146:0/362373693 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c058190 con 0x1a8d620 2024-09-13T18:27:14.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea2effd700 1 -- 172.21.15.146:0/362373693 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c05c7e0 con 0x1a8d620 2024-09-13T18:27:14.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea2effd700 1 -- 172.21.15.146:0/362373693 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c081a80 con 0x1a8d620 2024-09-13T18:27:14.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 -- 172.21.15.146:0/362373693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8d620 msgr2=0x1aa0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 --2- 172.21.15.146:0/362373693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8d620 0x1aa0620 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0070a0 tx=0x7fea3c083040 comp rx=0 tx=0).stop 2024-09-13T18:27:14.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 -- 172.21.15.146:0/362373693 shutdown_connections 2024-09-13T18:27:14.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 --2- 172.21.15.146:0/362373693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1aa0ba0 0x1a8c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 --2- 172.21.15.146:0/362373693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1a8d620 0x1aa0620 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 --2- 172.21.15.146:0/362373693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8cd00 0x1a8e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 -- 172.21.15.146:0/362373693 >> 172.21.15.146:0/362373693 conn(0x17f84d0 msgr2=0x1a8da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:14.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 -- 172.21.15.146:0/362373693 shutdown_connections 2024-09-13T18:27:14.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 -- 172.21.15.146:0/362373693 wait complete. 2024-09-13T18:27:14.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 -- start start 2024-09-13T18:27:14.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.750+0000 7fea51ec6740 1 Processor -- start 2024-09-13T18:27:14.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8cd00 0x1b11ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8d620 0x1a9f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 0x1b0ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ac4bc0 con 0x1aa0ba0 2024-09-13T18:27:14.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a8ca40 con 0x1a8d620 2024-09-13T18:27:14.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a8c800 con 0x1a8cd00 2024-09-13T18:27:14.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 10 client.0 _ll_register_callbacks cb 0x7fea38004b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:14.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea4d1b2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 0x1b0ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea4d1b2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 0x1b0ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52490/0 (socket says 172.21.15.146:52490) 2024-09-13T18:27:14.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea4d1b2700 1 -- 172.21.15.146:0/2740709798 learned_addr learned my addr 172.21.15.146:0/2740709798 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:14.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8cd00 0x1b11ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea47fff700 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8d620 0x1a9f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea4d1b2700 1 -- 172.21.15.146:0/2740709798 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8cd00 msgr2=0x1b11ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:14.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea4d1b2700 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8cd00 0x1b11ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea4d1b2700 1 -- 172.21.15.146:0/2740709798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8d620 msgr2=0x1a9f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:14.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea4d1b2700 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8d620 0x1a9f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea4d1b2700 1 -- 172.21.15.146:0/2740709798 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea481ed5f0 con 0x1aa0ba0 2024-09-13T18:27:14.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea4d1b2700 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 0x1b0ae10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fea4000b730 tx=0x7fea40004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8564b23d965352a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:14.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea277fe700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-13T18:27:14.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea277fe700 1 -- 172.21.15.146:0/2740709798 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40004450 con 0x1aa0ba0 2024-09-13T18:27:14.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea277fe700 1 -- 172.21.15.146:0/2740709798 <== mon.0 v2:172.21.15.146:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fea400045b0 con 0x1aa0ba0 2024-09-13T18:27:14.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 1 -- 172.21.15.146:0/2740709798 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea380133f0 con 0x1aa0ba0 2024-09-13T18:27:14.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea277fe700 1 -- 172.21.15.146:0/2740709798 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea40004770 con 0x1aa0ba0 2024-09-13T18:27:14.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 10 client.4950 fetch_fsmap learned FSMap version 15 2024-09-13T18:27:14.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 1 -- 172.21.15.146:0/2740709798 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x1971720 con 0x1aa0ba0 2024-09-13T18:27:14.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea277fe700 1 -- 172.21.15.146:0/2740709798 <== mon.0 v2:172.21.15.146:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea380133f0 con 0x1aa0ba0 2024-09-13T18:27:14.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea277fe700 1 -- 172.21.15.146:0/2740709798 <== mon.0 v2:172.21.15.146:3300/0 5 ==== fsmap(e 15) v1 ==== 4236+0+0 (secure 0 0 0) 0x7fea40019050 con 0x1aa0ba0 2024-09-13T18:27:14.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 10 client.4950 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T18:27:14.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 10 client.4950 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T18:27:14.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 20 client.4950 populate_metadata read hostname 'smithi146' 2024-09-13T18:27:14.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] conn(0x1b145c0 0x1b149a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:14.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 4 client.4950 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:27:14.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea51ec6740 1 -- 172.21.15.146:0/2740709798 --> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] -- command(tid 0: {"prefix": "status"}) v1 -- 0x18741e0 con 0x1b145c0 2024-09-13T18:27:14.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.754+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] conn(0x1b145c0 0x1b149a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.758+0000 7fea4c9b1700 1 -- 172.21.15.146:0/2740709798 >> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] conn(0x1b145c0 msgr2=0x1b149a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-13T18:27:14.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.758+0000 7fea4c9b1700 1 -- 172.21.15.146:0/2740709798 >> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] conn(0x1b145c0 msgr2=0x1b149a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:27:14.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.758+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] conn(0x1b145c0 0x1b149a0 crc :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:27:14.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.758+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] conn(0x1b145c0 0x1b149a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:27:14.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] conn(0x1b145c0 0x1b149a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:14.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea4c9b1700 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] conn(0x1b145c0 0x1b149a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b26ec2dac1bd2886 server_cookie=31fe1048f5c6c6ae in_seq=0 out_seq=0 2024-09-13T18:27:14.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea277fe700 10 client.4950 ms_handle_connect on v2:172.21.15.146:6834/1642500214 2024-09-13T18:27:14.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea277fe700 1 -- 172.21.15.146:0/2740709798 <== mds.1 v2:172.21.15.146:6834/1642500214 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x18741e0 con 0x1b145c0 2024-09-13T18:27:14.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea277fe700 10 client.4950 handle_command_reply: tid=0 2024-09-13T18:27:14.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea51ec6740 1 client.4950 shutdown 2024-09-13T18:27:14.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea51ec6740 1 -- 172.21.15.146:0/2740709798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 msgr2=0x1b0ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea51ec6740 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 0x1b0ae10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fea4000b730 tx=0x7fea40004b60 comp rx=0 tx=0).stop 2024-09-13T18:27:14.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea51ec6740 1 -- 172.21.15.146:0/2740709798 shutdown_connections 2024-09-13T18:27:14.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea51ec6740 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.146:6834/1642500214,v1:172.21.15.146:6835/1642500214] conn(0x1b145c0 0x1b149a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea51ec6740 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1aa0ba0 0x1b0ae10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea51ec6740 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1a8d620 0x1a9f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea51ec6740 1 --2- 172.21.15.146:0/2740709798 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x1a8cd00 0x1b11ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea51ec6740 1 -- 172.21.15.146:0/2740709798 >> 172.21.15.146:0/2740709798 conn(0x17f84d0 msgr2=0x1a6a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:14.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea277fe700 0 client.4950 ms_handle_reset on v2:172.21.15.146:6834/1642500214 2024-09-13T18:27:14.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea51ec6740 1 -- 172.21.15.146:0/2740709798 shutdown_connections 2024-09-13T18:27:14.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea51ec6740 1 -- 172.21.15.146:0/2740709798 wait complete. 2024-09-13T18:27:14.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.958+0000 7fea51ec6740 20 client.4950 trim_cache size 0 max 16384 2024-09-13T18:27:14.969 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:27:14.969 INFO:teuthology.orchestra.run.smithi146.stdout: "cluster_fsid": "509e851e-cdbc-4e57-a792-d3d1aeeeba86", 2024-09-13T18:27:14.969 INFO:teuthology.orchestra.run.smithi146.stdout: "whoami": 1, 2024-09-13T18:27:14.969 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4281, 2024-09-13T18:27:14.969 INFO:teuthology.orchestra.run.smithi146.stdout: "want_state": "up:active", 2024-09-13T18:27:14.969 INFO:teuthology.orchestra.run.smithi146.stdout: "state": "up:active", 2024-09-13T18:27:14.969 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs", 2024-09-13T18:27:14.970 INFO:teuthology.orchestra.run.smithi146.stdout: "rank_uptime": 16.504593968999998, 2024-09-13T18:27:14.970 INFO:teuthology.orchestra.run.smithi146.stdout: "mdsmap_epoch": 15, 2024-09-13T18:27:14.970 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_epoch": 25, 2024-09-13T18:27:14.970 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_epoch_barrier": 25, 2024-09-13T18:27:14.970 INFO:teuthology.orchestra.run.smithi146.stdout: "uptime": 48.740157209000003 2024-09-13T18:27:14.970 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:27:14.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.962+0000 7fea4ec15700 1 -- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea34066730 msgr2=0x7fea34068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.962+0000 7fea4ec15700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea34066730 0x7fea34068bf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fea38000f60 tx=0x7fea38007ef0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.962+0000 7fea4ec15700 1 -- 172.21.15.146:0/96812445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea480c7b20 msgr2=0x7fea481edb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:14.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.962+0000 7fea4ec15700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea480c7b20 0x7fea481edb10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c460 tx=0x7fea3c00c770 comp rx=0 tx=0).stop 2024-09-13T18:27:14.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.962+0000 7fea4ec15700 1 -- 172.21.15.146:0/96812445 shutdown_connections 2024-09-13T18:27:14.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.962+0000 7fea4ec15700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea34066730 0x7fea34068bf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.962+0000 7fea4ec15700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea48150900 0x7fea48142380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.962+0000 7fea4ec15700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea480c7b20 0x7fea481edb10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.962+0000 7fea4ec15700 1 --2- 172.21.15.146:0/96812445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea480c2670 0x7fea481483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:14.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.962+0000 7fea4ec15700 1 -- 172.21.15.146:0/96812445 >> 172.21.15.146:0/96812445 conn(0x7fea48066920 msgr2=0x7fea4806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:14.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.962+0000 7fea4ec15700 1 -- 172.21.15.146:0/96812445 shutdown_connections 2024-09-13T18:27:14.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:14.962+0000 7fea4ec15700 1 -- 172.21.15.146:0/96812445 wait complete. 2024-09-13T18:27:14.986 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T18:27:15.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa080845700 1 Processor -- start 2024-09-13T18:27:15.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa080845700 1 -- start start 2024-09-13T18:27:15.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa080845700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0740d89e0 0x7fa07404b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa080845700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa074009040 con 0x7fa074006af0 2024-09-13T18:27:15.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa080845700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0740df110 con 0x7fa07404bcd0 2024-09-13T18:27:15.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa080845700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa074046330 con 0x7fa0740d89e0 2024-09-13T18:27:15.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa07bfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa074006af0 0x7fa0740e9000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56706/0 (socket says 172.21.15.146:56706) 2024-09-13T18:27:15.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa07bfff700 1 -- 172.21.15.146:0/2828310756 learned_addr learned my addr 172.21.15.146:0/2828310756 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:15.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/2828310756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0740d89e0 0x7fa07404b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa07a7fc700 1 -- 172.21.15.146:0/2828310756 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210190544 0 0) 0x7fa074009040 con 0x7fa074006af0 2024-09-13T18:27:15.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa07a7fc700 1 -- 172.21.15.146:0/2828310756 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0640035d0 con 0x7fa074006af0 2024-09-13T18:27:15.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa07a7fc700 1 -- 172.21.15.146:0/2828310756 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443335569 0 0) 0x7fa0740df110 con 0x7fa07404bcd0 2024-09-13T18:27:15.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa07a7fc700 1 -- 172.21.15.146:0/2828310756 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa074009040 con 0x7fa07404bcd0 2024-09-13T18:27:15.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa07b7fe700 1 -- 172.21.15.146:0/2828310756 >> v1:172.21.15.154:6789/0 conn(0x7fa07404bcd0 legacy=0x7fa0740469e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa07b7fe700 1 -- 172.21.15.146:0/2828310756 >> v1:172.21.15.146:6789/0 conn(0x7fa074006af0 legacy=0x7fa0740e9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.102+0000 7fa07b7fe700 1 -- 172.21.15.146:0/2828310756 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0740de820 con 0x7fa0740d89e0 2024-09-13T18:27:15.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.106+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/2828310756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0740d89e0 0x7fa07404b790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa070009880 tx=0x7fa070009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b0d8c9260db4fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:15.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.106+0000 7fa07a7fc700 1 -- 172.21.15.146:0/2828310756 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070011830 con 0x7fa0740d89e0 2024-09-13T18:27:15.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.106+0000 7fa07a7fc700 1 -- 172.21.15.146:0/2828310756 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa070011990 con 0x7fa0740d89e0 2024-09-13T18:27:15.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.106+0000 7fa07a7fc700 1 -- 172.21.15.146:0/2828310756 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070011d20 con 0x7fa0740d89e0 2024-09-13T18:27:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.106+0000 7fa080845700 1 -- 172.21.15.146:0/2828310756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0740d89e0 msgr2=0x7fa07404b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.106+0000 7fa080845700 1 --2- 172.21.15.146:0/2828310756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0740d89e0 0x7fa07404b790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa070009880 tx=0x7fa070009b90 comp rx=0 tx=0).stop 2024-09-13T18:27:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.106+0000 7fa080845700 1 -- 172.21.15.146:0/2828310756 shutdown_connections 2024-09-13T18:27:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.106+0000 7fa080845700 1 --2- 172.21.15.146:0/2828310756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0740d89e0 0x7fa07404b790 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.106+0000 7fa080845700 1 -- 172.21.15.146:0/2828310756 >> 172.21.15.146:0/2828310756 conn(0x7fa074050e10 msgr2=0x7fa074051220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.106+0000 7fa080845700 1 -- 172.21.15.146:0/2828310756 shutdown_connections 2024-09-13T18:27:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.106+0000 7fa080845700 1 -- 172.21.15.146:0/2828310756 wait complete. 2024-09-13T18:27:15.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.106+0000 7fa080845700 1 Processor -- start 2024-09-13T18:27:15.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.106+0000 7fa080845700 1 -- start start 2024-09-13T18:27:15.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa080845700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa074006af0 0x7fa0740ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa080845700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa07404bcd0 0x7fa0740c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa080845700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0740d89e0 0x7fa0740cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa080845700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa074167fc0 con 0x7fa0740d89e0 2024-09-13T18:27:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa080845700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740cc710 con 0x7fa074006af0 2024-09-13T18:27:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa080845700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740ccfb0 con 0x7fa07404bcd0 2024-09-13T18:27:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa07bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0740d89e0 0x7fa0740cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa07bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0740d89e0 0x7fa0740cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52500/0 (socket says 172.21.15.146:52500) 2024-09-13T18:27:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa07bfff700 1 -- 172.21.15.146:0/1596438203 learned_addr learned my addr 172.21.15.146:0/1596438203 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa074006af0 0x7fa0740ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa07affd700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa07404bcd0 0x7fa0740c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa07b7fe700 1 -- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa07404bcd0 msgr2=0x7fa0740c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa07404bcd0 0x7fa0740c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa07b7fe700 1 -- 172.21.15.146:0/1596438203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0740d89e0 msgr2=0x7fa0740cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0740d89e0 0x7fa0740cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa07b7fe700 1 -- 172.21.15.146:0/1596438203 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0700094b0 con 0x7fa074006af0 2024-09-13T18:27:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa07affd700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa07404bcd0 0x7fa0740c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa07bfff700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0740d89e0 0x7fa0740cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa074006af0 0x7fa0740ca750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa070015000 tx=0x7fa070004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa063fff700 1 -- 172.21.15.146:0/1596438203 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07001a8c0 con 0x7fa074006af0 2024-09-13T18:27:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa063fff700 1 -- 172.21.15.146:0/1596438203 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07001e070 con 0x7fa074006af0 2024-09-13T18:27:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa080845700 1 -- 172.21.15.146:0/1596438203 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0740cbe50 con 0x7fa074006af0 2024-09-13T18:27:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa063fff700 1 -- 172.21.15.146:0/1596438203 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07001ac80 con 0x7fa074006af0 2024-09-13T18:27:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.110+0000 7fa080845700 1 -- 172.21.15.146:0/1596438203 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0740cc960 con 0x7fa074006af0 2024-09-13T18:27:15.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.114+0000 7fa063fff700 1 -- 172.21.15.146:0/1596438203 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa07002d450 con 0x7fa074006af0 2024-09-13T18:27:15.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.114+0000 7fa063fff700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa068066730 0x7fa068068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.114+0000 7fa07affd700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa068066730 0x7fa068068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.114+0000 7fa063fff700 1 -- 172.21.15.146:0/1596438203 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa070085150 con 0x7fa074006af0 2024-09-13T18:27:15.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.118+0000 7fa07affd700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa068066730 0x7fa068068bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa0740e2b70 tx=0x7fa06c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:15.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.118+0000 7fa084d58740 1 Processor -- start 2024-09-13T18:27:15.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.118+0000 7fa084d58740 1 -- start start 2024-09-13T18:27:15.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xcf5db0 0xce19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xce1f10 0xce22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xce2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0xc2cd80 con 0xcf5db0 2024-09-13T18:27:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0xc2a510 con 0xce2830 2024-09-13T18:27:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0xc25e00 con 0xce1f10 2024-09-13T18:27:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa07b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xcf5db0 0xce19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa07b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xcf5db0 0xce19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52502/0 (socket says 172.21.15.146:52502) 2024-09-13T18:27:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa07b7fe700 1 -- 172.21.15.146:0/2078086904 learned_addr learned my addr 172.21.15.146:0/2078086904 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa07affd700 1 --2- 172.21.15.146:0/2078086904 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xce1f10 0xce22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa07bfff700 1 --2- 172.21.15.146:0/2078086904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xce2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa07b7fe700 1 -- 172.21.15.146:0/2078086904 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xce1f10 msgr2=0xce22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/2078086904 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xce1f10 0xce22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa07b7fe700 1 -- 172.21.15.146:0/2078086904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 msgr2=0xce2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:15.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/2078086904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xce2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa07b7fe700 1 -- 172.21.15.146:0/2078086904 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xa98960 con 0xcf5db0 2024-09-13T18:27:15.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa07bfff700 1 --2- 172.21.15.146:0/2078086904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xce2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:15.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/2078086904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xcf5db0 0xce19d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa070055ec0 tx=0x7fa070055e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e600637eaf2e22f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:15.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa0617fa700 1 -- 172.21.15.146:0/2078086904 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070059cb0 con 0xcf5db0 2024-09-13T18:27:15.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa0617fa700 1 -- 172.21.15.146:0/2078086904 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa070056e90 con 0xcf5db0 2024-09-13T18:27:15.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa0617fa700 1 -- 172.21.15.146:0/2078086904 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07005e270 con 0xcf5db0 2024-09-13T18:27:15.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 -- 172.21.15.146:0/2078086904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xcf5db0 msgr2=0xce19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 --2- 172.21.15.146:0/2078086904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xcf5db0 0xce19d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa070055ec0 tx=0x7fa070055e30 comp rx=0 tx=0).stop 2024-09-13T18:27:15.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 -- 172.21.15.146:0/2078086904 shutdown_connections 2024-09-13T18:27:15.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 --2- 172.21.15.146:0/2078086904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xce2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 --2- 172.21.15.146:0/2078086904 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xce1f10 0xce22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 --2- 172.21.15.146:0/2078086904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xcf5db0 0xce19d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 -- 172.21.15.146:0/2078086904 >> 172.21.15.146:0/2078086904 conn(0xa4d4d0 msgr2=0xcf8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:15.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 -- 172.21.15.146:0/2078086904 shutdown_connections 2024-09-13T18:27:15.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 -- 172.21.15.146:0/2078086904 wait complete. 2024-09-13T18:27:15.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 -- start start 2024-09-13T18:27:15.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.122+0000 7fa084d58740 1 Processor -- start 2024-09-13T18:27:15.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 0xcf8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xcf90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xcf5db0 0xcbde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0xbe3730 con 0xce1f10 2024-09-13T18:27:15.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0xbe95b0 con 0xce2830 2024-09-13T18:27:15.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0xad1aa0 con 0xcf5db0 2024-09-13T18:27:15.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 10 client.0 _ll_register_callbacks cb 0xc1f610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:15.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 0xcf8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 0xcf8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52510/0 (socket says 172.21.15.146:52510) 2024-09-13T18:27:15.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07b7fe700 1 -- 172.21.15.146:0/2962202692 learned_addr learned my addr 172.21.15.146:0/2962202692 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:15.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07bfff700 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xcf5db0 0xcbde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07affd700 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xcf90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07b7fe700 1 -- 172.21.15.146:0/2962202692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xcf5db0 msgr2=0xcbde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:15.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xcf5db0 0xcbde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07b7fe700 1 -- 172.21.15.146:0/2962202692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 msgr2=0xcf90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:15.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xcf90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07b7fe700 1 -- 172.21.15.146:0/2962202692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0740cc960 con 0xce1f10 2024-09-13T18:27:15.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07affd700 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xcf90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:15.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07bfff700 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xcf5db0 0xcbde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:15.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 0xcf8b80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa070055ec0 tx=0x7fa070009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a66060836962fd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:15.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa05a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-13T18:27:15.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa05a7fc700 1 -- 172.21.15.146:0/2962202692 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070059cb0 con 0xce1f10 2024-09-13T18:27:15.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa05a7fc700 1 -- 172.21.15.146:0/2962202692 <== mon.0 v2:172.21.15.146:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa070056190 con 0xce1f10 2024-09-13T18:27:15.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 1 -- 172.21.15.146:0/2962202692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xa5a1e0 con 0xce1f10 2024-09-13T18:27:15.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa05a7fc700 1 -- 172.21.15.146:0/2962202692 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa070056350 con 0xce1f10 2024-09-13T18:27:15.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 10 client.4962 fetch_fsmap learned FSMap version 15 2024-09-13T18:27:15.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 1 -- 172.21.15.146:0/2962202692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0xbec220 con 0xce1f10 2024-09-13T18:27:15.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa05a7fc700 1 -- 172.21.15.146:0/2962202692 <== mon.0 v2:172.21.15.146:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xa5a1e0 con 0xce1f10 2024-09-13T18:27:15.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa05a7fc700 1 -- 172.21.15.146:0/2962202692 <== mon.0 v2:172.21.15.146:3300/0 5 ==== fsmap(e 15) v1 ==== 4236+0+0 (secure 0 0 0) 0x7fa070078080 con 0xce1f10 2024-09-13T18:27:15.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 10 client.4962 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T18:27:15.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 10 client.4962 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T18:27:15.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 20 client.4962 populate_metadata read hostname 'smithi146' 2024-09-13T18:27:15.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0xcbca50 0xcbce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 4 client.4962 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:15.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa084d58740 1 -- 172.21.15.146:0/2962202692 --> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xaa4460 con 0xcbca50 2024-09-13T18:27:15.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07affd700 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0xcbca50 0xcbce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa05a7fc700 10 client.4962 ms_handle_connect on v2:172.21.15.146:6836/31041557 2024-09-13T18:27:15.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.126+0000 7fa07affd700 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0xcbca50 0xcbce30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=129e27d6d01adde6 server_cookie=ee40199aaba53cae in_seq=0 out_seq=0 2024-09-13T18:27:15.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa05a7fc700 1 -- 172.21.15.146:0/2962202692 <== mds.0 v2:172.21.15.146:6836/31041557 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0xaa4460 con 0xcbca50 2024-09-13T18:27:15.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa05a7fc700 10 client.4962 handle_command_reply: tid=0 2024-09-13T18:27:15.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa084d58740 1 client.4962 shutdown 2024-09-13T18:27:15.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa084d58740 1 -- 172.21.15.146:0/2962202692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 msgr2=0xcf8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa084d58740 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 0xcf8b80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa070055ec0 tx=0x7fa070009b90 comp rx=0 tx=0).stop 2024-09-13T18:27:15.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa084d58740 1 -- 172.21.15.146:0/2962202692 shutdown_connections 2024-09-13T18:27:15.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa084d58740 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0xcbca50 0xcbce30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa084d58740 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xcf5db0 0xcbde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa05a7fc700 0 client.4962 ms_handle_reset on v2:172.21.15.146:6836/31041557 2024-09-13T18:27:15.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa084d58740 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xcf90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa084d58740 1 --2- 172.21.15.146:0/2962202692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 0xcf8b80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa084d58740 1 -- 172.21.15.146:0/2962202692 >> 172.21.15.146:0/2962202692 conn(0xa4d4d0 msgr2=0xcf6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:15.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa084d58740 1 -- 172.21.15.146:0/2962202692 shutdown_connections 2024-09-13T18:27:15.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa084d58740 1 -- 172.21.15.146:0/2962202692 wait complete. 2024-09-13T18:27:15.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.130+0000 7fa084d58740 20 client.4962 trim_cache size 0 max 16384 2024-09-13T18:27:15.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa084d58740 1 Processor -- start 2024-09-13T18:27:15.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa084d58740 1 -- start start 2024-09-13T18:27:15.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa084d58740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xcbca50 0xd0a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa084d58740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 0xcbd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa084d58740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xcbd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa084d58740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0xd0b1b0 con 0xce1f10 2024-09-13T18:27:15.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa084d58740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0xbe3730 con 0xce2830 2024-09-13T18:27:15.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa084d58740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0xcdfbd0 con 0xcbca50 2024-09-13T18:27:15.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa07affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 0xcbd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa07affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 0xcbd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52518/0 (socket says 172.21.15.146:52518) 2024-09-13T18:27:15.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa07affd700 1 -- 172.21.15.146:0/2065928323 learned_addr learned my addr 172.21.15.146:0/2065928323 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:15.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/2065928323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xcbca50 0xd0a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa07bfff700 1 --2- 172.21.15.146:0/2065928323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xcbd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa07affd700 1 -- 172.21.15.146:0/2065928323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xcbca50 msgr2=0xd0a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:15.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa07affd700 1 --2- 172.21.15.146:0/2065928323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xcbca50 0xd0a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa07affd700 1 -- 172.21.15.146:0/2065928323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 msgr2=0xcbd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:15.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa07affd700 1 --2- 172.21.15.146:0/2065928323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xcbd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa07affd700 1 -- 172.21.15.146:0/2065928323 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xbd2450 con 0xce1f10 2024-09-13T18:27:15.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.150+0000 7fa07affd700 1 --2- 172.21.15.146:0/2065928323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 0xcbd250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa06c008fd0 tx=0x7fa06c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9d3b478eccd21f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:15.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa061ffb700 1 -- 172.21.15.146:0/2065928323 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c027070 con 0xce1f10 2024-09-13T18:27:15.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa061ffb700 1 -- 172.21.15.146:0/2065928323 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c0144d0 con 0xce1f10 2024-09-13T18:27:15.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa061ffb700 1 -- 172.21.15.146:0/2065928323 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c014840 con 0xce1f10 2024-09-13T18:27:15.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 -- 172.21.15.146:0/2065928323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 msgr2=0xcbd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 --2- 172.21.15.146:0/2065928323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 0xcbd250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa06c008fd0 tx=0x7fa06c012000 comp rx=0 tx=0).stop 2024-09-13T18:27:15.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 -- 172.21.15.146:0/2065928323 shutdown_connections 2024-09-13T18:27:15.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 --2- 172.21.15.146:0/2065928323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xce2830 0xcbd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 --2- 172.21.15.146:0/2065928323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce1f10 0xcbd250 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 --2- 172.21.15.146:0/2065928323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xcbca50 0xd0a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 -- 172.21.15.146:0/2065928323 >> 172.21.15.146:0/2065928323 conn(0xa4d4d0 msgr2=0xd0a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:15.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 -- 172.21.15.146:0/2065928323 shutdown_connections 2024-09-13T18:27:15.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 -- 172.21.15.146:0/2065928323 wait complete. 2024-09-13T18:27:15.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 -- start start 2024-09-13T18:27:15.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 Processor -- start 2024-09-13T18:27:15.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xcbca50 0xd0a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xce1f10 0xd654f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce2830 0xd5ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0xbc4470 con 0xce2830 2024-09-13T18:27:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0xbf20a0 con 0xcbca50 2024-09-13T18:27:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0xcbdbf0 con 0xce1f10 2024-09-13T18:27:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 10 client.0 _ll_register_callbacks cb 0x7fa070052270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa07bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce2830 0xd5ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa07bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce2830 0xd5ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52526/0 (socket says 172.21.15.146:52526) 2024-09-13T18:27:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa07bfff700 1 -- 172.21.15.146:0/1858526215 learned_addr learned my addr 172.21.15.146:0/1858526215 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa07affd700 1 --2- 172.21.15.146:0/1858526215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xce1f10 0xd654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/1858526215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xcbca50 0xd0a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa07bfff700 1 -- 172.21.15.146:0/1858526215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xce1f10 msgr2=0xd654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa07bfff700 1 --2- 172.21.15.146:0/1858526215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xce1f10 0xd654f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa07bfff700 1 -- 172.21.15.146:0/1858526215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xcbca50 msgr2=0xd0a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa07bfff700 1 --2- 172.21.15.146:0/1858526215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xcbca50 0xd0a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa07bfff700 1 -- 172.21.15.146:0/1858526215 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa074009040 con 0xce2830 2024-09-13T18:27:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa07bfff700 1 --2- 172.21.15.146:0/1858526215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce2830 0xd5ba90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa07c064010 tx=0x7fa07c04ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce361a72cb2fe241 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa05a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-13T18:27:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa05a7fc700 1 -- 172.21.15.146:0/1858526215 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c090070 con 0xce2830 2024-09-13T18:27:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.154+0000 7fa084d58740 1 -- 172.21.15.146:0/1858526215 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa070056e90 con 0xce2830 2024-09-13T18:27:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa05a7fc700 1 -- 172.21.15.146:0/1858526215 <== mon.0 v2:172.21.15.146:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa07c04efc0 con 0xce2830 2024-09-13T18:27:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa05a7fc700 1 -- 172.21.15.146:0/1858526215 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa07c067750 con 0xce2830 2024-09-13T18:27:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 10 client.4968 fetch_fsmap learned FSMap version 15 2024-09-13T18:27:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 -- 172.21.15.146:0/1858526215 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0xbc6720 con 0xce2830 2024-09-13T18:27:15.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa05a7fc700 1 -- 172.21.15.146:0/1858526215 <== mon.0 v2:172.21.15.146:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa070056e90 con 0xce2830 2024-09-13T18:27:15.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa05a7fc700 1 -- 172.21.15.146:0/1858526215 <== mon.0 v2:172.21.15.146:3300/0 5 ==== fsmap(e 15) v1 ==== 4236+0+0 (secure 0 0 0) 0x7fa07c0678b0 con 0xce2830 2024-09-13T18:27:15.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 10 client.4968 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T18:27:15.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 10 client.4968 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T18:27:15.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 20 client.4968 populate_metadata read hostname 'smithi146' 2024-09-13T18:27:15.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 --2- 172.21.15.146:0/1858526215 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0xcf5db0 0xd5bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 4 client.4968 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:27:15.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 -- 172.21.15.146:0/1858526215 --> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] -- command(tid 0: {"prefix": "status"}) v1 -- 0xac91e0 con 0xcf5db0 2024-09-13T18:27:15.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/1858526215 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0xcf5db0 0xd5bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa05a7fc700 10 client.4968 ms_handle_connect on v2:172.21.15.146:6836/31041557 2024-09-13T18:27:15.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa07b7fe700 1 --2- 172.21.15.146:0/1858526215 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0xcf5db0 0xd5bfd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edbae7a3dc93a765 server_cookie=8af32d339a6ec575 in_seq=0 out_seq=0 2024-09-13T18:27:15.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa05a7fc700 1 -- 172.21.15.146:0/1858526215 <== mds.0 v2:172.21.15.146:6836/31041557 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0xac91e0 con 0xcf5db0 2024-09-13T18:27:15.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa05a7fc700 10 client.4968 handle_command_reply: tid=0 2024-09-13T18:27:15.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 client.4968 shutdown 2024-09-13T18:27:15.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 -- 172.21.15.146:0/1858526215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce2830 msgr2=0xd5ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 --2- 172.21.15.146:0/1858526215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce2830 0xd5ba90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa07c064010 tx=0x7fa07c04ef50 comp rx=0 tx=0).stop 2024-09-13T18:27:15.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 -- 172.21.15.146:0/1858526215 shutdown_connections 2024-09-13T18:27:15.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 --2- 172.21.15.146:0/1858526215 >> [v2:172.21.15.146:6836/31041557,v1:172.21.15.146:6837/31041557] conn(0xcf5db0 0xd5bfd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 --2- 172.21.15.146:0/1858526215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0xce2830 0xd5ba90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa05a7fc700 0 client.4968 ms_handle_reset on v2:172.21.15.146:6836/31041557 2024-09-13T18:27:15.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 --2- 172.21.15.146:0/1858526215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xce1f10 0xd654f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 --2- 172.21.15.146:0/1858526215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xcbca50 0xd0a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 -- 172.21.15.146:0/1858526215 >> 172.21.15.146:0/1858526215 conn(0xa4d4d0 msgr2=0xcbce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:15.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 -- 172.21.15.146:0/1858526215 shutdown_connections 2024-09-13T18:27:15.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 1 -- 172.21.15.146:0/1858526215 wait complete. 2024-09-13T18:27:15.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa084d58740 20 client.4968 trim_cache size 0 max 16384 2024-09-13T18:27:15.173 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:27:15.173 INFO:teuthology.orchestra.run.smithi146.stdout: "cluster_fsid": "509e851e-cdbc-4e57-a792-d3d1aeeeba86", 2024-09-13T18:27:15.173 INFO:teuthology.orchestra.run.smithi146.stdout: "whoami": 0, 2024-09-13T18:27:15.173 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4290, 2024-09-13T18:27:15.173 INFO:teuthology.orchestra.run.smithi146.stdout: "want_state": "up:active", 2024-09-13T18:27:15.173 INFO:teuthology.orchestra.run.smithi146.stdout: "state": "up:active", 2024-09-13T18:27:15.174 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs", 2024-09-13T18:27:15.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rank_uptime": 42.836510042, 2024-09-13T18:27:15.174 INFO:teuthology.orchestra.run.smithi146.stdout: "mdsmap_epoch": 15, 2024-09-13T18:27:15.174 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_epoch": 25, 2024-09-13T18:27:15.174 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_epoch_barrier": 25, 2024-09-13T18:27:15.174 INFO:teuthology.orchestra.run.smithi146.stdout: "uptime": 48.931429432999998 2024-09-13T18:27:15.174 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:27:15.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa080845700 1 -- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa068066730 msgr2=0x7fa068068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa080845700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa068066730 0x7fa068068bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa0740e2b70 tx=0x7fa06c009040 comp rx=0 tx=0).stop 2024-09-13T18:27:15.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa080845700 1 -- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa074006af0 msgr2=0x7fa0740ca750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa080845700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa074006af0 0x7fa0740ca750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa070015000 tx=0x7fa070004770 comp rx=0 tx=0).stop 2024-09-13T18:27:15.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa080845700 1 -- 172.21.15.146:0/1596438203 shutdown_connections 2024-09-13T18:27:15.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa080845700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa068066730 0x7fa068068bf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa080845700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0740d89e0 0x7fa0740cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa080845700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa07404bcd0 0x7fa0740c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa080845700 1 --2- 172.21.15.146:0/1596438203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa074006af0 0x7fa0740ca750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.158+0000 7fa080845700 1 -- 172.21.15.146:0/1596438203 >> 172.21.15.146:0/1596438203 conn(0x7fa074050e10 msgr2=0x7fa0740d7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:15.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.162+0000 7fa080845700 1 -- 172.21.15.146:0/1596438203 shutdown_connections 2024-09-13T18:27:15.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.162+0000 7fa080845700 1 -- 172.21.15.146:0/1596438203 wait complete. 2024-09-13T18:27:15.184 INFO:teuthology.run_tasks:Running task sequential... 2024-09-13T18:27:15.195 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-13T18:27:15.195 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-13T18:27:15.195 INFO:teuthology.task.print:Enabling mgr modules 2024-09-13T18:27:15.195 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-13T18:27:15.225 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-13T18:27:15.225 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-13T18:27:15.225 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-13T18:27:15.226 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T18:27:15.226 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T18:27:15.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:15.226 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-13T18:27:15.234 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-13T18:27:15.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:15.235 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T18:27:15.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:15.249 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-09-13T18:27:15.390 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-13T18:27:15.391 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T18:27:15.391 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T18:27:15.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.506+0000 7f962f59e700 1 Processor -- start 2024-09-13T18:27:15.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.506+0000 7f962f59e700 1 -- start start 2024-09-13T18:27:15.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962f59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96300c8f70 0x7f96300a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300c7530 0x7f96300cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962f59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9630159f80 con 0x7f96300c9350 2024-09-13T18:27:15.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962f59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96300ca1d0 con 0x7f96300c8f70 2024-09-13T18:27:15.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9630159670 con 0x7f96300c7530 2024-09-13T18:27:15.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962e59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96300c8f70 0x7f96300a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962e59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96300c8f70 0x7f96300a9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52538/0 (socket says 172.21.15.146:52538) 2024-09-13T18:27:15.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962e59c700 1 -- 172.21.15.146:0/2826038557 learned_addr learned my addr 172.21.15.146:0/2826038557 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:15.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962ed9d700 1 --2- 172.21.15.146:0/2826038557 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300c7530 0x7f96300cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962e59c700 1 -- 172.21.15.146:0/2826038557 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300c7530 msgr2=0x7f96300cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:15.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962e59c700 1 --2- 172.21.15.146:0/2826038557 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300c7530 0x7f96300cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962e59c700 1 -- 172.21.15.146:0/2826038557 >> v1:172.21.15.154:6789/0 conn(0x7f96300c9350 legacy=0x7f96300723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962e59c700 1 -- 172.21.15.146:0/2826038557 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96300aa3a0 con 0x7f96300c8f70 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962e59c700 1 --2- 172.21.15.146:0/2826038557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300c8f70 0x7f96300a9cf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9620009910 tx=0x7f9620009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90412b410c24f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962d59a700 1 -- 172.21.15.146:0/2826038557 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962000ba60 con 0x7f96300c8f70 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962d59a700 1 -- 172.21.15.146:0/2826038557 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962000bbc0 con 0x7f96300c8f70 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962d59a700 1 -- 172.21.15.146:0/2826038557 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620009e60 con 0x7f96300c8f70 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962f59e700 1 -- 172.21.15.146:0/2826038557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300c8f70 msgr2=0x7f96300a9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962f59e700 1 --2- 172.21.15.146:0/2826038557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300c8f70 0x7f96300a9cf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9620009910 tx=0x7f9620009c20 comp rx=0 tx=0).stop 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962f59e700 1 -- 172.21.15.146:0/2826038557 shutdown_connections 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962f59e700 1 --2- 172.21.15.146:0/2826038557 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300c7530 0x7f96300cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962f59e700 1 --2- 172.21.15.146:0/2826038557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300c8f70 0x7f96300a9cf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962f59e700 1 -- 172.21.15.146:0/2826038557 >> 172.21.15.146:0/2826038557 conn(0x7f9630066090 msgr2=0x7f9630066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962f59e700 1 -- 172.21.15.146:0/2826038557 shutdown_connections 2024-09-13T18:27:15.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.510+0000 7f962f59e700 1 -- 172.21.15.146:0/2826038557 wait complete. 2024-09-13T18:27:15.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.514+0000 7f962f59e700 1 Processor -- start 2024-09-13T18:27:15.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.514+0000 7f962f59e700 1 -- start start 2024-09-13T18:27:15.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.514+0000 7f962f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300c7530 0x7f96301ef430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.514+0000 7f962f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300c8f70 0x7f96301efe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.514+0000 7f962f59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96300c9350 0x7f96300ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.514+0000 7f962f59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96301efa10 con 0x7f96300c8f70 2024-09-13T18:27:15.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.514+0000 7f962f59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96300aefb0 con 0x7f96300c9350 2024-09-13T18:27:15.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.514+0000 7f962f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96300adbf0 con 0x7f96300c7530 2024-09-13T18:27:15.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.514+0000 7f962dd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300c8f70 0x7f96301efe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.514+0000 7f962e59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300c7530 0x7f96301ef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.514+0000 7f962dd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300c8f70 0x7f96301efe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52550/0 (socket says 172.21.15.146:52550) 2024-09-13T18:27:15.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.514+0000 7f962dd9b700 1 -- 172.21.15.146:0/2856814485 learned_addr learned my addr 172.21.15.146:0/2856814485 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:15.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f962ed9d700 1 --2- 172.21.15.146:0/2856814485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96300c9350 0x7f96300ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f962dd9b700 1 -- 172.21.15.146:0/2856814485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300c7530 msgr2=0x7f96301ef430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f962dd9b700 1 --2- 172.21.15.146:0/2856814485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300c7530 0x7f96301ef430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f962dd9b700 1 -- 172.21.15.146:0/2856814485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96300c9350 msgr2=0x7f96300ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f962dd9b700 1 --2- 172.21.15.146:0/2856814485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96300c9350 0x7f96300ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f962dd9b700 1 -- 172.21.15.146:0/2856814485 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9620009570 con 0x7f96300c8f70 2024-09-13T18:27:15.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f962dd9b700 1 --2- 172.21.15.146:0/2856814485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300c8f70 0x7f96301efe90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f961800c500 tx=0x7f961800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:15.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f9626ffd700 1 -- 172.21.15.146:0/2856814485 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96180052d0 con 0x7f96300c8f70 2024-09-13T18:27:15.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f9626ffd700 1 -- 172.21.15.146:0/2856814485 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9618005430 con 0x7f96300c8f70 2024-09-13T18:27:15.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f962f59e700 1 -- 172.21.15.146:0/2856814485 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96300aded0 con 0x7f96300c8f70 2024-09-13T18:27:15.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f9626ffd700 1 -- 172.21.15.146:0/2856814485 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96180055a0 con 0x7f96300c8f70 2024-09-13T18:27:15.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f962f59e700 1 -- 172.21.15.146:0/2856814485 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96300af290 con 0x7f96300c8f70 2024-09-13T18:27:15.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f962f59e700 1 -- 172.21.15.146:0/2856814485 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963015aaf0 con 0x7f96300c8f70 2024-09-13T18:27:15.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.518+0000 7f9626ffd700 1 -- 172.21.15.146:0/2856814485 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f96180038e0 con 0x7f96300c8f70 2024-09-13T18:27:15.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.522+0000 7f9626ffd700 1 --2- 172.21.15.146:0/2856814485 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9610066760 0x7f9610068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:15.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.522+0000 7f9626ffd700 1 -- 172.21.15.146:0/2856814485 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f96180837f0 con 0x7f96300c8f70 2024-09-13T18:27:15.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.522+0000 7f962e59c700 1 --2- 172.21.15.146:0/2856814485 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9610066760 0x7f9610068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:15.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.522+0000 7f962e59c700 1 --2- 172.21.15.146:0/2856814485 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9610066760 0x7f9610068c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9620000c00 tx=0x7f9620008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:15.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.530+0000 7f9626ffd700 1 -- 172.21.15.146:0/2856814485 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f963015aaf0 con 0x7f96300c8f70 2024-09-13T18:27:15.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.738+0000 7f962f59e700 1 -- 172.21.15.146:0/2856814485 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96300ae3f0 con 0x7f96300c8f70 2024-09-13T18:27:15.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.742+0000 7f9626ffd700 1 -- 172.21.15.146:0/2856814485 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+88 (secure 0 0 0) 0x7f96300ae3f0 con 0x7f96300c8f70 2024-09-13T18:27:15.749 INFO:teuthology.orchestra.run.smithi146.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T18:27:15.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.746+0000 7f962f59e700 1 -- 172.21.15.146:0/2856814485 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9610066760 msgr2=0x7f9610068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.746+0000 7f962f59e700 1 --2- 172.21.15.146:0/2856814485 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9610066760 0x7f9610068c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9620000c00 tx=0x7f9620008030 comp rx=0 tx=0).stop 2024-09-13T18:27:15.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.746+0000 7f962f59e700 1 -- 172.21.15.146:0/2856814485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300c8f70 msgr2=0x7f96301efe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:15.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.746+0000 7f962f59e700 1 --2- 172.21.15.146:0/2856814485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300c8f70 0x7f96301efe90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f961800c500 tx=0x7f961800c810 comp rx=0 tx=0).stop 2024-09-13T18:27:15.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.746+0000 7f962f59e700 1 -- 172.21.15.146:0/2856814485 shutdown_connections 2024-09-13T18:27:15.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.746+0000 7f962f59e700 1 --2- 172.21.15.146:0/2856814485 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9610066760 0x7f9610068c20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.746+0000 7f962f59e700 1 --2- 172.21.15.146:0/2856814485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96300c9350 0x7f96300ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.746+0000 7f962f59e700 1 --2- 172.21.15.146:0/2856814485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300c8f70 0x7f96301efe90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.746+0000 7f962f59e700 1 --2- 172.21.15.146:0/2856814485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300c7530 0x7f96301ef430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:15.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.746+0000 7f962f59e700 1 -- 172.21.15.146:0/2856814485 >> 172.21.15.146:0/2856814485 conn(0x7f9630066090 msgr2=0x7f963006c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:15.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.746+0000 7f962f59e700 1 -- 172.21.15.146:0/2856814485 shutdown_connections 2024-09-13T18:27:15.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:15.746+0000 7f962f59e700 1 -- 172.21.15.146:0/2856814485 wait complete. 2024-09-13T18:27:15.771 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T18:27:15.771 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T18:27:15.771 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-13T18:27:15.771 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T18:27:15.771 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T18:27:15.771 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T18:27:15.771 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T18:27:15.771 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T18:27:15.771 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T18:27:15.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:15.772 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-13T18:27:15.777 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:27:15.777 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:27:15.777 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:27:15.777 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:27:15.777 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-13T18:27:15.777 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42520sec preferred_lft 42520sec 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T18:27:15.778 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 02:42:8a:41:83:6b brd ff:ff:ff:ff:ff:ff 2024-09-13T18:27:15.779 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T18:27:15.779 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:27:15.779 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T18:27:15.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:15.779 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-09-13T18:27:15.779 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-13T18:27:15.779 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add name ceph-brx type bridge 2024-09-13T18:27:15.779 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr flush dev ceph-brx 2024-09-13T18:27:15.779 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set ceph-brx up 2024-09-13T18:27:15.779 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T18:27:15.780 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-13T18:27:15.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:15.818 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T18:27:15.828 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-13T18:27:15.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:15.829 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-13T18:27:15.881 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-13T18:27:15.881 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T18:27:15.881 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T18:27:15.881 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T18:27:15.881 INFO:teuthology.orchestra.run.smithi146.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T18:27:15.881 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-09-13T18:27:15.881 INFO:teuthology.orchestra.run.smithi146.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T18:27:15.881 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T18:27:15.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:15.882 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-09-13T18:27:15.882 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-13T18:27:15.882 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T18:27:15.882 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T18:27:15.882 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T18:27:15.882 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-13T18:27:15.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:15.922 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-13T18:27:15.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:15.927 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list-id 2024-09-13T18:27:15.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:15.975 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-09-13T18:27:15.975 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-13T18:27:15.975 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T18:27:15.975 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-13T18:27:15.975 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-13T18:27:16.008 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-13T18:27:16.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:16.008 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-09-13T18:27:16.008 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-13T18:27:16.009 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-13T18:27:16.009 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T18:27:16.009 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-13T18:27:16.009 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-13T18:27:16.009 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-13T18:27:16.009 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-13T18:27:16.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:16.067 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-09-13T18:27:16.067 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-13T18:27:16.067 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-09-13T18:27:16.067 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T18:27:16.067 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-13T18:27:16.092 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T18:27:16.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:27:16.092 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T18:27:16.097 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T18:27:16.097 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:27:16.097 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T18:27:16.103 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-13T18:27:16.116 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:27:16.116 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:27:16.179 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:27:16.180 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:27:16.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:16.181 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-13T18:27:16.185 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T18:27:16.185 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T18:27:16.188 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-13T18:27:16.205 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:27:16.205 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:27:16.216 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:27:16.217 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:27:16.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:16.218 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-13T18:27:16.276 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi146.stderr:2024-09-13T18:27:16.266+0000 7f3b59630640 -1 init, newargv = 0x55f14d8f9240 newargc=15 2024-09-13T18:27:16.277 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi146.stderr:ceph-fuse[33473]: starting ceph client 2024-09-13T18:27:16.286 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi146.stderr:ceph-fuse[33473]: starting fuse 2024-09-13T18:27:17.324 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-13T18:27:17.342 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:27:17.343 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:27:17.353 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:27:17.354 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:27:17.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:17.355 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-13T18:27:17.364 INFO:teuthology.orchestra.run.smithi146.stdout:52 2024-09-13T18:27:17.365 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-09-13T18:27:17.365 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:27:17.365 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-13T18:27:17.365 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-13T18:27:17.365 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-13T18:27:17.365 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-13T18:27:17.365 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-13T18:27:17.365 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-13T18:27:17.365 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T18:27:17.365 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-13T18:27:17.365 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T18:27:17.366 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-13T18:27:17.367 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.0")) 2024-09-13T18:27:17.367 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-13T18:27:17.446 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.0.33473.asok 2024-09-13T18:27:17.631 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33473.asok 2024-09-13T18:27:17.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:17.632 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33473.asok status 2024-09-13T18:27:17.782 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:27:17.782 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-13T18:27:17.782 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T18:27:17.782 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "0", 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "33473", 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4748, 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 4748 2024-09-13T18:27:17.783 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "172.21.15.146:0", 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1528125864 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "172.21.15.146:0", 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1528125864 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.4748 172.21.15.146:0/1528125864", 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "172.21.15.146:0/1528125864", 2024-09-13T18:27:17.784 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-13T18:27:17.785 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 15, 2024-09-13T18:27:17.785 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 25, 2024-09-13T18:27:17.785 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-13T18:27:17.785 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-13T18:27:17.785 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-13T18:27:17.785 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:27:17.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:17.792 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-13T18:27:17.792 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T18:27:17.800 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-13T18:27:17.801 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T18:27:17.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:27:17.801 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T18:27:17.816 INFO:teuthology.run_tasks:Running task mds_thrash... 2024-09-13T18:27:17.829 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:27:17.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.938+0000 7ff864e90700 1 Processor -- start 2024-09-13T18:27:17.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.938+0000 7ff864e90700 1 -- start start 2024-09-13T18:27:17.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.938+0000 7ff864e90700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff860153bc0 0x7ff8600c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:17.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.938+0000 7ff864e90700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff8600c6a50 0x7ff860160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:17.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.938+0000 7ff864e90700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff860153fa0 con 0x7ff8600c6610 2024-09-13T18:27:17.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.938+0000 7ff864e90700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8600aa500 con 0x7ff8600c6a50 2024-09-13T18:27:17.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.938+0000 7ff864e90700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86015a7e0 con 0x7ff860153bc0 2024-09-13T18:27:17.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.938+0000 7ff85effd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff8600c6610 0x7ff8600cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56710/0 (socket says 172.21.15.146:56710) 2024-09-13T18:27:17.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.938+0000 7ff85effd700 1 -- 172.21.15.146:0/2347808017 learned_addr learned my addr 172.21.15.146:0/2347808017 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:17.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.938+0000 7ff85ffff700 1 --2- 172.21.15.146:0/2347808017 >> v2:172.21.15.146:3300/0 conn(0x7ff8600c6a50 0x7ff860160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:17.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.938+0000 7ff85f7fe700 1 --2- 172.21.15.146:0/2347808017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff860153bc0 0x7ff8600c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff85e7fc700 1 -- 172.21.15.146:0/2347808017 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2482252247 0 0) 0x7ff860153fa0 con 0x7ff8600c6610 2024-09-13T18:27:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff85e7fc700 1 -- 172.21.15.146:0/2347808017 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8480035d0 con 0x7ff8600c6610 2024-09-13T18:27:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff85ffff700 1 -- 172.21.15.146:0/2347808017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff860153bc0 msgr2=0x7ff8600c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff85ffff700 1 --2- 172.21.15.146:0/2347808017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff860153bc0 0x7ff8600c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff85ffff700 1 -- 172.21.15.146:0/2347808017 >> v1:172.21.15.146:6789/0 conn(0x7ff8600c6610 legacy=0x7ff8600cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff85ffff700 1 -- 172.21.15.146:0/2347808017 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8600c9320 con 0x7ff8600c6a50 2024-09-13T18:27:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff85ffff700 1 --2- 172.21.15.146:0/2347808017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8600c6a50 0x7ff860160c20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff858009d90 tx=0x7ff85800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52033be55c971b90 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff85e7fc700 1 -- 172.21.15.146:0/2347808017 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85800e070 con 0x7ff8600c6a50 2024-09-13T18:27:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff85e7fc700 1 -- 172.21.15.146:0/2347808017 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85800bb20 con 0x7ff8600c6a50 2024-09-13T18:27:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff85e7fc700 1 -- 172.21.15.146:0/2347808017 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff858012430 con 0x7ff8600c6a50 2024-09-13T18:27:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff864e90700 1 -- 172.21.15.146:0/2347808017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8600c6a50 msgr2=0x7ff860160c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff864e90700 1 --2- 172.21.15.146:0/2347808017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8600c6a50 0x7ff860160c20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff858009d90 tx=0x7ff85800b330 comp rx=0 tx=0).stop 2024-09-13T18:27:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff864e90700 1 -- 172.21.15.146:0/2347808017 shutdown_connections 2024-09-13T18:27:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff864e90700 1 --2- 172.21.15.146:0/2347808017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8600c6a50 0x7ff860160c20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff864e90700 1 --2- 172.21.15.146:0/2347808017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff860153bc0 0x7ff8600c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff864e90700 1 -- 172.21.15.146:0/2347808017 >> 172.21.15.146:0/2347808017 conn(0x7ff860066090 msgr2=0x7ff860066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff864e90700 1 -- 172.21.15.146:0/2347808017 shutdown_connections 2024-09-13T18:27:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff864e90700 1 -- 172.21.15.146:0/2347808017 wait complete. 2024-09-13T18:27:17.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff864e90700 1 Processor -- start 2024-09-13T18:27:17.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.942+0000 7ff864e90700 1 -- start start 2024-09-13T18:27:17.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff864e90700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8600c6610 0x7ff8600afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:17.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff864e90700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8600c6a50 0x7ff8601efe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:17.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff864e90700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff860153bc0 0x7ff860150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:17.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff864e90700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff860154b60 con 0x7ff8600c6a50 2024-09-13T18:27:17.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff864e90700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8601560a0 con 0x7ff8600c6610 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff864e90700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff860154db0 con 0x7ff860153bc0 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff85effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8600c6a50 0x7ff8601efe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff85effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8600c6a50 0x7ff8601efe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52556/0 (socket says 172.21.15.146:52556) 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff85effd700 1 -- 172.21.15.146:0/3443994936 learned_addr learned my addr 172.21.15.146:0/3443994936 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff85ffff700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff860153bc0 0x7ff860150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff85f7fe700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8600c6610 0x7ff8600afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff85effd700 1 -- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff860153bc0 msgr2=0x7ff860150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff85effd700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff860153bc0 0x7ff860150660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff85effd700 1 -- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8600c6610 msgr2=0x7ff8600afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff85effd700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8600c6610 0x7ff8600afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff85effd700 1 -- 172.21.15.146:0/3443994936 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff858009a70 con 0x7ff8600c6a50 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff85f7fe700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8600c6610 0x7ff8600afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff85effd700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8600c6a50 0x7ff8601efe30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff850005930 tx=0x7ff85000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:17.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff847fff700 1 -- 172.21.15.146:0/3443994936 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850005d90 con 0x7ff8600c6a50 2024-09-13T18:27:17.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff847fff700 1 -- 172.21.15.146:0/3443994936 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8500070d0 con 0x7ff8600c6a50 2024-09-13T18:27:17.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff864e90700 1 -- 172.21.15.146:0/3443994936 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8601ecbf0 con 0x7ff8600c6a50 2024-09-13T18:27:17.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff847fff700 1 -- 172.21.15.146:0/3443994936 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850007480 con 0x7ff8600c6a50 2024-09-13T18:27:17.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff864e90700 1 -- 172.21.15.146:0/3443994936 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8601ed1b0 con 0x7ff8600c6a50 2024-09-13T18:27:17.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.946+0000 7ff864e90700 1 -- 172.21.15.146:0/3443994936 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff86015a2c0 con 0x7ff8600c6a50 2024-09-13T18:27:17.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.950+0000 7ff847fff700 1 -- 172.21.15.146:0/3443994936 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff8500075e0 con 0x7ff8600c6a50 2024-09-13T18:27:17.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.950+0000 7ff847fff700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff84c066730 0x7ff84c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:17.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.950+0000 7ff847fff700 1 -- 172.21.15.146:0/3443994936 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff850083980 con 0x7ff8600c6a50 2024-09-13T18:27:17.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.950+0000 7ff85f7fe700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff84c066730 0x7ff84c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:17.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.950+0000 7ff85f7fe700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff84c066730 0x7ff84c068bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff854009ad0 tx=0x7ff854008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:17.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:17.954+0000 7ff847fff700 1 -- 172.21.15.146:0/3443994936 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff850050a90 con 0x7ff8600c6a50 2024-09-13T18:27:18.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.126+0000 7ff864e90700 1 -- 172.21.15.146:0/3443994936 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff86006f690 con 0x7ff8600c6a50 2024-09-13T18:27:18.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.130+0000 7ff847fff700 1 -- 172.21.15.146:0/3443994936 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7ff8500540b0 con 0x7ff8600c6a50 2024-09-13T18:27:18.139 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:18.139 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:27:18.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.134+0000 7ff864e90700 1 -- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff84c066730 msgr2=0x7ff84c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.134+0000 7ff864e90700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff84c066730 0x7ff84c068bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff854009ad0 tx=0x7ff854008040 comp rx=0 tx=0).stop 2024-09-13T18:27:18.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.134+0000 7ff864e90700 1 -- 172.21.15.146:0/3443994936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8600c6a50 msgr2=0x7ff8601efe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.134+0000 7ff864e90700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8600c6a50 0x7ff8601efe30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff850005930 tx=0x7ff85000ca90 comp rx=0 tx=0).stop 2024-09-13T18:27:18.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.134+0000 7ff864e90700 1 -- 172.21.15.146:0/3443994936 shutdown_connections 2024-09-13T18:27:18.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.134+0000 7ff864e90700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff84c066730 0x7ff84c068bf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.134+0000 7ff864e90700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff860153bc0 0x7ff860150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.134+0000 7ff864e90700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8600c6a50 0x7ff8601efe30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.134+0000 7ff864e90700 1 --2- 172.21.15.146:0/3443994936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8600c6610 0x7ff8600afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.134+0000 7ff864e90700 1 -- 172.21.15.146:0/3443994936 >> 172.21.15.146:0/3443994936 conn(0x7ff860066090 msgr2=0x7ff8601513e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:18.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.134+0000 7ff864e90700 1 -- 172.21.15.146:0/3443994936 shutdown_connections 2024-09-13T18:27:18.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.134+0000 7ff864e90700 1 -- 172.21.15.146:0/3443994936 wait complete. 2024-09-13T18:27:18.154 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:26:24.867515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:26:30.782269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:26:31.323277+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:26:34.393386+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:27:18.154 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:27:18.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.258+0000 7fb72d544700 1 Processor -- start 2024-09-13T18:27:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.258+0000 7fb72d544700 1 -- start start 2024-09-13T18:27:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.258+0000 7fb72d544700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb7281538c0 0x7fb7280ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.258+0000 7fb72d544700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7280cb340 con 0x7fb7280cc680 2024-09-13T18:27:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.258+0000 7fb72d544700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72815a510 con 0x7fb72815a080 2024-09-13T18:27:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.258+0000 7fb72d544700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7280cb5e0 con 0x7fb7281538c0 2024-09-13T18:27:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.258+0000 7fb727fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb7281538c0 0x7fb7280ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.258+0000 7fb7277fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb7280cc680 0x7fb7280c7580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56720/0 (socket says 172.21.15.146:56720) 2024-09-13T18:27:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.258+0000 7fb7277fe700 1 -- 172.21.15.146:0/3185827033 learned_addr learned my addr 172.21.15.146:0/3185827033 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb726ffd700 1 -- 172.21.15.146:0/3185827033 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524639663 0 0) 0x7fb7280cb340 con 0x7fb7280cc680 2024-09-13T18:27:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb726ffd700 1 -- 172.21.15.146:0/3185827033 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7140035d0 con 0x7fb7280cc680 2024-09-13T18:27:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb727fff700 1 -- 172.21.15.146:0/3185827033 >> v1:172.21.15.154:6789/0 conn(0x7fb72815a080 legacy=0x7fb7280cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb727fff700 1 -- 172.21.15.146:0/3185827033 >> v1:172.21.15.146:6789/0 conn(0x7fb7280cc680 legacy=0x7fb7280c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb727fff700 1 -- 172.21.15.146:0/3185827033 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7280cb8c0 con 0x7fb7281538c0 2024-09-13T18:27:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb727fff700 1 --2- 172.21.15.146:0/3185827033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7281538c0 0x7fb7280ca240 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb720009880 tx=0x7fb720009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2634e8191e108cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb726ffd700 1 -- 172.21.15.146:0/3185827033 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720011830 con 0x7fb7281538c0 2024-09-13T18:27:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb726ffd700 1 -- 172.21.15.146:0/3185827033 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb720011990 con 0x7fb7281538c0 2024-09-13T18:27:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb726ffd700 1 -- 172.21.15.146:0/3185827033 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720011d20 con 0x7fb7281538c0 2024-09-13T18:27:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb72d544700 1 -- 172.21.15.146:0/3185827033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7281538c0 msgr2=0x7fb7280ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb72d544700 1 --2- 172.21.15.146:0/3185827033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7281538c0 0x7fb7280ca240 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb720009880 tx=0x7fb720009b90 comp rx=0 tx=0).stop 2024-09-13T18:27:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb72d544700 1 -- 172.21.15.146:0/3185827033 shutdown_connections 2024-09-13T18:27:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb72d544700 1 --2- 172.21.15.146:0/3185827033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7281538c0 0x7fb7280ca240 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb72d544700 1 -- 172.21.15.146:0/3185827033 >> 172.21.15.146:0/3185827033 conn(0x7fb728066270 msgr2=0x7fb728066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb72d544700 1 -- 172.21.15.146:0/3185827033 shutdown_connections 2024-09-13T18:27:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb72d544700 1 -- 172.21.15.146:0/3185827033 wait complete. 2024-09-13T18:27:18.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb72d544700 1 Processor -- start 2024-09-13T18:27:18.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.262+0000 7fb72d544700 1 -- start start 2024-09-13T18:27:18.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72d544700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7280cc680 0x7fb7280b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72d544700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7281538c0 0x7fb72815b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72d544700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb72815a080 0x7fb72815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72d544700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72815b300 con 0x7fb72815a080 2024-09-13T18:27:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72d544700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72815cb30 con 0x7fb7281538c0 2024-09-13T18:27:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72d544700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72815d730 con 0x7fb7280cc680 2024-09-13T18:27:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72cd43700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb72815a080 0x7fb72815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72cd43700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb72815a080 0x7fb72815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52578/0 (socket says 172.21.15.146:52578) 2024-09-13T18:27:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72cd43700 1 -- 172.21.15.146:0/3517420593 learned_addr learned my addr 172.21.15.146:0/3517420593 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb727fff700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7280cc680 0x7fb7280b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb7277fe700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7281538c0 0x7fb72815b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72cd43700 1 -- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7280cc680 msgr2=0x7fb7280b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72cd43700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7280cc680 0x7fb7280b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72cd43700 1 -- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7281538c0 msgr2=0x7fb72815b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72cd43700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7281538c0 0x7fb72815b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72cd43700 1 -- 172.21.15.146:0/3517420593 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7200094b0 con 0x7fb72815a080 2024-09-13T18:27:18.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb7277fe700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7281538c0 0x7fb72815b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:18.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72cd43700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb72815a080 0x7fb72815bcd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb71000c430 tx=0x7fb71000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:18.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb724ff9700 1 -- 172.21.15.146:0/3517420593 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7100052d0 con 0x7fb72815a080 2024-09-13T18:27:18.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb724ff9700 1 -- 172.21.15.146:0/3517420593 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb710005430 con 0x7fb72815a080 2024-09-13T18:27:18.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72d544700 1 -- 172.21.15.146:0/3517420593 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72815cdf0 con 0x7fb72815a080 2024-09-13T18:27:18.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb724ff9700 1 -- 172.21.15.146:0/3517420593 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7100055a0 con 0x7fb72815a080 2024-09-13T18:27:18.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.266+0000 7fb72d544700 1 -- 172.21.15.146:0/3517420593 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72815d100 con 0x7fb72815a080 2024-09-13T18:27:18.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.270+0000 7fb72d544700 1 -- 172.21.15.146:0/3517420593 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72815b4b0 con 0x7fb72815a080 2024-09-13T18:27:18.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.270+0000 7fb724ff9700 1 -- 172.21.15.146:0/3517420593 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb7100038e0 con 0x7fb72815a080 2024-09-13T18:27:18.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.270+0000 7fb724ff9700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb708066730 0x7fb708068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.270+0000 7fb724ff9700 1 -- 172.21.15.146:0/3517420593 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb710083760 con 0x7fb72815a080 2024-09-13T18:27:18.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.270+0000 7fb727fff700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb708066730 0x7fb708068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.270+0000 7fb727fff700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb708066730 0x7fb708068bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb7200090b0 tx=0x7fb720008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:18.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.278+0000 7fb724ff9700 1 -- 172.21.15.146:0/3517420593 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb72815b4b0 con 0x7fb72815a080 2024-09-13T18:27:18.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.446+0000 7fb72d544700 1 -- 172.21.15.146:0/3517420593 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7280c99c0 con 0x7fb72815a080 2024-09-13T18:27:18.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.450+0000 7fb724ff9700 1 -- 172.21.15.146:0/3517420593 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb710050870 con 0x7fb72815a080 2024-09-13T18:27:18.457 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-09-13T18:27:18.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.450+0000 7fb72d544700 1 -- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb708066730 msgr2=0x7fb708068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.450+0000 7fb72d544700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb708066730 0x7fb708068bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb7200090b0 tx=0x7fb720008030 comp rx=0 tx=0).stop 2024-09-13T18:27:18.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.450+0000 7fb72d544700 1 -- 172.21.15.146:0/3517420593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb72815a080 msgr2=0x7fb72815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.450+0000 7fb72d544700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb72815a080 0x7fb72815bcd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb71000c430 tx=0x7fb71000c7f0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.450+0000 7fb72d544700 1 -- 172.21.15.146:0/3517420593 shutdown_connections 2024-09-13T18:27:18.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.450+0000 7fb72d544700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb708066730 0x7fb708068bf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.450+0000 7fb72d544700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb72815a080 0x7fb72815bcd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.450+0000 7fb72d544700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7281538c0 0x7fb72815b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.450+0000 7fb72d544700 1 --2- 172.21.15.146:0/3517420593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7280cc680 0x7fb7280b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.450+0000 7fb72d544700 1 -- 172.21.15.146:0/3517420593 >> 172.21.15.146:0/3517420593 conn(0x7fb728066270 msgr2=0x7fb72814e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:18.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.450+0000 7fb72d544700 1 -- 172.21.15.146:0/3517420593 shutdown_connections 2024-09-13T18:27:18.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.450+0000 7fb72d544700 1 -- 172.21.15.146:0/3517420593 wait complete. 2024-09-13T18:27:18.472 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:27:18.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.574+0000 7fa73747a700 1 Processor -- start 2024-09-13T18:27:18.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.574+0000 7fa73747a700 1 -- start start 2024-09-13T18:27:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa73747a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa73015e650 0x7fa730070b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa73747a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa7300710d0 0x7fa730071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa73747a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7300c6380 con 0x7fa730157f80 2024-09-13T18:27:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa73747a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa730158360 con 0x7fa7300710d0 2024-09-13T18:27:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa73747a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300c60a0 con 0x7fa73015e650 2024-09-13T18:27:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa735216700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa730157f80 0x7fa7300c6c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56722/0 (socket says 172.21.15.146:56722) 2024-09-13T18:27:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa735216700 1 -- 172.21.15.146:0/4211140770 learned_addr learned my addr 172.21.15.146:0/4211140770 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa734a15700 1 --2- 172.21.15.146:0/4211140770 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa73015e650 0x7fa730070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa735a17700 1 --2- 172.21.15.146:0/4211140770 >> v2:172.21.15.154:3300/0 conn(0x7fa7300710d0 0x7fa730071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa727fff700 1 -- 172.21.15.146:0/4211140770 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495480544 0 0) 0x7fa7300c6380 con 0x7fa730157f80 2024-09-13T18:27:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa727fff700 1 -- 172.21.15.146:0/4211140770 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa71c0035d0 con 0x7fa730157f80 2024-09-13T18:27:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa727fff700 1 -- 172.21.15.146:0/4211140770 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3244679620 0 0) 0x7fa71c0035d0 con 0x7fa730157f80 2024-09-13T18:27:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa734a15700 1 -- 172.21.15.146:0/4211140770 >> v2:172.21.15.154:3300/0 conn(0x7fa7300710d0 msgr2=0x7fa730071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa734a15700 1 --2- 172.21.15.146:0/4211140770 >> v2:172.21.15.154:3300/0 conn(0x7fa7300710d0 0x7fa730071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa734a15700 1 -- 172.21.15.146:0/4211140770 >> v1:172.21.15.146:6789/0 conn(0x7fa730157f80 legacy=0x7fa7300c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa734a15700 1 -- 172.21.15.146:0/4211140770 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7300c0ba0 con 0x7fa73015e650 2024-09-13T18:27:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa734a15700 1 --2- 172.21.15.146:0/4211140770 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa73015e650 0x7fa730070b90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009850 tx=0x7fa72c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2883cc7927bbb2d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa727fff700 1 -- 172.21.15.146:0/4211140770 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720001670 con 0x7fa73015e650 2024-09-13T18:27:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa727fff700 1 -- 172.21.15.146:0/4211140770 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c004030 con 0x7fa73015e650 2024-09-13T18:27:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa727fff700 1 -- 172.21.15.146:0/4211140770 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0043a0 con 0x7fa73015e650 2024-09-13T18:27:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa73747a700 1 -- 172.21.15.146:0/4211140770 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa73015e650 msgr2=0x7fa730070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa73747a700 1 --2- 172.21.15.146:0/4211140770 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa73015e650 0x7fa730070b90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009850 tx=0x7fa72c009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa73747a700 1 -- 172.21.15.146:0/4211140770 shutdown_connections 2024-09-13T18:27:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa73747a700 1 --2- 172.21.15.146:0/4211140770 >> v2:172.21.15.154:3300/0 conn(0x7fa7300710d0 0x7fa730071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa73747a700 1 --2- 172.21.15.146:0/4211140770 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa73015e650 0x7fa730070b90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa73747a700 1 -- 172.21.15.146:0/4211140770 >> 172.21.15.146:0/4211140770 conn(0x7fa73006f5c0 msgr2=0x7fa73006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:18.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa73747a700 1 -- 172.21.15.146:0/4211140770 shutdown_connections 2024-09-13T18:27:18.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.578+0000 7fa73747a700 1 -- 172.21.15.146:0/4211140770 wait complete. 2024-09-13T18:27:18.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa73747a700 1 Processor -- start 2024-09-13T18:27:18.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa73747a700 1 -- start start 2024-09-13T18:27:18.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa73747a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7300710d0 0x7fa7301f3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa73747a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa730157f80 0x7fa7301e5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa73747a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa73015e650 0x7fa730153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa73747a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa730156fc0 con 0x7fa730157f80 2024-09-13T18:27:18.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa73747a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7301f4220 con 0x7fa73015e650 2024-09-13T18:27:18.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa73747a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7301571f0 con 0x7fa7300710d0 2024-09-13T18:27:18.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa734a15700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa730157f80 0x7fa7301e5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa734a15700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa730157f80 0x7fa7301e5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52584/0 (socket says 172.21.15.146:52584) 2024-09-13T18:27:18.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa734a15700 1 -- 172.21.15.146:0/2651723898 learned_addr learned my addr 172.21.15.146:0/2651723898 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa735216700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7300710d0 0x7fa7301f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa735a17700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa73015e650 0x7fa730153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa734a15700 1 -- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7300710d0 msgr2=0x7fa7301f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa734a15700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7300710d0 0x7fa7301f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa734a15700 1 -- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa73015e650 msgr2=0x7fa730153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa734a15700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa73015e650 0x7fa730153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa734a15700 1 -- 172.21.15.146:0/2651723898 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c0094b0 con 0x7fa730157f80 2024-09-13T18:27:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa734a15700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa730157f80 0x7fa7301e5860 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0090b0 tx=0x7fa72c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa725ffb700 1 -- 172.21.15.146:0/2651723898 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c003b70 con 0x7fa730157f80 2024-09-13T18:27:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa725ffb700 1 -- 172.21.15.146:0/2651723898 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c01f070 con 0x7fa730157f80 2024-09-13T18:27:18.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa73747a700 1 -- 172.21.15.146:0/2651723898 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7301f3f40 con 0x7fa730157f80 2024-09-13T18:27:18.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa725ffb700 1 -- 172.21.15.146:0/2651723898 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c011630 con 0x7fa730157f80 2024-09-13T18:27:18.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.582+0000 7fa73747a700 1 -- 172.21.15.146:0/2651723898 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7301e5340 con 0x7fa730157f80 2024-09-13T18:27:18.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.586+0000 7fa73747a700 1 -- 172.21.15.146:0/2651723898 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7300c5c60 con 0x7fa730157f80 2024-09-13T18:27:18.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.586+0000 7fa725ffb700 1 -- 172.21.15.146:0/2651723898 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa72c021030 con 0x7fa730157f80 2024-09-13T18:27:18.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.586+0000 7fa725ffb700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa714066760 0x7fa714068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.586+0000 7fa725ffb700 1 -- 172.21.15.146:0/2651723898 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa72c085720 con 0x7fa730157f80 2024-09-13T18:27:18.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.586+0000 7fa735216700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa714066760 0x7fa714068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.590+0000 7fa735216700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa714066760 0x7fa714068c20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa7200011e0 tx=0x7fa720001da0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:18.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.594+0000 7fa725ffb700 1 -- 172.21.15.146:0/2651723898 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa72c052830 con 0x7fa730157f80 2024-09-13T18:27:18.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.762+0000 7fa73747a700 1 -- 172.21.15.146:0/2651723898 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7300c67a0 con 0x7fa730157f80 2024-09-13T18:27:18.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.762+0000 7fa725ffb700 1 -- 172.21.15.146:0/2651723898 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa72c055e50 con 0x7fa730157f80 2024-09-13T18:27:18.772 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:27:18.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.766+0000 7fa73747a700 1 -- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa714066760 msgr2=0x7fa714068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.766+0000 7fa73747a700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa714066760 0x7fa714068c20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa7200011e0 tx=0x7fa720001da0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.766+0000 7fa73747a700 1 -- 172.21.15.146:0/2651723898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa730157f80 msgr2=0x7fa7301e5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.766+0000 7fa73747a700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa730157f80 0x7fa7301e5860 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0090b0 tx=0x7fa72c004770 comp rx=0 tx=0).stop 2024-09-13T18:27:18.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.766+0000 7fa73747a700 1 -- 172.21.15.146:0/2651723898 shutdown_connections 2024-09-13T18:27:18.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.766+0000 7fa73747a700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa714066760 0x7fa714068c20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.766+0000 7fa73747a700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa73015e650 0x7fa730153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.766+0000 7fa73747a700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa730157f80 0x7fa7301e5860 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.766+0000 7fa73747a700 1 --2- 172.21.15.146:0/2651723898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7300710d0 0x7fa7301f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.766+0000 7fa73747a700 1 -- 172.21.15.146:0/2651723898 >> 172.21.15.146:0/2651723898 conn(0x7fa73006f5c0 msgr2=0x7fa7300c99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:18.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.766+0000 7fa73747a700 1 -- 172.21.15.146:0/2651723898 shutdown_connections 2024-09-13T18:27:18.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.766+0000 7fa73747a700 1 -- 172.21.15.146:0/2651723898 wait complete. 2024-09-13T18:27:18.787 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:27:18.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.902+0000 7f27be9d5700 1 Processor -- start 2024-09-13T18:27:18.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.902+0000 7f27be9d5700 1 -- start start 2024-09-13T18:27:18.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.902+0000 7f27be9d5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27b81539c0 0x7f27b80c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.902+0000 7f27be9d5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27b815a570 0x7f27b80c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.902+0000 7f27be9d5700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b80aa010 con 0x7f27b806ffd0 2024-09-13T18:27:18.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.902+0000 7f27be9d5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b80aa5d0 con 0x7f27b81539c0 2024-09-13T18:27:18.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.902+0000 7f27be9d5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80aa2f0 con 0x7f27b815a570 2024-09-13T18:27:18.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.902+0000 7f27be1d4700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f27b806ffd0 0x7f27b80caa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56726/0 (socket says 172.21.15.146:56726) 2024-09-13T18:27:18.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.902+0000 7f27be1d4700 1 -- 172.21.15.146:0/161988481 learned_addr learned my addr 172.21.15.146:0/161988481 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:18.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.902+0000 7f27bd9d3700 1 --2- 172.21.15.146:0/161988481 >> v2:172.21.15.154:3300/0 conn(0x7f27b81539c0 0x7f27b80c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.902+0000 7f27bd1d2700 1 --2- 172.21.15.146:0/161988481 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27b815a570 0x7f27b80c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.902+0000 7f27bc9d1700 1 -- 172.21.15.146:0/161988481 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459629441 0 0) 0x7f27b80aa010 con 0x7f27b806ffd0 2024-09-13T18:27:18.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27bc9d1700 1 -- 172.21.15.146:0/161988481 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27a40035d0 con 0x7f27b806ffd0 2024-09-13T18:27:18.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27bd9d3700 1 -- 172.21.15.146:0/161988481 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27b815a570 msgr2=0x7f27b80c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:18.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27bd9d3700 1 --2- 172.21.15.146:0/161988481 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27b815a570 0x7f27b80c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27bd9d3700 1 -- 172.21.15.146:0/161988481 >> v1:172.21.15.146:6789/0 conn(0x7f27b806ffd0 legacy=0x7f27b80caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27bd9d3700 1 -- 172.21.15.146:0/161988481 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8159a90 con 0x7f27b81539c0 2024-09-13T18:27:18.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27bd9d3700 1 --2- 172.21.15.146:0/161988481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b81539c0 0x7f27b80c8de0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009850 tx=0x7f27b4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef076d8a653750db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:18.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27bc9d1700 1 -- 172.21.15.146:0/161988481 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4011830 con 0x7f27b81539c0 2024-09-13T18:27:18.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27bc9d1700 1 -- 172.21.15.146:0/161988481 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b4011990 con 0x7f27b81539c0 2024-09-13T18:27:18.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27bc9d1700 1 -- 172.21.15.146:0/161988481 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4011d40 con 0x7f27b81539c0 2024-09-13T18:27:18.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27be9d5700 1 -- 172.21.15.146:0/161988481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b81539c0 msgr2=0x7f27b80c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27be9d5700 1 --2- 172.21.15.146:0/161988481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b81539c0 0x7f27b80c8de0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009850 tx=0x7f27b4009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:18.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27be9d5700 1 -- 172.21.15.146:0/161988481 shutdown_connections 2024-09-13T18:27:18.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27be9d5700 1 --2- 172.21.15.146:0/161988481 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27b815a570 0x7f27b80c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27be9d5700 1 --2- 172.21.15.146:0/161988481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b81539c0 0x7f27b80c8de0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27be9d5700 1 -- 172.21.15.146:0/161988481 >> 172.21.15.146:0/161988481 conn(0x7f27b8066090 msgr2=0x7f27b8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:18.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27be9d5700 1 -- 172.21.15.146:0/161988481 shutdown_connections 2024-09-13T18:27:18.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.906+0000 7f27be9d5700 1 -- 172.21.15.146:0/161988481 wait complete. 2024-09-13T18:27:18.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be9d5700 1 Processor -- start 2024-09-13T18:27:18.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be9d5700 1 -- start start 2024-09-13T18:27:18.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be9d5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27b806ffd0 0x7f27b80bfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be9d5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b81539c0 0x7f27b80c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be9d5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27b815a570 0x7f27b81eff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be9d5700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80c0230 con 0x7f27b815a570 2024-09-13T18:27:18.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be9d5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80c0450 con 0x7f27b81539c0 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be9d5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b81eca20 con 0x7f27b806ffd0 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be1d4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27b815a570 0x7f27b81eff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be1d4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27b815a570 0x7f27b81eff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52586/0 (socket says 172.21.15.146:52586) 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be1d4700 1 -- 172.21.15.146:0/1827782338 learned_addr learned my addr 172.21.15.146:0/1827782338 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27bd9d3700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27b806ffd0 0x7f27b80bfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27bd1d2700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b81539c0 0x7f27b80c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be1d4700 1 -- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27b806ffd0 msgr2=0x7f27b80bfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be1d4700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27b806ffd0 0x7f27b80bfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be1d4700 1 -- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b81539c0 msgr2=0x7f27b80c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be1d4700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b81539c0 0x7f27b80c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be1d4700 1 -- 172.21.15.146:0/1827782338 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b40094b0 con 0x7f27b815a570 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27bd1d2700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b81539c0 0x7f27b80c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be1d4700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27b815a570 0x7f27b81eff10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f27b000cb40 tx=0x7f27b000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:18.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27ae7fc700 1 -- 172.21.15.146:0/1827782338 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b000a7b0 con 0x7f27b815a570 2024-09-13T18:27:18.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27ae7fc700 1 -- 172.21.15.146:0/1827782338 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b000a910 con 0x7f27b815a570 2024-09-13T18:27:18.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be9d5700 1 -- 172.21.15.146:0/1827782338 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b81ecd00 con 0x7f27b815a570 2024-09-13T18:27:18.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27ae7fc700 1 -- 172.21.15.146:0/1827782338 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b000ac80 con 0x7f27b815a570 2024-09-13T18:27:18.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.910+0000 7f27be9d5700 1 -- 172.21.15.146:0/1827782338 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b81ed2c0 con 0x7f27b815a570 2024-09-13T18:27:18.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.914+0000 7f27be9d5700 1 -- 172.21.15.146:0/1827782338 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b8066470 con 0x7f27b815a570 2024-09-13T18:27:18.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.914+0000 7f27ae7fc700 1 -- 172.21.15.146:0/1827782338 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f27b001b3b0 con 0x7f27b815a570 2024-09-13T18:27:18.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.914+0000 7f27ae7fc700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f279c066730 0x7f279c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:18.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.914+0000 7f27ae7fc700 1 -- 172.21.15.146:0/1827782338 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f27b0083de0 con 0x7f27b815a570 2024-09-13T18:27:18.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.914+0000 7f27bd9d3700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f279c066730 0x7f279c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:18.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.918+0000 7f27bd9d3700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f279c066730 0x7f279c068bf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f27b40090b0 tx=0x7f27b4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:18.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:18.926+0000 7f27ae7fc700 1 -- 172.21.15.146:0/1827782338 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f27b0050e70 con 0x7f27b815a570 2024-09-13T18:27:19.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.090+0000 7f27be9d5700 1 -- 172.21.15.146:0/1827782338 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27b8070820 con 0x7f27b815a570 2024-09-13T18:27:19.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.090+0000 7f27ae7fc700 1 -- 172.21.15.146:0/1827782338 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f27b00170a0 con 0x7f27b815a570 2024-09-13T18:27:19.100 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-09-13T18:27:19.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.094+0000 7f27be9d5700 1 -- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f279c066730 msgr2=0x7f279c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.094+0000 7f27be9d5700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f279c066730 0x7f279c068bf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f27b40090b0 tx=0x7f27b4008030 comp rx=0 tx=0).stop 2024-09-13T18:27:19.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.094+0000 7f27be9d5700 1 -- 172.21.15.146:0/1827782338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27b815a570 msgr2=0x7f27b81eff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.094+0000 7f27be9d5700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27b815a570 0x7f27b81eff10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f27b000cb40 tx=0x7f27b000cb70 comp rx=0 tx=0).stop 2024-09-13T18:27:19.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.094+0000 7f27be9d5700 1 -- 172.21.15.146:0/1827782338 shutdown_connections 2024-09-13T18:27:19.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.094+0000 7f27be9d5700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f279c066730 0x7f279c068bf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.094+0000 7f27be9d5700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27b815a570 0x7f27b81eff10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.094+0000 7f27be9d5700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b81539c0 0x7f27b80c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.094+0000 7f27be9d5700 1 --2- 172.21.15.146:0/1827782338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27b806ffd0 0x7f27b80bfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.094+0000 7f27be9d5700 1 -- 172.21.15.146:0/1827782338 >> 172.21.15.146:0/1827782338 conn(0x7f27b8066090 msgr2=0x7f27b8150f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:19.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.094+0000 7f27be9d5700 1 -- 172.21.15.146:0/1827782338 shutdown_connections 2024-09-13T18:27:19.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.094+0000 7f27be9d5700 1 -- 172.21.15.146:0/1827782338 wait complete. 2024-09-13T18:27:19.115 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:27:19.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa41622f700 1 Processor -- start 2024-09-13T18:27:19.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa41622f700 1 -- start start 2024-09-13T18:27:19.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa41622f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4100c57f0 0x7fa4100cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa41622f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa4100ca020 0x7fa4100ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa41622f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4100c5bd0 con 0x7fa4100cad90 2024-09-13T18:27:19.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa41622f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa41015a160 con 0x7fa4100ca020 2024-09-13T18:27:19.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa41622f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa410159830 con 0x7fa4100c57f0 2024-09-13T18:27:19.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa40f7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4100c57f0 0x7fa4100cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa40effd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa4100ca020 0x7fa4100ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa40effd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa4100ca020 0x7fa4100ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:38140/0 (socket says 172.21.15.146:38140) 2024-09-13T18:27:19.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa40f7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4100c57f0 0x7fa4100cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:52222/0 (socket says 172.21.15.146:52222) 2024-09-13T18:27:19.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa40effd700 1 -- 172.21.15.146:0/2364320272 learned_addr learned my addr 172.21.15.146:0/2364320272 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:19.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa40ffff700 1 --1- 172.21.15.146:0/2364320272 >> v1:172.21.15.154:6789/0 conn(0x7fa4100cad90 0x7fa4100cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:59116/0 (socket says 172.21.15.146:59116) 2024-09-13T18:27:19.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa40f7fe700 1 -- 172.21.15.146:0/2364320272 >> v2:172.21.15.154:3300/0 conn(0x7fa4100ca020 msgr2=0x7fa4100ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:19.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa40f7fe700 1 --2- 172.21.15.146:0/2364320272 >> v2:172.21.15.154:3300/0 conn(0x7fa4100ca020 0x7fa4100ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa40f7fe700 1 -- 172.21.15.146:0/2364320272 >> v1:172.21.15.154:6789/0 conn(0x7fa4100cad90 legacy=0x7fa4100cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa40f7fe700 1 -- 172.21.15.146:0/2364320272 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4100c9bb0 con 0x7fa4100c57f0 2024-09-13T18:27:19.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.230+0000 7fa40effd700 1 --2- 172.21.15.146:0/2364320272 >> v2:172.21.15.154:3300/0 conn(0x7fa4100ca020 0x7fa4100ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:19.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa40f7fe700 1 --2- 172.21.15.146:0/2364320272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4100c57f0 0x7fa4100cb930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009850 tx=0x7fa3f8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25854ff610752c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:19.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa40e7fc700 1 -- 172.21.15.146:0/2364320272 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8011830 con 0x7fa4100c57f0 2024-09-13T18:27:19.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa40e7fc700 1 -- 172.21.15.146:0/2364320272 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f8011990 con 0x7fa4100c57f0 2024-09-13T18:27:19.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa40e7fc700 1 -- 172.21.15.146:0/2364320272 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8011d40 con 0x7fa4100c57f0 2024-09-13T18:27:19.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa41622f700 1 -- 172.21.15.146:0/2364320272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4100c57f0 msgr2=0x7fa4100cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa41622f700 1 --2- 172.21.15.146:0/2364320272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4100c57f0 0x7fa4100cb930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009850 tx=0x7fa3f8009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:19.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa41622f700 1 -- 172.21.15.146:0/2364320272 shutdown_connections 2024-09-13T18:27:19.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa41622f700 1 --2- 172.21.15.146:0/2364320272 >> v2:172.21.15.154:3300/0 conn(0x7fa4100ca020 0x7fa4100ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa41622f700 1 --2- 172.21.15.146:0/2364320272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4100c57f0 0x7fa4100cb930 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa41622f700 1 -- 172.21.15.146:0/2364320272 >> 172.21.15.146:0/2364320272 conn(0x7fa410071d30 msgr2=0x7fa410072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:19.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa41622f700 1 -- 172.21.15.146:0/2364320272 shutdown_connections 2024-09-13T18:27:19.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa41622f700 1 -- 172.21.15.146:0/2364320272 wait complete. 2024-09-13T18:27:19.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa41622f700 1 Processor -- start 2024-09-13T18:27:19.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.234+0000 7fa41622f700 1 -- start start 2024-09-13T18:27:19.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa41622f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4100c57f0 0x7fa4101e28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa41622f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4100ca020 0x7fa4101f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa41622f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4100cad90 0x7fa410156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa41622f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4101f2f90 con 0x7fa4100c57f0 2024-09-13T18:27:19.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa41622f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4101e5140 con 0x7fa4100cad90 2024-09-13T18:27:19.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa41622f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4101e5420 con 0x7fa4100ca020 2024-09-13T18:27:19.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa40f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4100c57f0 0x7fa4101e28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa40f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4100c57f0 0x7fa4101e28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52596/0 (socket says 172.21.15.146:52596) 2024-09-13T18:27:19.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa40f7fe700 1 -- 172.21.15.146:0/2603366905 learned_addr learned my addr 172.21.15.146:0/2603366905 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:19.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa40ffff700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4100cad90 0x7fa410156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa40effd700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4100ca020 0x7fa4101f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa40f7fe700 1 -- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4100ca020 msgr2=0x7fa4101f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa40f7fe700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4100ca020 0x7fa4101f3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa40f7fe700 1 -- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4100cad90 msgr2=0x7fa410156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa40f7fe700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4100cad90 0x7fa410156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa40f7fe700 1 -- 172.21.15.146:0/2603366905 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa400009570 con 0x7fa4100c57f0 2024-09-13T18:27:19.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa40f7fe700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4100c57f0 0x7fa4101e28f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa410159970 tx=0x7fa3f8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:19.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa407fff700 1 -- 172.21.15.146:0/2603366905 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8004480 con 0x7fa4100c57f0 2024-09-13T18:27:19.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa407fff700 1 -- 172.21.15.146:0/2603366905 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f801f070 con 0x7fa4100c57f0 2024-09-13T18:27:19.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa41622f700 1 -- 172.21.15.146:0/2603366905 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f80094b0 con 0x7fa4100c57f0 2024-09-13T18:27:19.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa407fff700 1 -- 172.21.15.146:0/2603366905 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80038e0 con 0x7fa4100c57f0 2024-09-13T18:27:19.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.238+0000 7fa41622f700 1 -- 172.21.15.146:0/2603366905 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4101f2940 con 0x7fa4100c57f0 2024-09-13T18:27:19.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.242+0000 7fa407fff700 1 -- 172.21.15.146:0/2603366905 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3f8021030 con 0x7fa4100c57f0 2024-09-13T18:27:19.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.242+0000 7fa407fff700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa3f0066730 0x7fa3f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.242+0000 7fa407fff700 1 -- 172.21.15.146:0/2603366905 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa3f8085740 con 0x7fa4100c57f0 2024-09-13T18:27:19.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.242+0000 7fa41622f700 1 -- 172.21.15.146:0/2603366905 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4100c9730 con 0x7fa4100c57f0 2024-09-13T18:27:19.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.242+0000 7fa40effd700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa3f0066730 0x7fa3f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.242+0000 7fa40effd700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa3f0066730 0x7fa3f0068bf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa410157b50 tx=0x7fa4000092e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:19.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.246+0000 7fa407fff700 1 -- 172.21.15.146:0/2603366905 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa3f8052970 con 0x7fa4100c57f0 2024-09-13T18:27:19.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa41622f700 1 -- 172.21.15.146:0/2603366905 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa4100cb570 con 0x7fa4100c57f0 2024-09-13T18:27:19.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa407fff700 1 -- 172.21.15.146:0/2603366905 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa3f8055f90 con 0x7fa4100c57f0 2024-09-13T18:27:19.425 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:27:19.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa41622f700 1 -- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa3f0066730 msgr2=0x7fa3f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa41622f700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa3f0066730 0x7fa3f0068bf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa410157b50 tx=0x7fa4000092e0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa41622f700 1 -- 172.21.15.146:0/2603366905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4100c57f0 msgr2=0x7fa4101e28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa41622f700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4100c57f0 0x7fa4101e28f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa410159970 tx=0x7fa3f8004770 comp rx=0 tx=0).stop 2024-09-13T18:27:19.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa41622f700 1 -- 172.21.15.146:0/2603366905 shutdown_connections 2024-09-13T18:27:19.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa41622f700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa3f0066730 0x7fa3f0068bf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa41622f700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4100cad90 0x7fa410156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa41622f700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4100ca020 0x7fa4101f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa41622f700 1 --2- 172.21.15.146:0/2603366905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4100c57f0 0x7fa4101e28f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa41622f700 1 -- 172.21.15.146:0/2603366905 >> 172.21.15.146:0/2603366905 conn(0x7fa410071d30 msgr2=0x7fa4100c7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:19.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa41622f700 1 -- 172.21.15.146:0/2603366905 shutdown_connections 2024-09-13T18:27:19.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.418+0000 7fa41622f700 1 -- 172.21.15.146:0/2603366905 wait complete. 2024-09-13T18:27:19.440 INFO:tasks.mds_thrash:mds thrasher using random seed: 1726252039 2024-09-13T18:27:19.440 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:27:19.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 Processor -- start 2024-09-13T18:27:19.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 -- start start 2024-09-13T18:27:19.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f418806f830 0x7f41880661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f418806fc10 0x7f418806cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f418806d2c0 0x7f41880c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f418815d920 con 0x7f418806f830 2024-09-13T18:27:19.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41880c6640 con 0x7f418806d2c0 2024-09-13T18:27:19.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418815e8b0 con 0x7f418806fc10 2024-09-13T18:27:19.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418dbe2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f418806f830 0x7f41880661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418dbe2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f418806f830 0x7f41880661b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52602/0 (socket says 172.21.15.146:52602) 2024-09-13T18:27:19.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418dbe2700 1 -- 172.21.15.146:0/358614436 learned_addr learned my addr 172.21.15.146:0/358614436 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:19.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418d3e1700 1 --2- 172.21.15.146:0/358614436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f418806fc10 0x7f418806cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418e3e3700 1 --2- 172.21.15.146:0/358614436 >> v2:172.21.15.154:3300/0 conn(0x7f418806d2c0 0x7f41880c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418dbe2700 1 -- 172.21.15.146:0/358614436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f418806fc10 msgr2=0x7f418806cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:19.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418dbe2700 1 --2- 172.21.15.146:0/358614436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f418806fc10 0x7f418806cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418dbe2700 1 -- 172.21.15.146:0/358614436 >> v2:172.21.15.154:3300/0 conn(0x7f418806d2c0 msgr2=0x7f41880c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:19.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418dbe2700 1 --2- 172.21.15.146:0/358614436 >> v2:172.21.15.154:3300/0 conn(0x7f418806d2c0 0x7f41880c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418dbe2700 1 -- 172.21.15.146:0/358614436 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41880666f0 con 0x7f418806f830 2024-09-13T18:27:19.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418e3e3700 1 --2- 172.21.15.146:0/358614436 >> v2:172.21.15.154:3300/0 conn(0x7f418806d2c0 0x7f41880c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:19.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418dbe2700 1 --2- 172.21.15.146:0/358614436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f418806f830 0x7f41880661b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f41880c93f0 tx=0x7f4178009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf3b40a3f973e16 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:19.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418cbe0700 1 -- 172.21.15.146:0/358614436 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417800b8a0 con 0x7f418806f830 2024-09-13T18:27:19.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418cbe0700 1 -- 172.21.15.146:0/358614436 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417800ba00 con 0x7f418806f830 2024-09-13T18:27:19.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418cbe0700 1 -- 172.21.15.146:0/358614436 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417800bd90 con 0x7f418806f830 2024-09-13T18:27:19.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 -- 172.21.15.146:0/358614436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f418806f830 msgr2=0x7f41880661b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 --2- 172.21.15.146:0/358614436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f418806f830 0x7f41880661b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f41880c93f0 tx=0x7f4178009bf0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 -- 172.21.15.146:0/358614436 shutdown_connections 2024-09-13T18:27:19.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 --2- 172.21.15.146:0/358614436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f418806fc10 0x7f418806cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 --2- 172.21.15.146:0/358614436 >> v2:172.21.15.154:3300/0 conn(0x7f418806d2c0 0x7f41880c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 --2- 172.21.15.146:0/358614436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f418806f830 0x7f41880661b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 -- 172.21.15.146:0/358614436 >> 172.21.15.146:0/358614436 conn(0x7f4188066b00 msgr2=0x7f418806aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:19.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 -- 172.21.15.146:0/358614436 shutdown_connections 2024-09-13T18:27:19.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.546+0000 7f418ebe4700 1 -- 172.21.15.146:0/358614436 wait complete. 2024-09-13T18:27:19.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418ebe4700 1 Processor -- start 2024-09-13T18:27:19.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418ebe4700 1 -- start start 2024-09-13T18:27:19.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418ebe4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f418806d2c0 0x7f4188159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418ebe4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f418806f830 0x7f41881e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418ebe4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f418806fc10 0x7f41881e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418ebe4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4188159630 con 0x7f418806d2c0 2024-09-13T18:27:19.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418ebe4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41881e4710 con 0x7f418806f830 2024-09-13T18:27:19.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418ebe4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4188159f90 con 0x7f418806fc10 2024-09-13T18:27:19.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418dbe2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f418806d2c0 0x7f4188159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418dbe2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f418806d2c0 0x7f4188159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52618/0 (socket says 172.21.15.146:52618) 2024-09-13T18:27:19.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418dbe2700 1 -- 172.21.15.146:0/1935992817 learned_addr learned my addr 172.21.15.146:0/1935992817 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:19.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418e3e3700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f418806fc10 0x7f41881e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418d3e1700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f418806f830 0x7f41881e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418dbe2700 1 -- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f418806fc10 msgr2=0x7f41881e90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418dbe2700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f418806fc10 0x7f41881e90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418dbe2700 1 -- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f418806f830 msgr2=0x7f41881e8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418dbe2700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f418806f830 0x7f41881e8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418dbe2700 1 -- 172.21.15.146:0/1935992817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4178009570 con 0x7f418806d2c0 2024-09-13T18:27:19.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418d3e1700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f418806f830 0x7f41881e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:19.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f418dbe2700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f418806d2c0 0x7f4188159a50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4178000c00 tx=0x7f4178004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:19.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.550+0000 7f417e7fc700 1 -- 172.21.15.146:0/1935992817 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41780038c0 con 0x7f418806d2c0 2024-09-13T18:27:19.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.554+0000 7f417e7fc700 1 -- 172.21.15.146:0/1935992817 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4178003a20 con 0x7f418806d2c0 2024-09-13T18:27:19.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.554+0000 7f418ebe4700 1 -- 172.21.15.146:0/1935992817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41881e3e70 con 0x7f418806d2c0 2024-09-13T18:27:19.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.554+0000 7f417e7fc700 1 -- 172.21.15.146:0/1935992817 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178003db0 con 0x7f418806d2c0 2024-09-13T18:27:19.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.554+0000 7f418ebe4700 1 -- 172.21.15.146:0/1935992817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41881e4430 con 0x7f418806d2c0 2024-09-13T18:27:19.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.554+0000 7f418ebe4700 1 -- 172.21.15.146:0/1935992817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4188159770 con 0x7f418806d2c0 2024-09-13T18:27:19.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.554+0000 7f417e7fc700 1 -- 172.21.15.146:0/1935992817 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4178011420 con 0x7f418806d2c0 2024-09-13T18:27:19.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.554+0000 7f417e7fc700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f416c066730 0x7f416c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.554+0000 7f417e7fc700 1 -- 172.21.15.146:0/1935992817 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4178020030 con 0x7f418806d2c0 2024-09-13T18:27:19.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.554+0000 7f418d3e1700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f416c066730 0x7f416c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.558+0000 7f418d3e1700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f416c066730 0x7f416c068bf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f41880c2350 tx=0x7f4184008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:19.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.562+0000 7f417e7fc700 1 -- 172.21.15.146:0/1935992817 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4178052b20 con 0x7f418806d2c0 2024-09-13T18:27:19.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.734+0000 7f418ebe4700 1 -- 172.21.15.146:0/1935992817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41880c5f40 con 0x7f418806d2c0 2024-09-13T18:27:19.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.738+0000 7f417e7fc700 1 -- 172.21.15.146:0/1935992817 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f417802a090 con 0x7f418806d2c0 2024-09-13T18:27:19.747 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:19.747 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:26:36.757374+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:27:19.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.742+0000 7f418ebe4700 1 -- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f416c066730 msgr2=0x7f416c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.742+0000 7f418ebe4700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f416c066730 0x7f416c068bf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f41880c2350 tx=0x7f4184008040 comp rx=0 tx=0).stop 2024-09-13T18:27:19.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.742+0000 7f418ebe4700 1 -- 172.21.15.146:0/1935992817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f418806d2c0 msgr2=0x7f4188159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.742+0000 7f418ebe4700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f418806d2c0 0x7f4188159a50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4178000c00 tx=0x7f4178004770 comp rx=0 tx=0).stop 2024-09-13T18:27:19.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.742+0000 7f418ebe4700 1 -- 172.21.15.146:0/1935992817 shutdown_connections 2024-09-13T18:27:19.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.742+0000 7f418ebe4700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f416c066730 0x7f416c068bf0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.742+0000 7f418ebe4700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f418806fc10 0x7f41881e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.742+0000 7f418ebe4700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f418806f830 0x7f41881e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.742+0000 7f418ebe4700 1 --2- 172.21.15.146:0/1935992817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f418806d2c0 0x7f4188159a50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.742+0000 7f418ebe4700 1 -- 172.21.15.146:0/1935992817 >> 172.21.15.146:0/1935992817 conn(0x7f4188066b00 msgr2=0x7f4188073c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:19.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.742+0000 7f418ebe4700 1 -- 172.21.15.146:0/1935992817 shutdown_connections 2024-09-13T18:27:19.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.742+0000 7f418ebe4700 1 -- 172.21.15.146:0/1935992817 wait complete. 2024-09-13T18:27:19.762 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:26:24.867515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:26:30.782269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:26:31.323277+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:26:34.393386+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:27:19.762 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:27:19.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.874+0000 7febddd17700 1 Processor -- start 2024-09-13T18:27:19.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.874+0000 7febddd17700 1 -- start start 2024-09-13T18:27:19.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.874+0000 7febddd17700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7febd80caed0 0x7febd806eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.874+0000 7febddd17700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd806fef0 con 0x7febd806c500 2024-09-13T18:27:19.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.874+0000 7febddd17700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd8070790 con 0x7febd80cb2b0 2024-09-13T18:27:19.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.874+0000 7febddd17700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd80aa770 con 0x7febd80caed0 2024-09-13T18:27:19.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.874+0000 7febdd516700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7febd806c500 0x7febd80a9500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56732/0 (socket says 172.21.15.146:56732) 2024-09-13T18:27:19.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.874+0000 7febdd516700 1 -- 172.21.15.146:0/3949707571 learned_addr learned my addr 172.21.15.146:0/3949707571 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:19.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.874+0000 7febdcd15700 1 --2- 172.21.15.146:0/3949707571 >> v2:172.21.15.154:3300/0 conn(0x7febd80caed0 0x7febd806eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.874+0000 7febd77fe700 1 -- 172.21.15.146:0/3949707571 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99078753 0 0) 0x7febd806fef0 con 0x7febd806c500 2024-09-13T18:27:19.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.874+0000 7febd77fe700 1 -- 172.21.15.146:0/3949707571 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febc00035d0 con 0x7febd806c500 2024-09-13T18:27:19.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febdcd15700 1 -- 172.21.15.146:0/3949707571 >> v1:172.21.15.154:6789/0 conn(0x7febd80cb2b0 legacy=0x7febd8072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febdcd15700 1 -- 172.21.15.146:0/3949707571 >> v1:172.21.15.146:6789/0 conn(0x7febd806c500 legacy=0x7febd80a9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febdcd15700 1 -- 172.21.15.146:0/3949707571 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd806f5c0 con 0x7febd80caed0 2024-09-13T18:27:19.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febdcd15700 1 --2- 172.21.15.146:0/3949707571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd80caed0 0x7febd806eec0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7febcc009850 tx=0x7febcc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=287b0cb3ab123335 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:19.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febd77fe700 1 -- 172.21.15.146:0/3949707571 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc011830 con 0x7febd80caed0 2024-09-13T18:27:19.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febd77fe700 1 -- 172.21.15.146:0/3949707571 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc011990 con 0x7febd80caed0 2024-09-13T18:27:19.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febd77fe700 1 -- 172.21.15.146:0/3949707571 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc011d40 con 0x7febd80caed0 2024-09-13T18:27:19.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febddd17700 1 -- 172.21.15.146:0/3949707571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd80caed0 msgr2=0x7febd806eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febddd17700 1 --2- 172.21.15.146:0/3949707571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd80caed0 0x7febd806eec0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7febcc009850 tx=0x7febcc009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:19.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febddd17700 1 -- 172.21.15.146:0/3949707571 shutdown_connections 2024-09-13T18:27:19.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febddd17700 1 --2- 172.21.15.146:0/3949707571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd80caed0 0x7febd806eec0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febddd17700 1 -- 172.21.15.146:0/3949707571 >> 172.21.15.146:0/3949707571 conn(0x7febd8066090 msgr2=0x7febd8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:19.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febddd17700 1 -- 172.21.15.146:0/3949707571 shutdown_connections 2024-09-13T18:27:19.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febddd17700 1 -- 172.21.15.146:0/3949707571 wait complete. 2024-09-13T18:27:19.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febddd17700 1 Processor -- start 2024-09-13T18:27:19.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.878+0000 7febddd17700 1 -- start start 2024-09-13T18:27:19.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febddd17700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd806c500 0x7febd81deb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febddd17700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd80caed0 0x7febd81f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febddd17700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd80cb2b0 0x7febd81ddf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febddd17700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd80c81a0 con 0x7febd80cb2b0 2024-09-13T18:27:19.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febddd17700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd80c7f00 con 0x7febd80caed0 2024-09-13T18:27:19.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febddd17700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd81ed3e0 con 0x7febd806c500 2024-09-13T18:27:19.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febdd516700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd80cb2b0 0x7febd81ddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febdd516700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd80cb2b0 0x7febd81ddf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52630/0 (socket says 172.21.15.146:52630) 2024-09-13T18:27:19.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febdd516700 1 -- 172.21.15.146:0/1587410871 learned_addr learned my addr 172.21.15.146:0/1587410871 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:19.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febdcd15700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd806c500 0x7febd81deb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febd7fff700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd80caed0 0x7febd81f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febdcd15700 1 -- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd80caed0 msgr2=0x7febd81f08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febdcd15700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd80caed0 0x7febd81f08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febdcd15700 1 -- 172.21.15.146:0/1587410871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd80cb2b0 msgr2=0x7febd81ddf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:19.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febdcd15700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd80cb2b0 0x7febd81ddf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:19.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febdcd15700 1 -- 172.21.15.146:0/1587410871 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc0094b0 con 0x7febd806c500 2024-09-13T18:27:19.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febdd516700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd80cb2b0 0x7febd81ddf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:19.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febdcd15700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd806c500 0x7febd81deb70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7febcc016000 tx=0x7febcc004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:19.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febd57fa700 1 -- 172.21.15.146:0/1587410871 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc01a990 con 0x7febd806c500 2024-09-13T18:27:19.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febd57fa700 1 -- 172.21.15.146:0/1587410871 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc01f070 con 0x7febd806c500 2024-09-13T18:27:19.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febddd17700 1 -- 172.21.15.146:0/1587410871 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd81ed6c0 con 0x7febd806c500 2024-09-13T18:27:19.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febd57fa700 1 -- 172.21.15.146:0/1587410871 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc01ad10 con 0x7febd806c500 2024-09-13T18:27:19.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.882+0000 7febddd17700 1 -- 172.21.15.146:0/1587410871 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd81f00c0 con 0x7febd806c500 2024-09-13T18:27:19.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.886+0000 7febddd17700 1 -- 172.21.15.146:0/1587410871 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd814de20 con 0x7febd806c500 2024-09-13T18:27:19.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.886+0000 7febd57fa700 1 -- 172.21.15.146:0/1587410871 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7febcc01b430 con 0x7febd806c500 2024-09-13T18:27:19.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.886+0000 7febd57fa700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7febc4066730 0x7febc4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:19.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.886+0000 7febd57fa700 1 -- 172.21.15.146:0/1587410871 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7febcc085810 con 0x7febd806c500 2024-09-13T18:27:19.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.886+0000 7febd7fff700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7febc4066730 0x7febc4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:19.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.886+0000 7febd7fff700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7febc4066730 0x7febc4068bf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7febd814ebd0 tx=0x7febc8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:19.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:19.894+0000 7febd57fa700 1 -- 172.21.15.146:0/1587410871 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7febcc052a00 con 0x7febd806c500 2024-09-13T18:27:20.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.066+0000 7febddd17700 1 -- 172.21.15.146:0/1587410871 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febd814db40 con 0x7febd806c500 2024-09-13T18:27:20.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.070+0000 7febd57fa700 1 -- 172.21.15.146:0/1587410871 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7febcc056020 con 0x7febd806c500 2024-09-13T18:27:20.077 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-09-13T18:27:20.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.074+0000 7febddd17700 1 -- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7febc4066730 msgr2=0x7febc4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.074+0000 7febddd17700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7febc4066730 0x7febc4068bf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7febd814ebd0 tx=0x7febc8009040 comp rx=0 tx=0).stop 2024-09-13T18:27:20.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.074+0000 7febddd17700 1 -- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd806c500 msgr2=0x7febd81deb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.074+0000 7febddd17700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd806c500 0x7febd81deb70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7febcc016000 tx=0x7febcc004770 comp rx=0 tx=0).stop 2024-09-13T18:27:20.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.074+0000 7febddd17700 1 -- 172.21.15.146:0/1587410871 shutdown_connections 2024-09-13T18:27:20.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.074+0000 7febddd17700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7febc4066730 0x7febc4068bf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.074+0000 7febddd17700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd80cb2b0 0x7febd81ddf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.074+0000 7febddd17700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd80caed0 0x7febd81f08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.074+0000 7febddd17700 1 --2- 172.21.15.146:0/1587410871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd806c500 0x7febd81deb70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.074+0000 7febddd17700 1 -- 172.21.15.146:0/1587410871 >> 172.21.15.146:0/1587410871 conn(0x7febd8066090 msgr2=0x7febd80c6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:20.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.074+0000 7febddd17700 1 -- 172.21.15.146:0/1587410871 shutdown_connections 2024-09-13T18:27:20.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.074+0000 7febddd17700 1 -- 172.21.15.146:0/1587410871 wait complete. 2024-09-13T18:27:20.097 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:27:20.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.214+0000 7f0024dbd700 1 Processor -- start 2024-09-13T18:27:20.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.214+0000 7f0024dbd700 1 -- start start 2024-09-13T18:27:20.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.214+0000 7f0024dbd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00200cf580 0x7f00200cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.214+0000 7f0024dbd700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00200cff70 con 0x7f00201539e0 2024-09-13T18:27:20.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.214+0000 7f0024dbd700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00200c5e50 con 0x7f002015d960 2024-09-13T18:27:20.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.214+0000 7f0024dbd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00200cc3a0 con 0x7f00200cf580 2024-09-13T18:27:20.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.214+0000 7f001f7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f00201539e0 0x7f00200b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56746/0 (socket says 172.21.15.146:56746) 2024-09-13T18:27:20.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f001f7fe700 1 -- 172.21.15.146:0/4202895565 learned_addr learned my addr 172.21.15.146:0/4202895565 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:20.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f001ffff700 1 --2- 172.21.15.146:0/4202895565 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00200cf580 0x7f00200cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f001effd700 1 -- 172.21.15.146:0/4202895565 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732631787 0 0) 0x7f00200cff70 con 0x7f00201539e0 2024-09-13T18:27:20.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f001effd700 1 -- 172.21.15.146:0/4202895565 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f000c0035d0 con 0x7f00201539e0 2024-09-13T18:27:20.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f001effd700 1 -- 172.21.15.146:0/4202895565 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223453734 0 0) 0x7f0010001110 con 0x7f002015d960 2024-09-13T18:27:20.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f001ffff700 1 -- 172.21.15.146:0/4202895565 >> v1:172.21.15.154:6789/0 conn(0x7f002015d960 legacy=0x7f00200c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f001ffff700 1 -- 172.21.15.146:0/4202895565 >> v1:172.21.15.146:6789/0 conn(0x7f00201539e0 legacy=0x7f00200b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f001ffff700 1 -- 172.21.15.146:0/4202895565 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00200cb480 con 0x7f00200cf580 2024-09-13T18:27:20.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f001ffff700 1 --2- 172.21.15.146:0/4202895565 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00200cf580 0x7f00200cfa30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0018009df0 tx=0x7f001800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=565d57863ac67f91 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:20.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f001effd700 1 -- 172.21.15.146:0/4202895565 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001800e070 con 0x7f00200cf580 2024-09-13T18:27:20.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f001effd700 1 -- 172.21.15.146:0/4202895565 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001800bae0 con 0x7f00200cf580 2024-09-13T18:27:20.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f001effd700 1 -- 172.21.15.146:0/4202895565 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001800be50 con 0x7f00200cf580 2024-09-13T18:27:20.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f0024dbd700 1 -- 172.21.15.146:0/4202895565 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00200cf580 msgr2=0x7f00200cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f0024dbd700 1 --2- 172.21.15.146:0/4202895565 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00200cf580 0x7f00200cfa30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0018009df0 tx=0x7f001800b330 comp rx=0 tx=0).stop 2024-09-13T18:27:20.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f0024dbd700 1 -- 172.21.15.146:0/4202895565 shutdown_connections 2024-09-13T18:27:20.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f0024dbd700 1 --2- 172.21.15.146:0/4202895565 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00200cf580 0x7f00200cfa30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f0024dbd700 1 -- 172.21.15.146:0/4202895565 >> 172.21.15.146:0/4202895565 conn(0x7f0020071ec0 msgr2=0x7f00200722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:20.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f0024dbd700 1 -- 172.21.15.146:0/4202895565 shutdown_connections 2024-09-13T18:27:20.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.218+0000 7f0024dbd700 1 -- 172.21.15.146:0/4202895565 wait complete. 2024-09-13T18:27:20.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f0024dbd700 1 Processor -- start 2024-09-13T18:27:20.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f0024dbd700 1 -- start start 2024-09-13T18:27:20.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f0024dbd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00200cf580 0x7f00200c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f0024dbd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00201539e0 0x7f00201efee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f0024dbd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f002015d960 0x7f002014c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f0024dbd700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00200c8660 con 0x7f002015d960 2024-09-13T18:27:20.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f0024dbd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00200c6c10 con 0x7f00200cf580 2024-09-13T18:27:20.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f0024dbd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002014c620 con 0x7f00201539e0 2024-09-13T18:27:20.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f002015d960 0x7f002014c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f002015d960 0x7f002014c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52646/0 (socket says 172.21.15.146:52646) 2024-09-13T18:27:20.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001ffff700 1 -- 172.21.15.146:0/2742519700 learned_addr learned my addr 172.21.15.146:0/2742519700 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:20.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f0017fff700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00201539e0 0x7f00201efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001f7fe700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00200cf580 0x7f00200c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001ffff700 1 -- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00201539e0 msgr2=0x7f00201efee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001ffff700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00201539e0 0x7f00201efee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001ffff700 1 -- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00200cf580 msgr2=0x7f00200c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001ffff700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00200cf580 0x7f00200c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001ffff700 1 -- 172.21.15.146:0/2742519700 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0018009a70 con 0x7f002015d960 2024-09-13T18:27:20.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001f7fe700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00200cf580 0x7f00200c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:20.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001ffff700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f002015d960 0x7f002014c0e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0020067de0 tx=0x7f0018004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:20.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001cff9700 1 -- 172.21.15.146:0/2742519700 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018012c00 con 0x7f002015d960 2024-09-13T18:27:20.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001cff9700 1 -- 172.21.15.146:0/2742519700 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0018012d60 con 0x7f002015d960 2024-09-13T18:27:20.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f0024dbd700 1 -- 172.21.15.146:0/2742519700 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00200c8100 con 0x7f002015d960 2024-09-13T18:27:20.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f001cff9700 1 -- 172.21.15.146:0/2742519700 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018025660 con 0x7f002015d960 2024-09-13T18:27:20.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.222+0000 7f0024dbd700 1 -- 172.21.15.146:0/2742519700 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00200c6550 con 0x7f002015d960 2024-09-13T18:27:20.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.226+0000 7f0024dbd700 1 -- 172.21.15.146:0/2742519700 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00200c63b0 con 0x7f002015d960 2024-09-13T18:27:20.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.226+0000 7f001cff9700 1 -- 172.21.15.146:0/2742519700 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0018023070 con 0x7f002015d960 2024-09-13T18:27:20.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.226+0000 7f001cff9700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0000066730 0x7f0000068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.226+0000 7f001cff9700 1 -- 172.21.15.146:0/2742519700 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0018085720 con 0x7f002015d960 2024-09-13T18:27:20.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.226+0000 7f001f7fe700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0000066730 0x7f0000068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.226+0000 7f001f7fe700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0000066730 0x7f0000068bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0008005930 tx=0x7f0008009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:20.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.230+0000 7f001cff9700 1 -- 172.21.15.146:0/2742519700 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f00200c63b0 con 0x7f002015d960 2024-09-13T18:27:20.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.402+0000 7f0024dbd700 1 -- 172.21.15.146:0/2742519700 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00200c6d50 con 0x7f002015d960 2024-09-13T18:27:20.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.406+0000 7f001cff9700 1 -- 172.21.15.146:0/2742519700 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f00200c6d50 con 0x7f002015d960 2024-09-13T18:27:20.412 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:27:20.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.406+0000 7f0024dbd700 1 -- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0000066730 msgr2=0x7f0000068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.406+0000 7f0024dbd700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0000066730 0x7f0000068bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0008005930 tx=0x7f0008009040 comp rx=0 tx=0).stop 2024-09-13T18:27:20.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.406+0000 7f0024dbd700 1 -- 172.21.15.146:0/2742519700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f002015d960 msgr2=0x7f002014c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.406+0000 7f0024dbd700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f002015d960 0x7f002014c0e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0020067de0 tx=0x7f0018004af0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.406+0000 7f0024dbd700 1 -- 172.21.15.146:0/2742519700 shutdown_connections 2024-09-13T18:27:20.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.406+0000 7f0024dbd700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0000066730 0x7f0000068bf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.406+0000 7f0024dbd700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f002015d960 0x7f002014c0e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.406+0000 7f0024dbd700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00201539e0 0x7f00201efee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.406+0000 7f0024dbd700 1 --2- 172.21.15.146:0/2742519700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00200cf580 0x7f00200c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.406+0000 7f0024dbd700 1 -- 172.21.15.146:0/2742519700 >> 172.21.15.146:0/2742519700 conn(0x7f0020071ec0 msgr2=0x7f00200b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:20.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.406+0000 7f0024dbd700 1 -- 172.21.15.146:0/2742519700 shutdown_connections 2024-09-13T18:27:20.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.406+0000 7f0024dbd700 1 -- 172.21.15.146:0/2742519700 wait complete. 2024-09-13T18:27:20.431 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:27:20.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.538+0000 7f11f70f2700 1 Processor -- start 2024-09-13T18:27:20.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.538+0000 7f11f70f2700 1 -- start start 2024-09-13T18:27:20.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f70f2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11f015d960 0x7f11f00c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f70f2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f00c5e50 con 0x7f11f01539e0 2024-09-13T18:27:20.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f70f2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f00cb480 con 0x7f11f00ca500 2024-09-13T18:27:20.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f70f2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f00cc3a0 con 0x7f11f015d960 2024-09-13T18:27:20.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f60f0700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f11f01539e0 0x7f11f00b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56756/0 (socket says 172.21.15.146:56756) 2024-09-13T18:27:20.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f60f0700 1 -- 172.21.15.146:0/1938206352 learned_addr learned my addr 172.21.15.146:0/1938206352 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:20.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f58ef700 1 --2- 172.21.15.146:0/1938206352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11f015d960 0x7f11f00c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443534100 0 0) 0x7f11ec001110 con 0x7f11f01539e0 2024-09-13T18:27:20.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11dc0035d0 con 0x7f11f01539e0 2024-09-13T18:27:20.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294842415 0 0) 0x7f11f00cb480 con 0x7f11f00ca500 2024-09-13T18:27:20.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11ec001110 con 0x7f11f00ca500 2024-09-13T18:27:20.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2257769208 0 0) 0x7f11dc0035d0 con 0x7f11f01539e0 2024-09-13T18:27:20.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11f015d960 msgr2=0x7f11f00c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:20.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 --2- 172.21.15.146:0/1938206352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11f015d960 0x7f11f00c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 >> v1:172.21.15.154:6789/0 conn(0x7f11f00ca500 legacy=0x7f11f00cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f00cb720 con 0x7f11f01539e0 2024-09-13T18:27:20.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f11ec001800 con 0x7f11f01539e0 2024-09-13T18:27:20.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 >> v1:172.21.15.146:6789/0 conn(0x7f11f01539e0 legacy=0x7f11f00b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 --2- 172.21.15.146:0/1938206352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11dc003c50 0x7f11dc004050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 --2- 172.21.15.146:0/1938206352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11dc004870 0x7f11dc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 --2- 172.21.15.146:0/1938206352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11dc005230 0x7f11dc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f00cc3a0 con 0x7f11dc005230 2024-09-13T18:27:20.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc00c080 con 0x7f11dc003c50 2024-09-13T18:27:20.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc00c240 con 0x7f11dc004870 2024-09-13T18:27:20.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f58ef700 1 --2- 172.21.15.146:0/1938206352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11f015d960 0x7f11f00c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:20.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f68f1700 1 --2- 172.21.15.146:0/1938206352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11dc005230 0x7f11dc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f60f0700 1 --2- 172.21.15.146:0/1938206352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11dc003c50 0x7f11dc004050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f58ef700 1 --2- 172.21.15.146:0/1938206352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11dc004870 0x7f11dc004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f68f1700 1 -- 172.21.15.146:0/1938206352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11dc004870 msgr2=0x7f11dc004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:20.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f68f1700 1 --2- 172.21.15.146:0/1938206352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11dc004870 0x7f11dc004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f68f1700 1 -- 172.21.15.146:0/1938206352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11dc003c50 msgr2=0x7f11dc004050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:20.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f68f1700 1 --2- 172.21.15.146:0/1938206352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11dc003c50 0x7f11dc004050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f68f1700 1 -- 172.21.15.146:0/1938206352 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc0035d0 con 0x7f11dc005230 2024-09-13T18:27:20.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f68f1700 1 --2- 172.21.15.146:0/1938206352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11dc005230 0x7f11dc00ba30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f11e800c320 tx=0x7f11dc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab0b8b3e59d5350 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:20.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8001f20 con 0x7f11dc005230 2024-09-13T18:27:20.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e8003a20 con 0x7f11dc005230 2024-09-13T18:27:20.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.542+0000 7f11f50ee700 1 -- 172.21.15.146:0/1938206352 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800cea0 con 0x7f11dc005230 2024-09-13T18:27:20.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.546+0000 7f11f70f2700 1 -- 172.21.15.146:0/1938206352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11dc005230 msgr2=0x7f11dc00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.546+0000 7f11f70f2700 1 --2- 172.21.15.146:0/1938206352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11dc005230 0x7f11dc00ba30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f11e800c320 tx=0x7f11dc0058a0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.546+0000 7f11f68f1700 1 -- 172.21.15.146:0/1938206352 reap_dead start 2024-09-13T18:27:20.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.546+0000 7f11f70f2700 1 -- 172.21.15.146:0/1938206352 shutdown_connections 2024-09-13T18:27:20.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.546+0000 7f11f70f2700 1 -- 172.21.15.146:0/1938206352 >> 172.21.15.146:0/1938206352 conn(0x7f11f0071ec0 msgr2=0x7f11f00722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:20.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.546+0000 7f11f70f2700 1 -- 172.21.15.146:0/1938206352 shutdown_connections 2024-09-13T18:27:20.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.546+0000 7f11f70f2700 1 -- 172.21.15.146:0/1938206352 wait complete. 2024-09-13T18:27:20.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.546+0000 7f11f70f2700 1 Processor -- start 2024-09-13T18:27:20.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.546+0000 7f11f70f2700 1 -- start start 2024-09-13T18:27:20.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f70f2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11dc005230 0x7f11f00c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f70f2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f01ef7b0 0x7f11f0147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f70f2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11f0148400 0x7f11f01f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f70f2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f01472b0 con 0x7f11f0148400 2024-09-13T18:27:20.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f70f2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f01efcf0 con 0x7f11f01ef7b0 2024-09-13T18:27:20.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f70f2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f0148880 con 0x7f11dc005230 2024-09-13T18:27:20.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f68f1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11f0148400 0x7f11f01f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f58ef700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f01ef7b0 0x7f11f0147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f60f0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11dc005230 0x7f11f00c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f58ef700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f01ef7b0 0x7f11f0147ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:38214/0 (socket says 172.21.15.146:38214) 2024-09-13T18:27:20.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f58ef700 1 -- 172.21.15.146:0/3852978073 learned_addr learned my addr 172.21.15.146:0/3852978073 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:20.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f68f1700 1 -- 172.21.15.146:0/3852978073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11dc005230 msgr2=0x7f11f00c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f68f1700 1 --2- 172.21.15.146:0/3852978073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11dc005230 0x7f11f00c41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f68f1700 1 -- 172.21.15.146:0/3852978073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f01ef7b0 msgr2=0x7f11f0147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f68f1700 1 --2- 172.21.15.146:0/3852978073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f01ef7b0 0x7f11f0147ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f68f1700 1 -- 172.21.15.146:0/3852978073 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f014b8f0 con 0x7f11f0148400 2024-09-13T18:27:20.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f58ef700 1 --2- 172.21.15.146:0/3852978073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f01ef7b0 0x7f11f0147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:20.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f68f1700 1 --2- 172.21.15.146:0/3852978073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11f0148400 0x7f11f01f22a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f11e800c2f0 tx=0x7f11e8001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:20.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11e6ffd700 1 -- 172.21.15.146:0/3852978073 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8003da0 con 0x7f11f0148400 2024-09-13T18:27:20.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11e6ffd700 1 -- 172.21.15.146:0/3852978073 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e80050d0 con 0x7f11f0148400 2024-09-13T18:27:20.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f70f2700 1 -- 172.21.15.146:0/3852978073 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f01466a0 con 0x7f11f0148400 2024-09-13T18:27:20.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11e6ffd700 1 -- 172.21.15.146:0/3852978073 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8005480 con 0x7f11f0148400 2024-09-13T18:27:20.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f70f2700 1 -- 172.21.15.146:0/3852978073 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f0146c60 con 0x7f11f0148400 2024-09-13T18:27:20.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.550+0000 7f11f70f2700 1 -- 172.21.15.146:0/3852978073 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f00cc7c0 con 0x7f11f0148400 2024-09-13T18:27:20.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.554+0000 7f11e6ffd700 1 -- 172.21.15.146:0/3852978073 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f11e8019070 con 0x7f11f0148400 2024-09-13T18:27:20.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.554+0000 7f11e6ffd700 1 --2- 172.21.15.146:0/3852978073 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f11d4066760 0x7f11d4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.554+0000 7f11e6ffd700 1 -- 172.21.15.146:0/3852978073 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f11e8084fc0 con 0x7f11f0148400 2024-09-13T18:27:20.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.554+0000 7f11f60f0700 1 --2- 172.21.15.146:0/3852978073 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f11d4066760 0x7f11d4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.554+0000 7f11f60f0700 1 --2- 172.21.15.146:0/3852978073 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f11d4066760 0x7f11d4068c20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f11ec000c00 tx=0x7f11ec007a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:20.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.562+0000 7f11e6ffd700 1 -- 172.21.15.146:0/3852978073 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f11e8052030 con 0x7f11f0148400 2024-09-13T18:27:20.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.730+0000 7f11f70f2700 1 -- 172.21.15.146:0/3852978073 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11f0147b70 con 0x7f11f0148400 2024-09-13T18:27:20.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.730+0000 7f11e6ffd700 1 -- 172.21.15.146:0/3852978073 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11e8002080 con 0x7f11f0148400 2024-09-13T18:27:20.740 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-09-13T18:27:20.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.734+0000 7f11f70f2700 1 -- 172.21.15.146:0/3852978073 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f11d4066760 msgr2=0x7f11d4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.734+0000 7f11f70f2700 1 --2- 172.21.15.146:0/3852978073 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f11d4066760 0x7f11d4068c20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f11ec000c00 tx=0x7f11ec007a50 comp rx=0 tx=0).stop 2024-09-13T18:27:20.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.734+0000 7f11f70f2700 1 -- 172.21.15.146:0/3852978073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11f0148400 msgr2=0x7f11f01f22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.734+0000 7f11f70f2700 1 --2- 172.21.15.146:0/3852978073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11f0148400 0x7f11f01f22a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f11e800c2f0 tx=0x7f11e8001650 comp rx=0 tx=0).stop 2024-09-13T18:27:20.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.734+0000 7f11f70f2700 1 -- 172.21.15.146:0/3852978073 shutdown_connections 2024-09-13T18:27:20.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.734+0000 7f11f70f2700 1 --2- 172.21.15.146:0/3852978073 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f11d4066760 0x7f11d4068c20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.734+0000 7f11f70f2700 1 --2- 172.21.15.146:0/3852978073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11f0148400 0x7f11f01f22a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.734+0000 7f11f70f2700 1 --2- 172.21.15.146:0/3852978073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f01ef7b0 0x7f11f0147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.734+0000 7f11f70f2700 1 --2- 172.21.15.146:0/3852978073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11dc005230 0x7f11f00c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.734+0000 7f11f70f2700 1 -- 172.21.15.146:0/3852978073 >> 172.21.15.146:0/3852978073 conn(0x7f11f0071ec0 msgr2=0x7f11f00b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:20.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.734+0000 7f11f70f2700 1 -- 172.21.15.146:0/3852978073 shutdown_connections 2024-09-13T18:27:20.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.734+0000 7f11f70f2700 1 -- 172.21.15.146:0/3852978073 wait complete. 2024-09-13T18:27:20.754 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:27:20.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d988b2700 1 Processor -- start 2024-09-13T18:27:20.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d988b2700 1 -- start start 2024-09-13T18:27:20.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d988b2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3d940c4ec0 0x7f3d94158790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d988b2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d9406f990 0x7f3d94151090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d988b2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d94066710 con 0x7f3d9406fd90 2024-09-13T18:27:20.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d988b2700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9406ea00 con 0x7f3d940c4ec0 2024-09-13T18:27:20.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d988b2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9406ee90 con 0x7f3d9406f990 2024-09-13T18:27:20.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d93fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f3d9406fd90 0x7f3d941572e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56770/0 (socket says 172.21.15.146:56770) 2024-09-13T18:27:20.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d93fff700 1 -- 172.21.15.146:0/519595622 learned_addr learned my addr 172.21.15.146:0/519595622 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d937fe700 1 --2- 172.21.15.146:0/519595622 >> v2:172.21.15.154:3300/0 conn(0x7f3d940c4ec0 0x7f3d94158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d92ffd700 1 --2- 172.21.15.146:0/519595622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d9406f990 0x7f3d94151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d927fc700 1 -- 172.21.15.146:0/519595622 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396880181 0 0) 0x7f3d94066710 con 0x7f3d9406fd90 2024-09-13T18:27:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d927fc700 1 -- 172.21.15.146:0/519595622 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d7c0035d0 con 0x7f3d9406fd90 2024-09-13T18:27:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.862+0000 7f3d927fc700 1 -- 172.21.15.146:0/519595622 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 853319520 0 0) 0x7f3d7c0035d0 con 0x7f3d9406fd90 2024-09-13T18:27:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d937fe700 1 -- 172.21.15.146:0/519595622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d9406f990 msgr2=0x7f3d94151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d937fe700 1 --2- 172.21.15.146:0/519595622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d9406f990 0x7f3d94151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d937fe700 1 -- 172.21.15.146:0/519595622 >> v1:172.21.15.146:6789/0 conn(0x7f3d9406fd90 legacy=0x7f3d941572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d937fe700 1 -- 172.21.15.146:0/519595622 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d94066450 con 0x7f3d940c4ec0 2024-09-13T18:27:20.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d937fe700 1 --2- 172.21.15.146:0/519595622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d940c4ec0 0x7f3d94158790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009850 tx=0x7f3d84009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd5723f7729b6f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:20.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d927fc700 1 -- 172.21.15.146:0/519595622 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c0019f0 con 0x7f3d940c4ec0 2024-09-13T18:27:20.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d927fc700 1 -- 172.21.15.146:0/519595622 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d84004030 con 0x7f3d940c4ec0 2024-09-13T18:27:20.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d927fc700 1 -- 172.21.15.146:0/519595622 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d840043e0 con 0x7f3d940c4ec0 2024-09-13T18:27:20.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d988b2700 1 -- 172.21.15.146:0/519595622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d940c4ec0 msgr2=0x7f3d94158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d988b2700 1 --2- 172.21.15.146:0/519595622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d940c4ec0 0x7f3d94158790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009850 tx=0x7f3d84009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:20.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d988b2700 1 -- 172.21.15.146:0/519595622 shutdown_connections 2024-09-13T18:27:20.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d988b2700 1 --2- 172.21.15.146:0/519595622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d9406f990 0x7f3d94151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d988b2700 1 --2- 172.21.15.146:0/519595622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d940c4ec0 0x7f3d94158790 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d988b2700 1 -- 172.21.15.146:0/519595622 >> 172.21.15.146:0/519595622 conn(0x7f3d94066b00 msgr2=0x7f3d9406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:20.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d988b2700 1 -- 172.21.15.146:0/519595622 shutdown_connections 2024-09-13T18:27:20.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.866+0000 7f3d988b2700 1 -- 172.21.15.146:0/519595622 wait complete. 2024-09-13T18:27:20.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d988b2700 1 Processor -- start 2024-09-13T18:27:20.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d988b2700 1 -- start start 2024-09-13T18:27:20.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d988b2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d9406f990 0x7f3d940ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d988b2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d9406fd90 0x7f3d940ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d988b2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d940c4ec0 0x7f3d94140c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d988b2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9414a910 con 0x7f3d9406fd90 2024-09-13T18:27:20.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d988b2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d94146fc0 con 0x7f3d9406f990 2024-09-13T18:27:20.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d988b2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d94147420 con 0x7f3d940c4ec0 2024-09-13T18:27:20.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d92ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d9406fd90 0x7f3d940ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d92ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d9406fd90 0x7f3d940ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52664/0 (socket says 172.21.15.146:52664) 2024-09-13T18:27:20.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d92ffd700 1 -- 172.21.15.146:0/3996956806 learned_addr learned my addr 172.21.15.146:0/3996956806 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:20.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d93fff700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d940c4ec0 0x7f3d94140c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d937fe700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d9406f990 0x7f3d940ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d92ffd700 1 -- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d940c4ec0 msgr2=0x7f3d94140c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d92ffd700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d940c4ec0 0x7f3d94140c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d92ffd700 1 -- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d9406f990 msgr2=0x7f3d940ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:20.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d92ffd700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d9406f990 0x7f3d940ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:20.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d92ffd700 1 -- 172.21.15.146:0/3996956806 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d840094b0 con 0x7f3d9406fd90 2024-09-13T18:27:20.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.870+0000 7f3d937fe700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d9406f990 0x7f3d940ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:20.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.874+0000 7f3d92ffd700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d9406fd90 0x7f3d940ca990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800e9a0 tx=0x7f3d8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:20.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.874+0000 7f3d7bfff700 1 -- 172.21.15.146:0/3996956806 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88009720 con 0x7f3d9406fd90 2024-09-13T18:27:20.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.874+0000 7f3d7bfff700 1 -- 172.21.15.146:0/3996956806 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8800c850 con 0x7f3d9406fd90 2024-09-13T18:27:20.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.874+0000 7f3d988b2700 1 -- 172.21.15.146:0/3996956806 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d941476e0 con 0x7f3d9406fd90 2024-09-13T18:27:20.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.874+0000 7f3d7bfff700 1 -- 172.21.15.146:0/3996956806 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8800cc00 con 0x7f3d9406fd90 2024-09-13T18:27:20.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.874+0000 7f3d988b2700 1 -- 172.21.15.146:0/3996956806 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d94149f70 con 0x7f3d9406fd90 2024-09-13T18:27:20.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.874+0000 7f3d7bfff700 1 -- 172.21.15.146:0/3996956806 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3d88013470 con 0x7f3d9406fd90 2024-09-13T18:27:20.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.874+0000 7f3d988b2700 1 -- 172.21.15.146:0/3996956806 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d940c2260 con 0x7f3d9406fd90 2024-09-13T18:27:20.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.874+0000 7f3d7bfff700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d80066730 0x7f3d80068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:20.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.874+0000 7f3d7bfff700 1 -- 172.21.15.146:0/3996956806 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3d88084510 con 0x7f3d9406fd90 2024-09-13T18:27:20.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.874+0000 7f3d937fe700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d80066730 0x7f3d80068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:20.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.878+0000 7f3d937fe700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d80066730 0x7f3d80068bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d840090b0 tx=0x7f3d84013040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:20.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:20.882+0000 7f3d7bfff700 1 -- 172.21.15.146:0/3996956806 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3d88051510 con 0x7f3d9406fd90 2024-09-13T18:27:21.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.058+0000 7f3d988b2700 1 -- 172.21.15.146:0/3996956806 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3d940c52a0 con 0x7f3d9406fd90 2024-09-13T18:27:21.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.062+0000 7f3d7bfff700 1 -- 172.21.15.146:0/3996956806 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3d88054b30 con 0x7f3d9406fd90 2024-09-13T18:27:21.069 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:27:21.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.062+0000 7f3d988b2700 1 -- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d80066730 msgr2=0x7f3d80068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.062+0000 7f3d988b2700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d80066730 0x7f3d80068bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d840090b0 tx=0x7f3d84013040 comp rx=0 tx=0).stop 2024-09-13T18:27:21.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.062+0000 7f3d988b2700 1 -- 172.21.15.146:0/3996956806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d9406fd90 msgr2=0x7f3d940ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.062+0000 7f3d988b2700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d9406fd90 0x7f3d940ca990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800e9a0 tx=0x7f3d8800ecb0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.062+0000 7f3d988b2700 1 -- 172.21.15.146:0/3996956806 shutdown_connections 2024-09-13T18:27:21.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.062+0000 7f3d988b2700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d80066730 0x7f3d80068bf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.062+0000 7f3d988b2700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d940c4ec0 0x7f3d94140c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.062+0000 7f3d988b2700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d9406fd90 0x7f3d940ca990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.062+0000 7f3d988b2700 1 --2- 172.21.15.146:0/3996956806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d9406f990 0x7f3d940ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.062+0000 7f3d988b2700 1 -- 172.21.15.146:0/3996956806 >> 172.21.15.146:0/3996956806 conn(0x7f3d94066b00 msgr2=0x7f3d940c81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:21.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.062+0000 7f3d988b2700 1 -- 172.21.15.146:0/3996956806 shutdown_connections 2024-09-13T18:27:21.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.062+0000 7f3d988b2700 1 -- 172.21.15.146:0/3996956806 wait complete. 2024-09-13T18:27:21.084 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2024-09-13T18:27:21.084 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:27:21.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff3a605700 1 Processor -- start 2024-09-13T18:27:21.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff3a605700 1 -- start start 2024-09-13T18:27:21.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff3a605700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff34154a40 0x7eff340721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff3a605700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7eff3406f7d0 0x7eff34157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff3a605700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7eff340cac60 0x7eff340cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff3a605700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff34072810 con 0x7eff3406f7d0 2024-09-13T18:27:21.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff3a605700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff34157910 con 0x7eff340cac60 2024-09-13T18:27:21.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff3a605700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340c8c70 con 0x7eff34154a40 2024-09-13T18:27:21.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff38e02700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7eff3406f7d0 0x7eff34157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff38e02700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7eff3406f7d0 0x7eff34157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44118/0 (socket says 172.21.15.146:44118) 2024-09-13T18:27:21.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff38e02700 1 -- 172.21.15.146:0/1260813762 learned_addr learned my addr 172.21.15.146:0/1260813762 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:21.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff39e04700 1 --2- 172.21.15.146:0/1260813762 >> v2:172.21.15.154:3300/0 conn(0x7eff340cac60 0x7eff340cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff39603700 1 --2- 172.21.15.146:0/1260813762 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff34154a40 0x7eff340721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff38e02700 1 -- 172.21.15.146:0/1260813762 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff34154a40 msgr2=0x7eff340721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:21.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff38e02700 1 --2- 172.21.15.146:0/1260813762 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff34154a40 0x7eff340721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff38e02700 1 -- 172.21.15.146:0/1260813762 >> v2:172.21.15.154:3300/0 conn(0x7eff340cac60 msgr2=0x7eff340cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:21.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff38e02700 1 --2- 172.21.15.146:0/1260813762 >> v2:172.21.15.154:3300/0 conn(0x7eff340cac60 0x7eff340cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff38e02700 1 -- 172.21.15.146:0/1260813762 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff340c8f50 con 0x7eff3406f7d0 2024-09-13T18:27:21.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff38e02700 1 --2- 172.21.15.146:0/1260813762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff3406f7d0 0x7eff34157d30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7eff3406c490 tx=0x7eff1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7208678743940f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:21.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff33fff700 1 -- 172.21.15.146:0/1260813762 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c00e070 con 0x7eff3406f7d0 2024-09-13T18:27:21.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff33fff700 1 -- 172.21.15.146:0/1260813762 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1c00b910 con 0x7eff3406f7d0 2024-09-13T18:27:21.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.186+0000 7eff33fff700 1 -- 172.21.15.146:0/1260813762 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c00bcc0 con 0x7eff3406f7d0 2024-09-13T18:27:21.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 -- 172.21.15.146:0/1260813762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff3406f7d0 msgr2=0x7eff34157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 --2- 172.21.15.146:0/1260813762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff3406f7d0 0x7eff34157d30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7eff3406c490 tx=0x7eff1c00b330 comp rx=0 tx=0).stop 2024-09-13T18:27:21.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 -- 172.21.15.146:0/1260813762 shutdown_connections 2024-09-13T18:27:21.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 --2- 172.21.15.146:0/1260813762 >> v2:172.21.15.154:3300/0 conn(0x7eff340cac60 0x7eff340cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 --2- 172.21.15.146:0/1260813762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff3406f7d0 0x7eff34157d30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 --2- 172.21.15.146:0/1260813762 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff34154a40 0x7eff340721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 -- 172.21.15.146:0/1260813762 >> 172.21.15.146:0/1260813762 conn(0x7eff34066090 msgr2=0x7eff34066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:21.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 -- 172.21.15.146:0/1260813762 shutdown_connections 2024-09-13T18:27:21.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 -- 172.21.15.146:0/1260813762 wait complete. 2024-09-13T18:27:21.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 Processor -- start 2024-09-13T18:27:21.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 -- start start 2024-09-13T18:27:21.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff3406f7d0 0x7eff3406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff340cac60 0x7eff341e2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff34154a40 0x7eff341e07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff341ed230 con 0x7eff34154a40 2024-09-13T18:27:21.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3415b620 con 0x7eff3406f7d0 2024-09-13T18:27:21.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff3a605700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3415a9a0 con 0x7eff340cac60 2024-09-13T18:27:21.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff39e04700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff34154a40 0x7eff341e07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff39603700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff3406f7d0 0x7eff3406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff39e04700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff34154a40 0x7eff341e07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44120/0 (socket says 172.21.15.146:44120) 2024-09-13T18:27:21.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff38e02700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff340cac60 0x7eff341e2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff39e04700 1 -- 172.21.15.146:0/709335342 learned_addr learned my addr 172.21.15.146:0/709335342 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:21.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff39e04700 1 -- 172.21.15.146:0/709335342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff340cac60 msgr2=0x7eff341e2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff39e04700 1 --2- 172.21.15.146:0/709335342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff340cac60 0x7eff341e2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff39e04700 1 -- 172.21.15.146:0/709335342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff3406f7d0 msgr2=0x7eff3406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff39e04700 1 --2- 172.21.15.146:0/709335342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff3406f7d0 0x7eff3406d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.190+0000 7eff39e04700 1 -- 172.21.15.146:0/709335342 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c009b30 con 0x7eff34154a40 2024-09-13T18:27:21.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.194+0000 7eff39e04700 1 --2- 172.21.15.146:0/709335342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff34154a40 0x7eff341e07e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7eff2400e7f0 tx=0x7eff2400eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:21.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.194+0000 7eff31ffb700 1 -- 172.21.15.146:0/709335342 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24009540 con 0x7eff34154a40 2024-09-13T18:27:21.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.194+0000 7eff31ffb700 1 -- 172.21.15.146:0/709335342 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff24013680 con 0x7eff34154a40 2024-09-13T18:27:21.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.194+0000 7eff3a605700 1 -- 172.21.15.146:0/709335342 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3415ac80 con 0x7eff34154a40 2024-09-13T18:27:21.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.194+0000 7eff31ffb700 1 -- 172.21.15.146:0/709335342 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24013a10 con 0x7eff34154a40 2024-09-13T18:27:21.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.194+0000 7eff3a605700 1 -- 172.21.15.146:0/709335342 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3415b240 con 0x7eff34154a40 2024-09-13T18:27:21.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.194+0000 7eff3a605700 1 -- 172.21.15.146:0/709335342 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff340c9e00 con 0x7eff34154a40 2024-09-13T18:27:21.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.194+0000 7eff31ffb700 1 -- 172.21.15.146:0/709335342 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7eff24013b70 con 0x7eff34154a40 2024-09-13T18:27:21.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.194+0000 7eff31ffb700 1 --2- 172.21.15.146:0/709335342 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7eff14066760 0x7eff14068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.194+0000 7eff31ffb700 1 -- 172.21.15.146:0/709335342 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7eff24084220 con 0x7eff34154a40 2024-09-13T18:27:21.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.198+0000 7eff39603700 1 --2- 172.21.15.146:0/709335342 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7eff14066760 0x7eff14068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.198+0000 7eff39603700 1 --2- 172.21.15.146:0/709335342 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7eff14066760 0x7eff14068c20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009a00 tx=0x7eff2c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:21.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.202+0000 7eff31ffb700 1 -- 172.21.15.146:0/709335342 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7eff24051210 con 0x7eff34154a40 2024-09-13T18:27:21.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.406+0000 7eff3a605700 1 -- 172.21.15.146:0/709335342 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff3415b760 con 0x7eff34154a40 2024-09-13T18:27:21.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.410+0000 7eff31ffb700 1 -- 172.21.15.146:0/709335342 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6286 (secure 0 0 0) 0x7eff3415b760 con 0x7eff34154a40 2024-09-13T18:27:21.417 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:21.417 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:27:09.524480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"e","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:27:21.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.410+0000 7eff3a605700 1 -- 172.21.15.146:0/709335342 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7eff14066760 msgr2=0x7eff14068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.410+0000 7eff3a605700 1 --2- 172.21.15.146:0/709335342 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7eff14066760 0x7eff14068c20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009a00 tx=0x7eff2c008040 comp rx=0 tx=0).stop 2024-09-13T18:27:21.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.410+0000 7eff3a605700 1 -- 172.21.15.146:0/709335342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff34154a40 msgr2=0x7eff341e07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.410+0000 7eff3a605700 1 --2- 172.21.15.146:0/709335342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff34154a40 0x7eff341e07e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7eff2400e7f0 tx=0x7eff2400eb00 comp rx=0 tx=0).stop 2024-09-13T18:27:21.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.410+0000 7eff3a605700 1 -- 172.21.15.146:0/709335342 shutdown_connections 2024-09-13T18:27:21.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.410+0000 7eff3a605700 1 --2- 172.21.15.146:0/709335342 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7eff14066760 0x7eff14068c20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.410+0000 7eff3a605700 1 --2- 172.21.15.146:0/709335342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff34154a40 0x7eff341e07e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.410+0000 7eff3a605700 1 --2- 172.21.15.146:0/709335342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff340cac60 0x7eff341e2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.410+0000 7eff3a605700 1 --2- 172.21.15.146:0/709335342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff3406f7d0 0x7eff3406d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.410+0000 7eff3a605700 1 -- 172.21.15.146:0/709335342 >> 172.21.15.146:0/709335342 conn(0x7eff34066090 msgr2=0x7eff340afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:21.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.410+0000 7eff3a605700 1 -- 172.21.15.146:0/709335342 shutdown_connections 2024-09-13T18:27:21.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.410+0000 7eff3a605700 1 -- 172.21.15.146:0/709335342 wait complete. 2024-09-13T18:27:21.421 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 15 2024-09-13T18:27:21.431 INFO:tasks.mds_thrash:Ready to start thrashing 2024-09-13T18:27:21.432 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2024-09-13T18:27:21.432 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:27:21.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 Processor -- start 2024-09-13T18:27:21.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 -- start start 2024-09-13T18:27:21.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd530157e10 0x7fd5300c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd53006fe60 0x7fd530070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5300c5bd0 0x7fd5300c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd53015e690 con 0x7fd530157e10 2024-09-13T18:27:21.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5300adfe0 con 0x7fd53006fe60 2024-09-13T18:27:21.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300c7570 con 0x7fd5300c5bd0 2024-09-13T18:27:21.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd535eff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd530157e10 0x7fd5300c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd535eff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd530157e10 0x7fd5300c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44136/0 (socket says 172.21.15.146:44136) 2024-09-13T18:27:21.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd535eff700 1 -- 172.21.15.146:0/1771075314 learned_addr learned my addr 172.21.15.146:0/1771075314 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:21.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd5356fe700 1 --2- 172.21.15.146:0/1771075314 >> v2:172.21.15.154:3300/0 conn(0x7fd53006fe60 0x7fd530070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd536700700 1 --2- 172.21.15.146:0/1771075314 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5300c5bd0 0x7fd5300c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd535eff700 1 -- 172.21.15.146:0/1771075314 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5300c5bd0 msgr2=0x7fd5300c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:21.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd535eff700 1 --2- 172.21.15.146:0/1771075314 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5300c5bd0 0x7fd5300c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd535eff700 1 -- 172.21.15.146:0/1771075314 >> v2:172.21.15.154:3300/0 conn(0x7fd53006fe60 msgr2=0x7fd530070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:21.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd535eff700 1 --2- 172.21.15.146:0/1771075314 >> v2:172.21.15.154:3300/0 conn(0x7fd53006fe60 0x7fd530070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd535eff700 1 -- 172.21.15.146:0/1771075314 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53015eb80 con 0x7fd530157e10 2024-09-13T18:27:21.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd536700700 1 --2- 172.21.15.146:0/1771075314 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5300c5bd0 0x7fd5300c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:21.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd535eff700 1 --2- 172.21.15.146:0/1771075314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd530157e10 0x7fd5300c6df0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd53015b820 tx=0x7fd52c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77935898f72a63a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:21.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd534efd700 1 -- 172.21.15.146:0/1771075314 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c00b8a0 con 0x7fd530157e10 2024-09-13T18:27:21.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd534efd700 1 -- 172.21.15.146:0/1771075314 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52c00ba00 con 0x7fd530157e10 2024-09-13T18:27:21.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd534efd700 1 -- 172.21.15.146:0/1771075314 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c00bd90 con 0x7fd530157e10 2024-09-13T18:27:21.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 -- 172.21.15.146:0/1771075314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd530157e10 msgr2=0x7fd5300c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 --2- 172.21.15.146:0/1771075314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd530157e10 0x7fd5300c6df0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd53015b820 tx=0x7fd52c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 -- 172.21.15.146:0/1771075314 shutdown_connections 2024-09-13T18:27:21.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 --2- 172.21.15.146:0/1771075314 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5300c5bd0 0x7fd5300c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 --2- 172.21.15.146:0/1771075314 >> v2:172.21.15.154:3300/0 conn(0x7fd53006fe60 0x7fd530070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 --2- 172.21.15.146:0/1771075314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd530157e10 0x7fd5300c6df0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 -- 172.21.15.146:0/1771075314 >> 172.21.15.146:0/1771075314 conn(0x7fd5300cb1d0 msgr2=0x7fd5300cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:21.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 -- 172.21.15.146:0/1771075314 shutdown_connections 2024-09-13T18:27:21.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.534+0000 7fd538163700 1 -- 172.21.15.146:0/1771075314 wait complete. 2024-09-13T18:27:21.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.538+0000 7fd538163700 1 Processor -- start 2024-09-13T18:27:21.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.538+0000 7fd538163700 1 -- start start 2024-09-13T18:27:21.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.538+0000 7fd538163700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd53006fe60 0x7fd5300c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.538+0000 7fd538163700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5300c5bd0 0x7fd530150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.538+0000 7fd538163700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd530157e10 0x7fd5301522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.538+0000 7fd538163700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300c25d0 con 0x7fd53006fe60 2024-09-13T18:27:21.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.538+0000 7fd538163700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd530151ba0 con 0x7fd5300c5bd0 2024-09-13T18:27:21.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.538+0000 7fd538163700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53015da10 con 0x7fd530157e10 2024-09-13T18:27:21.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.538+0000 7fd535eff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd53006fe60 0x7fd5300c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.538+0000 7fd535eff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd53006fe60 0x7fd5300c3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44148/0 (socket says 172.21.15.146:44148) 2024-09-13T18:27:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.538+0000 7fd535eff700 1 -- 172.21.15.146:0/3625307029 learned_addr learned my addr 172.21.15.146:0/3625307029 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.538+0000 7fd536700700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd530157e10 0x7fd5301522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.538+0000 7fd5356fe700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5300c5bd0 0x7fd530150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd535eff700 1 -- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd530157e10 msgr2=0x7fd5301522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd535eff700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd530157e10 0x7fd5301522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd535eff700 1 -- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5300c5bd0 msgr2=0x7fd530150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd535eff700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5300c5bd0 0x7fd530150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd535eff700 1 -- 172.21.15.146:0/3625307029 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c009570 con 0x7fd53006fe60 2024-09-13T18:27:21.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd535eff700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd53006fe60 0x7fd5300c3520 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009860 tx=0x7fd52c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:21.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd5267fc700 1 -- 172.21.15.146:0/3625307029 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c0036c0 con 0x7fd53006fe60 2024-09-13T18:27:21.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd5267fc700 1 -- 172.21.15.146:0/3625307029 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52c01f070 con 0x7fd53006fe60 2024-09-13T18:27:21.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd538163700 1 -- 172.21.15.146:0/3625307029 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5301f7020 con 0x7fd53006fe60 2024-09-13T18:27:21.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd5267fc700 1 -- 172.21.15.146:0/3625307029 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c003a60 con 0x7fd53006fe60 2024-09-13T18:27:21.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd538163700 1 -- 172.21.15.146:0/3625307029 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5301515e0 con 0x7fd53006fe60 2024-09-13T18:27:21.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd538163700 1 -- 172.21.15.146:0/3625307029 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd530151ff0 con 0x7fd53006fe60 2024-09-13T18:27:21.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd5267fc700 1 -- 172.21.15.146:0/3625307029 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd52c021030 con 0x7fd53006fe60 2024-09-13T18:27:21.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd5267fc700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd514066730 0x7fd514068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd5267fc700 1 -- 172.21.15.146:0/3625307029 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd52c085a80 con 0x7fd53006fe60 2024-09-13T18:27:21.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.542+0000 7fd5356fe700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd514066730 0x7fd514068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.546+0000 7fd5356fe700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd514066730 0x7fd514068bf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd5300ad710 tx=0x7fd520008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:21.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.546+0000 7fd5267fc700 1 -- 172.21.15.146:0/3625307029 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd52c052b70 con 0x7fd53006fe60 2024-09-13T18:27:21.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.722+0000 7fd538163700 1 -- 172.21.15.146:0/3625307029 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd53006f660 con 0x7fd514066730 2024-09-13T18:27:21.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.746+0000 7fd5267fc700 1 -- 172.21.15.146:0/3625307029 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345369 (secure 0 0 0) 0x7fd52000b620 con 0x7fd514066730 2024-09-13T18:27:21.756 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:21.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.750+0000 7fd538163700 1 -- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd514066730 msgr2=0x7fd514068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.750+0000 7fd538163700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd514066730 0x7fd514068bf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd5300ad710 tx=0x7fd520008040 comp rx=0 tx=0).stop 2024-09-13T18:27:21.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.750+0000 7fd538163700 1 -- 172.21.15.146:0/3625307029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd53006fe60 msgr2=0x7fd5300c3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.750+0000 7fd538163700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd53006fe60 0x7fd5300c3520 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009860 tx=0x7fd52c004770 comp rx=0 tx=0).stop 2024-09-13T18:27:21.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.750+0000 7fd538163700 1 -- 172.21.15.146:0/3625307029 shutdown_connections 2024-09-13T18:27:21.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.750+0000 7fd538163700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd514066730 0x7fd514068bf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.750+0000 7fd538163700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd530157e10 0x7fd5301522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.750+0000 7fd538163700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5300c5bd0 0x7fd530150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.750+0000 7fd538163700 1 --2- 172.21.15.146:0/3625307029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd53006fe60 0x7fd5300c3520 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.750+0000 7fd538163700 1 -- 172.21.15.146:0/3625307029 >> 172.21.15.146:0/3625307029 conn(0x7fd5300cb1d0 msgr2=0x7fd53006dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:21.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.750+0000 7fd538163700 1 -- 172.21.15.146:0/3625307029 shutdown_connections 2024-09-13T18:27:21.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.750+0000 7fd538163700 1 -- 172.21.15.146:0/3625307029 wait complete. 2024-09-13T18:27:21.761 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:27:21.775 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-09-13T18:27:21.590165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598119,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222528,"kb_used_data":2304,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754752192,"statfs":{"total":773094113280,"available":772866244608,"internally_reserved":0,"allocated":2359296,"data_stored":2021622,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":225431099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1766,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.758288"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.520387+0000","last_change":"2024-09-13T18:26:31.972797+0000","last_active":"2024-09-13T18:27:08.520387+0000","last_peered":"2024-09-13T18:27:08.520387+0000","last_clean":"2024-09-13T18:27:08.520387+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:27:08.520387+0000","last_undegraded":"2024-09-13T18:27:08.520387+0000","last_fullsized":"2024-09-13T18:27:08.520387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:50.511747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384385+0000","last_change":"2024-09-13T18:26:32.349901+0000","last_active":"2024-09-13T18:26:36.384385+0000","last_peered":"2024-09-13T18:26:36.384385+0000","last_clean":"2024-09-13T18:26:36.384385+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:26:36.384385+0000","last_undegraded":"2024-09-13T18:26:36.384385+0000","last_fullsized":"2024-09-13T18:26:36.384385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:24.978813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768058+0000","last_change":"2024-09-13T18:26:36.407579+0000","last_active":"2024-09-13T18:26:36.768058+0000","last_peered":"2024-09-13T18:26:36.768058+0000","last_clean":"2024-09-13T18:26:36.768058+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:26:36.768058+0000","last_undegraded":"2024-09-13T18:26:36.768058+0000","last_fullsized":"2024-09-13T18:26:36.768058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:53:56.370013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343814+0000","last_change":"2024-09-13T18:26:31.980716+0000","last_active":"2024-09-13T18:26:37.343814+0000","last_peered":"2024-09-13T18:26:37.343814+0000","last_clean":"2024-09-13T18:26:37.343814+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:26:37.343814+0000","last_undegraded":"2024-09-13T18:26:37.343814+0000","last_fullsized":"2024-09-13T18:26:37.343814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:20.020903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388711+0000","last_change":"2024-09-13T18:26:32.342690+0000","last_active":"2024-09-13T18:26:36.388711+0000","last_peered":"2024-09-13T18:26:36.388711+0000","last_clean":"2024-09-13T18:26:36.388711+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:26:36.388711+0000","last_undegraded":"2024-09-13T18:26:36.388711+0000","last_fullsized":"2024-09-13T18:26:36.388711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.212930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.398481+0000","last_change":"2024-09-13T18:26:36.396545+0000","last_active":"2024-09-13T18:26:36.398481+0000","last_peered":"2024-09-13T18:26:36.398481+0000","last_clean":"2024-09-13T18:26:36.398481+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:26:36.398481+0000","last_undegraded":"2024-09-13T18:26:36.398481+0000","last_fullsized":"2024-09-13T18:26:36.398481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:38.760215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.523368+0000","last_change":"2024-09-13T18:26:31.977823+0000","last_active":"2024-09-13T18:27:08.523368+0000","last_peered":"2024-09-13T18:27:08.523368+0000","last_clean":"2024-09-13T18:27:08.523368+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:27:08.523368+0000","last_undegraded":"2024-09-13T18:27:08.523368+0000","last_fullsized":"2024-09-13T18:27:08.523368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:46.956527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394225+0000","last_change":"2024-09-13T18:26:32.361288+0000","last_active":"2024-09-13T18:26:36.394225+0000","last_peered":"2024-09-13T18:26:36.394225+0000","last_clean":"2024-09-13T18:26:36.394225+0000","last_became_active":"2024-09-13T18:26:32.360918+0000","last_became_peered":"2024-09-13T18:26:32.360918+0000","last_unstale":"2024-09-13T18:26:36.394225+0000","last_undegraded":"2024-09-13T18:26:36.394225+0000","last_fullsized":"2024-09-13T18:26:36.394225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:30.143436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408364+0000","last_change":"2024-09-13T18:26:36.406675+0000","last_active":"2024-09-13T18:26:36.408364+0000","last_peered":"2024-09-13T18:26:36.408364+0000","last_clean":"2024-09-13T18:26:36.408364+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:26:36.408364+0000","last_undegraded":"2024-09-13T18:26:36.408364+0000","last_fullsized":"2024-09-13T18:26:36.408364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:36.896027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.347551+0000","last_change":"2024-09-13T18:26:31.979860+0000","last_active":"2024-09-13T18:26:37.347551+0000","last_peered":"2024-09-13T18:26:37.347551+0000","last_clean":"2024-09-13T18:26:37.347551+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:26:37.347551+0000","last_undegraded":"2024-09-13T18:26:37.347551+0000","last_fullsized":"2024-09-13T18:26:37.347551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:45.323631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393203+0000","last_change":"2024-09-13T18:26:32.351494+0000","last_active":"2024-09-13T18:26:36.393203+0000","last_peered":"2024-09-13T18:26:36.393203+0000","last_clean":"2024-09-13T18:26:36.393203+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:26:36.393203+0000","last_undegraded":"2024-09-13T18:26:36.393203+0000","last_fullsized":"2024-09-13T18:26:36.393203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:52.804526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400955+0000","last_change":"2024-09-13T18:26:36.398763+0000","last_active":"2024-09-13T18:26:36.400955+0000","last_peered":"2024-09-13T18:26:36.400955+0000","last_clean":"2024-09-13T18:26:36.400955+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:26:36.400955+0000","last_undegraded":"2024-09-13T18:26:36.400955+0000","last_fullsized":"2024-09-13T18:26:36.400955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:35.417438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897834+0000","last_change":"2024-09-13T18:26:32.360976+0000","last_active":"2024-09-13T18:26:36.897834+0000","last_peered":"2024-09-13T18:26:36.897834+0000","last_clean":"2024-09-13T18:26:36.897834+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:26:36.897834+0000","last_undegraded":"2024-09-13T18:26:36.897834+0000","last_fullsized":"2024-09-13T18:26:36.897834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:53:52.373515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.477712+0000","last_change":"2024-09-13T18:26:31.977891+0000","last_active":"2024-09-13T18:27:03.477712+0000","last_peered":"2024-09-13T18:27:03.477712+0000","last_clean":"2024-09-13T18:27:03.477712+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:27:03.477712+0000","last_undegraded":"2024-09-13T18:27:03.477712+0000","last_fullsized":"2024-09-13T18:27:03.477712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:01.453094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400563+0000","last_change":"2024-09-13T18:26:36.397484+0000","last_active":"2024-09-13T18:26:36.400563+0000","last_peered":"2024-09-13T18:26:36.400563+0000","last_clean":"2024-09-13T18:26:36.400563+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:26:36.400563+0000","last_undegraded":"2024-09-13T18:26:36.400563+0000","last_fullsized":"2024-09-13T18:26:36.400563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:44.686244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386958+0000","last_change":"2024-09-13T18:26:32.345564+0000","last_active":"2024-09-13T18:26:36.386958+0000","last_peered":"2024-09-13T18:26:36.386958+0000","last_clean":"2024-09-13T18:26:36.386958+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:26:36.386958+0000","last_undegraded":"2024-09-13T18:26:36.386958+0000","last_fullsized":"2024-09-13T18:26:36.386958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:54.331806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767096+0000","last_change":"2024-09-13T18:26:31.974107+0000","last_active":"2024-09-13T18:26:36.767096+0000","last_peered":"2024-09-13T18:26:36.767096+0000","last_clean":"2024-09-13T18:26:36.767096+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:26:36.767096+0000","last_undegraded":"2024-09-13T18:26:36.767096+0000","last_fullsized":"2024-09-13T18:26:36.767096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:37.547480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408253+0000","last_change":"2024-09-13T18:26:36.405986+0000","last_active":"2024-09-13T18:26:36.408253+0000","last_peered":"2024-09-13T18:26:36.408253+0000","last_clean":"2024-09-13T18:26:36.408253+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:26:36.408253+0000","last_undegraded":"2024-09-13T18:26:36.408253+0000","last_fullsized":"2024-09-13T18:26:36.408253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:30.052430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383772+0000","last_change":"2024-09-13T18:26:32.340616+0000","last_active":"2024-09-13T18:26:36.383772+0000","last_peered":"2024-09-13T18:26:36.383772+0000","last_clean":"2024-09-13T18:26:36.383772+0000","last_became_active":"2024-09-13T18:26:32.339405+0000","last_became_peered":"2024-09-13T18:26:32.339405+0000","last_unstale":"2024-09-13T18:26:36.383772+0000","last_undegraded":"2024-09-13T18:26:36.383772+0000","last_fullsized":"2024-09-13T18:26:36.383772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:26.081896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.474579+0000","last_change":"2024-09-13T18:26:31.968895+0000","last_active":"2024-09-13T18:27:03.474579+0000","last_peered":"2024-09-13T18:27:03.474579+0000","last_clean":"2024-09-13T18:27:03.474579+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:27:03.474579+0000","last_undegraded":"2024-09-13T18:27:03.474579+0000","last_fullsized":"2024-09-13T18:27:03.474579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:15.525097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767834+0000","last_change":"2024-09-13T18:26:36.406484+0000","last_active":"2024-09-13T18:26:36.767834+0000","last_peered":"2024-09-13T18:26:36.767834+0000","last_clean":"2024-09-13T18:26:36.767834+0000","last_became_active":"2024-09-13T18:26:36.406161+0000","last_became_peered":"2024-09-13T18:26:36.406161+0000","last_unstale":"2024-09-13T18:26:36.767834+0000","last_undegraded":"2024-09-13T18:26:36.767834+0000","last_fullsized":"2024-09-13T18:26:36.767834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:53.330919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391435+0000","last_change":"2024-09-13T18:26:32.360406+0000","last_active":"2024-09-13T18:26:36.391435+0000","last_peered":"2024-09-13T18:26:36.391435+0000","last_clean":"2024-09-13T18:26:36.391435+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:26:36.391435+0000","last_undegraded":"2024-09-13T18:26:36.391435+0000","last_fullsized":"2024-09-13T18:26:36.391435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:37.417707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.476272+0000","last_change":"2024-09-13T18:26:31.976872+0000","last_active":"2024-09-13T18:27:03.476272+0000","last_peered":"2024-09-13T18:27:03.476272+0000","last_clean":"2024-09-13T18:27:03.476272+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:27:03.476272+0000","last_undegraded":"2024-09-13T18:27:03.476272+0000","last_fullsized":"2024-09-13T18:27:03.476272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:26.163029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408244+0000","last_change":"2024-09-13T18:26:36.405722+0000","last_active":"2024-09-13T18:26:36.408244+0000","last_peered":"2024-09-13T18:26:36.408244+0000","last_clean":"2024-09-13T18:26:36.408244+0000","last_became_active":"2024-09-13T18:26:36.405310+0000","last_became_peered":"2024-09-13T18:26:36.405310+0000","last_unstale":"2024-09-13T18:26:36.408244+0000","last_undegraded":"2024-09-13T18:26:36.408244+0000","last_fullsized":"2024-09-13T18:26:36.408244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:09.815655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.389080+0000","last_change":"2024-09-13T18:26:32.343915+0000","last_active":"2024-09-13T18:26:36.389080+0000","last_peered":"2024-09-13T18:26:36.389080+0000","last_clean":"2024-09-13T18:26:36.389080+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:26:36.389080+0000","last_undegraded":"2024-09-13T18:26:36.389080+0000","last_fullsized":"2024-09-13T18:26:36.389080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:19.357243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.477810+0000","last_change":"2024-09-13T18:26:31.979944+0000","last_active":"2024-09-13T18:27:03.477810+0000","last_peered":"2024-09-13T18:27:03.477810+0000","last_clean":"2024-09-13T18:27:03.477810+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:27:03.477810+0000","last_undegraded":"2024-09-13T18:27:03.477810+0000","last_fullsized":"2024-09-13T18:27:03.477810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:21.721921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.411958+0000","last_change":"2024-09-13T18:26:36.410028+0000","last_active":"2024-09-13T18:26:36.411958+0000","last_peered":"2024-09-13T18:26:36.411958+0000","last_clean":"2024-09-13T18:26:36.411958+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:26:36.411958+0000","last_undegraded":"2024-09-13T18:26:36.411958+0000","last_fullsized":"2024-09-13T18:26:36.411958+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:32.264341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387875+0000","last_change":"2024-09-13T18:26:32.358980+0000","last_active":"2024-09-13T18:26:36.387875+0000","last_peered":"2024-09-13T18:26:36.387875+0000","last_clean":"2024-09-13T18:26:36.387875+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:26:36.387875+0000","last_undegraded":"2024-09-13T18:26:36.387875+0000","last_fullsized":"2024-09-13T18:26:36.387875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:08.748139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.521355+0000","last_change":"2024-09-13T18:26:31.968972+0000","last_active":"2024-09-13T18:27:08.521355+0000","last_peered":"2024-09-13T18:27:08.521355+0000","last_clean":"2024-09-13T18:27:08.521355+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:27:08.521355+0000","last_undegraded":"2024-09-13T18:27:08.521355+0000","last_fullsized":"2024-09-13T18:27:08.521355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:56.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768011+0000","last_change":"2024-09-13T18:26:36.407034+0000","last_active":"2024-09-13T18:26:36.768011+0000","last_peered":"2024-09-13T18:26:36.768011+0000","last_clean":"2024-09-13T18:26:36.768011+0000","last_became_active":"2024-09-13T18:26:36.406473+0000","last_became_peered":"2024-09-13T18:26:36.406473+0000","last_unstale":"2024-09-13T18:26:36.768011+0000","last_undegraded":"2024-09-13T18:26:36.768011+0000","last_fullsized":"2024-09-13T18:26:36.768011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:53.815901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386420+0000","last_change":"2024-09-13T18:26:32.342823+0000","last_active":"2024-09-13T18:26:36.386420+0000","last_peered":"2024-09-13T18:26:36.386420+0000","last_clean":"2024-09-13T18:26:36.386420+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:26:36.386420+0000","last_undegraded":"2024-09-13T18:26:36.386420+0000","last_fullsized":"2024-09-13T18:26:36.386420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:28.051252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.349179+0000","last_change":"2024-09-13T18:26:31.978976+0000","last_active":"2024-09-13T18:26:37.349179+0000","last_peered":"2024-09-13T18:26:37.349179+0000","last_clean":"2024-09-13T18:26:37.349179+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:26:37.349179+0000","last_undegraded":"2024-09-13T18:26:37.349179+0000","last_fullsized":"2024-09-13T18:26:37.349179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:11.279557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103455+0000","last_change":"2024-09-13T18:26:36.408844+0000","last_active":"2024-09-13T18:26:37.103455+0000","last_peered":"2024-09-13T18:26:37.103455+0000","last_clean":"2024-09-13T18:26:37.103455+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:26:37.103455+0000","last_undegraded":"2024-09-13T18:26:37.103455+0000","last_fullsized":"2024-09-13T18:26:37.103455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:06.969342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393043+0000","last_change":"2024-09-13T18:26:32.355038+0000","last_active":"2024-09-13T18:26:36.393043+0000","last_peered":"2024-09-13T18:26:36.393043+0000","last_clean":"2024-09-13T18:26:36.393043+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:26:36.393043+0000","last_undegraded":"2024-09-13T18:26:36.393043+0000","last_fullsized":"2024-09-13T18:26:36.393043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.159231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386562+0000","last_change":"2024-09-13T18:26:31.972931+0000","last_active":"2024-09-13T18:26:36.386562+0000","last_peered":"2024-09-13T18:26:36.386562+0000","last_clean":"2024-09-13T18:26:36.386562+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:26:36.386562+0000","last_undegraded":"2024-09-13T18:26:36.386562+0000","last_fullsized":"2024-09-13T18:26:36.386562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:05.646924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766310+0000","last_change":"2024-09-13T18:26:36.398876+0000","last_active":"2024-09-13T18:26:36.766310+0000","last_peered":"2024-09-13T18:26:36.766310+0000","last_clean":"2024-09-13T18:26:36.766310+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:26:36.766310+0000","last_undegraded":"2024-09-13T18:26:36.766310+0000","last_fullsized":"2024-09-13T18:26:36.766310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:56.918239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392625+0000","last_change":"2024-09-13T18:26:32.353111+0000","last_active":"2024-09-13T18:26:36.392625+0000","last_peered":"2024-09-13T18:26:36.392625+0000","last_clean":"2024-09-13T18:26:36.392625+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:26:36.392625+0000","last_undegraded":"2024-09-13T18:26:36.392625+0000","last_fullsized":"2024-09-13T18:26:36.392625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:43.927936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.772320+0000","last_change":"2024-09-13T18:26:31.976075+0000","last_active":"2024-09-13T18:26:36.772320+0000","last_peered":"2024-09-13T18:26:36.772320+0000","last_clean":"2024-09-13T18:26:36.772320+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:26:36.772320+0000","last_undegraded":"2024-09-13T18:26:36.772320+0000","last_fullsized":"2024-09-13T18:26:36.772320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:03.693518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766877+0000","last_change":"2024-09-13T18:26:36.401777+0000","last_active":"2024-09-13T18:26:36.766877+0000","last_peered":"2024-09-13T18:26:36.766877+0000","last_clean":"2024-09-13T18:26:36.766877+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:26:36.766877+0000","last_undegraded":"2024-09-13T18:26:36.766877+0000","last_fullsized":"2024-09-13T18:26:36.766877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:43.440284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384991+0000","last_change":"2024-09-13T18:26:32.343791+0000","last_active":"2024-09-13T18:26:36.384991+0000","last_peered":"2024-09-13T18:26:36.384991+0000","last_clean":"2024-09-13T18:26:36.384991+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:26:36.384991+0000","last_undegraded":"2024-09-13T18:26:36.384991+0000","last_fullsized":"2024-09-13T18:26:36.384991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:36.296571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.521069+0000","last_change":"2024-09-13T18:26:31.970786+0000","last_active":"2024-09-13T18:27:08.521069+0000","last_peered":"2024-09-13T18:27:08.521069+0000","last_clean":"2024-09-13T18:27:08.521069+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:27:08.521069+0000","last_undegraded":"2024-09-13T18:27:08.521069+0000","last_fullsized":"2024-09-13T18:27:08.521069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:54.584009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767929+0000","last_change":"2024-09-13T18:26:36.401529+0000","last_active":"2024-09-13T18:26:36.767929+0000","last_peered":"2024-09-13T18:26:36.767929+0000","last_clean":"2024-09-13T18:26:36.767929+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:26:36.767929+0000","last_undegraded":"2024-09-13T18:26:36.767929+0000","last_fullsized":"2024-09-13T18:26:36.767929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:47.318347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.382662+0000","last_change":"2024-09-13T18:26:32.350470+0000","last_active":"2024-09-13T18:26:36.382662+0000","last_peered":"2024-09-13T18:26:36.382662+0000","last_clean":"2024-09-13T18:26:36.382662+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:26:36.382662+0000","last_undegraded":"2024-09-13T18:26:36.382662+0000","last_fullsized":"2024-09-13T18:26:36.382662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:55.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.522206+0000","last_change":"2024-09-13T18:26:31.972587+0000","last_active":"2024-09-13T18:27:08.522206+0000","last_peered":"2024-09-13T18:27:08.522206+0000","last_clean":"2024-09-13T18:27:08.522206+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:27:08.522206+0000","last_undegraded":"2024-09-13T18:27:08.522206+0000","last_fullsized":"2024-09-13T18:27:08.522206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:27.547763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103735+0000","last_change":"2024-09-13T18:26:36.405086+0000","last_active":"2024-09-13T18:26:37.103735+0000","last_peered":"2024-09-13T18:26:37.103735+0000","last_clean":"2024-09-13T18:26:37.103735+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:26:37.103735+0000","last_undegraded":"2024-09-13T18:26:37.103735+0000","last_fullsized":"2024-09-13T18:26:37.103735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:02:51.694634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388036+0000","last_change":"2024-09-13T18:26:32.350510+0000","last_active":"2024-09-13T18:26:36.388036+0000","last_peered":"2024-09-13T18:26:36.388036+0000","last_clean":"2024-09-13T18:26:36.388036+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:26:36.388036+0000","last_undegraded":"2024-09-13T18:26:36.388036+0000","last_fullsized":"2024-09-13T18:26:36.388036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:40.850255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.479714+0000","last_change":"2024-09-13T18:26:31.978355+0000","last_active":"2024-09-13T18:27:03.479714+0000","last_peered":"2024-09-13T18:27:03.479714+0000","last_clean":"2024-09-13T18:27:03.479714+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:27:03.479714+0000","last_undegraded":"2024-09-13T18:27:03.479714+0000","last_fullsized":"2024-09-13T18:27:03.479714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:12.946518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404778+0000","last_change":"2024-09-13T18:26:26.931403+0000","last_active":"2024-09-13T18:26:36.404778+0000","last_peered":"2024-09-13T18:26:36.404778+0000","last_clean":"2024-09-13T18:26:36.404778+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:26:36.404778+0000","last_undegraded":"2024-09-13T18:26:36.404778+0000","last_fullsized":"2024-09-13T18:26:36.404778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_clean_scrub_stamp":"2024-09-13T18:26:24.880028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:30.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897926+0000","last_change":"2024-09-13T18:26:36.409827+0000","last_active":"2024-09-13T18:26:36.897926+0000","last_peered":"2024-09-13T18:26:36.897926+0000","last_clean":"2024-09-13T18:26:36.897926+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:26:36.897926+0000","last_undegraded":"2024-09-13T18:26:36.897926+0000","last_fullsized":"2024-09-13T18:26:36.897926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:50.482400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405746+0000","last_change":"2024-09-13T18:26:32.356850+0000","last_active":"2024-09-13T18:26:36.405746+0000","last_peered":"2024-09-13T18:26:36.405746+0000","last_clean":"2024-09-13T18:26:36.405746+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:26:36.405746+0000","last_undegraded":"2024-09-13T18:26:36.405746+0000","last_fullsized":"2024-09-13T18:26:36.405746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:38.710828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.520921+0000","last_change":"2024-09-13T18:26:31.975942+0000","last_active":"2024-09-13T18:27:08.520921+0000","last_peered":"2024-09-13T18:27:08.520921+0000","last_clean":"2024-09-13T18:27:08.520921+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:27:08.520921+0000","last_undegraded":"2024-09-13T18:27:08.520921+0000","last_fullsized":"2024-09-13T18:27:08.520921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:05.840582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897805+0000","last_change":"2024-09-13T18:26:36.409802+0000","last_active":"2024-09-13T18:26:36.897805+0000","last_peered":"2024-09-13T18:26:36.897805+0000","last_clean":"2024-09-13T18:26:36.897805+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:26:36.897805+0000","last_undegraded":"2024-09-13T18:26:36.897805+0000","last_fullsized":"2024-09-13T18:26:36.897805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:38.178151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394309+0000","last_change":"2024-09-13T18:26:32.355181+0000","last_active":"2024-09-13T18:26:36.394309+0000","last_peered":"2024-09-13T18:26:36.394309+0000","last_clean":"2024-09-13T18:26:36.394309+0000","last_became_active":"2024-09-13T18:26:32.353043+0000","last_became_peered":"2024-09-13T18:26:32.353043+0000","last_unstale":"2024-09-13T18:26:36.394309+0000","last_undegraded":"2024-09-13T18:26:36.394309+0000","last_fullsized":"2024-09-13T18:26:36.394309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:04:17.000915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767488+0000","last_change":"2024-09-13T18:26:31.966618+0000","last_active":"2024-09-13T18:26:36.767488+0000","last_peered":"2024-09-13T18:26:36.767488+0000","last_clean":"2024-09-13T18:26:36.767488+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:26:36.767488+0000","last_undegraded":"2024-09-13T18:26:36.767488+0000","last_fullsized":"2024-09-13T18:26:36.767488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:54:26.422611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103645+0000","last_change":"2024-09-13T18:26:36.394344+0000","last_active":"2024-09-13T18:26:37.103645+0000","last_peered":"2024-09-13T18:26:37.103645+0000","last_clean":"2024-09-13T18:26:37.103645+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:26:37.103645+0000","last_undegraded":"2024-09-13T18:26:37.103645+0000","last_fullsized":"2024-09-13T18:26:37.103645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:18.425816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405784+0000","last_change":"2024-09-13T18:26:32.365655+0000","last_active":"2024-09-13T18:26:36.405784+0000","last_peered":"2024-09-13T18:26:36.405784+0000","last_clean":"2024-09-13T18:26:36.405784+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:26:36.405784+0000","last_undegraded":"2024-09-13T18:26:36.405784+0000","last_fullsized":"2024-09-13T18:26:36.405784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:40.288128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343912+0000","last_change":"2024-09-13T18:26:31.970652+0000","last_active":"2024-09-13T18:26:37.343912+0000","last_peered":"2024-09-13T18:26:37.343912+0000","last_clean":"2024-09-13T18:26:37.343912+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:26:37.343912+0000","last_undegraded":"2024-09-13T18:26:37.343912+0000","last_fullsized":"2024-09-13T18:26:37.343912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:11.169661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.770631+0000","last_change":"2024-09-13T18:26:36.410280+0000","last_active":"2024-09-13T18:26:36.770631+0000","last_peered":"2024-09-13T18:26:36.770631+0000","last_clean":"2024-09-13T18:26:36.770631+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:26:36.770631+0000","last_undegraded":"2024-09-13T18:26:36.770631+0000","last_fullsized":"2024-09-13T18:26:36.770631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:01.395290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405642+0000","last_change":"2024-09-13T18:26:32.361575+0000","last_active":"2024-09-13T18:26:36.405642+0000","last_peered":"2024-09-13T18:26:36.405642+0000","last_clean":"2024-09-13T18:26:36.405642+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:26:36.405642+0000","last_undegraded":"2024-09-13T18:26:36.405642+0000","last_fullsized":"2024-09-13T18:26:36.405642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:08.818877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"25'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.346552+0000","last_change":"2024-09-13T18:26:31.977029+0000","last_active":"2024-09-13T18:26:37.346552+0000","last_peered":"2024-09-13T18:26:37.346552+0000","last_clean":"2024-09-13T18:26:37.346552+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:26:37.346552+0000","last_undegraded":"2024-09-13T18:26:37.346552+0000","last_fullsized":"2024-09-13T18:26:37.346552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:13.701689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407565+0000","last_change":"2024-09-13T18:26:36.403479+0000","last_active":"2024-09-13T18:26:36.407565+0000","last_peered":"2024-09-13T18:26:36.407565+0000","last_clean":"2024-09-13T18:26:36.407565+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:26:36.407565+0000","last_undegraded":"2024-09-13T18:26:36.407565+0000","last_fullsized":"2024-09-13T18:26:36.407565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:51.095852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404976+0000","last_change":"2024-09-13T18:26:32.365143+0000","last_active":"2024-09-13T18:26:36.404976+0000","last_peered":"2024-09-13T18:26:36.404976+0000","last_clean":"2024-09-13T18:26:36.404976+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:26:36.404976+0000","last_undegraded":"2024-09-13T18:26:36.404976+0000","last_fullsized":"2024-09-13T18:26:36.404976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:46.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.526361+0000","last_change":"2024-09-13T18:26:31.965802+0000","last_active":"2024-09-13T18:27:08.526361+0000","last_peered":"2024-09-13T18:27:08.526361+0000","last_clean":"2024-09-13T18:27:08.526361+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:27:08.526361+0000","last_undegraded":"2024-09-13T18:27:08.526361+0000","last_fullsized":"2024-09-13T18:27:08.526361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:12.759181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103768+0000","last_change":"2024-09-13T18:26:36.399235+0000","last_active":"2024-09-13T18:26:37.103768+0000","last_peered":"2024-09-13T18:26:37.103768+0000","last_clean":"2024-09-13T18:26:37.103768+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:26:37.103768+0000","last_undegraded":"2024-09-13T18:26:37.103768+0000","last_fullsized":"2024-09-13T18:26:37.103768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:13:54.711083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897760+0000","last_change":"2024-09-13T18:26:32.357403+0000","last_active":"2024-09-13T18:26:36.897760+0000","last_peered":"2024-09-13T18:26:36.897760+0000","last_clean":"2024-09-13T18:26:36.897760+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:26:36.897760+0000","last_undegraded":"2024-09-13T18:26:36.897760+0000","last_fullsized":"2024-09-13T18:26:36.897760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:12.600397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.525532+0000","last_change":"2024-09-13T18:26:31.976148+0000","last_active":"2024-09-13T18:27:08.525532+0000","last_peered":"2024-09-13T18:27:08.525532+0000","last_clean":"2024-09-13T18:27:08.525532+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:27:08.525532+0000","last_undegraded":"2024-09-13T18:27:08.525532+0000","last_fullsized":"2024-09-13T18:27:08.525532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:38.023636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769582+0000","last_change":"2024-09-13T18:26:36.406177+0000","last_active":"2024-09-13T18:26:36.769582+0000","last_peered":"2024-09-13T18:26:36.769582+0000","last_clean":"2024-09-13T18:26:36.769582+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:26:36.769582+0000","last_undegraded":"2024-09-13T18:26:36.769582+0000","last_fullsized":"2024-09-13T18:26:36.769582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:00.309995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405459+0000","last_change":"2024-09-13T18:26:32.361726+0000","last_active":"2024-09-13T18:26:36.405459+0000","last_peered":"2024-09-13T18:26:36.405459+0000","last_clean":"2024-09-13T18:26:36.405459+0000","last_became_active":"2024-09-13T18:26:32.361432+0000","last_became_peered":"2024-09-13T18:26:32.361432+0000","last_unstale":"2024-09-13T18:26:36.405459+0000","last_undegraded":"2024-09-13T18:26:36.405459+0000","last_fullsized":"2024-09-13T18:26:36.405459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:42.021515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.480098+0000","last_change":"2024-09-13T18:26:31.982114+0000","last_active":"2024-09-13T18:27:03.480098+0000","last_peered":"2024-09-13T18:27:03.480098+0000","last_clean":"2024-09-13T18:27:03.480098+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:27:03.480098+0000","last_undegraded":"2024-09-13T18:27:03.480098+0000","last_fullsized":"2024-09-13T18:27:03.480098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.266528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769049+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:26:36.769049+0000","last_peered":"2024-09-13T18:26:36.769049+0000","last_clean":"2024-09-13T18:26:36.769049+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:26:36.769049+0000","last_undegraded":"2024-09-13T18:26:36.769049+0000","last_fullsized":"2024-09-13T18:26:36.769049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:58.853971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388651+0000","last_change":"2024-09-13T18:26:32.351145+0000","last_active":"2024-09-13T18:26:36.388651+0000","last_peered":"2024-09-13T18:26:36.388651+0000","last_clean":"2024-09-13T18:26:36.388651+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:26:36.388651+0000","last_undegraded":"2024-09-13T18:26:36.388651+0000","last_fullsized":"2024-09-13T18:26:36.388651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:57.002435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.519027+0000","last_change":"2024-09-13T18:26:31.961202+0000","last_active":"2024-09-13T18:27:08.519027+0000","last_peered":"2024-09-13T18:27:08.519027+0000","last_clean":"2024-09-13T18:27:08.519027+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:27:08.519027+0000","last_undegraded":"2024-09-13T18:27:08.519027+0000","last_fullsized":"2024-09-13T18:27:08.519027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:27.340127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.772209+0000","last_change":"2024-09-13T18:26:36.410014+0000","last_active":"2024-09-13T18:26:36.772209+0000","last_peered":"2024-09-13T18:26:36.772209+0000","last_clean":"2024-09-13T18:26:36.772209+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:26:36.772209+0000","last_undegraded":"2024-09-13T18:26:36.772209+0000","last_fullsized":"2024-09-13T18:26:36.772209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:24.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405669+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:26:36.405669+0000","last_peered":"2024-09-13T18:26:36.405669+0000","last_clean":"2024-09-13T18:26:36.405669+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:26:36.405669+0000","last_undegraded":"2024-09-13T18:26:36.405669+0000","last_fullsized":"2024-09-13T18:26:36.405669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:40.102815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'11","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.474634+0000","last_change":"2024-09-13T18:26:31.974110+0000","last_active":"2024-09-13T18:27:03.474634+0000","last_peered":"2024-09-13T18:27:03.474634+0000","last_clean":"2024-09-13T18:27:03.474634+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:27:03.474634+0000","last_undegraded":"2024-09-13T18:27:03.474634+0000","last_fullsized":"2024-09-13T18:27:03.474634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:56.058561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766576+0000","last_change":"2024-09-13T18:26:36.394080+0000","last_active":"2024-09-13T18:26:36.766576+0000","last_peered":"2024-09-13T18:26:36.766576+0000","last_clean":"2024-09-13T18:26:36.766576+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:26:36.766576+0000","last_undegraded":"2024-09-13T18:26:36.766576+0000","last_fullsized":"2024-09-13T18:26:36.766576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:47.121814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392195+0000","last_change":"2024-09-13T18:26:32.362873+0000","last_active":"2024-09-13T18:26:36.392195+0000","last_peered":"2024-09-13T18:26:36.392195+0000","last_clean":"2024-09-13T18:26:36.392195+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:26:36.392195+0000","last_undegraded":"2024-09-13T18:26:36.392195+0000","last_fullsized":"2024-09-13T18:26:36.392195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:37.598189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.344708+0000","last_change":"2024-09-13T18:26:31.976401+0000","last_active":"2024-09-13T18:26:37.344708+0000","last_peered":"2024-09-13T18:26:37.344708+0000","last_clean":"2024-09-13T18:26:37.344708+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:26:37.344708+0000","last_undegraded":"2024-09-13T18:26:37.344708+0000","last_fullsized":"2024-09-13T18:26:37.344708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:26.434292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408317+0000","last_change":"2024-09-13T18:26:36.406228+0000","last_active":"2024-09-13T18:26:36.408317+0000","last_peered":"2024-09-13T18:26:36.408317+0000","last_clean":"2024-09-13T18:26:36.408317+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:26:36.408317+0000","last_undegraded":"2024-09-13T18:26:36.408317+0000","last_fullsized":"2024-09-13T18:26:36.408317+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:49.198397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388733+0000","last_change":"2024-09-13T18:26:32.350841+0000","last_active":"2024-09-13T18:26:36.388733+0000","last_peered":"2024-09-13T18:26:36.388733+0000","last_clean":"2024-09-13T18:26:36.388733+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:26:36.388733+0000","last_undegraded":"2024-09-13T18:26:36.388733+0000","last_fullsized":"2024-09-13T18:26:36.388733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:05.137882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.476898+0000","last_change":"2024-09-13T18:26:31.978214+0000","last_active":"2024-09-13T18:27:03.476898+0000","last_peered":"2024-09-13T18:27:03.476898+0000","last_clean":"2024-09-13T18:27:03.476898+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:27:03.476898+0000","last_undegraded":"2024-09-13T18:27:03.476898+0000","last_fullsized":"2024-09-13T18:27:03.476898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:17.381375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768498+0000","last_change":"2024-09-13T18:26:36.406090+0000","last_active":"2024-09-13T18:26:36.768498+0000","last_peered":"2024-09-13T18:26:36.768498+0000","last_clean":"2024-09-13T18:26:36.768498+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:26:36.768498+0000","last_undegraded":"2024-09-13T18:26:36.768498+0000","last_fullsized":"2024-09-13T18:26:36.768498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:09.865147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391759+0000","last_change":"2024-09-13T18:26:32.357701+0000","last_active":"2024-09-13T18:26:36.391759+0000","last_peered":"2024-09-13T18:26:36.391759+0000","last_clean":"2024-09-13T18:26:36.391759+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:26:36.391759+0000","last_undegraded":"2024-09-13T18:26:36.391759+0000","last_fullsized":"2024-09-13T18:26:36.391759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:47.587683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.479895+0000","last_change":"2024-09-13T18:26:31.980133+0000","last_active":"2024-09-13T18:27:03.479895+0000","last_peered":"2024-09-13T18:27:03.479895+0000","last_clean":"2024-09-13T18:27:03.479895+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:27:03.479895+0000","last_undegraded":"2024-09-13T18:27:03.479895+0000","last_fullsized":"2024-09-13T18:27:03.479895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:05.354335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768120+0000","last_change":"2024-09-13T18:26:36.406553+0000","last_active":"2024-09-13T18:26:36.768120+0000","last_peered":"2024-09-13T18:26:36.768120+0000","last_clean":"2024-09-13T18:26:36.768120+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:26:36.768120+0000","last_undegraded":"2024-09-13T18:26:36.768120+0000","last_fullsized":"2024-09-13T18:26:36.768120+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:02.980826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103481+0000","last_change":"2024-09-13T18:26:36.406986+0000","last_active":"2024-09-13T18:26:37.103481+0000","last_peered":"2024-09-13T18:26:37.103481+0000","last_clean":"2024-09-13T18:26:37.103481+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:26:37.103481+0000","last_undegraded":"2024-09-13T18:26:37.103481+0000","last_fullsized":"2024-09-13T18:26:37.103481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:53.984231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384974+0000","last_change":"2024-09-13T18:26:32.349081+0000","last_active":"2024-09-13T18:26:36.384974+0000","last_peered":"2024-09-13T18:26:36.384974+0000","last_clean":"2024-09-13T18:26:36.384974+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:26:36.384974+0000","last_undegraded":"2024-09-13T18:26:36.384974+0000","last_fullsized":"2024-09-13T18:26:36.384974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:47.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.771163+0000","last_change":"2024-09-13T18:26:31.968944+0000","last_active":"2024-09-13T18:26:36.771163+0000","last_peered":"2024-09-13T18:26:36.771163+0000","last_clean":"2024-09-13T18:26:36.771163+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:26:36.771163+0000","last_undegraded":"2024-09-13T18:26:36.771163+0000","last_fullsized":"2024-09-13T18:26:36.771163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:29.363312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.898021+0000","last_change":"2024-09-13T18:26:36.396418+0000","last_active":"2024-09-13T18:26:36.898021+0000","last_peered":"2024-09-13T18:26:36.898021+0000","last_clean":"2024-09-13T18:26:36.898021+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:26:36.898021+0000","last_undegraded":"2024-09-13T18:26:36.898021+0000","last_fullsized":"2024-09-13T18:26:36.898021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:03.252773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383753+0000","last_change":"2024-09-13T18:26:32.351369+0000","last_active":"2024-09-13T18:26:36.383753+0000","last_peered":"2024-09-13T18:26:36.383753+0000","last_clean":"2024-09-13T18:26:36.383753+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:26:36.383753+0000","last_undegraded":"2024-09-13T18:26:36.383753+0000","last_fullsized":"2024-09-13T18:26:36.383753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:10.892347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.522340+0000","last_change":"2024-09-13T18:26:31.972048+0000","last_active":"2024-09-13T18:27:08.522340+0000","last_peered":"2024-09-13T18:27:08.522340+0000","last_clean":"2024-09-13T18:27:08.522340+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:27:08.522340+0000","last_undegraded":"2024-09-13T18:27:08.522340+0000","last_fullsized":"2024-09-13T18:27:08.522340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:28.430119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.523049+0000","last_change":"2024-09-13T18:26:31.975310+0000","last_active":"2024-09-13T18:27:08.523049+0000","last_peered":"2024-09-13T18:27:08.523049+0000","last_clean":"2024-09-13T18:27:08.523049+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:27:08.523049+0000","last_undegraded":"2024-09-13T18:27:08.523049+0000","last_fullsized":"2024-09-13T18:27:08.523049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:52.915310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388817+0000","last_change":"2024-09-13T18:26:32.345825+0000","last_active":"2024-09-13T18:26:36.388817+0000","last_peered":"2024-09-13T18:26:36.388817+0000","last_clean":"2024-09-13T18:26:36.388817+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:26:36.388817+0000","last_undegraded":"2024-09-13T18:26:36.388817+0000","last_fullsized":"2024-09-13T18:26:36.388817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:25.200489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.395253+0000","last_change":"2024-09-13T18:26:36.391520+0000","last_active":"2024-09-13T18:26:36.395253+0000","last_peered":"2024-09-13T18:26:36.395253+0000","last_clean":"2024-09-13T18:26:36.395253+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:26:36.395253+0000","last_undegraded":"2024-09-13T18:26:36.395253+0000","last_fullsized":"2024-09-13T18:26:36.395253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:42.638713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'6","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.479127+0000","last_change":"2024-09-13T18:26:31.976603+0000","last_active":"2024-09-13T18:27:03.479127+0000","last_peered":"2024-09-13T18:27:03.479127+0000","last_clean":"2024-09-13T18:27:03.479127+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:27:03.479127+0000","last_undegraded":"2024-09-13T18:27:03.479127+0000","last_fullsized":"2024-09-13T18:27:03.479127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:03.965376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1016,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387960+0000","last_change":"2024-09-13T18:26:32.352982+0000","last_active":"2024-09-13T18:26:36.387960+0000","last_peered":"2024-09-13T18:26:36.387960+0000","last_clean":"2024-09-13T18:26:36.387960+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:26:36.387960+0000","last_undegraded":"2024-09-13T18:26:36.387960+0000","last_fullsized":"2024-09-13T18:26:36.387960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:55.670360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769131+0000","last_change":"2024-09-13T18:26:36.403817+0000","last_active":"2024-09-13T18:26:36.769131+0000","last_peered":"2024-09-13T18:26:36.769131+0000","last_clean":"2024-09-13T18:26:36.769131+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:26:36.769131+0000","last_undegraded":"2024-09-13T18:26:36.769131+0000","last_fullsized":"2024-09-13T18:26:36.769131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:34.892901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897728+0000","last_change":"2024-09-13T18:26:32.357856+0000","last_active":"2024-09-13T18:26:36.897728+0000","last_peered":"2024-09-13T18:26:36.897728+0000","last_clean":"2024-09-13T18:26:36.897728+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:26:36.897728+0000","last_undegraded":"2024-09-13T18:26:36.897728+0000","last_fullsized":"2024-09-13T18:26:36.897728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:26.316214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.375208+0000","last_change":"2024-09-13T18:26:32.350966+0000","last_active":"2024-09-13T18:26:36.375208+0000","last_peered":"2024-09-13T18:26:36.375208+0000","last_clean":"2024-09-13T18:26:36.375208+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:26:36.375208+0000","last_undegraded":"2024-09-13T18:26:36.375208+0000","last_fullsized":"2024-09-13T18:26:36.375208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:52.632112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391246+0000","last_change":"2024-09-13T18:26:32.355806+0000","last_active":"2024-09-13T18:26:36.391246+0000","last_peered":"2024-09-13T18:26:36.391246+0000","last_clean":"2024-09-13T18:26:36.391246+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:26:36.391246+0000","last_undegraded":"2024-09-13T18:26:36.391246+0000","last_fullsized":"2024-09-13T18:26:36.391246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.988231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388740+0000","last_change":"2024-09-13T18:26:32.340965+0000","last_active":"2024-09-13T18:26:36.388740+0000","last_peered":"2024-09-13T18:26:36.388740+0000","last_clean":"2024-09-13T18:26:36.388740+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:26:36.388740+0000","last_undegraded":"2024-09-13T18:26:36.388740+0000","last_fullsized":"2024-09-13T18:26:36.388740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:01:04.716919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384429+0000","last_change":"2024-09-13T18:26:32.349625+0000","last_active":"2024-09-13T18:26:36.384429+0000","last_peered":"2024-09-13T18:26:36.384429+0000","last_clean":"2024-09-13T18:26:36.384429+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:26:36.384429+0000","last_undegraded":"2024-09-13T18:26:36.384429+0000","last_fullsized":"2024-09-13T18:26:36.384429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.519754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386397+0000","last_change":"2024-09-13T18:26:32.349170+0000","last_active":"2024-09-13T18:26:36.386397+0000","last_peered":"2024-09-13T18:26:36.386397+0000","last_clean":"2024-09-13T18:26:36.386397+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:26:36.386397+0000","last_undegraded":"2024-09-13T18:26:36.386397+0000","last_fullsized":"2024-09-13T18:26:36.386397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:43.942732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897998+0000","last_change":"2024-09-13T18:26:32.364786+0000","last_active":"2024-09-13T18:26:36.897998+0000","last_peered":"2024-09-13T18:26:36.897998+0000","last_clean":"2024-09-13T18:26:36.897998+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:26:36.897998+0000","last_undegraded":"2024-09-13T18:26:36.897998+0000","last_fullsized":"2024-09-13T18:26:36.897998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:22.949875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387927+0000","last_change":"2024-09-13T18:26:32.347541+0000","last_active":"2024-09-13T18:26:36.387927+0000","last_peered":"2024-09-13T18:26:36.387927+0000","last_clean":"2024-09-13T18:26:36.387927+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:26:36.387927+0000","last_undegraded":"2024-09-13T18:26:36.387927+0000","last_fullsized":"2024-09-13T18:26:36.387927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:58.144128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390892+0000","last_change":"2024-09-13T18:26:32.353359+0000","last_active":"2024-09-13T18:26:36.390892+0000","last_peered":"2024-09-13T18:26:36.390892+0000","last_clean":"2024-09-13T18:26:36.390892+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:26:36.390892+0000","last_undegraded":"2024-09-13T18:26:36.390892+0000","last_fullsized":"2024-09-13T18:26:36.390892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:09.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.380537+0000","last_change":"2024-09-13T18:26:32.349566+0000","last_active":"2024-09-13T18:26:36.380537+0000","last_peered":"2024-09-13T18:26:36.380537+0000","last_clean":"2024-09-13T18:26:36.380537+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:26:36.380537+0000","last_undegraded":"2024-09-13T18:26:36.380537+0000","last_fullsized":"2024-09-13T18:26:36.380537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:22.341899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391168+0000","last_change":"2024-09-13T18:26:32.358640+0000","last_active":"2024-09-13T18:26:36.391168+0000","last_peered":"2024-09-13T18:26:36.391168+0000","last_clean":"2024-09-13T18:26:36.391168+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:26:36.391168+0000","last_undegraded":"2024-09-13T18:26:36.391168+0000","last_fullsized":"2024-09-13T18:26:36.391168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:40.174754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388633+0000","last_change":"2024-09-13T18:26:32.353675+0000","last_active":"2024-09-13T18:26:36.388633+0000","last_peered":"2024-09-13T18:26:36.388633+0000","last_clean":"2024-09-13T18:26:36.388633+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:26:36.388633+0000","last_undegraded":"2024-09-13T18:26:36.388633+0000","last_fullsized":"2024-09-13T18:26:36.388633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:58.501843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897877+0000","last_change":"2024-09-13T18:26:32.353223+0000","last_active":"2024-09-13T18:26:36.897877+0000","last_peered":"2024-09-13T18:26:36.897877+0000","last_clean":"2024-09-13T18:26:36.897877+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:26:36.897877+0000","last_undegraded":"2024-09-13T18:26:36.897877+0000","last_fullsized":"2024-09-13T18:26:36.897877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:45.898213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407736+0000","last_change":"2024-09-13T18:26:32.357354+0000","last_active":"2024-09-13T18:26:36.407736+0000","last_peered":"2024-09-13T18:26:36.407736+0000","last_clean":"2024-09-13T18:26:36.407736+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:26:36.407736+0000","last_undegraded":"2024-09-13T18:26:36.407736+0000","last_fullsized":"2024-09-13T18:26:36.407736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:19.185008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.376351+0000","last_change":"2024-09-13T18:26:32.348832+0000","last_active":"2024-09-13T18:26:36.376351+0000","last_peered":"2024-09-13T18:26:36.376351+0000","last_clean":"2024-09-13T18:26:36.376351+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:26:36.376351+0000","last_undegraded":"2024-09-13T18:26:36.376351+0000","last_fullsized":"2024-09-13T18:26:36.376351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:11:59.874789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393723+0000","last_change":"2024-09-13T18:26:32.356902+0000","last_active":"2024-09-13T18:26:36.393723+0000","last_peered":"2024-09-13T18:26:36.393723+0000","last_clean":"2024-09-13T18:26:36.393723+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:26:36.393723+0000","last_undegraded":"2024-09-13T18:26:36.393723+0000","last_fullsized":"2024-09-13T18:26:36.393723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:18.362772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390031+0000","last_change":"2024-09-13T18:26:32.347047+0000","last_active":"2024-09-13T18:26:36.390031+0000","last_peered":"2024-09-13T18:26:36.390031+0000","last_clean":"2024-09-13T18:26:36.390031+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:26:36.390031+0000","last_undegraded":"2024-09-13T18:26:36.390031+0000","last_fullsized":"2024-09-13T18:26:36.390031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:40.882789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.385066+0000","last_change":"2024-09-13T18:26:32.349651+0000","last_active":"2024-09-13T18:26:36.385066+0000","last_peered":"2024-09-13T18:26:36.385066+0000","last_clean":"2024-09-13T18:26:36.385066+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:26:36.385066+0000","last_undegraded":"2024-09-13T18:26:36.385066+0000","last_fullsized":"2024-09-13T18:26:36.385066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:33.356563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388522+0000","last_change":"2024-09-13T18:26:32.344607+0000","last_active":"2024-09-13T18:26:36.388522+0000","last_peered":"2024-09-13T18:26:36.388522+0000","last_clean":"2024-09-13T18:26:36.388522+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:26:36.388522+0000","last_undegraded":"2024-09-13T18:26:36.388522+0000","last_fullsized":"2024-09-13T18:26:36.388522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:31.658172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387737+0000","last_change":"2024-09-13T18:26:32.347717+0000","last_active":"2024-09-13T18:26:36.387737+0000","last_peered":"2024-09-13T18:26:36.387737+0000","last_clean":"2024-09-13T18:26:36.387737+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:26:36.387737+0000","last_undegraded":"2024-09-13T18:26:36.387737+0000","last_fullsized":"2024-09-13T18:26:36.387737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:47.187837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392104+0000","last_change":"2024-09-13T18:26:32.359506+0000","last_active":"2024-09-13T18:26:36.392104+0000","last_peered":"2024-09-13T18:26:36.392104+0000","last_clean":"2024-09-13T18:26:36.392104+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:26:36.392104+0000","last_undegraded":"2024-09-13T18:26:36.392104+0000","last_fullsized":"2024-09-13T18:26:36.392104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:39:57.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897951+0000","last_change":"2024-09-13T18:26:32.357729+0000","last_active":"2024-09-13T18:26:36.897951+0000","last_peered":"2024-09-13T18:26:36.897951+0000","last_clean":"2024-09-13T18:26:36.897951+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:26:36.897951+0000","last_undegraded":"2024-09-13T18:26:36.897951+0000","last_fullsized":"2024-09-13T18:26:36.897951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:05.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897974+0000","last_change":"2024-09-13T18:26:32.357980+0000","last_active":"2024-09-13T18:26:36.897974+0000","last_peered":"2024-09-13T18:26:36.897974+0000","last_clean":"2024-09-13T18:26:36.897974+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:26:36.897974+0000","last_undegraded":"2024-09-13T18:26:36.897974+0000","last_fullsized":"2024-09-13T18:26:36.897974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:00.858702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405699+0000","last_change":"2024-09-13T18:26:32.365638+0000","last_active":"2024-09-13T18:26:36.405699+0000","last_peered":"2024-09-13T18:26:36.405699+0000","last_clean":"2024-09-13T18:26:36.405699+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:26:36.405699+0000","last_undegraded":"2024-09-13T18:26:36.405699+0000","last_fullsized":"2024-09-13T18:26:36.405699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.065853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383235+0000","last_change":"2024-09-13T18:26:32.345496+0000","last_active":"2024-09-13T18:26:36.383235+0000","last_peered":"2024-09-13T18:26:36.383235+0000","last_clean":"2024-09-13T18:26:36.383235+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:26:36.383235+0000","last_undegraded":"2024-09-13T18:26:36.383235+0000","last_fullsized":"2024-09-13T18:26:36.383235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:22.447315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404895+0000","last_change":"2024-09-13T18:26:32.356901+0000","last_active":"2024-09-13T18:26:36.404895+0000","last_peered":"2024-09-13T18:26:36.404895+0000","last_clean":"2024-09-13T18:26:36.404895+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:26:36.404895+0000","last_undegraded":"2024-09-13T18:26:36.404895+0000","last_fullsized":"2024-09-13T18:26:36.404895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:03.311603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391082+0000","last_change":"2024-09-13T18:26:32.355203+0000","last_active":"2024-09-13T18:26:36.391082+0000","last_peered":"2024-09-13T18:26:36.391082+0000","last_clean":"2024-09-13T18:26:36.391082+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:26:36.391082+0000","last_undegraded":"2024-09-13T18:26:36.391082+0000","last_fullsized":"2024-09-13T18:26:36.391082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:52.769708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405315+0000","last_change":"2024-09-13T18:26:32.358190+0000","last_active":"2024-09-13T18:26:36.405315+0000","last_peered":"2024-09-13T18:26:36.405315+0000","last_clean":"2024-09-13T18:26:36.405315+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:26:36.405315+0000","last_undegraded":"2024-09-13T18:26:36.405315+0000","last_fullsized":"2024-09-13T18:26:36.405315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:35.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393258+0000","last_change":"2024-09-13T18:26:32.355121+0000","last_active":"2024-09-13T18:26:36.393258+0000","last_peered":"2024-09-13T18:26:36.393258+0000","last_clean":"2024-09-13T18:26:36.393258+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:26:36.393258+0000","last_undegraded":"2024-09-13T18:26:36.393258+0000","last_fullsized":"2024-09-13T18:26:36.393258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:48.427332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388810+0000","last_change":"2024-09-13T18:26:32.341497+0000","last_active":"2024-09-13T18:26:36.388810+0000","last_peered":"2024-09-13T18:26:36.388810+0000","last_clean":"2024-09-13T18:26:36.388810+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:26:36.388810+0000","last_undegraded":"2024-09-13T18:26:36.388810+0000","last_fullsized":"2024-09-13T18:26:36.388810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:15.510048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386627+0000","last_change":"2024-09-13T18:26:32.350106+0000","last_active":"2024-09-13T18:26:36.386627+0000","last_peered":"2024-09-13T18:26:36.386627+0000","last_clean":"2024-09-13T18:26:36.386627+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:26:36.386627+0000","last_undegraded":"2024-09-13T18:26:36.386627+0000","last_fullsized":"2024-09-13T18:26:36.386627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:12.387451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405597+0000","last_change":"2024-09-13T18:26:32.362196+0000","last_active":"2024-09-13T18:26:36.405597+0000","last_peered":"2024-09-13T18:26:36.405597+0000","last_clean":"2024-09-13T18:26:36.405597+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:26:36.405597+0000","last_undegraded":"2024-09-13T18:26:36.405597+0000","last_fullsized":"2024-09-13T18:26:36.405597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:14.946904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7751,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":15502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509453,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":107598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509453,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":344064,"data_stored":695251,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]}]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":103839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":108074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":103991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":104551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":104117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":339968,"data_stored":694201,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:27:21.781 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:27:21.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.882+0000 7f0087fff700 1 Processor -- start 2024-09-13T18:27:21.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.882+0000 7f0087fff700 1 -- start start 2024-09-13T18:27:21.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0087fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f00880c6610 0x7f00880cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0087fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00880c6a80 0x7f00880cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0087fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0088153fa0 con 0x7f0088153bc0 2024-09-13T18:27:21.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0087fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00880aa500 con 0x7f00880c6610 2024-09-13T18:27:21.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0087fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008815a7e0 con 0x7f00880c6a80 2024-09-13T18:27:21.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f007e7fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f00880c6610 0x7f00880cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f00877fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00880c6a80 0x7f00880cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f007e7fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f00880c6610 0x7f00880cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:45252/0 (socket says 172.21.15.146:45252) 2024-09-13T18:27:21.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0086ffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0088153bc0 0x7f00880c8de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:54232/0 (socket says 172.21.15.146:54232) 2024-09-13T18:27:21.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f007e7fc700 1 -- 172.21.15.146:0/334996785 learned_addr learned my addr 172.21.15.146:0/334996785 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:21.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f007e7fc700 1 -- 172.21.15.146:0/334996785 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00880c6a80 msgr2=0x7f00880cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:21.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f007e7fc700 1 --2- 172.21.15.146:0/334996785 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00880c6a80 0x7f00880cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f007e7fc700 1 -- 172.21.15.146:0/334996785 >> v1:172.21.15.154:6789/0 conn(0x7f0088153bc0 legacy=0x7f00880c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f007e7fc700 1 -- 172.21.15.146:0/334996785 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880aa220 con 0x7f00880c6610 2024-09-13T18:27:21.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f007e7fc700 1 --2- 172.21.15.146:0/334996785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00880c6610 0x7f00880cac60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0070009910 tx=0x7f0070009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d300db7a6ea95c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:21.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f00867fc700 1 -- 172.21.15.146:0/334996785 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070011830 con 0x7f00880c6610 2024-09-13T18:27:21.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f00867fc700 1 -- 172.21.15.146:0/334996785 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0070011990 con 0x7f00880c6610 2024-09-13T18:27:21.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f00867fc700 1 -- 172.21.15.146:0/334996785 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070011d40 con 0x7f00880c6610 2024-09-13T18:27:21.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0087fff700 1 -- 172.21.15.146:0/334996785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00880c6610 msgr2=0x7f00880cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0087fff700 1 --2- 172.21.15.146:0/334996785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00880c6610 0x7f00880cac60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0070009910 tx=0x7f0070009c20 comp rx=0 tx=0).stop 2024-09-13T18:27:21.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0087fff700 1 -- 172.21.15.146:0/334996785 shutdown_connections 2024-09-13T18:27:21.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0087fff700 1 --2- 172.21.15.146:0/334996785 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00880c6a80 0x7f00880cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0087fff700 1 --2- 172.21.15.146:0/334996785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00880c6610 0x7f00880cac60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0087fff700 1 -- 172.21.15.146:0/334996785 >> 172.21.15.146:0/334996785 conn(0x7f0088066090 msgr2=0x7f0088066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:21.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0087fff700 1 -- 172.21.15.146:0/334996785 shutdown_connections 2024-09-13T18:27:21.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.886+0000 7f0087fff700 1 -- 172.21.15.146:0/334996785 wait complete. 2024-09-13T18:27:21.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f0087fff700 1 Processor -- start 2024-09-13T18:27:21.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f0087fff700 1 -- start start 2024-09-13T18:27:21.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f0087fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00880c6610 0x7f00880ac030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f0087fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00880c6a80 0x7f00881547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f0087fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0088153bc0 0x7f00880af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f0087fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00880ac570 con 0x7f00880c6a80 2024-09-13T18:27:21.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f0087fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0088155330 con 0x7f00880c6610 2024-09-13T18:27:21.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f0087fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0088155050 con 0x7f0088153bc0 2024-09-13T18:27:21.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f007e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00880c6a80 0x7f00881547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f007e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00880c6a80 0x7f00881547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44164/0 (socket says 172.21.15.146:44164) 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f007e7fc700 1 -- 172.21.15.146:0/1854947793 learned_addr learned my addr 172.21.15.146:0/1854947793 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f0086ffd700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00880c6610 0x7f00880ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f00877fe700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0088153bc0 0x7f00880af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f007e7fc700 1 -- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0088153bc0 msgr2=0x7f00880af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f007e7fc700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0088153bc0 0x7f00880af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f007e7fc700 1 -- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00880c6610 msgr2=0x7f00880ac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f007e7fc700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00880c6610 0x7f00880ac030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f007e7fc700 1 -- 172.21.15.146:0/1854947793 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0070009570 con 0x7f00880c6a80 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f007e7fc700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00880c6a80 0x7f00881547c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0070009c60 tx=0x7f0070004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f007ffff700 1 -- 172.21.15.146:0/1854947793 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070004300 con 0x7f00880c6a80 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f007ffff700 1 -- 172.21.15.146:0/1854947793 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007001f070 con 0x7f00880c6a80 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f007ffff700 1 -- 172.21.15.146:0/1854947793 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00700036c0 con 0x7f00880c6a80 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f0087fff700 1 -- 172.21.15.146:0/1854947793 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00880aca20 con 0x7f00880c6a80 2024-09-13T18:27:21.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.890+0000 7f0087fff700 1 -- 172.21.15.146:0/1854947793 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00880acfe0 con 0x7f00880c6a80 2024-09-13T18:27:21.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.894+0000 7f007ffff700 1 -- 172.21.15.146:0/1854947793 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0070021030 con 0x7f00880c6a80 2024-09-13T18:27:21.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.894+0000 7f0087fff700 1 -- 172.21.15.146:0/1854947793 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00880a9de0 con 0x7f00880c6a80 2024-09-13T18:27:21.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.894+0000 7f007ffff700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0068066730 0x7f0068068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:21.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.894+0000 7f007ffff700 1 -- 172.21.15.146:0/1854947793 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0070085820 con 0x7f00880c6a80 2024-09-13T18:27:21.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.894+0000 7f0086ffd700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0068066730 0x7f0068068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:21.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.894+0000 7f0086ffd700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0068066730 0x7f0068068bf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0078000f60 tx=0x7f0078007f30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:21.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:21.898+0000 7f007ffff700 1 -- 172.21.15.146:0/1854947793 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f00700527d0 con 0x7f00880c6a80 2024-09-13T18:27:22.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.066+0000 7f0087fff700 1 -- 172.21.15.146:0/1854947793 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0088159fe0 con 0x7f0068066730 2024-09-13T18:27:22.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.086+0000 7f007ffff700 1 -- 172.21.15.146:0/1854947793 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345369 (secure 0 0 0) 0x7f0078004b30 con 0x7f0068066730 2024-09-13T18:27:22.099 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:22.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.094+0000 7f0087fff700 1 -- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0068066730 msgr2=0x7f0068068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.094+0000 7f0087fff700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0068066730 0x7f0068068bf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0078000f60 tx=0x7f0078007f30 comp rx=0 tx=0).stop 2024-09-13T18:27:22.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.094+0000 7f0087fff700 1 -- 172.21.15.146:0/1854947793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00880c6a80 msgr2=0x7f00881547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.094+0000 7f0087fff700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00880c6a80 0x7f00881547c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0070009c60 tx=0x7f0070004770 comp rx=0 tx=0).stop 2024-09-13T18:27:22.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.094+0000 7f0087fff700 1 -- 172.21.15.146:0/1854947793 shutdown_connections 2024-09-13T18:27:22.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.094+0000 7f0087fff700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0068066730 0x7f0068068bf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.094+0000 7f0087fff700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0088153bc0 0x7f00880af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.094+0000 7f0087fff700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00880c6a80 0x7f00881547c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.094+0000 7f0087fff700 1 --2- 172.21.15.146:0/1854947793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00880c6610 0x7f00880ac030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.094+0000 7f0087fff700 1 -- 172.21.15.146:0/1854947793 >> 172.21.15.146:0/1854947793 conn(0x7f0088066090 msgr2=0x7f00880c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:22.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.094+0000 7f0087fff700 1 -- 172.21.15.146:0/1854947793 shutdown_connections 2024-09-13T18:27:22.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.094+0000 7f0087fff700 1 -- 172.21.15.146:0/1854947793 wait complete. 2024-09-13T18:27:22.103 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:27:22.115 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-09-13T18:27:21.590165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598119,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222528,"kb_used_data":2304,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754752192,"statfs":{"total":773094113280,"available":772866244608,"internally_reserved":0,"allocated":2359296,"data_stored":2021622,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":225431099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1766,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.758288"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.520387+0000","last_change":"2024-09-13T18:26:31.972797+0000","last_active":"2024-09-13T18:27:08.520387+0000","last_peered":"2024-09-13T18:27:08.520387+0000","last_clean":"2024-09-13T18:27:08.520387+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:27:08.520387+0000","last_undegraded":"2024-09-13T18:27:08.520387+0000","last_fullsized":"2024-09-13T18:27:08.520387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:50.511747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384385+0000","last_change":"2024-09-13T18:26:32.349901+0000","last_active":"2024-09-13T18:26:36.384385+0000","last_peered":"2024-09-13T18:26:36.384385+0000","last_clean":"2024-09-13T18:26:36.384385+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:26:36.384385+0000","last_undegraded":"2024-09-13T18:26:36.384385+0000","last_fullsized":"2024-09-13T18:26:36.384385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:24.978813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768058+0000","last_change":"2024-09-13T18:26:36.407579+0000","last_active":"2024-09-13T18:26:36.768058+0000","last_peered":"2024-09-13T18:26:36.768058+0000","last_clean":"2024-09-13T18:26:36.768058+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:26:36.768058+0000","last_undegraded":"2024-09-13T18:26:36.768058+0000","last_fullsized":"2024-09-13T18:26:36.768058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:53:56.370013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343814+0000","last_change":"2024-09-13T18:26:31.980716+0000","last_active":"2024-09-13T18:26:37.343814+0000","last_peered":"2024-09-13T18:26:37.343814+0000","last_clean":"2024-09-13T18:26:37.343814+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:26:37.343814+0000","last_undegraded":"2024-09-13T18:26:37.343814+0000","last_fullsized":"2024-09-13T18:26:37.343814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:20.020903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388711+0000","last_change":"2024-09-13T18:26:32.342690+0000","last_active":"2024-09-13T18:26:36.388711+0000","last_peered":"2024-09-13T18:26:36.388711+0000","last_clean":"2024-09-13T18:26:36.388711+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:26:36.388711+0000","last_undegraded":"2024-09-13T18:26:36.388711+0000","last_fullsized":"2024-09-13T18:26:36.388711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.212930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.398481+0000","last_change":"2024-09-13T18:26:36.396545+0000","last_active":"2024-09-13T18:26:36.398481+0000","last_peered":"2024-09-13T18:26:36.398481+0000","last_clean":"2024-09-13T18:26:36.398481+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:26:36.398481+0000","last_undegraded":"2024-09-13T18:26:36.398481+0000","last_fullsized":"2024-09-13T18:26:36.398481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:38.760215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.523368+0000","last_change":"2024-09-13T18:26:31.977823+0000","last_active":"2024-09-13T18:27:08.523368+0000","last_peered":"2024-09-13T18:27:08.523368+0000","last_clean":"2024-09-13T18:27:08.523368+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:27:08.523368+0000","last_undegraded":"2024-09-13T18:27:08.523368+0000","last_fullsized":"2024-09-13T18:27:08.523368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:46.956527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394225+0000","last_change":"2024-09-13T18:26:32.361288+0000","last_active":"2024-09-13T18:26:36.394225+0000","last_peered":"2024-09-13T18:26:36.394225+0000","last_clean":"2024-09-13T18:26:36.394225+0000","last_became_active":"2024-09-13T18:26:32.360918+0000","last_became_peered":"2024-09-13T18:26:32.360918+0000","last_unstale":"2024-09-13T18:26:36.394225+0000","last_undegraded":"2024-09-13T18:26:36.394225+0000","last_fullsized":"2024-09-13T18:26:36.394225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:30.143436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408364+0000","last_change":"2024-09-13T18:26:36.406675+0000","last_active":"2024-09-13T18:26:36.408364+0000","last_peered":"2024-09-13T18:26:36.408364+0000","last_clean":"2024-09-13T18:26:36.408364+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:26:36.408364+0000","last_undegraded":"2024-09-13T18:26:36.408364+0000","last_fullsized":"2024-09-13T18:26:36.408364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:36.896027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.347551+0000","last_change":"2024-09-13T18:26:31.979860+0000","last_active":"2024-09-13T18:26:37.347551+0000","last_peered":"2024-09-13T18:26:37.347551+0000","last_clean":"2024-09-13T18:26:37.347551+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:26:37.347551+0000","last_undegraded":"2024-09-13T18:26:37.347551+0000","last_fullsized":"2024-09-13T18:26:37.347551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:45.323631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393203+0000","last_change":"2024-09-13T18:26:32.351494+0000","last_active":"2024-09-13T18:26:36.393203+0000","last_peered":"2024-09-13T18:26:36.393203+0000","last_clean":"2024-09-13T18:26:36.393203+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:26:36.393203+0000","last_undegraded":"2024-09-13T18:26:36.393203+0000","last_fullsized":"2024-09-13T18:26:36.393203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:52.804526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400955+0000","last_change":"2024-09-13T18:26:36.398763+0000","last_active":"2024-09-13T18:26:36.400955+0000","last_peered":"2024-09-13T18:26:36.400955+0000","last_clean":"2024-09-13T18:26:36.400955+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:26:36.400955+0000","last_undegraded":"2024-09-13T18:26:36.400955+0000","last_fullsized":"2024-09-13T18:26:36.400955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:35.417438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897834+0000","last_change":"2024-09-13T18:26:32.360976+0000","last_active":"2024-09-13T18:26:36.897834+0000","last_peered":"2024-09-13T18:26:36.897834+0000","last_clean":"2024-09-13T18:26:36.897834+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:26:36.897834+0000","last_undegraded":"2024-09-13T18:26:36.897834+0000","last_fullsized":"2024-09-13T18:26:36.897834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:53:52.373515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.477712+0000","last_change":"2024-09-13T18:26:31.977891+0000","last_active":"2024-09-13T18:27:03.477712+0000","last_peered":"2024-09-13T18:27:03.477712+0000","last_clean":"2024-09-13T18:27:03.477712+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:27:03.477712+0000","last_undegraded":"2024-09-13T18:27:03.477712+0000","last_fullsized":"2024-09-13T18:27:03.477712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:01.453094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400563+0000","last_change":"2024-09-13T18:26:36.397484+0000","last_active":"2024-09-13T18:26:36.400563+0000","last_peered":"2024-09-13T18:26:36.400563+0000","last_clean":"2024-09-13T18:26:36.400563+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:26:36.400563+0000","last_undegraded":"2024-09-13T18:26:36.400563+0000","last_fullsized":"2024-09-13T18:26:36.400563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:44.686244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386958+0000","last_change":"2024-09-13T18:26:32.345564+0000","last_active":"2024-09-13T18:26:36.386958+0000","last_peered":"2024-09-13T18:26:36.386958+0000","last_clean":"2024-09-13T18:26:36.386958+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:26:36.386958+0000","last_undegraded":"2024-09-13T18:26:36.386958+0000","last_fullsized":"2024-09-13T18:26:36.386958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:54.331806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767096+0000","last_change":"2024-09-13T18:26:31.974107+0000","last_active":"2024-09-13T18:26:36.767096+0000","last_peered":"2024-09-13T18:26:36.767096+0000","last_clean":"2024-09-13T18:26:36.767096+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:26:36.767096+0000","last_undegraded":"2024-09-13T18:26:36.767096+0000","last_fullsized":"2024-09-13T18:26:36.767096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:37.547480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408253+0000","last_change":"2024-09-13T18:26:36.405986+0000","last_active":"2024-09-13T18:26:36.408253+0000","last_peered":"2024-09-13T18:26:36.408253+0000","last_clean":"2024-09-13T18:26:36.408253+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:26:36.408253+0000","last_undegraded":"2024-09-13T18:26:36.408253+0000","last_fullsized":"2024-09-13T18:26:36.408253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:30.052430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383772+0000","last_change":"2024-09-13T18:26:32.340616+0000","last_active":"2024-09-13T18:26:36.383772+0000","last_peered":"2024-09-13T18:26:36.383772+0000","last_clean":"2024-09-13T18:26:36.383772+0000","last_became_active":"2024-09-13T18:26:32.339405+0000","last_became_peered":"2024-09-13T18:26:32.339405+0000","last_unstale":"2024-09-13T18:26:36.383772+0000","last_undegraded":"2024-09-13T18:26:36.383772+0000","last_fullsized":"2024-09-13T18:26:36.383772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:26.081896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.474579+0000","last_change":"2024-09-13T18:26:31.968895+0000","last_active":"2024-09-13T18:27:03.474579+0000","last_peered":"2024-09-13T18:27:03.474579+0000","last_clean":"2024-09-13T18:27:03.474579+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:27:03.474579+0000","last_undegraded":"2024-09-13T18:27:03.474579+0000","last_fullsized":"2024-09-13T18:27:03.474579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:15.525097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767834+0000","last_change":"2024-09-13T18:26:36.406484+0000","last_active":"2024-09-13T18:26:36.767834+0000","last_peered":"2024-09-13T18:26:36.767834+0000","last_clean":"2024-09-13T18:26:36.767834+0000","last_became_active":"2024-09-13T18:26:36.406161+0000","last_became_peered":"2024-09-13T18:26:36.406161+0000","last_unstale":"2024-09-13T18:26:36.767834+0000","last_undegraded":"2024-09-13T18:26:36.767834+0000","last_fullsized":"2024-09-13T18:26:36.767834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:53.330919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391435+0000","last_change":"2024-09-13T18:26:32.360406+0000","last_active":"2024-09-13T18:26:36.391435+0000","last_peered":"2024-09-13T18:26:36.391435+0000","last_clean":"2024-09-13T18:26:36.391435+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:26:36.391435+0000","last_undegraded":"2024-09-13T18:26:36.391435+0000","last_fullsized":"2024-09-13T18:26:36.391435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:37.417707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.476272+0000","last_change":"2024-09-13T18:26:31.976872+0000","last_active":"2024-09-13T18:27:03.476272+0000","last_peered":"2024-09-13T18:27:03.476272+0000","last_clean":"2024-09-13T18:27:03.476272+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:27:03.476272+0000","last_undegraded":"2024-09-13T18:27:03.476272+0000","last_fullsized":"2024-09-13T18:27:03.476272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:26.163029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408244+0000","last_change":"2024-09-13T18:26:36.405722+0000","last_active":"2024-09-13T18:26:36.408244+0000","last_peered":"2024-09-13T18:26:36.408244+0000","last_clean":"2024-09-13T18:26:36.408244+0000","last_became_active":"2024-09-13T18:26:36.405310+0000","last_became_peered":"2024-09-13T18:26:36.405310+0000","last_unstale":"2024-09-13T18:26:36.408244+0000","last_undegraded":"2024-09-13T18:26:36.408244+0000","last_fullsized":"2024-09-13T18:26:36.408244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:09.815655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.389080+0000","last_change":"2024-09-13T18:26:32.343915+0000","last_active":"2024-09-13T18:26:36.389080+0000","last_peered":"2024-09-13T18:26:36.389080+0000","last_clean":"2024-09-13T18:26:36.389080+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:26:36.389080+0000","last_undegraded":"2024-09-13T18:26:36.389080+0000","last_fullsized":"2024-09-13T18:26:36.389080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:19.357243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.477810+0000","last_change":"2024-09-13T18:26:31.979944+0000","last_active":"2024-09-13T18:27:03.477810+0000","last_peered":"2024-09-13T18:27:03.477810+0000","last_clean":"2024-09-13T18:27:03.477810+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:27:03.477810+0000","last_undegraded":"2024-09-13T18:27:03.477810+0000","last_fullsized":"2024-09-13T18:27:03.477810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:21.721921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.411958+0000","last_change":"2024-09-13T18:26:36.410028+0000","last_active":"2024-09-13T18:26:36.411958+0000","last_peered":"2024-09-13T18:26:36.411958+0000","last_clean":"2024-09-13T18:26:36.411958+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:26:36.411958+0000","last_undegraded":"2024-09-13T18:26:36.411958+0000","last_fullsized":"2024-09-13T18:26:36.411958+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:32.264341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387875+0000","last_change":"2024-09-13T18:26:32.358980+0000","last_active":"2024-09-13T18:26:36.387875+0000","last_peered":"2024-09-13T18:26:36.387875+0000","last_clean":"2024-09-13T18:26:36.387875+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:26:36.387875+0000","last_undegraded":"2024-09-13T18:26:36.387875+0000","last_fullsized":"2024-09-13T18:26:36.387875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:08.748139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.521355+0000","last_change":"2024-09-13T18:26:31.968972+0000","last_active":"2024-09-13T18:27:08.521355+0000","last_peered":"2024-09-13T18:27:08.521355+0000","last_clean":"2024-09-13T18:27:08.521355+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:27:08.521355+0000","last_undegraded":"2024-09-13T18:27:08.521355+0000","last_fullsized":"2024-09-13T18:27:08.521355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:56.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768011+0000","last_change":"2024-09-13T18:26:36.407034+0000","last_active":"2024-09-13T18:26:36.768011+0000","last_peered":"2024-09-13T18:26:36.768011+0000","last_clean":"2024-09-13T18:26:36.768011+0000","last_became_active":"2024-09-13T18:26:36.406473+0000","last_became_peered":"2024-09-13T18:26:36.406473+0000","last_unstale":"2024-09-13T18:26:36.768011+0000","last_undegraded":"2024-09-13T18:26:36.768011+0000","last_fullsized":"2024-09-13T18:26:36.768011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:53.815901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386420+0000","last_change":"2024-09-13T18:26:32.342823+0000","last_active":"2024-09-13T18:26:36.386420+0000","last_peered":"2024-09-13T18:26:36.386420+0000","last_clean":"2024-09-13T18:26:36.386420+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:26:36.386420+0000","last_undegraded":"2024-09-13T18:26:36.386420+0000","last_fullsized":"2024-09-13T18:26:36.386420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:28.051252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.349179+0000","last_change":"2024-09-13T18:26:31.978976+0000","last_active":"2024-09-13T18:26:37.349179+0000","last_peered":"2024-09-13T18:26:37.349179+0000","last_clean":"2024-09-13T18:26:37.349179+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:26:37.349179+0000","last_undegraded":"2024-09-13T18:26:37.349179+0000","last_fullsized":"2024-09-13T18:26:37.349179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:11.279557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103455+0000","last_change":"2024-09-13T18:26:36.408844+0000","last_active":"2024-09-13T18:26:37.103455+0000","last_peered":"2024-09-13T18:26:37.103455+0000","last_clean":"2024-09-13T18:26:37.103455+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:26:37.103455+0000","last_undegraded":"2024-09-13T18:26:37.103455+0000","last_fullsized":"2024-09-13T18:26:37.103455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:06.969342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393043+0000","last_change":"2024-09-13T18:26:32.355038+0000","last_active":"2024-09-13T18:26:36.393043+0000","last_peered":"2024-09-13T18:26:36.393043+0000","last_clean":"2024-09-13T18:26:36.393043+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:26:36.393043+0000","last_undegraded":"2024-09-13T18:26:36.393043+0000","last_fullsized":"2024-09-13T18:26:36.393043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.159231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386562+0000","last_change":"2024-09-13T18:26:31.972931+0000","last_active":"2024-09-13T18:26:36.386562+0000","last_peered":"2024-09-13T18:26:36.386562+0000","last_clean":"2024-09-13T18:26:36.386562+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:26:36.386562+0000","last_undegraded":"2024-09-13T18:26:36.386562+0000","last_fullsized":"2024-09-13T18:26:36.386562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:05.646924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766310+0000","last_change":"2024-09-13T18:26:36.398876+0000","last_active":"2024-09-13T18:26:36.766310+0000","last_peered":"2024-09-13T18:26:36.766310+0000","last_clean":"2024-09-13T18:26:36.766310+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:26:36.766310+0000","last_undegraded":"2024-09-13T18:26:36.766310+0000","last_fullsized":"2024-09-13T18:26:36.766310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:56.918239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392625+0000","last_change":"2024-09-13T18:26:32.353111+0000","last_active":"2024-09-13T18:26:36.392625+0000","last_peered":"2024-09-13T18:26:36.392625+0000","last_clean":"2024-09-13T18:26:36.392625+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:26:36.392625+0000","last_undegraded":"2024-09-13T18:26:36.392625+0000","last_fullsized":"2024-09-13T18:26:36.392625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:43.927936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.772320+0000","last_change":"2024-09-13T18:26:31.976075+0000","last_active":"2024-09-13T18:26:36.772320+0000","last_peered":"2024-09-13T18:26:36.772320+0000","last_clean":"2024-09-13T18:26:36.772320+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:26:36.772320+0000","last_undegraded":"2024-09-13T18:26:36.772320+0000","last_fullsized":"2024-09-13T18:26:36.772320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:03.693518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766877+0000","last_change":"2024-09-13T18:26:36.401777+0000","last_active":"2024-09-13T18:26:36.766877+0000","last_peered":"2024-09-13T18:26:36.766877+0000","last_clean":"2024-09-13T18:26:36.766877+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:26:36.766877+0000","last_undegraded":"2024-09-13T18:26:36.766877+0000","last_fullsized":"2024-09-13T18:26:36.766877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:43.440284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384991+0000","last_change":"2024-09-13T18:26:32.343791+0000","last_active":"2024-09-13T18:26:36.384991+0000","last_peered":"2024-09-13T18:26:36.384991+0000","last_clean":"2024-09-13T18:26:36.384991+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:26:36.384991+0000","last_undegraded":"2024-09-13T18:26:36.384991+0000","last_fullsized":"2024-09-13T18:26:36.384991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:36.296571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.521069+0000","last_change":"2024-09-13T18:26:31.970786+0000","last_active":"2024-09-13T18:27:08.521069+0000","last_peered":"2024-09-13T18:27:08.521069+0000","last_clean":"2024-09-13T18:27:08.521069+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:27:08.521069+0000","last_undegraded":"2024-09-13T18:27:08.521069+0000","last_fullsized":"2024-09-13T18:27:08.521069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:54.584009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767929+0000","last_change":"2024-09-13T18:26:36.401529+0000","last_active":"2024-09-13T18:26:36.767929+0000","last_peered":"2024-09-13T18:26:36.767929+0000","last_clean":"2024-09-13T18:26:36.767929+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:26:36.767929+0000","last_undegraded":"2024-09-13T18:26:36.767929+0000","last_fullsized":"2024-09-13T18:26:36.767929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:47.318347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.382662+0000","last_change":"2024-09-13T18:26:32.350470+0000","last_active":"2024-09-13T18:26:36.382662+0000","last_peered":"2024-09-13T18:26:36.382662+0000","last_clean":"2024-09-13T18:26:36.382662+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:26:36.382662+0000","last_undegraded":"2024-09-13T18:26:36.382662+0000","last_fullsized":"2024-09-13T18:26:36.382662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:55.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.522206+0000","last_change":"2024-09-13T18:26:31.972587+0000","last_active":"2024-09-13T18:27:08.522206+0000","last_peered":"2024-09-13T18:27:08.522206+0000","last_clean":"2024-09-13T18:27:08.522206+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:27:08.522206+0000","last_undegraded":"2024-09-13T18:27:08.522206+0000","last_fullsized":"2024-09-13T18:27:08.522206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:27.547763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103735+0000","last_change":"2024-09-13T18:26:36.405086+0000","last_active":"2024-09-13T18:26:37.103735+0000","last_peered":"2024-09-13T18:26:37.103735+0000","last_clean":"2024-09-13T18:26:37.103735+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:26:37.103735+0000","last_undegraded":"2024-09-13T18:26:37.103735+0000","last_fullsized":"2024-09-13T18:26:37.103735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:02:51.694634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388036+0000","last_change":"2024-09-13T18:26:32.350510+0000","last_active":"2024-09-13T18:26:36.388036+0000","last_peered":"2024-09-13T18:26:36.388036+0000","last_clean":"2024-09-13T18:26:36.388036+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:26:36.388036+0000","last_undegraded":"2024-09-13T18:26:36.388036+0000","last_fullsized":"2024-09-13T18:26:36.388036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:40.850255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.479714+0000","last_change":"2024-09-13T18:26:31.978355+0000","last_active":"2024-09-13T18:27:03.479714+0000","last_peered":"2024-09-13T18:27:03.479714+0000","last_clean":"2024-09-13T18:27:03.479714+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:27:03.479714+0000","last_undegraded":"2024-09-13T18:27:03.479714+0000","last_fullsized":"2024-09-13T18:27:03.479714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:12.946518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404778+0000","last_change":"2024-09-13T18:26:26.931403+0000","last_active":"2024-09-13T18:26:36.404778+0000","last_peered":"2024-09-13T18:26:36.404778+0000","last_clean":"2024-09-13T18:26:36.404778+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:26:36.404778+0000","last_undegraded":"2024-09-13T18:26:36.404778+0000","last_fullsized":"2024-09-13T18:26:36.404778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_clean_scrub_stamp":"2024-09-13T18:26:24.880028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:30.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897926+0000","last_change":"2024-09-13T18:26:36.409827+0000","last_active":"2024-09-13T18:26:36.897926+0000","last_peered":"2024-09-13T18:26:36.897926+0000","last_clean":"2024-09-13T18:26:36.897926+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:26:36.897926+0000","last_undegraded":"2024-09-13T18:26:36.897926+0000","last_fullsized":"2024-09-13T18:26:36.897926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:50.482400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405746+0000","last_change":"2024-09-13T18:26:32.356850+0000","last_active":"2024-09-13T18:26:36.405746+0000","last_peered":"2024-09-13T18:26:36.405746+0000","last_clean":"2024-09-13T18:26:36.405746+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:26:36.405746+0000","last_undegraded":"2024-09-13T18:26:36.405746+0000","last_fullsized":"2024-09-13T18:26:36.405746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:38.710828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.520921+0000","last_change":"2024-09-13T18:26:31.975942+0000","last_active":"2024-09-13T18:27:08.520921+0000","last_peered":"2024-09-13T18:27:08.520921+0000","last_clean":"2024-09-13T18:27:08.520921+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:27:08.520921+0000","last_undegraded":"2024-09-13T18:27:08.520921+0000","last_fullsized":"2024-09-13T18:27:08.520921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:05.840582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897805+0000","last_change":"2024-09-13T18:26:36.409802+0000","last_active":"2024-09-13T18:26:36.897805+0000","last_peered":"2024-09-13T18:26:36.897805+0000","last_clean":"2024-09-13T18:26:36.897805+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:26:36.897805+0000","last_undegraded":"2024-09-13T18:26:36.897805+0000","last_fullsized":"2024-09-13T18:26:36.897805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:38.178151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394309+0000","last_change":"2024-09-13T18:26:32.355181+0000","last_active":"2024-09-13T18:26:36.394309+0000","last_peered":"2024-09-13T18:26:36.394309+0000","last_clean":"2024-09-13T18:26:36.394309+0000","last_became_active":"2024-09-13T18:26:32.353043+0000","last_became_peered":"2024-09-13T18:26:32.353043+0000","last_unstale":"2024-09-13T18:26:36.394309+0000","last_undegraded":"2024-09-13T18:26:36.394309+0000","last_fullsized":"2024-09-13T18:26:36.394309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:04:17.000915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767488+0000","last_change":"2024-09-13T18:26:31.966618+0000","last_active":"2024-09-13T18:26:36.767488+0000","last_peered":"2024-09-13T18:26:36.767488+0000","last_clean":"2024-09-13T18:26:36.767488+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:26:36.767488+0000","last_undegraded":"2024-09-13T18:26:36.767488+0000","last_fullsized":"2024-09-13T18:26:36.767488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:54:26.422611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103645+0000","last_change":"2024-09-13T18:26:36.394344+0000","last_active":"2024-09-13T18:26:37.103645+0000","last_peered":"2024-09-13T18:26:37.103645+0000","last_clean":"2024-09-13T18:26:37.103645+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:26:37.103645+0000","last_undegraded":"2024-09-13T18:26:37.103645+0000","last_fullsized":"2024-09-13T18:26:37.103645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:18.425816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405784+0000","last_change":"2024-09-13T18:26:32.365655+0000","last_active":"2024-09-13T18:26:36.405784+0000","last_peered":"2024-09-13T18:26:36.405784+0000","last_clean":"2024-09-13T18:26:36.405784+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:26:36.405784+0000","last_undegraded":"2024-09-13T18:26:36.405784+0000","last_fullsized":"2024-09-13T18:26:36.405784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:40.288128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343912+0000","last_change":"2024-09-13T18:26:31.970652+0000","last_active":"2024-09-13T18:26:37.343912+0000","last_peered":"2024-09-13T18:26:37.343912+0000","last_clean":"2024-09-13T18:26:37.343912+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:26:37.343912+0000","last_undegraded":"2024-09-13T18:26:37.343912+0000","last_fullsized":"2024-09-13T18:26:37.343912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:11.169661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.770631+0000","last_change":"2024-09-13T18:26:36.410280+0000","last_active":"2024-09-13T18:26:36.770631+0000","last_peered":"2024-09-13T18:26:36.770631+0000","last_clean":"2024-09-13T18:26:36.770631+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:26:36.770631+0000","last_undegraded":"2024-09-13T18:26:36.770631+0000","last_fullsized":"2024-09-13T18:26:36.770631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:01.395290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405642+0000","last_change":"2024-09-13T18:26:32.361575+0000","last_active":"2024-09-13T18:26:36.405642+0000","last_peered":"2024-09-13T18:26:36.405642+0000","last_clean":"2024-09-13T18:26:36.405642+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:26:36.405642+0000","last_undegraded":"2024-09-13T18:26:36.405642+0000","last_fullsized":"2024-09-13T18:26:36.405642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:08.818877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"25'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.346552+0000","last_change":"2024-09-13T18:26:31.977029+0000","last_active":"2024-09-13T18:26:37.346552+0000","last_peered":"2024-09-13T18:26:37.346552+0000","last_clean":"2024-09-13T18:26:37.346552+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:26:37.346552+0000","last_undegraded":"2024-09-13T18:26:37.346552+0000","last_fullsized":"2024-09-13T18:26:37.346552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:13.701689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407565+0000","last_change":"2024-09-13T18:26:36.403479+0000","last_active":"2024-09-13T18:26:36.407565+0000","last_peered":"2024-09-13T18:26:36.407565+0000","last_clean":"2024-09-13T18:26:36.407565+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:26:36.407565+0000","last_undegraded":"2024-09-13T18:26:36.407565+0000","last_fullsized":"2024-09-13T18:26:36.407565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:51.095852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404976+0000","last_change":"2024-09-13T18:26:32.365143+0000","last_active":"2024-09-13T18:26:36.404976+0000","last_peered":"2024-09-13T18:26:36.404976+0000","last_clean":"2024-09-13T18:26:36.404976+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:26:36.404976+0000","last_undegraded":"2024-09-13T18:26:36.404976+0000","last_fullsized":"2024-09-13T18:26:36.404976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:46.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.526361+0000","last_change":"2024-09-13T18:26:31.965802+0000","last_active":"2024-09-13T18:27:08.526361+0000","last_peered":"2024-09-13T18:27:08.526361+0000","last_clean":"2024-09-13T18:27:08.526361+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:27:08.526361+0000","last_undegraded":"2024-09-13T18:27:08.526361+0000","last_fullsized":"2024-09-13T18:27:08.526361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:12.759181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103768+0000","last_change":"2024-09-13T18:26:36.399235+0000","last_active":"2024-09-13T18:26:37.103768+0000","last_peered":"2024-09-13T18:26:37.103768+0000","last_clean":"2024-09-13T18:26:37.103768+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:26:37.103768+0000","last_undegraded":"2024-09-13T18:26:37.103768+0000","last_fullsized":"2024-09-13T18:26:37.103768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:13:54.711083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897760+0000","last_change":"2024-09-13T18:26:32.357403+0000","last_active":"2024-09-13T18:26:36.897760+0000","last_peered":"2024-09-13T18:26:36.897760+0000","last_clean":"2024-09-13T18:26:36.897760+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:26:36.897760+0000","last_undegraded":"2024-09-13T18:26:36.897760+0000","last_fullsized":"2024-09-13T18:26:36.897760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:12.600397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.525532+0000","last_change":"2024-09-13T18:26:31.976148+0000","last_active":"2024-09-13T18:27:08.525532+0000","last_peered":"2024-09-13T18:27:08.525532+0000","last_clean":"2024-09-13T18:27:08.525532+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:27:08.525532+0000","last_undegraded":"2024-09-13T18:27:08.525532+0000","last_fullsized":"2024-09-13T18:27:08.525532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:38.023636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769582+0000","last_change":"2024-09-13T18:26:36.406177+0000","last_active":"2024-09-13T18:26:36.769582+0000","last_peered":"2024-09-13T18:26:36.769582+0000","last_clean":"2024-09-13T18:26:36.769582+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:26:36.769582+0000","last_undegraded":"2024-09-13T18:26:36.769582+0000","last_fullsized":"2024-09-13T18:26:36.769582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:00.309995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405459+0000","last_change":"2024-09-13T18:26:32.361726+0000","last_active":"2024-09-13T18:26:36.405459+0000","last_peered":"2024-09-13T18:26:36.405459+0000","last_clean":"2024-09-13T18:26:36.405459+0000","last_became_active":"2024-09-13T18:26:32.361432+0000","last_became_peered":"2024-09-13T18:26:32.361432+0000","last_unstale":"2024-09-13T18:26:36.405459+0000","last_undegraded":"2024-09-13T18:26:36.405459+0000","last_fullsized":"2024-09-13T18:26:36.405459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:42.021515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.480098+0000","last_change":"2024-09-13T18:26:31.982114+0000","last_active":"2024-09-13T18:27:03.480098+0000","last_peered":"2024-09-13T18:27:03.480098+0000","last_clean":"2024-09-13T18:27:03.480098+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:27:03.480098+0000","last_undegraded":"2024-09-13T18:27:03.480098+0000","last_fullsized":"2024-09-13T18:27:03.480098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.266528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769049+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:26:36.769049+0000","last_peered":"2024-09-13T18:26:36.769049+0000","last_clean":"2024-09-13T18:26:36.769049+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:26:36.769049+0000","last_undegraded":"2024-09-13T18:26:36.769049+0000","last_fullsized":"2024-09-13T18:26:36.769049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:58.853971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388651+0000","last_change":"2024-09-13T18:26:32.351145+0000","last_active":"2024-09-13T18:26:36.388651+0000","last_peered":"2024-09-13T18:26:36.388651+0000","last_clean":"2024-09-13T18:26:36.388651+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:26:36.388651+0000","last_undegraded":"2024-09-13T18:26:36.388651+0000","last_fullsized":"2024-09-13T18:26:36.388651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:57.002435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.519027+0000","last_change":"2024-09-13T18:26:31.961202+0000","last_active":"2024-09-13T18:27:08.519027+0000","last_peered":"2024-09-13T18:27:08.519027+0000","last_clean":"2024-09-13T18:27:08.519027+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:27:08.519027+0000","last_undegraded":"2024-09-13T18:27:08.519027+0000","last_fullsized":"2024-09-13T18:27:08.519027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:27.340127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.772209+0000","last_change":"2024-09-13T18:26:36.410014+0000","last_active":"2024-09-13T18:26:36.772209+0000","last_peered":"2024-09-13T18:26:36.772209+0000","last_clean":"2024-09-13T18:26:36.772209+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:26:36.772209+0000","last_undegraded":"2024-09-13T18:26:36.772209+0000","last_fullsized":"2024-09-13T18:26:36.772209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:24.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405669+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:26:36.405669+0000","last_peered":"2024-09-13T18:26:36.405669+0000","last_clean":"2024-09-13T18:26:36.405669+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:26:36.405669+0000","last_undegraded":"2024-09-13T18:26:36.405669+0000","last_fullsized":"2024-09-13T18:26:36.405669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:40.102815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'11","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.474634+0000","last_change":"2024-09-13T18:26:31.974110+0000","last_active":"2024-09-13T18:27:03.474634+0000","last_peered":"2024-09-13T18:27:03.474634+0000","last_clean":"2024-09-13T18:27:03.474634+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:27:03.474634+0000","last_undegraded":"2024-09-13T18:27:03.474634+0000","last_fullsized":"2024-09-13T18:27:03.474634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:56.058561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766576+0000","last_change":"2024-09-13T18:26:36.394080+0000","last_active":"2024-09-13T18:26:36.766576+0000","last_peered":"2024-09-13T18:26:36.766576+0000","last_clean":"2024-09-13T18:26:36.766576+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:26:36.766576+0000","last_undegraded":"2024-09-13T18:26:36.766576+0000","last_fullsized":"2024-09-13T18:26:36.766576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:47.121814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392195+0000","last_change":"2024-09-13T18:26:32.362873+0000","last_active":"2024-09-13T18:26:36.392195+0000","last_peered":"2024-09-13T18:26:36.392195+0000","last_clean":"2024-09-13T18:26:36.392195+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:26:36.392195+0000","last_undegraded":"2024-09-13T18:26:36.392195+0000","last_fullsized":"2024-09-13T18:26:36.392195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:37.598189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.344708+0000","last_change":"2024-09-13T18:26:31.976401+0000","last_active":"2024-09-13T18:26:37.344708+0000","last_peered":"2024-09-13T18:26:37.344708+0000","last_clean":"2024-09-13T18:26:37.344708+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:26:37.344708+0000","last_undegraded":"2024-09-13T18:26:37.344708+0000","last_fullsized":"2024-09-13T18:26:37.344708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:26.434292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408317+0000","last_change":"2024-09-13T18:26:36.406228+0000","last_active":"2024-09-13T18:26:36.408317+0000","last_peered":"2024-09-13T18:26:36.408317+0000","last_clean":"2024-09-13T18:26:36.408317+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:26:36.408317+0000","last_undegraded":"2024-09-13T18:26:36.408317+0000","last_fullsized":"2024-09-13T18:26:36.408317+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:49.198397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388733+0000","last_change":"2024-09-13T18:26:32.350841+0000","last_active":"2024-09-13T18:26:36.388733+0000","last_peered":"2024-09-13T18:26:36.388733+0000","last_clean":"2024-09-13T18:26:36.388733+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:26:36.388733+0000","last_undegraded":"2024-09-13T18:26:36.388733+0000","last_fullsized":"2024-09-13T18:26:36.388733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:05.137882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.476898+0000","last_change":"2024-09-13T18:26:31.978214+0000","last_active":"2024-09-13T18:27:03.476898+0000","last_peered":"2024-09-13T18:27:03.476898+0000","last_clean":"2024-09-13T18:27:03.476898+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:27:03.476898+0000","last_undegraded":"2024-09-13T18:27:03.476898+0000","last_fullsized":"2024-09-13T18:27:03.476898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:17.381375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768498+0000","last_change":"2024-09-13T18:26:36.406090+0000","last_active":"2024-09-13T18:26:36.768498+0000","last_peered":"2024-09-13T18:26:36.768498+0000","last_clean":"2024-09-13T18:26:36.768498+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:26:36.768498+0000","last_undegraded":"2024-09-13T18:26:36.768498+0000","last_fullsized":"2024-09-13T18:26:36.768498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:09.865147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391759+0000","last_change":"2024-09-13T18:26:32.357701+0000","last_active":"2024-09-13T18:26:36.391759+0000","last_peered":"2024-09-13T18:26:36.391759+0000","last_clean":"2024-09-13T18:26:36.391759+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:26:36.391759+0000","last_undegraded":"2024-09-13T18:26:36.391759+0000","last_fullsized":"2024-09-13T18:26:36.391759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:47.587683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.479895+0000","last_change":"2024-09-13T18:26:31.980133+0000","last_active":"2024-09-13T18:27:03.479895+0000","last_peered":"2024-09-13T18:27:03.479895+0000","last_clean":"2024-09-13T18:27:03.479895+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:27:03.479895+0000","last_undegraded":"2024-09-13T18:27:03.479895+0000","last_fullsized":"2024-09-13T18:27:03.479895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:05.354335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768120+0000","last_change":"2024-09-13T18:26:36.406553+0000","last_active":"2024-09-13T18:26:36.768120+0000","last_peered":"2024-09-13T18:26:36.768120+0000","last_clean":"2024-09-13T18:26:36.768120+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:26:36.768120+0000","last_undegraded":"2024-09-13T18:26:36.768120+0000","last_fullsized":"2024-09-13T18:26:36.768120+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:02.980826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103481+0000","last_change":"2024-09-13T18:26:36.406986+0000","last_active":"2024-09-13T18:26:37.103481+0000","last_peered":"2024-09-13T18:26:37.103481+0000","last_clean":"2024-09-13T18:26:37.103481+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:26:37.103481+0000","last_undegraded":"2024-09-13T18:26:37.103481+0000","last_fullsized":"2024-09-13T18:26:37.103481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:53.984231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384974+0000","last_change":"2024-09-13T18:26:32.349081+0000","last_active":"2024-09-13T18:26:36.384974+0000","last_peered":"2024-09-13T18:26:36.384974+0000","last_clean":"2024-09-13T18:26:36.384974+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:26:36.384974+0000","last_undegraded":"2024-09-13T18:26:36.384974+0000","last_fullsized":"2024-09-13T18:26:36.384974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:47.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.771163+0000","last_change":"2024-09-13T18:26:31.968944+0000","last_active":"2024-09-13T18:26:36.771163+0000","last_peered":"2024-09-13T18:26:36.771163+0000","last_clean":"2024-09-13T18:26:36.771163+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:26:36.771163+0000","last_undegraded":"2024-09-13T18:26:36.771163+0000","last_fullsized":"2024-09-13T18:26:36.771163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:29.363312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.898021+0000","last_change":"2024-09-13T18:26:36.396418+0000","last_active":"2024-09-13T18:26:36.898021+0000","last_peered":"2024-09-13T18:26:36.898021+0000","last_clean":"2024-09-13T18:26:36.898021+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:26:36.898021+0000","last_undegraded":"2024-09-13T18:26:36.898021+0000","last_fullsized":"2024-09-13T18:26:36.898021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:03.252773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383753+0000","last_change":"2024-09-13T18:26:32.351369+0000","last_active":"2024-09-13T18:26:36.383753+0000","last_peered":"2024-09-13T18:26:36.383753+0000","last_clean":"2024-09-13T18:26:36.383753+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:26:36.383753+0000","last_undegraded":"2024-09-13T18:26:36.383753+0000","last_fullsized":"2024-09-13T18:26:36.383753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:10.892347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.522340+0000","last_change":"2024-09-13T18:26:31.972048+0000","last_active":"2024-09-13T18:27:08.522340+0000","last_peered":"2024-09-13T18:27:08.522340+0000","last_clean":"2024-09-13T18:27:08.522340+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:27:08.522340+0000","last_undegraded":"2024-09-13T18:27:08.522340+0000","last_fullsized":"2024-09-13T18:27:08.522340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:28.430119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.523049+0000","last_change":"2024-09-13T18:26:31.975310+0000","last_active":"2024-09-13T18:27:08.523049+0000","last_peered":"2024-09-13T18:27:08.523049+0000","last_clean":"2024-09-13T18:27:08.523049+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:27:08.523049+0000","last_undegraded":"2024-09-13T18:27:08.523049+0000","last_fullsized":"2024-09-13T18:27:08.523049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:52.915310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388817+0000","last_change":"2024-09-13T18:26:32.345825+0000","last_active":"2024-09-13T18:26:36.388817+0000","last_peered":"2024-09-13T18:26:36.388817+0000","last_clean":"2024-09-13T18:26:36.388817+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:26:36.388817+0000","last_undegraded":"2024-09-13T18:26:36.388817+0000","last_fullsized":"2024-09-13T18:26:36.388817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:25.200489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.395253+0000","last_change":"2024-09-13T18:26:36.391520+0000","last_active":"2024-09-13T18:26:36.395253+0000","last_peered":"2024-09-13T18:26:36.395253+0000","last_clean":"2024-09-13T18:26:36.395253+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:26:36.395253+0000","last_undegraded":"2024-09-13T18:26:36.395253+0000","last_fullsized":"2024-09-13T18:26:36.395253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:42.638713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'6","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.479127+0000","last_change":"2024-09-13T18:26:31.976603+0000","last_active":"2024-09-13T18:27:03.479127+0000","last_peered":"2024-09-13T18:27:03.479127+0000","last_clean":"2024-09-13T18:27:03.479127+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:27:03.479127+0000","last_undegraded":"2024-09-13T18:27:03.479127+0000","last_fullsized":"2024-09-13T18:27:03.479127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:03.965376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1016,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387960+0000","last_change":"2024-09-13T18:26:32.352982+0000","last_active":"2024-09-13T18:26:36.387960+0000","last_peered":"2024-09-13T18:26:36.387960+0000","last_clean":"2024-09-13T18:26:36.387960+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:26:36.387960+0000","last_undegraded":"2024-09-13T18:26:36.387960+0000","last_fullsized":"2024-09-13T18:26:36.387960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:55.670360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769131+0000","last_change":"2024-09-13T18:26:36.403817+0000","last_active":"2024-09-13T18:26:36.769131+0000","last_peered":"2024-09-13T18:26:36.769131+0000","last_clean":"2024-09-13T18:26:36.769131+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:26:36.769131+0000","last_undegraded":"2024-09-13T18:26:36.769131+0000","last_fullsized":"2024-09-13T18:26:36.769131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:34.892901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897728+0000","last_change":"2024-09-13T18:26:32.357856+0000","last_active":"2024-09-13T18:26:36.897728+0000","last_peered":"2024-09-13T18:26:36.897728+0000","last_clean":"2024-09-13T18:26:36.897728+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:26:36.897728+0000","last_undegraded":"2024-09-13T18:26:36.897728+0000","last_fullsized":"2024-09-13T18:26:36.897728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:26.316214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.375208+0000","last_change":"2024-09-13T18:26:32.350966+0000","last_active":"2024-09-13T18:26:36.375208+0000","last_peered":"2024-09-13T18:26:36.375208+0000","last_clean":"2024-09-13T18:26:36.375208+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:26:36.375208+0000","last_undegraded":"2024-09-13T18:26:36.375208+0000","last_fullsized":"2024-09-13T18:26:36.375208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:52.632112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391246+0000","last_change":"2024-09-13T18:26:32.355806+0000","last_active":"2024-09-13T18:26:36.391246+0000","last_peered":"2024-09-13T18:26:36.391246+0000","last_clean":"2024-09-13T18:26:36.391246+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:26:36.391246+0000","last_undegraded":"2024-09-13T18:26:36.391246+0000","last_fullsized":"2024-09-13T18:26:36.391246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.988231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388740+0000","last_change":"2024-09-13T18:26:32.340965+0000","last_active":"2024-09-13T18:26:36.388740+0000","last_peered":"2024-09-13T18:26:36.388740+0000","last_clean":"2024-09-13T18:26:36.388740+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:26:36.388740+0000","last_undegraded":"2024-09-13T18:26:36.388740+0000","last_fullsized":"2024-09-13T18:26:36.388740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:01:04.716919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384429+0000","last_change":"2024-09-13T18:26:32.349625+0000","last_active":"2024-09-13T18:26:36.384429+0000","last_peered":"2024-09-13T18:26:36.384429+0000","last_clean":"2024-09-13T18:26:36.384429+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:26:36.384429+0000","last_undegraded":"2024-09-13T18:26:36.384429+0000","last_fullsized":"2024-09-13T18:26:36.384429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.519754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386397+0000","last_change":"2024-09-13T18:26:32.349170+0000","last_active":"2024-09-13T18:26:36.386397+0000","last_peered":"2024-09-13T18:26:36.386397+0000","last_clean":"2024-09-13T18:26:36.386397+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:26:36.386397+0000","last_undegraded":"2024-09-13T18:26:36.386397+0000","last_fullsized":"2024-09-13T18:26:36.386397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:43.942732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897998+0000","last_change":"2024-09-13T18:26:32.364786+0000","last_active":"2024-09-13T18:26:36.897998+0000","last_peered":"2024-09-13T18:26:36.897998+0000","last_clean":"2024-09-13T18:26:36.897998+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:26:36.897998+0000","last_undegraded":"2024-09-13T18:26:36.897998+0000","last_fullsized":"2024-09-13T18:26:36.897998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:22.949875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387927+0000","last_change":"2024-09-13T18:26:32.347541+0000","last_active":"2024-09-13T18:26:36.387927+0000","last_peered":"2024-09-13T18:26:36.387927+0000","last_clean":"2024-09-13T18:26:36.387927+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:26:36.387927+0000","last_undegraded":"2024-09-13T18:26:36.387927+0000","last_fullsized":"2024-09-13T18:26:36.387927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:58.144128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390892+0000","last_change":"2024-09-13T18:26:32.353359+0000","last_active":"2024-09-13T18:26:36.390892+0000","last_peered":"2024-09-13T18:26:36.390892+0000","last_clean":"2024-09-13T18:26:36.390892+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:26:36.390892+0000","last_undegraded":"2024-09-13T18:26:36.390892+0000","last_fullsized":"2024-09-13T18:26:36.390892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:09.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.380537+0000","last_change":"2024-09-13T18:26:32.349566+0000","last_active":"2024-09-13T18:26:36.380537+0000","last_peered":"2024-09-13T18:26:36.380537+0000","last_clean":"2024-09-13T18:26:36.380537+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:26:36.380537+0000","last_undegraded":"2024-09-13T18:26:36.380537+0000","last_fullsized":"2024-09-13T18:26:36.380537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:22.341899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391168+0000","last_change":"2024-09-13T18:26:32.358640+0000","last_active":"2024-09-13T18:26:36.391168+0000","last_peered":"2024-09-13T18:26:36.391168+0000","last_clean":"2024-09-13T18:26:36.391168+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:26:36.391168+0000","last_undegraded":"2024-09-13T18:26:36.391168+0000","last_fullsized":"2024-09-13T18:26:36.391168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:40.174754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388633+0000","last_change":"2024-09-13T18:26:32.353675+0000","last_active":"2024-09-13T18:26:36.388633+0000","last_peered":"2024-09-13T18:26:36.388633+0000","last_clean":"2024-09-13T18:26:36.388633+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:26:36.388633+0000","last_undegraded":"2024-09-13T18:26:36.388633+0000","last_fullsized":"2024-09-13T18:26:36.388633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:58.501843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897877+0000","last_change":"2024-09-13T18:26:32.353223+0000","last_active":"2024-09-13T18:26:36.897877+0000","last_peered":"2024-09-13T18:26:36.897877+0000","last_clean":"2024-09-13T18:26:36.897877+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:26:36.897877+0000","last_undegraded":"2024-09-13T18:26:36.897877+0000","last_fullsized":"2024-09-13T18:26:36.897877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:45.898213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407736+0000","last_change":"2024-09-13T18:26:32.357354+0000","last_active":"2024-09-13T18:26:36.407736+0000","last_peered":"2024-09-13T18:26:36.407736+0000","last_clean":"2024-09-13T18:26:36.407736+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:26:36.407736+0000","last_undegraded":"2024-09-13T18:26:36.407736+0000","last_fullsized":"2024-09-13T18:26:36.407736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:19.185008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.376351+0000","last_change":"2024-09-13T18:26:32.348832+0000","last_active":"2024-09-13T18:26:36.376351+0000","last_peered":"2024-09-13T18:26:36.376351+0000","last_clean":"2024-09-13T18:26:36.376351+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:26:36.376351+0000","last_undegraded":"2024-09-13T18:26:36.376351+0000","last_fullsized":"2024-09-13T18:26:36.376351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:11:59.874789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393723+0000","last_change":"2024-09-13T18:26:32.356902+0000","last_active":"2024-09-13T18:26:36.393723+0000","last_peered":"2024-09-13T18:26:36.393723+0000","last_clean":"2024-09-13T18:26:36.393723+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:26:36.393723+0000","last_undegraded":"2024-09-13T18:26:36.393723+0000","last_fullsized":"2024-09-13T18:26:36.393723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:18.362772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390031+0000","last_change":"2024-09-13T18:26:32.347047+0000","last_active":"2024-09-13T18:26:36.390031+0000","last_peered":"2024-09-13T18:26:36.390031+0000","last_clean":"2024-09-13T18:26:36.390031+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:26:36.390031+0000","last_undegraded":"2024-09-13T18:26:36.390031+0000","last_fullsized":"2024-09-13T18:26:36.390031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:40.882789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.385066+0000","last_change":"2024-09-13T18:26:32.349651+0000","last_active":"2024-09-13T18:26:36.385066+0000","last_peered":"2024-09-13T18:26:36.385066+0000","last_clean":"2024-09-13T18:26:36.385066+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:26:36.385066+0000","last_undegraded":"2024-09-13T18:26:36.385066+0000","last_fullsized":"2024-09-13T18:26:36.385066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:33.356563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388522+0000","last_change":"2024-09-13T18:26:32.344607+0000","last_active":"2024-09-13T18:26:36.388522+0000","last_peered":"2024-09-13T18:26:36.388522+0000","last_clean":"2024-09-13T18:26:36.388522+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:26:36.388522+0000","last_undegraded":"2024-09-13T18:26:36.388522+0000","last_fullsized":"2024-09-13T18:26:36.388522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:31.658172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387737+0000","last_change":"2024-09-13T18:26:32.347717+0000","last_active":"2024-09-13T18:26:36.387737+0000","last_peered":"2024-09-13T18:26:36.387737+0000","last_clean":"2024-09-13T18:26:36.387737+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:26:36.387737+0000","last_undegraded":"2024-09-13T18:26:36.387737+0000","last_fullsized":"2024-09-13T18:26:36.387737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:47.187837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392104+0000","last_change":"2024-09-13T18:26:32.359506+0000","last_active":"2024-09-13T18:26:36.392104+0000","last_peered":"2024-09-13T18:26:36.392104+0000","last_clean":"2024-09-13T18:26:36.392104+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:26:36.392104+0000","last_undegraded":"2024-09-13T18:26:36.392104+0000","last_fullsized":"2024-09-13T18:26:36.392104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:39:57.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897951+0000","last_change":"2024-09-13T18:26:32.357729+0000","last_active":"2024-09-13T18:26:36.897951+0000","last_peered":"2024-09-13T18:26:36.897951+0000","last_clean":"2024-09-13T18:26:36.897951+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:26:36.897951+0000","last_undegraded":"2024-09-13T18:26:36.897951+0000","last_fullsized":"2024-09-13T18:26:36.897951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:05.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897974+0000","last_change":"2024-09-13T18:26:32.357980+0000","last_active":"2024-09-13T18:26:36.897974+0000","last_peered":"2024-09-13T18:26:36.897974+0000","last_clean":"2024-09-13T18:26:36.897974+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:26:36.897974+0000","last_undegraded":"2024-09-13T18:26:36.897974+0000","last_fullsized":"2024-09-13T18:26:36.897974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:00.858702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405699+0000","last_change":"2024-09-13T18:26:32.365638+0000","last_active":"2024-09-13T18:26:36.405699+0000","last_peered":"2024-09-13T18:26:36.405699+0000","last_clean":"2024-09-13T18:26:36.405699+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:26:36.405699+0000","last_undegraded":"2024-09-13T18:26:36.405699+0000","last_fullsized":"2024-09-13T18:26:36.405699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.065853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383235+0000","last_change":"2024-09-13T18:26:32.345496+0000","last_active":"2024-09-13T18:26:36.383235+0000","last_peered":"2024-09-13T18:26:36.383235+0000","last_clean":"2024-09-13T18:26:36.383235+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:26:36.383235+0000","last_undegraded":"2024-09-13T18:26:36.383235+0000","last_fullsized":"2024-09-13T18:26:36.383235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:22.447315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404895+0000","last_change":"2024-09-13T18:26:32.356901+0000","last_active":"2024-09-13T18:26:36.404895+0000","last_peered":"2024-09-13T18:26:36.404895+0000","last_clean":"2024-09-13T18:26:36.404895+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:26:36.404895+0000","last_undegraded":"2024-09-13T18:26:36.404895+0000","last_fullsized":"2024-09-13T18:26:36.404895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:03.311603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391082+0000","last_change":"2024-09-13T18:26:32.355203+0000","last_active":"2024-09-13T18:26:36.391082+0000","last_peered":"2024-09-13T18:26:36.391082+0000","last_clean":"2024-09-13T18:26:36.391082+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:26:36.391082+0000","last_undegraded":"2024-09-13T18:26:36.391082+0000","last_fullsized":"2024-09-13T18:26:36.391082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:52.769708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405315+0000","last_change":"2024-09-13T18:26:32.358190+0000","last_active":"2024-09-13T18:26:36.405315+0000","last_peered":"2024-09-13T18:26:36.405315+0000","last_clean":"2024-09-13T18:26:36.405315+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:26:36.405315+0000","last_undegraded":"2024-09-13T18:26:36.405315+0000","last_fullsized":"2024-09-13T18:26:36.405315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:35.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393258+0000","last_change":"2024-09-13T18:26:32.355121+0000","last_active":"2024-09-13T18:26:36.393258+0000","last_peered":"2024-09-13T18:26:36.393258+0000","last_clean":"2024-09-13T18:26:36.393258+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:26:36.393258+0000","last_undegraded":"2024-09-13T18:26:36.393258+0000","last_fullsized":"2024-09-13T18:26:36.393258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:48.427332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388810+0000","last_change":"2024-09-13T18:26:32.341497+0000","last_active":"2024-09-13T18:26:36.388810+0000","last_peered":"2024-09-13T18:26:36.388810+0000","last_clean":"2024-09-13T18:26:36.388810+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:26:36.388810+0000","last_undegraded":"2024-09-13T18:26:36.388810+0000","last_fullsized":"2024-09-13T18:26:36.388810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:15.510048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386627+0000","last_change":"2024-09-13T18:26:32.350106+0000","last_active":"2024-09-13T18:26:36.386627+0000","last_peered":"2024-09-13T18:26:36.386627+0000","last_clean":"2024-09-13T18:26:36.386627+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:26:36.386627+0000","last_undegraded":"2024-09-13T18:26:36.386627+0000","last_fullsized":"2024-09-13T18:26:36.386627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:12.387451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405597+0000","last_change":"2024-09-13T18:26:32.362196+0000","last_active":"2024-09-13T18:26:36.405597+0000","last_peered":"2024-09-13T18:26:36.405597+0000","last_clean":"2024-09-13T18:26:36.405597+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:26:36.405597+0000","last_undegraded":"2024-09-13T18:26:36.405597+0000","last_fullsized":"2024-09-13T18:26:36.405597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:14.946904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7751,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":15502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509453,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":107598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509453,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":344064,"data_stored":695251,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]}]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":103839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":108074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":103991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":104551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":104117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":339968,"data_stored":694201,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:27:22.120 INFO:tasks.mds_thrash.ceph_manager:clean! 2024-09-13T18:27:22.121 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:27:22.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.226+0000 7fc38238e700 1 Processor -- start 2024-09-13T18:27:22.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.226+0000 7fc38238e700 1 -- start start 2024-09-13T18:27:22.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.226+0000 7fc38238e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc37c0a9840 0x7fc37c15e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.226+0000 7fc38238e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37c161550 con 0x7fc37c0caed0 2024-09-13T18:27:22.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.226+0000 7fc38238e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37c0aa800 con 0x7fc37c0cb2b0 2024-09-13T18:27:22.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.226+0000 7fc38238e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37c072a30 con 0x7fc37c0a9840 2024-09-13T18:27:22.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.226+0000 7fc38138c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc37c0caed0 0x7fc37c1619e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49126/0 (socket says 172.21.15.146:49126) 2024-09-13T18:27:22.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.226+0000 7fc38138c700 1 -- 172.21.15.146:0/1112601289 learned_addr learned my addr 172.21.15.146:0/1112601289 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:22.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.226+0000 7fc381b8d700 1 --2- 172.21.15.146:0/1112601289 >> v2:172.21.15.146:3300/0 conn(0x7fc37c0a9840 0x7fc37c15e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.226+0000 7fc373fff700 1 -- 172.21.15.146:0/1112601289 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 405199765 0 0) 0x7fc37c161550 con 0x7fc37c0caed0 2024-09-13T18:27:22.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.226+0000 7fc373fff700 1 -- 172.21.15.146:0/1112601289 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3680035d0 con 0x7fc37c0caed0 2024-09-13T18:27:22.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.226+0000 7fc381b8d700 1 -- 172.21.15.146:0/1112601289 >> v1:172.21.15.154:6789/0 conn(0x7fc37c0cb2b0 legacy=0x7fc37c15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc381b8d700 1 -- 172.21.15.146:0/1112601289 >> v1:172.21.15.146:6789/0 conn(0x7fc37c0caed0 legacy=0x7fc37c1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc381b8d700 1 -- 172.21.15.146:0/1112601289 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c161220 con 0x7fc37c0a9840 2024-09-13T18:27:22.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc381b8d700 1 --2- 172.21.15.146:0/1112601289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c0a9840 0x7fc37c15e810 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc378009850 tx=0x7fc378009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fee61e51701d3c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:22.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc373fff700 1 -- 172.21.15.146:0/1112601289 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378011830 con 0x7fc37c0a9840 2024-09-13T18:27:22.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc373fff700 1 -- 172.21.15.146:0/1112601289 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc378011990 con 0x7fc37c0a9840 2024-09-13T18:27:22.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc373fff700 1 -- 172.21.15.146:0/1112601289 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378011d00 con 0x7fc37c0a9840 2024-09-13T18:27:22.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 -- 172.21.15.146:0/1112601289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c0a9840 msgr2=0x7fc37c15e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 --2- 172.21.15.146:0/1112601289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c0a9840 0x7fc37c15e810 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc378009850 tx=0x7fc378009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:22.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 -- 172.21.15.146:0/1112601289 shutdown_connections 2024-09-13T18:27:22.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 --2- 172.21.15.146:0/1112601289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c0a9840 0x7fc37c15e810 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 -- 172.21.15.146:0/1112601289 >> 172.21.15.146:0/1112601289 conn(0x7fc37c066090 msgr2=0x7fc37c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:22.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 -- 172.21.15.146:0/1112601289 shutdown_connections 2024-09-13T18:27:22.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 -- 172.21.15.146:0/1112601289 wait complete. 2024-09-13T18:27:22.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 Processor -- start 2024-09-13T18:27:22.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 -- start start 2024-09-13T18:27:22.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c0a9840 0x7fc37c0ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc37c0caed0 0x7fc37c06d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc37c0cb2b0 0x7fc37c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c0acc60 con 0x7fc37c0a9840 2024-09-13T18:27:22.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c0ad6e0 con 0x7fc37c0cb2b0 2024-09-13T18:27:22.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38238e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c0abd80 con 0x7fc37c0caed0 2024-09-13T18:27:22.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38138c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c0a9840 0x7fc37c0ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38138c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c0a9840 0x7fc37c0ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44186/0 (socket says 172.21.15.146:44186) 2024-09-13T18:27:22.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38138c700 1 -- 172.21.15.146:0/2443397746 learned_addr learned my addr 172.21.15.146:0/2443397746 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:22.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc381b8d700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc37c0cb2b0 0x7fc37c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc380b8b700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc37c0caed0 0x7fc37c06d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38138c700 1 -- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc37c0caed0 msgr2=0x7fc37c06d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38138c700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc37c0caed0 0x7fc37c06d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38138c700 1 -- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc37c0cb2b0 msgr2=0x7fc37c06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38138c700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc37c0cb2b0 0x7fc37c06da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38138c700 1 -- 172.21.15.146:0/2443397746 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3780094b0 con 0x7fc37c0a9840 2024-09-13T18:27:22.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc381b8d700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc37c0cb2b0 0x7fc37c06da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:22.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.230+0000 7fc38138c700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c0a9840 0x7fc37c0ad1a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc37400cbf0 tx=0x7fc37400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:22.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.234+0000 7fc371ffb700 1 -- 172.21.15.146:0/2443397746 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37400a9a0 con 0x7fc37c0a9840 2024-09-13T18:27:22.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.234+0000 7fc371ffb700 1 -- 172.21.15.146:0/2443397746 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37400ab00 con 0x7fc37c0a9840 2024-09-13T18:27:22.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.234+0000 7fc38238e700 1 -- 172.21.15.146:0/2443397746 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37c0ac020 con 0x7fc37c0a9840 2024-09-13T18:27:22.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.234+0000 7fc371ffb700 1 -- 172.21.15.146:0/2443397746 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37400ae90 con 0x7fc37c0a9840 2024-09-13T18:27:22.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.234+0000 7fc38238e700 1 -- 172.21.15.146:0/2443397746 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37c0ac5a0 con 0x7fc37c0a9840 2024-09-13T18:27:22.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.234+0000 7fc371ffb700 1 -- 172.21.15.146:0/2443397746 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc37401b3b0 con 0x7fc37c0a9840 2024-09-13T18:27:22.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.234+0000 7fc38238e700 1 -- 172.21.15.146:0/2443397746 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37c06d370 con 0x7fc37c0a9840 2024-09-13T18:27:22.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.234+0000 7fc371ffb700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc360066730 0x7fc360068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.234+0000 7fc371ffb700 1 -- 172.21.15.146:0/2443397746 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc374083ff0 con 0x7fc37c0a9840 2024-09-13T18:27:22.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.234+0000 7fc380b8b700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc360066730 0x7fc360068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.238+0000 7fc380b8b700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc360066730 0x7fc360068bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0c9b50 tx=0x7fc36c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:22.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.242+0000 7fc371ffb700 1 -- 172.21.15.146:0/2443397746 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc374051030 con 0x7fc37c0a9840 2024-09-13T18:27:22.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.410+0000 7fc38238e700 1 -- 172.21.15.146:0/2443397746 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc37c066470 con 0x7fc360066730 2024-09-13T18:27:22.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.434+0000 7fc371ffb700 1 -- 172.21.15.146:0/2443397746 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345369 (secure 0 0 0) 0x7fc37c066470 con 0x7fc360066730 2024-09-13T18:27:22.443 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:22.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.438+0000 7fc38238e700 1 -- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc360066730 msgr2=0x7fc360068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.438+0000 7fc38238e700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc360066730 0x7fc360068bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0c9b50 tx=0x7fc36c009040 comp rx=0 tx=0).stop 2024-09-13T18:27:22.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.438+0000 7fc38238e700 1 -- 172.21.15.146:0/2443397746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c0a9840 msgr2=0x7fc37c0ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.438+0000 7fc38238e700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c0a9840 0x7fc37c0ad1a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc37400cbf0 tx=0x7fc37400ccd0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.438+0000 7fc38238e700 1 -- 172.21.15.146:0/2443397746 shutdown_connections 2024-09-13T18:27:22.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.438+0000 7fc38238e700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc360066730 0x7fc360068bf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.438+0000 7fc38238e700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc37c0cb2b0 0x7fc37c06da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.438+0000 7fc38238e700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc37c0caed0 0x7fc37c06d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.438+0000 7fc38238e700 1 --2- 172.21.15.146:0/2443397746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c0a9840 0x7fc37c0ad1a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.438+0000 7fc38238e700 1 -- 172.21.15.146:0/2443397746 >> 172.21.15.146:0/2443397746 conn(0x7fc37c066090 msgr2=0x7fc37c15ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:22.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.438+0000 7fc38238e700 1 -- 172.21.15.146:0/2443397746 shutdown_connections 2024-09-13T18:27:22.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.438+0000 7fc38238e700 1 -- 172.21.15.146:0/2443397746 wait complete. 2024-09-13T18:27:22.447 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:27:22.458 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-09-13T18:27:21.590165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598119,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222528,"kb_used_data":2304,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754752192,"statfs":{"total":773094113280,"available":772866244608,"internally_reserved":0,"allocated":2359296,"data_stored":2021622,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":225431099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1766,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.758288"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.520387+0000","last_change":"2024-09-13T18:26:31.972797+0000","last_active":"2024-09-13T18:27:08.520387+0000","last_peered":"2024-09-13T18:27:08.520387+0000","last_clean":"2024-09-13T18:27:08.520387+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:27:08.520387+0000","last_undegraded":"2024-09-13T18:27:08.520387+0000","last_fullsized":"2024-09-13T18:27:08.520387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:50.511747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384385+0000","last_change":"2024-09-13T18:26:32.349901+0000","last_active":"2024-09-13T18:26:36.384385+0000","last_peered":"2024-09-13T18:26:36.384385+0000","last_clean":"2024-09-13T18:26:36.384385+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:26:36.384385+0000","last_undegraded":"2024-09-13T18:26:36.384385+0000","last_fullsized":"2024-09-13T18:26:36.384385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:24.978813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768058+0000","last_change":"2024-09-13T18:26:36.407579+0000","last_active":"2024-09-13T18:26:36.768058+0000","last_peered":"2024-09-13T18:26:36.768058+0000","last_clean":"2024-09-13T18:26:36.768058+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:26:36.768058+0000","last_undegraded":"2024-09-13T18:26:36.768058+0000","last_fullsized":"2024-09-13T18:26:36.768058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:53:56.370013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343814+0000","last_change":"2024-09-13T18:26:31.980716+0000","last_active":"2024-09-13T18:26:37.343814+0000","last_peered":"2024-09-13T18:26:37.343814+0000","last_clean":"2024-09-13T18:26:37.343814+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:26:37.343814+0000","last_undegraded":"2024-09-13T18:26:37.343814+0000","last_fullsized":"2024-09-13T18:26:37.343814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:20.020903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388711+0000","last_change":"2024-09-13T18:26:32.342690+0000","last_active":"2024-09-13T18:26:36.388711+0000","last_peered":"2024-09-13T18:26:36.388711+0000","last_clean":"2024-09-13T18:26:36.388711+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:26:36.388711+0000","last_undegraded":"2024-09-13T18:26:36.388711+0000","last_fullsized":"2024-09-13T18:26:36.388711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.212930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.398481+0000","last_change":"2024-09-13T18:26:36.396545+0000","last_active":"2024-09-13T18:26:36.398481+0000","last_peered":"2024-09-13T18:26:36.398481+0000","last_clean":"2024-09-13T18:26:36.398481+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:26:36.398481+0000","last_undegraded":"2024-09-13T18:26:36.398481+0000","last_fullsized":"2024-09-13T18:26:36.398481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:38.760215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.523368+0000","last_change":"2024-09-13T18:26:31.977823+0000","last_active":"2024-09-13T18:27:08.523368+0000","last_peered":"2024-09-13T18:27:08.523368+0000","last_clean":"2024-09-13T18:27:08.523368+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:27:08.523368+0000","last_undegraded":"2024-09-13T18:27:08.523368+0000","last_fullsized":"2024-09-13T18:27:08.523368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:46.956527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394225+0000","last_change":"2024-09-13T18:26:32.361288+0000","last_active":"2024-09-13T18:26:36.394225+0000","last_peered":"2024-09-13T18:26:36.394225+0000","last_clean":"2024-09-13T18:26:36.394225+0000","last_became_active":"2024-09-13T18:26:32.360918+0000","last_became_peered":"2024-09-13T18:26:32.360918+0000","last_unstale":"2024-09-13T18:26:36.394225+0000","last_undegraded":"2024-09-13T18:26:36.394225+0000","last_fullsized":"2024-09-13T18:26:36.394225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:30.143436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408364+0000","last_change":"2024-09-13T18:26:36.406675+0000","last_active":"2024-09-13T18:26:36.408364+0000","last_peered":"2024-09-13T18:26:36.408364+0000","last_clean":"2024-09-13T18:26:36.408364+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:26:36.408364+0000","last_undegraded":"2024-09-13T18:26:36.408364+0000","last_fullsized":"2024-09-13T18:26:36.408364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:36.896027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.347551+0000","last_change":"2024-09-13T18:26:31.979860+0000","last_active":"2024-09-13T18:26:37.347551+0000","last_peered":"2024-09-13T18:26:37.347551+0000","last_clean":"2024-09-13T18:26:37.347551+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:26:37.347551+0000","last_undegraded":"2024-09-13T18:26:37.347551+0000","last_fullsized":"2024-09-13T18:26:37.347551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:45.323631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393203+0000","last_change":"2024-09-13T18:26:32.351494+0000","last_active":"2024-09-13T18:26:36.393203+0000","last_peered":"2024-09-13T18:26:36.393203+0000","last_clean":"2024-09-13T18:26:36.393203+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:26:36.393203+0000","last_undegraded":"2024-09-13T18:26:36.393203+0000","last_fullsized":"2024-09-13T18:26:36.393203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:52.804526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400955+0000","last_change":"2024-09-13T18:26:36.398763+0000","last_active":"2024-09-13T18:26:36.400955+0000","last_peered":"2024-09-13T18:26:36.400955+0000","last_clean":"2024-09-13T18:26:36.400955+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:26:36.400955+0000","last_undegraded":"2024-09-13T18:26:36.400955+0000","last_fullsized":"2024-09-13T18:26:36.400955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:35.417438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897834+0000","last_change":"2024-09-13T18:26:32.360976+0000","last_active":"2024-09-13T18:26:36.897834+0000","last_peered":"2024-09-13T18:26:36.897834+0000","last_clean":"2024-09-13T18:26:36.897834+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:26:36.897834+0000","last_undegraded":"2024-09-13T18:26:36.897834+0000","last_fullsized":"2024-09-13T18:26:36.897834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:53:52.373515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.477712+0000","last_change":"2024-09-13T18:26:31.977891+0000","last_active":"2024-09-13T18:27:03.477712+0000","last_peered":"2024-09-13T18:27:03.477712+0000","last_clean":"2024-09-13T18:27:03.477712+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:27:03.477712+0000","last_undegraded":"2024-09-13T18:27:03.477712+0000","last_fullsized":"2024-09-13T18:27:03.477712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:01.453094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.400563+0000","last_change":"2024-09-13T18:26:36.397484+0000","last_active":"2024-09-13T18:26:36.400563+0000","last_peered":"2024-09-13T18:26:36.400563+0000","last_clean":"2024-09-13T18:26:36.400563+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:26:36.400563+0000","last_undegraded":"2024-09-13T18:26:36.400563+0000","last_fullsized":"2024-09-13T18:26:36.400563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:44.686244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386958+0000","last_change":"2024-09-13T18:26:32.345564+0000","last_active":"2024-09-13T18:26:36.386958+0000","last_peered":"2024-09-13T18:26:36.386958+0000","last_clean":"2024-09-13T18:26:36.386958+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:26:36.386958+0000","last_undegraded":"2024-09-13T18:26:36.386958+0000","last_fullsized":"2024-09-13T18:26:36.386958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:54.331806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767096+0000","last_change":"2024-09-13T18:26:31.974107+0000","last_active":"2024-09-13T18:26:36.767096+0000","last_peered":"2024-09-13T18:26:36.767096+0000","last_clean":"2024-09-13T18:26:36.767096+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:26:36.767096+0000","last_undegraded":"2024-09-13T18:26:36.767096+0000","last_fullsized":"2024-09-13T18:26:36.767096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:37.547480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408253+0000","last_change":"2024-09-13T18:26:36.405986+0000","last_active":"2024-09-13T18:26:36.408253+0000","last_peered":"2024-09-13T18:26:36.408253+0000","last_clean":"2024-09-13T18:26:36.408253+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:26:36.408253+0000","last_undegraded":"2024-09-13T18:26:36.408253+0000","last_fullsized":"2024-09-13T18:26:36.408253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:30.052430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383772+0000","last_change":"2024-09-13T18:26:32.340616+0000","last_active":"2024-09-13T18:26:36.383772+0000","last_peered":"2024-09-13T18:26:36.383772+0000","last_clean":"2024-09-13T18:26:36.383772+0000","last_became_active":"2024-09-13T18:26:32.339405+0000","last_became_peered":"2024-09-13T18:26:32.339405+0000","last_unstale":"2024-09-13T18:26:36.383772+0000","last_undegraded":"2024-09-13T18:26:36.383772+0000","last_fullsized":"2024-09-13T18:26:36.383772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:26:26.081896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.474579+0000","last_change":"2024-09-13T18:26:31.968895+0000","last_active":"2024-09-13T18:27:03.474579+0000","last_peered":"2024-09-13T18:27:03.474579+0000","last_clean":"2024-09-13T18:27:03.474579+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:27:03.474579+0000","last_undegraded":"2024-09-13T18:27:03.474579+0000","last_fullsized":"2024-09-13T18:27:03.474579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:15.525097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767834+0000","last_change":"2024-09-13T18:26:36.406484+0000","last_active":"2024-09-13T18:26:36.767834+0000","last_peered":"2024-09-13T18:26:36.767834+0000","last_clean":"2024-09-13T18:26:36.767834+0000","last_became_active":"2024-09-13T18:26:36.406161+0000","last_became_peered":"2024-09-13T18:26:36.406161+0000","last_unstale":"2024-09-13T18:26:36.767834+0000","last_undegraded":"2024-09-13T18:26:36.767834+0000","last_fullsized":"2024-09-13T18:26:36.767834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:53.330919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391435+0000","last_change":"2024-09-13T18:26:32.360406+0000","last_active":"2024-09-13T18:26:36.391435+0000","last_peered":"2024-09-13T18:26:36.391435+0000","last_clean":"2024-09-13T18:26:36.391435+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:26:36.391435+0000","last_undegraded":"2024-09-13T18:26:36.391435+0000","last_fullsized":"2024-09-13T18:26:36.391435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:37.417707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.476272+0000","last_change":"2024-09-13T18:26:31.976872+0000","last_active":"2024-09-13T18:27:03.476272+0000","last_peered":"2024-09-13T18:27:03.476272+0000","last_clean":"2024-09-13T18:27:03.476272+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:27:03.476272+0000","last_undegraded":"2024-09-13T18:27:03.476272+0000","last_fullsized":"2024-09-13T18:27:03.476272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:26.163029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408244+0000","last_change":"2024-09-13T18:26:36.405722+0000","last_active":"2024-09-13T18:26:36.408244+0000","last_peered":"2024-09-13T18:26:36.408244+0000","last_clean":"2024-09-13T18:26:36.408244+0000","last_became_active":"2024-09-13T18:26:36.405310+0000","last_became_peered":"2024-09-13T18:26:36.405310+0000","last_unstale":"2024-09-13T18:26:36.408244+0000","last_undegraded":"2024-09-13T18:26:36.408244+0000","last_fullsized":"2024-09-13T18:26:36.408244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:09.815655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.389080+0000","last_change":"2024-09-13T18:26:32.343915+0000","last_active":"2024-09-13T18:26:36.389080+0000","last_peered":"2024-09-13T18:26:36.389080+0000","last_clean":"2024-09-13T18:26:36.389080+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:26:36.389080+0000","last_undegraded":"2024-09-13T18:26:36.389080+0000","last_fullsized":"2024-09-13T18:26:36.389080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:19.357243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.477810+0000","last_change":"2024-09-13T18:26:31.979944+0000","last_active":"2024-09-13T18:27:03.477810+0000","last_peered":"2024-09-13T18:27:03.477810+0000","last_clean":"2024-09-13T18:27:03.477810+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:27:03.477810+0000","last_undegraded":"2024-09-13T18:27:03.477810+0000","last_fullsized":"2024-09-13T18:27:03.477810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:21.721921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.411958+0000","last_change":"2024-09-13T18:26:36.410028+0000","last_active":"2024-09-13T18:26:36.411958+0000","last_peered":"2024-09-13T18:26:36.411958+0000","last_clean":"2024-09-13T18:26:36.411958+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:26:36.411958+0000","last_undegraded":"2024-09-13T18:26:36.411958+0000","last_fullsized":"2024-09-13T18:26:36.411958+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:32.264341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387875+0000","last_change":"2024-09-13T18:26:32.358980+0000","last_active":"2024-09-13T18:26:36.387875+0000","last_peered":"2024-09-13T18:26:36.387875+0000","last_clean":"2024-09-13T18:26:36.387875+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:26:36.387875+0000","last_undegraded":"2024-09-13T18:26:36.387875+0000","last_fullsized":"2024-09-13T18:26:36.387875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:08.748139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.521355+0000","last_change":"2024-09-13T18:26:31.968972+0000","last_active":"2024-09-13T18:27:08.521355+0000","last_peered":"2024-09-13T18:27:08.521355+0000","last_clean":"2024-09-13T18:27:08.521355+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:27:08.521355+0000","last_undegraded":"2024-09-13T18:27:08.521355+0000","last_fullsized":"2024-09-13T18:27:08.521355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:56.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768011+0000","last_change":"2024-09-13T18:26:36.407034+0000","last_active":"2024-09-13T18:26:36.768011+0000","last_peered":"2024-09-13T18:26:36.768011+0000","last_clean":"2024-09-13T18:26:36.768011+0000","last_became_active":"2024-09-13T18:26:36.406473+0000","last_became_peered":"2024-09-13T18:26:36.406473+0000","last_unstale":"2024-09-13T18:26:36.768011+0000","last_undegraded":"2024-09-13T18:26:36.768011+0000","last_fullsized":"2024-09-13T18:26:36.768011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:53.815901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386420+0000","last_change":"2024-09-13T18:26:32.342823+0000","last_active":"2024-09-13T18:26:36.386420+0000","last_peered":"2024-09-13T18:26:36.386420+0000","last_clean":"2024-09-13T18:26:36.386420+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:26:36.386420+0000","last_undegraded":"2024-09-13T18:26:36.386420+0000","last_fullsized":"2024-09-13T18:26:36.386420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:28.051252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.349179+0000","last_change":"2024-09-13T18:26:31.978976+0000","last_active":"2024-09-13T18:26:37.349179+0000","last_peered":"2024-09-13T18:26:37.349179+0000","last_clean":"2024-09-13T18:26:37.349179+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:26:37.349179+0000","last_undegraded":"2024-09-13T18:26:37.349179+0000","last_fullsized":"2024-09-13T18:26:37.349179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:11.279557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103455+0000","last_change":"2024-09-13T18:26:36.408844+0000","last_active":"2024-09-13T18:26:37.103455+0000","last_peered":"2024-09-13T18:26:37.103455+0000","last_clean":"2024-09-13T18:26:37.103455+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:26:37.103455+0000","last_undegraded":"2024-09-13T18:26:37.103455+0000","last_fullsized":"2024-09-13T18:26:37.103455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:06.969342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393043+0000","last_change":"2024-09-13T18:26:32.355038+0000","last_active":"2024-09-13T18:26:36.393043+0000","last_peered":"2024-09-13T18:26:36.393043+0000","last_clean":"2024-09-13T18:26:36.393043+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:26:36.393043+0000","last_undegraded":"2024-09-13T18:26:36.393043+0000","last_fullsized":"2024-09-13T18:26:36.393043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.159231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386562+0000","last_change":"2024-09-13T18:26:31.972931+0000","last_active":"2024-09-13T18:26:36.386562+0000","last_peered":"2024-09-13T18:26:36.386562+0000","last_clean":"2024-09-13T18:26:36.386562+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:26:36.386562+0000","last_undegraded":"2024-09-13T18:26:36.386562+0000","last_fullsized":"2024-09-13T18:26:36.386562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:05.646924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766310+0000","last_change":"2024-09-13T18:26:36.398876+0000","last_active":"2024-09-13T18:26:36.766310+0000","last_peered":"2024-09-13T18:26:36.766310+0000","last_clean":"2024-09-13T18:26:36.766310+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:26:36.766310+0000","last_undegraded":"2024-09-13T18:26:36.766310+0000","last_fullsized":"2024-09-13T18:26:36.766310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:56.918239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392625+0000","last_change":"2024-09-13T18:26:32.353111+0000","last_active":"2024-09-13T18:26:36.392625+0000","last_peered":"2024-09-13T18:26:36.392625+0000","last_clean":"2024-09-13T18:26:36.392625+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:26:36.392625+0000","last_undegraded":"2024-09-13T18:26:36.392625+0000","last_fullsized":"2024-09-13T18:26:36.392625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:43.927936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.772320+0000","last_change":"2024-09-13T18:26:31.976075+0000","last_active":"2024-09-13T18:26:36.772320+0000","last_peered":"2024-09-13T18:26:36.772320+0000","last_clean":"2024-09-13T18:26:36.772320+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:26:36.772320+0000","last_undegraded":"2024-09-13T18:26:36.772320+0000","last_fullsized":"2024-09-13T18:26:36.772320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:03.693518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766877+0000","last_change":"2024-09-13T18:26:36.401777+0000","last_active":"2024-09-13T18:26:36.766877+0000","last_peered":"2024-09-13T18:26:36.766877+0000","last_clean":"2024-09-13T18:26:36.766877+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:26:36.766877+0000","last_undegraded":"2024-09-13T18:26:36.766877+0000","last_fullsized":"2024-09-13T18:26:36.766877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:43.440284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384991+0000","last_change":"2024-09-13T18:26:32.343791+0000","last_active":"2024-09-13T18:26:36.384991+0000","last_peered":"2024-09-13T18:26:36.384991+0000","last_clean":"2024-09-13T18:26:36.384991+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:26:36.384991+0000","last_undegraded":"2024-09-13T18:26:36.384991+0000","last_fullsized":"2024-09-13T18:26:36.384991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:36.296571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.521069+0000","last_change":"2024-09-13T18:26:31.970786+0000","last_active":"2024-09-13T18:27:08.521069+0000","last_peered":"2024-09-13T18:27:08.521069+0000","last_clean":"2024-09-13T18:27:08.521069+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:27:08.521069+0000","last_undegraded":"2024-09-13T18:27:08.521069+0000","last_fullsized":"2024-09-13T18:27:08.521069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:54.584009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767929+0000","last_change":"2024-09-13T18:26:36.401529+0000","last_active":"2024-09-13T18:26:36.767929+0000","last_peered":"2024-09-13T18:26:36.767929+0000","last_clean":"2024-09-13T18:26:36.767929+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:26:36.767929+0000","last_undegraded":"2024-09-13T18:26:36.767929+0000","last_fullsized":"2024-09-13T18:26:36.767929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:47.318347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.382662+0000","last_change":"2024-09-13T18:26:32.350470+0000","last_active":"2024-09-13T18:26:36.382662+0000","last_peered":"2024-09-13T18:26:36.382662+0000","last_clean":"2024-09-13T18:26:36.382662+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:26:36.382662+0000","last_undegraded":"2024-09-13T18:26:36.382662+0000","last_fullsized":"2024-09-13T18:26:36.382662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:55.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.522206+0000","last_change":"2024-09-13T18:26:31.972587+0000","last_active":"2024-09-13T18:27:08.522206+0000","last_peered":"2024-09-13T18:27:08.522206+0000","last_clean":"2024-09-13T18:27:08.522206+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:27:08.522206+0000","last_undegraded":"2024-09-13T18:27:08.522206+0000","last_fullsized":"2024-09-13T18:27:08.522206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:27.547763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103735+0000","last_change":"2024-09-13T18:26:36.405086+0000","last_active":"2024-09-13T18:26:37.103735+0000","last_peered":"2024-09-13T18:26:37.103735+0000","last_clean":"2024-09-13T18:26:37.103735+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:26:37.103735+0000","last_undegraded":"2024-09-13T18:26:37.103735+0000","last_fullsized":"2024-09-13T18:26:37.103735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:02:51.694634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388036+0000","last_change":"2024-09-13T18:26:32.350510+0000","last_active":"2024-09-13T18:26:36.388036+0000","last_peered":"2024-09-13T18:26:36.388036+0000","last_clean":"2024-09-13T18:26:36.388036+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:26:36.388036+0000","last_undegraded":"2024-09-13T18:26:36.388036+0000","last_fullsized":"2024-09-13T18:26:36.388036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:40.850255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.479714+0000","last_change":"2024-09-13T18:26:31.978355+0000","last_active":"2024-09-13T18:27:03.479714+0000","last_peered":"2024-09-13T18:27:03.479714+0000","last_clean":"2024-09-13T18:27:03.479714+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:27:03.479714+0000","last_undegraded":"2024-09-13T18:27:03.479714+0000","last_fullsized":"2024-09-13T18:27:03.479714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:12.946518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404778+0000","last_change":"2024-09-13T18:26:26.931403+0000","last_active":"2024-09-13T18:26:36.404778+0000","last_peered":"2024-09-13T18:26:36.404778+0000","last_clean":"2024-09-13T18:26:36.404778+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:26:36.404778+0000","last_undegraded":"2024-09-13T18:26:36.404778+0000","last_fullsized":"2024-09-13T18:26:36.404778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_clean_scrub_stamp":"2024-09-13T18:26:24.880028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:30.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897926+0000","last_change":"2024-09-13T18:26:36.409827+0000","last_active":"2024-09-13T18:26:36.897926+0000","last_peered":"2024-09-13T18:26:36.897926+0000","last_clean":"2024-09-13T18:26:36.897926+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:26:36.897926+0000","last_undegraded":"2024-09-13T18:26:36.897926+0000","last_fullsized":"2024-09-13T18:26:36.897926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:50.482400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405746+0000","last_change":"2024-09-13T18:26:32.356850+0000","last_active":"2024-09-13T18:26:36.405746+0000","last_peered":"2024-09-13T18:26:36.405746+0000","last_clean":"2024-09-13T18:26:36.405746+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:26:36.405746+0000","last_undegraded":"2024-09-13T18:26:36.405746+0000","last_fullsized":"2024-09-13T18:26:36.405746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:38.710828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.520921+0000","last_change":"2024-09-13T18:26:31.975942+0000","last_active":"2024-09-13T18:27:08.520921+0000","last_peered":"2024-09-13T18:27:08.520921+0000","last_clean":"2024-09-13T18:27:08.520921+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:27:08.520921+0000","last_undegraded":"2024-09-13T18:27:08.520921+0000","last_fullsized":"2024-09-13T18:27:08.520921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:05.840582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897805+0000","last_change":"2024-09-13T18:26:36.409802+0000","last_active":"2024-09-13T18:26:36.897805+0000","last_peered":"2024-09-13T18:26:36.897805+0000","last_clean":"2024-09-13T18:26:36.897805+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:26:36.897805+0000","last_undegraded":"2024-09-13T18:26:36.897805+0000","last_fullsized":"2024-09-13T18:26:36.897805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:38.178151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.394309+0000","last_change":"2024-09-13T18:26:32.355181+0000","last_active":"2024-09-13T18:26:36.394309+0000","last_peered":"2024-09-13T18:26:36.394309+0000","last_clean":"2024-09-13T18:26:36.394309+0000","last_became_active":"2024-09-13T18:26:32.353043+0000","last_became_peered":"2024-09-13T18:26:32.353043+0000","last_unstale":"2024-09-13T18:26:36.394309+0000","last_undegraded":"2024-09-13T18:26:36.394309+0000","last_fullsized":"2024-09-13T18:26:36.394309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:04:17.000915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.767488+0000","last_change":"2024-09-13T18:26:31.966618+0000","last_active":"2024-09-13T18:26:36.767488+0000","last_peered":"2024-09-13T18:26:36.767488+0000","last_clean":"2024-09-13T18:26:36.767488+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:26:36.767488+0000","last_undegraded":"2024-09-13T18:26:36.767488+0000","last_fullsized":"2024-09-13T18:26:36.767488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:54:26.422611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103645+0000","last_change":"2024-09-13T18:26:36.394344+0000","last_active":"2024-09-13T18:26:37.103645+0000","last_peered":"2024-09-13T18:26:37.103645+0000","last_clean":"2024-09-13T18:26:37.103645+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:26:37.103645+0000","last_undegraded":"2024-09-13T18:26:37.103645+0000","last_fullsized":"2024-09-13T18:26:37.103645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:18.425816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405784+0000","last_change":"2024-09-13T18:26:32.365655+0000","last_active":"2024-09-13T18:26:36.405784+0000","last_peered":"2024-09-13T18:26:36.405784+0000","last_clean":"2024-09-13T18:26:36.405784+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:26:36.405784+0000","last_undegraded":"2024-09-13T18:26:36.405784+0000","last_fullsized":"2024-09-13T18:26:36.405784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:40.288128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.343912+0000","last_change":"2024-09-13T18:26:31.970652+0000","last_active":"2024-09-13T18:26:37.343912+0000","last_peered":"2024-09-13T18:26:37.343912+0000","last_clean":"2024-09-13T18:26:37.343912+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:26:37.343912+0000","last_undegraded":"2024-09-13T18:26:37.343912+0000","last_fullsized":"2024-09-13T18:26:37.343912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:11.169661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.770631+0000","last_change":"2024-09-13T18:26:36.410280+0000","last_active":"2024-09-13T18:26:36.770631+0000","last_peered":"2024-09-13T18:26:36.770631+0000","last_clean":"2024-09-13T18:26:36.770631+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:26:36.770631+0000","last_undegraded":"2024-09-13T18:26:36.770631+0000","last_fullsized":"2024-09-13T18:26:36.770631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:01.395290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405642+0000","last_change":"2024-09-13T18:26:32.361575+0000","last_active":"2024-09-13T18:26:36.405642+0000","last_peered":"2024-09-13T18:26:36.405642+0000","last_clean":"2024-09-13T18:26:36.405642+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:26:36.405642+0000","last_undegraded":"2024-09-13T18:26:36.405642+0000","last_fullsized":"2024-09-13T18:26:36.405642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:08.818877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"25'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.346552+0000","last_change":"2024-09-13T18:26:31.977029+0000","last_active":"2024-09-13T18:26:37.346552+0000","last_peered":"2024-09-13T18:26:37.346552+0000","last_clean":"2024-09-13T18:26:37.346552+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:26:37.346552+0000","last_undegraded":"2024-09-13T18:26:37.346552+0000","last_fullsized":"2024-09-13T18:26:37.346552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:13.701689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407565+0000","last_change":"2024-09-13T18:26:36.403479+0000","last_active":"2024-09-13T18:26:36.407565+0000","last_peered":"2024-09-13T18:26:36.407565+0000","last_clean":"2024-09-13T18:26:36.407565+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:26:36.407565+0000","last_undegraded":"2024-09-13T18:26:36.407565+0000","last_fullsized":"2024-09-13T18:26:36.407565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:51.095852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404976+0000","last_change":"2024-09-13T18:26:32.365143+0000","last_active":"2024-09-13T18:26:36.404976+0000","last_peered":"2024-09-13T18:26:36.404976+0000","last_clean":"2024-09-13T18:26:36.404976+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:26:36.404976+0000","last_undegraded":"2024-09-13T18:26:36.404976+0000","last_fullsized":"2024-09-13T18:26:36.404976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:46.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.526361+0000","last_change":"2024-09-13T18:26:31.965802+0000","last_active":"2024-09-13T18:27:08.526361+0000","last_peered":"2024-09-13T18:27:08.526361+0000","last_clean":"2024-09-13T18:27:08.526361+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:27:08.526361+0000","last_undegraded":"2024-09-13T18:27:08.526361+0000","last_fullsized":"2024-09-13T18:27:08.526361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:12.759181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103768+0000","last_change":"2024-09-13T18:26:36.399235+0000","last_active":"2024-09-13T18:26:37.103768+0000","last_peered":"2024-09-13T18:26:37.103768+0000","last_clean":"2024-09-13T18:26:37.103768+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:26:37.103768+0000","last_undegraded":"2024-09-13T18:26:37.103768+0000","last_fullsized":"2024-09-13T18:26:37.103768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:13:54.711083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897760+0000","last_change":"2024-09-13T18:26:32.357403+0000","last_active":"2024-09-13T18:26:36.897760+0000","last_peered":"2024-09-13T18:26:36.897760+0000","last_clean":"2024-09-13T18:26:36.897760+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:26:36.897760+0000","last_undegraded":"2024-09-13T18:26:36.897760+0000","last_fullsized":"2024-09-13T18:26:36.897760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:12.600397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.525532+0000","last_change":"2024-09-13T18:26:31.976148+0000","last_active":"2024-09-13T18:27:08.525532+0000","last_peered":"2024-09-13T18:27:08.525532+0000","last_clean":"2024-09-13T18:27:08.525532+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:27:08.525532+0000","last_undegraded":"2024-09-13T18:27:08.525532+0000","last_fullsized":"2024-09-13T18:27:08.525532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:38.023636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769582+0000","last_change":"2024-09-13T18:26:36.406177+0000","last_active":"2024-09-13T18:26:36.769582+0000","last_peered":"2024-09-13T18:26:36.769582+0000","last_clean":"2024-09-13T18:26:36.769582+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:26:36.769582+0000","last_undegraded":"2024-09-13T18:26:36.769582+0000","last_fullsized":"2024-09-13T18:26:36.769582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:00.309995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405459+0000","last_change":"2024-09-13T18:26:32.361726+0000","last_active":"2024-09-13T18:26:36.405459+0000","last_peered":"2024-09-13T18:26:36.405459+0000","last_clean":"2024-09-13T18:26:36.405459+0000","last_became_active":"2024-09-13T18:26:32.361432+0000","last_became_peered":"2024-09-13T18:26:32.361432+0000","last_unstale":"2024-09-13T18:26:36.405459+0000","last_undegraded":"2024-09-13T18:26:36.405459+0000","last_fullsized":"2024-09-13T18:26:36.405459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:42.021515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.480098+0000","last_change":"2024-09-13T18:26:31.982114+0000","last_active":"2024-09-13T18:27:03.480098+0000","last_peered":"2024-09-13T18:27:03.480098+0000","last_clean":"2024-09-13T18:27:03.480098+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:27:03.480098+0000","last_undegraded":"2024-09-13T18:27:03.480098+0000","last_fullsized":"2024-09-13T18:27:03.480098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.266528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769049+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:26:36.769049+0000","last_peered":"2024-09-13T18:26:36.769049+0000","last_clean":"2024-09-13T18:26:36.769049+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:26:36.769049+0000","last_undegraded":"2024-09-13T18:26:36.769049+0000","last_fullsized":"2024-09-13T18:26:36.769049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:58.853971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388651+0000","last_change":"2024-09-13T18:26:32.351145+0000","last_active":"2024-09-13T18:26:36.388651+0000","last_peered":"2024-09-13T18:26:36.388651+0000","last_clean":"2024-09-13T18:26:36.388651+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:26:36.388651+0000","last_undegraded":"2024-09-13T18:26:36.388651+0000","last_fullsized":"2024-09-13T18:26:36.388651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:57.002435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.519027+0000","last_change":"2024-09-13T18:26:31.961202+0000","last_active":"2024-09-13T18:27:08.519027+0000","last_peered":"2024-09-13T18:27:08.519027+0000","last_clean":"2024-09-13T18:27:08.519027+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:27:08.519027+0000","last_undegraded":"2024-09-13T18:27:08.519027+0000","last_fullsized":"2024-09-13T18:27:08.519027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:27.340127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.772209+0000","last_change":"2024-09-13T18:26:36.410014+0000","last_active":"2024-09-13T18:26:36.772209+0000","last_peered":"2024-09-13T18:26:36.772209+0000","last_clean":"2024-09-13T18:26:36.772209+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:26:36.772209+0000","last_undegraded":"2024-09-13T18:26:36.772209+0000","last_fullsized":"2024-09-13T18:26:36.772209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:24.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405669+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:26:36.405669+0000","last_peered":"2024-09-13T18:26:36.405669+0000","last_clean":"2024-09-13T18:26:36.405669+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:26:36.405669+0000","last_undegraded":"2024-09-13T18:26:36.405669+0000","last_fullsized":"2024-09-13T18:26:36.405669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:40.102815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'11","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.474634+0000","last_change":"2024-09-13T18:26:31.974110+0000","last_active":"2024-09-13T18:27:03.474634+0000","last_peered":"2024-09-13T18:27:03.474634+0000","last_clean":"2024-09-13T18:27:03.474634+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:27:03.474634+0000","last_undegraded":"2024-09-13T18:27:03.474634+0000","last_fullsized":"2024-09-13T18:27:03.474634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:56.058561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.766576+0000","last_change":"2024-09-13T18:26:36.394080+0000","last_active":"2024-09-13T18:26:36.766576+0000","last_peered":"2024-09-13T18:26:36.766576+0000","last_clean":"2024-09-13T18:26:36.766576+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:26:36.766576+0000","last_undegraded":"2024-09-13T18:26:36.766576+0000","last_fullsized":"2024-09-13T18:26:36.766576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:47.121814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392195+0000","last_change":"2024-09-13T18:26:32.362873+0000","last_active":"2024-09-13T18:26:36.392195+0000","last_peered":"2024-09-13T18:26:36.392195+0000","last_clean":"2024-09-13T18:26:36.392195+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:26:36.392195+0000","last_undegraded":"2024-09-13T18:26:36.392195+0000","last_fullsized":"2024-09-13T18:26:36.392195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:37.598189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.344708+0000","last_change":"2024-09-13T18:26:31.976401+0000","last_active":"2024-09-13T18:26:37.344708+0000","last_peered":"2024-09-13T18:26:37.344708+0000","last_clean":"2024-09-13T18:26:37.344708+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:26:37.344708+0000","last_undegraded":"2024-09-13T18:26:37.344708+0000","last_fullsized":"2024-09-13T18:26:37.344708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:26.434292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.408317+0000","last_change":"2024-09-13T18:26:36.406228+0000","last_active":"2024-09-13T18:26:36.408317+0000","last_peered":"2024-09-13T18:26:36.408317+0000","last_clean":"2024-09-13T18:26:36.408317+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:26:36.408317+0000","last_undegraded":"2024-09-13T18:26:36.408317+0000","last_fullsized":"2024-09-13T18:26:36.408317+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:49.198397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388733+0000","last_change":"2024-09-13T18:26:32.350841+0000","last_active":"2024-09-13T18:26:36.388733+0000","last_peered":"2024-09-13T18:26:36.388733+0000","last_clean":"2024-09-13T18:26:36.388733+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:26:36.388733+0000","last_undegraded":"2024-09-13T18:26:36.388733+0000","last_fullsized":"2024-09-13T18:26:36.388733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:05.137882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.476898+0000","last_change":"2024-09-13T18:26:31.978214+0000","last_active":"2024-09-13T18:27:03.476898+0000","last_peered":"2024-09-13T18:27:03.476898+0000","last_clean":"2024-09-13T18:27:03.476898+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:27:03.476898+0000","last_undegraded":"2024-09-13T18:27:03.476898+0000","last_fullsized":"2024-09-13T18:27:03.476898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:17.381375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768498+0000","last_change":"2024-09-13T18:26:36.406090+0000","last_active":"2024-09-13T18:26:36.768498+0000","last_peered":"2024-09-13T18:26:36.768498+0000","last_clean":"2024-09-13T18:26:36.768498+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:26:36.768498+0000","last_undegraded":"2024-09-13T18:26:36.768498+0000","last_fullsized":"2024-09-13T18:26:36.768498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:09.865147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391759+0000","last_change":"2024-09-13T18:26:32.357701+0000","last_active":"2024-09-13T18:26:36.391759+0000","last_peered":"2024-09-13T18:26:36.391759+0000","last_clean":"2024-09-13T18:26:36.391759+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:26:36.391759+0000","last_undegraded":"2024-09-13T18:26:36.391759+0000","last_fullsized":"2024-09-13T18:26:36.391759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:47.587683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.479895+0000","last_change":"2024-09-13T18:26:31.980133+0000","last_active":"2024-09-13T18:27:03.479895+0000","last_peered":"2024-09-13T18:27:03.479895+0000","last_clean":"2024-09-13T18:27:03.479895+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:27:03.479895+0000","last_undegraded":"2024-09-13T18:27:03.479895+0000","last_fullsized":"2024-09-13T18:27:03.479895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:05.354335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.768120+0000","last_change":"2024-09-13T18:26:36.406553+0000","last_active":"2024-09-13T18:26:36.768120+0000","last_peered":"2024-09-13T18:26:36.768120+0000","last_clean":"2024-09-13T18:26:36.768120+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:26:36.768120+0000","last_undegraded":"2024-09-13T18:26:36.768120+0000","last_fullsized":"2024-09-13T18:26:36.768120+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:02.980826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:37.103481+0000","last_change":"2024-09-13T18:26:36.406986+0000","last_active":"2024-09-13T18:26:37.103481+0000","last_peered":"2024-09-13T18:26:37.103481+0000","last_clean":"2024-09-13T18:26:37.103481+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:26:37.103481+0000","last_undegraded":"2024-09-13T18:26:37.103481+0000","last_fullsized":"2024-09-13T18:26:37.103481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:53.984231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384974+0000","last_change":"2024-09-13T18:26:32.349081+0000","last_active":"2024-09-13T18:26:36.384974+0000","last_peered":"2024-09-13T18:26:36.384974+0000","last_clean":"2024-09-13T18:26:36.384974+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:26:36.384974+0000","last_undegraded":"2024-09-13T18:26:36.384974+0000","last_fullsized":"2024-09-13T18:26:36.384974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:47.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.771163+0000","last_change":"2024-09-13T18:26:31.968944+0000","last_active":"2024-09-13T18:26:36.771163+0000","last_peered":"2024-09-13T18:26:36.771163+0000","last_clean":"2024-09-13T18:26:36.771163+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:26:36.771163+0000","last_undegraded":"2024-09-13T18:26:36.771163+0000","last_fullsized":"2024-09-13T18:26:36.771163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:29.363312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.898021+0000","last_change":"2024-09-13T18:26:36.396418+0000","last_active":"2024-09-13T18:26:36.898021+0000","last_peered":"2024-09-13T18:26:36.898021+0000","last_clean":"2024-09-13T18:26:36.898021+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:26:36.898021+0000","last_undegraded":"2024-09-13T18:26:36.898021+0000","last_fullsized":"2024-09-13T18:26:36.898021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:03.252773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383753+0000","last_change":"2024-09-13T18:26:32.351369+0000","last_active":"2024-09-13T18:26:36.383753+0000","last_peered":"2024-09-13T18:26:36.383753+0000","last_clean":"2024-09-13T18:26:36.383753+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:26:36.383753+0000","last_undegraded":"2024-09-13T18:26:36.383753+0000","last_fullsized":"2024-09-13T18:26:36.383753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:10.892347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.522340+0000","last_change":"2024-09-13T18:26:31.972048+0000","last_active":"2024-09-13T18:27:08.522340+0000","last_peered":"2024-09-13T18:27:08.522340+0000","last_clean":"2024-09-13T18:27:08.522340+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:27:08.522340+0000","last_undegraded":"2024-09-13T18:27:08.522340+0000","last_fullsized":"2024-09-13T18:27:08.522340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:28.430119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:08.523049+0000","last_change":"2024-09-13T18:26:31.975310+0000","last_active":"2024-09-13T18:27:08.523049+0000","last_peered":"2024-09-13T18:27:08.523049+0000","last_clean":"2024-09-13T18:27:08.523049+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:27:08.523049+0000","last_undegraded":"2024-09-13T18:27:08.523049+0000","last_fullsized":"2024-09-13T18:27:08.523049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:52.915310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388817+0000","last_change":"2024-09-13T18:26:32.345825+0000","last_active":"2024-09-13T18:26:36.388817+0000","last_peered":"2024-09-13T18:26:36.388817+0000","last_clean":"2024-09-13T18:26:36.388817+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:26:36.388817+0000","last_undegraded":"2024-09-13T18:26:36.388817+0000","last_fullsized":"2024-09-13T18:26:36.388817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:25.200489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.395253+0000","last_change":"2024-09-13T18:26:36.391520+0000","last_active":"2024-09-13T18:26:36.395253+0000","last_peered":"2024-09-13T18:26:36.395253+0000","last_clean":"2024-09-13T18:26:36.395253+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:26:36.395253+0000","last_undegraded":"2024-09-13T18:26:36.395253+0000","last_fullsized":"2024-09-13T18:26:36.395253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:42.638713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'6","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:27:03.479127+0000","last_change":"2024-09-13T18:26:31.976603+0000","last_active":"2024-09-13T18:27:03.479127+0000","last_peered":"2024-09-13T18:27:03.479127+0000","last_clean":"2024-09-13T18:27:03.479127+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:27:03.479127+0000","last_undegraded":"2024-09-13T18:27:03.479127+0000","last_fullsized":"2024-09-13T18:27:03.479127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:03.965376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1016,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387960+0000","last_change":"2024-09-13T18:26:32.352982+0000","last_active":"2024-09-13T18:26:36.387960+0000","last_peered":"2024-09-13T18:26:36.387960+0000","last_clean":"2024-09-13T18:26:36.387960+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:26:36.387960+0000","last_undegraded":"2024-09-13T18:26:36.387960+0000","last_fullsized":"2024-09-13T18:26:36.387960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:55.670360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.769131+0000","last_change":"2024-09-13T18:26:36.403817+0000","last_active":"2024-09-13T18:26:36.769131+0000","last_peered":"2024-09-13T18:26:36.769131+0000","last_clean":"2024-09-13T18:26:36.769131+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:26:36.769131+0000","last_undegraded":"2024-09-13T18:26:36.769131+0000","last_fullsized":"2024-09-13T18:26:36.769131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:34.892901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897728+0000","last_change":"2024-09-13T18:26:32.357856+0000","last_active":"2024-09-13T18:26:36.897728+0000","last_peered":"2024-09-13T18:26:36.897728+0000","last_clean":"2024-09-13T18:26:36.897728+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:26:36.897728+0000","last_undegraded":"2024-09-13T18:26:36.897728+0000","last_fullsized":"2024-09-13T18:26:36.897728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:26.316214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.375208+0000","last_change":"2024-09-13T18:26:32.350966+0000","last_active":"2024-09-13T18:26:36.375208+0000","last_peered":"2024-09-13T18:26:36.375208+0000","last_clean":"2024-09-13T18:26:36.375208+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:26:36.375208+0000","last_undegraded":"2024-09-13T18:26:36.375208+0000","last_fullsized":"2024-09-13T18:26:36.375208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:52.632112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391246+0000","last_change":"2024-09-13T18:26:32.355806+0000","last_active":"2024-09-13T18:26:36.391246+0000","last_peered":"2024-09-13T18:26:36.391246+0000","last_clean":"2024-09-13T18:26:36.391246+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:26:36.391246+0000","last_undegraded":"2024-09-13T18:26:36.391246+0000","last_fullsized":"2024-09-13T18:26:36.391246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.988231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388740+0000","last_change":"2024-09-13T18:26:32.340965+0000","last_active":"2024-09-13T18:26:36.388740+0000","last_peered":"2024-09-13T18:26:36.388740+0000","last_clean":"2024-09-13T18:26:36.388740+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:26:36.388740+0000","last_undegraded":"2024-09-13T18:26:36.388740+0000","last_fullsized":"2024-09-13T18:26:36.388740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:01:04.716919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.384429+0000","last_change":"2024-09-13T18:26:32.349625+0000","last_active":"2024-09-13T18:26:36.384429+0000","last_peered":"2024-09-13T18:26:36.384429+0000","last_clean":"2024-09-13T18:26:36.384429+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:26:36.384429+0000","last_undegraded":"2024-09-13T18:26:36.384429+0000","last_fullsized":"2024-09-13T18:26:36.384429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.519754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386397+0000","last_change":"2024-09-13T18:26:32.349170+0000","last_active":"2024-09-13T18:26:36.386397+0000","last_peered":"2024-09-13T18:26:36.386397+0000","last_clean":"2024-09-13T18:26:36.386397+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:26:36.386397+0000","last_undegraded":"2024-09-13T18:26:36.386397+0000","last_fullsized":"2024-09-13T18:26:36.386397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:43.942732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897998+0000","last_change":"2024-09-13T18:26:32.364786+0000","last_active":"2024-09-13T18:26:36.897998+0000","last_peered":"2024-09-13T18:26:36.897998+0000","last_clean":"2024-09-13T18:26:36.897998+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:26:36.897998+0000","last_undegraded":"2024-09-13T18:26:36.897998+0000","last_fullsized":"2024-09-13T18:26:36.897998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:22.949875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387927+0000","last_change":"2024-09-13T18:26:32.347541+0000","last_active":"2024-09-13T18:26:36.387927+0000","last_peered":"2024-09-13T18:26:36.387927+0000","last_clean":"2024-09-13T18:26:36.387927+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:26:36.387927+0000","last_undegraded":"2024-09-13T18:26:36.387927+0000","last_fullsized":"2024-09-13T18:26:36.387927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:58.144128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390892+0000","last_change":"2024-09-13T18:26:32.353359+0000","last_active":"2024-09-13T18:26:36.390892+0000","last_peered":"2024-09-13T18:26:36.390892+0000","last_clean":"2024-09-13T18:26:36.390892+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:26:36.390892+0000","last_undegraded":"2024-09-13T18:26:36.390892+0000","last_fullsized":"2024-09-13T18:26:36.390892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:09.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.380537+0000","last_change":"2024-09-13T18:26:32.349566+0000","last_active":"2024-09-13T18:26:36.380537+0000","last_peered":"2024-09-13T18:26:36.380537+0000","last_clean":"2024-09-13T18:26:36.380537+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:26:36.380537+0000","last_undegraded":"2024-09-13T18:26:36.380537+0000","last_fullsized":"2024-09-13T18:26:36.380537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:22.341899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391168+0000","last_change":"2024-09-13T18:26:32.358640+0000","last_active":"2024-09-13T18:26:36.391168+0000","last_peered":"2024-09-13T18:26:36.391168+0000","last_clean":"2024-09-13T18:26:36.391168+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:26:36.391168+0000","last_undegraded":"2024-09-13T18:26:36.391168+0000","last_fullsized":"2024-09-13T18:26:36.391168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:40.174754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388633+0000","last_change":"2024-09-13T18:26:32.353675+0000","last_active":"2024-09-13T18:26:36.388633+0000","last_peered":"2024-09-13T18:26:36.388633+0000","last_clean":"2024-09-13T18:26:36.388633+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:26:36.388633+0000","last_undegraded":"2024-09-13T18:26:36.388633+0000","last_fullsized":"2024-09-13T18:26:36.388633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:58.501843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897877+0000","last_change":"2024-09-13T18:26:32.353223+0000","last_active":"2024-09-13T18:26:36.897877+0000","last_peered":"2024-09-13T18:26:36.897877+0000","last_clean":"2024-09-13T18:26:36.897877+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:26:36.897877+0000","last_undegraded":"2024-09-13T18:26:36.897877+0000","last_fullsized":"2024-09-13T18:26:36.897877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:45.898213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.407736+0000","last_change":"2024-09-13T18:26:32.357354+0000","last_active":"2024-09-13T18:26:36.407736+0000","last_peered":"2024-09-13T18:26:36.407736+0000","last_clean":"2024-09-13T18:26:36.407736+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:26:36.407736+0000","last_undegraded":"2024-09-13T18:26:36.407736+0000","last_fullsized":"2024-09-13T18:26:36.407736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:19.185008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.376351+0000","last_change":"2024-09-13T18:26:32.348832+0000","last_active":"2024-09-13T18:26:36.376351+0000","last_peered":"2024-09-13T18:26:36.376351+0000","last_clean":"2024-09-13T18:26:36.376351+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:26:36.376351+0000","last_undegraded":"2024-09-13T18:26:36.376351+0000","last_fullsized":"2024-09-13T18:26:36.376351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:11:59.874789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393723+0000","last_change":"2024-09-13T18:26:32.356902+0000","last_active":"2024-09-13T18:26:36.393723+0000","last_peered":"2024-09-13T18:26:36.393723+0000","last_clean":"2024-09-13T18:26:36.393723+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:26:36.393723+0000","last_undegraded":"2024-09-13T18:26:36.393723+0000","last_fullsized":"2024-09-13T18:26:36.393723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:18.362772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.390031+0000","last_change":"2024-09-13T18:26:32.347047+0000","last_active":"2024-09-13T18:26:36.390031+0000","last_peered":"2024-09-13T18:26:36.390031+0000","last_clean":"2024-09-13T18:26:36.390031+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:26:36.390031+0000","last_undegraded":"2024-09-13T18:26:36.390031+0000","last_fullsized":"2024-09-13T18:26:36.390031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:40.882789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.385066+0000","last_change":"2024-09-13T18:26:32.349651+0000","last_active":"2024-09-13T18:26:36.385066+0000","last_peered":"2024-09-13T18:26:36.385066+0000","last_clean":"2024-09-13T18:26:36.385066+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:26:36.385066+0000","last_undegraded":"2024-09-13T18:26:36.385066+0000","last_fullsized":"2024-09-13T18:26:36.385066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:33.356563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388522+0000","last_change":"2024-09-13T18:26:32.344607+0000","last_active":"2024-09-13T18:26:36.388522+0000","last_peered":"2024-09-13T18:26:36.388522+0000","last_clean":"2024-09-13T18:26:36.388522+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:26:36.388522+0000","last_undegraded":"2024-09-13T18:26:36.388522+0000","last_fullsized":"2024-09-13T18:26:36.388522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:31.658172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.387737+0000","last_change":"2024-09-13T18:26:32.347717+0000","last_active":"2024-09-13T18:26:36.387737+0000","last_peered":"2024-09-13T18:26:36.387737+0000","last_clean":"2024-09-13T18:26:36.387737+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:26:36.387737+0000","last_undegraded":"2024-09-13T18:26:36.387737+0000","last_fullsized":"2024-09-13T18:26:36.387737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:47.187837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.392104+0000","last_change":"2024-09-13T18:26:32.359506+0000","last_active":"2024-09-13T18:26:36.392104+0000","last_peered":"2024-09-13T18:26:36.392104+0000","last_clean":"2024-09-13T18:26:36.392104+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:26:36.392104+0000","last_undegraded":"2024-09-13T18:26:36.392104+0000","last_fullsized":"2024-09-13T18:26:36.392104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:39:57.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897951+0000","last_change":"2024-09-13T18:26:32.357729+0000","last_active":"2024-09-13T18:26:36.897951+0000","last_peered":"2024-09-13T18:26:36.897951+0000","last_clean":"2024-09-13T18:26:36.897951+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:26:36.897951+0000","last_undegraded":"2024-09-13T18:26:36.897951+0000","last_fullsized":"2024-09-13T18:26:36.897951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:05.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.897974+0000","last_change":"2024-09-13T18:26:32.357980+0000","last_active":"2024-09-13T18:26:36.897974+0000","last_peered":"2024-09-13T18:26:36.897974+0000","last_clean":"2024-09-13T18:26:36.897974+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:26:36.897974+0000","last_undegraded":"2024-09-13T18:26:36.897974+0000","last_fullsized":"2024-09-13T18:26:36.897974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:00.858702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405699+0000","last_change":"2024-09-13T18:26:32.365638+0000","last_active":"2024-09-13T18:26:36.405699+0000","last_peered":"2024-09-13T18:26:36.405699+0000","last_clean":"2024-09-13T18:26:36.405699+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:26:36.405699+0000","last_undegraded":"2024-09-13T18:26:36.405699+0000","last_fullsized":"2024-09-13T18:26:36.405699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.065853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.383235+0000","last_change":"2024-09-13T18:26:32.345496+0000","last_active":"2024-09-13T18:26:36.383235+0000","last_peered":"2024-09-13T18:26:36.383235+0000","last_clean":"2024-09-13T18:26:36.383235+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:26:36.383235+0000","last_undegraded":"2024-09-13T18:26:36.383235+0000","last_fullsized":"2024-09-13T18:26:36.383235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:22.447315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.404895+0000","last_change":"2024-09-13T18:26:32.356901+0000","last_active":"2024-09-13T18:26:36.404895+0000","last_peered":"2024-09-13T18:26:36.404895+0000","last_clean":"2024-09-13T18:26:36.404895+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:26:36.404895+0000","last_undegraded":"2024-09-13T18:26:36.404895+0000","last_fullsized":"2024-09-13T18:26:36.404895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:03.311603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.391082+0000","last_change":"2024-09-13T18:26:32.355203+0000","last_active":"2024-09-13T18:26:36.391082+0000","last_peered":"2024-09-13T18:26:36.391082+0000","last_clean":"2024-09-13T18:26:36.391082+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:26:36.391082+0000","last_undegraded":"2024-09-13T18:26:36.391082+0000","last_fullsized":"2024-09-13T18:26:36.391082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:52.769708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405315+0000","last_change":"2024-09-13T18:26:32.358190+0000","last_active":"2024-09-13T18:26:36.405315+0000","last_peered":"2024-09-13T18:26:36.405315+0000","last_clean":"2024-09-13T18:26:36.405315+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:26:36.405315+0000","last_undegraded":"2024-09-13T18:26:36.405315+0000","last_fullsized":"2024-09-13T18:26:36.405315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:35.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.393258+0000","last_change":"2024-09-13T18:26:32.355121+0000","last_active":"2024-09-13T18:26:36.393258+0000","last_peered":"2024-09-13T18:26:36.393258+0000","last_clean":"2024-09-13T18:26:36.393258+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:26:36.393258+0000","last_undegraded":"2024-09-13T18:26:36.393258+0000","last_fullsized":"2024-09-13T18:26:36.393258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:48.427332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.388810+0000","last_change":"2024-09-13T18:26:32.341497+0000","last_active":"2024-09-13T18:26:36.388810+0000","last_peered":"2024-09-13T18:26:36.388810+0000","last_clean":"2024-09-13T18:26:36.388810+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:26:36.388810+0000","last_undegraded":"2024-09-13T18:26:36.388810+0000","last_fullsized":"2024-09-13T18:26:36.388810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:15.510048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.386627+0000","last_change":"2024-09-13T18:26:32.350106+0000","last_active":"2024-09-13T18:26:36.386627+0000","last_peered":"2024-09-13T18:26:36.386627+0000","last_clean":"2024-09-13T18:26:36.386627+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:26:36.386627+0000","last_undegraded":"2024-09-13T18:26:36.386627+0000","last_fullsized":"2024-09-13T18:26:36.386627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:12.387451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T18:26:36.405597+0000","last_change":"2024-09-13T18:26:32.362196+0000","last_active":"2024-09-13T18:26:36.405597+0000","last_peered":"2024-09-13T18:26:36.405597+0000","last_clean":"2024-09-13T18:26:36.405597+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:26:36.405597+0000","last_undegraded":"2024-09-13T18:26:36.405597+0000","last_fullsized":"2024-09-13T18:26:36.405597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:14.946904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7751,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":15502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509453,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":107598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509453,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":344064,"data_stored":695251,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]}]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":103839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":108074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":103991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":104551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":104117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":339968,"data_stored":694201,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:27:22.463 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:27:22.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4db09b6700 1 Processor -- start 2024-09-13T18:27:22.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4db09b6700 1 -- start start 2024-09-13T18:27:22.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4db09b6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4dac0cf550 0x7f4dac1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4db09b6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dac0cb5b0 0x7f4dac1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4db09b6700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dac15a540 con 0x7f4dac0cf170 2024-09-13T18:27:22.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4db09b6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dac0c5220 con 0x7f4dac0cf550 2024-09-13T18:27:22.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4db09b6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac159ca0 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4daad9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4dac0cf170 0x7f4dac0c56b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49132/0 (socket says 172.21.15.146:49132) 2024-09-13T18:27:22.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4daad9d700 1 -- 172.21.15.146:0/3895240144 learned_addr learned my addr 172.21.15.146:0/3895240144 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:22.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4daa59c700 1 --2- 172.21.15.146:0/3895240144 >> v2:172.21.15.154:3300/0 conn(0x7f4dac0cf550 0x7f4dac1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4dab59e700 1 --2- 172.21.15.146:0/3895240144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dac0cb5b0 0x7f4dac1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4da9d9b700 1 -- 172.21.15.146:0/3895240144 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670730806 0 0) 0x7f4dac15a540 con 0x7f4dac0cf170 2024-09-13T18:27:22.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4da9d9b700 1 -- 172.21.15.146:0/3895240144 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d980035d0 con 0x7f4dac0cf170 2024-09-13T18:27:22.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4daa59c700 1 -- 172.21.15.146:0/3895240144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dac0cb5b0 msgr2=0x7f4dac1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:22.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4da9d9b700 1 -- 172.21.15.146:0/3895240144 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1031869044 0 0) 0x7f4d980035d0 con 0x7f4dac0cf170 2024-09-13T18:27:22.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4daa59c700 1 --2- 172.21.15.146:0/3895240144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dac0cb5b0 0x7f4dac1534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4daa59c700 1 -- 172.21.15.146:0/3895240144 >> v1:172.21.15.146:6789/0 conn(0x7f4dac0cf170 legacy=0x7f4dac0c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.566+0000 7f4daa59c700 1 -- 172.21.15.146:0/3895240144 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dac159f80 con 0x7f4dac0cf550 2024-09-13T18:27:22.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4daa59c700 1 --2- 172.21.15.146:0/3895240144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dac0cf550 0x7f4dac1595f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009850 tx=0x7f4d94009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b036a768e79d44d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:22.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4da9d9b700 1 -- 172.21.15.146:0/3895240144 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c0019f0 con 0x7f4dac0cf550 2024-09-13T18:27:22.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4da9d9b700 1 -- 172.21.15.146:0/3895240144 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d94004030 con 0x7f4dac0cf550 2024-09-13T18:27:22.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4da9d9b700 1 -- 172.21.15.146:0/3895240144 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d940043e0 con 0x7f4dac0cf550 2024-09-13T18:27:22.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4db09b6700 1 -- 172.21.15.146:0/3895240144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dac0cf550 msgr2=0x7f4dac1595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4db09b6700 1 --2- 172.21.15.146:0/3895240144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dac0cf550 0x7f4dac1595f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009850 tx=0x7f4d94009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:22.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4db09b6700 1 -- 172.21.15.146:0/3895240144 shutdown_connections 2024-09-13T18:27:22.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4db09b6700 1 --2- 172.21.15.146:0/3895240144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dac0cb5b0 0x7f4dac1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4db09b6700 1 --2- 172.21.15.146:0/3895240144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dac0cf550 0x7f4dac1595f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4db09b6700 1 -- 172.21.15.146:0/3895240144 >> 172.21.15.146:0/3895240144 conn(0x7f4dac072220 msgr2=0x7f4dac066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:22.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4db09b6700 1 -- 172.21.15.146:0/3895240144 shutdown_connections 2024-09-13T18:27:22.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4db09b6700 1 -- 172.21.15.146:0/3895240144 wait complete. 2024-09-13T18:27:22.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4db09b6700 1 Processor -- start 2024-09-13T18:27:22.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.570+0000 7f4db09b6700 1 -- start start 2024-09-13T18:27:22.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4db09b6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dac0cb5b0 0x7f4dac0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4db09b6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dac0cf170 0x7f4dac0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4db09b6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dac0cf550 0x7f4dac0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4db09b6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac158710 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4db09b6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac157f00 con 0x7f4dac0cf550 2024-09-13T18:27:22.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4db09b6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac158960 con 0x7f4dac0cf170 2024-09-13T18:27:22.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4daad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dac0cb5b0 0x7f4dac0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4daad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dac0cb5b0 0x7f4dac0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44188/0 (socket says 172.21.15.146:44188) 2024-09-13T18:27:22.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4daad9d700 1 -- 172.21.15.146:0/3699017142 learned_addr learned my addr 172.21.15.146:0/3699017142 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:22.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4dab59e700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dac0cf550 0x7f4dac0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4daa59c700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dac0cf170 0x7f4dac0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4daad9d700 1 -- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dac0cf170 msgr2=0x7f4dac0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4daad9d700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dac0cf170 0x7f4dac0b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4daad9d700 1 -- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dac0cf550 msgr2=0x7f4dac0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4daad9d700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dac0cf550 0x7f4dac0ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4daad9d700 1 -- 172.21.15.146:0/3699017142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d940094b0 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4dab59e700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dac0cf550 0x7f4dac0ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:22.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4daad9d700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dac0cb5b0 0x7f4dac0c66f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c001940 tx=0x7f4d9c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:22.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4da37fe700 1 -- 172.21.15.146:0/3699017142 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c00ad80 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4da37fe700 1 -- 172.21.15.146:0/3699017142 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9c0070d0 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4db09b6700 1 -- 172.21.15.146:0/3699017142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dac1572f0 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4da37fe700 1 -- 172.21.15.146:0/3699017142 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c007480 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4db09b6700 1 -- 172.21.15.146:0/3699017142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dac1578b0 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4db09b6700 1 -- 172.21.15.146:0/3699017142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dac066850 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.574+0000 7f4da37fe700 1 -- 172.21.15.146:0/3699017142 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4d9c0075e0 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.578+0000 7f4da37fe700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4d8c066730 0x7f4d8c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.578+0000 7f4da37fe700 1 -- 172.21.15.146:0/3699017142 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4d9c084290 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.578+0000 7f4daa59c700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4d8c066730 0x7f4d8c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.578+0000 7f4daa59c700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4d8c066730 0x7f4d8c068bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4d940090b0 tx=0x7f4d94008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:22.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.590+0000 7f4da37fe700 1 -- 172.21.15.146:0/3699017142 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4d9c0511c0 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.766+0000 7f4db09b6700 1 -- 172.21.15.146:0/3699017142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4dac0cb180 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.770+0000 7f4da37fe700 1 -- 172.21.15.146:0/3699017142 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18724 (secure 0 0 0) 0x7f4d9c0547e0 con 0x7f4dac0cb5b0 2024-09-13T18:27:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:22.779 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":27,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:27:22.593938+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.13","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:27:22.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.774+0000 7f4db09b6700 1 -- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4d8c066730 msgr2=0x7f4d8c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.774+0000 7f4db09b6700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4d8c066730 0x7f4d8c068bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4d940090b0 tx=0x7f4d94008030 comp rx=0 tx=0).stop 2024-09-13T18:27:22.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.774+0000 7f4db09b6700 1 -- 172.21.15.146:0/3699017142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dac0cb5b0 msgr2=0x7f4dac0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.774+0000 7f4db09b6700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dac0cb5b0 0x7f4dac0c66f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c001940 tx=0x7f4d9c00a340 comp rx=0 tx=0).stop 2024-09-13T18:27:22.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.774+0000 7f4db09b6700 1 -- 172.21.15.146:0/3699017142 shutdown_connections 2024-09-13T18:27:22.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.774+0000 7f4db09b6700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4d8c066730 0x7f4d8c068bf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.774+0000 7f4db09b6700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dac0cf550 0x7f4dac0ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.774+0000 7f4db09b6700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dac0cf170 0x7f4dac0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.774+0000 7f4db09b6700 1 --2- 172.21.15.146:0/3699017142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dac0cb5b0 0x7f4dac0c66f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.774+0000 7f4db09b6700 1 -- 172.21.15.146:0/3699017142 >> 172.21.15.146:0/3699017142 conn(0x7f4dac072220 msgr2=0x7f4dac152970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:22.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.774+0000 7f4db09b6700 1 -- 172.21.15.146:0/3699017142 shutdown_connections 2024-09-13T18:27:22.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.774+0000 7f4db09b6700 1 -- 172.21.15.146:0/3699017142 wait complete. 2024-09-13T18:27:22.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:26:24.867515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:26:30.782269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:26:31.323277+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:26:34.393386+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:27:22.795 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:27:22.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.902+0000 7f37c3fff700 1 Processor -- start 2024-09-13T18:27:22.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.902+0000 7f37c3fff700 1 -- start start 2024-09-13T18:27:22.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.902+0000 7f37c3fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f37c415d960 0x7f37c40c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.902+0000 7f37c3fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c40c5e50 con 0x7f37c41539e0 2024-09-13T18:27:22.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.902+0000 7f37c3fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c40cb480 con 0x7f37c40ca500 2024-09-13T18:27:22.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.902+0000 7f37c3fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c40cc3a0 con 0x7f37c415d960 2024-09-13T18:27:22.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.902+0000 7f37c27fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f37c415d960 0x7f37c40c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.902+0000 7f37c27fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f37c415d960 0x7f37c40c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44190/0 (socket says 172.21.15.146:44190) 2024-09-13T18:27:22.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.902+0000 7f37c27fc700 1 -- 172.21.15.146:0/1644953102 learned_addr learned my addr 172.21.15.146:0/1644953102 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:22.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.902+0000 7f37c2ffd700 1 --1- 172.21.15.146:0/1644953102 >> v1:172.21.15.146:6789/0 conn(0x7f37c41539e0 0x7f37c40b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49146/0 (socket says 172.21.15.146:49146) 2024-09-13T18:27:22.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.902+0000 7f37c27fc700 1 -- 172.21.15.146:0/1644953102 >> v1:172.21.15.154:6789/0 conn(0x7f37c40ca500 legacy=0x7f37c40cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:22.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.902+0000 7f37c27fc700 1 -- 172.21.15.146:0/1644953102 >> v1:172.21.15.146:6789/0 conn(0x7f37c41539e0 legacy=0x7f37c40b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.902+0000 7f37c27fc700 1 -- 172.21.15.146:0/1644953102 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c40cb720 con 0x7f37c415d960 2024-09-13T18:27:22.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.906+0000 7f37c27fc700 1 --2- 172.21.15.146:0/1644953102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37c415d960 0x7f37c40c9fc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009940 tx=0x7f37ac009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e82aeb17160679 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:22.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.906+0000 7f37c1ffb700 1 -- 172.21.15.146:0/1644953102 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac004230 con 0x7f37c415d960 2024-09-13T18:27:22.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.906+0000 7f37c1ffb700 1 -- 172.21.15.146:0/1644953102 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac009e90 con 0x7f37c415d960 2024-09-13T18:27:22.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.906+0000 7f37c1ffb700 1 -- 172.21.15.146:0/1644953102 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0045b0 con 0x7f37c415d960 2024-09-13T18:27:22.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.906+0000 7f37c3fff700 1 -- 172.21.15.146:0/1644953102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37c415d960 msgr2=0x7f37c40c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.906+0000 7f37c3fff700 1 --2- 172.21.15.146:0/1644953102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37c415d960 0x7f37c40c9fc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009940 tx=0x7f37ac009c50 comp rx=0 tx=0).stop 2024-09-13T18:27:22.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.906+0000 7f37c3fff700 1 -- 172.21.15.146:0/1644953102 shutdown_connections 2024-09-13T18:27:22.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.906+0000 7f37c3fff700 1 --2- 172.21.15.146:0/1644953102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37c415d960 0x7f37c40c9fc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.906+0000 7f37c3fff700 1 -- 172.21.15.146:0/1644953102 >> 172.21.15.146:0/1644953102 conn(0x7f37c4071ec0 msgr2=0x7f37c40722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:22.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.906+0000 7f37c3fff700 1 -- 172.21.15.146:0/1644953102 shutdown_connections 2024-09-13T18:27:22.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.906+0000 7f37c3fff700 1 -- 172.21.15.146:0/1644953102 wait complete. 2024-09-13T18:27:22.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c3fff700 1 Processor -- start 2024-09-13T18:27:22.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c3fff700 1 -- start start 2024-09-13T18:27:22.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37c40ca500 0x7f37c41477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c41539e0 0x7f37c414eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37c415d960 0x7f37c414b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c3fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c414e140 con 0x7f37c415d960 2024-09-13T18:27:22.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c3fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c4143560 con 0x7f37c41539e0 2024-09-13T18:27:22.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c414d110 con 0x7f37c40ca500 2024-09-13T18:27:22.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37c415d960 0x7f37c414b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37c415d960 0x7f37c414b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44206/0 (socket says 172.21.15.146:44206) 2024-09-13T18:27:22.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c37fe700 1 -- 172.21.15.146:0/3215740456 learned_addr learned my addr 172.21.15.146:0/3215740456 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:22.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c27fc700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c41539e0 0x7f37c414eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c2ffd700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37c40ca500 0x7f37c41477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c37fe700 1 -- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37c40ca500 msgr2=0x7f37c41477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c37fe700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37c40ca500 0x7f37c41477f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c37fe700 1 -- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c41539e0 msgr2=0x7f37c414eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:22.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c37fe700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c41539e0 0x7f37c414eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:22.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c37fe700 1 -- 172.21.15.146:0/3215740456 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac009570 con 0x7f37c415d960 2024-09-13T18:27:22.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c27fc700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c41539e0 0x7f37c414eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:22.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.910+0000 7f37c37fe700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37c415d960 0x7f37c414b350 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00cc10 tx=0x7f37bc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:22.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.914+0000 7f37bb7fe700 1 -- 172.21.15.146:0/3215740456 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00abb0 con 0x7f37c415d960 2024-09-13T18:27:22.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.914+0000 7f37bb7fe700 1 -- 172.21.15.146:0/3215740456 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc00ad10 con 0x7f37c415d960 2024-09-13T18:27:22.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.914+0000 7f37c3fff700 1 -- 172.21.15.146:0/3215740456 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c414d3d0 con 0x7f37c415d960 2024-09-13T18:27:22.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.914+0000 7f37bb7fe700 1 -- 172.21.15.146:0/3215740456 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc012660 con 0x7f37c415d960 2024-09-13T18:27:22.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.914+0000 7f37c3fff700 1 -- 172.21.15.146:0/3215740456 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c414e350 con 0x7f37c415d960 2024-09-13T18:27:22.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.914+0000 7f37c3fff700 1 -- 172.21.15.146:0/3215740456 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c4148d00 con 0x7f37c415d960 2024-09-13T18:27:22.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.914+0000 7f37bb7fe700 1 -- 172.21.15.146:0/3215740456 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f37bc0127c0 con 0x7f37c415d960 2024-09-13T18:27:22.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.914+0000 7f37bb7fe700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37a4066730 0x7f37a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:22.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.914+0000 7f37bb7fe700 1 -- 172.21.15.146:0/3215740456 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f37bc084da0 con 0x7f37c415d960 2024-09-13T18:27:22.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.914+0000 7f37c2ffd700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37a4066730 0x7f37a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:22.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.918+0000 7f37c2ffd700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37a4066730 0x7f37a4068bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37b4000f60 tx=0x7f37b4007d70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:22.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:22.926+0000 7f37bb7fe700 1 -- 172.21.15.146:0/3215740456 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f37c4148d00 con 0x7f37c415d960 2024-09-13T18:27:23.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.090+0000 7f37c3fff700 1 -- 172.21.15.146:0/3215740456 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37c4148d00 con 0x7f37c415d960 2024-09-13T18:27:23.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.090+0000 7f37bb7fe700 1 -- 172.21.15.146:0/3215740456 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f37c4148d00 con 0x7f37c415d960 2024-09-13T18:27:23.097 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-09-13T18:27:23.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.094+0000 7f37c3fff700 1 -- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37a4066730 msgr2=0x7f37a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.094+0000 7f37c3fff700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37a4066730 0x7f37a4068bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37b4000f60 tx=0x7f37b4007d70 comp rx=0 tx=0).stop 2024-09-13T18:27:23.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.094+0000 7f37c3fff700 1 -- 172.21.15.146:0/3215740456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37c415d960 msgr2=0x7f37c414b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.094+0000 7f37c3fff700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37c415d960 0x7f37c414b350 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00cc10 tx=0x7f37bc00a060 comp rx=0 tx=0).stop 2024-09-13T18:27:23.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.094+0000 7f37c3fff700 1 -- 172.21.15.146:0/3215740456 shutdown_connections 2024-09-13T18:27:23.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.094+0000 7f37c3fff700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37a4066730 0x7f37a4068bf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.094+0000 7f37c3fff700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37c415d960 0x7f37c414b350 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.094+0000 7f37c3fff700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c41539e0 0x7f37c414eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.094+0000 7f37c3fff700 1 --2- 172.21.15.146:0/3215740456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37c40ca500 0x7f37c41477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.094+0000 7f37c3fff700 1 -- 172.21.15.146:0/3215740456 >> 172.21.15.146:0/3215740456 conn(0x7f37c4071ec0 msgr2=0x7f37c40b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:23.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.094+0000 7f37c3fff700 1 -- 172.21.15.146:0/3215740456 shutdown_connections 2024-09-13T18:27:23.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.094+0000 7f37c3fff700 1 -- 172.21.15.146:0/3215740456 wait complete. 2024-09-13T18:27:23.116 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:27:23.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.230+0000 7f96f0b73700 1 Processor -- start 2024-09-13T18:27:23.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.230+0000 7f96f0b73700 1 -- start start 2024-09-13T18:27:23.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96f0b73700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96ec1539c0 0x7f96ec0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96f0b73700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f96ec06ffd0 0x7f96ec0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96f0b73700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ec0aa010 con 0x7f96ec15a570 2024-09-13T18:27:23.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96f0b73700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec0aa5d0 con 0x7f96ec1539c0 2024-09-13T18:27:23.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96f0b73700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec0aa2f0 con 0x7f96ec06ffd0 2024-09-13T18:27:23.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96eb7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96ec1539c0 0x7f96ec0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96eb7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96ec1539c0 0x7f96ec0c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44220/0 (socket says 172.21.15.146:44220) 2024-09-13T18:27:23.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96eb7fe700 1 -- 172.21.15.146:0/1653560617 learned_addr learned my addr 172.21.15.146:0/1653560617 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:23.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96ebfff700 1 --2- 172.21.15.146:0/1653560617 >> v2:172.21.15.154:3300/0 conn(0x7f96ec06ffd0 0x7f96ec0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96eb7fe700 1 -- 172.21.15.146:0/1653560617 >> v2:172.21.15.154:3300/0 conn(0x7f96ec06ffd0 msgr2=0x7f96ec0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:23.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96eb7fe700 1 --2- 172.21.15.146:0/1653560617 >> v2:172.21.15.154:3300/0 conn(0x7f96ec06ffd0 0x7f96ec0caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96eb7fe700 1 -- 172.21.15.146:0/1653560617 >> v1:172.21.15.154:6789/0 conn(0x7f96ec15a570 legacy=0x7f96ec0c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96eb7fe700 1 -- 172.21.15.146:0/1653560617 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec159a90 con 0x7f96ec1539c0 2024-09-13T18:27:23.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96eb7fe700 1 --2- 172.21.15.146:0/1653560617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96ec1539c0 0x7f96ec0c8de0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f96ec06eca0 tx=0x7f96d4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f94897ea382fe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:23.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96ea7fc700 1 -- 172.21.15.146:0/1653560617 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4011830 con 0x7f96ec1539c0 2024-09-13T18:27:23.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96ea7fc700 1 -- 172.21.15.146:0/1653560617 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d4011990 con 0x7f96ec1539c0 2024-09-13T18:27:23.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96ea7fc700 1 -- 172.21.15.146:0/1653560617 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4011d20 con 0x7f96ec1539c0 2024-09-13T18:27:23.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96f0b73700 1 -- 172.21.15.146:0/1653560617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96ec1539c0 msgr2=0x7f96ec0c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96f0b73700 1 --2- 172.21.15.146:0/1653560617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96ec1539c0 0x7f96ec0c8de0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f96ec06eca0 tx=0x7f96d4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96f0b73700 1 -- 172.21.15.146:0/1653560617 shutdown_connections 2024-09-13T18:27:23.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96f0b73700 1 --2- 172.21.15.146:0/1653560617 >> v2:172.21.15.154:3300/0 conn(0x7f96ec06ffd0 0x7f96ec0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96f0b73700 1 --2- 172.21.15.146:0/1653560617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96ec1539c0 0x7f96ec0c8de0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96f0b73700 1 -- 172.21.15.146:0/1653560617 >> 172.21.15.146:0/1653560617 conn(0x7f96ec066090 msgr2=0x7f96ec066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:23.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96f0b73700 1 -- 172.21.15.146:0/1653560617 shutdown_connections 2024-09-13T18:27:23.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.234+0000 7f96f0b73700 1 -- 172.21.15.146:0/1653560617 wait complete. 2024-09-13T18:27:23.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96f0b73700 1 Processor -- start 2024-09-13T18:27:23.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96f0b73700 1 -- start start 2024-09-13T18:27:23.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96f0b73700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96ec06ffd0 0x7f96ec157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96f0b73700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96ec1539c0 0x7f96ec06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96f0b73700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96ec15a570 0x7f96ec06db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96f0b73700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec06cef0 con 0x7f96ec1539c0 2024-09-13T18:27:23.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96f0b73700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec152cb0 con 0x7f96ec06ffd0 2024-09-13T18:27:23.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96f0b73700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec06c770 con 0x7f96ec15a570 2024-09-13T18:27:23.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96eaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96ec1539c0 0x7f96ec06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96eaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96ec1539c0 0x7f96ec06d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44226/0 (socket says 172.21.15.146:44226) 2024-09-13T18:27:23.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96eaffd700 1 -- 172.21.15.146:0/999963721 learned_addr learned my addr 172.21.15.146:0/999963721 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:23.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96eb7fe700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96ec06ffd0 0x7f96ec157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96ebfff700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96ec15a570 0x7f96ec06db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96eaffd700 1 -- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96ec15a570 msgr2=0x7f96ec06db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96eaffd700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96ec15a570 0x7f96ec06db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96eaffd700 1 -- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96ec06ffd0 msgr2=0x7f96ec157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96eaffd700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96ec06ffd0 0x7f96ec157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96eaffd700 1 -- 172.21.15.146:0/999963721 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d4009570 con 0x7f96ec1539c0 2024-09-13T18:27:23.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96ebfff700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96ec15a570 0x7f96ec06db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:23.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96eaffd700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96ec1539c0 0x7f96ec06d5f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00c560 tx=0x7f96dc00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:23.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96e3fff700 1 -- 172.21.15.146:0/999963721 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0052d0 con 0x7f96ec1539c0 2024-09-13T18:27:23.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.238+0000 7f96f0b73700 1 -- 172.21.15.146:0/999963721 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ec1ef870 con 0x7f96ec1539c0 2024-09-13T18:27:23.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.242+0000 7f96e3fff700 1 -- 172.21.15.146:0/999963721 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96dc005430 con 0x7f96ec1539c0 2024-09-13T18:27:23.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.242+0000 7f96f0b73700 1 -- 172.21.15.146:0/999963721 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec06d0d0 con 0x7f96ec1539c0 2024-09-13T18:27:23.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.242+0000 7f96e3fff700 1 -- 172.21.15.146:0/999963721 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0055a0 con 0x7f96ec1539c0 2024-09-13T18:27:23.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.242+0000 7f96f0b73700 1 -- 172.21.15.146:0/999963721 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec0c53c0 con 0x7f96ec1539c0 2024-09-13T18:27:23.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.242+0000 7f96e3fff700 1 -- 172.21.15.146:0/999963721 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f96dc0038a0 con 0x7f96ec1539c0 2024-09-13T18:27:23.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.242+0000 7f96e3fff700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f96cc066730 0x7f96cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.242+0000 7f96e3fff700 1 -- 172.21.15.146:0/999963721 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f96dc0836b0 con 0x7f96ec1539c0 2024-09-13T18:27:23.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.242+0000 7f96eb7fe700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f96cc066730 0x7f96cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.246+0000 7f96eb7fe700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f96cc066730 0x7f96cc068bf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f96d4000c00 tx=0x7f96d4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:23.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.250+0000 7f96e3fff700 1 -- 172.21.15.146:0/999963721 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f96dc050720 con 0x7f96ec1539c0 2024-09-13T18:27:23.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.418+0000 7f96f0b73700 1 -- 172.21.15.146:0/999963721 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96ec0c92f0 con 0x7f96ec1539c0 2024-09-13T18:27:23.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.418+0000 7f96e3fff700 1 -- 172.21.15.146:0/999963721 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96dc053d40 con 0x7f96ec1539c0 2024-09-13T18:27:23.427 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:27:23.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.422+0000 7f96f0b73700 1 -- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f96cc066730 msgr2=0x7f96cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.422+0000 7f96f0b73700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f96cc066730 0x7f96cc068bf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f96d4000c00 tx=0x7f96d4008030 comp rx=0 tx=0).stop 2024-09-13T18:27:23.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.422+0000 7f96f0b73700 1 -- 172.21.15.146:0/999963721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96ec1539c0 msgr2=0x7f96ec06d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.422+0000 7f96f0b73700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96ec1539c0 0x7f96ec06d5f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00c560 tx=0x7f96dc00c870 comp rx=0 tx=0).stop 2024-09-13T18:27:23.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.422+0000 7f96f0b73700 1 -- 172.21.15.146:0/999963721 shutdown_connections 2024-09-13T18:27:23.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.422+0000 7f96f0b73700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f96cc066730 0x7f96cc068bf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.422+0000 7f96f0b73700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96ec15a570 0x7f96ec06db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.422+0000 7f96f0b73700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96ec1539c0 0x7f96ec06d5f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.422+0000 7f96f0b73700 1 --2- 172.21.15.146:0/999963721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96ec06ffd0 0x7f96ec157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.422+0000 7f96f0b73700 1 -- 172.21.15.146:0/999963721 >> 172.21.15.146:0/999963721 conn(0x7f96ec066090 msgr2=0x7f96ec0c4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:23.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.422+0000 7f96f0b73700 1 -- 172.21.15.146:0/999963721 shutdown_connections 2024-09-13T18:27:23.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.422+0000 7f96f0b73700 1 -- 172.21.15.146:0/999963721 wait complete. 2024-09-13T18:27:23.442 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:27:23.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.550+0000 7f5235214700 1 Processor -- start 2024-09-13T18:27:23.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.550+0000 7f5235214700 1 -- start start 2024-09-13T18:27:23.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.550+0000 7f5235214700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f52301539e0 0x7f52300b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.550+0000 7f5235214700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f523015d960 0x7f52300c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.550+0000 7f5235214700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52300c5e50 con 0x7f52300ca500 2024-09-13T18:27:23.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.550+0000 7f5235214700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52300cc3a0 con 0x7f523015d960 2024-09-13T18:27:23.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.550+0000 7f5235214700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52300cb480 con 0x7f52301539e0 2024-09-13T18:27:23.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.550+0000 7f5234a13700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f52300ca500 0x7f52300cf580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49148/0 (socket says 172.21.15.146:49148) 2024-09-13T18:27:23.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.550+0000 7f5234a13700 1 -- 172.21.15.146:0/1148039635 learned_addr learned my addr 172.21.15.146:0/1148039635 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:23.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f522f7fe700 1 --2- 172.21.15.146:0/1148039635 >> v2:172.21.15.146:3300/0 conn(0x7f523015d960 0x7f52300c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f522ffff700 1 --2- 172.21.15.146:0/1148039635 >> v2:172.21.15.154:3300/0 conn(0x7f52301539e0 0x7f52300b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f522effd700 1 -- 172.21.15.146:0/1148039635 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281989316 0 0) 0x7f5228001490 con 0x7f52300ca500 2024-09-13T18:27:23.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f522effd700 1 -- 172.21.15.146:0/1148039635 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52180035d0 con 0x7f52300ca500 2024-09-13T18:27:23.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f522ffff700 1 -- 172.21.15.146:0/1148039635 >> v2:172.21.15.146:3300/0 conn(0x7f523015d960 msgr2=0x7f52300c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:23.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f522ffff700 1 --2- 172.21.15.146:0/1148039635 >> v2:172.21.15.146:3300/0 conn(0x7f523015d960 0x7f52300c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f522ffff700 1 -- 172.21.15.146:0/1148039635 >> v1:172.21.15.146:6789/0 conn(0x7f52300ca500 legacy=0x7f52300cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f522ffff700 1 -- 172.21.15.146:0/1148039635 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52300cb720 con 0x7f52301539e0 2024-09-13T18:27:23.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f522ffff700 1 --2- 172.21.15.146:0/1148039635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52301539e0 0x7f52300b3370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5224009850 tx=0x7f5224009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=515325718e334153 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:23.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f522effd700 1 -- 172.21.15.146:0/1148039635 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224011830 con 0x7f52301539e0 2024-09-13T18:27:23.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f522effd700 1 -- 172.21.15.146:0/1148039635 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5224011990 con 0x7f52301539e0 2024-09-13T18:27:23.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f522effd700 1 -- 172.21.15.146:0/1148039635 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224011d00 con 0x7f52301539e0 2024-09-13T18:27:23.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f5235214700 1 -- 172.21.15.146:0/1148039635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52301539e0 msgr2=0x7f52300b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f5235214700 1 --2- 172.21.15.146:0/1148039635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52301539e0 0x7f52300b3370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5224009850 tx=0x7f5224009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:23.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f5235214700 1 -- 172.21.15.146:0/1148039635 shutdown_connections 2024-09-13T18:27:23.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f5235214700 1 --2- 172.21.15.146:0/1148039635 >> v2:172.21.15.146:3300/0 conn(0x7f523015d960 0x7f52300c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f5235214700 1 --2- 172.21.15.146:0/1148039635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52301539e0 0x7f52300b3370 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f5235214700 1 -- 172.21.15.146:0/1148039635 >> 172.21.15.146:0/1148039635 conn(0x7f5230071ec0 msgr2=0x7f52300722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:23.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f5235214700 1 -- 172.21.15.146:0/1148039635 shutdown_connections 2024-09-13T18:27:23.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.554+0000 7f5235214700 1 -- 172.21.15.146:0/1148039635 wait complete. 2024-09-13T18:27:23.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f5235214700 1 Processor -- start 2024-09-13T18:27:23.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f5235214700 1 -- start start 2024-09-13T18:27:23.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f5235214700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52300ca500 0x7f52301efc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f5235214700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52301539e0 0x7f523014ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f5235214700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f523015d960 0x7f523014b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f5235214700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523014dbc0 con 0x7f52301539e0 2024-09-13T18:27:23.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f5235214700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52301f0180 con 0x7f523015d960 2024-09-13T18:27:23.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f5235214700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523014e560 con 0x7f52300ca500 2024-09-13T18:27:23.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f522f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52301539e0 0x7f523014ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f522f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52301539e0 0x7f523014ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44244/0 (socket says 172.21.15.146:44244) 2024-09-13T18:27:23.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f5234a13700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f523015d960 0x7f523014b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f522ffff700 1 --2- 172.21.15.146:0/1542685334 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52300ca500 0x7f52301efc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f522f7fe700 1 -- 172.21.15.146:0/1542685334 learned_addr learned my addr 172.21.15.146:0/1542685334 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:23.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f522f7fe700 1 -- 172.21.15.146:0/1542685334 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52300ca500 msgr2=0x7f52301efc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f522f7fe700 1 --2- 172.21.15.146:0/1542685334 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52300ca500 0x7f52301efc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f522f7fe700 1 -- 172.21.15.146:0/1542685334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f523015d960 msgr2=0x7f523014b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f522f7fe700 1 --2- 172.21.15.146:0/1542685334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f523015d960 0x7f523014b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f522f7fe700 1 -- 172.21.15.146:0/1542685334 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240094b0 con 0x7f52301539e0 2024-09-13T18:27:23.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f522f7fe700 1 --2- 172.21.15.146:0/1542685334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52301539e0 0x7f523014ce50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f522000e840 tx=0x7f522000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:23.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f522cff9700 1 -- 172.21.15.146:0/1542685334 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200095a0 con 0x7f52301539e0 2024-09-13T18:27:23.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f522cff9700 1 -- 172.21.15.146:0/1542685334 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5220013680 con 0x7f52301539e0 2024-09-13T18:27:23.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f5235214700 1 -- 172.21.15.146:0/1542685334 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523014d390 con 0x7f52301539e0 2024-09-13T18:27:23.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f522cff9700 1 -- 172.21.15.146:0/1542685334 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200139f0 con 0x7f52301539e0 2024-09-13T18:27:23.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.558+0000 7f5235214700 1 -- 172.21.15.146:0/1542685334 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523014d8e0 con 0x7f52301539e0 2024-09-13T18:27:23.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.562+0000 7f5235214700 1 -- 172.21.15.146:0/1542685334 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52300cab20 con 0x7f52301539e0 2024-09-13T18:27:23.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.562+0000 7f522cff9700 1 -- 172.21.15.146:0/1542685334 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5220013b50 con 0x7f52301539e0 2024-09-13T18:27:23.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.562+0000 7f522cff9700 1 --2- 172.21.15.146:0/1542685334 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f521c066760 0x7f521c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.562+0000 7f522cff9700 1 -- 172.21.15.146:0/1542685334 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f52200847a0 con 0x7f52301539e0 2024-09-13T18:27:23.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.562+0000 7f522ffff700 1 --2- 172.21.15.146:0/1542685334 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f521c066760 0x7f521c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.566+0000 7f522ffff700 1 --2- 172.21.15.146:0/1542685334 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f521c066760 0x7f521c068c20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5224000c00 tx=0x7f5224008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:23.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.570+0000 7f522cff9700 1 -- 172.21.15.146:0/1542685334 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f52200516c0 con 0x7f52301539e0 2024-09-13T18:27:23.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f5235214700 1 -- 172.21.15.146:0/1542685334 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52300cc7a0 con 0x7f52301539e0 2024-09-13T18:27:23.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f522cff9700 1 -- 172.21.15.146:0/1542685334 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5220054ce0 con 0x7f52301539e0 2024-09-13T18:27:23.749 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-09-13T18:27:23.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f5235214700 1 -- 172.21.15.146:0/1542685334 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f521c066760 msgr2=0x7f521c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f5235214700 1 --2- 172.21.15.146:0/1542685334 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f521c066760 0x7f521c068c20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5224000c00 tx=0x7f5224008030 comp rx=0 tx=0).stop 2024-09-13T18:27:23.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f5235214700 1 -- 172.21.15.146:0/1542685334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52301539e0 msgr2=0x7f523014ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f5235214700 1 --2- 172.21.15.146:0/1542685334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52301539e0 0x7f523014ce50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f522000e840 tx=0x7f522000eb50 comp rx=0 tx=0).stop 2024-09-13T18:27:23.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f5235214700 1 -- 172.21.15.146:0/1542685334 shutdown_connections 2024-09-13T18:27:23.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f5235214700 1 --2- 172.21.15.146:0/1542685334 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f521c066760 0x7f521c068c20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f5235214700 1 --2- 172.21.15.146:0/1542685334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f523015d960 0x7f523014b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f5235214700 1 --2- 172.21.15.146:0/1542685334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52301539e0 0x7f523014ce50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f5235214700 1 --2- 172.21.15.146:0/1542685334 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52300ca500 0x7f52301efc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f5235214700 1 -- 172.21.15.146:0/1542685334 >> 172.21.15.146:0/1542685334 conn(0x7f5230071ec0 msgr2=0x7f52300b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:23.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f5235214700 1 -- 172.21.15.146:0/1542685334 shutdown_connections 2024-09-13T18:27:23.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.742+0000 7f5235214700 1 -- 172.21.15.146:0/1542685334 wait complete. 2024-09-13T18:27:23.765 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:27:23.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.882+0000 7f5fb72c9700 1 Processor -- start 2024-09-13T18:27:23.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.882+0000 7f5fb72c9700 1 -- start start 2024-09-13T18:27:23.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.882+0000 7f5fb72c9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5fb015d960 0x7f5fb00c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.882+0000 7f5fb72c9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5fb00ca500 0x7f5fb00cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.882+0000 7f5fb72c9700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb00c5e50 con 0x7f5fb01539e0 2024-09-13T18:27:23.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.882+0000 7f5fb72c9700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb00cc3a0 con 0x7f5fb015d960 2024-09-13T18:27:23.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.882+0000 7f5fb72c9700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb00cb480 con 0x7f5fb00ca500 2024-09-13T18:27:23.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.882+0000 7f5fb62c7700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5fb01539e0 0x7f5fb00b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49154/0 (socket says 172.21.15.146:49154) 2024-09-13T18:27:23.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.882+0000 7f5fb5ac6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5fb015d960 0x7f5fb00c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb62c7700 1 -- 172.21.15.146:0/717449292 learned_addr learned my addr 172.21.15.146:0/717449292 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:23.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb6ac8700 1 --2- 172.21.15.146:0/717449292 >> v2:172.21.15.154:3300/0 conn(0x7f5fb00ca500 0x7f5fb00cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb5ac6700 1 -- 172.21.15.146:0/717449292 >> v2:172.21.15.154:3300/0 conn(0x7f5fb00ca500 msgr2=0x7f5fb00cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:23.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb5ac6700 1 --2- 172.21.15.146:0/717449292 >> v2:172.21.15.154:3300/0 conn(0x7f5fb00ca500 0x7f5fb00cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb5ac6700 1 -- 172.21.15.146:0/717449292 >> v1:172.21.15.146:6789/0 conn(0x7f5fb01539e0 legacy=0x7f5fb00b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb5ac6700 1 -- 172.21.15.146:0/717449292 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb00cb720 con 0x7f5fb015d960 2024-09-13T18:27:23.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb5ac6700 1 --2- 172.21.15.146:0/717449292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fb015d960 0x7f5fb00c9fc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009850 tx=0x7f5fac009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27883a78a34d3c2e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:23.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb52c5700 1 -- 172.21.15.146:0/717449292 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac011830 con 0x7f5fb015d960 2024-09-13T18:27:23.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb52c5700 1 -- 172.21.15.146:0/717449292 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac011990 con 0x7f5fb015d960 2024-09-13T18:27:23.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb52c5700 1 -- 172.21.15.146:0/717449292 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac011d20 con 0x7f5fb015d960 2024-09-13T18:27:23.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb72c9700 1 -- 172.21.15.146:0/717449292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fb015d960 msgr2=0x7f5fb00c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb72c9700 1 --2- 172.21.15.146:0/717449292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fb015d960 0x7f5fb00c9fc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009850 tx=0x7f5fac009b60 comp rx=0 tx=0).stop 2024-09-13T18:27:23.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb72c9700 1 -- 172.21.15.146:0/717449292 shutdown_connections 2024-09-13T18:27:23.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb72c9700 1 --2- 172.21.15.146:0/717449292 >> v2:172.21.15.154:3300/0 conn(0x7f5fb00ca500 0x7f5fb00cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb72c9700 1 --2- 172.21.15.146:0/717449292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fb015d960 0x7f5fb00c9fc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb72c9700 1 -- 172.21.15.146:0/717449292 >> 172.21.15.146:0/717449292 conn(0x7f5fb0071ec0 msgr2=0x7f5fb00722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:23.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb72c9700 1 -- 172.21.15.146:0/717449292 shutdown_connections 2024-09-13T18:27:23.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.886+0000 7f5fb72c9700 1 -- 172.21.15.146:0/717449292 wait complete. 2024-09-13T18:27:23.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb72c9700 1 Processor -- start 2024-09-13T18:27:23.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb72c9700 1 -- start start 2024-09-13T18:27:23.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb72c9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb00ca500 0x7f5fb0150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb72c9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fb01539e0 0x7f5fb00c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb72c9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fb015d960 0x7f5fb00c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb72c9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb0154640 con 0x7f5fb01539e0 2024-09-13T18:27:23.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb72c9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb014e840 con 0x7f5fb00ca500 2024-09-13T18:27:23.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb72c9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb014dc30 con 0x7f5fb015d960 2024-09-13T18:27:23.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb5ac6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fb01539e0 0x7f5fb00c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb5ac6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fb01539e0 0x7f5fb00c7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44264/0 (socket says 172.21.15.146:44264) 2024-09-13T18:27:23.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb5ac6700 1 -- 172.21.15.146:0/4094449327 learned_addr learned my addr 172.21.15.146:0/4094449327 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:23.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb62c7700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb00ca500 0x7f5fb0150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb6ac8700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fb015d960 0x7f5fb00c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb5ac6700 1 -- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fb015d960 msgr2=0x7f5fb00c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb5ac6700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fb015d960 0x7f5fb00c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb5ac6700 1 -- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb00ca500 msgr2=0x7f5fb0150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:23.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb5ac6700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb00ca500 0x7f5fb0150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:23.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb5ac6700 1 -- 172.21.15.146:0/4094449327 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac0094b0 con 0x7f5fb01539e0 2024-09-13T18:27:23.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb5ac6700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fb01539e0 0x7f5fb00c7470 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0090b0 tx=0x7f5fac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:23.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fa6ffd700 1 -- 172.21.15.146:0/4094449327 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac004550 con 0x7f5fb01539e0 2024-09-13T18:27:23.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fa6ffd700 1 -- 172.21.15.146:0/4094449327 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac01e070 con 0x7f5fb01539e0 2024-09-13T18:27:23.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb72c9700 1 -- 172.21.15.146:0/4094449327 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb014df10 con 0x7f5fb01539e0 2024-09-13T18:27:23.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fa6ffd700 1 -- 172.21.15.146:0/4094449327 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac003900 con 0x7f5fb01539e0 2024-09-13T18:27:23.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.890+0000 7f5fb72c9700 1 -- 172.21.15.146:0/4094449327 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb014e4d0 con 0x7f5fb01539e0 2024-09-13T18:27:23.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.894+0000 7f5fb72c9700 1 -- 172.21.15.146:0/4094449327 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb00cc7c0 con 0x7f5fb01539e0 2024-09-13T18:27:23.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.894+0000 7f5fa6ffd700 1 -- 172.21.15.146:0/4094449327 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5fac003a60 con 0x7f5fb01539e0 2024-09-13T18:27:23.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.894+0000 7f5fa6ffd700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5f94066730 0x7f5f94068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:23.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.894+0000 7f5fa6ffd700 1 -- 172.21.15.146:0/4094449327 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5fac085160 con 0x7f5fb01539e0 2024-09-13T18:27:23.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.894+0000 7f5fb62c7700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5f94066730 0x7f5f94068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:23.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.894+0000 7f5fb62c7700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5f94066730 0x7f5f94068bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0005930 tx=0x7f5fa0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:23.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:23.902+0000 7f5fa6ffd700 1 -- 172.21.15.146:0/4094449327 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5fac052070 con 0x7f5fb01539e0 2024-09-13T18:27:24.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.070+0000 7f5fb72c9700 1 -- 172.21.15.146:0/4094449327 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5fb014e980 con 0x7f5fb01539e0 2024-09-13T18:27:24.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.074+0000 7f5fa6ffd700 1 -- 172.21.15.146:0/4094449327 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5fac055690 con 0x7f5fb01539e0 2024-09-13T18:27:24.080 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:27:24.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.074+0000 7f5fb72c9700 1 -- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5f94066730 msgr2=0x7f5f94068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.074+0000 7f5fb72c9700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5f94066730 0x7f5f94068bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0005930 tx=0x7f5fa0009040 comp rx=0 tx=0).stop 2024-09-13T18:27:24.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.074+0000 7f5fb72c9700 1 -- 172.21.15.146:0/4094449327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fb01539e0 msgr2=0x7f5fb00c7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.074+0000 7f5fb72c9700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fb01539e0 0x7f5fb00c7470 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0090b0 tx=0x7f5fac004770 comp rx=0 tx=0).stop 2024-09-13T18:27:24.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.074+0000 7f5fb72c9700 1 -- 172.21.15.146:0/4094449327 shutdown_connections 2024-09-13T18:27:24.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.074+0000 7f5fb72c9700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5f94066730 0x7f5f94068bf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.074+0000 7f5fb72c9700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fb015d960 0x7f5fb00c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.074+0000 7f5fb72c9700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fb01539e0 0x7f5fb00c7470 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.074+0000 7f5fb72c9700 1 --2- 172.21.15.146:0/4094449327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb00ca500 0x7f5fb0150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.074+0000 7f5fb72c9700 1 -- 172.21.15.146:0/4094449327 >> 172.21.15.146:0/4094449327 conn(0x7f5fb0071ec0 msgr2=0x7f5fb00b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:24.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.074+0000 7f5fb72c9700 1 -- 172.21.15.146:0/4094449327 shutdown_connections 2024-09-13T18:27:24.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.074+0000 7f5fb72c9700 1 -- 172.21.15.146:0/4094449327 wait complete. 2024-09-13T18:27:24.095 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:27:24.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.198+0000 7f64962fb700 1 Processor -- start 2024-09-13T18:27:24.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.198+0000 7f64962fb700 1 -- start start 2024-09-13T18:27:24.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.198+0000 7f64962fb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f64900ca210 0x7f64900b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:24.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.198+0000 7f64962fb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6490071390 0x7f64900cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:24.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.198+0000 7f64962fb700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64900cc670 con 0x7f64900cbd50 2024-09-13T18:27:24.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.198+0000 7f64962fb700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64900caa30 con 0x7f6490071390 2024-09-13T18:27:24.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.198+0000 7f64962fb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64900cb810 con 0x7f64900ca210 2024-09-13T18:27:24.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.198+0000 7f6495afa700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6490071390 0x7f64900cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:24.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.198+0000 7f64952f9700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f64900cbd50 0x7f64900701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49164/0 (socket says 172.21.15.146:49164) 2024-09-13T18:27:24.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.198+0000 7f64952f9700 1 -- 172.21.15.146:0/2104641717 learned_addr learned my addr 172.21.15.146:0/2104641717 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:24.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.198+0000 7f6494af8700 1 --2- 172.21.15.146:0/2104641717 >> v2:172.21.15.154:3300/0 conn(0x7f64900ca210 0x7f64900b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:24.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f6495afa700 1 -- 172.21.15.146:0/2104641717 >> v2:172.21.15.154:3300/0 conn(0x7f64900ca210 msgr2=0x7f64900b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:24.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f6495afa700 1 --2- 172.21.15.146:0/2104641717 >> v2:172.21.15.154:3300/0 conn(0x7f64900ca210 0x7f64900b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f6495afa700 1 -- 172.21.15.146:0/2104641717 >> v1:172.21.15.146:6789/0 conn(0x7f64900cbd50 legacy=0x7f64900701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f6495afa700 1 -- 172.21.15.146:0/2104641717 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649015da20 con 0x7f6490071390 2024-09-13T18:27:24.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f6495afa700 1 --2- 172.21.15.146:0/2104641717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6490071390 0x7f64900cc130 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f64900b18a0 tx=0x7f648800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a1b63e7503003c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:24.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f6483fff700 1 -- 172.21.15.146:0/2104641717 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648800e070 con 0x7f6490071390 2024-09-13T18:27:24.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f6483fff700 1 -- 172.21.15.146:0/2104641717 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648800bbd0 con 0x7f6490071390 2024-09-13T18:27:24.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f6483fff700 1 -- 172.21.15.146:0/2104641717 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488012430 con 0x7f6490071390 2024-09-13T18:27:24.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f64962fb700 1 -- 172.21.15.146:0/2104641717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6490071390 msgr2=0x7f64900cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f64962fb700 1 --2- 172.21.15.146:0/2104641717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6490071390 0x7f64900cc130 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f64900b18a0 tx=0x7f648800b330 comp rx=0 tx=0).stop 2024-09-13T18:27:24.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f64962fb700 1 -- 172.21.15.146:0/2104641717 shutdown_connections 2024-09-13T18:27:24.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f64962fb700 1 --2- 172.21.15.146:0/2104641717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6490071390 0x7f64900cc130 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f64962fb700 1 --2- 172.21.15.146:0/2104641717 >> v2:172.21.15.154:3300/0 conn(0x7f64900ca210 0x7f64900b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f64962fb700 1 -- 172.21.15.146:0/2104641717 >> 172.21.15.146:0/2104641717 conn(0x7f6490066270 msgr2=0x7f6490066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:24.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f64962fb700 1 -- 172.21.15.146:0/2104641717 shutdown_connections 2024-09-13T18:27:24.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f64962fb700 1 -- 172.21.15.146:0/2104641717 wait complete. 2024-09-13T18:27:24.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f64962fb700 1 Processor -- start 2024-09-13T18:27:24.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.202+0000 7f64962fb700 1 -- start start 2024-09-13T18:27:24.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64962fb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6490071390 0x7f64901e6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:24.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64962fb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64900ca210 0x7f64901f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:24.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64962fb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64900cbd50 0x7f64901f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:24.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64962fb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64901e2aa0 con 0x7f6490071390 2024-09-13T18:27:24.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64962fb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64901e2d40 con 0x7f64900ca210 2024-09-13T18:27:24.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64962fb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64901e2fe0 con 0x7f64900cbd50 2024-09-13T18:27:24.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64952f9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6490071390 0x7f64901e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:24.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64952f9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6490071390 0x7f64901e6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44274/0 (socket says 172.21.15.146:44274) 2024-09-13T18:27:24.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64952f9700 1 -- 172.21.15.146:0/3032412221 learned_addr learned my addr 172.21.15.146:0/3032412221 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:24.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f6494af8700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64900ca210 0x7f64901f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:24.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f6495afa700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64900cbd50 0x7f64901f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:24.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64952f9700 1 -- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64900cbd50 msgr2=0x7f64901f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64952f9700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64900cbd50 0x7f64901f31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64952f9700 1 -- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64900ca210 msgr2=0x7f64901f2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64952f9700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64900ca210 0x7f64901f2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64952f9700 1 -- 172.21.15.146:0/3032412221 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6488009a70 con 0x7f6490071390 2024-09-13T18:27:24.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f6494af8700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64900ca210 0x7f64901f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:24.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64952f9700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6490071390 0x7f64901e6790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f648c001eb0 tx=0x7f648c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:24.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f6481ffb700 1 -- 172.21.15.146:0/3032412221 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c005d90 con 0x7f6490071390 2024-09-13T18:27:24.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f6481ffb700 1 -- 172.21.15.146:0/3032412221 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648c0070d0 con 0x7f6490071390 2024-09-13T18:27:24.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64962fb700 1 -- 172.21.15.146:0/3032412221 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64901e32c0 con 0x7f6490071390 2024-09-13T18:27:24.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f6481ffb700 1 -- 172.21.15.146:0/3032412221 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c007440 con 0x7f6490071390 2024-09-13T18:27:24.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.206+0000 7f64962fb700 1 -- 172.21.15.146:0/3032412221 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64901f2760 con 0x7f6490071390 2024-09-13T18:27:24.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.210+0000 7f64962fb700 1 -- 172.21.15.146:0/3032412221 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64901f2980 con 0x7f6490071390 2024-09-13T18:27:24.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.210+0000 7f6481ffb700 1 -- 172.21.15.146:0/3032412221 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f648c0075a0 con 0x7f6490071390 2024-09-13T18:27:24.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.210+0000 7f6481ffb700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6474066730 0x7f6474068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:24.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.210+0000 7f6481ffb700 1 -- 172.21.15.146:0/3032412221 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f648c083600 con 0x7f6490071390 2024-09-13T18:27:24.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.210+0000 7f6494af8700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6474066730 0x7f6474068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:24.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.210+0000 7f6494af8700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6474066730 0x7f6474068bf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f64900b3af0 tx=0x7f6484008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:24.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.218+0000 7f6481ffb700 1 -- 172.21.15.146:0/3032412221 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f648c0505c0 con 0x7f6490071390 2024-09-13T18:27:24.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.430+0000 7f64962fb700 1 -- 172.21.15.146:0/3032412221 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64900cc670 con 0x7f6490071390 2024-09-13T18:27:24.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.430+0000 7f6481ffb700 1 -- 172.21.15.146:0/3032412221 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6286 (secure 0 0 0) 0x7f64900cc670 con 0x7f6490071390 2024-09-13T18:27:24.440 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:24.440 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:27:09.524480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"e","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:27:24.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.434+0000 7f64962fb700 1 -- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6474066730 msgr2=0x7f6474068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.434+0000 7f64962fb700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6474066730 0x7f6474068bf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f64900b3af0 tx=0x7f6484008040 comp rx=0 tx=0).stop 2024-09-13T18:27:24.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.434+0000 7f64962fb700 1 -- 172.21.15.146:0/3032412221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6490071390 msgr2=0x7f64901e6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.434+0000 7f64962fb700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6490071390 0x7f64901e6790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f648c001eb0 tx=0x7f648c00c7b0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.434+0000 7f64962fb700 1 -- 172.21.15.146:0/3032412221 shutdown_connections 2024-09-13T18:27:24.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.434+0000 7f64962fb700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6474066730 0x7f6474068bf0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.434+0000 7f64962fb700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64900cbd50 0x7f64901f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.434+0000 7f64962fb700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64900ca210 0x7f64901f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.434+0000 7f64962fb700 1 --2- 172.21.15.146:0/3032412221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6490071390 0x7f64901e6790 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.434+0000 7f64962fb700 1 -- 172.21.15.146:0/3032412221 >> 172.21.15.146:0/3032412221 conn(0x7f6490066270 msgr2=0x7f6490151b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:24.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.434+0000 7f64962fb700 1 -- 172.21.15.146:0/3032412221 shutdown_connections 2024-09-13T18:27:24.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.434+0000 7f64962fb700 1 -- 172.21.15.146:0/3032412221 wait complete. 2024-09-13T18:27:24.444 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 15 2024-09-13T18:27:24.455 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:27:24.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4ddae71700 1 Processor -- start 2024-09-13T18:27:24.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4ddae71700 1 -- start start 2024-09-13T18:27:24.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4ddae71700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40c6a80 0x7f4dd4161c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:24.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4ddae71700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd4153fa0 con 0x7f4dd40c6610 2024-09-13T18:27:24.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4ddae71700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd40aa500 con 0x7f4dd4153bc0 2024-09-13T18:27:24.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4ddae71700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd415a7e0 con 0x7f4dd40c6a80 2024-09-13T18:27:24.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4dd966e700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4dd40c6610 0x7f4dd40cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49174/0 (socket says 172.21.15.146:49174) 2024-09-13T18:27:24.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4dd966e700 1 -- 172.21.15.146:0/2992343900 learned_addr learned my addr 172.21.15.146:0/2992343900 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:27:24.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4dda670700 1 --2- 172.21.15.146:0/2992343900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40c6a80 0x7f4dd4161c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:24.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4dd8e6d700 1 -- 172.21.15.146:0/2992343900 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285608804 0 0) 0x7f4dd4153fa0 con 0x7f4dd40c6610 2024-09-13T18:27:24.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4dd8e6d700 1 -- 172.21.15.146:0/2992343900 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dc00035d0 con 0x7f4dd40c6610 2024-09-13T18:27:24.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4dda670700 1 -- 172.21.15.146:0/2992343900 >> v1:172.21.15.154:6789/0 conn(0x7f4dd4153bc0 legacy=0x7f4dd40c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4dd8e6d700 1 -- 172.21.15.146:0/2992343900 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280981 0 0) 0x7f4dd40aa500 con 0x7f4dd4153bc0 2024-09-13T18:27:24.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4dda670700 1 -- 172.21.15.146:0/2992343900 >> v1:172.21.15.146:6789/0 conn(0x7f4dd40c6610 legacy=0x7f4dd40cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.554+0000 7f4dda670700 1 -- 172.21.15.146:0/2992343900 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd40aa220 con 0x7f4dd40c6a80 2024-09-13T18:27:24.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.558+0000 7f4dda670700 1 --2- 172.21.15.146:0/2992343900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40c6a80 0x7f4dd4161c30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009dc0 tx=0x7f4dcc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0200ee6b10a98c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:24.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.558+0000 7f4dd8e6d700 1 -- 172.21.15.146:0/2992343900 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc00e070 con 0x7f4dd40c6a80 2024-09-13T18:27:24.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.558+0000 7f4dd8e6d700 1 -- 172.21.15.146:0/2992343900 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc003a40 con 0x7f4dd40c6a80 2024-09-13T18:27:24.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.558+0000 7f4dd8e6d700 1 -- 172.21.15.146:0/2992343900 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8001670 con 0x7f4dd40c6a80 2024-09-13T18:27:24.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.558+0000 7f4ddae71700 1 -- 172.21.15.146:0/2992343900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40c6a80 msgr2=0x7f4dd4161c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.558+0000 7f4ddae71700 1 --2- 172.21.15.146:0/2992343900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40c6a80 0x7f4dd4161c30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009dc0 tx=0x7f4dcc00b330 comp rx=0 tx=0).stop 2024-09-13T18:27:24.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.558+0000 7f4ddae71700 1 -- 172.21.15.146:0/2992343900 shutdown_connections 2024-09-13T18:27:24.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.558+0000 7f4ddae71700 1 --2- 172.21.15.146:0/2992343900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40c6a80 0x7f4dd4161c30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.558+0000 7f4ddae71700 1 -- 172.21.15.146:0/2992343900 >> 172.21.15.146:0/2992343900 conn(0x7f4dd4066090 msgr2=0x7f4dd4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:24.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.558+0000 7f4ddae71700 1 -- 172.21.15.146:0/2992343900 shutdown_connections 2024-09-13T18:27:24.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.558+0000 7f4ddae71700 1 -- 172.21.15.146:0/2992343900 wait complete. 2024-09-13T18:27:24.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.558+0000 7f4ddae71700 1 Processor -- start 2024-09-13T18:27:24.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.558+0000 7f4ddae71700 1 -- start start 2024-09-13T18:27:24.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4ddae71700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dd40c6610 0x7f4dd4151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:24.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4ddae71700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40c6a80 0x7f4dd40ac660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:24.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4ddae71700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd4153bc0 0x7f4dd40acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:24.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4ddae71700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40ad710 con 0x7f4dd40c6610 2024-09-13T18:27:24.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4ddae71700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40c25d0 con 0x7f4dd4153bc0 2024-09-13T18:27:24.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4ddae71700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40ad430 con 0x7f4dd40c6a80 2024-09-13T18:27:24.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dd9e6f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dd40c6610 0x7f4dd4151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:24.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dd9e6f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dd40c6610 0x7f4dd4151ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44290/0 (socket says 172.21.15.146:44290) 2024-09-13T18:27:24.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dd9e6f700 1 -- 172.21.15.146:0/738224251 learned_addr learned my addr 172.21.15.146:0/738224251 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:27:24.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dda670700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd4153bc0 0x7f4dd40acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:24.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dd966e700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40c6a80 0x7f4dd40ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:24.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dd9e6f700 1 -- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40c6a80 msgr2=0x7f4dd40ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dd9e6f700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40c6a80 0x7f4dd40ac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dd9e6f700 1 -- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd4153bc0 msgr2=0x7f4dd40acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dd9e6f700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd4153bc0 0x7f4dd40acba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dd9e6f700 1 -- 172.21.15.146:0/738224251 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc009a70 con 0x7f4dd40c6610 2024-09-13T18:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dd9e6f700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dd40c6610 0x7f4dd4151ed0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000c520 tx=0x7f4dd000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dc67fc700 1 -- 172.21.15.146:0/738224251 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00054e0 con 0x7f4dd40c6610 2024-09-13T18:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dc67fc700 1 -- 172.21.15.146:0/738224251 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd00036a0 con 0x7f4dd40c6610 2024-09-13T18:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4ddae71700 1 -- 172.21.15.146:0/738224251 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd40c1f80 con 0x7f4dd40c6610 2024-09-13T18:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4dc67fc700 1 -- 172.21.15.146:0/738224251 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0003a50 con 0x7f4dd40c6610 2024-09-13T18:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4ddae71700 1 -- 172.21.15.146:0/738224251 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd406e900 con 0x7f4dd40c6610 2024-09-13T18:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.562+0000 7f4ddae71700 1 -- 172.21.15.146:0/738224251 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd40a9de0 con 0x7f4dd40c6610 2024-09-13T18:27:24.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.566+0000 7f4dc67fc700 1 -- 172.21.15.146:0/738224251 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4dd0003bb0 con 0x7f4dd40c6610 2024-09-13T18:27:24.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.566+0000 7f4dc67fc700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4db8066730 0x7f4db8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:24.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.566+0000 7f4dc67fc700 1 -- 172.21.15.146:0/738224251 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4dd00838b0 con 0x7f4dd40c6610 2024-09-13T18:27:24.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.566+0000 7f4dd966e700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4db8066730 0x7f4db8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:24.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.566+0000 7f4dd966e700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4db8066730 0x7f4db8068bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8001b70 tx=0x7f4dc8001b00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:24.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.570+0000 7f4dc67fc700 1 -- 172.21.15.146:0/738224251 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4dd0050870 con 0x7f4dd40c6610 2024-09-13T18:27:24.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.742+0000 7f4ddae71700 1 -- 172.21.15.146:0/738224251 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4dd406f690 con 0x7f4dd40c6610 2024-09-13T18:27:24.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.746+0000 7f4dc67fc700 1 -- 172.21.15.146:0/738224251 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18724 (secure 0 0 0) 0x7f4dd0003e60 con 0x7f4dd40c6610 2024-09-13T18:27:24.755 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:27:24.755 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":27,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:27:22.593938+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.13","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:27:24.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.750+0000 7f4ddae71700 1 -- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4db8066730 msgr2=0x7f4db8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.750+0000 7f4ddae71700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4db8066730 0x7f4db8068bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8001b70 tx=0x7f4dc8001b00 comp rx=0 tx=0).stop 2024-09-13T18:27:24.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.750+0000 7f4ddae71700 1 -- 172.21.15.146:0/738224251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dd40c6610 msgr2=0x7f4dd4151ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:24.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.750+0000 7f4ddae71700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dd40c6610 0x7f4dd4151ed0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000c520 tx=0x7f4dd000c830 comp rx=0 tx=0).stop 2024-09-13T18:27:24.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.750+0000 7f4ddae71700 1 -- 172.21.15.146:0/738224251 shutdown_connections 2024-09-13T18:27:24.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.750+0000 7f4ddae71700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4db8066730 0x7f4db8068bf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.750+0000 7f4ddae71700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd4153bc0 0x7f4dd40acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.750+0000 7f4ddae71700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40c6a80 0x7f4dd40ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.750+0000 7f4ddae71700 1 --2- 172.21.15.146:0/738224251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dd40c6610 0x7f4dd4151ed0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:24.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.750+0000 7f4ddae71700 1 -- 172.21.15.146:0/738224251 >> 172.21.15.146:0/738224251 conn(0x7f4dd4066090 msgr2=0x7f4dd40c0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:24.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.750+0000 7f4ddae71700 1 -- 172.21.15.146:0/738224251 shutdown_connections 2024-09-13T18:27:24.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:27:24.750+0000 7f4ddae71700 1 -- 172.21.15.146:0/738224251 wait complete. 2024-09-13T18:27:24.770 DEBUG:tasks.mds_thrash:Yielding 2024-09-13T18:27:24.770 INFO:teuthology.run_tasks:Running task workunit... 2024-09-13T18:27:24.785 INFO:tasks.workunit:Pulling workunits from ref 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:27:24.785 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-13T18:27:24.785 INFO:tasks.workunit:timeout=3h 2024-09-13T18:27:24.785 INFO:tasks.workunit:cleanup=True 2024-09-13T18:27:24.786 DEBUG:teuthology.orchestra.run.smithi146:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-13T18:27:24.786 INFO:tasks.mds_thrash.fs.[cephfs]:starting mds_do_thrash for fs cephfs 2024-09-13T18:27:24.786 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 38 secs before thrashing 2024-09-13T18:27:24.794 INFO:teuthology.orchestra.run.smithi146.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-13T18:27:24.794 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-13T18:27:24.794 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 34h/52d Inode: 1 Links: 2 2024-09-13T18:27:24.794 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:27:24.794 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-13T18:27:24.794 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-13 18:26:37.328439674 +0000 2024-09-13T18:27:24.794 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-13 18:26:45.374173348 +0000 2024-09-13T18:27:24.794 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: - 2024-09-13T18:27:24.794 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-13T18:27:24.794 DEBUG:teuthology.orchestra.run.smithi146:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-13T18:27:24.826 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:27:24.834 INFO:tasks.workunit.client.0.smithi146.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-13T18:27:37.169 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:27:37.172 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:28:02.787 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:02.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.894+0000 7f2edd250700 1 Processor -- start 2024-09-13T18:28:02.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.894+0000 7f2edd250700 1 -- start start 2024-09-13T18:28:02.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2ed80c6960 0x7f2ed80cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:02.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ed80cc6e0 0x7f2ed806e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:02.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed8070dc0 con 0x7f2ed8151080 2024-09-13T18:28:02.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed80ccaf0 con 0x7f2ed80cc6e0 2024-09-13T18:28:02.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed8157470 con 0x7f2ed80c6960 2024-09-13T18:28:02.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edca4f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ed80cc6e0 0x7f2ed806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:02.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edca4f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ed80cc6e0 0x7f2ed806e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51554/0 (socket says 172.21.15.146:51554) 2024-09-13T18:28:02.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edca4f700 1 -- 172.21.15.146:0/3958381240 learned_addr learned my addr 172.21.15.146:0/3958381240 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:02.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2ed77fe700 1 --2- 172.21.15.146:0/3958381240 >> v2:172.21.15.154:3300/0 conn(0x7f2ed80c6960 0x7f2ed80cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:02.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edca4f700 1 -- 172.21.15.146:0/3958381240 >> v2:172.21.15.154:3300/0 conn(0x7f2ed80c6960 msgr2=0x7f2ed80cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:02.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edca4f700 1 --2- 172.21.15.146:0/3958381240 >> v2:172.21.15.154:3300/0 conn(0x7f2ed80c6960 0x7f2ed80cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:02.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edca4f700 1 -- 172.21.15.146:0/3958381240 >> v1:172.21.15.154:6789/0 conn(0x7f2ed8151080 legacy=0x7f2ed806f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:02.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edca4f700 1 -- 172.21.15.146:0/3958381240 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed8157750 con 0x7f2ed80cc6e0 2024-09-13T18:28:02.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edca4f700 1 --2- 172.21.15.146:0/3958381240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed80cc6e0 0x7f2ed806e650 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2ed806b060 tx=0x7f2ed000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2540a678da1c31f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:02.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2ed6ffd700 1 -- 172.21.15.146:0/3958381240 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000e070 con 0x7f2ed80cc6e0 2024-09-13T18:28:02.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2ed6ffd700 1 -- 172.21.15.146:0/3958381240 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed0003a40 con 0x7f2ed80cc6e0 2024-09-13T18:28:02.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2ed6ffd700 1 -- 172.21.15.146:0/3958381240 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0003df0 con 0x7f2ed80cc6e0 2024-09-13T18:28:02.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 -- 172.21.15.146:0/3958381240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed80cc6e0 msgr2=0x7f2ed806e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:02.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 --2- 172.21.15.146:0/3958381240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed80cc6e0 0x7f2ed806e650 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2ed806b060 tx=0x7f2ed000b330 comp rx=0 tx=0).stop 2024-09-13T18:28:02.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 -- 172.21.15.146:0/3958381240 shutdown_connections 2024-09-13T18:28:02.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 --2- 172.21.15.146:0/3958381240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed80cc6e0 0x7f2ed806e650 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:02.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 --2- 172.21.15.146:0/3958381240 >> v2:172.21.15.154:3300/0 conn(0x7f2ed80c6960 0x7f2ed80cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:02.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 -- 172.21.15.146:0/3958381240 >> 172.21.15.146:0/3958381240 conn(0x7f2ed8071ed0 msgr2=0x7f2ed80722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:02.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 -- 172.21.15.146:0/3958381240 shutdown_connections 2024-09-13T18:28:02.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 -- 172.21.15.146:0/3958381240 wait complete. 2024-09-13T18:28:02.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 Processor -- start 2024-09-13T18:28:02.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.898+0000 7f2edd250700 1 -- start start 2024-09-13T18:28:02.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2edd250700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed80c6960 0x7f2ed80cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:02.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2edd250700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ed80cc6e0 0x7f2ed80c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:02.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2edd250700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed8151080 0x7f2ed81e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:02.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2edd250700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed81402a0 con 0x7f2ed80c6960 2024-09-13T18:28:02.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2edd250700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed814bfd0 con 0x7f2ed80cc6e0 2024-09-13T18:28:02.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2edd250700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80cbca0 con 0x7f2ed8151080 2024-09-13T18:28:02.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed80c6960 0x7f2ed80cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:02.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed80c6960 0x7f2ed80cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51560/0 (socket says 172.21.15.146:51560) 2024-09-13T18:28:02.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed7fff700 1 -- 172.21.15.146:0/4196524897 learned_addr learned my addr 172.21.15.146:0/4196524897 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:02.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2edca4f700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed8151080 0x7f2ed81e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:02.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed77fe700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ed80cc6e0 0x7f2ed80c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:02.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed7fff700 1 -- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed8151080 msgr2=0x7f2ed81e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:02.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed7fff700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed8151080 0x7f2ed81e3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:02.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed7fff700 1 -- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ed80cc6e0 msgr2=0x7f2ed80c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:02.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed7fff700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ed80cc6e0 0x7f2ed80c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:02.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed7fff700 1 -- 172.21.15.146:0/4196524897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed0009b30 con 0x7f2ed80c6960 2024-09-13T18:28:02.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed7fff700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed80c6960 0x7f2ed80cb760 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800b4d0 tx=0x7f2ec800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:02.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed4ff9700 1 -- 172.21.15.146:0/4196524897 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8011830 con 0x7f2ed80c6960 2024-09-13T18:28:02.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed4ff9700 1 -- 172.21.15.146:0/4196524897 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec8011990 con 0x7f2ed80c6960 2024-09-13T18:28:02.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2edd250700 1 -- 172.21.15.146:0/4196524897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed814d210 con 0x7f2ed80c6960 2024-09-13T18:28:02.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed4ff9700 1 -- 172.21.15.146:0/4196524897 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8011d40 con 0x7f2ed80c6960 2024-09-13T18:28:02.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2edd250700 1 -- 172.21.15.146:0/4196524897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed81404f0 con 0x7f2ed80c6960 2024-09-13T18:28:02.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed4ff9700 1 -- 172.21.15.146:0/4196524897 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2ec801a400 con 0x7f2ed80c6960 2024-09-13T18:28:02.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2edd250700 1 -- 172.21.15.146:0/4196524897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed8074430 con 0x7f2ed80c6960 2024-09-13T18:28:02.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed4ff9700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2ec4066730 0x7f2ec4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:02.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.902+0000 7f2ed4ff9700 1 -- 172.21.15.146:0/4196524897 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2ec8084160 con 0x7f2ed80c6960 2024-09-13T18:28:02.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.906+0000 7f2ed77fe700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2ec4066730 0x7f2ec4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:02.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.906+0000 7f2ed77fe700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2ec4066730 0x7f2ec4068bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2ed806b4e0 tx=0x7f2ecc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:02.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:02.910+0000 7f2ed4ff9700 1 -- 172.21.15.146:0/4196524897 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2ec80511a0 con 0x7f2ed80c6960 2024-09-13T18:28:03.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.106+0000 7f2edd250700 1 -- 172.21.15.146:0/4196524897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ed8157140 con 0x7f2ed80c6960 2024-09-13T18:28:03.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.106+0000 7f2ed4ff9700 1 -- 172.21.15.146:0/4196524897 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6286 (secure 0 0 0) 0x7f2ec80547c0 con 0x7f2ed80c6960 2024-09-13T18:28:03.115 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:03.116 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:27:09.524480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"e","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:03.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.110+0000 7f2edd250700 1 -- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2ec4066730 msgr2=0x7f2ec4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:03.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.110+0000 7f2edd250700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2ec4066730 0x7f2ec4068bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2ed806b4e0 tx=0x7f2ecc008040 comp rx=0 tx=0).stop 2024-09-13T18:28:03.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.110+0000 7f2edd250700 1 -- 172.21.15.146:0/4196524897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed80c6960 msgr2=0x7f2ed80cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.110+0000 7f2edd250700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed80c6960 0x7f2ed80cb760 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800b4d0 tx=0x7f2ec800b890 comp rx=0 tx=0).stop 2024-09-13T18:28:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.110+0000 7f2edd250700 1 -- 172.21.15.146:0/4196524897 shutdown_connections 2024-09-13T18:28:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.110+0000 7f2edd250700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2ec4066730 0x7f2ec4068bf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.110+0000 7f2edd250700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed8151080 0x7f2ed81e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.110+0000 7f2edd250700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ed80cc6e0 0x7f2ed80c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.110+0000 7f2edd250700 1 --2- 172.21.15.146:0/4196524897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed80c6960 0x7f2ed80cb760 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.110+0000 7f2edd250700 1 -- 172.21.15.146:0/4196524897 >> 172.21.15.146:0/4196524897 conn(0x7f2ed8071ed0 msgr2=0x7f2ed806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.110+0000 7f2edd250700 1 -- 172.21.15.146:0/4196524897 shutdown_connections 2024-09-13T18:28:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.110+0000 7f2edd250700 1 -- 172.21.15.146:0/4196524897 wait complete. 2024-09-13T18:28:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 15 2024-09-13T18:28:03.131 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=2) 2024-09-13T18:28:03.131 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-09-13T18:28:03.131 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:28:03.141 INFO:tasks.ceph.mds.e:Stopped 2024-09-13T18:28:03.141 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:03.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75af33c700 1 Processor -- start 2024-09-13T18:28:03.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75af33c700 1 -- start start 2024-09-13T18:28:03.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75af33c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f75a8151590 0x7f75a81587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:03.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75af33c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a8158d10 con 0x7f75a806db40 2024-09-13T18:28:03.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75af33c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a8066500 con 0x7f75a8157fd0 2024-09-13T18:28:03.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75af33c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a80c69b0 con 0x7f75a8151590 2024-09-13T18:28:03.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75ae33a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f75a8151590 0x7f75a81587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:03.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75ae33a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f75a8151590 0x7f75a81587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51576/0 (socket says 172.21.15.146:51576) 2024-09-13T18:28:03.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75ae33a700 1 -- 172.21.15.146:0/733874554 learned_addr learned my addr 172.21.15.146:0/733874554 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:03.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75ae33a700 1 -- 172.21.15.146:0/733874554 >> v1:172.21.15.154:6789/0 conn(0x7f75a8157fd0 legacy=0x7f75a80c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:03.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75ae33a700 1 -- 172.21.15.146:0/733874554 >> v1:172.21.15.146:6789/0 conn(0x7f75a806db40 legacy=0x7f75a8161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:03.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75ad338700 1 -- 172.21.15.146:0/733874554 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200127138 0 0) 0x7f75a8158d10 con 0x7f75a806db40 2024-09-13T18:28:03.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75ae33a700 1 -- 172.21.15.146:0/733874554 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a806f740 con 0x7f75a8151590 2024-09-13T18:28:03.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75ae33a700 1 --2- 172.21.15.146:0/733874554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a8151590 0x7f75a81587d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009910 tx=0x7f75a4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e49443d9f407d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:03.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75ad338700 1 -- 172.21.15.146:0/733874554 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a400ba60 con 0x7f75a8151590 2024-09-13T18:28:03.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75ad338700 1 -- 172.21.15.146:0/733874554 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a400bbc0 con 0x7f75a8151590 2024-09-13T18:28:03.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75ad338700 1 -- 172.21.15.146:0/733874554 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4009e60 con 0x7f75a8151590 2024-09-13T18:28:03.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75af33c700 1 -- 172.21.15.146:0/733874554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a8151590 msgr2=0x7f75a81587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:03.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.302+0000 7f75af33c700 1 --2- 172.21.15.146:0/733874554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a8151590 0x7f75a81587d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009910 tx=0x7f75a4009c20 comp rx=0 tx=0).stop 2024-09-13T18:28:03.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75af33c700 1 -- 172.21.15.146:0/733874554 shutdown_connections 2024-09-13T18:28:03.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75af33c700 1 --2- 172.21.15.146:0/733874554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a8151590 0x7f75a81587d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:03.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75af33c700 1 -- 172.21.15.146:0/733874554 >> 172.21.15.146:0/733874554 conn(0x7f75a8066b00 msgr2=0x7f75a806aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:03.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75af33c700 1 -- 172.21.15.146:0/733874554 shutdown_connections 2024-09-13T18:28:03.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75af33c700 1 -- 172.21.15.146:0/733874554 wait complete. 2024-09-13T18:28:03.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75af33c700 1 Processor -- start 2024-09-13T18:28:03.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75af33c700 1 -- start start 2024-09-13T18:28:03.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75af33c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75a806db40 0x7f75a8153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:03.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75af33c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75a8151590 0x7f75a8147bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:03.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75af33c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a8157fd0 0x7f75a814f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:03.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75af33c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a8152990 con 0x7f75a8157fd0 2024-09-13T18:28:03.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75af33c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a8152c70 con 0x7f75a8151590 2024-09-13T18:28:03.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75af33c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a8153640 con 0x7f75a806db40 2024-09-13T18:28:03.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75aeb3b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a8157fd0 0x7f75a814f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:03.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75aeb3b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a8157fd0 0x7f75a814f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51584/0 (socket says 172.21.15.146:51584) 2024-09-13T18:28:03.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75aeb3b700 1 -- 172.21.15.146:0/2678839403 learned_addr learned my addr 172.21.15.146:0/2678839403 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:03.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75adb39700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75a8151590 0x7f75a8147bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:03.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75ae33a700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75a806db40 0x7f75a8153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:03.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75aeb3b700 1 -- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75a806db40 msgr2=0x7f75a8153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:03.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75aeb3b700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75a806db40 0x7f75a8153100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:03.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75aeb3b700 1 -- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75a8151590 msgr2=0x7f75a8147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:03.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75aeb3b700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75a8151590 0x7f75a8147bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:03.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75aeb3b700 1 -- 172.21.15.146:0/2678839403 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a4009570 con 0x7f75a8157fd0 2024-09-13T18:28:03.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75adb39700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75a8151590 0x7f75a8147bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:03.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f75aeb3b700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a8157fd0 0x7f75a814f540 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f75a000cca0 tx=0x7f75a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:03.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.306+0000 7f759effd700 1 -- 172.21.15.146:0/2678839403 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a000cea0 con 0x7f75a8157fd0 2024-09-13T18:28:03.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.310+0000 7f759effd700 1 -- 172.21.15.146:0/2678839403 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a0003e70 con 0x7f75a8157fd0 2024-09-13T18:28:03.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.310+0000 7f75af33c700 1 -- 172.21.15.146:0/2678839403 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a8154180 con 0x7f75a8157fd0 2024-09-13T18:28:03.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.310+0000 7f759effd700 1 -- 172.21.15.146:0/2678839403 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a000abb0 con 0x7f75a8157fd0 2024-09-13T18:28:03.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.310+0000 7f75af33c700 1 -- 172.21.15.146:0/2678839403 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a8154740 con 0x7f75a8157fd0 2024-09-13T18:28:03.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.310+0000 7f75af33c700 1 -- 172.21.15.146:0/2678839403 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a8152db0 con 0x7f75a8157fd0 2024-09-13T18:28:03.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.310+0000 7f759effd700 1 -- 172.21.15.146:0/2678839403 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f75a000ad10 con 0x7f75a8157fd0 2024-09-13T18:28:03.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.310+0000 7f759effd700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f758c066760 0x7f758c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:03.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.310+0000 7f759effd700 1 -- 172.21.15.146:0/2678839403 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f75a0084010 con 0x7f75a8157fd0 2024-09-13T18:28:03.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.310+0000 7f75ae33a700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f758c066760 0x7f758c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:03.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.314+0000 7f75ae33a700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f758c066760 0x7f758c068c20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f75a4006f80 tx=0x7f75a4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:03.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.318+0000 7f759effd700 1 -- 172.21.15.146:0/2678839403 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f75a0051050 con 0x7f75a8157fd0 2024-09-13T18:28:03.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.514+0000 7f75af33c700 1 -- 172.21.15.146:0/2678839403 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75a806d780 con 0x7f75a8157fd0 2024-09-13T18:28:03.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.514+0000 7f759effd700 1 -- 172.21.15.146:0/2678839403 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6286 (secure 0 0 0) 0x7f75a0017090 con 0x7f75a8157fd0 2024-09-13T18:28:03.524 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:03.524 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:27:09.524480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"e","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:03.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.518+0000 7f75af33c700 1 -- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f758c066760 msgr2=0x7f758c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:03.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.518+0000 7f75af33c700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f758c066760 0x7f758c068c20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f75a4006f80 tx=0x7f75a4008030 comp rx=0 tx=0).stop 2024-09-13T18:28:03.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.518+0000 7f75af33c700 1 -- 172.21.15.146:0/2678839403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a8157fd0 msgr2=0x7f75a814f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:03.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.518+0000 7f75af33c700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a8157fd0 0x7f75a814f540 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f75a000cca0 tx=0x7f75a000a340 comp rx=0 tx=0).stop 2024-09-13T18:28:03.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.518+0000 7f75af33c700 1 -- 172.21.15.146:0/2678839403 shutdown_connections 2024-09-13T18:28:03.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.518+0000 7f75af33c700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f758c066760 0x7f758c068c20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:03.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.518+0000 7f75af33c700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a8157fd0 0x7f75a814f540 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:03.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.518+0000 7f75af33c700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75a8151590 0x7f75a8147bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:03.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.518+0000 7f75af33c700 1 --2- 172.21.15.146:0/2678839403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75a806db40 0x7f75a8153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:03.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.518+0000 7f75af33c700 1 -- 172.21.15.146:0/2678839403 >> 172.21.15.146:0/2678839403 conn(0x7f75a8066b00 msgr2=0x7f75a8150310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:03.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.518+0000 7f75af33c700 1 -- 172.21.15.146:0/2678839403 shutdown_connections 2024-09-13T18:28:03.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:03.518+0000 7f75af33c700 1 -- 172.21.15.146:0/2678839403 wait complete. 2024-09-13T18:28:03.527 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 15 2024-09-13T18:28:03.536 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:28:05.537 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:05.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 Processor -- start 2024-09-13T18:28:05.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 -- start start 2024-09-13T18:28:05.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab740d8b80 0x7fab74007950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:05.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fab740e1cc0 0x7fab740e2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:05.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab740478d0 con 0x7fab7404b7e0 2024-09-13T18:28:05.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab74008110 con 0x7fab740e1cc0 2024-09-13T18:28:05.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab74047b90 con 0x7fab740d8b80 2024-09-13T18:28:05.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab81ef4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fab740e1cc0 0x7fab740e2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:05.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab81ef4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fab740e1cc0 0x7fab740e2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51592/0 (socket says 172.21.15.146:51592) 2024-09-13T18:28:05.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab81ef4700 1 -- 172.21.15.146:0/34763052 learned_addr learned my addr 172.21.15.146:0/34763052 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:05.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab816f3700 1 --2- 172.21.15.146:0/34763052 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab740d8b80 0x7fab74007950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:05.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab81ef4700 1 -- 172.21.15.146:0/34763052 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab740d8b80 msgr2=0x7fab74007950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:05.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab81ef4700 1 --2- 172.21.15.146:0/34763052 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab740d8b80 0x7fab74007950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:05.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab81ef4700 1 -- 172.21.15.146:0/34763052 >> v1:172.21.15.146:6789/0 conn(0x7fab7404b7e0 legacy=0x7fab74045f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:05.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab81ef4700 1 -- 172.21.15.146:0/34763052 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab74047e70 con 0x7fab740e1cc0 2024-09-13T18:28:05.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab816f3700 1 --2- 172.21.15.146:0/34763052 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab740d8b80 0x7fab74007950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:05.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab81ef4700 1 --2- 172.21.15.146:0/34763052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab740e1cc0 0x7fab740e2170 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0731e0 tx=0x7fab7c061ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0fd891e268a5d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:05.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab73fff700 1 -- 172.21.15.146:0/34763052 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c0739c0 con 0x7fab740e1cc0 2024-09-13T18:28:05.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab73fff700 1 -- 172.21.15.146:0/34763052 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c075a30 con 0x7fab740e1cc0 2024-09-13T18:28:05.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab73fff700 1 -- 172.21.15.146:0/34763052 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c075de0 con 0x7fab740e1cc0 2024-09-13T18:28:05.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 -- 172.21.15.146:0/34763052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab740e1cc0 msgr2=0x7fab740e2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:05.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 --2- 172.21.15.146:0/34763052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab740e1cc0 0x7fab740e2170 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0731e0 tx=0x7fab7c061ef0 comp rx=0 tx=0).stop 2024-09-13T18:28:05.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 -- 172.21.15.146:0/34763052 shutdown_connections 2024-09-13T18:28:05.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 --2- 172.21.15.146:0/34763052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab740e1cc0 0x7fab740e2170 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:05.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 --2- 172.21.15.146:0/34763052 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab740d8b80 0x7fab74007950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:05.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 -- 172.21.15.146:0/34763052 >> 172.21.15.146:0/34763052 conn(0x7fab7404d8e0 msgr2=0x7fab7404dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:05.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 -- 172.21.15.146:0/34763052 shutdown_connections 2024-09-13T18:28:05.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.662+0000 7fab826f5700 1 -- 172.21.15.146:0/34763052 wait complete. 2024-09-13T18:28:05.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab826f5700 1 Processor -- start 2024-09-13T18:28:05.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab826f5700 1 -- start start 2024-09-13T18:28:05.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab826f5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7404b7e0 0x7fab740dc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:05.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab826f5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab740d8b80 0x7fab7404a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:05.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab826f5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab740e1cc0 0x7fab74177fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:05.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab826f5700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab740dd390 con 0x7fab740e1cc0 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab826f5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab740ddf40 con 0x7fab7404b7e0 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab826f5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab740dcb30 con 0x7fab740d8b80 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab81ef4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab740e1cc0 0x7fab74177fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab81ef4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab740e1cc0 0x7fab74177fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51608/0 (socket says 172.21.15.146:51608) 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab81ef4700 1 -- 172.21.15.146:0/2757526866 learned_addr learned my addr 172.21.15.146:0/2757526866 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab80ef2700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab740d8b80 0x7fab7404a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab816f3700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7404b7e0 0x7fab740dc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab81ef4700 1 -- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab740d8b80 msgr2=0x7fab7404a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab81ef4700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab740d8b80 0x7fab7404a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab81ef4700 1 -- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7404b7e0 msgr2=0x7fab740dc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab81ef4700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7404b7e0 0x7fab740dc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab81ef4700 1 -- 172.21.15.146:0/2757526866 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c04f5c0 con 0x7fab740e1cc0 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab81ef4700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab740e1cc0 0x7fab74177fb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fab7c000b60 tx=0x7fab7c04f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:05.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab71ffb700 1 -- 172.21.15.146:0/2757526866 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c099070 con 0x7fab740e1cc0 2024-09-13T18:28:05.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab71ffb700 1 -- 172.21.15.146:0/2757526866 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c067c50 con 0x7fab740e1cc0 2024-09-13T18:28:05.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab71ffb700 1 -- 172.21.15.146:0/2757526866 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c067fc0 con 0x7fab740e1cc0 2024-09-13T18:28:05.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab826f5700 1 -- 172.21.15.146:0/2757526866 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab740deda0 con 0x7fab740e1cc0 2024-09-13T18:28:05.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab826f5700 1 -- 172.21.15.146:0/2757526866 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab740de7e0 con 0x7fab740e1cc0 2024-09-13T18:28:05.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.666+0000 7fab826f5700 1 -- 172.21.15.146:0/2757526866 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7404bbe0 con 0x7fab740e1cc0 2024-09-13T18:28:05.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.670+0000 7fab71ffb700 1 -- 172.21.15.146:0/2757526866 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fab7c09c070 con 0x7fab740e1cc0 2024-09-13T18:28:05.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.670+0000 7fab71ffb700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab60066730 0x7fab60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:05.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.670+0000 7fab71ffb700 1 -- 172.21.15.146:0/2757526866 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fab7c100490 con 0x7fab740e1cc0 2024-09-13T18:28:05.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.670+0000 7fab816f3700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab60066730 0x7fab60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:05.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.670+0000 7fab816f3700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab60066730 0x7fab60068bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fab7800a9a0 tx=0x7fab78008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:05.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.678+0000 7fab71ffb700 1 -- 172.21.15.146:0/2757526866 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fab7c0cd6d0 con 0x7fab740e1cc0 2024-09-13T18:28:05.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab826f5700 1 -- 172.21.15.146:0/2757526866 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab74046a60 con 0x7fab740e1cc0 2024-09-13T18:28:05.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab71ffb700 1 -- 172.21.15.146:0/2757526866 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6286 (secure 0 0 0) 0x7fab7c0d0cf0 con 0x7fab740e1cc0 2024-09-13T18:28:05.913 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:05.913 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:27:09.524480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"e","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:05.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab826f5700 1 -- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab60066730 msgr2=0x7fab60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:05.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab826f5700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab60066730 0x7fab60068bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fab7800a9a0 tx=0x7fab78008040 comp rx=0 tx=0).stop 2024-09-13T18:28:05.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab826f5700 1 -- 172.21.15.146:0/2757526866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab740e1cc0 msgr2=0x7fab74177fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:05.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab826f5700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab740e1cc0 0x7fab74177fb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fab7c000b60 tx=0x7fab7c04f750 comp rx=0 tx=0).stop 2024-09-13T18:28:05.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab826f5700 1 -- 172.21.15.146:0/2757526866 shutdown_connections 2024-09-13T18:28:05.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab826f5700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab60066730 0x7fab60068bf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:05.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab826f5700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab740e1cc0 0x7fab74177fb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:05.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab826f5700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab740d8b80 0x7fab7404a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:05.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab826f5700 1 --2- 172.21.15.146:0/2757526866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7404b7e0 0x7fab740dc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:05.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab826f5700 1 -- 172.21.15.146:0/2757526866 >> 172.21.15.146:0/2757526866 conn(0x7fab7404d8e0 msgr2=0x7fab740e11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:05.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab826f5700 1 -- 172.21.15.146:0/2757526866 shutdown_connections 2024-09-13T18:28:05.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:05.906+0000 7fab826f5700 1 -- 172.21.15.146:0/2757526866 wait complete. 2024-09-13T18:28:05.916 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 15 2024-09-13T18:28:05.925 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:28:07.225 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:28:07.227 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:28:07.926 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:08.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.050+0000 7f197d630700 1 Processor -- start 2024-09-13T18:28:08.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.050+0000 7f197d630700 1 -- start start 2024-09-13T18:28:08.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.050+0000 7f197d630700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f19780c61b0 0x7f197815d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:08.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.050+0000 7f197d630700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19780c5440 0x7f1978153340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:08.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.050+0000 7f197d630700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1978066780 con 0x7f19780c6590 2024-09-13T18:28:08.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.050+0000 7f197d630700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19781594a0 con 0x7f19780c61b0 2024-09-13T18:28:08.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.050+0000 7f197d630700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197815d960 con 0x7f19780c5440 2024-09-13T18:28:08.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.050+0000 7f197ce2f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19780c5440 0x7f1978153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:08.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.050+0000 7f1977fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f19780c61b0 0x7f197815d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:08.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.050+0000 7f19777fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f19780c6590 0x7f197806cd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:54166/0 (socket says 172.21.15.146:54166) 2024-09-13T18:28:08.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.050+0000 7f197ce2f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19780c5440 0x7f1978153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:52906/0 (socket says 172.21.15.146:52906) 2024-09-13T18:28:08.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.050+0000 7f19777fe700 1 -- 172.21.15.146:0/135563285 learned_addr learned my addr 172.21.15.146:0/135563285 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:28:08.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197ce2f700 1 -- 172.21.15.146:0/135563285 >> v2:172.21.15.154:3300/0 conn(0x7f19780c61b0 msgr2=0x7f197815d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:08.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197ce2f700 1 --2- 172.21.15.146:0/135563285 >> v2:172.21.15.154:3300/0 conn(0x7f19780c61b0 0x7f197815d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:08.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197ce2f700 1 -- 172.21.15.146:0/135563285 >> v1:172.21.15.154:6789/0 conn(0x7f19780c6590 legacy=0x7f197806cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:08.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197ce2f700 1 -- 172.21.15.146:0/135563285 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197806eec0 con 0x7f19780c5440 2024-09-13T18:28:08.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197ce2f700 1 --2- 172.21.15.146:0/135563285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19780c5440 0x7f1978153340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1970009d90 tx=0x7f197000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12d2ce7750972a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:08.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f1976ffd700 1 -- 172.21.15.146:0/135563285 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197000e070 con 0x7f19780c5440 2024-09-13T18:28:08.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f1976ffd700 1 -- 172.21.15.146:0/135563285 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197000bb20 con 0x7f19780c5440 2024-09-13T18:28:08.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f1976ffd700 1 -- 172.21.15.146:0/135563285 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970012430 con 0x7f19780c5440 2024-09-13T18:28:08.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197d630700 1 -- 172.21.15.146:0/135563285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19780c5440 msgr2=0x7f1978153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:08.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197d630700 1 --2- 172.21.15.146:0/135563285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19780c5440 0x7f1978153340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1970009d90 tx=0x7f197000b330 comp rx=0 tx=0).stop 2024-09-13T18:28:08.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197d630700 1 -- 172.21.15.146:0/135563285 shutdown_connections 2024-09-13T18:28:08.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197d630700 1 --2- 172.21.15.146:0/135563285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19780c5440 0x7f1978153340 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:08.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197d630700 1 --2- 172.21.15.146:0/135563285 >> v2:172.21.15.154:3300/0 conn(0x7f19780c61b0 0x7f197815d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:08.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197d630700 1 -- 172.21.15.146:0/135563285 >> 172.21.15.146:0/135563285 conn(0x7f1978066b00 msgr2=0x7f197806aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:08.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197d630700 1 -- 172.21.15.146:0/135563285 shutdown_connections 2024-09-13T18:28:08.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197d630700 1 -- 172.21.15.146:0/135563285 wait complete. 2024-09-13T18:28:08.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197d630700 1 Processor -- start 2024-09-13T18:28:08.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.054+0000 7f197d630700 1 -- start start 2024-09-13T18:28:08.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f197d630700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19780c5440 0x7f19781529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:08.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f197d630700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19780c61b0 0x7f197814eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:08.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f197d630700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19780c6590 0x7f1978147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f197d630700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19780c7db0 con 0x7f19780c5440 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f197d630700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197815b720 con 0x7f19780c61b0 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f197d630700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19781f2ad0 con 0x7f19780c6590 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1977fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19780c5440 0x7f19781529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1977fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19780c5440 0x7f19781529b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51612/0 (socket says 172.21.15.146:51612) 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1977fff700 1 -- 172.21.15.146:0/1755295007 learned_addr learned my addr 172.21.15.146:0/1755295007 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f197ce2f700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19780c6590 0x7f1978147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f19777fe700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19780c61b0 0x7f197814eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1977fff700 1 -- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19780c6590 msgr2=0x7f1978147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1977fff700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19780c6590 0x7f1978147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1977fff700 1 -- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19780c61b0 msgr2=0x7f197814eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1977fff700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19780c61b0 0x7f197814eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1977fff700 1 -- 172.21.15.146:0/1755295007 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1970009a70 con 0x7f19780c5440 2024-09-13T18:28:08.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1977fff700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19780c5440 0x7f19781529b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f196800e9d0 tx=0x7f196800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:08.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1974ff9700 1 -- 172.21.15.146:0/1755295007 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968009720 con 0x7f19780c5440 2024-09-13T18:28:08.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f197d630700 1 -- 172.21.15.146:0/1755295007 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1978151d90 con 0x7f19780c5440 2024-09-13T18:28:08.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f197d630700 1 -- 172.21.15.146:0/1755295007 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19781f21a0 con 0x7f19780c5440 2024-09-13T18:28:08.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1974ff9700 1 -- 172.21.15.146:0/1755295007 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196800c850 con 0x7f19780c5440 2024-09-13T18:28:08.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1974ff9700 1 -- 172.21.15.146:0/1755295007 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196800cbc0 con 0x7f19780c5440 2024-09-13T18:28:08.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f197d630700 1 -- 172.21.15.146:0/1755295007 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1964005310 con 0x7f19780c5440 2024-09-13T18:28:08.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1974ff9700 1 -- 172.21.15.146:0/1755295007 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f196800cd20 con 0x7f19780c5440 2024-09-13T18:28:08.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.058+0000 7f1974ff9700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1960066880 0x7f1960068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:08.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.062+0000 7f1974ff9700 1 -- 172.21.15.146:0/1755295007 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1968084600 con 0x7f19780c5440 2024-09-13T18:28:08.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.062+0000 7f19777fe700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1960066880 0x7f1960068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:08.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.062+0000 7f19777fe700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1960066880 0x7f1960068d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f19780c8710 tx=0x7f196c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:08.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.066+0000 7f1974ff9700 1 -- 172.21.15.146:0/1755295007 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f19680515b0 con 0x7f19780c5440 2024-09-13T18:28:08.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.298+0000 7f197d630700 1 -- 172.21.15.146:0/1755295007 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19640055c0 con 0x7f19780c5440 2024-09-13T18:28:08.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.298+0000 7f1974ff9700 1 -- 172.21.15.146:0/1755295007 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6286 (secure 0 0 0) 0x7f1968054bd0 con 0x7f19780c5440 2024-09-13T18:28:08.306 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:08.306 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:27:09.524480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"e","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:08.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.302+0000 7f197d630700 1 -- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1960066880 msgr2=0x7f1960068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:08.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.302+0000 7f197d630700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1960066880 0x7f1960068d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f19780c8710 tx=0x7f196c009040 comp rx=0 tx=0).stop 2024-09-13T18:28:08.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.302+0000 7f197d630700 1 -- 172.21.15.146:0/1755295007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19780c5440 msgr2=0x7f19781529b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:08.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.302+0000 7f197d630700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19780c5440 0x7f19781529b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f196800e9d0 tx=0x7f196800ed90 comp rx=0 tx=0).stop 2024-09-13T18:28:08.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.302+0000 7f197d630700 1 -- 172.21.15.146:0/1755295007 shutdown_connections 2024-09-13T18:28:08.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.302+0000 7f197d630700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1960066880 0x7f1960068d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:08.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.302+0000 7f197d630700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19780c6590 0x7f1978147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:08.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.302+0000 7f197d630700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19780c61b0 0x7f197814eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:08.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.302+0000 7f197d630700 1 --2- 172.21.15.146:0/1755295007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19780c5440 0x7f19781529b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:08.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.302+0000 7f197d630700 1 -- 172.21.15.146:0/1755295007 >> 172.21.15.146:0/1755295007 conn(0x7f1978066b00 msgr2=0x7f19780cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:08.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.302+0000 7f197d630700 1 -- 172.21.15.146:0/1755295007 shutdown_connections 2024-09-13T18:28:08.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:08.302+0000 7f197d630700 1 -- 172.21.15.146:0/1755295007 wait complete. 2024-09-13T18:28:08.314 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 15 2024-09-13T18:28:08.318 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:28:10.320 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:10.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.410+0000 7fc87359e700 1 Processor -- start 2024-09-13T18:28:10.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.410+0000 7fc87359e700 1 -- start start 2024-09-13T18:28:10.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.410+0000 7fc87359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc87406e7c0 0x7fc8740b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.410+0000 7fc87359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc87406eba0 0x7fc8740c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.410+0000 7fc87359e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8740c1020 con 0x7fc8741598f0 2024-09-13T18:28:10.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.410+0000 7fc87359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8741593f0 con 0x7fc87406eba0 2024-09-13T18:28:10.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.410+0000 7fc87359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740c6a70 con 0x7fc87406e7c0 2024-09-13T18:28:10.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.410+0000 7fc872d9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc8741598f0 0x7fc8741531a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57598/0 (socket says 172.21.15.146:57598) 2024-09-13T18:28:10.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.410+0000 7fc872d9d700 1 -- 172.21.15.146:0/1916296380 learned_addr learned my addr 172.21.15.146:0/1916296380 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:28:10.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.410+0000 7fc87259c700 1 --2- 172.21.15.146:0/1916296380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc87406e7c0 0x7fc8740b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.410+0000 7fc86bfff700 1 --2- 172.21.15.146:0/1916296380 >> v2:172.21.15.154:3300/0 conn(0x7fc87406eba0 0x7fc8740c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.410+0000 7fc871d9b700 1 -- 172.21.15.146:0/1916296380 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426637200 0 0) 0x7fc8740c1020 con 0x7fc8741598f0 2024-09-13T18:28:10.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc871d9b700 1 -- 172.21.15.146:0/1916296380 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8600035d0 con 0x7fc8741598f0 2024-09-13T18:28:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87259c700 1 -- 172.21.15.146:0/1916296380 >> v2:172.21.15.154:3300/0 conn(0x7fc87406eba0 msgr2=0x7fc8740c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87259c700 1 --2- 172.21.15.146:0/1916296380 >> v2:172.21.15.154:3300/0 conn(0x7fc87406eba0 0x7fc8740c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87259c700 1 -- 172.21.15.146:0/1916296380 >> v1:172.21.15.146:6789/0 conn(0x7fc8741598f0 legacy=0x7fc8741531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc871d9b700 1 -- 172.21.15.146:0/1916296380 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3019311466 0 0) 0x7fc8600035d0 con 0x7fc8741598f0 2024-09-13T18:28:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87259c700 1 -- 172.21.15.146:0/1916296380 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8740c1290 con 0x7fc87406e7c0 2024-09-13T18:28:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87259c700 1 --2- 172.21.15.146:0/1916296380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc87406e7c0 0x7fc8740b11a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc864009850 tx=0x7fc864009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=167ad91fdc9e7bf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc871d9b700 1 -- 172.21.15.146:0/1916296380 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0019f0 con 0x7fc87406e7c0 2024-09-13T18:28:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc871d9b700 1 -- 172.21.15.146:0/1916296380 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc864004030 con 0x7fc87406e7c0 2024-09-13T18:28:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc871d9b700 1 -- 172.21.15.146:0/1916296380 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8640043a0 con 0x7fc87406e7c0 2024-09-13T18:28:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87359e700 1 -- 172.21.15.146:0/1916296380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc87406e7c0 msgr2=0x7fc8740b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87359e700 1 --2- 172.21.15.146:0/1916296380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc87406e7c0 0x7fc8740b11a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc864009850 tx=0x7fc864009b60 comp rx=0 tx=0).stop 2024-09-13T18:28:10.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87359e700 1 -- 172.21.15.146:0/1916296380 shutdown_connections 2024-09-13T18:28:10.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87359e700 1 --2- 172.21.15.146:0/1916296380 >> v2:172.21.15.154:3300/0 conn(0x7fc87406eba0 0x7fc8740c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87359e700 1 --2- 172.21.15.146:0/1916296380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc87406e7c0 0x7fc8740b11a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87359e700 1 -- 172.21.15.146:0/1916296380 >> 172.21.15.146:0/1916296380 conn(0x7fc874071ed0 msgr2=0x7fc8740722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:10.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87359e700 1 -- 172.21.15.146:0/1916296380 shutdown_connections 2024-09-13T18:28:10.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87359e700 1 -- 172.21.15.146:0/1916296380 wait complete. 2024-09-13T18:28:10.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87359e700 1 Processor -- start 2024-09-13T18:28:10.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.414+0000 7fc87359e700 1 -- start start 2024-09-13T18:28:10.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc87359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc87406e7c0 0x7fc8741f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc87359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc87406eba0 0x7fc8741ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc87359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8741598f0 0x7fc87406fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc87359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc874070360 con 0x7fc8741598f0 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc87359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741f29c0 con 0x7fc87406e7c0 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc87359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740c89b0 con 0x7fc87406eba0 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc872d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8741598f0 0x7fc87406fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc872d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8741598f0 0x7fc87406fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51614/0 (socket says 172.21.15.146:51614) 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc872d9d700 1 -- 172.21.15.146:0/2785277703 learned_addr learned my addr 172.21.15.146:0/2785277703 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc86bfff700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc87406eba0 0x7fc8741ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc87259c700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc87406e7c0 0x7fc8741f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc872d9d700 1 -- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc87406eba0 msgr2=0x7fc8741ef3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc872d9d700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc87406eba0 0x7fc8741ef3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc872d9d700 1 -- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc87406e7c0 msgr2=0x7fc8741f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc872d9d700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc87406e7c0 0x7fc8741f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc872d9d700 1 -- 172.21.15.146:0/2785277703 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8640094b0 con 0x7fc8741598f0 2024-09-13T18:28:10.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc87259c700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc87406e7c0 0x7fc8741f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:28:10.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc872d9d700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8741598f0 0x7fc87406fe20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc86c001ba0 tx=0x7fc86c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:10.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc86affd700 1 -- 172.21.15.146:0/2785277703 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00ab70 con 0x7fc8741598f0 2024-09-13T18:28:10.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc86affd700 1 -- 172.21.15.146:0/2785277703 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c00acd0 con 0x7fc8741598f0 2024-09-13T18:28:10.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc86affd700 1 -- 172.21.15.146:0/2785277703 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c007310 con 0x7fc8741598f0 2024-09-13T18:28:10.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc87359e700 1 -- 172.21.15.146:0/2785277703 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8740c8c90 con 0x7fc8741598f0 2024-09-13T18:28:10.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc87359e700 1 -- 172.21.15.146:0/2785277703 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8741f2f40 con 0x7fc8741598f0 2024-09-13T18:28:10.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.418+0000 7fc87359e700 1 -- 172.21.15.146:0/2785277703 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8741f3160 con 0x7fc8741598f0 2024-09-13T18:28:10.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.422+0000 7fc86affd700 1 -- 172.21.15.146:0/2785277703 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc86c007470 con 0x7fc8741598f0 2024-09-13T18:28:10.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.422+0000 7fc86affd700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc854066730 0x7fc854068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.422+0000 7fc86affd700 1 -- 172.21.15.146:0/2785277703 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc86c083fc0 con 0x7fc8741598f0 2024-09-13T18:28:10.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.422+0000 7fc87259c700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc854066730 0x7fc854068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.422+0000 7fc87259c700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc854066730 0x7fc854068bf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc8640090b0 tx=0x7fc864013040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:10.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.426+0000 7fc86affd700 1 -- 172.21.15.146:0/2785277703 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc86c0b5db0 con 0x7fc8741598f0 2024-09-13T18:28:10.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc87359e700 1 -- 172.21.15.146:0/2785277703 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8740b11a0 con 0x7fc8741598f0 2024-09-13T18:28:10.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc86affd700 1 -- 172.21.15.146:0/2785277703 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6286 (secure 0 0 0) 0x7fc86c018020 con 0x7fc8741598f0 2024-09-13T18:28:10.665 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:10.665 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:27:09.524480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"e","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:10.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc87359e700 1 -- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc854066730 msgr2=0x7fc854068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc87359e700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc854066730 0x7fc854068bf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc8640090b0 tx=0x7fc864013040 comp rx=0 tx=0).stop 2024-09-13T18:28:10.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc87359e700 1 -- 172.21.15.146:0/2785277703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8741598f0 msgr2=0x7fc87406fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc87359e700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8741598f0 0x7fc87406fe20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc86c001ba0 tx=0x7fc86c00cf50 comp rx=0 tx=0).stop 2024-09-13T18:28:10.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc87359e700 1 -- 172.21.15.146:0/2785277703 shutdown_connections 2024-09-13T18:28:10.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc87359e700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc854066730 0x7fc854068bf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc87359e700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8741598f0 0x7fc87406fe20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc87359e700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc87406eba0 0x7fc8741ef3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc87359e700 1 --2- 172.21.15.146:0/2785277703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc87406e7c0 0x7fc8741f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc87359e700 1 -- 172.21.15.146:0/2785277703 >> 172.21.15.146:0/2785277703 conn(0x7fc874071ed0 msgr2=0x7fc8740722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:10.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc87359e700 1 -- 172.21.15.146:0/2785277703 shutdown_connections 2024-09-13T18:28:10.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:10.658+0000 7fc87359e700 1 -- 172.21.15.146:0/2785277703 wait complete. 2024-09-13T18:28:10.668 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 15 2024-09-13T18:28:10.677 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:28:12.678 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:12.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99ed476700 1 Processor -- start 2024-09-13T18:28:12.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99ed476700 1 -- start start 2024-09-13T18:28:12.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99ed476700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99e8150610 0x7f99e806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:12.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99ed476700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e8071410 con 0x7f99e8071610 2024-09-13T18:28:12.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99ed476700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e8156090 con 0x7f99e8150230 2024-09-13T18:28:12.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99ed476700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e81596b0 con 0x7f99e8150610 2024-09-13T18:28:12.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99ecc75700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f99e8071610 0x7f99e8159b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:36420/0 (socket says 172.21.15.146:36420) 2024-09-13T18:28:12.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99ecc75700 1 -- 172.21.15.146:0/2143438467 learned_addr learned my addr 172.21.15.146:0/2143438467 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:28:12.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99e77fe700 1 --2- 172.21.15.146:0/2143438467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99e8150610 0x7f99e806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:12.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380959777 0 0) 0x7f99e8071410 con 0x7f99e8071610 2024-09-13T18:28:12.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99d00035d0 con 0x7f99e8071610 2024-09-13T18:28:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 961783878 0 0) 0x7f99d00035d0 con 0x7f99e8071610 2024-09-13T18:28:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99e8150610 msgr2=0x7f99e806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99e6ffd700 1 --2- 172.21.15.146:0/2143438467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99e8150610 0x7f99e806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 >> v1:172.21.15.154:6789/0 conn(0x7f99e8150230 legacy=0x7f99e806ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e8156350 con 0x7f99e8071610 2024-09-13T18:28:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.770+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f99e00019f0 con 0x7f99e8071610 2024-09-13T18:28:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 >> v1:172.21.15.146:6789/0 conn(0x7f99e8071610 legacy=0x7f99e8159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e6ffd700 1 --2- 172.21.15.146:0/2143438467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99d0003ac0 0x7f99d0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e6ffd700 1 --2- 172.21.15.146:0/2143438467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99d0004870 0x7f99d0004cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e6ffd700 1 --2- 172.21.15.146:0/2143438467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d00051f0 0x7f99d000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e81596b0 con 0x7f99d0003ac0 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d000bf70 con 0x7f99d00051f0 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d000c0b0 con 0x7f99d0004870 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e7fff700 1 --2- 172.21.15.146:0/2143438467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99d0003ac0 0x7f99d0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99ecc75700 1 --2- 172.21.15.146:0/2143438467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d00051f0 0x7f99d000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e77fe700 1 --2- 172.21.15.146:0/2143438467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99d0004870 0x7f99d0004cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e7fff700 1 -- 172.21.15.146:0/2143438467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99d0004870 msgr2=0x7f99d0004cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e7fff700 1 --2- 172.21.15.146:0/2143438467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99d0004870 0x7f99d0004cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e7fff700 1 -- 172.21.15.146:0/2143438467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d00051f0 msgr2=0x7f99d000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e7fff700 1 --2- 172.21.15.146:0/2143438467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d00051f0 0x7f99d000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e7fff700 1 -- 172.21.15.146:0/2143438467 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d00035d0 con 0x7f99d0003ac0 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e7fff700 1 --2- 172.21.15.146:0/2143438467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99d0003ac0 0x7f99d0003ea0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f99d80019a0 tx=0x7f99d00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfefbdb2e962e4ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d8016070 con 0x7f99d0003ac0 2024-09-13T18:28:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d8006880 con 0x7f99d0003ac0 2024-09-13T18:28:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99e6ffd700 1 -- 172.21.15.146:0/2143438467 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d8001440 con 0x7f99d0003ac0 2024-09-13T18:28:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99ed476700 1 -- 172.21.15.146:0/2143438467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99d0003ac0 msgr2=0x7f99d0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99ed476700 1 --2- 172.21.15.146:0/2143438467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99d0003ac0 0x7f99d0003ea0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f99d80019a0 tx=0x7f99d00058a0 comp rx=0 tx=0).stop 2024-09-13T18:28:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99ecc75700 1 -- 172.21.15.146:0/2143438467 reap_dead start 2024-09-13T18:28:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99ed476700 1 -- 172.21.15.146:0/2143438467 shutdown_connections 2024-09-13T18:28:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99ed476700 1 -- 172.21.15.146:0/2143438467 >> 172.21.15.146:0/2143438467 conn(0x7f99e8071ed0 msgr2=0x7f99e80722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99ed476700 1 -- 172.21.15.146:0/2143438467 shutdown_connections 2024-09-13T18:28:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99ed476700 1 -- 172.21.15.146:0/2143438467 wait complete. 2024-09-13T18:28:12.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99ed476700 1 Processor -- start 2024-09-13T18:28:12.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.774+0000 7f99ed476700 1 -- start start 2024-09-13T18:28:12.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ed476700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99e8146f40 0x7f99e8147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:12.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ed476700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99e81573b0 0x7f99e81577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:12.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ed476700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99e8145830 0x7f99e8145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:12.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ed476700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e81eb210 con 0x7f99e8145830 2024-09-13T18:28:12.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ed476700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e81ea2d0 con 0x7f99e81573b0 2024-09-13T18:28:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ed476700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8157d30 con 0x7f99e8146f40 2024-09-13T18:28:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ecc75700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99e8145830 0x7f99e8145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ecc75700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99e8145830 0x7f99e8145cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44176/0 (socket says 172.21.15.146:44176) 2024-09-13T18:28:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ecc75700 1 -- 172.21.15.146:0/2568276910 learned_addr learned my addr 172.21.15.146:0/2568276910 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99e7fff700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99e8146f40 0x7f99e8147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99e77fe700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99e81573b0 0x7f99e81577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ecc75700 1 -- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99e8146f40 msgr2=0x7f99e8147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ecc75700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99e8146f40 0x7f99e8147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ecc75700 1 -- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99e81573b0 msgr2=0x7f99e81577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ecc75700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99e81573b0 0x7f99e81577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ecc75700 1 -- 172.21.15.146:0/2568276910 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e80c7340 con 0x7f99e8145830 2024-09-13T18:28:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ecc75700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99e8145830 0x7f99e8145cb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f99e806ca00 tx=0x7f99e0001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99e4ff9700 1 -- 172.21.15.146:0/2568276910 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e000e950 con 0x7f99e8145830 2024-09-13T18:28:12.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99e4ff9700 1 -- 172.21.15.146:0/2568276910 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e000ead0 con 0x7f99e8145830 2024-09-13T18:28:12.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ed476700 1 -- 172.21.15.146:0/2568276910 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e81ea590 con 0x7f99e8145830 2024-09-13T18:28:12.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99e4ff9700 1 -- 172.21.15.146:0/2568276910 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e000ee80 con 0x7f99e8145830 2024-09-13T18:28:12.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ed476700 1 -- 172.21.15.146:0/2568276910 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e81eab50 con 0x7f99e8145830 2024-09-13T18:28:12.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99ed476700 1 -- 172.21.15.146:0/2568276910 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e81571b0 con 0x7f99e8145830 2024-09-13T18:28:12.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99e4ff9700 1 -- 172.21.15.146:0/2568276910 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f99e001d460 con 0x7f99e8145830 2024-09-13T18:28:12.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99e4ff9700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f99d4066760 0x7f99d4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:12.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.778+0000 7f99e4ff9700 1 -- 172.21.15.146:0/2568276910 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f99e0084600 con 0x7f99e8145830 2024-09-13T18:28:12.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.782+0000 7f99e7fff700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f99d4066760 0x7f99d4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:12.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.782+0000 7f99e7fff700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f99d4066760 0x7f99d4068c20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f99e806c100 tx=0x7f99d8009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:12.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.786+0000 7f99e4ff9700 1 -- 172.21.15.146:0/2568276910 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f99e0051640 con 0x7f99e8145830 2024-09-13T18:28:12.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.982+0000 7f99ed476700 1 -- 172.21.15.146:0/2568276910 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99e8150610 con 0x7f99e8145830 2024-09-13T18:28:12.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.982+0000 7f99e4ff9700 1 -- 172.21.15.146:0/2568276910 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6286 (secure 0 0 0) 0x7f99e0054c60 con 0x7f99e8145830 2024-09-13T18:28:12.992 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:12.992 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:27:09.524480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"e","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:12.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.986+0000 7f99ed476700 1 -- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f99d4066760 msgr2=0x7f99d4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:12.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.986+0000 7f99ed476700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f99d4066760 0x7f99d4068c20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f99e806c100 tx=0x7f99d8009030 comp rx=0 tx=0).stop 2024-09-13T18:28:12.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.986+0000 7f99ed476700 1 -- 172.21.15.146:0/2568276910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99e8145830 msgr2=0x7f99e8145cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:12.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.986+0000 7f99ed476700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99e8145830 0x7f99e8145cb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f99e806ca00 tx=0x7f99e0001a90 comp rx=0 tx=0).stop 2024-09-13T18:28:12.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.986+0000 7f99ed476700 1 -- 172.21.15.146:0/2568276910 shutdown_connections 2024-09-13T18:28:12.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.986+0000 7f99ed476700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f99d4066760 0x7f99d4068c20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:12.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.986+0000 7f99ed476700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99e8145830 0x7f99e8145cb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:12.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.986+0000 7f99ed476700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99e81573b0 0x7f99e81577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:12.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.986+0000 7f99ed476700 1 --2- 172.21.15.146:0/2568276910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99e8146f40 0x7f99e8147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:12.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.986+0000 7f99ed476700 1 -- 172.21.15.146:0/2568276910 >> 172.21.15.146:0/2568276910 conn(0x7f99e8071ed0 msgr2=0x7f99e806d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:12.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.986+0000 7f99ed476700 1 -- 172.21.15.146:0/2568276910 shutdown_connections 2024-09-13T18:28:12.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:12.986+0000 7f99ed476700 1 -- 172.21.15.146:0/2568276910 wait complete. 2024-09-13T18:28:12.995 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 15 2024-09-13T18:28:13.004 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:28:15.006 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:15.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f5945e11700 1 Processor -- start 2024-09-13T18:28:15.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f5945e11700 1 -- start start 2024-09-13T18:28:15.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f5945e11700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f59400725b0 0x7f5940159f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f5945e11700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59400c2100 con 0x7f59400c7630 2024-09-13T18:28:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f5945e11700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5940153bd0 con 0x7f59401537f0 2024-09-13T18:28:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f5945e11700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59400c30c0 con 0x7f59400725b0 2024-09-13T18:28:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f593ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f59400725b0 0x7f5940159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f593ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f59400725b0 0x7f5940159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44192/0 (socket says 172.21.15.146:44192) 2024-09-13T18:28:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f593ffff700 1 -- 172.21.15.146:0/258177592 learned_addr learned my addr 172.21.15.146:0/258177592 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f593ffff700 1 -- 172.21.15.146:0/258177592 >> v1:172.21.15.154:6789/0 conn(0x7f59401537f0 legacy=0x7f59400c19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f593ffff700 1 -- 172.21.15.146:0/258177592 >> v1:172.21.15.146:6789/0 conn(0x7f59400c7630 legacy=0x7f59400c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f593ffff700 1 -- 172.21.15.146:0/258177592 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59400c8240 con 0x7f59400725b0 2024-09-13T18:28:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f593ffff700 1 --2- 172.21.15.146:0/258177592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59400725b0 0x7f5940159f10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f592c009910 tx=0x7f592c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71e1bdaf6f87b0a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f593f7fe700 1 -- 172.21.15.146:0/258177592 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c004230 con 0x7f59400725b0 2024-09-13T18:28:15.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f593f7fe700 1 -- 172.21.15.146:0/258177592 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592c004390 con 0x7f59400725b0 2024-09-13T18:28:15.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f593f7fe700 1 -- 172.21.15.146:0/258177592 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c009e60 con 0x7f59400725b0 2024-09-13T18:28:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f5945e11700 1 -- 172.21.15.146:0/258177592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59400725b0 msgr2=0x7f5940159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f5945e11700 1 --2- 172.21.15.146:0/258177592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59400725b0 0x7f5940159f10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f592c009910 tx=0x7f592c009c20 comp rx=0 tx=0).stop 2024-09-13T18:28:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f5945e11700 1 -- 172.21.15.146:0/258177592 shutdown_connections 2024-09-13T18:28:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f5945e11700 1 --2- 172.21.15.146:0/258177592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59400725b0 0x7f5940159f10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f5945e11700 1 -- 172.21.15.146:0/258177592 >> 172.21.15.146:0/258177592 conn(0x7f5940071a90 msgr2=0x7f5940071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f5945e11700 1 -- 172.21.15.146:0/258177592 shutdown_connections 2024-09-13T18:28:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.107+0000 7f5945e11700 1 -- 172.21.15.146:0/258177592 wait complete. 2024-09-13T18:28:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5945e11700 1 Processor -- start 2024-09-13T18:28:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5945e11700 1 -- start start 2024-09-13T18:28:15.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5945e11700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59400725b0 0x7f5940151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5945e11700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59400c7630 0x7f59401500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5945e11700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59401537f0 0x7f5940150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5945e11700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59401e4050 con 0x7f59400725b0 2024-09-13T18:28:15.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5945e11700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59401f0830 con 0x7f59401537f0 2024-09-13T18:28:15.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5945e11700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59401476a0 con 0x7f59400c7630 2024-09-13T18:28:15.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5944e0f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59400725b0 0x7f5940151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5944e0f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59400725b0 0x7f5940151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44196/0 (socket says 172.21.15.146:44196) 2024-09-13T18:28:15.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5944e0f700 1 -- 172.21.15.146:0/3787793177 learned_addr learned my addr 172.21.15.146:0/3787793177 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:15.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f593ffff700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59400c7630 0x7f59401500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5945610700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59401537f0 0x7f5940150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5944e0f700 1 -- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59400c7630 msgr2=0x7f59401500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5944e0f700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59400c7630 0x7f59401500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5944e0f700 1 -- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59401537f0 msgr2=0x7f5940150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5944e0f700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59401537f0 0x7f5940150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5944e0f700 1 -- 172.21.15.146:0/3787793177 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c009570 con 0x7f59400725b0 2024-09-13T18:28:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5945610700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59401537f0 0x7f5940150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5944e0f700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59400725b0 0x7f5940151530 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f593400b4a0 tx=0x7f593400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f593d7fa700 1 -- 172.21.15.146:0/3787793177 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934011830 con 0x7f59400725b0 2024-09-13T18:28:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f593d7fa700 1 -- 172.21.15.146:0/3787793177 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5934011990 con 0x7f59400725b0 2024-09-13T18:28:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f593d7fa700 1 -- 172.21.15.146:0/3787793177 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934011d40 con 0x7f59400725b0 2024-09-13T18:28:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5945e11700 1 -- 172.21.15.146:0/3787793177 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59401ef9d0 con 0x7f59400725b0 2024-09-13T18:28:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5945e11700 1 -- 172.21.15.146:0/3787793177 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59401eff90 con 0x7f59400725b0 2024-09-13T18:28:15.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.111+0000 7f5945e11700 1 -- 172.21.15.146:0/3787793177 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594006a8e0 con 0x7f59400725b0 2024-09-13T18:28:15.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.115+0000 7f593d7fa700 1 -- 172.21.15.146:0/3787793177 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f593401a400 con 0x7f59400725b0 2024-09-13T18:28:15.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.115+0000 7f593d7fa700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5924066730 0x7f5924068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.115+0000 7f593d7fa700 1 -- 172.21.15.146:0/3787793177 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5934084220 con 0x7f59400725b0 2024-09-13T18:28:15.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.115+0000 7f593ffff700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5924066730 0x7f5924068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.115+0000 7f593ffff700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5924066730 0x7f5924068bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f592c009910 tx=0x7f592c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.123+0000 7f593d7fa700 1 -- 172.21.15.146:0/3787793177 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f59340511d0 con 0x7f59400725b0 2024-09-13T18:28:15.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.319+0000 7f5945e11700 1 -- 172.21.15.146:0/3787793177 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f594014fe40 con 0x7f59400725b0 2024-09-13T18:28:15.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.319+0000 7f593d7fa700 1 -- 172.21.15.146:0/3787793177 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6286 (secure 0 0 0) 0x7f59340547f0 con 0x7f59400725b0 2024-09-13T18:28:15.327 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:15.327 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:27:09.524480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"e","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6833/2835746745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2835746745},{"type":"v1","addr":"172.21.15.146:6833","nonce":2835746745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:15.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.323+0000 7f5945e11700 1 -- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5924066730 msgr2=0x7f5924068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.323+0000 7f5945e11700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5924066730 0x7f5924068bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f592c009910 tx=0x7f592c008030 comp rx=0 tx=0).stop 2024-09-13T18:28:15.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.323+0000 7f5945e11700 1 -- 172.21.15.146:0/3787793177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59400725b0 msgr2=0x7f5940151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.323+0000 7f5945e11700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59400725b0 0x7f5940151530 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f593400b4a0 tx=0x7f593400b860 comp rx=0 tx=0).stop 2024-09-13T18:28:15.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.323+0000 7f5945e11700 1 -- 172.21.15.146:0/3787793177 shutdown_connections 2024-09-13T18:28:15.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.323+0000 7f5945e11700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5924066730 0x7f5924068bf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.323+0000 7f5945e11700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59401537f0 0x7f5940150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.323+0000 7f5945e11700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59400c7630 0x7f59401500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.323+0000 7f5945e11700 1 --2- 172.21.15.146:0/3787793177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59400725b0 0x7f5940151530 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.323+0000 7f5945e11700 1 -- 172.21.15.146:0/3787793177 >> 172.21.15.146:0/3787793177 conn(0x7f5940071a90 msgr2=0x7f594006b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.323+0000 7f5945e11700 1 -- 172.21.15.146:0/3787793177 shutdown_connections 2024-09-13T18:28:15.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:15.323+0000 7f5945e11700 1 -- 172.21.15.146:0/3787793177 wait complete. 2024-09-13T18:28:15.331 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 15 2024-09-13T18:28:15.340 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:28:17.341 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:17.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 Processor -- start 2024-09-13T18:28:17.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 -- start start 2024-09-13T18:28:17.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97b014fb50 0x7f97b00c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f97b00c6770 0x7f97b00c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b0156810 con 0x7f97b00c7b00 2024-09-13T18:28:17.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b01559b0 con 0x7f97b00c6770 2024-09-13T18:28:17.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b0155c90 con 0x7f97b014fb50 2024-09-13T18:28:17.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97affff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f97b00c6770 0x7f97b00c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97affff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f97b00c6770 0x7f97b00c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44212/0 (socket says 172.21.15.146:44212) 2024-09-13T18:28:17.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97affff700 1 -- 172.21.15.146:0/586251461 learned_addr learned my addr 172.21.15.146:0/586251461 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:17.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97aeffd700 1 --1- 172.21.15.146:0/586251461 >> v1:172.21.15.146:6789/0 conn(0x7f97b00c7b00 0x7f97b00c6060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:36442/0 (socket says 172.21.15.146:36442) 2024-09-13T18:28:17.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97af7fe700 1 --2- 172.21.15.146:0/586251461 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97b014fb50 0x7f97b00c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97affff700 1 -- 172.21.15.146:0/586251461 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97b014fb50 msgr2=0x7f97b00c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97affff700 1 --2- 172.21.15.146:0/586251461 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97b014fb50 0x7f97b00c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97affff700 1 -- 172.21.15.146:0/586251461 >> v1:172.21.15.146:6789/0 conn(0x7f97b00c7b00 legacy=0x7f97b00c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97affff700 1 -- 172.21.15.146:0/586251461 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b0155f70 con 0x7f97b00c6770 2024-09-13T18:28:17.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97affff700 1 --2- 172.21.15.146:0/586251461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97b00c6770 0x7f97b00c8510 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009e50 tx=0x7f97a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de17bb25d7a3ff49 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97ae7fc700 1 -- 172.21.15.146:0/586251461 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a800e070 con 0x7f97b00c6770 2024-09-13T18:28:17.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97ae7fc700 1 -- 172.21.15.146:0/586251461 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a800b910 con 0x7f97b00c6770 2024-09-13T18:28:17.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97ae7fc700 1 -- 172.21.15.146:0/586251461 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a800bcc0 con 0x7f97b00c6770 2024-09-13T18:28:17.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 -- 172.21.15.146:0/586251461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97b00c6770 msgr2=0x7f97b00c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 --2- 172.21.15.146:0/586251461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97b00c6770 0x7f97b00c8510 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009e50 tx=0x7f97a800b330 comp rx=0 tx=0).stop 2024-09-13T18:28:17.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 -- 172.21.15.146:0/586251461 shutdown_connections 2024-09-13T18:28:17.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 --2- 172.21.15.146:0/586251461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97b00c6770 0x7f97b00c8510 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 --2- 172.21.15.146:0/586251461 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97b014fb50 0x7f97b00c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 -- 172.21.15.146:0/586251461 >> 172.21.15.146:0/586251461 conn(0x7f97b0066b00 msgr2=0x7f97b006aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 -- 172.21.15.146:0/586251461 shutdown_connections 2024-09-13T18:28:17.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.463+0000 7f97b4e96700 1 -- 172.21.15.146:0/586251461 wait complete. 2024-09-13T18:28:17.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97b4e96700 1 Processor -- start 2024-09-13T18:28:17.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97b4e96700 1 -- start start 2024-09-13T18:28:17.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97b4e96700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97b00c6770 0x7f97b0153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97b4e96700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97b00c7b00 0x7f97b00c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97b4e96700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97b014fb50 0x7f97b00c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97b4e96700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b0142fd0 con 0x7f97b014fb50 2024-09-13T18:28:17.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97b4e96700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b0158bf0 con 0x7f97b00c6770 2024-09-13T18:28:17.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97b4e96700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b01edfd0 con 0x7f97b00c7b00 2024-09-13T18:28:17.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97affff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97b014fb50 0x7f97b00c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97affff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97b014fb50 0x7f97b00c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44222/0 (socket says 172.21.15.146:44222) 2024-09-13T18:28:17.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97affff700 1 -- 172.21.15.146:0/3360819036 learned_addr learned my addr 172.21.15.146:0/3360819036 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:17.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97af7fe700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97b00c6770 0x7f97b0153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97aeffd700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97b00c7b00 0x7f97b00c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97affff700 1 -- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97b00c7b00 msgr2=0x7f97b00c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97affff700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97b00c7b00 0x7f97b00c4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97affff700 1 -- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97b00c6770 msgr2=0x7f97b0153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97affff700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97b00c6770 0x7f97b0153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97affff700 1 -- 172.21.15.146:0/3360819036 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8009b30 con 0x7f97b014fb50 2024-09-13T18:28:17.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97af7fe700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97b00c6770 0x7f97b0153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:17.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97affff700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97b014fb50 0x7f97b00c5260 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f97b01569a0 tx=0x7f97a8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f9797fff700 1 -- 172.21.15.146:0/3360819036 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a80092e0 con 0x7f97b014fb50 2024-09-13T18:28:17.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f9797fff700 1 -- 172.21.15.146:0/3360819036 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a801c440 con 0x7f97b014fb50 2024-09-13T18:28:17.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f9797fff700 1 -- 172.21.15.146:0/3360819036 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a801c810 con 0x7f97b014fb50 2024-09-13T18:28:17.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97b4e96700 1 -- 172.21.15.146:0/3360819036 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b01ee290 con 0x7f97b014fb50 2024-09-13T18:28:17.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.467+0000 7f97b4e96700 1 -- 172.21.15.146:0/3360819036 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b0156ea0 con 0x7f97b014fb50 2024-09-13T18:28:17.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.471+0000 7f9797fff700 1 -- 172.21.15.146:0/3360819036 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f97a801c970 con 0x7f97b014fb50 2024-09-13T18:28:17.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.471+0000 7f9797fff700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f979c066730 0x7f979c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.471+0000 7f9797fff700 1 -- 172.21.15.146:0/3360819036 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f97a8085ef0 con 0x7f97b014fb50 2024-09-13T18:28:17.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.471+0000 7f97af7fe700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f979c066730 0x7f979c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.471+0000 7f97af7fe700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f979c066730 0x7f979c068bf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f97a000abd0 tx=0x7f97a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.475+0000 7f97b4e96700 1 -- 172.21.15.146:0/3360819036 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b006dba0 con 0x7f97b014fb50 2024-09-13T18:28:17.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.483+0000 7f9797fff700 1 -- 172.21.15.146:0/3360819036 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f97a8052ee0 con 0x7f97b014fb50 2024-09-13T18:28:17.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.683+0000 7f97b4e96700 1 -- 172.21.15.146:0/3360819036 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97b00664c0 con 0x7f97b014fb50 2024-09-13T18:28:17.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.683+0000 7f9797fff700 1 -- 172.21.15.146:0/3360819036 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5527 (secure 0 0 0) 0x7f97a8056500 con 0x7f97b014fb50 2024-09-13T18:28:17.690 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:17.690 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:15.565185+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:17.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.687+0000 7f97b4e96700 1 -- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f979c066730 msgr2=0x7f979c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.687+0000 7f97b4e96700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f979c066730 0x7f979c068bf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f97a000abd0 tx=0x7f97a0008040 comp rx=0 tx=0).stop 2024-09-13T18:28:17.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.687+0000 7f97b4e96700 1 -- 172.21.15.146:0/3360819036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97b014fb50 msgr2=0x7f97b00c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.687+0000 7f97b4e96700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97b014fb50 0x7f97b00c5260 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f97b01569a0 tx=0x7f97a8004af0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.687+0000 7f97b4e96700 1 -- 172.21.15.146:0/3360819036 shutdown_connections 2024-09-13T18:28:17.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.687+0000 7f97b4e96700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f979c066730 0x7f979c068bf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.687+0000 7f97b4e96700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97b014fb50 0x7f97b00c5260 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.687+0000 7f97b4e96700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97b00c7b00 0x7f97b00c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.687+0000 7f97b4e96700 1 --2- 172.21.15.146:0/3360819036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97b00c6770 0x7f97b0153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.687+0000 7f97b4e96700 1 -- 172.21.15.146:0/3360819036 >> 172.21.15.146:0/3360819036 conn(0x7f97b0066b00 msgr2=0x7f97b00c7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.687+0000 7f97b4e96700 1 -- 172.21.15.146:0/3360819036 shutdown_connections 2024-09-13T18:28:17.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.687+0000 7f97b4e96700 1 -- 172.21.15.146:0/3360819036 wait complete. 2024-09-13T18:28:17.694 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 16 2024-09-13T18:28:17.703 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2024-09-13T18:28:17.703 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:17.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3dffff700 1 Processor -- start 2024-09-13T18:28:17.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3dffff700 1 -- start start 2024-09-13T18:28:17.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3dffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe3e00cbd50 0x7fe3e00b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3dffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e0071390 0x7fe3e00cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3dffff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e00cc670 con 0x7fe3e00ca210 2024-09-13T18:28:17.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3dffff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e00caa30 con 0x7fe3e00cbd50 2024-09-13T18:28:17.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3dffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00cb810 con 0x7fe3e0071390 2024-09-13T18:28:17.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3de7fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe3e00ca210 0x7fe3e00701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:33024/0 (socket says 172.21.15.146:33024) 2024-09-13T18:28:17.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3deffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe3e00cbd50 0x7fe3e00b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3de7fc700 1 -- 172.21.15.146:0/3469872386 learned_addr learned my addr 172.21.15.146:0/3469872386 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:28:17.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3df7fe700 1 --2- 172.21.15.146:0/3469872386 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e0071390 0x7fe3e00cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3deffd700 1 -- 172.21.15.146:0/3469872386 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e0071390 msgr2=0x7fe3e00cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3deffd700 1 --2- 172.21.15.146:0/3469872386 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e0071390 0x7fe3e00cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3deffd700 1 -- 172.21.15.146:0/3469872386 >> v1:172.21.15.154:6789/0 conn(0x7fe3e00ca210 legacy=0x7fe3e00701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3deffd700 1 -- 172.21.15.146:0/3469872386 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e015da20 con 0x7fe3e00cbd50 2024-09-13T18:28:17.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3ddffb700 1 -- 172.21.15.146:0/3469872386 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886652995 0 0) 0x7fe3d4001110 con 0x7fe3e00ca210 2024-09-13T18:28:17.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3df7fe700 1 --2- 172.21.15.146:0/3469872386 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e0071390 0x7fe3e00cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:28:17.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3deffd700 1 --2- 172.21.15.146:0/3469872386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e00cbd50 0x7fe3e00b1c40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009850 tx=0x7fe3d0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb4285942b996e4b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3ddffb700 1 -- 172.21.15.146:0/3469872386 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0011830 con 0x7fe3e00cbd50 2024-09-13T18:28:17.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3ddffb700 1 -- 172.21.15.146:0/3469872386 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d0011990 con 0x7fe3e00cbd50 2024-09-13T18:28:17.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.791+0000 7fe3ddffb700 1 -- 172.21.15.146:0/3469872386 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0011d40 con 0x7fe3e00cbd50 2024-09-13T18:28:17.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 -- 172.21.15.146:0/3469872386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e00cbd50 msgr2=0x7fe3e00b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 --2- 172.21.15.146:0/3469872386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e00cbd50 0x7fe3e00b1c40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009850 tx=0x7fe3d0009b60 comp rx=0 tx=0).stop 2024-09-13T18:28:17.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 -- 172.21.15.146:0/3469872386 shutdown_connections 2024-09-13T18:28:17.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 --2- 172.21.15.146:0/3469872386 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e0071390 0x7fe3e00cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 --2- 172.21.15.146:0/3469872386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e00cbd50 0x7fe3e00b1c40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 -- 172.21.15.146:0/3469872386 >> 172.21.15.146:0/3469872386 conn(0x7fe3e0066270 msgr2=0x7fe3e0066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 -- 172.21.15.146:0/3469872386 shutdown_connections 2024-09-13T18:28:17.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 -- 172.21.15.146:0/3469872386 wait complete. 2024-09-13T18:28:17.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 Processor -- start 2024-09-13T18:28:17.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 -- start start 2024-09-13T18:28:17.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e0071390 0x7fe3e00c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e00ca210 0x7fe3e00cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e00cbd50 0x7fe3e00ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e01f3350 con 0x7fe3e00ca210 2024-09-13T18:28:17.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00c9050 con 0x7fe3e0071390 2024-09-13T18:28:17.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3dffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e01f3840 con 0x7fe3e00cbd50 2024-09-13T18:28:17.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3de7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e00ca210 0x7fe3e00cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3de7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e00ca210 0x7fe3e00cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44228/0 (socket says 172.21.15.146:44228) 2024-09-13T18:28:17.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3de7fc700 1 -- 172.21.15.146:0/3273698803 learned_addr learned my addr 172.21.15.146:0/3273698803 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:17.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3deffd700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e0071390 0x7fe3e00c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3df7fe700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e00cbd50 0x7fe3e00ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3de7fc700 1 -- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e00cbd50 msgr2=0x7fe3e00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3de7fc700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e00cbd50 0x7fe3e00ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3de7fc700 1 -- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e0071390 msgr2=0x7fe3e00c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3de7fc700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e0071390 0x7fe3e00c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.795+0000 7fe3de7fc700 1 -- 172.21.15.146:0/3273698803 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d8009a70 con 0x7fe3e00ca210 2024-09-13T18:28:17.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.799+0000 7fe3de7fc700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e00ca210 0x7fe3e00cf1e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400c750 tx=0x7fe3d400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.799+0000 7fe3c77fe700 1 -- 172.21.15.146:0/3273698803 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4004ca0 con 0x7fe3e00ca210 2024-09-13T18:28:17.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.799+0000 7fe3c77fe700 1 -- 172.21.15.146:0/3273698803 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d4004e20 con 0x7fe3e00ca210 2024-09-13T18:28:17.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.799+0000 7fe3dffff700 1 -- 172.21.15.146:0/3273698803 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d00094b0 con 0x7fe3e00ca210 2024-09-13T18:28:17.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.799+0000 7fe3c77fe700 1 -- 172.21.15.146:0/3273698803 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4007310 con 0x7fe3e00ca210 2024-09-13T18:28:17.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.799+0000 7fe3dffff700 1 -- 172.21.15.146:0/3273698803 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e00cde20 con 0x7fe3e00ca210 2024-09-13T18:28:17.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.799+0000 7fe3dffff700 1 -- 172.21.15.146:0/3273698803 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c0005310 con 0x7fe3e00ca210 2024-09-13T18:28:17.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.799+0000 7fe3c77fe700 1 -- 172.21.15.146:0/3273698803 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe3d4007470 con 0x7fe3e00ca210 2024-09-13T18:28:17.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.799+0000 7fe3c77fe700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe3cc066760 0x7fe3cc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.799+0000 7fe3c77fe700 1 -- 172.21.15.146:0/3273698803 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fe3d4084070 con 0x7fe3e00ca210 2024-09-13T18:28:17.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.799+0000 7fe3deffd700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe3cc066760 0x7fe3cc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.803+0000 7fe3deffd700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe3cc066760 0x7fe3cc068c20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00090b0 tx=0x7fe3d001c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:17.807+0000 7fe3c77fe700 1 -- 172.21.15.146:0/3273698803 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe3d4050f40 con 0x7fe3e00ca210 2024-09-13T18:28:18.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3dffff700 1 -- 172.21.15.146:0/3273698803 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3c00055c0 con 0x7fe3e00ca210 2024-09-13T18:28:18.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3c77fe700 1 -- 172.21.15.146:0/3273698803 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5527 (secure 0 0 0) 0x7fe3d4007720 con 0x7fe3e00ca210 2024-09-13T18:28:18.033 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:18.034 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:15.565185+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:18.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3dffff700 1 -- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe3cc066760 msgr2=0x7fe3cc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:18.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3dffff700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe3cc066760 0x7fe3cc068c20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00090b0 tx=0x7fe3d001c040 comp rx=0 tx=0).stop 2024-09-13T18:28:18.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3dffff700 1 -- 172.21.15.146:0/3273698803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e00ca210 msgr2=0x7fe3e00cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:18.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3dffff700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e00ca210 0x7fe3e00cf1e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400c750 tx=0x7fe3d400ca60 comp rx=0 tx=0).stop 2024-09-13T18:28:18.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3dffff700 1 -- 172.21.15.146:0/3273698803 shutdown_connections 2024-09-13T18:28:18.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3dffff700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe3cc066760 0x7fe3cc068c20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:18.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3dffff700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e00cbd50 0x7fe3e00ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:18.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3dffff700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e00ca210 0x7fe3e00cf1e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:18.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3dffff700 1 --2- 172.21.15.146:0/3273698803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e0071390 0x7fe3e00c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:18.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3dffff700 1 -- 172.21.15.146:0/3273698803 >> 172.21.15.146:0/3273698803 conn(0x7fe3e0066270 msgr2=0x7fe3e00b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:18.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3dffff700 1 -- 172.21.15.146:0/3273698803 shutdown_connections 2024-09-13T18:28:18.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:18.027+0000 7fe3dffff700 1 -- 172.21.15.146:0/3273698803 wait complete. 2024-09-13T18:28:18.037 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 16 2024-09-13T18:28:18.046 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 16, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 16, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4247": { "addr": "172.21.15.154:6839/3978828499", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 3978828499, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 3978828499, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4247, "incarnation": 16, "join_fscid": -1, "name": "f", "rank": 2, "state": "up:replay", "state_seq": 1 }, "gid_4281": { "addr": "172.21.15.146:6835/1642500214", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 1642500214, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 1642500214, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4281, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 11 }, "gid_4290": { "addr": "172.21.15.146:6837/31041557", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 31041557, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 31041557, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4290, "incarnation": 4, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 28, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:28:15.565185+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4290, "mds_1": 4281, "mds_2": 4247 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:28:20.047 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:20.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a2fe1700 1 Processor -- start 2024-09-13T18:28:20.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a2fe1700 1 -- start start 2024-09-13T18:28:20.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a2fe1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa094046df0 0x7fa0940e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:20.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a2fe1700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0940dd3a0 con 0x7fa09404d0f0 2024-09-13T18:28:20.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a2fe1700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0940dcb80 con 0x7fa0940d6720 2024-09-13T18:28:20.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a2fe1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa094047200 con 0x7fa094046df0 2024-09-13T18:28:20.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a17de700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa09404d0f0 0x7fa09404de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:36446/0 (socket says 172.21.15.146:36446) 2024-09-13T18:28:20.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a17de700 1 -- 172.21.15.146:0/1832437169 learned_addr learned my addr 172.21.15.146:0/1832437169 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:28:20.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a27e0700 1 --2- 172.21.15.146:0/1832437169 >> v2:172.21.15.154:3300/0 conn(0x7fa094046df0 0x7fa0940e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:20.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148347675 0 0) 0x7fa0940dd3a0 con 0x7fa09404d0f0 2024-09-13T18:28:20.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0880035d0 con 0x7fa09404d0f0 2024-09-13T18:28:20.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4118587566 0 0) 0x7fa0880035d0 con 0x7fa09404d0f0 2024-09-13T18:28:20.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 >> v2:172.21.15.154:3300/0 conn(0x7fa094046df0 msgr2=0x7fa0940e0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:20.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 --2- 172.21.15.146:0/1832437169 >> v2:172.21.15.154:3300/0 conn(0x7fa094046df0 0x7fa0940e0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:20.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 >> v1:172.21.15.154:6789/0 conn(0x7fa0940d6720 legacy=0x7fa0940dc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:20.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0940dcde0 con 0x7fa09404d0f0 2024-09-13T18:28:20.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7fa08c001670 con 0x7fa09404d0f0 2024-09-13T18:28:20.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 >> v1:172.21.15.146:6789/0 conn(0x7fa09404d0f0 legacy=0x7fa09404de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:20.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 --2- 172.21.15.146:0/1832437169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa088003a20 0x7fa088003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:20.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 --2- 172.21.15.146:0/1832437169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa088004870 0x7fa088004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:20.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 --2- 172.21.15.146:0/1832437169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa088005230 0x7fa08800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:20.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa094047200 con 0x7fa088003a20 2024-09-13T18:28:20.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a1fdf700 1 --2- 172.21.15.146:0/1832437169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa088003a20 0x7fa088003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:20.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08800c130 con 0x7fa088004870 2024-09-13T18:28:20.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08800c2f0 con 0x7fa088005230 2024-09-13T18:28:20.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a17de700 1 --2- 172.21.15.146:0/1832437169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa088004870 0x7fa088004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:20.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a1fdf700 1 -- 172.21.15.146:0/1832437169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa088005230 msgr2=0x7fa08800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:20.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a1fdf700 1 --2- 172.21.15.146:0/1832437169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa088005230 0x7fa08800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:20.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a1fdf700 1 -- 172.21.15.146:0/1832437169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa088004870 msgr2=0x7fa088004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:20.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a1fdf700 1 --2- 172.21.15.146:0/1832437169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa088004870 0x7fa088004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:20.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a1fdf700 1 -- 172.21.15.146:0/1832437169 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0880035d0 con 0x7fa088003a20 2024-09-13T18:28:20.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a27e0700 1 -- 172.21.15.146:0/1832437169 reap_dead start 2024-09-13T18:28:20.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a1fdf700 1 --2- 172.21.15.146:0/1832437169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa088003a20 0x7fa088003e00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa0980017e0 tx=0x7fa0880058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e67231ab0f0f32 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:20.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0980052d0 con 0x7fa088003a20 2024-09-13T18:28:20.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa098005430 con 0x7fa088003a20 2024-09-13T18:28:20.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.171+0000 7fa0a0fdd700 1 -- 172.21.15.146:0/1832437169 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0980055a0 con 0x7fa088003a20 2024-09-13T18:28:20.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1832437169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa088003a20 msgr2=0x7fa088003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:20.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 --2- 172.21.15.146:0/1832437169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa088003a20 0x7fa088003e00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa0980017e0 tx=0x7fa0880058a0 comp rx=0 tx=0).stop 2024-09-13T18:28:20.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1832437169 shutdown_connections 2024-09-13T18:28:20.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 --2- 172.21.15.146:0/1832437169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa088003a20 0x7fa088003e00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:20.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1832437169 >> 172.21.15.146:0/1832437169 conn(0x7fa09400dc40 msgr2=0x7fa09400e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:20.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1832437169 shutdown_connections 2024-09-13T18:28:20.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1832437169 wait complete. 2024-09-13T18:28:20.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 Processor -- start 2024-09-13T18:28:20.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 -- start start 2024-09-13T18:28:20.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa088003a20 0x7fa094037380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:20.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa094049210 0x7fa0940cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:20.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0940cfd30 0x7fa094172200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:20.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940d01e0 con 0x7fa088003a20 2024-09-13T18:28:20.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09404a9b0 con 0x7fa094049210 2024-09-13T18:28:20.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09404a280 con 0x7fa0940cfd30 2024-09-13T18:28:20.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a1fdf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa088003a20 0x7fa094037380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:20.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a1fdf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa088003a20 0x7fa094037380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44246/0 (socket says 172.21.15.146:44246) 2024-09-13T18:28:20.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a1fdf700 1 -- 172.21.15.146:0/1138457514 learned_addr learned my addr 172.21.15.146:0/1138457514 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:20.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a27e0700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0940cfd30 0x7fa094172200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:20.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a17de700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa094049210 0x7fa0940cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:20.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a1fdf700 1 -- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0940cfd30 msgr2=0x7fa094172200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:20.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a1fdf700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0940cfd30 0x7fa094172200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:20.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a1fdf700 1 -- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa094049210 msgr2=0x7fa0940cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:20.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a1fdf700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa094049210 0x7fa0940cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:20.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a1fdf700 1 -- 172.21.15.146:0/1138457514 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09404ba60 con 0x7fa088003a20 2024-09-13T18:28:20.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a1fdf700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa088003a20 0x7fa094037380 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa098005040 tx=0x7fa098001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:20.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0927fc700 1 -- 172.21.15.146:0/1138457514 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0980055a0 con 0x7fa088003a20 2024-09-13T18:28:20.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0927fc700 1 -- 172.21.15.146:0/1138457514 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa098001290 con 0x7fa088003a20 2024-09-13T18:28:20.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1138457514 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09404bd40 con 0x7fa088003a20 2024-09-13T18:28:20.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0927fc700 1 -- 172.21.15.146:0/1138457514 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098007260 con 0x7fa088003a20 2024-09-13T18:28:20.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.175+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1138457514 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09404af00 con 0x7fa088003a20 2024-09-13T18:28:20.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.179+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1138457514 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa084005310 con 0x7fa088003a20 2024-09-13T18:28:20.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.179+0000 7fa0927fc700 1 -- 172.21.15.146:0/1138457514 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa0980073c0 con 0x7fa088003a20 2024-09-13T18:28:20.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.179+0000 7fa0927fc700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa088068330 0x7fa08806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:20.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.179+0000 7fa0927fc700 1 -- 172.21.15.146:0/1138457514 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fa098084e00 con 0x7fa088003a20 2024-09-13T18:28:20.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.179+0000 7fa0a17de700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa088068330 0x7fa08806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:20.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.179+0000 7fa0a17de700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa088068330 0x7fa08806a7f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa09404f640 tx=0x7fa08c00a040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:20.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.187+0000 7fa0927fc700 1 -- 172.21.15.146:0/1138457514 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa098051df0 con 0x7fa088003a20 2024-09-13T18:28:20.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1138457514 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0840055c0 con 0x7fa088003a20 2024-09-13T18:28:20.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0927fc700 1 -- 172.21.15.146:0/1138457514 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5527 (secure 0 0 0) 0x7fa098055410 con 0x7fa088003a20 2024-09-13T18:28:20.405 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:20.405 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:15.565185+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:20.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa088068330 msgr2=0x7fa08806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:20.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0a2fe1700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa088068330 0x7fa08806a7f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa09404f640 tx=0x7fa08c00a040 comp rx=0 tx=0).stop 2024-09-13T18:28:20.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1138457514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa088003a20 msgr2=0x7fa094037380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:20.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0a2fe1700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa088003a20 0x7fa094037380 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa098005040 tx=0x7fa098001120 comp rx=0 tx=0).stop 2024-09-13T18:28:20.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1138457514 shutdown_connections 2024-09-13T18:28:20.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0a2fe1700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa088068330 0x7fa08806a7f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:20.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0a2fe1700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0940cfd30 0x7fa094172200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:20.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0a2fe1700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa094049210 0x7fa0940cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:20.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0a2fe1700 1 --2- 172.21.15.146:0/1138457514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa088003a20 0x7fa094037380 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:20.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1138457514 >> 172.21.15.146:0/1138457514 conn(0x7fa09400dc40 msgr2=0x7fa0940d4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:20.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1138457514 shutdown_connections 2024-09-13T18:28:20.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:20.399+0000 7fa0a2fe1700 1 -- 172.21.15.146:0/1138457514 wait complete. 2024-09-13T18:28:20.411 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 16 2024-09-13T18:28:20.421 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:28:22.422 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:22.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.515+0000 7f8b0dc7e700 1 Processor -- start 2024-09-13T18:28:22.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.515+0000 7f8b0dc7e700 1 -- start start 2024-09-13T18:28:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b08070940 0x7f8b08071210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8b08070d20 0x7f8b08150200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8b080b1be0 0x7f8b08150740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b08071830 con 0x7f8b08070d20 2024-09-13T18:28:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b080c24b0 con 0x7f8b080b1be0 2024-09-13T18:28:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b080c2790 con 0x7f8b08070940 2024-09-13T18:28:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b07fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8b08070d20 0x7f8b08150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b07fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8b08070d20 0x7f8b08150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60970/0 (socket says 172.21.15.146:60970) 2024-09-13T18:28:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b07fff700 1 -- 172.21.15.146:0/2171339298 learned_addr learned my addr 172.21.15.146:0/2171339298 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0d47d700 1 --2- 172.21.15.146:0/2171339298 >> v2:172.21.15.154:3300/0 conn(0x7f8b080b1be0 0x7f8b08150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0cc7c700 1 --2- 172.21.15.146:0/2171339298 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b08070940 0x7f8b08071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b07fff700 1 -- 172.21.15.146:0/2171339298 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b08070940 msgr2=0x7f8b08071210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b07fff700 1 --2- 172.21.15.146:0/2171339298 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b08070940 0x7f8b08071210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b07fff700 1 -- 172.21.15.146:0/2171339298 >> v2:172.21.15.154:3300/0 conn(0x7f8b080b1be0 msgr2=0x7f8b08150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b07fff700 1 --2- 172.21.15.146:0/2171339298 >> v2:172.21.15.154:3300/0 conn(0x7f8b080b1be0 0x7f8b08150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b07fff700 1 -- 172.21.15.146:0/2171339298 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b080c2a70 con 0x7f8b08070d20 2024-09-13T18:28:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0cc7c700 1 --2- 172.21.15.146:0/2171339298 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b08070940 0x7f8b08071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b07fff700 1 --2- 172.21.15.146:0/2171339298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08070d20 0x7f8b08150200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8b080c6a10 tx=0x7f8af400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5284e74edcc9a514 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b077fe700 1 -- 172.21.15.146:0/2171339298 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af400dd90 con 0x7f8b08070d20 2024-09-13T18:28:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b077fe700 1 -- 172.21.15.146:0/2171339298 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af4010470 con 0x7f8b08070d20 2024-09-13T18:28:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b077fe700 1 -- 172.21.15.146:0/2171339298 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4010800 con 0x7f8b08070d20 2024-09-13T18:28:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2171339298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08070d20 msgr2=0x7f8b08150200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 --2- 172.21.15.146:0/2171339298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08070d20 0x7f8b08150200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8b080c6a10 tx=0x7f8af400be90 comp rx=0 tx=0).stop 2024-09-13T18:28:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2171339298 shutdown_connections 2024-09-13T18:28:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 --2- 172.21.15.146:0/2171339298 >> v2:172.21.15.154:3300/0 conn(0x7f8b080b1be0 0x7f8b08150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 --2- 172.21.15.146:0/2171339298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08070d20 0x7f8b08150200 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 --2- 172.21.15.146:0/2171339298 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b08070940 0x7f8b08071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2171339298 >> 172.21.15.146:0/2171339298 conn(0x7f8b08071a90 msgr2=0x7f8b08071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2171339298 shutdown_connections 2024-09-13T18:28:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2171339298 wait complete. 2024-09-13T18:28:22.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 Processor -- start 2024-09-13T18:28:22.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.519+0000 7f8b0dc7e700 1 -- start start 2024-09-13T18:28:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0dc7e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b08070940 0x7f8b08152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0dc7e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b08070d20 0x7f8b0814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0dc7e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b080b1be0 0x7f8b08154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0dc7e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b081eddf0 con 0x7f8b080b1be0 2024-09-13T18:28:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0dc7e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b081ea990 con 0x7f8b08070d20 2024-09-13T18:28:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0dc7e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b081eac70 con 0x7f8b08070940 2024-09-13T18:28:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0d47d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b080b1be0 0x7f8b08154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0d47d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b080b1be0 0x7f8b08154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60974/0 (socket says 172.21.15.146:60974) 2024-09-13T18:28:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0d47d700 1 -- 172.21.15.146:0/2896561654 learned_addr learned my addr 172.21.15.146:0/2896561654 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b07fff700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b08070d20 0x7f8b0814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0cc7c700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b08070940 0x7f8b08152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0d47d700 1 -- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b08070940 msgr2=0x7f8b08152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0d47d700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b08070940 0x7f8b08152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0d47d700 1 -- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b08070d20 msgr2=0x7f8b0814f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0d47d700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b08070d20 0x7f8b0814f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0d47d700 1 -- 172.21.15.146:0/2896561654 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af400b810 con 0x7f8b080b1be0 2024-09-13T18:28:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0d47d700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b080b1be0 0x7f8b08154530 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009fd0 tx=0x7f8b0000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b057fa700 1 -- 172.21.15.146:0/2896561654 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0000ede0 con 0x7f8b080b1be0 2024-09-13T18:28:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b057fa700 1 -- 172.21.15.146:0/2896561654 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b00014420 con 0x7f8b080b1be0 2024-09-13T18:28:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b057fa700 1 -- 172.21.15.146:0/2896561654 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b000147d0 con 0x7f8b080b1be0 2024-09-13T18:28:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2896561654 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b081eaf50 con 0x7f8b080b1be0 2024-09-13T18:28:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2896561654 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b081eb750 con 0x7f8b080b1be0 2024-09-13T18:28:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2896561654 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b081eb470 con 0x7f8b080b1be0 2024-09-13T18:28:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b057fa700 1 -- 172.21.15.146:0/2896561654 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b00014930 con 0x7f8b080b1be0 2024-09-13T18:28:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b057fa700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8aec066730 0x7f8aec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.523+0000 7f8b057fa700 1 -- 172.21.15.146:0/2896561654 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f8b00084c90 con 0x7f8b080b1be0 2024-09-13T18:28:22.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.527+0000 7f8b0cc7c700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8aec066730 0x7f8aec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:22.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.527+0000 7f8b0cc7c700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8aec066730 0x7f8aec068bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8afc006fd0 tx=0x7f8afc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:22.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.531+0000 7f8b057fa700 1 -- 172.21.15.146:0/2896561654 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8b081eb470 con 0x7f8b080b1be0 2024-09-13T18:28:22.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2896561654 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b081eb470 con 0x7f8b080b1be0 2024-09-13T18:28:22.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b057fa700 1 -- 172.21.15.146:0/2896561654 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5529 (secure 0 0 0) 0x7f8b081eb470 con 0x7f8b080b1be0 2024-09-13T18:28:22.744 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:22.744 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:20.978460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:resolve","state_seq":30,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:22.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8aec066730 msgr2=0x7f8aec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:22.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b0dc7e700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8aec066730 0x7f8aec068bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8afc006fd0 tx=0x7f8afc008040 comp rx=0 tx=0).stop 2024-09-13T18:28:22.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2896561654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b080b1be0 msgr2=0x7f8b08154530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:22.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b0dc7e700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b080b1be0 0x7f8b08154530 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009fd0 tx=0x7f8b0000e3c0 comp rx=0 tx=0).stop 2024-09-13T18:28:22.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2896561654 shutdown_connections 2024-09-13T18:28:22.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b0dc7e700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8aec066730 0x7f8aec068bf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:22.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b0dc7e700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b080b1be0 0x7f8b08154530 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:22.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b0dc7e700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b08070d20 0x7f8b0814f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:22.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b0dc7e700 1 --2- 172.21.15.146:0/2896561654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b08070940 0x7f8b08152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:22.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2896561654 >> 172.21.15.146:0/2896561654 conn(0x7f8b08071a90 msgr2=0x7f8b0806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:22.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2896561654 shutdown_connections 2024-09-13T18:28:22.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:22.739+0000 7f8b0dc7e700 1 -- 172.21.15.146:0/2896561654 wait complete. 2024-09-13T18:28:22.748 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 17 2024-09-13T18:28:22.757 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:28:24.758 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:24.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.859+0000 7f6a0f59e700 1 Processor -- start 2024-09-13T18:28:24.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.859+0000 7f6a0f59e700 1 -- start start 2024-09-13T18:28:24.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0f59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6a10154280 0x7f6a1015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0f59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6a100c6960 0x7f6a100c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0f59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a100c3060 con 0x7f6a100cc1a0 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0f59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a100c1e90 con 0x7f6a100c6960 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0f59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1006e9c0 con 0x7f6a10154280 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a07fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6a100c6960 0x7f6a100c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a07fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6a100c6960 0x7f6a100c12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60990/0 (socket says 172.21.15.146:60990) 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a07fff700 1 -- 172.21.15.146:0/1871204997 learned_addr learned my addr 172.21.15.146:0/1871204997 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0e59c700 1 --2- 172.21.15.146:0/1871204997 >> v2:172.21.15.154:3300/0 conn(0x7f6a10154280 0x7f6a1015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a07fff700 1 -- 172.21.15.146:0/1871204997 >> v2:172.21.15.154:3300/0 conn(0x7f6a10154280 msgr2=0x7f6a1015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a07fff700 1 --2- 172.21.15.146:0/1871204997 >> v2:172.21.15.154:3300/0 conn(0x7f6a10154280 0x7f6a1015ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a07fff700 1 -- 172.21.15.146:0/1871204997 >> v1:172.21.15.154:6789/0 conn(0x7f6a100cc1a0 legacy=0x7f6a10160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a07fff700 1 -- 172.21.15.146:0/1871204997 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1006e3b0 con 0x7f6a100c6960 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a07fff700 1 --2- 172.21.15.146:0/1871204997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a100c6960 0x7f6a100c12d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6a100c9ad0 tx=0x7f69f8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1184f6121fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0dd9b700 1 -- 172.21.15.146:0/1871204997 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f8004230 con 0x7f6a100c6960 2024-09-13T18:28:24.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0dd9b700 1 -- 172.21.15.146:0/1871204997 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f8004390 con 0x7f6a100c6960 2024-09-13T18:28:24.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0dd9b700 1 -- 172.21.15.146:0/1871204997 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f8009e60 con 0x7f6a100c6960 2024-09-13T18:28:24.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0f59e700 1 -- 172.21.15.146:0/1871204997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a100c6960 msgr2=0x7f6a100c12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:24.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0f59e700 1 --2- 172.21.15.146:0/1871204997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a100c6960 0x7f6a100c12d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6a100c9ad0 tx=0x7f69f8009c20 comp rx=0 tx=0).stop 2024-09-13T18:28:24.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0f59e700 1 -- 172.21.15.146:0/1871204997 shutdown_connections 2024-09-13T18:28:24.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0f59e700 1 --2- 172.21.15.146:0/1871204997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a100c6960 0x7f6a100c12d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0f59e700 1 --2- 172.21.15.146:0/1871204997 >> v2:172.21.15.154:3300/0 conn(0x7f6a10154280 0x7f6a1015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0f59e700 1 -- 172.21.15.146:0/1871204997 >> 172.21.15.146:0/1871204997 conn(0x7f6a10071ed0 msgr2=0x7f6a100722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:24.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0f59e700 1 -- 172.21.15.146:0/1871204997 shutdown_connections 2024-09-13T18:28:24.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.863+0000 7f6a0f59e700 1 -- 172.21.15.146:0/1871204997 wait complete. 2024-09-13T18:28:24.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0f59e700 1 Processor -- start 2024-09-13T18:28:24.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0f59e700 1 -- start start 2024-09-13T18:28:24.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a100c6960 0x7f6a101f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:24.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a100cc1a0 0x7f6a101f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:24.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0f59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a10154280 0x7f6a100c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0f59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a101e74d0 con 0x7f6a100c6960 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0f59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1015c5f0 con 0x7f6a10154280 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a101f72b0 con 0x7f6a100cc1a0 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0e59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a100c6960 0x7f6a101f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0e59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a100c6960 0x7f6a101f37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60992/0 (socket says 172.21.15.146:60992) 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0e59c700 1 -- 172.21.15.146:0/777182781 learned_addr learned my addr 172.21.15.146:0/777182781 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a07fff700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a100cc1a0 0x7f6a101f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0ed9d700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a10154280 0x7f6a100c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0e59c700 1 -- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a100cc1a0 msgr2=0x7f6a101f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0e59c700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a100cc1a0 0x7f6a101f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0e59c700 1 -- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a10154280 msgr2=0x7f6a100c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0e59c700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a10154280 0x7f6a100c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0e59c700 1 -- 172.21.15.146:0/777182781 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f8009570 con 0x7f6a100c6960 2024-09-13T18:28:24.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0e59c700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a100c6960 0x7f6a101f37f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000d830 tx=0x7f6a0000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:24.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a06ffd700 1 -- 172.21.15.146:0/777182781 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a000125f0 con 0x7f6a100c6960 2024-09-13T18:28:24.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a06ffd700 1 -- 172.21.15.146:0/777182781 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a00012750 con 0x7f6a100c6960 2024-09-13T18:28:24.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0f59e700 1 -- 172.21.15.146:0/777182781 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a101f3d30 con 0x7f6a100c6960 2024-09-13T18:28:24.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a06ffd700 1 -- 172.21.15.146:0/777182781 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00012b00 con 0x7f6a100c6960 2024-09-13T18:28:24.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0f59e700 1 -- 172.21.15.146:0/777182781 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a101f3fa0 con 0x7f6a100c6960 2024-09-13T18:28:24.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.867+0000 7f6a0f59e700 1 -- 172.21.15.146:0/777182781 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a100c2200 con 0x7f6a100c6960 2024-09-13T18:28:24.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.871+0000 7f6a06ffd700 1 -- 172.21.15.146:0/777182781 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6a00012c60 con 0x7f6a100c6960 2024-09-13T18:28:24.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.871+0000 7f6a06ffd700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69f0066760 0x7f69f0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:24.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.871+0000 7f6a06ffd700 1 -- 172.21.15.146:0/777182781 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f6a000846d0 con 0x7f6a100c6960 2024-09-13T18:28:24.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.871+0000 7f6a07fff700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69f0066760 0x7f69f0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:24.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.871+0000 7f6a07fff700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69f0066760 0x7f69f0068c20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6a100c6da0 tx=0x7f69f8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:24.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:24.879+0000 7f6a06ffd700 1 -- 172.21.15.146:0/777182781 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6a000515d0 con 0x7f6a100c6960 2024-09-13T18:28:25.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.071+0000 7f6a0f59e700 1 -- 172.21.15.146:0/777182781 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a100c2d00 con 0x7f6a100c6960 2024-09-13T18:28:25.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.075+0000 7f6a06ffd700 1 -- 172.21.15.146:0/777182781 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5529 (secure 0 0 0) 0x7f6a00054bf0 con 0x7f6a100c6960 2024-09-13T18:28:25.080 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:25.080 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:20.978460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:resolve","state_seq":30,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:25.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.079+0000 7f6a0f59e700 1 -- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69f0066760 msgr2=0x7f69f0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:25.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.079+0000 7f6a0f59e700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69f0066760 0x7f69f0068c20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6a100c6da0 tx=0x7f69f8008030 comp rx=0 tx=0).stop 2024-09-13T18:28:25.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.079+0000 7f6a0f59e700 1 -- 172.21.15.146:0/777182781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a100c6960 msgr2=0x7f6a101f37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:25.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.079+0000 7f6a0f59e700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a100c6960 0x7f6a101f37f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000d830 tx=0x7f6a0000db40 comp rx=0 tx=0).stop 2024-09-13T18:28:25.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.079+0000 7f6a0f59e700 1 -- 172.21.15.146:0/777182781 shutdown_connections 2024-09-13T18:28:25.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.079+0000 7f6a0f59e700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69f0066760 0x7f69f0068c20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:25.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.079+0000 7f6a0f59e700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a10154280 0x7f6a100c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:25.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.079+0000 7f6a0f59e700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a100cc1a0 0x7f6a101f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:25.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.079+0000 7f6a0f59e700 1 --2- 172.21.15.146:0/777182781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a100c6960 0x7f6a101f37f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:25.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.079+0000 7f6a0f59e700 1 -- 172.21.15.146:0/777182781 >> 172.21.15.146:0/777182781 conn(0x7f6a10071ed0 msgr2=0x7f6a100722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:25.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.079+0000 7f6a0f59e700 1 -- 172.21.15.146:0/777182781 shutdown_connections 2024-09-13T18:28:25.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:25.079+0000 7f6a0f59e700 1 -- 172.21.15.146:0/777182781 wait complete. 2024-09-13T18:28:25.086 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 17 2024-09-13T18:28:25.100 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:28:27.100 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:27.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb5af9a700 1 Processor -- start 2024-09-13T18:28:27.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb5af9a700 1 -- start start 2024-09-13T18:28:27.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb5af9a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7feb54066300 0x7feb54156da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:27.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb5af9a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb5406d220 0x7feb540c4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:27.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb5af9a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb540c4690 con 0x7feb541504f0 2024-09-13T18:28:27.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb5af9a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb540c27e0 con 0x7feb54066300 2024-09-13T18:28:27.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb5af9a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5406d6b0 con 0x7feb5406d220 2024-09-13T18:28:27.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb59797700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7feb54066300 0x7feb54156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:27.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb59797700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7feb54066300 0x7feb54156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32772/0 (socket says 172.21.15.146:32772) 2024-09-13T18:28:27.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb59797700 1 -- 172.21.15.146:0/3228915067 learned_addr learned my addr 172.21.15.146:0/3228915067 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:27.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb5a799700 1 --2- 172.21.15.146:0/3228915067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb5406d220 0x7feb540c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:27.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb59797700 1 -- 172.21.15.146:0/3228915067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb5406d220 msgr2=0x7feb540c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:27.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb59797700 1 --2- 172.21.15.146:0/3228915067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb5406d220 0x7feb540c4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:27.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb59797700 1 -- 172.21.15.146:0/3228915067 >> v1:172.21.15.146:6789/0 conn(0x7feb541504f0 legacy=0x7feb540c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:27.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb59797700 1 -- 172.21.15.146:0/3228915067 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb54156330 con 0x7feb54066300 2024-09-13T18:28:27.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb59797700 1 --2- 172.21.15.146:0/3228915067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb54066300 0x7feb54156da0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feb44009910 tx=0x7feb44009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b050f3e9e43a01b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:27.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.207+0000 7feb58f96700 1 -- 172.21.15.146:0/3228915067 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4400ba60 con 0x7feb54066300 2024-09-13T18:28:27.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb58f96700 1 -- 172.21.15.146:0/3228915067 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4400bbc0 con 0x7feb54066300 2024-09-13T18:28:27.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb58f96700 1 -- 172.21.15.146:0/3228915067 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44009e60 con 0x7feb54066300 2024-09-13T18:28:27.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 -- 172.21.15.146:0/3228915067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb54066300 msgr2=0x7feb54156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:27.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 --2- 172.21.15.146:0/3228915067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb54066300 0x7feb54156da0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feb44009910 tx=0x7feb44009c20 comp rx=0 tx=0).stop 2024-09-13T18:28:27.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 -- 172.21.15.146:0/3228915067 shutdown_connections 2024-09-13T18:28:27.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 --2- 172.21.15.146:0/3228915067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb5406d220 0x7feb540c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:27.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 --2- 172.21.15.146:0/3228915067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb54066300 0x7feb54156da0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:27.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 -- 172.21.15.146:0/3228915067 >> 172.21.15.146:0/3228915067 conn(0x7feb54066b00 msgr2=0x7feb5406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:27.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 -- 172.21.15.146:0/3228915067 shutdown_connections 2024-09-13T18:28:27.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 -- 172.21.15.146:0/3228915067 wait complete. 2024-09-13T18:28:27.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 Processor -- start 2024-09-13T18:28:27.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 -- start start 2024-09-13T18:28:27.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb54066300 0x7feb5414f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:27.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb5406d220 0x7feb541f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:27.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb541504f0 0x7feb54154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:27.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb54146030 con 0x7feb5406d220 2024-09-13T18:28:27.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb54146310 con 0x7feb541504f0 2024-09-13T18:28:27.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb5af9a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb541465f0 con 0x7feb54066300 2024-09-13T18:28:27.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.211+0000 7feb59797700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb5406d220 0x7feb541f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:27.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb59797700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb5406d220 0x7feb541f1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32776/0 (socket says 172.21.15.146:32776) 2024-09-13T18:28:27.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb59797700 1 -- 172.21.15.146:0/444212296 learned_addr learned my addr 172.21.15.146:0/444212296 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:27.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb59f98700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb54066300 0x7feb5414f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:27.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb5a799700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb541504f0 0x7feb54154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:27.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb59797700 1 -- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb54066300 msgr2=0x7feb5414f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:27.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb59797700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb54066300 0x7feb5414f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:27.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb59797700 1 -- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb541504f0 msgr2=0x7feb54154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:27.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb59797700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb541504f0 0x7feb54154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:27.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb59797700 1 -- 172.21.15.146:0/444212296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb44009570 con 0x7feb5406d220 2024-09-13T18:28:27.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb5a799700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb541504f0 0x7feb54154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:27.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb59797700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb5406d220 0x7feb541f1ee0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feb54070790 tx=0x7feb44004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:27.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb4a7fc700 1 -- 172.21.15.146:0/444212296 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44004510 con 0x7feb5406d220 2024-09-13T18:28:27.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb4a7fc700 1 -- 172.21.15.146:0/444212296 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4401f070 con 0x7feb5406d220 2024-09-13T18:28:27.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb5af9a700 1 -- 172.21.15.146:0/444212296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5414bc20 con 0x7feb5406d220 2024-09-13T18:28:27.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb4a7fc700 1 -- 172.21.15.146:0/444212296 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44003900 con 0x7feb5406d220 2024-09-13T18:28:27.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb5af9a700 1 -- 172.21.15.146:0/444212296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5414c1e0 con 0x7feb5406d220 2024-09-13T18:28:27.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb5af9a700 1 -- 172.21.15.146:0/444212296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5406b3f0 con 0x7feb5406d220 2024-09-13T18:28:27.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb4a7fc700 1 -- 172.21.15.146:0/444212296 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feb44003a60 con 0x7feb5406d220 2024-09-13T18:28:27.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb4a7fc700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7feb38066730 0x7feb38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:27.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.215+0000 7feb4a7fc700 1 -- 172.21.15.146:0/444212296 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7feb44021070 con 0x7feb5406d220 2024-09-13T18:28:27.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.219+0000 7feb59f98700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7feb38066730 0x7feb38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:27.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.219+0000 7feb59f98700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7feb38066730 0x7feb38068bf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7feb50000f60 tx=0x7feb50009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:27.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.223+0000 7feb4a7fc700 1 -- 172.21.15.146:0/444212296 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feb4408b020 con 0x7feb5406d220 2024-09-13T18:28:27.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.427+0000 7feb5af9a700 1 -- 172.21.15.146:0/444212296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb540c4e90 con 0x7feb5406d220 2024-09-13T18:28:27.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.431+0000 7feb4a7fc700 1 -- 172.21.15.146:0/444212296 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5531 (secure 0 0 0) 0x7feb44052d70 con 0x7feb5406d220 2024-09-13T18:28:27.436 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:27.436 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:25.999263+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:reconnect","state_seq":32,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:27.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.431+0000 7feb5af9a700 1 -- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7feb38066730 msgr2=0x7feb38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:27.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.431+0000 7feb5af9a700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7feb38066730 0x7feb38068bf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7feb50000f60 tx=0x7feb50009040 comp rx=0 tx=0).stop 2024-09-13T18:28:27.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.431+0000 7feb5af9a700 1 -- 172.21.15.146:0/444212296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb5406d220 msgr2=0x7feb541f1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:27.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.431+0000 7feb5af9a700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb5406d220 0x7feb541f1ee0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feb54070790 tx=0x7feb44004770 comp rx=0 tx=0).stop 2024-09-13T18:28:27.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.431+0000 7feb5af9a700 1 -- 172.21.15.146:0/444212296 shutdown_connections 2024-09-13T18:28:27.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.431+0000 7feb5af9a700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7feb38066730 0x7feb38068bf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:27.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.431+0000 7feb5af9a700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb541504f0 0x7feb54154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:27.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.431+0000 7feb5af9a700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb5406d220 0x7feb541f1ee0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:27.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.431+0000 7feb5af9a700 1 --2- 172.21.15.146:0/444212296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb54066300 0x7feb5414f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:27.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.431+0000 7feb5af9a700 1 -- 172.21.15.146:0/444212296 >> 172.21.15.146:0/444212296 conn(0x7feb54066b00 msgr2=0x7feb54146e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:27.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.431+0000 7feb5af9a700 1 -- 172.21.15.146:0/444212296 shutdown_connections 2024-09-13T18:28:27.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:27.431+0000 7feb5af9a700 1 -- 172.21.15.146:0/444212296 wait complete. 2024-09-13T18:28:27.440 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 18 2024-09-13T18:28:27.452 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:28:27.613 INFO:tasks.workunit.client.0.smithi146.stderr:Note: switching to '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'. 2024-09-13T18:28:27.613 INFO:tasks.workunit.client.0.smithi146.stderr: 2024-09-13T18:28:27.614 INFO:tasks.workunit.client.0.smithi146.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-13T18:28:27.614 INFO:tasks.workunit.client.0.smithi146.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-13T18:28:27.614 INFO:tasks.workunit.client.0.smithi146.stderr:state without impacting any branches by switching back to a branch. 2024-09-13T18:28:27.614 INFO:tasks.workunit.client.0.smithi146.stderr: 2024-09-13T18:28:27.614 INFO:tasks.workunit.client.0.smithi146.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-13T18:28:27.614 INFO:tasks.workunit.client.0.smithi146.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-13T18:28:27.614 INFO:tasks.workunit.client.0.smithi146.stderr: 2024-09-13T18:28:27.614 INFO:tasks.workunit.client.0.smithi146.stderr: git switch -c 2024-09-13T18:28:27.614 INFO:tasks.workunit.client.0.smithi146.stderr: 2024-09-13T18:28:27.614 INFO:tasks.workunit.client.0.smithi146.stderr:Or undo this operation with: 2024-09-13T18:28:27.614 INFO:tasks.workunit.client.0.smithi146.stderr: 2024-09-13T18:28:27.614 INFO:tasks.workunit.client.0.smithi146.stderr: git switch - 2024-09-13T18:28:27.615 INFO:tasks.workunit.client.0.smithi146.stderr: 2024-09-13T18:28:27.615 INFO:tasks.workunit.client.0.smithi146.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-13T18:28:27.615 INFO:tasks.workunit.client.0.smithi146.stderr: 2024-09-13T18:28:27.615 INFO:tasks.workunit.client.0.smithi146.stderr:HEAD is now at 6c6b083f148 Merge PR #56690 into wip-rishabh-testing-20240910.100730-reef 2024-09-13T18:28:27.620 DEBUG:teuthology.orchestra.run.smithi146:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-13T18:28:27.650 INFO:tasks.workunit.client.0.smithi146.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-13T18:28:27.653 INFO:tasks.workunit.client.0.smithi146.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T18:28:27.653 INFO:tasks.workunit.client.0.smithi146.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-13T18:28:28.489 INFO:tasks.workunit.client.0.smithi146.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-13T18:28:28.570 INFO:tasks.workunit.client.0.smithi146.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-13T18:28:28.632 INFO:tasks.workunit.client.0.smithi146.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T18:28:28.634 INFO:tasks.workunit.client.0.smithi146.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T18:28:28.634 INFO:tasks.workunit.client.0.smithi146.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-13T18:28:28.697 INFO:tasks.workunit.client.0.smithi146.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T18:28:28.703 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:28:28.703 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-13T18:28:28.708 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-09-13T18:28:28.710 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-09-13T18:28:28.710 DEBUG:teuthology.orchestra.run.smithi146:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-13T18:28:29.452 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:29.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483f59e700 1 Processor -- start 2024-09-13T18:28:29.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483f59e700 1 -- start start 2024-09-13T18:28:29.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483f59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4840154a40 0x7f48400721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:29.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483f59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48400c8c70 con 0x7f484006f7d0 2024-09-13T18:28:29.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483f59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48400c8f50 con 0x7f48400cac60 2024-09-13T18:28:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483f59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4840072810 con 0x7f4840154a40 2024-09-13T18:28:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483dd9b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f484006f7d0 0x7f4840157d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:38974/0 (socket says 172.21.15.146:38974) 2024-09-13T18:28:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483dd9b700 1 -- 172.21.15.146:0/484558261 learned_addr learned my addr 172.21.15.146:0/484558261 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:28:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483e59c700 1 --2- 172.21.15.146:0/484558261 >> v2:172.21.15.154:3300/0 conn(0x7f4840154a40 0x7f48400721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483d59a700 1 -- 172.21.15.146:0/484558261 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118101470 0 0) 0x7f48400c8c70 con 0x7f484006f7d0 2024-09-13T18:28:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483d59a700 1 -- 172.21.15.146:0/484558261 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48280035d0 con 0x7f484006f7d0 2024-09-13T18:28:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483d59a700 1 -- 172.21.15.146:0/484558261 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308226590 0 0) 0x7f48400c8f50 con 0x7f48400cac60 2024-09-13T18:28:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483d59a700 1 -- 172.21.15.146:0/484558261 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48400c8c70 con 0x7f48400cac60 2024-09-13T18:28:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483e59c700 1 -- 172.21.15.146:0/484558261 >> v1:172.21.15.154:6789/0 conn(0x7f48400cac60 legacy=0x7f4840161c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483d59a700 1 -- 172.21.15.146:0/484558261 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1079467873 0 0) 0x7f48280035d0 con 0x7f484006f7d0 2024-09-13T18:28:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483e59c700 1 -- 172.21.15.146:0/484558261 >> v1:172.21.15.146:6789/0 conn(0x7f484006f7d0 legacy=0x7f4840157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.567+0000 7f483e59c700 1 -- 172.21.15.146:0/484558261 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4840157530 con 0x7f4840154a40 2024-09-13T18:28:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.571+0000 7f483e59c700 1 --2- 172.21.15.146:0/484558261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4840154a40 0x7f48400721f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4834009850 tx=0x7f4834009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f447db2be1f0ab6d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:29.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.571+0000 7f483d59a700 1 -- 172.21.15.146:0/484558261 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830001670 con 0x7f4840154a40 2024-09-13T18:28:29.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.571+0000 7f483d59a700 1 -- 172.21.15.146:0/484558261 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4834004030 con 0x7f4840154a40 2024-09-13T18:28:29.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.571+0000 7f483d59a700 1 -- 172.21.15.146:0/484558261 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48340043c0 con 0x7f4840154a40 2024-09-13T18:28:29.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.571+0000 7f483f59e700 1 -- 172.21.15.146:0/484558261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4840154a40 msgr2=0x7f48400721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:29.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.571+0000 7f483f59e700 1 --2- 172.21.15.146:0/484558261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4840154a40 0x7f48400721f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4834009850 tx=0x7f4834009b60 comp rx=0 tx=0).stop 2024-09-13T18:28:29.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.571+0000 7f483f59e700 1 -- 172.21.15.146:0/484558261 shutdown_connections 2024-09-13T18:28:29.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.571+0000 7f483f59e700 1 --2- 172.21.15.146:0/484558261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4840154a40 0x7f48400721f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:29.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.571+0000 7f483f59e700 1 -- 172.21.15.146:0/484558261 >> 172.21.15.146:0/484558261 conn(0x7f4840066090 msgr2=0x7f4840066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:29.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.571+0000 7f483f59e700 1 -- 172.21.15.146:0/484558261 shutdown_connections 2024-09-13T18:28:29.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.571+0000 7f483f59e700 1 -- 172.21.15.146:0/484558261 wait complete. 2024-09-13T18:28:29.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.571+0000 7f483f59e700 1 Processor -- start 2024-09-13T18:28:29.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.571+0000 7f483f59e700 1 -- start start 2024-09-13T18:28:29.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f484006f7d0 0x7f48401529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:29.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48400cac60 0x7f48400c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:29.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483f59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4840154a40 0x7f48401f0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:29.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483f59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4840157910 con 0x7f48400cac60 2024-09-13T18:28:29.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483f59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484014e640 con 0x7f4840154a40 2024-09-13T18:28:29.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48400c45a0 con 0x7f484006f7d0 2024-09-13T18:28:29.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483dd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48400cac60 0x7f48400c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:29.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483dd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48400cac60 0x7f48400c2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32780/0 (socket says 172.21.15.146:32780) 2024-09-13T18:28:29.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483dd9b700 1 -- 172.21.15.146:0/888970959 learned_addr learned my addr 172.21.15.146:0/888970959 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:29.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483e59c700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f484006f7d0 0x7f48401529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:29.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483ed9d700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4840154a40 0x7f48401f0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:29.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483dd9b700 1 -- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f484006f7d0 msgr2=0x7f48401529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:29.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483dd9b700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f484006f7d0 0x7f48401529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:29.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483dd9b700 1 -- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4840154a40 msgr2=0x7f48401f0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:29.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483dd9b700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4840154a40 0x7f48401f0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:29.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483dd9b700 1 -- 172.21.15.146:0/888970959 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48340094b0 con 0x7f48400cac60 2024-09-13T18:28:29.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483dd9b700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48400cac60 0x7f48400c2c70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f483000c6b0 tx=0x7f483000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:29.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f482effd700 1 -- 172.21.15.146:0/888970959 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830004ca0 con 0x7f48400cac60 2024-09-13T18:28:29.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f482effd700 1 -- 172.21.15.146:0/888970959 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4830004e00 con 0x7f48400cac60 2024-09-13T18:28:29.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483f59e700 1 -- 172.21.15.146:0/888970959 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484014da00 con 0x7f48400cac60 2024-09-13T18:28:29.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f482effd700 1 -- 172.21.15.146:0/888970959 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300072f0 con 0x7f48400cac60 2024-09-13T18:28:29.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.575+0000 7f483f59e700 1 -- 172.21.15.146:0/888970959 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484014e2d0 con 0x7f48400cac60 2024-09-13T18:28:29.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.579+0000 7f483f59e700 1 -- 172.21.15.146:0/888970959 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48400c8f50 con 0x7f48400cac60 2024-09-13T18:28:29.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.579+0000 7f482effd700 1 -- 172.21.15.146:0/888970959 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4830007450 con 0x7f48400cac60 2024-09-13T18:28:29.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.579+0000 7f482effd700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4820066730 0x7f4820068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:29.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.579+0000 7f482effd700 1 -- 172.21.15.146:0/888970959 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f48300840c0 con 0x7f48400cac60 2024-09-13T18:28:29.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.579+0000 7f483e59c700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4820066730 0x7f4820068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:29.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.579+0000 7f483e59c700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4820066730 0x7f4820068bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f48340090b0 tx=0x7f4834008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:29.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.587+0000 7f482effd700 1 -- 172.21.15.146:0/888970959 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f48300510b0 con 0x7f48400cac60 2024-09-13T18:28:29.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.795+0000 7f483f59e700 1 -- 172.21.15.146:0/888970959 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48400b00d0 con 0x7f48400cac60 2024-09-13T18:28:29.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.799+0000 7f482effd700 1 -- 172.21.15.146:0/888970959 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5531 (secure 0 0 0) 0x7f48300546d0 con 0x7f48400cac60 2024-09-13T18:28:29.804 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:29.804 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:25.999263+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:reconnect","state_seq":32,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:29.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.799+0000 7f483f59e700 1 -- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4820066730 msgr2=0x7f4820068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:29.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.799+0000 7f483f59e700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4820066730 0x7f4820068bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f48340090b0 tx=0x7f4834008030 comp rx=0 tx=0).stop 2024-09-13T18:28:29.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.799+0000 7f483f59e700 1 -- 172.21.15.146:0/888970959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48400cac60 msgr2=0x7f48400c2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:29.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.799+0000 7f483f59e700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48400cac60 0x7f48400c2c70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f483000c6b0 tx=0x7f483000c9c0 comp rx=0 tx=0).stop 2024-09-13T18:28:29.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.799+0000 7f483f59e700 1 -- 172.21.15.146:0/888970959 shutdown_connections 2024-09-13T18:28:29.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.799+0000 7f483f59e700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4820066730 0x7f4820068bf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:29.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.799+0000 7f483f59e700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4840154a40 0x7f48401f0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:29.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.799+0000 7f483f59e700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48400cac60 0x7f48400c2c70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:29.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.799+0000 7f483f59e700 1 --2- 172.21.15.146:0/888970959 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f484006f7d0 0x7f48401529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:29.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.799+0000 7f483f59e700 1 -- 172.21.15.146:0/888970959 >> 172.21.15.146:0/888970959 conn(0x7f4840066090 msgr2=0x7f48401565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:29.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.799+0000 7f483f59e700 1 -- 172.21.15.146:0/888970959 shutdown_connections 2024-09-13T18:28:29.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:29.799+0000 7f483f59e700 1 -- 172.21.15.146:0/888970959 wait complete. 2024-09-13T18:28:29.808 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 18 2024-09-13T18:28:29.820 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 18, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 18, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4247": { "addr": "172.21.15.154:6839/3978828499", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 3978828499, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 3978828499, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4247, "incarnation": 16, "join_fscid": -1, "name": "f", "rank": 2, "state": "up:reconnect", "state_seq": 32 }, "gid_4281": { "addr": "172.21.15.146:6835/1642500214", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 1642500214, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 1642500214, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4281, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 11 }, "gid_4290": { "addr": "172.21.15.146:6837/31041557", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 31041557, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 31041557, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4290, "incarnation": 4, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 28, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:28:25.999263+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4290, "mds_1": 4281, "mds_2": 4247 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:28:31.821 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:31.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.939+0000 7fe59b59e700 1 Processor -- start 2024-09-13T18:28:31.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.939+0000 7fe59b59e700 1 -- start start 2024-09-13T18:28:31.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.939+0000 7fe59b59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe59c151580 0x7fe59c0c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:31.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59b59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe59c0afbc0 0x7fe59c0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:31.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59b59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59c0c43c0 con 0x7fe59c15a970 2024-09-13T18:28:31.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59b59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59c0c22a0 con 0x7fe59c151580 2024-09-13T18:28:31.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59b59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59c157730 con 0x7fe59c0afbc0 2024-09-13T18:28:31.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59a59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe59c151580 0x7fe59c0c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:31.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59a59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe59c151580 0x7fe59c0c7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36540/0 (socket says 172.21.15.146:36540) 2024-09-13T18:28:31.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59a59c700 1 -- 172.21.15.146:0/1813976123 learned_addr learned my addr 172.21.15.146:0/1813976123 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:31.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe599d9b700 1 --2- 172.21.15.146:0/1813976123 >> v2:172.21.15.154:3300/0 conn(0x7fe59c0afbc0 0x7fe59c0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:31.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59a59c700 1 -- 172.21.15.146:0/1813976123 >> v2:172.21.15.154:3300/0 conn(0x7fe59c0afbc0 msgr2=0x7fe59c0c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:31.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59a59c700 1 --2- 172.21.15.146:0/1813976123 >> v2:172.21.15.154:3300/0 conn(0x7fe59c0afbc0 0x7fe59c0c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:31.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59a59c700 1 -- 172.21.15.146:0/1813976123 >> v1:172.21.15.154:6789/0 conn(0x7fe59c15a970 legacy=0x7fe59c0c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:31.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59a59c700 1 -- 172.21.15.146:0/1813976123 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c157a10 con 0x7fe59c151580 2024-09-13T18:28:31.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59a59c700 1 --2- 172.21.15.146:0/1813976123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59c151580 0x7fe59c0c7c70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0b3440 tx=0x7fe590009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1207a3a71ef4592 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:31.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59959a700 1 -- 172.21.15.146:0/1813976123 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590011830 con 0x7fe59c151580 2024-09-13T18:28:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59959a700 1 -- 172.21.15.146:0/1813976123 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe590011990 con 0x7fe59c151580 2024-09-13T18:28:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59959a700 1 -- 172.21.15.146:0/1813976123 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590011d20 con 0x7fe59c151580 2024-09-13T18:28:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59b59e700 1 -- 172.21.15.146:0/1813976123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59c151580 msgr2=0x7fe59c0c7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59b59e700 1 --2- 172.21.15.146:0/1813976123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59c151580 0x7fe59c0c7c70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0b3440 tx=0x7fe590009bf0 comp rx=0 tx=0).stop 2024-09-13T18:28:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59b59e700 1 -- 172.21.15.146:0/1813976123 shutdown_connections 2024-09-13T18:28:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59b59e700 1 --2- 172.21.15.146:0/1813976123 >> v2:172.21.15.154:3300/0 conn(0x7fe59c0afbc0 0x7fe59c0c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59b59e700 1 --2- 172.21.15.146:0/1813976123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59c151580 0x7fe59c0c7c70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59b59e700 1 -- 172.21.15.146:0/1813976123 >> 172.21.15.146:0/1813976123 conn(0x7fe59c06f5c0 msgr2=0x7fe59c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59b59e700 1 -- 172.21.15.146:0/1813976123 shutdown_connections 2024-09-13T18:28:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.943+0000 7fe59b59e700 1 -- 172.21.15.146:0/1813976123 wait complete. 2024-09-13T18:28:31.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59b59e700 1 Processor -- start 2024-09-13T18:28:31.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59b59e700 1 -- start start 2024-09-13T18:28:31.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe59c0afbc0 0x7fe59c1f0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:31.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59b59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe59c151580 0x7fe59c0c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:31.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59c15a970 0x7fe59c154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:31.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59b59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c0c3a40 con 0x7fe59c15a970 2024-09-13T18:28:31.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59b59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c153da0 con 0x7fe59c151580 2024-09-13T18:28:31.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c0cb420 con 0x7fe59c0afbc0 2024-09-13T18:28:31.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59c15a970 0x7fe59c154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:31.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59c15a970 0x7fe59c154230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36544/0 (socket says 172.21.15.146:36544) 2024-09-13T18:28:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59ad9d700 1 -- 172.21.15.146:0/3402204601 learned_addr learned my addr 172.21.15.146:0/3402204601 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe599d9b700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe59c151580 0x7fe59c0c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59a59c700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe59c0afbc0 0x7fe59c1f0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59ad9d700 1 -- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe59c0afbc0 msgr2=0x7fe59c1f0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59ad9d700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe59c0afbc0 0x7fe59c1f0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59ad9d700 1 -- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe59c151580 msgr2=0x7fe59c0c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59ad9d700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe59c151580 0x7fe59c0c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59ad9d700 1 -- 172.21.15.146:0/3402204601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe590009570 con 0x7fe59c15a970 2024-09-13T18:28:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59ad9d700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59c15a970 0x7fe59c154230 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe59400caa0 tx=0x7fe59400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe58affd700 1 -- 172.21.15.146:0/3402204601 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe594003c20 con 0x7fe59c15a970 2024-09-13T18:28:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe58affd700 1 -- 172.21.15.146:0/3402204601 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe594003d80 con 0x7fe59c15a970 2024-09-13T18:28:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59b59e700 1 -- 172.21.15.146:0/3402204601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59c0cb160 con 0x7fe59c15a970 2024-09-13T18:28:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe58affd700 1 -- 172.21.15.146:0/3402204601 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59400aa20 con 0x7fe59c15a970 2024-09-13T18:28:31.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.947+0000 7fe59b59e700 1 -- 172.21.15.146:0/3402204601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c1f13b0 con 0x7fe59c15a970 2024-09-13T18:28:31.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.951+0000 7fe59b59e700 1 -- 172.21.15.146:0/3402204601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59c153f10 con 0x7fe59c15a970 2024-09-13T18:28:31.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.951+0000 7fe58affd700 1 -- 172.21.15.146:0/3402204601 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe59400ab80 con 0x7fe59c15a970 2024-09-13T18:28:31.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.951+0000 7fe58affd700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe57c066730 0x7fe57c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:31.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.951+0000 7fe58affd700 1 -- 172.21.15.146:0/3402204601 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fe594084310 con 0x7fe59c15a970 2024-09-13T18:28:31.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.951+0000 7fe59a59c700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe57c066730 0x7fe57c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:31.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.951+0000 7fe59a59c700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe57c066730 0x7fe57c068bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe590000c00 tx=0x7fe590008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:31.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:31.959+0000 7fe58affd700 1 -- 172.21.15.146:0/3402204601 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe5940513c0 con 0x7fe59c15a970 2024-09-13T18:28:32.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.167+0000 7fe59b59e700 1 -- 172.21.15.146:0/3402204601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe59c151960 con 0x7fe59c15a970 2024-09-13T18:28:32.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.167+0000 7fe58affd700 1 -- 172.21.15.146:0/3402204601 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5528 (secure 0 0 0) 0x7fe5940549e0 con 0x7fe59c15a970 2024-09-13T18:28:32.176 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:32.176 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:31.025614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:rejoin","state_seq":34,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:32.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.171+0000 7fe59b59e700 1 -- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe57c066730 msgr2=0x7fe57c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:32.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.171+0000 7fe59b59e700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe57c066730 0x7fe57c068bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe590000c00 tx=0x7fe590008030 comp rx=0 tx=0).stop 2024-09-13T18:28:32.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.171+0000 7fe59b59e700 1 -- 172.21.15.146:0/3402204601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59c15a970 msgr2=0x7fe59c154230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:32.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.171+0000 7fe59b59e700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59c15a970 0x7fe59c154230 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe59400caa0 tx=0x7fe59400cdb0 comp rx=0 tx=0).stop 2024-09-13T18:28:32.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.171+0000 7fe59b59e700 1 -- 172.21.15.146:0/3402204601 shutdown_connections 2024-09-13T18:28:32.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.171+0000 7fe59b59e700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe57c066730 0x7fe57c068bf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:32.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.171+0000 7fe59b59e700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59c15a970 0x7fe59c154230 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:32.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.171+0000 7fe59b59e700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe59c151580 0x7fe59c0c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:32.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.171+0000 7fe59b59e700 1 --2- 172.21.15.146:0/3402204601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe59c0afbc0 0x7fe59c1f0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:32.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.171+0000 7fe59b59e700 1 -- 172.21.15.146:0/3402204601 >> 172.21.15.146:0/3402204601 conn(0x7fe59c06f5c0 msgr2=0x7fe59c0c2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:32.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.171+0000 7fe59b59e700 1 -- 172.21.15.146:0/3402204601 shutdown_connections 2024-09-13T18:28:32.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:32.171+0000 7fe59b59e700 1 -- 172.21.15.146:0/3402204601 wait complete. 2024-09-13T18:28:32.179 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 19 2024-09-13T18:28:32.190 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:28:34.191 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:34.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.307+0000 7fd55ffff700 1 Processor -- start 2024-09-13T18:28:34.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.307+0000 7fd55ffff700 1 -- start start 2024-09-13T18:28:34.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.307+0000 7fd55ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd5600ca210 0x7fd5600b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:34.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.307+0000 7fd55ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd560071390 0x7fd5600cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:34.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.307+0000 7fd55ffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5600cc670 con 0x7fd5600cbd50 2024-09-13T18:28:34.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.307+0000 7fd55ffff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5600caa30 con 0x7fd560071390 2024-09-13T18:28:34.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.307+0000 7fd55ffff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5600cb810 con 0x7fd5600ca210 2024-09-13T18:28:34.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.307+0000 7fd55effd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd5600cbd50 0x7fd5600701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41170/0 (socket says 172.21.15.146:41170) 2024-09-13T18:28:34.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.307+0000 7fd55effd700 1 -- 172.21.15.146:0/213220107 learned_addr learned my addr 172.21.15.146:0/213220107 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:28:34.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.307+0000 7fd55f7fe700 1 --2- 172.21.15.146:0/213220107 >> v2:172.21.15.146:3300/0 conn(0x7fd560071390 0x7fd5600cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:34.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.307+0000 7fd55e7fc700 1 --2- 172.21.15.146:0/213220107 >> v2:172.21.15.154:3300/0 conn(0x7fd5600ca210 0x7fd5600b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:34.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55f7fe700 1 -- 172.21.15.146:0/213220107 >> v2:172.21.15.154:3300/0 conn(0x7fd5600ca210 msgr2=0x7fd5600b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:34.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55f7fe700 1 --2- 172.21.15.146:0/213220107 >> v2:172.21.15.154:3300/0 conn(0x7fd5600ca210 0x7fd5600b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:34.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55f7fe700 1 -- 172.21.15.146:0/213220107 >> v1:172.21.15.146:6789/0 conn(0x7fd5600cbd50 legacy=0x7fd5600701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:34.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55f7fe700 1 -- 172.21.15.146:0/213220107 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56015da20 con 0x7fd560071390 2024-09-13T18:28:34.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55f7fe700 1 --2- 172.21.15.146:0/213220107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd560071390 0x7fd5600cc130 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd5600b18a0 tx=0x7fd55800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef68178378660f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:34.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55dffb700 1 -- 172.21.15.146:0/213220107 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55800e070 con 0x7fd560071390 2024-09-13T18:28:34.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55dffb700 1 -- 172.21.15.146:0/213220107 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55800bb20 con 0x7fd560071390 2024-09-13T18:28:34.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55dffb700 1 -- 172.21.15.146:0/213220107 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558012430 con 0x7fd560071390 2024-09-13T18:28:34.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55ffff700 1 -- 172.21.15.146:0/213220107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd560071390 msgr2=0x7fd5600cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:34.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55ffff700 1 --2- 172.21.15.146:0/213220107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd560071390 0x7fd5600cc130 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd5600b18a0 tx=0x7fd55800b330 comp rx=0 tx=0).stop 2024-09-13T18:28:34.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55ffff700 1 -- 172.21.15.146:0/213220107 shutdown_connections 2024-09-13T18:28:34.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55ffff700 1 --2- 172.21.15.146:0/213220107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd560071390 0x7fd5600cc130 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:34.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55ffff700 1 --2- 172.21.15.146:0/213220107 >> v2:172.21.15.154:3300/0 conn(0x7fd5600ca210 0x7fd5600b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:34.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55ffff700 1 -- 172.21.15.146:0/213220107 >> 172.21.15.146:0/213220107 conn(0x7fd560066270 msgr2=0x7fd560066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:34.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55ffff700 1 -- 172.21.15.146:0/213220107 shutdown_connections 2024-09-13T18:28:34.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55ffff700 1 -- 172.21.15.146:0/213220107 wait complete. 2024-09-13T18:28:34.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.311+0000 7fd55ffff700 1 Processor -- start 2024-09-13T18:28:34.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55ffff700 1 -- start start 2024-09-13T18:28:34.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560071390 0x7fd5601e2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:34.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5600ca210 0x7fd5601efe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:34.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5600cbd50 0x7fd5600c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:34.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600c6b20 con 0x7fd5600cbd50 2024-09-13T18:28:34.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55ffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56015bd10 con 0x7fd560071390 2024-09-13T18:28:34.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56015bff0 con 0x7fd5600ca210 2024-09-13T18:28:34.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5600cbd50 0x7fd5600c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:34.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5600cbd50 0x7fd5600c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36562/0 (socket says 172.21.15.146:36562) 2024-09-13T18:28:34.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55f7fe700 1 -- 172.21.15.146:0/2795127465 learned_addr learned my addr 172.21.15.146:0/2795127465 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:34.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55effd700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560071390 0x7fd5601e2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:34.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55e7fc700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5600ca210 0x7fd5601efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:34.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55f7fe700 1 -- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5600ca210 msgr2=0x7fd5601efe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:34.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55f7fe700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5600ca210 0x7fd5601efe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:34.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55f7fe700 1 -- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560071390 msgr2=0x7fd5601e2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:34.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55f7fe700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560071390 0x7fd5601e2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:34.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55f7fe700 1 -- 172.21.15.146:0/2795127465 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd558009a70 con 0x7fd5600cbd50 2024-09-13T18:28:34.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55e7fc700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5600ca210 0x7fd5601efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:34.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55effd700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560071390 0x7fd5601e2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:34.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55f7fe700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5600cbd50 0x7fd5600c65e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd558014040 tx=0x7fd558004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:34.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd5477fe700 1 -- 172.21.15.146:0/2795127465 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558012430 con 0x7fd5600cbd50 2024-09-13T18:28:34.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd5477fe700 1 -- 172.21.15.146:0/2795127465 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd558020070 con 0x7fd5600cbd50 2024-09-13T18:28:34.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55ffff700 1 -- 172.21.15.146:0/2795127465 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56015a830 con 0x7fd5600cbd50 2024-09-13T18:28:34.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd5477fe700 1 -- 172.21.15.146:0/2795127465 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558004280 con 0x7fd5600cbd50 2024-09-13T18:28:34.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.315+0000 7fd55ffff700 1 -- 172.21.15.146:0/2795127465 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56015adf0 con 0x7fd5600cbd50 2024-09-13T18:28:34.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.319+0000 7fd5477fe700 1 -- 172.21.15.146:0/2795127465 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd5580043e0 con 0x7fd5600cbd50 2024-09-13T18:28:34.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.319+0000 7fd55ffff700 1 -- 172.21.15.146:0/2795127465 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56015a690 con 0x7fd5600cbd50 2024-09-13T18:28:34.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.319+0000 7fd5477fe700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd54c066730 0x7fd54c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:34.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.319+0000 7fd5477fe700 1 -- 172.21.15.146:0/2795127465 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fd558022070 con 0x7fd5600cbd50 2024-09-13T18:28:34.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.319+0000 7fd55effd700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd54c066730 0x7fd54c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:34.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.319+0000 7fd55effd700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd54c066730 0x7fd54c068bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd554000f60 tx=0x7fd554009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:34.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.327+0000 7fd5477fe700 1 -- 172.21.15.146:0/2795127465 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd56015a690 con 0x7fd5600cbd50 2024-09-13T18:28:34.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.535+0000 7fd55ffff700 1 -- 172.21.15.146:0/2795127465 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd56015a690 con 0x7fd5600cbd50 2024-09-13T18:28:34.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.535+0000 7fd5477fe700 1 -- 172.21.15.146:0/2795127465 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5528 (secure 0 0 0) 0x7fd56015a690 con 0x7fd5600cbd50 2024-09-13T18:28:34.542 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:34.542 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:31.025614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:rejoin","state_seq":34,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:34.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.539+0000 7fd55ffff700 1 -- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd54c066730 msgr2=0x7fd54c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:34.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.539+0000 7fd55ffff700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd54c066730 0x7fd54c068bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd554000f60 tx=0x7fd554009040 comp rx=0 tx=0).stop 2024-09-13T18:28:34.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.539+0000 7fd55ffff700 1 -- 172.21.15.146:0/2795127465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5600cbd50 msgr2=0x7fd5600c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:34.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.539+0000 7fd55ffff700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5600cbd50 0x7fd5600c65e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd558014040 tx=0x7fd558004af0 comp rx=0 tx=0).stop 2024-09-13T18:28:34.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.539+0000 7fd55ffff700 1 -- 172.21.15.146:0/2795127465 shutdown_connections 2024-09-13T18:28:34.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.539+0000 7fd55ffff700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd54c066730 0x7fd54c068bf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:34.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.539+0000 7fd55ffff700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5600cbd50 0x7fd5600c65e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:34.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.539+0000 7fd55ffff700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5600ca210 0x7fd5601efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:34.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.539+0000 7fd55ffff700 1 --2- 172.21.15.146:0/2795127465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560071390 0x7fd5601e2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:34.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.539+0000 7fd55ffff700 1 -- 172.21.15.146:0/2795127465 >> 172.21.15.146:0/2795127465 conn(0x7fd560066270 msgr2=0x7fd560149d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:34.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.539+0000 7fd55ffff700 1 -- 172.21.15.146:0/2795127465 shutdown_connections 2024-09-13T18:28:34.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:34.539+0000 7fd55ffff700 1 -- 172.21.15.146:0/2795127465 wait complete. 2024-09-13T18:28:34.546 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 19 2024-09-13T18:28:34.557 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:28:36.066 INFO:tasks.workunit.client.0.smithi146.stderr:Cloning into 'ffsb'... 2024-09-13T18:28:36.558 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:36.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a351e700 1 Processor -- start 2024-09-13T18:28:36.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a351e700 1 -- start start 2024-09-13T18:28:36.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a351e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe49c06eba0 0x7fe49c0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:36.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a351e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe49c15a000 0x7fe49c0c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:36.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a351e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49c0c1020 con 0x7fe49c06e7c0 2024-09-13T18:28:36.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a351e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c1593f0 con 0x7fe49c06eba0 2024-09-13T18:28:36.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a351e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0c6a70 con 0x7fe49c15a000 2024-09-13T18:28:36.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a251c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fe49c06e7c0 0x7fe49c1598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41176/0 (socket says 172.21.15.146:41176) 2024-09-13T18:28:36.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a251c700 1 -- 172.21.15.146:0/742239075 learned_addr learned my addr 172.21.15.146:0/742239075 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:28:36.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a2d1d700 1 --2- 172.21.15.146:0/742239075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe49c15a000 0x7fe49c0c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:36.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a1d1b700 1 --2- 172.21.15.146:0/742239075 >> v2:172.21.15.154:3300/0 conn(0x7fe49c06eba0 0x7fe49c0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:36.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a151a700 1 -- 172.21.15.146:0/742239075 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050052292 0 0) 0x7fe49c0c1020 con 0x7fe49c06e7c0 2024-09-13T18:28:36.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a151a700 1 -- 172.21.15.146:0/742239075 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4880035d0 con 0x7fe49c06e7c0 2024-09-13T18:28:36.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a2d1d700 1 -- 172.21.15.146:0/742239075 >> v2:172.21.15.154:3300/0 conn(0x7fe49c06eba0 msgr2=0x7fe49c0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:36.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a2d1d700 1 --2- 172.21.15.146:0/742239075 >> v2:172.21.15.154:3300/0 conn(0x7fe49c06eba0 0x7fe49c0b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a2d1d700 1 -- 172.21.15.146:0/742239075 >> v1:172.21.15.146:6789/0 conn(0x7fe49c06e7c0 legacy=0x7fe49c1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a151a700 1 -- 172.21.15.146:0/742239075 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1462194709 0 0) 0x7fe4880035d0 con 0x7fe49c06e7c0 2024-09-13T18:28:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a2d1d700 1 -- 172.21.15.146:0/742239075 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c0c1290 con 0x7fe49c15a000 2024-09-13T18:28:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a2d1d700 1 --2- 172.21.15.146:0/742239075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe49c15a000 0x7fe49c0c2ce0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe494009d90 tx=0x7fe49400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac4b79406d18fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a151a700 1 -- 172.21.15.146:0/742239075 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49400e070 con 0x7fe49c15a000 2024-09-13T18:28:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a151a700 1 -- 172.21.15.146:0/742239075 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe494003a40 con 0x7fe49c15a000 2024-09-13T18:28:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.687+0000 7fe4a151a700 1 -- 172.21.15.146:0/742239075 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498001670 con 0x7fe49c15a000 2024-09-13T18:28:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 -- 172.21.15.146:0/742239075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe49c15a000 msgr2=0x7fe49c0c2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 --2- 172.21.15.146:0/742239075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe49c15a000 0x7fe49c0c2ce0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe494009d90 tx=0x7fe49400b330 comp rx=0 tx=0).stop 2024-09-13T18:28:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 -- 172.21.15.146:0/742239075 shutdown_connections 2024-09-13T18:28:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 --2- 172.21.15.146:0/742239075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe49c15a000 0x7fe49c0c2ce0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 --2- 172.21.15.146:0/742239075 >> v2:172.21.15.154:3300/0 conn(0x7fe49c06eba0 0x7fe49c0b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 -- 172.21.15.146:0/742239075 >> 172.21.15.146:0/742239075 conn(0x7fe49c071ed0 msgr2=0x7fe49c0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:36.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 -- 172.21.15.146:0/742239075 shutdown_connections 2024-09-13T18:28:36.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 -- 172.21.15.146:0/742239075 wait complete. 2024-09-13T18:28:36.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 Processor -- start 2024-09-13T18:28:36.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 -- start start 2024-09-13T18:28:36.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe49c06e7c0 0x7fe49c148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:36.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe49c06eba0 0x7fe49c1eec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:36.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe49c15a000 0x7fe49c1ef180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:36.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c1ef6c0 con 0x7fe49c06eba0 2024-09-13T18:28:36.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c152670 con 0x7fe49c15a000 2024-09-13T18:28:36.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c152930 con 0x7fe49c06e7c0 2024-09-13T18:28:36.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a1d1b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe49c06eba0 0x7fe49c1eec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a1d1b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe49c06eba0 0x7fe49c1eec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36572/0 (socket says 172.21.15.146:36572) 2024-09-13T18:28:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a1d1b700 1 -- 172.21.15.146:0/4196927450 learned_addr learned my addr 172.21.15.146:0/4196927450 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a2d1d700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe49c15a000 0x7fe49c1ef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a251c700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe49c06e7c0 0x7fe49c148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a1d1b700 1 -- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe49c06e7c0 msgr2=0x7fe49c148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a1d1b700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe49c06e7c0 0x7fe49c148c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a1d1b700 1 -- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe49c15a000 msgr2=0x7fe49c1ef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a1d1b700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe49c15a000 0x7fe49c1ef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a1d1b700 1 -- 172.21.15.146:0/4196927450 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe494009a70 con 0x7fe49c06eba0 2024-09-13T18:28:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a1d1b700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe49c06eba0 0x7fe49c1eec40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00e840 tx=0x7fe48c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe492ffd700 1 -- 172.21.15.146:0/4196927450 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c0095a0 con 0x7fe49c06eba0 2024-09-13T18:28:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe492ffd700 1 -- 172.21.15.146:0/4196927450 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c013680 con 0x7fe49c06eba0 2024-09-13T18:28:36.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe492ffd700 1 -- 172.21.15.146:0/4196927450 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c0139f0 con 0x7fe49c06eba0 2024-09-13T18:28:36.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 -- 172.21.15.146:0/4196927450 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49c152c10 con 0x7fe49c06eba0 2024-09-13T18:28:36.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.691+0000 7fe4a351e700 1 -- 172.21.15.146:0/4196927450 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c0ca460 con 0x7fe49c06eba0 2024-09-13T18:28:36.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.695+0000 7fe4a351e700 1 -- 172.21.15.146:0/4196927450 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49c0b11a0 con 0x7fe49c06eba0 2024-09-13T18:28:36.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.695+0000 7fe492ffd700 1 -- 172.21.15.146:0/4196927450 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe48c013b50 con 0x7fe49c06eba0 2024-09-13T18:28:36.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.695+0000 7fe492ffd700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe480066730 0x7fe480068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:36.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.695+0000 7fe492ffd700 1 -- 172.21.15.146:0/4196927450 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fe48c084810 con 0x7fe49c06eba0 2024-09-13T18:28:36.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.695+0000 7fe4a251c700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe480066730 0x7fe480068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:36.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.699+0000 7fe4a251c700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe480066730 0x7fe480068bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe4980011e0 tx=0x7fe498007a60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:36.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.703+0000 7fe492ffd700 1 -- 172.21.15.146:0/4196927450 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe48c051770 con 0x7fe49c06eba0 2024-09-13T18:28:36.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.951+0000 7fe4a351e700 1 -- 172.21.15.146:0/4196927450 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe49c06c040 con 0x7fe49c06eba0 2024-09-13T18:28:36.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.951+0000 7fe492ffd700 1 -- 172.21.15.146:0/4196927450 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5528 (secure 0 0 0) 0x7fe48c054d90 con 0x7fe49c06eba0 2024-09-13T18:28:36.957 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:36.957 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:36.053759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:active","state_seq":36,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:36.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.955+0000 7fe4a351e700 1 -- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe480066730 msgr2=0x7fe480068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:36.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.955+0000 7fe4a351e700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe480066730 0x7fe480068bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe4980011e0 tx=0x7fe498007a60 comp rx=0 tx=0).stop 2024-09-13T18:28:36.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.955+0000 7fe4a351e700 1 -- 172.21.15.146:0/4196927450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe49c06eba0 msgr2=0x7fe49c1eec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:36.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.955+0000 7fe4a351e700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe49c06eba0 0x7fe49c1eec40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00e840 tx=0x7fe48c00eb50 comp rx=0 tx=0).stop 2024-09-13T18:28:36.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.955+0000 7fe4a351e700 1 -- 172.21.15.146:0/4196927450 shutdown_connections 2024-09-13T18:28:36.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.955+0000 7fe4a351e700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe480066730 0x7fe480068bf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:36.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.955+0000 7fe4a351e700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe49c15a000 0x7fe49c1ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:36.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.955+0000 7fe4a351e700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe49c06eba0 0x7fe49c1eec40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:36.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.955+0000 7fe4a351e700 1 --2- 172.21.15.146:0/4196927450 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe49c06e7c0 0x7fe49c148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:36.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.955+0000 7fe4a351e700 1 -- 172.21.15.146:0/4196927450 >> 172.21.15.146:0/4196927450 conn(0x7fe49c071ed0 msgr2=0x7fe49c0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:36.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.955+0000 7fe4a351e700 1 -- 172.21.15.146:0/4196927450 shutdown_connections 2024-09-13T18:28:36.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:36.955+0000 7fe4a351e700 1 -- 172.21.15.146:0/4196927450 wait complete. 2024-09-13T18:28:36.963 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 20 2024-09-13T18:28:36.976 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=2, replacing gid=4269 2024-09-13T18:28:36.977 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.e 2024-09-13T18:28:37.245 INFO:tasks.ceph.mds.c.smithi146.stderr:2024-09-13T18:28:37.239+0000 7f6be472a700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-13T18:28:37.304 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:28:37.306 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:28:39.262 INFO:tasks.workunit.client.0.smithi146.stderr:Updating files: 81% (48/59) Updating files: 83% (49/59) Updating files: 84% (50/59) Updating files: 86% (51/59) Updating files: 88% (52/59) Updating files: 89% (53/59) Updating files: 91% (54/59) Updating files: 93% (55/59) Updating files: 94% (56/59) Updating files: 96% (57/59) Updating files: 98% (58/59) Updating files: 100% (59/59) Updating files: 100% (59/59), done. 2024-09-13T18:28:39.647 INFO:tasks.workunit.client.0.smithi146.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-13T18:28:40.668 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether build environment is sane... yes 2024-09-13T18:28:40.695 INFO:tasks.workunit.client.0.smithi146.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-13T18:28:40.696 INFO:tasks.workunit.client.0.smithi146.stdout:checking for gawk... gawk 2024-09-13T18:28:41.077 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether make sets $(MAKE)... yes 2024-09-13T18:28:41.179 INFO:tasks.workunit.client.0.smithi146.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-13T18:28:41.207 INFO:tasks.workunit.client.0.smithi146.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-13T18:28:41.213 INFO:tasks.workunit.client.0.smithi146.stdout:checking for style of include used by make... GNU 2024-09-13T18:28:41.222 INFO:tasks.workunit.client.0.smithi146.stdout:checking for gcc... gcc 2024-09-13T18:28:41.409 INFO:tasks.workunit.client.0.smithi146.stdout:checking for C compiler default output file name... a.out 2024-09-13T18:28:41.415 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether the C compiler works... yes 2024-09-13T18:28:41.422 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether we are cross compiling... no 2024-09-13T18:28:41.534 INFO:tasks.workunit.client.0.smithi146.stdout:checking for suffix of executables... 2024-09-13T18:28:41.617 INFO:tasks.workunit.client.0.smithi146.stdout:checking for suffix of object files... o 2024-09-13T18:28:41.698 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether we are using the GNU C compiler... yes 2024-09-13T18:28:41.790 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether gcc accepts -g... yes 2024-09-13T18:28:41.928 INFO:tasks.workunit.client.0.smithi146.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-13T18:28:42.188 INFO:tasks.workunit.client.0.smithi146.stdout:checking dependency style of gcc... gcc3 2024-09-13T18:28:42.321 INFO:tasks.workunit.client.0.smithi146.stdout:checking how to run the C preprocessor... gcc -E 2024-09-13T18:28:42.416 INFO:tasks.workunit.client.0.smithi146.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-13T18:28:42.419 INFO:tasks.workunit.client.0.smithi146.stdout:checking for egrep... /usr/bin/grep -E 2024-09-13T18:28:42.442 INFO:tasks.workunit.client.0.smithi146.stdout:checking for AIX... no 2024-09-13T18:28:42.481 INFO:tasks.workunit.client.0.smithi146.stdout:checking for gcc... (cached) gcc 2024-09-13T18:28:42.506 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-13T18:28:42.509 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-13T18:28:42.510 INFO:tasks.workunit.client.0.smithi146.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-13T18:28:42.512 INFO:tasks.workunit.client.0.smithi146.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-13T18:28:42.660 INFO:tasks.workunit.client.0.smithi146.stdout:checking for main in -lm... yes 2024-09-13T18:28:42.808 INFO:tasks.workunit.client.0.smithi146.stdout:checking for main in -lpthread... yes 2024-09-13T18:28:42.978 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2024-09-13T18:28:42.978 INFO:tasks.ceph.mds.e:Restarting daemon 2024-09-13T18:28:42.978 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-09-13T18:28:42.980 INFO:tasks.ceph.mds.e:Started 2024-09-13T18:28:42.980 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:43.076 INFO:tasks.ceph.mds.e.smithi146.stdout:starting mds.e at 2024-09-13T18:28:43.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.099+0000 7fce000ca700 1 Processor -- start 2024-09-13T18:28:43.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.099+0000 7fce000ca700 1 -- start start 2024-09-13T18:28:43.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcdf80c83e0 0x7fcdf80c7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:43.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80c87c0 0x7fcdf80c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:43.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcdf80c6b40 0x7fcdf80c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:43.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf80c1880 con 0x7fcdf80c83e0 2024-09-13T18:28:43.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf80716e0 con 0x7fcdf80c6b40 2024-09-13T18:28:43.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8072260 con 0x7fcdf80c87c0 2024-09-13T18:28:43.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfde66700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcdf80c83e0 0x7fcdf80c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:43.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfde66700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcdf80c83e0 0x7fcdf80c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38588/0 (socket says 172.21.15.146:38588) 2024-09-13T18:28:43.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfde66700 1 -- 172.21.15.146:0/3350601353 learned_addr learned my addr 172.21.15.146:0/3350601353 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:43.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfd665700 1 --2- 172.21.15.146:0/3350601353 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80c87c0 0x7fcdf80c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:43.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfe667700 1 --2- 172.21.15.146:0/3350601353 >> v2:172.21.15.154:3300/0 conn(0x7fcdf80c6b40 0x7fcdf80c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:43.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfde66700 1 -- 172.21.15.146:0/3350601353 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80c87c0 msgr2=0x7fcdf80c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:43.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfde66700 1 --2- 172.21.15.146:0/3350601353 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80c87c0 0x7fcdf80c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:43.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfde66700 1 -- 172.21.15.146:0/3350601353 >> v2:172.21.15.154:3300/0 conn(0x7fcdf80c6b40 msgr2=0x7fcdf80c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:43.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfde66700 1 --2- 172.21.15.146:0/3350601353 >> v2:172.21.15.154:3300/0 conn(0x7fcdf80c6b40 0x7fcdf80c2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:43.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfde66700 1 -- 172.21.15.146:0/3350601353 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80c7a90 con 0x7fcdf80c83e0 2024-09-13T18:28:43.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfe667700 1 --2- 172.21.15.146:0/3350601353 >> v2:172.21.15.154:3300/0 conn(0x7fcdf80c6b40 0x7fcdf80c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:28:43.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfde66700 1 --2- 172.21.15.146:0/3350601353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80c83e0 0x7fcdf80c7550 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80c2610 tx=0x7fcdf4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897547293f52e564 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:43.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfce64700 1 -- 172.21.15.146:0/3350601353 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4011830 con 0x7fcdf80c83e0 2024-09-13T18:28:43.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfce64700 1 -- 172.21.15.146:0/3350601353 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf4011990 con 0x7fcdf80c83e0 2024-09-13T18:28:43.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fcdfce64700 1 -- 172.21.15.146:0/3350601353 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4011d20 con 0x7fcdf80c83e0 2024-09-13T18:28:43.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 -- 172.21.15.146:0/3350601353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80c83e0 msgr2=0x7fcdf80c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:43.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 --2- 172.21.15.146:0/3350601353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80c83e0 0x7fcdf80c7550 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80c2610 tx=0x7fcdf4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:28:43.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 -- 172.21.15.146:0/3350601353 shutdown_connections 2024-09-13T18:28:43.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 --2- 172.21.15.146:0/3350601353 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80c87c0 0x7fcdf80c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:43.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 --2- 172.21.15.146:0/3350601353 >> v2:172.21.15.154:3300/0 conn(0x7fcdf80c6b40 0x7fcdf80c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:43.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 --2- 172.21.15.146:0/3350601353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80c83e0 0x7fcdf80c7550 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:43.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 -- 172.21.15.146:0/3350601353 >> 172.21.15.146:0/3350601353 conn(0x7fcdf80cc140 msgr2=0x7fcdf80cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:43.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 -- 172.21.15.146:0/3350601353 shutdown_connections 2024-09-13T18:28:43.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.103+0000 7fce000ca700 1 -- 172.21.15.146:0/3350601353 wait complete. 2024-09-13T18:28:43.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.107+0000 7fce000ca700 1 Processor -- start 2024-09-13T18:28:43.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.107+0000 7fce000ca700 1 -- start start 2024-09-13T18:28:43.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.107+0000 7fce000ca700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80c6b40 0x7fcdf80b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:43.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.107+0000 7fce000ca700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80c83e0 0x7fcdf8153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:43.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.107+0000 7fce000ca700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80c87c0 0x7fcdf81469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:43.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.107+0000 7fce000ca700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80b0c90 con 0x7fcdf80c87c0 2024-09-13T18:28:43.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.107+0000 7fce000ca700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80aef80 con 0x7fcdf80c6b40 2024-09-13T18:28:43.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.107+0000 7fce000ca700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8070300 con 0x7fcdf80c83e0 2024-09-13T18:28:43.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdfe667700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80c87c0 0x7fcdf81469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:43.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdfe667700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80c87c0 0x7fcdf81469b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38602/0 (socket says 172.21.15.146:38602) 2024-09-13T18:28:43.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdfe667700 1 -- 172.21.15.146:0/2892762115 learned_addr learned my addr 172.21.15.146:0/2892762115 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:43.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdfde66700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80c6b40 0x7fcdf80b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:43.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdfd665700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80c83e0 0x7fcdf8153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:43.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdfe667700 1 -- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80c83e0 msgr2=0x7fcdf8153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:43.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdfe667700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80c83e0 0x7fcdf8153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:43.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdfe667700 1 -- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80c6b40 msgr2=0x7fcdf80b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:43.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdfe667700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80c6b40 0x7fcdf80b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:43.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdfe667700 1 -- 172.21.15.146:0/2892762115 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf4009570 con 0x7fcdf80c87c0 2024-09-13T18:28:43.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdfe667700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80c87c0 0x7fcdf81469b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0007e70 tx=0x7fcdf000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:43.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdee7fc700 1 -- 172.21.15.146:0/2892762115 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf000dd20 con 0x7fcdf80c87c0 2024-09-13T18:28:43.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdee7fc700 1 -- 172.21.15.146:0/2892762115 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf000f040 con 0x7fcdf80c87c0 2024-09-13T18:28:43.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fce000ca700 1 -- 172.21.15.146:0/2892762115 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf80ae9c0 con 0x7fcdf80c87c0 2024-09-13T18:28:43.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdee7fc700 1 -- 172.21.15.146:0/2892762115 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0003c80 con 0x7fcdf80c87c0 2024-09-13T18:28:43.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fce000ca700 1 -- 172.21.15.146:0/2892762115 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf80b1620 con 0x7fcdf80c87c0 2024-09-13T18:28:43.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fce000ca700 1 -- 172.21.15.146:0/2892762115 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf80b0e00 con 0x7fcdf80c87c0 2024-09-13T18:28:43.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.111+0000 7fcdee7fc700 1 -- 172.21.15.146:0/2892762115 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcdf0004030 con 0x7fcdf80c87c0 2024-09-13T18:28:43.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.115+0000 7fcdee7fc700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcddc066730 0x7fcddc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:43.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.115+0000 7fcdee7fc700 1 -- 172.21.15.146:0/2892762115 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fcdf0084400 con 0x7fcdf80c87c0 2024-09-13T18:28:43.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.115+0000 7fcdfde66700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcddc066730 0x7fcddc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:43.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.115+0000 7fcdfde66700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcddc066730 0x7fcddc068bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4000c00 tx=0x7fcdf4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:43.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.119+0000 7fcdee7fc700 1 -- 172.21.15.146:0/2892762115 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcdf80b0e00 con 0x7fcdf80c87c0 2024-09-13T18:28:43.133 INFO:tasks.workunit.client.0.smithi146.stdout:checking for ANSI C header files... yes 2024-09-13T18:28:43.247 INFO:tasks.workunit.client.0.smithi146.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-13T18:28:43.345 INFO:tasks.workunit.client.0.smithi146.stdout:checking for sys/types.h... yes 2024-09-13T18:28:43.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.355+0000 7fce000ca700 1 -- 172.21.15.146:0/2892762115 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdf80b0ab0 con 0x7fcdf80c87c0 2024-09-13T18:28:43.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.359+0000 7fcdee7fc700 1 -- 172.21.15.146:0/2892762115 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5528 (secure 0 0 0) 0x7fcdf00513f0 con 0x7fcdf80c87c0 2024-09-13T18:28:43.364 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:43.364 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:36.053759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:active","state_seq":36,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:43.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.359+0000 7fce000ca700 1 -- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcddc066730 msgr2=0x7fcddc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:43.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.359+0000 7fce000ca700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcddc066730 0x7fcddc068bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4000c00 tx=0x7fcdf4008030 comp rx=0 tx=0).stop 2024-09-13T18:28:43.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.359+0000 7fce000ca700 1 -- 172.21.15.146:0/2892762115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80c87c0 msgr2=0x7fcdf81469b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:43.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.359+0000 7fce000ca700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80c87c0 0x7fcdf81469b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0007e70 tx=0x7fcdf000d3b0 comp rx=0 tx=0).stop 2024-09-13T18:28:43.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.359+0000 7fce000ca700 1 -- 172.21.15.146:0/2892762115 shutdown_connections 2024-09-13T18:28:43.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.359+0000 7fce000ca700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcddc066730 0x7fcddc068bf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:43.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.359+0000 7fce000ca700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf80c87c0 0x7fcdf81469b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:43.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.359+0000 7fce000ca700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdf80c83e0 0x7fcdf8153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:43.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.359+0000 7fce000ca700 1 --2- 172.21.15.146:0/2892762115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdf80c6b40 0x7fcdf80b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:43.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.359+0000 7fce000ca700 1 -- 172.21.15.146:0/2892762115 >> 172.21.15.146:0/2892762115 conn(0x7fcdf80cc140 msgr2=0x7fcdf80ca9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:43.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.359+0000 7fce000ca700 1 -- 172.21.15.146:0/2892762115 shutdown_connections 2024-09-13T18:28:43.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:43.359+0000 7fce000ca700 1 -- 172.21.15.146:0/2892762115 wait complete. 2024-09-13T18:28:43.368 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 20 2024-09-13T18:28:43.379 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2024-09-13T18:28:43.450 INFO:tasks.workunit.client.0.smithi146.stdout:checking for sys/stat.h... yes 2024-09-13T18:28:43.563 INFO:tasks.workunit.client.0.smithi146.stdout:checking for stdlib.h... yes 2024-09-13T18:28:43.678 INFO:tasks.workunit.client.0.smithi146.stdout:checking for string.h... yes 2024-09-13T18:28:43.815 INFO:tasks.workunit.client.0.smithi146.stdout:checking for memory.h... yes 2024-09-13T18:28:43.935 INFO:tasks.workunit.client.0.smithi146.stdout:checking for strings.h... yes 2024-09-13T18:28:44.203 INFO:tasks.workunit.client.0.smithi146.stdout:checking for inttypes.h... yes 2024-09-13T18:28:44.338 INFO:tasks.workunit.client.0.smithi146.stdout:checking for stdint.h... yes 2024-09-13T18:28:44.470 INFO:tasks.workunit.client.0.smithi146.stdout:checking for unistd.h... yes 2024-09-13T18:28:44.623 INFO:tasks.workunit.client.0.smithi146.stdout:checking pthread.h usability... yes 2024-09-13T18:28:44.672 INFO:tasks.workunit.client.0.smithi146.stdout:checking pthread.h presence... yes 2024-09-13T18:28:44.674 INFO:tasks.workunit.client.0.smithi146.stdout:checking for pthread.h... yes 2024-09-13T18:28:44.809 INFO:tasks.workunit.client.0.smithi146.stdout:checking fcntl.h usability... yes 2024-09-13T18:28:44.863 INFO:tasks.workunit.client.0.smithi146.stdout:checking fcntl.h presence... yes 2024-09-13T18:28:44.865 INFO:tasks.workunit.client.0.smithi146.stdout:checking for fcntl.h... yes 2024-09-13T18:28:44.993 INFO:tasks.workunit.client.0.smithi146.stdout:checking limits.h usability... yes 2024-09-13T18:28:45.036 INFO:tasks.workunit.client.0.smithi146.stdout:checking limits.h presence... yes 2024-09-13T18:28:45.038 INFO:tasks.workunit.client.0.smithi146.stdout:checking for limits.h... yes 2024-09-13T18:28:45.079 INFO:tasks.workunit.client.0.smithi146.stdout:checking for stdint.h... (cached) yes 2024-09-13T18:28:45.186 INFO:tasks.workunit.client.0.smithi146.stdout:checking sys/time.h usability... yes 2024-09-13T18:28:45.229 INFO:tasks.workunit.client.0.smithi146.stdout:checking sys/time.h presence... yes 2024-09-13T18:28:45.231 INFO:tasks.workunit.client.0.smithi146.stdout:checking for sys/time.h... yes 2024-09-13T18:28:45.274 INFO:tasks.workunit.client.0.smithi146.stdout:checking for unistd.h... (cached) yes 2024-09-13T18:28:45.380 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:45.386 INFO:tasks.workunit.client.0.smithi146.stdout:checking sys/vfs.h usability... yes 2024-09-13T18:28:45.428 INFO:tasks.workunit.client.0.smithi146.stdout:checking sys/vfs.h presence... yes 2024-09-13T18:28:45.429 INFO:tasks.workunit.client.0.smithi146.stdout:checking for sys/vfs.h... yes 2024-09-13T18:28:45.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92965f6700 1 Processor -- start 2024-09-13T18:28:45.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92965f6700 1 -- start start 2024-09-13T18:28:45.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92965f6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f929006e7c0 0x7f92900b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:45.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92965f6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f929015a000 0x7f92900c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:45.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92965f6700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92900c1020 con 0x7f929006eba0 2024-09-13T18:28:45.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92965f6700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92901593f0 con 0x7f929006e7c0 2024-09-13T18:28:45.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92965f6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92900c6a70 con 0x7f929015a000 2024-09-13T18:28:45.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92955f4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f929006e7c0 0x7f92900b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:45.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f9294df3700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f929006eba0 0x7f92901598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:54620/0 (socket says 172.21.15.146:54620) 2024-09-13T18:28:45.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92955f4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f929006e7c0 0x7f92900b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38618/0 (socket says 172.21.15.146:38618) 2024-09-13T18:28:45.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f9294df3700 1 -- 172.21.15.146:0/30620578 learned_addr learned my addr 172.21.15.146:0/30620578 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:28:45.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f9295df5700 1 --2- 172.21.15.146:0/30620578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f929015a000 0x7f92900c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:45.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92955f4700 1 -- 172.21.15.146:0/30620578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f929015a000 msgr2=0x7f92900c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:45.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92955f4700 1 --2- 172.21.15.146:0/30620578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f929015a000 0x7f92900c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:45.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92955f4700 1 -- 172.21.15.146:0/30620578 >> v1:172.21.15.146:6789/0 conn(0x7f929006eba0 legacy=0x7f92901598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:45.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92955f4700 1 -- 172.21.15.146:0/30620578 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92900c1290 con 0x7f929006e7c0 2024-09-13T18:28:45.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f92955f4700 1 --2- 172.21.15.146:0/30620578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f929006e7c0 0x7f92900b11a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9288009d90 tx=0x7f928800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e82397a952d427 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:45.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f9287fff700 1 -- 172.21.15.146:0/30620578 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928800e070 con 0x7f929006e7c0 2024-09-13T18:28:45.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f9287fff700 1 -- 172.21.15.146:0/30620578 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928800bbd0 con 0x7f929006e7c0 2024-09-13T18:28:45.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.495+0000 7f9287fff700 1 -- 172.21.15.146:0/30620578 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9288012430 con 0x7f929006e7c0 2024-09-13T18:28:45.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 -- 172.21.15.146:0/30620578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f929006e7c0 msgr2=0x7f92900b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:45.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 --2- 172.21.15.146:0/30620578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f929006e7c0 0x7f92900b11a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9288009d90 tx=0x7f928800b330 comp rx=0 tx=0).stop 2024-09-13T18:28:45.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 -- 172.21.15.146:0/30620578 shutdown_connections 2024-09-13T18:28:45.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 --2- 172.21.15.146:0/30620578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f929015a000 0x7f92900c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:45.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 --2- 172.21.15.146:0/30620578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f929006e7c0 0x7f92900b11a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:45.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 -- 172.21.15.146:0/30620578 >> 172.21.15.146:0/30620578 conn(0x7f9290071ed0 msgr2=0x7f92900722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:45.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 -- 172.21.15.146:0/30620578 shutdown_connections 2024-09-13T18:28:45.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 -- 172.21.15.146:0/30620578 wait complete. 2024-09-13T18:28:45.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 Processor -- start 2024-09-13T18:28:45.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 -- start start 2024-09-13T18:28:45.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f929006e7c0 0x7f9290150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:45.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f929006eba0 0x7f92901e6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:45.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f929015a000 0x7f92900c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92901f3000 con 0x7f929006e7c0 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9290151290 con 0x7f929006eba0 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92901f2170 con 0x7f929015a000 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92955f4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f929006e7c0 0x7f9290150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92955f4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f929006e7c0 0x7f9290150d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38628/0 (socket says 172.21.15.146:38628) 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92955f4700 1 -- 172.21.15.146:0/182939117 learned_addr learned my addr 172.21.15.146:0/182939117 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f9295df5700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f929015a000 0x7f92900c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f9294df3700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f929006eba0 0x7f92901e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92955f4700 1 -- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f929015a000 msgr2=0x7f92900c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92955f4700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f929015a000 0x7f92900c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92955f4700 1 -- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f929006eba0 msgr2=0x7f92901e6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92955f4700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f929006eba0 0x7f92901e6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92955f4700 1 -- 172.21.15.146:0/182939117 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9288009a70 con 0x7f929006e7c0 2024-09-13T18:28:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f9294df3700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f929006eba0 0x7f92901e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:28:45.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92955f4700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f929006e7c0 0x7f9290150d50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f929006da00 tx=0x7f9288004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:45.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f9285ffb700 1 -- 172.21.15.146:0/182939117 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9288012430 con 0x7f929006e7c0 2024-09-13T18:28:45.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f9285ffb700 1 -- 172.21.15.146:0/182939117 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9288020070 con 0x7f929006e7c0 2024-09-13T18:28:45.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f9285ffb700 1 -- 172.21.15.146:0/182939117 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9288004280 con 0x7f929006e7c0 2024-09-13T18:28:45.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 -- 172.21.15.146:0/182939117 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92901f23a0 con 0x7f929006e7c0 2024-09-13T18:28:45.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.499+0000 7f92965f6700 1 -- 172.21.15.146:0/182939117 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92901f2940 con 0x7f929006e7c0 2024-09-13T18:28:45.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.503+0000 7f92965f6700 1 -- 172.21.15.146:0/182939117 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92900b11a0 con 0x7f929006e7c0 2024-09-13T18:28:45.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.503+0000 7f9285ffb700 1 -- 172.21.15.146:0/182939117 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f92880043e0 con 0x7f929006e7c0 2024-09-13T18:28:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.503+0000 7f9285ffb700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9274066760 0x7f9274068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.503+0000 7f9285ffb700 1 -- 172.21.15.146:0/182939117 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f9288022070 con 0x7f929006e7c0 2024-09-13T18:28:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.503+0000 7f9294df3700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9274066760 0x7f9274068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.503+0000 7f9294df3700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9274066760 0x7f9274068c20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9280007fd0 tx=0x7f9280009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:45.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.507+0000 7f9285ffb700 1 -- 172.21.15.146:0/182939117 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f92880532e0 con 0x7f929006e7c0 2024-09-13T18:28:45.543 INFO:tasks.workunit.client.0.smithi146.stdout:checking sys/limits.h usability... no 2024-09-13T18:28:45.589 INFO:tasks.workunit.client.0.smithi146.stdout:checking sys/limits.h presence... no 2024-09-13T18:28:45.590 INFO:tasks.workunit.client.0.smithi146.stdout:checking for sys/limits.h... no 2024-09-13T18:28:45.654 INFO:tasks.workunit.client.0.smithi146.stdout:checking for an ANSI C-conforming const... yes 2024-09-13T18:28:45.723 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-13T18:28:45.727 INFO:tasks.workunit.client.0.smithi146.stdout:checking for function prototypes... yes 2024-09-13T18:28:45.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.751+0000 7f92965f6700 1 -- 172.21.15.146:0/182939117 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f929006e310 con 0x7f929006e7c0 2024-09-13T18:28:45.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.751+0000 7f9285ffb700 1 -- 172.21.15.146:0/182939117 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f9288056900 con 0x7f929006e7c0 2024-09-13T18:28:45.760 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:28:45.760 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5235,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:36.053759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:active","state_seq":36,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:45.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.755+0000 7f92965f6700 1 -- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9274066760 msgr2=0x7f9274068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:45.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.755+0000 7f92965f6700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9274066760 0x7f9274068c20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9280007fd0 tx=0x7f9280009040 comp rx=0 tx=0).stop 2024-09-13T18:28:45.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.755+0000 7f92965f6700 1 -- 172.21.15.146:0/182939117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f929006e7c0 msgr2=0x7f9290150d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:45.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.755+0000 7f92965f6700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f929006e7c0 0x7f9290150d50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f929006da00 tx=0x7f9288004af0 comp rx=0 tx=0).stop 2024-09-13T18:28:45.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.755+0000 7f92965f6700 1 -- 172.21.15.146:0/182939117 shutdown_connections 2024-09-13T18:28:45.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.755+0000 7f92965f6700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9274066760 0x7f9274068c20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:45.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.755+0000 7f92965f6700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f929015a000 0x7f92900c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:45.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.755+0000 7f92965f6700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f929006eba0 0x7f92901e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:45.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.755+0000 7f92965f6700 1 --2- 172.21.15.146:0/182939117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f929006e7c0 0x7f9290150d50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:45.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.755+0000 7f92965f6700 1 -- 172.21.15.146:0/182939117 >> 172.21.15.146:0/182939117 conn(0x7f9290071ed0 msgr2=0x7f929014eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:45.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.755+0000 7f92965f6700 1 -- 172.21.15.146:0/182939117 shutdown_connections 2024-09-13T18:28:45.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:28:45.755+0000 7f92965f6700 1 -- 172.21.15.146:0/182939117 wait complete. 2024-09-13T18:28:45.765 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 21 2024-09-13T18:28:45.776 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2024-09-13T18:28:45.777 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 25 secs before thrashing 2024-09-13T18:28:45.811 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether setvbuf arguments are reversed... no 2024-09-13T18:28:46.295 INFO:tasks.workunit.client.0.smithi146.stdout:checking for system... yes 2024-09-13T18:28:46.465 INFO:tasks.workunit.client.0.smithi146.stdout:checking for gettimeofday... yes 2024-09-13T18:28:46.623 INFO:tasks.workunit.client.0.smithi146.stdout:checking for mkdir... yes 2024-09-13T18:28:46.788 INFO:tasks.workunit.client.0.smithi146.stdout:checking for strerror... yes 2024-09-13T18:28:46.949 INFO:tasks.workunit.client.0.smithi146.stdout:checking for open64... yes 2024-09-13T18:28:47.098 INFO:tasks.workunit.client.0.smithi146.stdout:checking for stat64... yes 2024-09-13T18:28:47.257 INFO:tasks.workunit.client.0.smithi146.stdout:checking for fseeko64... yes 2024-09-13T18:28:47.900 INFO:tasks.workunit.client.0.smithi146.stdout:checking for lrand48_r... yes 2024-09-13T18:28:48.057 INFO:tasks.workunit.client.0.smithi146.stdout:checking for srand48_r... yes 2024-09-13T18:28:48.124 INFO:tasks.workunit.client.0.smithi146.stdout:configure: creating ./config.status 2024-09-13T18:28:48.398 INFO:tasks.workunit.client.0.smithi146.stdout:config.status: creating Makefile 2024-09-13T18:28:48.440 INFO:tasks.workunit.client.0.smithi146.stdout:config.status: creating config.h 2024-09-13T18:28:48.906 INFO:tasks.workunit.client.0.smithi146.stdout:config.status: executing depfiles commands 2024-09-13T18:28:49.029 INFO:tasks.workunit.client.0.smithi146.stdout:config.status: executing default commands 2024-09-13T18:28:49.187 INFO:tasks.workunit.client.0.smithi146.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-09-13T18:28:49.207 INFO:tasks.workunit.client.0.smithi146.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-09-13T18:28:49.210 INFO:tasks.workunit.client.0.smithi146.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-09-13T18:28:49.210 INFO:tasks.workunit.client.0.smithi146.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-09-13T18:28:49.210 INFO:tasks.workunit.client.0.smithi146.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-09-13T18:28:49.210 INFO:tasks.workunit.client.0.smithi146.stderr: any GNU archive site. 2024-09-13T18:28:49.220 INFO:tasks.workunit.client.0.smithi146.stdout: cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-09-13T18:28:49.230 INFO:tasks.workunit.client.0.smithi146.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-09-13T18:28:49.232 INFO:tasks.workunit.client.0.smithi146.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-09-13T18:28:49.233 INFO:tasks.workunit.client.0.smithi146.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-09-13T18:28:49.233 INFO:tasks.workunit.client.0.smithi146.stderr: You might want to install the `Automake' and `Perl' packages. 2024-09-13T18:28:49.233 INFO:tasks.workunit.client.0.smithi146.stderr: Grab them from any GNU archive site. 2024-09-13T18:28:52.277 INFO:tasks.workunit.client.0.smithi146.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-09-13T18:28:52.542 INFO:tasks.workunit.client.0.smithi146.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-13T18:28:52.542 INFO:tasks.workunit.client.0.smithi146.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-13T18:28:52.542 INFO:tasks.workunit.client.0.smithi146.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-13T18:28:52.542 INFO:tasks.workunit.client.0.smithi146.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-13T18:28:52.726 INFO:tasks.workunit.client.0.smithi146.stdout:/bin/bash ./config.status --recheck 2024-09-13T18:28:52.834 INFO:tasks.workunit.client.0.smithi146.stdout:running CONFIG_SHELL=/bin/bash /bin/bash ./configure --no-create --no-recursion 2024-09-13T18:28:53.493 INFO:tasks.workunit.client.0.smithi146.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-13T18:28:54.545 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether build environment is sane... yes 2024-09-13T18:28:54.587 INFO:tasks.workunit.client.0.smithi146.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-13T18:28:54.612 INFO:tasks.workunit.client.0.smithi146.stdout:checking for gawk... gawk 2024-09-13T18:28:54.694 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether make sets $(MAKE)... yes 2024-09-13T18:28:54.825 INFO:tasks.workunit.client.0.smithi146.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-13T18:28:54.845 INFO:tasks.workunit.client.0.smithi146.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-13T18:28:54.898 INFO:tasks.workunit.client.0.smithi146.stdout:checking for style of include used by make... GNU 2024-09-13T18:28:54.936 INFO:tasks.workunit.client.0.smithi146.stdout:checking for gcc... gcc 2024-09-13T18:28:55.252 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether the C compiler works... yes 2024-09-13T18:28:55.269 INFO:tasks.workunit.client.0.smithi146.stdout:checking for C compiler default output file name... a.out 2024-09-13T18:28:56.389 INFO:tasks.workunit.client.0.smithi146.stdout:checking for suffix of executables... 2024-09-13T18:28:56.563 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether we are cross compiling... no 2024-09-13T18:28:56.716 INFO:tasks.workunit.client.0.smithi146.stdout:checking for suffix of object files... o 2024-09-13T18:28:56.839 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether we are using the GNU C compiler... yes 2024-09-13T18:28:56.957 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether gcc accepts -g... yes 2024-09-13T18:28:57.107 INFO:tasks.workunit.client.0.smithi146.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-13T18:28:57.363 INFO:tasks.workunit.client.0.smithi146.stdout:checking dependency style of gcc... gcc3 2024-09-13T18:28:58.504 INFO:tasks.workunit.client.0.smithi146.stdout:checking how to run the C preprocessor... gcc -E 2024-09-13T18:28:58.825 INFO:tasks.workunit.client.0.smithi146.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-13T18:28:58.846 INFO:tasks.workunit.client.0.smithi146.stdout:checking for egrep... /usr/bin/grep -E 2024-09-13T18:29:00.210 INFO:tasks.workunit.client.0.smithi146.stdout:checking for ANSI C header files... yes 2024-09-13T18:29:00.369 INFO:tasks.workunit.client.0.smithi146.stdout:checking for sys/types.h... yes 2024-09-13T18:29:00.624 INFO:tasks.workunit.client.0.smithi146.stdout:checking for sys/stat.h... yes 2024-09-13T18:29:00.992 INFO:tasks.workunit.client.0.smithi146.stdout:checking for stdlib.h... yes 2024-09-13T18:29:01.158 INFO:tasks.workunit.client.0.smithi146.stdout:checking for string.h... yes 2024-09-13T18:29:02.287 INFO:tasks.workunit.client.0.smithi146.stdout:checking for memory.h... yes 2024-09-13T18:29:03.492 INFO:tasks.workunit.client.0.smithi146.stdout:checking for strings.h... yes 2024-09-13T18:29:04.367 INFO:tasks.workunit.client.0.smithi146.stdout:checking for inttypes.h... yes 2024-09-13T18:29:04.539 INFO:tasks.workunit.client.0.smithi146.stdout:checking for stdint.h... yes 2024-09-13T18:29:04.716 INFO:tasks.workunit.client.0.smithi146.stdout:checking for unistd.h... yes 2024-09-13T18:29:05.687 INFO:tasks.workunit.client.0.smithi146.stdout:checking minix/config.h usability... no 2024-09-13T18:29:06.626 INFO:tasks.workunit.client.0.smithi146.stdout:checking minix/config.h presence... no 2024-09-13T18:29:06.640 INFO:tasks.workunit.client.0.smithi146.stdout:checking for minix/config.h... no 2024-09-13T18:29:07.367 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:29:07.370 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:29:07.499 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-09-13T18:29:07.552 INFO:tasks.workunit.client.0.smithi146.stdout:checking for gcc... (cached) gcc 2024-09-13T18:29:08.291 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-13T18:29:08.315 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-13T18:29:08.333 INFO:tasks.workunit.client.0.smithi146.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-13T18:29:08.351 INFO:tasks.workunit.client.0.smithi146.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-13T18:29:08.538 INFO:tasks.workunit.client.0.smithi146.stdout:checking for main in -lm... yes 2024-09-13T18:29:09.012 INFO:tasks.workunit.client.0.smithi146.stdout:checking for main in -lpthread... yes 2024-09-13T18:29:09.463 INFO:tasks.workunit.client.0.smithi146.stdout:checking for ANSI C header files... (cached) yes 2024-09-13T18:29:09.625 INFO:tasks.workunit.client.0.smithi146.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-13T18:29:09.792 INFO:tasks.workunit.client.0.smithi146.stdout:checking pthread.h usability... yes 2024-09-13T18:29:09.893 INFO:tasks.workunit.client.0.smithi146.stdout:checking pthread.h presence... yes 2024-09-13T18:29:09.908 INFO:tasks.workunit.client.0.smithi146.stdout:checking for pthread.h... yes 2024-09-13T18:29:10.274 INFO:tasks.workunit.client.0.smithi146.stdout:checking fcntl.h usability... yes 2024-09-13T18:29:10.370 INFO:tasks.workunit.client.0.smithi146.stdout:checking fcntl.h presence... yes 2024-09-13T18:29:10.386 INFO:tasks.workunit.client.0.smithi146.stdout:checking for fcntl.h... yes 2024-09-13T18:29:10.564 INFO:tasks.workunit.client.0.smithi146.stdout:checking limits.h usability... yes 2024-09-13T18:29:10.778 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:10.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 Processor -- start 2024-09-13T18:29:10.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 -- start start 2024-09-13T18:29:10.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c9c0cb770 0x7f0c9c0cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:10.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9c15ec90 0x7f0c9c15f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:10.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9c0cb480 con 0x7f0c9c0cf6c0 2024-09-13T18:29:10.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9c161410 con 0x7f0c9c0cb770 2024-09-13T18:29:10.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c1616f0 con 0x7f0c9c15ec90 2024-09-13T18:29:10.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca160f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c9c0cb770 0x7f0c9c0cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:10.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca160f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c9c0cb770 0x7f0c9c0cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:47868/0 (socket says 172.21.15.146:47868) 2024-09-13T18:29:10.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca160f700 1 -- 172.21.15.146:0/1221127052 learned_addr learned my addr 172.21.15.146:0/1221127052 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:10.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca2611700 1 --2- 172.21.15.146:0/1221127052 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9c15ec90 0x7f0c9c15f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:10.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca160f700 1 -- 172.21.15.146:0/1221127052 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9c15ec90 msgr2=0x7f0c9c15f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:10.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca160f700 1 --2- 172.21.15.146:0/1221127052 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9c15ec90 0x7f0c9c15f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:10.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca160f700 1 -- 172.21.15.146:0/1221127052 >> v1:172.21.15.146:6789/0 conn(0x7f0c9c0cf6c0 legacy=0x7f0c9c15e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:10.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca160f700 1 -- 172.21.15.146:0/1221127052 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9c1619d0 con 0x7f0c9c0cb770 2024-09-13T18:29:10.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca160f700 1 --2- 172.21.15.146:0/1221127052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c9c0cb770 0x7f0c9c0cae60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009e50 tx=0x7f0c8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a3d2939e1dab45 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:10.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca0e0e700 1 -- 172.21.15.146:0/1221127052 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00e070 con 0x7f0c9c0cb770 2024-09-13T18:29:10.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca0e0e700 1 -- 172.21.15.146:0/1221127052 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8c00b9c0 con 0x7f0c9c0cb770 2024-09-13T18:29:10.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca0e0e700 1 -- 172.21.15.146:0/1221127052 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00bd70 con 0x7f0c9c0cb770 2024-09-13T18:29:10.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 -- 172.21.15.146:0/1221127052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c9c0cb770 msgr2=0x7f0c9c0cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:10.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 --2- 172.21.15.146:0/1221127052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c9c0cb770 0x7f0c9c0cae60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009e50 tx=0x7f0c8c00b330 comp rx=0 tx=0).stop 2024-09-13T18:29:10.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 -- 172.21.15.146:0/1221127052 shutdown_connections 2024-09-13T18:29:10.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 --2- 172.21.15.146:0/1221127052 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9c15ec90 0x7f0c9c15f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:10.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 --2- 172.21.15.146:0/1221127052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c9c0cb770 0x7f0c9c0cae60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:10.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 -- 172.21.15.146:0/1221127052 >> 172.21.15.146:0/1221127052 conn(0x7f0c9c071d90 msgr2=0x7f0c9c0721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:10.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.894+0000 7f0ca4074700 1 -- 172.21.15.146:0/1221127052 shutdown_connections 2024-09-13T18:29:10.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca4074700 1 -- 172.21.15.146:0/1221127052 wait complete. 2024-09-13T18:29:10.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca4074700 1 Processor -- start 2024-09-13T18:29:10.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca4074700 1 -- start start 2024-09-13T18:29:10.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca4074700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c9c0cb770 0x7f0c9c0c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:10.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca4074700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9c0cf6c0 0x7f0c9c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:10.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca4074700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c9c15ec90 0x7f0c9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:10.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca4074700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c0b1c60 con 0x7f0c9c0cb770 2024-09-13T18:29:10.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca4074700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c0b4470 con 0x7f0c9c15ec90 2024-09-13T18:29:10.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca4074700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c0c41c0 con 0x7f0c9c0cf6c0 2024-09-13T18:29:10.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca1e10700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c9c0cb770 0x7f0c9c0c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:10.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca1e10700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c9c0cb770 0x7f0c9c0c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:47882/0 (socket says 172.21.15.146:47882) 2024-09-13T18:29:10.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca1e10700 1 -- 172.21.15.146:0/501332767 learned_addr learned my addr 172.21.15.146:0/501332767 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:10.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca160f700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9c0cf6c0 0x7f0c9c06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:10.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca2611700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c9c15ec90 0x7f0c9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:10.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca1e10700 1 -- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9c0cf6c0 msgr2=0x7f0c9c06c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:10.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca1e10700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9c0cf6c0 0x7f0c9c06c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:10.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca1e10700 1 -- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c9c15ec90 msgr2=0x7f0c9c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:10.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca1e10700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c9c15ec90 0x7f0c9c06d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:10.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca1e10700 1 -- 172.21.15.146:0/501332767 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c009b30 con 0x7f0c9c0cb770 2024-09-13T18:29:10.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0ca1e10700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c9c0cb770 0x7f0c9c0c7c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800c500 tx=0x7f0c9800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:10.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.898+0000 7f0c927fc700 1 -- 172.21.15.146:0/501332767 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c980052d0 con 0x7f0c9c0cb770 2024-09-13T18:29:10.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.902+0000 7f0ca4074700 1 -- 172.21.15.146:0/501332767 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c0c5780 con 0x7f0c9c0cb770 2024-09-13T18:29:10.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.902+0000 7f0c927fc700 1 -- 172.21.15.146:0/501332767 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c98005430 con 0x7f0c9c0cb770 2024-09-13T18:29:10.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.902+0000 7f0ca4074700 1 -- 172.21.15.146:0/501332767 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c06d190 con 0x7f0c9c0cb770 2024-09-13T18:29:10.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.902+0000 7f0c927fc700 1 -- 172.21.15.146:0/501332767 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c980055a0 con 0x7f0c9c0cb770 2024-09-13T18:29:10.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.902+0000 7f0ca4074700 1 -- 172.21.15.146:0/501332767 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9c0caa80 con 0x7f0c9c0cb770 2024-09-13T18:29:10.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.902+0000 7f0c927fc700 1 -- 172.21.15.146:0/501332767 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0c98006a80 con 0x7f0c9c0cb770 2024-09-13T18:29:10.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.902+0000 7f0c927fc700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c88066730 0x7f0c88068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:10.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.902+0000 7f0c927fc700 1 -- 172.21.15.146:0/501332767 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f0c980840e0 con 0x7f0c9c0cb770 2024-09-13T18:29:10.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.902+0000 7f0ca160f700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c88066730 0x7f0c88068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:10.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.906+0000 7f0ca160f700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c88066730 0x7f0c88068bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009e50 tx=0x7f0c8c009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:10.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:10.910+0000 7f0c927fc700 1 -- 172.21.15.146:0/501332767 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0c98051040 con 0x7f0c9c0cb770 2024-09-13T18:29:11.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.122+0000 7f0ca4074700 1 -- 172.21.15.146:0/501332767 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c9c0c2460 con 0x7f0c9c0cb770 2024-09-13T18:29:11.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.122+0000 7f0c927fc700 1 -- 172.21.15.146:0/501332767 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f0c98054660 con 0x7f0c9c0cb770 2024-09-13T18:29:11.133 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:11.133 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5235,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:36.053759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:active","state_seq":36,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:11.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.126+0000 7f0ca4074700 1 -- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c88066730 msgr2=0x7f0c88068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:11.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.126+0000 7f0ca4074700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c88066730 0x7f0c88068bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009e50 tx=0x7f0c8c009000 comp rx=0 tx=0).stop 2024-09-13T18:29:11.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.126+0000 7f0ca4074700 1 -- 172.21.15.146:0/501332767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c9c0cb770 msgr2=0x7f0c9c0c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:11.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.126+0000 7f0ca4074700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c9c0cb770 0x7f0c9c0c7c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800c500 tx=0x7f0c9800c8c0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.126+0000 7f0ca4074700 1 -- 172.21.15.146:0/501332767 shutdown_connections 2024-09-13T18:29:11.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.126+0000 7f0ca4074700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c88066730 0x7f0c88068bf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.126+0000 7f0ca4074700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c9c15ec90 0x7f0c9c06d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.126+0000 7f0ca4074700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9c0cf6c0 0x7f0c9c06c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.126+0000 7f0ca4074700 1 --2- 172.21.15.146:0/501332767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c9c0cb770 0x7f0c9c0c7c40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.126+0000 7f0ca4074700 1 -- 172.21.15.146:0/501332767 >> 172.21.15.146:0/501332767 conn(0x7f0c9c071d90 msgr2=0x7f0c9c160000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:11.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.126+0000 7f0ca4074700 1 -- 172.21.15.146:0/501332767 shutdown_connections 2024-09-13T18:29:11.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.126+0000 7f0ca4074700 1 -- 172.21.15.146:0/501332767 wait complete. 2024-09-13T18:29:11.135 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 21 2024-09-13T18:29:11.145 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=2) 2024-09-13T18:29:11.145 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-09-13T18:29:11.145 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:29:11.162 INFO:tasks.ceph.mds.f:Stopped 2024-09-13T18:29:11.162 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:11.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.266+0000 7f2d27449700 1 Processor -- start 2024-09-13T18:29:11.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.266+0000 7f2d27449700 1 -- start start 2024-09-13T18:29:11.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d27449700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200721f0 0x7f2d20157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:11.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d27449700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2d200cac60 0x7f2d200cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:11.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d27449700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d200c8c70 con 0x7f2d20154a40 2024-09-13T18:29:11.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d27449700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d20072810 con 0x7f2d200cac60 2024-09-13T18:29:11.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d27449700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d20157910 con 0x7f2d200721f0 2024-09-13T18:29:11.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d25c46700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200721f0 0x7f2d20157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:11.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d26c48700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2d200cac60 0x7f2d200cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:11.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d26447700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2d20154a40 0x7f2d2006f7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:56354/0 (socket says 172.21.15.146:56354) 2024-09-13T18:29:11.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d26447700 1 -- 172.21.15.146:0/4010522776 learned_addr learned my addr 172.21.15.146:0/4010522776 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:29:11.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d25445700 1 -- 172.21.15.146:0/4010522776 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019380264 0 0) 0x7f2d200c8c70 con 0x7f2d20154a40 2024-09-13T18:29:11.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d25445700 1 -- 172.21.15.146:0/4010522776 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d080035d0 con 0x7f2d20154a40 2024-09-13T18:29:11.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d26c48700 1 -- 172.21.15.146:0/4010522776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200721f0 msgr2=0x7f2d20157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:11.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d26c48700 1 --2- 172.21.15.146:0/4010522776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200721f0 0x7f2d20157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d26c48700 1 -- 172.21.15.146:0/4010522776 >> v1:172.21.15.154:6789/0 conn(0x7f2d20154a40 legacy=0x7f2d2006f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:11.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d26c48700 1 -- 172.21.15.146:0/4010522776 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d200c8f50 con 0x7f2d200cac60 2024-09-13T18:29:11.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d25c46700 1 --2- 172.21.15.146:0/4010522776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200721f0 0x7f2d20157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:29:11.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d26c48700 1 --2- 172.21.15.146:0/4010522776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200cac60 0x7f2d200cb0f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009850 tx=0x7f2d18009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4893f77cb26b9088 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:11.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d25445700 1 -- 172.21.15.146:0/4010522776 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18011830 con 0x7f2d200cac60 2024-09-13T18:29:11.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d25445700 1 -- 172.21.15.146:0/4010522776 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d18011990 con 0x7f2d200cac60 2024-09-13T18:29:11.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d25445700 1 -- 172.21.15.146:0/4010522776 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18011d00 con 0x7f2d200cac60 2024-09-13T18:29:11.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d27449700 1 -- 172.21.15.146:0/4010522776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200cac60 msgr2=0x7f2d200cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:11.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d27449700 1 --2- 172.21.15.146:0/4010522776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200cac60 0x7f2d200cb0f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009850 tx=0x7f2d18009b60 comp rx=0 tx=0).stop 2024-09-13T18:29:11.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d27449700 1 -- 172.21.15.146:0/4010522776 shutdown_connections 2024-09-13T18:29:11.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d27449700 1 --2- 172.21.15.146:0/4010522776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200cac60 0x7f2d200cb0f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d27449700 1 --2- 172.21.15.146:0/4010522776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200721f0 0x7f2d20157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d27449700 1 -- 172.21.15.146:0/4010522776 >> 172.21.15.146:0/4010522776 conn(0x7f2d20066090 msgr2=0x7f2d20066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:11.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d27449700 1 -- 172.21.15.146:0/4010522776 shutdown_connections 2024-09-13T18:29:11.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.270+0000 7f2d27449700 1 -- 172.21.15.146:0/4010522776 wait complete. 2024-09-13T18:29:11.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d27449700 1 Processor -- start 2024-09-13T18:29:11.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d27449700 1 -- start start 2024-09-13T18:29:11.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d27449700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200721f0 0x7f2d201ec970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:11.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d27449700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200cac60 0x7f2d200c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:11.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d27449700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d20154a40 0x7f2d200c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:11.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d27449700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2006eb20 con 0x7f2d20154a40 2024-09-13T18:29:11.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d27449700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200c3b10 con 0x7f2d200cac60 2024-09-13T18:29:11.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d27449700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d20153310 con 0x7f2d200721f0 2024-09-13T18:29:11.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d26c48700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d20154a40 0x7f2d200c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:11.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d26c48700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d20154a40 0x7f2d200c44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53260/0 (socket says 172.21.15.146:53260) 2024-09-13T18:29:11.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d26c48700 1 -- 172.21.15.146:0/2054272976 learned_addr learned my addr 172.21.15.146:0/2054272976 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:11.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d25c46700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200cac60 0x7f2d200c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:11.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d26447700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200721f0 0x7f2d201ec970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:11.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d26c48700 1 -- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200721f0 msgr2=0x7f2d201ec970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:11.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d26c48700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200721f0 0x7f2d201ec970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d26c48700 1 -- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200cac60 msgr2=0x7f2d200c3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:11.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d26c48700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200cac60 0x7f2d200c3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d26c48700 1 -- 172.21.15.146:0/2054272976 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c009a70 con 0x7f2d20154a40 2024-09-13T18:29:11.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d25c46700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200cac60 0x7f2d200c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:29:11.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d26c48700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d20154a40 0x7f2d200c44e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2d180090b0 tx=0x7f2d18004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:11.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d16ffd700 1 -- 172.21.15.146:0/2054272976 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d180036c0 con 0x7f2d20154a40 2024-09-13T18:29:11.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d16ffd700 1 -- 172.21.15.146:0/2054272976 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1801e070 con 0x7f2d20154a40 2024-09-13T18:29:11.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d16ffd700 1 -- 172.21.15.146:0/2054272976 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18003aa0 con 0x7f2d20154a40 2024-09-13T18:29:11.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d27449700 1 -- 172.21.15.146:0/2054272976 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d180094b0 con 0x7f2d20154a40 2024-09-13T18:29:11.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.274+0000 7f2d27449700 1 -- 172.21.15.146:0/2054272976 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2006e840 con 0x7f2d20154a40 2024-09-13T18:29:11.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.278+0000 7f2d27449700 1 -- 172.21.15.146:0/2054272976 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d200c9e00 con 0x7f2d20154a40 2024-09-13T18:29:11.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.278+0000 7f2d16ffd700 1 -- 172.21.15.146:0/2054272976 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2d18003c00 con 0x7f2d20154a40 2024-09-13T18:29:11.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.278+0000 7f2d16ffd700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2d0c066730 0x7f2d0c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:11.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.278+0000 7f2d16ffd700 1 -- 172.21.15.146:0/2054272976 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f2d18085570 con 0x7f2d20154a40 2024-09-13T18:29:11.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.278+0000 7f2d26447700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2d0c066730 0x7f2d0c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:11.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.278+0000 7f2d26447700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2d0c066730 0x7f2d0c068bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2d10005930 tx=0x7f2d10009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:11.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.282+0000 7f2d16ffd700 1 -- 172.21.15.146:0/2054272976 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2d180524c0 con 0x7f2d20154a40 2024-09-13T18:29:11.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.490+0000 7f2d27449700 1 -- 172.21.15.146:0/2054272976 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d200c3c80 con 0x7f2d20154a40 2024-09-13T18:29:11.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.490+0000 7f2d16ffd700 1 -- 172.21.15.146:0/2054272976 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f2d18055ae0 con 0x7f2d20154a40 2024-09-13T18:29:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:11.499 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5235,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:36.053759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:active","state_seq":36,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.494+0000 7f2d27449700 1 -- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2d0c066730 msgr2=0x7f2d0c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.494+0000 7f2d27449700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2d0c066730 0x7f2d0c068bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2d10005930 tx=0x7f2d10009040 comp rx=0 tx=0).stop 2024-09-13T18:29:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.494+0000 7f2d27449700 1 -- 172.21.15.146:0/2054272976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d20154a40 msgr2=0x7f2d200c44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.494+0000 7f2d27449700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d20154a40 0x7f2d200c44e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2d180090b0 tx=0x7f2d18004770 comp rx=0 tx=0).stop 2024-09-13T18:29:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.494+0000 7f2d27449700 1 -- 172.21.15.146:0/2054272976 shutdown_connections 2024-09-13T18:29:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.494+0000 7f2d27449700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2d0c066730 0x7f2d0c068bf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.494+0000 7f2d27449700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d20154a40 0x7f2d200c44e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.494+0000 7f2d27449700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200cac60 0x7f2d200c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.494+0000 7f2d27449700 1 --2- 172.21.15.146:0/2054272976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200721f0 0x7f2d201ec970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.494+0000 7f2d27449700 1 -- 172.21.15.146:0/2054272976 >> 172.21.15.146:0/2054272976 conn(0x7f2d20066090 msgr2=0x7f2d200c5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:11.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.494+0000 7f2d27449700 1 -- 172.21.15.146:0/2054272976 shutdown_connections 2024-09-13T18:29:11.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:11.494+0000 7f2d27449700 1 -- 172.21.15.146:0/2054272976 wait complete. 2024-09-13T18:29:11.503 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 21 2024-09-13T18:29:11.514 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:29:13.515 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:13.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.630+0000 7f9487209700 1 Processor -- start 2024-09-13T18:29:13.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.630+0000 7f9487209700 1 -- start start 2024-09-13T18:29:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.630+0000 7f9487209700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f94800721f0 0x7f9480157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.630+0000 7f9487209700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f94800cac60 0x7f94800cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.630+0000 7f9487209700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94800c8c70 con 0x7f9480154a40 2024-09-13T18:29:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.630+0000 7f9487209700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9480072810 con 0x7f94800721f0 2024-09-13T18:29:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.630+0000 7f9487209700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9480157910 con 0x7f94800cac60 2024-09-13T18:29:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.630+0000 7f9486207700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9480154a40 0x7f948006f7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43904/0 (socket says 172.21.15.146:43904) 2024-09-13T18:29:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.630+0000 7f9486207700 1 -- 172.21.15.146:0/1025229260 learned_addr learned my addr 172.21.15.146:0/1025229260 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:29:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.630+0000 7f9485a06700 1 --2- 172.21.15.146:0/1025229260 >> v2:172.21.15.146:3300/0 conn(0x7f94800721f0 0x7f9480157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.630+0000 7f9486a08700 1 --2- 172.21.15.146:0/1025229260 >> v2:172.21.15.154:3300/0 conn(0x7f94800cac60 0x7f94800cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9485205700 1 -- 172.21.15.146:0/1025229260 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796366234 0 0) 0x7f94800c8c70 con 0x7f9480154a40 2024-09-13T18:29:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9485205700 1 -- 172.21.15.146:0/1025229260 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f946c0035d0 con 0x7f9480154a40 2024-09-13T18:29:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9485a06700 1 -- 172.21.15.146:0/1025229260 >> v2:172.21.15.154:3300/0 conn(0x7f94800cac60 msgr2=0x7f94800cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9485a06700 1 --2- 172.21.15.146:0/1025229260 >> v2:172.21.15.154:3300/0 conn(0x7f94800cac60 0x7f94800cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9485a06700 1 -- 172.21.15.146:0/1025229260 >> v1:172.21.15.146:6789/0 conn(0x7f9480154a40 legacy=0x7f948006f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9485a06700 1 -- 172.21.15.146:0/1025229260 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94800c8f50 con 0x7f94800721f0 2024-09-13T18:29:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9485a06700 1 --2- 172.21.15.146:0/1025229260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94800721f0 0x7f9480157d30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f94800cbd50 tx=0x7f9470009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c52440968993802 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9485205700 1 -- 172.21.15.146:0/1025229260 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470011830 con 0x7f94800721f0 2024-09-13T18:29:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9485205700 1 -- 172.21.15.146:0/1025229260 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9470011990 con 0x7f94800721f0 2024-09-13T18:29:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9485205700 1 -- 172.21.15.146:0/1025229260 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470011d00 con 0x7f94800721f0 2024-09-13T18:29:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 -- 172.21.15.146:0/1025229260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94800721f0 msgr2=0x7f9480157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 --2- 172.21.15.146:0/1025229260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94800721f0 0x7f9480157d30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f94800cbd50 tx=0x7f9470009b30 comp rx=0 tx=0).stop 2024-09-13T18:29:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 -- 172.21.15.146:0/1025229260 shutdown_connections 2024-09-13T18:29:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 --2- 172.21.15.146:0/1025229260 >> v2:172.21.15.154:3300/0 conn(0x7f94800cac60 0x7f94800cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 --2- 172.21.15.146:0/1025229260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94800721f0 0x7f9480157d30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 -- 172.21.15.146:0/1025229260 >> 172.21.15.146:0/1025229260 conn(0x7f9480066090 msgr2=0x7f9480066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 -- 172.21.15.146:0/1025229260 shutdown_connections 2024-09-13T18:29:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 -- 172.21.15.146:0/1025229260 wait complete. 2024-09-13T18:29:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 Processor -- start 2024-09-13T18:29:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 -- start start 2024-09-13T18:29:13.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94800721f0 0x7f94800afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94800cac60 0x7f948015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9480154a40 0x7f94801593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94800c3af0 con 0x7f94800721f0 2024-09-13T18:29:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94800c4090 con 0x7f9480154a40 2024-09-13T18:29:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9487209700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94800c3db0 con 0x7f94800cac60 2024-09-13T18:29:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9486207700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94800721f0 0x7f94800afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9486207700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94800721f0 0x7f94800afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53282/0 (socket says 172.21.15.146:53282) 2024-09-13T18:29:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.634+0000 7f9486207700 1 -- 172.21.15.146:0/1071139269 learned_addr learned my addr 172.21.15.146:0/1071139269 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9485a06700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94800cac60 0x7f948015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9486a08700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9480154a40 0x7f94801593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9486207700 1 -- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94800cac60 msgr2=0x7f948015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9486207700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94800cac60 0x7f948015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9486207700 1 -- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9480154a40 msgr2=0x7f94801593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9486207700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9480154a40 0x7f94801593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9486207700 1 -- 172.21.15.146:0/1071139269 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700094b0 con 0x7f94800721f0 2024-09-13T18:29:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9486a08700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9480154a40 0x7f94801593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:29:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9486207700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94800721f0 0x7f94800afa80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f947c00c4d0 tx=0x7f947c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9476ffd700 1 -- 172.21.15.146:0/1071139269 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c005d90 con 0x7f94800721f0 2024-09-13T18:29:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9476ffd700 1 -- 172.21.15.146:0/1071139269 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c0070d0 con 0x7f94800721f0 2024-09-13T18:29:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9476ffd700 1 -- 172.21.15.146:0/1071139269 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c007440 con 0x7f94800721f0 2024-09-13T18:29:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9487209700 1 -- 172.21.15.146:0/1071139269 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94800c4bc0 con 0x7f94800721f0 2024-09-13T18:29:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9487209700 1 -- 172.21.15.146:0/1071139269 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94800affc0 con 0x7f94800721f0 2024-09-13T18:29:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.638+0000 7f9487209700 1 -- 172.21.15.146:0/1071139269 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94800c9e00 con 0x7f94800721f0 2024-09-13T18:29:13.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.642+0000 7f9476ffd700 1 -- 172.21.15.146:0/1071139269 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f947c0075a0 con 0x7f94800721f0 2024-09-13T18:29:13.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.642+0000 7f9476ffd700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9464066730 0x7f9464068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:13.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.642+0000 7f9476ffd700 1 -- 172.21.15.146:0/1071139269 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f947c0837e0 con 0x7f94800721f0 2024-09-13T18:29:13.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.642+0000 7f9485a06700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9464066730 0x7f9464068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:13.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.642+0000 7f9485a06700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9464066730 0x7f9464068bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9470000c00 tx=0x7f9470008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:13.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.646+0000 7f9476ffd700 1 -- 172.21.15.146:0/1071139269 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f947c050880 con 0x7f94800721f0 2024-09-13T18:29:13.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.850+0000 7f9487209700 1 -- 172.21.15.146:0/1071139269 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94800ca8a0 con 0x7f94800721f0 2024-09-13T18:29:13.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.850+0000 7f9476ffd700 1 -- 172.21.15.146:0/1071139269 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f947c017090 con 0x7f94800721f0 2024-09-13T18:29:13.860 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:13.860 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5235,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:36.053759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:active","state_seq":36,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:13.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.854+0000 7f9487209700 1 -- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9464066730 msgr2=0x7f9464068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:13.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.854+0000 7f9487209700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9464066730 0x7f9464068bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9470000c00 tx=0x7f9470008030 comp rx=0 tx=0).stop 2024-09-13T18:29:13.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.854+0000 7f9487209700 1 -- 172.21.15.146:0/1071139269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94800721f0 msgr2=0x7f94800afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:13.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.854+0000 7f9487209700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94800721f0 0x7f94800afa80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f947c00c4d0 tx=0x7f947c00c7e0 comp rx=0 tx=0).stop 2024-09-13T18:29:13.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.854+0000 7f9487209700 1 -- 172.21.15.146:0/1071139269 shutdown_connections 2024-09-13T18:29:13.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.854+0000 7f9487209700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9464066730 0x7f9464068bf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:13.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.854+0000 7f9487209700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9480154a40 0x7f94801593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:13.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.854+0000 7f9487209700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94800cac60 0x7f948015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:13.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.854+0000 7f9487209700 1 --2- 172.21.15.146:0/1071139269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94800721f0 0x7f94800afa80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:13.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.854+0000 7f9487209700 1 -- 172.21.15.146:0/1071139269 >> 172.21.15.146:0/1071139269 conn(0x7f9480066090 msgr2=0x7f9480155f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:13.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.854+0000 7f9487209700 1 -- 172.21.15.146:0/1071139269 shutdown_connections 2024-09-13T18:29:13.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:13.854+0000 7f9487209700 1 -- 172.21.15.146:0/1071139269 wait complete. 2024-09-13T18:29:13.864 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 21 2024-09-13T18:29:13.874 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:29:15.876 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:16.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.990+0000 7f1ec22ef700 1 Processor -- start 2024-09-13T18:29:16.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec22ef700 1 -- start start 2024-09-13T18:29:16.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec22ef700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1ebc0c7310 0x7f1ebc161470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:16.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec22ef700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc0c8c70 con 0x7f1ebc0c5fc0 2024-09-13T18:29:16.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec22ef700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc0c8f50 con 0x7f1ebc0c63a0 2024-09-13T18:29:16.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec22ef700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc0c9bd0 con 0x7f1ebc0c7310 2024-09-13T18:29:16.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec12ed700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1ebc0c5fc0 0x7f1ebc0c6c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43916/0 (socket says 172.21.15.146:43916) 2024-09-13T18:29:16.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec12ed700 1 -- 172.21.15.146:0/4037889212 learned_addr learned my addr 172.21.15.146:0/4037889212 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:29:16.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec1aee700 1 --2- 172.21.15.146:0/4037889212 >> v2:172.21.15.154:3300/0 conn(0x7f1ebc0c7310 0x7f1ebc161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:16.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622185160 0 0) 0x7f1ebc0c8c70 con 0x7f1ebc0c5fc0 2024-09-13T18:29:16.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea80034f0 con 0x7f1ebc0c5fc0 2024-09-13T18:29:16.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2808253428 0 0) 0x7f1ea80034f0 con 0x7f1ebc0c5fc0 2024-09-13T18:29:16.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 >> v2:172.21.15.154:3300/0 conn(0x7f1ebc0c7310 msgr2=0x7f1ebc161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:16.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 --2- 172.21.15.146:0/4037889212 >> v2:172.21.15.154:3300/0 conn(0x7f1ebc0c7310 0x7f1ebc161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:16.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 >> v1:172.21.15.154:6789/0 conn(0x7f1ebc0c63a0 legacy=0x7f1ebc159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:16.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc0c9230 con 0x7f1ebc0c5fc0 2024-09-13T18:29:16.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f1eb8001670 con 0x7f1ebc0c5fc0 2024-09-13T18:29:16.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 >> v1:172.21.15.146:6789/0 conn(0x7f1ebc0c5fc0 legacy=0x7f1ebc0c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:16.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 --2- 172.21.15.146:0/4037889212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8003b80 0x7f1ea8003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:16.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 --2- 172.21.15.146:0/4037889212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ea8008950 0x7f1ea800ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:16.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 --2- 172.21.15.146:0/4037889212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ea800b5a0 0x7f1ea800da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:16.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0c9bd0 con 0x7f1ea8003b80 2024-09-13T18:29:16.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea800e340 con 0x7f1ea800b5a0 2024-09-13T18:29:16.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea800e500 con 0x7f1ea8008950 2024-09-13T18:29:16.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec12ed700 1 --2- 172.21.15.146:0/4037889212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8003b80 0x7f1ea8003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:16.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec0aec700 1 --2- 172.21.15.146:0/4037889212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ea8008950 0x7f1ea800ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:16.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec1aee700 1 --2- 172.21.15.146:0/4037889212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ea800b5a0 0x7f1ea800da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:16.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec12ed700 1 -- 172.21.15.146:0/4037889212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ea8008950 msgr2=0x7f1ea800ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:16.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec12ed700 1 --2- 172.21.15.146:0/4037889212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ea8008950 0x7f1ea800ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:16.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec12ed700 1 -- 172.21.15.146:0/4037889212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ea800b5a0 msgr2=0x7f1ea800da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:16.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec12ed700 1 --2- 172.21.15.146:0/4037889212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ea800b5a0 0x7f1ea800da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:16.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec1aee700 1 -- 172.21.15.146:0/4037889212 reap_dead start 2024-09-13T18:29:16.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec12ed700 1 -- 172.21.15.146:0/4037889212 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea80034f0 con 0x7f1ea8003b80 2024-09-13T18:29:16.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec0aec700 1 --2- 172.21.15.146:0/4037889212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ea8008950 0x7f1ea800ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:29:16.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1ec12ed700 1 --2- 172.21.15.146:0/4037889212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8003b80 0x7f1ea8003fa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800c320 tx=0x7f1ea80037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8f856930c5969 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:16.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8007c30 con 0x7f1ea8003b80 2024-09-13T18:29:16.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb800cde0 con 0x7f1ea8003b80 2024-09-13T18:29:16.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.994+0000 7f1eb3fff700 1 -- 172.21.15.146:0/4037889212 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8005ba0 con 0x7f1ea8003b80 2024-09-13T18:29:16.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.998+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4037889212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8003b80 msgr2=0x7f1ea8003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:16.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.998+0000 7f1ec22ef700 1 --2- 172.21.15.146:0/4037889212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8003b80 0x7f1ea8003fa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800c320 tx=0x7f1ea80037a0 comp rx=0 tx=0).stop 2024-09-13T18:29:16.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.998+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4037889212 shutdown_connections 2024-09-13T18:29:16.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.998+0000 7f1ec22ef700 1 --2- 172.21.15.146:0/4037889212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8003b80 0x7f1ea8003fa0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:16.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.998+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4037889212 >> 172.21.15.146:0/4037889212 conn(0x7f1ebc066090 msgr2=0x7f1ebc066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:16.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.998+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4037889212 shutdown_connections 2024-09-13T18:29:16.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.998+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4037889212 wait complete. 2024-09-13T18:29:16.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.998+0000 7f1ec22ef700 1 Processor -- start 2024-09-13T18:29:16.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:15.998+0000 7f1ec22ef700 1 -- start start 2024-09-13T18:29:16.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec22ef700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8003b80 0x7f1ebc06c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:16.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec22ef700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ebc1523b0 0x7f1ebc06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:16.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec22ef700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ebc153400 0x7f1ebc1ee260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:16.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec22ef700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc1de350 con 0x7f1ea8003b80 2024-09-13T18:29:16.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec22ef700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc14f680 con 0x7f1ebc153400 2024-09-13T18:29:16.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec22ef700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc14f3a0 con 0x7f1ebc1523b0 2024-09-13T18:29:16.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec12ed700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8003b80 0x7f1ebc06c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:16.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec12ed700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8003b80 0x7f1ebc06c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53292/0 (socket says 172.21.15.146:53292) 2024-09-13T18:29:16.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec12ed700 1 -- 172.21.15.146:0/4243999462 learned_addr learned my addr 172.21.15.146:0/4243999462 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:16.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec0aec700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ebc1523b0 0x7f1ebc06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:16.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec1aee700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ebc153400 0x7f1ebc1ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:16.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec12ed700 1 -- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ebc1523b0 msgr2=0x7f1ebc06cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:16.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec12ed700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ebc1523b0 0x7f1ebc06cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:16.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec12ed700 1 -- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ebc153400 msgr2=0x7f1ebc1ee260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:16.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec12ed700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ebc153400 0x7f1ebc1ee260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:16.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec12ed700 1 -- 172.21.15.146:0/4243999462 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc0c1610 con 0x7f1ea8003b80 2024-09-13T18:29:16.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec12ed700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8003b80 0x7f1ebc06c8a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800c910 tx=0x7f1eb8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:16.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1eb1ffb700 1 -- 172.21.15.146:0/4243999462 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8001640 con 0x7f1ea8003b80 2024-09-13T18:29:16.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1eb1ffb700 1 -- 172.21.15.146:0/4243999462 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb8004890 con 0x7f1ea8003b80 2024-09-13T18:29:16.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4243999462 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc0c18f0 con 0x7f1ea8003b80 2024-09-13T18:29:16.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1eb1ffb700 1 -- 172.21.15.146:0/4243999462 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8004c60 con 0x7f1ea8003b80 2024-09-13T18:29:16.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.002+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4243999462 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc06df80 con 0x7f1ea8003b80 2024-09-13T18:29:16.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.006+0000 7f1eb1ffb700 1 -- 172.21.15.146:0/4243999462 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1eb8019050 con 0x7f1ea8003b80 2024-09-13T18:29:16.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.006+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4243999462 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc0cb790 con 0x7f1ea8003b80 2024-09-13T18:29:16.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.006+0000 7f1eb1ffb700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1ea0066760 0x7f1ea0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:16.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.006+0000 7f1eb1ffb700 1 -- 172.21.15.146:0/4243999462 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f1eb8084cd0 con 0x7f1ea8003b80 2024-09-13T18:29:16.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.006+0000 7f1ec0aec700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1ea0066760 0x7f1ea0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:16.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.006+0000 7f1ec0aec700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1ea0066760 0x7f1ea0068c20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc14fa50 tx=0x7f1eac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:16.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.018+0000 7f1eb1ffb700 1 -- 172.21.15.146:0/4243999462 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1eb8051ba0 con 0x7f1ea8003b80 2024-09-13T18:29:16.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.222+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4243999462 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ebc1de350 con 0x7f1ea8003b80 2024-09-13T18:29:16.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.226+0000 7f1eb1ffb700 1 -- 172.21.15.146:0/4243999462 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f1eb8084090 con 0x7f1ea8003b80 2024-09-13T18:29:16.233 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:16.233 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5235,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:36.053759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:active","state_seq":36,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:16.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.226+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1ea0066760 msgr2=0x7f1ea0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:16.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.226+0000 7f1ec22ef700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1ea0066760 0x7f1ea0068c20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc14fa50 tx=0x7f1eac009040 comp rx=0 tx=0).stop 2024-09-13T18:29:16.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.226+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4243999462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8003b80 msgr2=0x7f1ebc06c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:16.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.226+0000 7f1ec22ef700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8003b80 0x7f1ebc06c8a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800c910 tx=0x7f1eb8001a00 comp rx=0 tx=0).stop 2024-09-13T18:29:16.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.226+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4243999462 shutdown_connections 2024-09-13T18:29:16.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.226+0000 7f1ec22ef700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1ea0066760 0x7f1ea0068c20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:16.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.226+0000 7f1ec22ef700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ebc153400 0x7f1ebc1ee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:16.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.226+0000 7f1ec22ef700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ebc1523b0 0x7f1ebc06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:16.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.226+0000 7f1ec22ef700 1 --2- 172.21.15.146:0/4243999462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8003b80 0x7f1ebc06c8a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:16.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.226+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4243999462 >> 172.21.15.146:0/4243999462 conn(0x7f1ebc066090 msgr2=0x7f1ebc155c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:16.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.226+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4243999462 shutdown_connections 2024-09-13T18:29:16.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:16.226+0000 7f1ec22ef700 1 -- 172.21.15.146:0/4243999462 wait complete. 2024-09-13T18:29:16.238 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 21 2024-09-13T18:29:16.248 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:29:18.249 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:18.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.362+0000 7f41f6d87700 1 Processor -- start 2024-09-13T18:29:18.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.362+0000 7f41f6d87700 1 -- start start 2024-09-13T18:29:18.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6d87700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f41f00721f0 0x7f41f0157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:18.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6d87700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f00cac60 0x7f41f00cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:18.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6d87700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f00c8c70 con 0x7f41f0154a40 2024-09-13T18:29:18.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6d87700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f0072810 con 0x7f41f00721f0 2024-09-13T18:29:18.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6d87700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f0157910 con 0x7f41f00cac60 2024-09-13T18:29:18.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6586700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f00cac60 0x7f41f00cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:18.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f5584700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f41f00721f0 0x7f41f0157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:18.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f5584700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f41f00721f0 0x7f41f0157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:42992/0 (socket says 172.21.15.146:42992) 2024-09-13T18:29:18.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f5584700 1 -- 172.21.15.146:0/989744709 learned_addr learned my addr 172.21.15.146:0/989744709 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:18.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f5d85700 1 --1- 172.21.15.146:0/989744709 >> v1:172.21.15.154:6789/0 conn(0x7f41f0154a40 0x7f41f006f7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:56368/0 (socket says 172.21.15.146:56368) 2024-09-13T18:29:18.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6586700 1 -- 172.21.15.146:0/989744709 >> v2:172.21.15.154:3300/0 conn(0x7f41f00721f0 msgr2=0x7f41f0157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:18.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6586700 1 --2- 172.21.15.146:0/989744709 >> v2:172.21.15.154:3300/0 conn(0x7f41f00721f0 0x7f41f0157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:18.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6586700 1 -- 172.21.15.146:0/989744709 >> v1:172.21.15.154:6789/0 conn(0x7f41f0154a40 legacy=0x7f41f006f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:18.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6586700 1 -- 172.21.15.146:0/989744709 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f00c8f50 con 0x7f41f00cac60 2024-09-13T18:29:18.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f5584700 1 --2- 172.21.15.146:0/989744709 >> v2:172.21.15.154:3300/0 conn(0x7f41f00721f0 0x7f41f0157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:29:18.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6586700 1 --2- 172.21.15.146:0/989744709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f00cac60 0x7f41f00cb0f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009d90 tx=0x7f41e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89f69e919470a39e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:18.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f4d83700 1 -- 172.21.15.146:0/989744709 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e800e070 con 0x7f41f00cac60 2024-09-13T18:29:18.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f4d83700 1 -- 172.21.15.146:0/989744709 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e800bb20 con 0x7f41f00cac60 2024-09-13T18:29:18.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f4d83700 1 -- 172.21.15.146:0/989744709 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8012430 con 0x7f41f00cac60 2024-09-13T18:29:18.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6d87700 1 -- 172.21.15.146:0/989744709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f00cac60 msgr2=0x7f41f00cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:18.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6d87700 1 --2- 172.21.15.146:0/989744709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f00cac60 0x7f41f00cb0f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009d90 tx=0x7f41e800b330 comp rx=0 tx=0).stop 2024-09-13T18:29:18.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6d87700 1 -- 172.21.15.146:0/989744709 shutdown_connections 2024-09-13T18:29:18.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6d87700 1 --2- 172.21.15.146:0/989744709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f00cac60 0x7f41f00cb0f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:18.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6d87700 1 --2- 172.21.15.146:0/989744709 >> v2:172.21.15.154:3300/0 conn(0x7f41f00721f0 0x7f41f0157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:18.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6d87700 1 -- 172.21.15.146:0/989744709 >> 172.21.15.146:0/989744709 conn(0x7f41f0066090 msgr2=0x7f41f0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:18.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6d87700 1 -- 172.21.15.146:0/989744709 shutdown_connections 2024-09-13T18:29:18.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.366+0000 7f41f6d87700 1 -- 172.21.15.146:0/989744709 wait complete. 2024-09-13T18:29:18.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f6d87700 1 Processor -- start 2024-09-13T18:29:18.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f6d87700 1 -- start start 2024-09-13T18:29:18.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f6d87700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41f00721f0 0x7f41f0159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:18.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f6d87700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f00cac60 0x7f41f015a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:18.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f6d87700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41f0154a40 0x7f41f01f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:18.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f6d87700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f006eb00 con 0x7f41f00721f0 2024-09-13T18:29:18.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f6d87700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f006e610 con 0x7f41f0154a40 2024-09-13T18:29:18.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f6d87700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f006e820 con 0x7f41f00cac60 2024-09-13T18:29:18.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f5d85700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41f00721f0 0x7f41f0159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:18.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f5d85700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41f00721f0 0x7f41f0159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53296/0 (socket says 172.21.15.146:53296) 2024-09-13T18:29:18.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f5d85700 1 -- 172.21.15.146:0/3104267392 learned_addr learned my addr 172.21.15.146:0/3104267392 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f5584700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f00cac60 0x7f41f015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f6586700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41f0154a40 0x7f41f01f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f5d85700 1 -- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f00cac60 msgr2=0x7f41f015a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f5d85700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f00cac60 0x7f41f015a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f5d85700 1 -- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41f0154a40 msgr2=0x7f41f01f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f5d85700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41f0154a40 0x7f41f01f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f5d85700 1 -- 172.21.15.146:0/3104267392 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec009570 con 0x7f41f00721f0 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f6586700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41f0154a40 0x7f41f01f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f5d85700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41f00721f0 0x7f41f0159cf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c410 tx=0x7f41e400c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41e27fc700 1 -- 172.21.15.146:0/3104267392 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4005cd0 con 0x7f41f00721f0 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41e27fc700 1 -- 172.21.15.146:0/3104267392 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e4005e30 con 0x7f41f00721f0 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41e27fc700 1 -- 172.21.15.146:0/3104267392 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e40072f0 con 0x7f41f00721f0 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f6d87700 1 -- 172.21.15.146:0/3104267392 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e8009a70 con 0x7f41f00721f0 2024-09-13T18:29:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.370+0000 7f41f6d87700 1 -- 172.21.15.146:0/3104267392 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f00c2550 con 0x7f41f00721f0 2024-09-13T18:29:18.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.374+0000 7f41f6d87700 1 -- 172.21.15.146:0/3104267392 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f00c9e00 con 0x7f41f00721f0 2024-09-13T18:29:18.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.374+0000 7f41e27fc700 1 -- 172.21.15.146:0/3104267392 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f41e4007450 con 0x7f41f00721f0 2024-09-13T18:29:18.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.374+0000 7f41e27fc700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f41d4066730 0x7f41d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:18.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.374+0000 7f41e27fc700 1 -- 172.21.15.146:0/3104267392 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f41e40839f0 con 0x7f41f00721f0 2024-09-13T18:29:18.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.374+0000 7f41f5584700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f41d4066730 0x7f41d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:18.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.374+0000 7f41f5584700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f41d4066730 0x7f41d4068bf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f41f01596a0 tx=0x7f41ec0092e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:18.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.382+0000 7f41e27fc700 1 -- 172.21.15.146:0/3104267392 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f41e4050840 con 0x7f41f00721f0 2024-09-13T18:29:18.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.594+0000 7f41f6d87700 1 -- 172.21.15.146:0/3104267392 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41f00c20d0 con 0x7f41f00721f0 2024-09-13T18:29:18.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.598+0000 7f41e27fc700 1 -- 172.21.15.146:0/3104267392 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f41f00c20d0 con 0x7f41f00721f0 2024-09-13T18:29:18.604 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:18.605 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5235,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:36.053759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:active","state_seq":36,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:18.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.598+0000 7f41f6d87700 1 -- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f41d4066730 msgr2=0x7f41d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:18.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.598+0000 7f41f6d87700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f41d4066730 0x7f41d4068bf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f41f01596a0 tx=0x7f41ec0092e0 comp rx=0 tx=0).stop 2024-09-13T18:29:18.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.598+0000 7f41f6d87700 1 -- 172.21.15.146:0/3104267392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41f00721f0 msgr2=0x7f41f0159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:18.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.598+0000 7f41f6d87700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41f00721f0 0x7f41f0159cf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c410 tx=0x7f41e400c720 comp rx=0 tx=0).stop 2024-09-13T18:29:18.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.598+0000 7f41f6d87700 1 -- 172.21.15.146:0/3104267392 shutdown_connections 2024-09-13T18:29:18.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.598+0000 7f41f6d87700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f41d4066730 0x7f41d4068bf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:18.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.598+0000 7f41f6d87700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41f0154a40 0x7f41f01f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:18.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.598+0000 7f41f6d87700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f00cac60 0x7f41f015a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:18.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.598+0000 7f41f6d87700 1 --2- 172.21.15.146:0/3104267392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41f00721f0 0x7f41f0159cf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:18.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.598+0000 7f41f6d87700 1 -- 172.21.15.146:0/3104267392 >> 172.21.15.146:0/3104267392 conn(0x7f41f0066090 msgr2=0x7f41f0152c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:18.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.598+0000 7f41f6d87700 1 -- 172.21.15.146:0/3104267392 shutdown_connections 2024-09-13T18:29:18.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:18.598+0000 7f41f6d87700 1 -- 172.21.15.146:0/3104267392 wait complete. 2024-09-13T18:29:18.608 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 21 2024-09-13T18:29:18.619 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:29:20.620 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:20.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.726+0000 7f2e3262b700 1 Processor -- start 2024-09-13T18:29:20.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.726+0000 7f2e3262b700 1 -- start start 2024-09-13T18:29:20.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e3262b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2e2c0721f0 0x7f2e2c157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:20.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e3262b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e2c0cac60 0x7f2e2c0cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:20.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e3262b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2c0c8c70 con 0x7f2e2c154a40 2024-09-13T18:29:20.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e3262b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2c072810 con 0x7f2e2c0721f0 2024-09-13T18:29:20.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e3262b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c157910 con 0x7f2e2c0cac60 2024-09-13T18:29:20.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e31629700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f2e2c154a40 0x7f2e2c06f7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43930/0 (socket says 172.21.15.146:43930) 2024-09-13T18:29:20.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e31629700 1 -- 172.21.15.146:0/3170897422 learned_addr learned my addr 172.21.15.146:0/3170897422 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:29:20.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e30e28700 1 --2- 172.21.15.146:0/3170897422 >> v2:172.21.15.154:3300/0 conn(0x7f2e2c0721f0 0x7f2e2c157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:20.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e31e2a700 1 --2- 172.21.15.146:0/3170897422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e2c0cac60 0x7f2e2c0cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:20.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e23fff700 1 -- 172.21.15.146:0/3170897422 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79817346 0 0) 0x7f2e2c0c8c70 con 0x7f2e2c154a40 2024-09-13T18:29:20.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e23fff700 1 -- 172.21.15.146:0/3170897422 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e180035d0 con 0x7f2e2c154a40 2024-09-13T18:29:20.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e30e28700 1 -- 172.21.15.146:0/3170897422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e2c0cac60 msgr2=0x7f2e2c0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:20.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e30e28700 1 --2- 172.21.15.146:0/3170897422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e2c0cac60 0x7f2e2c0cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:20.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e30e28700 1 -- 172.21.15.146:0/3170897422 >> v1:172.21.15.146:6789/0 conn(0x7f2e2c154a40 legacy=0x7f2e2c06f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:20.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e30e28700 1 -- 172.21.15.146:0/3170897422 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c0c8f50 con 0x7f2e2c0721f0 2024-09-13T18:29:20.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e31e2a700 1 --2- 172.21.15.146:0/3170897422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e2c0cac60 0x7f2e2c0cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:29:20.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e30e28700 1 --2- 172.21.15.146:0/3170897422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e2c0721f0 0x7f2e2c157d30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009850 tx=0x7f2e28009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2086c7425f26f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:20.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e23fff700 1 -- 172.21.15.146:0/3170897422 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28011830 con 0x7f2e2c0721f0 2024-09-13T18:29:20.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e23fff700 1 -- 172.21.15.146:0/3170897422 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e28011990 con 0x7f2e2c0721f0 2024-09-13T18:29:20.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e23fff700 1 -- 172.21.15.146:0/3170897422 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28011d00 con 0x7f2e2c0721f0 2024-09-13T18:29:20.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e3262b700 1 -- 172.21.15.146:0/3170897422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e2c0721f0 msgr2=0x7f2e2c157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:20.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e3262b700 1 --2- 172.21.15.146:0/3170897422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e2c0721f0 0x7f2e2c157d30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009850 tx=0x7f2e28009b60 comp rx=0 tx=0).stop 2024-09-13T18:29:20.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e3262b700 1 -- 172.21.15.146:0/3170897422 shutdown_connections 2024-09-13T18:29:20.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e3262b700 1 --2- 172.21.15.146:0/3170897422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e2c0cac60 0x7f2e2c0cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:20.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e3262b700 1 --2- 172.21.15.146:0/3170897422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e2c0721f0 0x7f2e2c157d30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:20.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e3262b700 1 -- 172.21.15.146:0/3170897422 >> 172.21.15.146:0/3170897422 conn(0x7f2e2c066090 msgr2=0x7f2e2c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:20.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e3262b700 1 -- 172.21.15.146:0/3170897422 shutdown_connections 2024-09-13T18:29:20.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.730+0000 7f2e3262b700 1 -- 172.21.15.146:0/3170897422 wait complete. 2024-09-13T18:29:20.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e3262b700 1 Processor -- start 2024-09-13T18:29:20.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e3262b700 1 -- start start 2024-09-13T18:29:20.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e3262b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e2c0721f0 0x7f2e2c0c2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:20.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e3262b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e2c0cac60 0x7f2e2c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:20.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e3262b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e2c154a40 0x7f2e2c148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:20.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e3262b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c153f90 con 0x7f2e2c0721f0 2024-09-13T18:29:20.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e3262b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0affb0 con 0x7f2e2c0cac60 2024-09-13T18:29:20.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e3262b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c153660 con 0x7f2e2c154a40 2024-09-13T18:29:20.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e31629700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e2c0721f0 0x7f2e2c0c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:20.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e31629700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e2c0721f0 0x7f2e2c0c2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53308/0 (socket says 172.21.15.146:53308) 2024-09-13T18:29:20.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e31629700 1 -- 172.21.15.146:0/2427615110 learned_addr learned my addr 172.21.15.146:0/2427615110 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:20.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e30e28700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e2c0cac60 0x7f2e2c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:20.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e31e2a700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e2c154a40 0x7f2e2c148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:20.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e31629700 1 -- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e2c154a40 msgr2=0x7f2e2c148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:20.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e31629700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e2c154a40 0x7f2e2c148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:20.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e31629700 1 -- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e2c0cac60 msgr2=0x7f2e2c06e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:20.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e31629700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e2c0cac60 0x7f2e2c06e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:20.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.734+0000 7f2e31629700 1 -- 172.21.15.146:0/2427615110 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e24009a70 con 0x7f2e2c0721f0 2024-09-13T18:29:20.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.738+0000 7f2e31629700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e2c0721f0 0x7f2e2c0c2090 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00c4f0 tx=0x7f2e1c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:20.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.738+0000 7f2e21ffb700 1 -- 172.21.15.146:0/2427615110 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c005d90 con 0x7f2e2c0721f0 2024-09-13T18:29:20.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.738+0000 7f2e21ffb700 1 -- 172.21.15.146:0/2427615110 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c0070d0 con 0x7f2e2c0721f0 2024-09-13T18:29:20.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.738+0000 7f2e3262b700 1 -- 172.21.15.146:0/2427615110 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e280094b0 con 0x7f2e2c0721f0 2024-09-13T18:29:20.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.738+0000 7f2e21ffb700 1 -- 172.21.15.146:0/2427615110 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c007440 con 0x7f2e2c0721f0 2024-09-13T18:29:20.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.738+0000 7f2e3262b700 1 -- 172.21.15.146:0/2427615110 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2c153c20 con 0x7f2e2c0721f0 2024-09-13T18:29:20.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.738+0000 7f2e3262b700 1 -- 172.21.15.146:0/2427615110 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2c0c9e00 con 0x7f2e2c0721f0 2024-09-13T18:29:20.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.738+0000 7f2e21ffb700 1 -- 172.21.15.146:0/2427615110 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2e1c0075a0 con 0x7f2e2c0721f0 2024-09-13T18:29:20.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.738+0000 7f2e21ffb700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2e10066730 0x7f2e10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:20.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.742+0000 7f2e21ffb700 1 -- 172.21.15.146:0/2427615110 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f2e1c083820 con 0x7f2e2c0721f0 2024-09-13T18:29:20.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.742+0000 7f2e30e28700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2e10066730 0x7f2e10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:20.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.742+0000 7f2e30e28700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2e10066730 0x7f2e10068bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2e280090b0 tx=0x7f2e2801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:20.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.750+0000 7f2e21ffb700 1 -- 172.21.15.146:0/2427615110 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2e1c050830 con 0x7f2e2c0721f0 2024-09-13T18:29:20.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.962+0000 7f2e3262b700 1 -- 172.21.15.146:0/2427615110 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e2c0c8c70 con 0x7f2e2c0721f0 2024-09-13T18:29:20.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.962+0000 7f2e21ffb700 1 -- 172.21.15.146:0/2427615110 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f2e1c017090 con 0x7f2e2c0721f0 2024-09-13T18:29:20.972 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:20.972 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5235,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:36.053759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:active","state_seq":36,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:20.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.966+0000 7f2e3262b700 1 -- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2e10066730 msgr2=0x7f2e10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:20.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.966+0000 7f2e3262b700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2e10066730 0x7f2e10068bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2e280090b0 tx=0x7f2e2801c040 comp rx=0 tx=0).stop 2024-09-13T18:29:20.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.966+0000 7f2e3262b700 1 -- 172.21.15.146:0/2427615110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e2c0721f0 msgr2=0x7f2e2c0c2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:20.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.966+0000 7f2e3262b700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e2c0721f0 0x7f2e2c0c2090 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00c4f0 tx=0x7f2e1c00c800 comp rx=0 tx=0).stop 2024-09-13T18:29:20.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.966+0000 7f2e3262b700 1 -- 172.21.15.146:0/2427615110 shutdown_connections 2024-09-13T18:29:20.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.966+0000 7f2e3262b700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2e10066730 0x7f2e10068bf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:20.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.966+0000 7f2e3262b700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e2c154a40 0x7f2e2c148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:20.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.966+0000 7f2e3262b700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e2c0cac60 0x7f2e2c06e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:20.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.966+0000 7f2e3262b700 1 --2- 172.21.15.146:0/2427615110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e2c0721f0 0x7f2e2c0c2090 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:20.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.966+0000 7f2e3262b700 1 -- 172.21.15.146:0/2427615110 >> 172.21.15.146:0/2427615110 conn(0x7f2e2c066090 msgr2=0x7f2e2c155370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:20.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.966+0000 7f2e3262b700 1 -- 172.21.15.146:0/2427615110 shutdown_connections 2024-09-13T18:29:20.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:20.966+0000 7f2e3262b700 1 -- 172.21.15.146:0/2427615110 wait complete. 2024-09-13T18:29:20.976 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 21 2024-09-13T18:29:20.987 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:29:22.988 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:23.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.102+0000 7f4ed9a13700 1 Processor -- start 2024-09-13T18:29:23.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.102+0000 7f4ed9a13700 1 -- start start 2024-09-13T18:29:23.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.102+0000 7f4ed9a13700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed40c5010 0x7f4ed40c6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.102+0000 7f4ed9a13700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ed40c53f0 0x7f4ed406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.102+0000 7f4ed9a13700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed4158040 con 0x7f4ed406da90 2024-09-13T18:29:23.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.102+0000 7f4ed9a13700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed40c7310 con 0x7f4ed40c53f0 2024-09-13T18:29:23.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.102+0000 7f4ed9a13700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed406d8f0 con 0x7f4ed40c5010 2024-09-13T18:29:23.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.102+0000 7f4ed27fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ed40c53f0 0x7f4ed406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.102+0000 7f4ed37fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4ed406da90 0x7f4ed414dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58162/0 (socket says 172.21.15.146:58162) 2024-09-13T18:29:23.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.102+0000 7f4ed37fe700 1 -- 172.21.15.146:0/1352941129 learned_addr learned my addr 172.21.15.146:0/1352941129 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:29:23.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.102+0000 7f4ed2ffd700 1 --2- 172.21.15.146:0/1352941129 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed40c5010 0x7f4ed40c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed27fc700 1 -- 172.21.15.146:0/1352941129 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed40c5010 msgr2=0x7f4ed40c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:23.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed27fc700 1 --2- 172.21.15.146:0/1352941129 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed40c5010 0x7f4ed40c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed27fc700 1 -- 172.21.15.146:0/1352941129 >> v1:172.21.15.146:6789/0 conn(0x7f4ed406da90 legacy=0x7f4ed414dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed27fc700 1 -- 172.21.15.146:0/1352941129 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed40665d0 con 0x7f4ed40c53f0 2024-09-13T18:29:23.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed27fc700 1 --2- 172.21.15.146:0/1352941129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ed40c53f0 0x7f4ed406d3b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80098b0 tx=0x7f4ec8009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2e7096690d0158 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:23.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed1ffb700 1 -- 172.21.15.146:0/1352941129 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8011830 con 0x7f4ed40c53f0 2024-09-13T18:29:23.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed1ffb700 1 -- 172.21.15.146:0/1352941129 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec8011990 con 0x7f4ed40c53f0 2024-09-13T18:29:23.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed1ffb700 1 -- 172.21.15.146:0/1352941129 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8011d20 con 0x7f4ed40c53f0 2024-09-13T18:29:23.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed9a13700 1 -- 172.21.15.146:0/1352941129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ed40c53f0 msgr2=0x7f4ed406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed9a13700 1 --2- 172.21.15.146:0/1352941129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ed40c53f0 0x7f4ed406d3b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80098b0 tx=0x7f4ec8009bc0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed9a13700 1 -- 172.21.15.146:0/1352941129 shutdown_connections 2024-09-13T18:29:23.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed9a13700 1 --2- 172.21.15.146:0/1352941129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ed40c53f0 0x7f4ed406d3b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed9a13700 1 --2- 172.21.15.146:0/1352941129 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed40c5010 0x7f4ed40c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed9a13700 1 -- 172.21.15.146:0/1352941129 >> 172.21.15.146:0/1352941129 conn(0x7f4ed4066920 msgr2=0x7f4ed406acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:23.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed9a13700 1 -- 172.21.15.146:0/1352941129 shutdown_connections 2024-09-13T18:29:23.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.106+0000 7f4ed9a13700 1 -- 172.21.15.146:0/1352941129 wait complete. 2024-09-13T18:29:23.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed9a13700 1 Processor -- start 2024-09-13T18:29:23.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed9a13700 1 -- start start 2024-09-13T18:29:23.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed9a13700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ed406da90 0x7f4ed41edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed9a13700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ed40c5010 0x7f4ed40cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed9a13700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed40c53f0 0x7f4ed41459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed9a13700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed414a6f0 con 0x7f4ed40c5010 2024-09-13T18:29:23.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed9a13700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41ee200 con 0x7f4ed406da90 2024-09-13T18:29:23.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed9a13700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed414e320 con 0x7f4ed40c53f0 2024-09-13T18:29:23.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed27fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ed40c5010 0x7f4ed40cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed27fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ed40c5010 0x7f4ed40cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53668/0 (socket says 172.21.15.146:53668) 2024-09-13T18:29:23.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed27fc700 1 -- 172.21.15.146:0/2901162134 learned_addr learned my addr 172.21.15.146:0/2901162134 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:23.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed2ffd700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ed406da90 0x7f4ed41edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed37fe700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed40c53f0 0x7f4ed41459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed27fc700 1 -- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed40c53f0 msgr2=0x7f4ed41459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed27fc700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed40c53f0 0x7f4ed41459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed27fc700 1 -- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ed406da90 msgr2=0x7f4ed41edcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed27fc700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ed406da90 0x7f4ed41edcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed27fc700 1 -- 172.21.15.146:0/2901162134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec80094b0 con 0x7f4ed40c5010 2024-09-13T18:29:23.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed2ffd700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ed406da90 0x7f4ed41edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:29:23.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed27fc700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ed40c5010 0x7f4ed40cbee0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8014040 tx=0x7f4ec8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:23.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ebbfff700 1 -- 172.21.15.146:0/2901162134 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80036c0 con 0x7f4ed40c5010 2024-09-13T18:29:23.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ebbfff700 1 -- 172.21.15.146:0/2901162134 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec801f070 con 0x7f4ed40c5010 2024-09-13T18:29:23.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed9a13700 1 -- 172.21.15.146:0/2901162134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed40cae20 con 0x7f4ed40c5010 2024-09-13T18:29:23.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ebbfff700 1 -- 172.21.15.146:0/2901162134 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8003a60 con 0x7f4ed40c5010 2024-09-13T18:29:23.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.110+0000 7f4ed9a13700 1 -- 172.21.15.146:0/2901162134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed40c9490 con 0x7f4ed40c5010 2024-09-13T18:29:23.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.114+0000 7f4ed9a13700 1 -- 172.21.15.146:0/2901162134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed40661b0 con 0x7f4ed40c5010 2024-09-13T18:29:23.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.114+0000 7f4ebbfff700 1 -- 172.21.15.146:0/2901162134 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4ec8003bc0 con 0x7f4ed40c5010 2024-09-13T18:29:23.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.114+0000 7f4ebbfff700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ec0066730 0x7f4ec0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.114+0000 7f4ebbfff700 1 -- 172.21.15.146:0/2901162134 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f4ec8021070 con 0x7f4ed40c5010 2024-09-13T18:29:23.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.114+0000 7f4ed2ffd700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ec0066730 0x7f4ec0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.114+0000 7f4ed2ffd700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ec0066730 0x7f4ec0068bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009a10 tx=0x7f4ec4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:23.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.118+0000 7f4ebbfff700 1 -- 172.21.15.146:0/2901162134 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4ec8052ed0 con 0x7f4ed40c5010 2024-09-13T18:29:23.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.330+0000 7f4ed9a13700 1 -- 172.21.15.146:0/2901162134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ed4074ed0 con 0x7f4ed40c5010 2024-09-13T18:29:23.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.330+0000 7f4ebbfff700 1 -- 172.21.15.146:0/2901162134 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f4ec802b020 con 0x7f4ed40c5010 2024-09-13T18:29:23.338 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:23.338 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5235,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:28:36.053759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":2,"incarnation":16,"state":"up:active","state_seq":36,"addr":"172.21.15.154:6839/3978828499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3978828499},{"type":"v1","addr":"172.21.15.154:6839","nonce":3978828499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:23.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.334+0000 7f4ed9a13700 1 -- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ec0066730 msgr2=0x7f4ec0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.334+0000 7f4ed9a13700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ec0066730 0x7f4ec0068bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009a10 tx=0x7f4ec4008040 comp rx=0 tx=0).stop 2024-09-13T18:29:23.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.334+0000 7f4ed9a13700 1 -- 172.21.15.146:0/2901162134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ed40c5010 msgr2=0x7f4ed40cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.334+0000 7f4ed9a13700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ed40c5010 0x7f4ed40cbee0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8014040 tx=0x7f4ec8004770 comp rx=0 tx=0).stop 2024-09-13T18:29:23.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.334+0000 7f4ed9a13700 1 -- 172.21.15.146:0/2901162134 shutdown_connections 2024-09-13T18:29:23.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.334+0000 7f4ed9a13700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ec0066730 0x7f4ec0068bf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.334+0000 7f4ed9a13700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed40c53f0 0x7f4ed41459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.334+0000 7f4ed9a13700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ed40c5010 0x7f4ed40cbee0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.334+0000 7f4ed9a13700 1 --2- 172.21.15.146:0/2901162134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ed406da90 0x7f4ed41edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.334+0000 7f4ed9a13700 1 -- 172.21.15.146:0/2901162134 >> 172.21.15.146:0/2901162134 conn(0x7f4ed4066920 msgr2=0x7f4ed414da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:23.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.334+0000 7f4ed9a13700 1 -- 172.21.15.146:0/2901162134 shutdown_connections 2024-09-13T18:29:23.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:23.334+0000 7f4ed9a13700 1 -- 172.21.15.146:0/2901162134 wait complete. 2024-09-13T18:29:23.342 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 21 2024-09-13T18:29:23.353 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:29:25.354 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:25.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f280999a700 1 Processor -- start 2024-09-13T18:29:25.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f280999a700 1 -- start start 2024-09-13T18:29:25.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f280999a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f28040cac60 0x7f28040cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f280999a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28040c8c70 con 0x7f28040721f0 2024-09-13T18:29:25.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f280999a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28040c8f50 con 0x7f2804154a40 2024-09-13T18:29:25.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f280999a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2804072810 con 0x7f28040cac60 2024-09-13T18:29:25.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f2809199700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f28040cac60 0x7f28040cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f2803fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f28040721f0 0x7f2804157d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58166/0 (socket says 172.21.15.146:58166) 2024-09-13T18:29:25.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f2803fff700 1 -- 172.21.15.146:0/2557249379 learned_addr learned my addr 172.21.15.146:0/2557249379 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:29:25.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f28037fe700 1 -- 172.21.15.146:0/2557249379 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807504180 0 0) 0x7f28040c8c70 con 0x7f28040721f0 2024-09-13T18:29:25.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f28037fe700 1 -- 172.21.15.146:0/2557249379 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27f40035d0 con 0x7f28040721f0 2024-09-13T18:29:25.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f2809199700 1 -- 172.21.15.146:0/2557249379 >> v1:172.21.15.154:6789/0 conn(0x7f2804154a40 legacy=0x7f280406f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f2809199700 1 -- 172.21.15.146:0/2557249379 >> v1:172.21.15.146:6789/0 conn(0x7f28040721f0 legacy=0x7f2804157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f2809199700 1 -- 172.21.15.146:0/2557249379 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2804157530 con 0x7f28040cac60 2024-09-13T18:29:25.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f2809199700 1 --2- 172.21.15.146:0/2557249379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28040cac60 0x7f28040cb0f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009d90 tx=0x7f27fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5538f438c79aac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f28037fe700 1 -- 172.21.15.146:0/2557249379 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc00e070 con 0x7f28040cac60 2024-09-13T18:29:25.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f28037fe700 1 -- 172.21.15.146:0/2557249379 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc00bb20 con 0x7f28040cac60 2024-09-13T18:29:25.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f28037fe700 1 -- 172.21.15.146:0/2557249379 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc012430 con 0x7f28040cac60 2024-09-13T18:29:25.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f280999a700 1 -- 172.21.15.146:0/2557249379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28040cac60 msgr2=0x7f28040cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f280999a700 1 --2- 172.21.15.146:0/2557249379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28040cac60 0x7f28040cb0f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009d90 tx=0x7f27fc00b330 comp rx=0 tx=0).stop 2024-09-13T18:29:25.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f280999a700 1 -- 172.21.15.146:0/2557249379 shutdown_connections 2024-09-13T18:29:25.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f280999a700 1 --2- 172.21.15.146:0/2557249379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28040cac60 0x7f28040cb0f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.470+0000 7f280999a700 1 -- 172.21.15.146:0/2557249379 >> 172.21.15.146:0/2557249379 conn(0x7f2804066090 msgr2=0x7f2804066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.474+0000 7f280999a700 1 -- 172.21.15.146:0/2557249379 shutdown_connections 2024-09-13T18:29:25.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.474+0000 7f280999a700 1 -- 172.21.15.146:0/2557249379 wait complete. 2024-09-13T18:29:25.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.474+0000 7f280999a700 1 Processor -- start 2024-09-13T18:29:25.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.474+0000 7f280999a700 1 -- start start 2024-09-13T18:29:25.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.474+0000 7f280999a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28040721f0 0x7f28041f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.474+0000 7f280999a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28040cac60 0x7f280406de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.474+0000 7f280999a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2804154a40 0x7f2804158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.474+0000 7f280999a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2804157910 con 0x7f28040cac60 2024-09-13T18:29:25.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.474+0000 7f280999a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2804151e40 con 0x7f2804154a40 2024-09-13T18:29:25.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.474+0000 7f280999a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040b02c0 con 0x7f28040721f0 2024-09-13T18:29:25.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.474+0000 7f2803fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28040cac60 0x7f280406de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f2803fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28040cac60 0x7f280406de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53696/0 (socket says 172.21.15.146:53696) 2024-09-13T18:29:25.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f2803fff700 1 -- 172.21.15.146:0/899176544 learned_addr learned my addr 172.21.15.146:0/899176544 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:25.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f2809199700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2804154a40 0x7f2804158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f2808998700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28040721f0 0x7f28041f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f2803fff700 1 -- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28040721f0 msgr2=0x7f28041f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f2803fff700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28040721f0 0x7f28041f2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f2803fff700 1 -- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2804154a40 msgr2=0x7f2804158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f2803fff700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2804154a40 0x7f2804158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f2803fff700 1 -- 172.21.15.146:0/899176544 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc009a70 con 0x7f28040cac60 2024-09-13T18:29:25.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f2803fff700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28040cac60 0x7f280406de00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f27f000c4d0 tx=0x7f27f000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f28017fa700 1 -- 172.21.15.146:0/899176544 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0005d90 con 0x7f28040cac60 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f28017fa700 1 -- 172.21.15.146:0/899176544 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f00070d0 con 0x7f28040cac60 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f280999a700 1 -- 172.21.15.146:0/899176544 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28041f25f0 con 0x7f28040cac60 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f28017fa700 1 -- 172.21.15.146:0/899176544 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0007480 con 0x7f28040cac60 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f280999a700 1 -- 172.21.15.146:0/899176544 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2804158620 con 0x7f28040cac60 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f280999a700 1 -- 172.21.15.146:0/899176544 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28041f2780 con 0x7f28040cac60 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f28017fa700 1 -- 172.21.15.146:0/899176544 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f27f00075e0 con 0x7f28040cac60 2024-09-13T18:29:25.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.478+0000 7f28017fa700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f27e8066730 0x7f27e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.482+0000 7f28017fa700 1 -- 172.21.15.146:0/899176544 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f27f0083a60 con 0x7f28040cac60 2024-09-13T18:29:25.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.482+0000 7f2808998700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f27e8066730 0x7f27e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.482+0000 7f2808998700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f27e8066730 0x7f27e8068bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f27f80016e0 tx=0x7f27f8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.486+0000 7f28017fa700 1 -- 172.21.15.146:0/899176544 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f27f0050a50 con 0x7f28040cac60 2024-09-13T18:29:25.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.698+0000 7f280999a700 1 -- 172.21.15.146:0/899176544 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28040c8c70 con 0x7f28040cac60 2024-09-13T18:29:25.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.698+0000 7f28017fa700 1 -- 172.21.15.146:0/899176544 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5527 (secure 0 0 0) 0x7f27f0054070 con 0x7f28040cac60 2024-09-13T18:29:25.708 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:25.708 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:25.579669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:25.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.702+0000 7f280999a700 1 -- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f27e8066730 msgr2=0x7f27e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.702+0000 7f280999a700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f27e8066730 0x7f27e8068bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f27f80016e0 tx=0x7f27f8009040 comp rx=0 tx=0).stop 2024-09-13T18:29:25.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.702+0000 7f280999a700 1 -- 172.21.15.146:0/899176544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28040cac60 msgr2=0x7f280406de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.702+0000 7f280999a700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28040cac60 0x7f280406de00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f27f000c4d0 tx=0x7f27f000c890 comp rx=0 tx=0).stop 2024-09-13T18:29:25.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.702+0000 7f280999a700 1 -- 172.21.15.146:0/899176544 shutdown_connections 2024-09-13T18:29:25.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.702+0000 7f280999a700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f27e8066730 0x7f27e8068bf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.702+0000 7f280999a700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2804154a40 0x7f2804158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.702+0000 7f280999a700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28040cac60 0x7f280406de00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.702+0000 7f280999a700 1 --2- 172.21.15.146:0/899176544 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28040721f0 0x7f28041f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.702+0000 7f280999a700 1 -- 172.21.15.146:0/899176544 >> 172.21.15.146:0/899176544 conn(0x7f2804066090 msgr2=0x7f2804155d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.702+0000 7f280999a700 1 -- 172.21.15.146:0/899176544 shutdown_connections 2024-09-13T18:29:25.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.702+0000 7f280999a700 1 -- 172.21.15.146:0/899176544 wait complete. 2024-09-13T18:29:25.712 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 22 2024-09-13T18:29:25.722 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2024-09-13T18:29:25.723 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:29:25.723 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:25.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf3fff700 1 Processor -- start 2024-09-13T18:29:25.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf3fff700 1 -- start start 2024-09-13T18:29:25.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4154a40 0x7efdf40721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf3fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf40c8c70 con 0x7efdf406f7d0 2024-09-13T18:29:25.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf3fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf40c8f50 con 0x7efdf40cac60 2024-09-13T18:29:25.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4072810 con 0x7efdf4154a40 2024-09-13T18:29:25.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf27fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7efdf406f7d0 0x7efdf4157d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58176/0 (socket says 172.21.15.146:58176) 2024-09-13T18:29:25.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf27fc700 1 -- 172.21.15.146:0/3326142043 learned_addr learned my addr 172.21.15.146:0/3326142043 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:29:25.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408138407 0 0) 0x7efdf40c8c70 con 0x7efdf406f7d0 2024-09-13T18:29:25.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efddc0035d0 con 0x7efdf406f7d0 2024-09-13T18:29:25.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf2ffd700 1 --2- 172.21.15.146:0/3326142043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4154a40 0x7efdf40721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1781994305 0 0) 0x7efddc0035d0 con 0x7efdf406f7d0 2024-09-13T18:29:25.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4154a40 msgr2=0x7efdf40721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 --2- 172.21.15.146:0/3326142043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4154a40 0x7efdf40721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 >> v1:172.21.15.154:6789/0 conn(0x7efdf40cac60 legacy=0x7efdf4161c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4157530 con 0x7efdf406f7d0 2024-09-13T18:29:25.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7efde4001670 con 0x7efdf406f7d0 2024-09-13T18:29:25.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 >> v1:172.21.15.146:6789/0 conn(0x7efdf406f7d0 legacy=0x7efdf4157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 --2- 172.21.15.146:0/3326142043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efddc003ac0 0x7efddc003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 --2- 172.21.15.146:0/3326142043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4154a40 0x7efddc004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 --2- 172.21.15.146:0/3326142043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efddc004db0 0x7efddc005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4072810 con 0x7efddc004db0 2024-09-13T18:29:25.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc00ff50 con 0x7efddc003ac0 2024-09-13T18:29:25.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc010110 con 0x7efdf4154a40 2024-09-13T18:29:25.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf37fe700 1 --2- 172.21.15.146:0/3326142043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efddc004db0 0x7efddc005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf27fc700 1 --2- 172.21.15.146:0/3326142043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4154a40 0x7efddc004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf2ffd700 1 --2- 172.21.15.146:0/3326142043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efddc003ac0 0x7efddc003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.826+0000 7efdf37fe700 1 -- 172.21.15.146:0/3326142043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4154a40 msgr2=0x7efddc004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf37fe700 1 --2- 172.21.15.146:0/3326142043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4154a40 0x7efddc004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf37fe700 1 -- 172.21.15.146:0/3326142043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efddc003ac0 msgr2=0x7efddc003ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf37fe700 1 --2- 172.21.15.146:0/3326142043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efddc003ac0 0x7efddc003ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf37fe700 1 -- 172.21.15.146:0/3326142043 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc0035d0 con 0x7efddc004db0 2024-09-13T18:29:25.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf37fe700 1 --2- 172.21.15.146:0/3326142043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efddc004db0 0x7efddc005230 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efdec00c160 tx=0x7efddc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4084ce129bd851a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec001db0 con 0x7efddc004db0 2024-09-13T18:29:25.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec00cca0 con 0x7efddc004db0 2024-09-13T18:29:25.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf1ffb700 1 -- 172.21.15.146:0/3326142043 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec003c60 con 0x7efddc004db0 2024-09-13T18:29:25.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 -- 172.21.15.146:0/3326142043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efddc004db0 msgr2=0x7efddc005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 --2- 172.21.15.146:0/3326142043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efddc004db0 0x7efddc005230 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efdec00c160 tx=0x7efddc0058a0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf37fe700 1 -- 172.21.15.146:0/3326142043 reap_dead start 2024-09-13T18:29:25.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 -- 172.21.15.146:0/3326142043 shutdown_connections 2024-09-13T18:29:25.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 -- 172.21.15.146:0/3326142043 >> 172.21.15.146:0/3326142043 conn(0x7efdf4066090 msgr2=0x7efdf4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 -- 172.21.15.146:0/3326142043 shutdown_connections 2024-09-13T18:29:25.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 -- 172.21.15.146:0/3326142043 wait complete. 2024-09-13T18:29:25.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 Processor -- start 2024-09-13T18:29:25.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 -- start start 2024-09-13T18:29:25.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4150b90 0x7efdf406c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdf406ca40 0x7efdf41561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf406e120 0x7efdf41566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4157910 con 0x7efdf406ca40 2024-09-13T18:29:25.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf41ef0d0 con 0x7efdf406e120 2024-09-13T18:29:25.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.830+0000 7efdf3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4159340 con 0x7efdf4150b90 2024-09-13T18:29:25.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf27fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdf406ca40 0x7efdf41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf27fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdf406ca40 0x7efdf41561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53726/0 (socket says 172.21.15.146:53726) 2024-09-13T18:29:25.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf27fc700 1 -- 172.21.15.146:0/3866855901 learned_addr learned my addr 172.21.15.146:0/3866855901 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:25.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf37fe700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf406e120 0x7efdf41566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf2ffd700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4150b90 0x7efdf406c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf27fc700 1 -- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4150b90 msgr2=0x7efdf406c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf27fc700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4150b90 0x7efdf406c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf27fc700 1 -- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf406e120 msgr2=0x7efdf41566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf27fc700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf406e120 0x7efdf41566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf27fc700 1 -- 172.21.15.146:0/3866855901 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf41ef4f0 con 0x7efdf406ca40 2024-09-13T18:29:25.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf2ffd700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4150b90 0x7efdf406c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:29:25.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf27fc700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdf406ca40 0x7efdf41561b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efde400e780 tx=0x7efde400ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efddb7fe700 1 -- 172.21.15.146:0/3866855901 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4012590 con 0x7efdf406ca40 2024-09-13T18:29:25.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efddb7fe700 1 -- 172.21.15.146:0/3866855901 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde40126f0 con 0x7efdf406ca40 2024-09-13T18:29:25.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf3fff700 1 -- 172.21.15.146:0/3866855901 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf4159060 con 0x7efdf406ca40 2024-09-13T18:29:25.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efddb7fe700 1 -- 172.21.15.146:0/3866855901 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4012aa0 con 0x7efdf406ca40 2024-09-13T18:29:25.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf3fff700 1 -- 172.21.15.146:0/3866855901 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf41eea10 con 0x7efdf406ca40 2024-09-13T18:29:25.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.834+0000 7efdf3fff700 1 -- 172.21.15.146:0/3866855901 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf41ef210 con 0x7efdf406ca40 2024-09-13T18:29:25.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.838+0000 7efddb7fe700 1 -- 172.21.15.146:0/3866855901 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efde4012c00 con 0x7efdf406ca40 2024-09-13T18:29:25.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.838+0000 7efddb7fe700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efde0066760 0x7efde0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.838+0000 7efddb7fe700 1 -- 172.21.15.146:0/3866855901 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7efde4084300 con 0x7efdf406ca40 2024-09-13T18:29:25.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.838+0000 7efdf2ffd700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efde0066760 0x7efde0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.838+0000 7efdf2ffd700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efde0066760 0x7efde0068c20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efde8009b80 tx=0x7efde8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:25.846+0000 7efddb7fe700 1 -- 172.21.15.146:0/3866855901 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efdf41ef210 con 0x7efdf406ca40 2024-09-13T18:29:26.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.050+0000 7efdf3fff700 1 -- 172.21.15.146:0/3866855901 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdf41ef210 con 0x7efdf406ca40 2024-09-13T18:29:26.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.054+0000 7efddb7fe700 1 -- 172.21.15.146:0/3866855901 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5527 (secure 0 0 0) 0x7efdf41ef210 con 0x7efdf406ca40 2024-09-13T18:29:26.061 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:26.061 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:25.579669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:26.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.058+0000 7efdf3fff700 1 -- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efde0066760 msgr2=0x7efde0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.058+0000 7efdf3fff700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efde0066760 0x7efde0068c20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efde8009b80 tx=0x7efde8008040 comp rx=0 tx=0).stop 2024-09-13T18:29:26.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.058+0000 7efdf3fff700 1 -- 172.21.15.146:0/3866855901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdf406ca40 msgr2=0x7efdf41561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.058+0000 7efdf3fff700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdf406ca40 0x7efdf41561b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efde400e780 tx=0x7efde400ea90 comp rx=0 tx=0).stop 2024-09-13T18:29:26.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.058+0000 7efdf3fff700 1 -- 172.21.15.146:0/3866855901 shutdown_connections 2024-09-13T18:29:26.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.058+0000 7efdf3fff700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efde0066760 0x7efde0068c20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.058+0000 7efdf3fff700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf406e120 0x7efdf41566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.058+0000 7efdf3fff700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdf406ca40 0x7efdf41561b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.058+0000 7efdf3fff700 1 --2- 172.21.15.146:0/3866855901 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf4150b90 0x7efdf406c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.058+0000 7efdf3fff700 1 -- 172.21.15.146:0/3866855901 >> 172.21.15.146:0/3866855901 conn(0x7efdf4066090 msgr2=0x7efdf40c53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:26.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.058+0000 7efdf3fff700 1 -- 172.21.15.146:0/3866855901 shutdown_connections 2024-09-13T18:29:26.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:26.058+0000 7efdf3fff700 1 -- 172.21.15.146:0/3866855901 wait complete. 2024-09-13T18:29:26.068 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 22 2024-09-13T18:29:26.083 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 22, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 22, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4281": { "addr": "172.21.15.146:6835/1642500214", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 1642500214, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 1642500214, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4281, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 11 }, "gid_4290": { "addr": "172.21.15.146:6837/31041557", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 31041557, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 31041557, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4290, "incarnation": 4, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_5235": { "addr": "172.21.15.146:6833/1498985282", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 1498985282, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 1498985282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5235, "incarnation": 22, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 29, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:29:25.579669+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4290, "mds_1": 4281, "mds_2": 5235 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:29:28.084 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:28.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9abfff700 1 Processor -- start 2024-09-13T18:29:28.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9abfff700 1 -- start start 2024-09-13T18:29:28.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9abfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc9ac154a40 0x7fc9ac0721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9abfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac0c8c70 con 0x7fc9ac0cac60 2024-09-13T18:29:28.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9abfff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac0c8f50 con 0x7fc9ac06f7d0 2024-09-13T18:29:28.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9abfff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ac072810 con 0x7fc9ac154a40 2024-09-13T18:29:28.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9ab7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc9ac0cac60 0x7fc9ac161c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58186/0 (socket says 172.21.15.146:58186) 2024-09-13T18:29:28.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9ab7fe700 1 -- 172.21.15.146:0/2807691882 learned_addr learned my addr 172.21.15.146:0/2807691882 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:29:28.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9aaffd700 1 --2- 172.21.15.146:0/2807691882 >> v2:172.21.15.154:3300/0 conn(0x7fc9ac154a40 0x7fc9ac0721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9a9ffb700 1 -- 172.21.15.146:0/2807691882 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903664300 0 0) 0x7fc9ac0c8c70 con 0x7fc9ac0cac60 2024-09-13T18:29:28.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9a9ffb700 1 -- 172.21.15.146:0/2807691882 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9980035d0 con 0x7fc9ac0cac60 2024-09-13T18:29:28.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9a9ffb700 1 -- 172.21.15.146:0/2807691882 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616403889 0 0) 0x7fc9ac0c8f50 con 0x7fc9ac06f7d0 2024-09-13T18:29:28.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9a9ffb700 1 -- 172.21.15.146:0/2807691882 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9ac0c8c70 con 0x7fc9ac06f7d0 2024-09-13T18:29:28.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9aaffd700 1 -- 172.21.15.146:0/2807691882 >> v1:172.21.15.154:6789/0 conn(0x7fc9ac06f7d0 legacy=0x7fc9ac157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9a9ffb700 1 -- 172.21.15.146:0/2807691882 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2454254541 0 0) 0x7fc9980035d0 con 0x7fc9ac0cac60 2024-09-13T18:29:28.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9aaffd700 1 -- 172.21.15.146:0/2807691882 >> v1:172.21.15.146:6789/0 conn(0x7fc9ac0cac60 legacy=0x7fc9ac161c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9aaffd700 1 -- 172.21.15.146:0/2807691882 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac157530 con 0x7fc9ac154a40 2024-09-13T18:29:28.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9aaffd700 1 --2- 172.21.15.146:0/2807691882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9ac154a40 0x7fc9ac0721f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009850 tx=0x7fc99c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36689cb1426cdaeb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.199+0000 7fc9a9ffb700 1 -- 172.21.15.146:0/2807691882 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4001670 con 0x7fc9ac154a40 2024-09-13T18:29:28.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.203+0000 7fc9a9ffb700 1 -- 172.21.15.146:0/2807691882 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c004030 con 0x7fc9ac154a40 2024-09-13T18:29:28.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.203+0000 7fc9a9ffb700 1 -- 172.21.15.146:0/2807691882 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0043a0 con 0x7fc9ac154a40 2024-09-13T18:29:28.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.203+0000 7fc9abfff700 1 -- 172.21.15.146:0/2807691882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9ac154a40 msgr2=0x7fc9ac0721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.203+0000 7fc9abfff700 1 --2- 172.21.15.146:0/2807691882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9ac154a40 0x7fc9ac0721f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009850 tx=0x7fc99c009b60 comp rx=0 tx=0).stop 2024-09-13T18:29:28.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.203+0000 7fc9abfff700 1 -- 172.21.15.146:0/2807691882 shutdown_connections 2024-09-13T18:29:28.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.203+0000 7fc9abfff700 1 --2- 172.21.15.146:0/2807691882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9ac154a40 0x7fc9ac0721f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.203+0000 7fc9abfff700 1 -- 172.21.15.146:0/2807691882 >> 172.21.15.146:0/2807691882 conn(0x7fc9ac066090 msgr2=0x7fc9ac066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.203+0000 7fc9abfff700 1 -- 172.21.15.146:0/2807691882 shutdown_connections 2024-09-13T18:29:28.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.203+0000 7fc9abfff700 1 -- 172.21.15.146:0/2807691882 wait complete. 2024-09-13T18:29:28.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.203+0000 7fc9abfff700 1 Processor -- start 2024-09-13T18:29:28.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.203+0000 7fc9abfff700 1 -- start start 2024-09-13T18:29:28.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9abfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9ac06f7d0 0x7fc9ac06d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9abfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9ac0cac60 0x7fc9ac14d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9abfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9ac154a40 0x7fc9ac0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9abfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac157910 con 0x7fc9ac154a40 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9abfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac06de60 con 0x7fc9ac06f7d0 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9abfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac06c950 con 0x7fc9ac0cac60 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9ab7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9ac154a40 0x7fc9ac0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9ab7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9ac154a40 0x7fc9ac0ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53738/0 (socket says 172.21.15.146:53738) 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9ab7fe700 1 -- 172.21.15.146:0/1742319346 learned_addr learned my addr 172.21.15.146:0/1742319346 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9aa7fc700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9ac0cac60 0x7fc9ac14d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9aaffd700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9ac06f7d0 0x7fc9ac06d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9ab7fe700 1 -- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9ac0cac60 msgr2=0x7fc9ac14d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9ab7fe700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9ac0cac60 0x7fc9ac14d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9ab7fe700 1 -- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9ac06f7d0 msgr2=0x7fc9ac06d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9ab7fe700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9ac06f7d0 0x7fc9ac06d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9ab7fe700 1 -- 172.21.15.146:0/1742319346 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c0094b0 con 0x7fc9ac154a40 2024-09-13T18:29:28.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9ab7fe700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9ac154a40 0x7fc9ac0ae200 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c520 tx=0x7fc9a400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9a37fe700 1 -- 172.21.15.146:0/1742319346 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4004ca0 con 0x7fc9ac154a40 2024-09-13T18:29:28.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9a37fe700 1 -- 172.21.15.146:0/1742319346 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a4004e00 con 0x7fc9ac154a40 2024-09-13T18:29:28.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9abfff700 1 -- 172.21.15.146:0/1742319346 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac0c1540 con 0x7fc9ac154a40 2024-09-13T18:29:28.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9a37fe700 1 -- 172.21.15.146:0/1742319346 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4007310 con 0x7fc9ac154a40 2024-09-13T18:29:28.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9abfff700 1 -- 172.21.15.146:0/1742319346 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac0c1b00 con 0x7fc9ac154a40 2024-09-13T18:29:28.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9abfff700 1 -- 172.21.15.146:0/1742319346 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac0c8f50 con 0x7fc9ac154a40 2024-09-13T18:29:28.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.207+0000 7fc9a37fe700 1 -- 172.21.15.146:0/1742319346 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc9a4007470 con 0x7fc9ac154a40 2024-09-13T18:29:28.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.211+0000 7fc9a37fe700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc98c066760 0x7fc98c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.211+0000 7fc9a37fe700 1 -- 172.21.15.146:0/1742319346 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fc9a4084300 con 0x7fc9ac154a40 2024-09-13T18:29:28.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.211+0000 7fc9aaffd700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc98c066760 0x7fc98c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.211+0000 7fc9aaffd700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc98c066760 0x7fc98c068c20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0090b0 tx=0x7fc99c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.215+0000 7fc9a37fe700 1 -- 172.21.15.146:0/1742319346 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc9a4051200 con 0x7fc9ac154a40 2024-09-13T18:29:28.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.423+0000 7fc9abfff700 1 -- 172.21.15.146:0/1742319346 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9ac0c1260 con 0x7fc9ac154a40 2024-09-13T18:29:28.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.423+0000 7fc9a37fe700 1 -- 172.21.15.146:0/1742319346 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5527 (secure 0 0 0) 0x7fc9a4007720 con 0x7fc9ac154a40 2024-09-13T18:29:28.431 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:28.431 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:25.579669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:28.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.427+0000 7fc9abfff700 1 -- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc98c066760 msgr2=0x7fc98c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.427+0000 7fc9abfff700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc98c066760 0x7fc98c068c20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0090b0 tx=0x7fc99c008030 comp rx=0 tx=0).stop 2024-09-13T18:29:28.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.427+0000 7fc9abfff700 1 -- 172.21.15.146:0/1742319346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9ac154a40 msgr2=0x7fc9ac0ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.427+0000 7fc9abfff700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9ac154a40 0x7fc9ac0ae200 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c520 tx=0x7fc9a400c8e0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.427+0000 7fc9abfff700 1 -- 172.21.15.146:0/1742319346 shutdown_connections 2024-09-13T18:29:28.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.427+0000 7fc9abfff700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc98c066760 0x7fc98c068c20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.427+0000 7fc9abfff700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9ac154a40 0x7fc9ac0ae200 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.427+0000 7fc9abfff700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9ac0cac60 0x7fc9ac14d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.427+0000 7fc9abfff700 1 --2- 172.21.15.146:0/1742319346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9ac06f7d0 0x7fc9ac06d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.427+0000 7fc9abfff700 1 -- 172.21.15.146:0/1742319346 >> 172.21.15.146:0/1742319346 conn(0x7fc9ac066090 msgr2=0x7fc9ac156720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.427+0000 7fc9abfff700 1 -- 172.21.15.146:0/1742319346 shutdown_connections 2024-09-13T18:29:28.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:28.427+0000 7fc9abfff700 1 -- 172.21.15.146:0/1742319346 wait complete. 2024-09-13T18:29:28.435 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 22 2024-09-13T18:29:28.446 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:29:30.447 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:30.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.563+0000 7f45e746b700 1 Processor -- start 2024-09-13T18:29:30.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.563+0000 7f45e746b700 1 -- start start 2024-09-13T18:29:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.563+0000 7f45e746b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45e00c63a0 0x7f45e0159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.563+0000 7f45e746b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e00c8c70 con 0x7f45e00c5fc0 2024-09-13T18:29:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.563+0000 7f45e746b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e00c8f50 con 0x7f45e00c7310 2024-09-13T18:29:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.563+0000 7f45e746b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00c9bd0 con 0x7f45e00c63a0 2024-09-13T18:29:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.563+0000 7f45e6469700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f45e00c5fc0 0x7f45e00c6c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58194/0 (socket says 172.21.15.146:58194) 2024-09-13T18:29:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.563+0000 7f45e6469700 1 -- 172.21.15.146:0/3612471863 learned_addr learned my addr 172.21.15.146:0/3612471863 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:29:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.563+0000 7f45e5c68700 1 --2- 172.21.15.146:0/3612471863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45e00c63a0 0x7f45e0159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e5467700 1 -- 172.21.15.146:0/3612471863 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946458349 0 0) 0x7f45e00c8c70 con 0x7f45e00c5fc0 2024-09-13T18:29:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e5467700 1 -- 172.21.15.146:0/3612471863 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45cc0035d0 con 0x7f45e00c5fc0 2024-09-13T18:29:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e5467700 1 -- 172.21.15.146:0/3612471863 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529670541 0 0) 0x7f45e00c8f50 con 0x7f45e00c7310 2024-09-13T18:29:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e5467700 1 -- 172.21.15.146:0/3612471863 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e00c8c70 con 0x7f45e00c7310 2024-09-13T18:29:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e5c68700 1 -- 172.21.15.146:0/3612471863 >> v1:172.21.15.154:6789/0 conn(0x7f45e00c7310 legacy=0x7f45e0160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e5467700 1 -- 172.21.15.146:0/3612471863 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 428798347 0 0) 0x7f45cc0035d0 con 0x7f45e00c5fc0 2024-09-13T18:29:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e5c68700 1 -- 172.21.15.146:0/3612471863 >> v1:172.21.15.146:6789/0 conn(0x7f45e00c5fc0 legacy=0x7f45e00c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e5c68700 1 -- 172.21.15.146:0/3612471863 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e00cacb0 con 0x7f45e00c63a0 2024-09-13T18:29:30.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e5c68700 1 --2- 172.21.15.146:0/3612471863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45e00c63a0 0x7f45e0159660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009dc0 tx=0x7f45dc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de33ee3c4def4767 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:30.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e5467700 1 -- 172.21.15.146:0/3612471863 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc00e070 con 0x7f45e00c63a0 2024-09-13T18:29:30.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e5467700 1 -- 172.21.15.146:0/3612471863 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc003a40 con 0x7f45e00c63a0 2024-09-13T18:29:30.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e5467700 1 -- 172.21.15.146:0/3612471863 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0001670 con 0x7f45e00c63a0 2024-09-13T18:29:30.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e746b700 1 -- 172.21.15.146:0/3612471863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45e00c63a0 msgr2=0x7f45e0159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e746b700 1 --2- 172.21.15.146:0/3612471863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45e00c63a0 0x7f45e0159660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009dc0 tx=0x7f45dc00b330 comp rx=0 tx=0).stop 2024-09-13T18:29:30.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e746b700 1 -- 172.21.15.146:0/3612471863 shutdown_connections 2024-09-13T18:29:30.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e746b700 1 --2- 172.21.15.146:0/3612471863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45e00c63a0 0x7f45e0159660 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e746b700 1 -- 172.21.15.146:0/3612471863 >> 172.21.15.146:0/3612471863 conn(0x7f45e0066090 msgr2=0x7f45e0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:30.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e746b700 1 -- 172.21.15.146:0/3612471863 shutdown_connections 2024-09-13T18:29:30.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.567+0000 7f45e746b700 1 -- 172.21.15.146:0/3612471863 wait complete. 2024-09-13T18:29:30.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e746b700 1 Processor -- start 2024-09-13T18:29:30.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e746b700 1 -- start start 2024-09-13T18:29:30.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e746b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45e00c5fc0 0x7f45e01e1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e746b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45e00c63a0 0x7f45e01df420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e746b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45e00c7310 0x7f45e014ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e746b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e01e0110 con 0x7f45e00c63a0 2024-09-13T18:29:30.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e746b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00ae9c0 con 0x7f45e00c7310 2024-09-13T18:29:30.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e746b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e01eeb80 con 0x7f45e00c5fc0 2024-09-13T18:29:30.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e5c68700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45e00c63a0 0x7f45e01df420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e5c68700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45e00c63a0 0x7f45e01df420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53752/0 (socket says 172.21.15.146:53752) 2024-09-13T18:29:30.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e5c68700 1 -- 172.21.15.146:0/1349751681 learned_addr learned my addr 172.21.15.146:0/1349751681 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:30.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e6469700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45e00c5fc0 0x7f45e01e1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e6c6a700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45e00c7310 0x7f45e014ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e5c68700 1 -- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45e00c5fc0 msgr2=0x7f45e01e1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e5c68700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45e00c5fc0 0x7f45e01e1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e5c68700 1 -- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45e00c7310 msgr2=0x7f45e014ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e5c68700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45e00c7310 0x7f45e014ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e5c68700 1 -- 172.21.15.146:0/1349751681 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc009a70 con 0x7f45e00c63a0 2024-09-13T18:29:30.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e5c68700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45e00c63a0 0x7f45e01df420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f45dc000f80 tx=0x7f45dc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:30.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45d6ffd700 1 -- 172.21.15.146:0/1349751681 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0046b0 con 0x7f45e00c63a0 2024-09-13T18:29:30.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45d6ffd700 1 -- 172.21.15.146:0/1349751681 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc004810 con 0x7f45e00c63a0 2024-09-13T18:29:30.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e746b700 1 -- 172.21.15.146:0/1349751681 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e00ae300 con 0x7f45e00c63a0 2024-09-13T18:29:30.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45d6ffd700 1 -- 172.21.15.146:0/1349751681 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc012650 con 0x7f45e00c63a0 2024-09-13T18:29:30.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.571+0000 7f45e746b700 1 -- 172.21.15.146:0/1349751681 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e01def00 con 0x7f45e00c63a0 2024-09-13T18:29:30.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.575+0000 7f45e746b700 1 -- 172.21.15.146:0/1349751681 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e00cb790 con 0x7f45e00c63a0 2024-09-13T18:29:30.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.575+0000 7f45d6ffd700 1 -- 172.21.15.146:0/1349751681 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f45dc0127b0 con 0x7f45e00c63a0 2024-09-13T18:29:30.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.575+0000 7f45d6ffd700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f45c4066730 0x7f45c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.575+0000 7f45d6ffd700 1 -- 172.21.15.146:0/1349751681 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f45dc085b40 con 0x7f45e00c63a0 2024-09-13T18:29:30.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.575+0000 7f45e6469700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f45c4066730 0x7f45c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.579+0000 7f45e6469700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f45c4066730 0x7f45c4068bf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f45d00058f0 tx=0x7f45d0001430 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:30.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.583+0000 7f45d6ffd700 1 -- 172.21.15.146:0/1349751681 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f45dc052b60 con 0x7f45e00c63a0 2024-09-13T18:29:30.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.795+0000 7f45e746b700 1 -- 172.21.15.146:0/1349751681 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45e00ae7c0 con 0x7f45e00c63a0 2024-09-13T18:29:30.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.795+0000 7f45d6ffd700 1 -- 172.21.15.146:0/1349751681 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5527 (secure 0 0 0) 0x7f45dc056180 con 0x7f45e00c63a0 2024-09-13T18:29:30.803 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:30.803 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:25.579669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:30.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.799+0000 7f45e746b700 1 -- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f45c4066730 msgr2=0x7f45c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.799+0000 7f45e746b700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f45c4066730 0x7f45c4068bf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f45d00058f0 tx=0x7f45d0001430 comp rx=0 tx=0).stop 2024-09-13T18:29:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.799+0000 7f45e746b700 1 -- 172.21.15.146:0/1349751681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45e00c63a0 msgr2=0x7f45e01df420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.799+0000 7f45e746b700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45e00c63a0 0x7f45e01df420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f45dc000f80 tx=0x7f45dc004af0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.799+0000 7f45e746b700 1 -- 172.21.15.146:0/1349751681 shutdown_connections 2024-09-13T18:29:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.799+0000 7f45e746b700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f45c4066730 0x7f45c4068bf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.799+0000 7f45e746b700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45e00c7310 0x7f45e014ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.799+0000 7f45e746b700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45e00c63a0 0x7f45e01df420 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.799+0000 7f45e746b700 1 --2- 172.21.15.146:0/1349751681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45e00c5fc0 0x7f45e01e1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.799+0000 7f45e746b700 1 -- 172.21.15.146:0/1349751681 >> 172.21.15.146:0/1349751681 conn(0x7f45e0066090 msgr2=0x7f45e00c6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.799+0000 7f45e746b700 1 -- 172.21.15.146:0/1349751681 shutdown_connections 2024-09-13T18:29:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:30.799+0000 7f45e746b700 1 -- 172.21.15.146:0/1349751681 wait complete. 2024-09-13T18:29:30.807 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 22 2024-09-13T18:29:30.817 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:29:32.818 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.931+0000 7efc94a6c700 1 Processor -- start 2024-09-13T18:29:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.931+0000 7efc94a6c700 1 -- start start 2024-09-13T18:29:32.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc94a6c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efc900c8f70 0x7efc900a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:32.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc94a6c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc900c9350 0x7efc900723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:32.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc94a6c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc90159f80 con 0x7efc900c7530 2024-09-13T18:29:32.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc94a6c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc900ca1d0 con 0x7efc900c8f70 2024-09-13T18:29:32.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc94a6c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc90159670 con 0x7efc900c9350 2024-09-13T18:29:32.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc8f7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efc900c8f70 0x7efc900a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:32.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc8f7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efc900c8f70 0x7efc900a9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55450/0 (socket says 172.21.15.146:55450) 2024-09-13T18:29:32.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc8f7fe700 1 -- 172.21.15.146:0/1166858425 learned_addr learned my addr 172.21.15.146:0/1166858425 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:32.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc87fff700 1 --2- 172.21.15.146:0/1166858425 >> v2:172.21.15.154:3300/0 conn(0x7efc900c9350 0x7efc900723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:32.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc8f7fe700 1 -- 172.21.15.146:0/1166858425 >> v2:172.21.15.154:3300/0 conn(0x7efc900c9350 msgr2=0x7efc900723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:32.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc8f7fe700 1 --2- 172.21.15.146:0/1166858425 >> v2:172.21.15.154:3300/0 conn(0x7efc900c9350 0x7efc900723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:32.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc8f7fe700 1 -- 172.21.15.146:0/1166858425 >> v1:172.21.15.154:6789/0 conn(0x7efc900c7530 legacy=0x7efc900cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:32.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc8f7fe700 1 -- 172.21.15.146:0/1166858425 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc900aa3a0 con 0x7efc900c8f70 2024-09-13T18:29:32.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc8f7fe700 1 --2- 172.21.15.146:0/1166858425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc900c8f70 0x7efc900a9cf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efc900c8810 tx=0x7efc78009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5203e4b1b41165 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:32.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc8effd700 1 -- 172.21.15.146:0/1166858425 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78011830 con 0x7efc900c8f70 2024-09-13T18:29:32.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc8effd700 1 -- 172.21.15.146:0/1166858425 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc78011990 con 0x7efc900c8f70 2024-09-13T18:29:32.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc8effd700 1 -- 172.21.15.146:0/1166858425 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78011d20 con 0x7efc900c8f70 2024-09-13T18:29:32.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc94a6c700 1 -- 172.21.15.146:0/1166858425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc900c8f70 msgr2=0x7efc900a9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:32.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc94a6c700 1 --2- 172.21.15.146:0/1166858425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc900c8f70 0x7efc900a9cf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efc900c8810 tx=0x7efc78009bf0 comp rx=0 tx=0).stop 2024-09-13T18:29:32.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc94a6c700 1 -- 172.21.15.146:0/1166858425 shutdown_connections 2024-09-13T18:29:32.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc94a6c700 1 --2- 172.21.15.146:0/1166858425 >> v2:172.21.15.154:3300/0 conn(0x7efc900c9350 0x7efc900723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:32.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc94a6c700 1 --2- 172.21.15.146:0/1166858425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc900c8f70 0x7efc900a9cf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:32.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc94a6c700 1 -- 172.21.15.146:0/1166858425 >> 172.21.15.146:0/1166858425 conn(0x7efc90066090 msgr2=0x7efc90066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:32.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc94a6c700 1 -- 172.21.15.146:0/1166858425 shutdown_connections 2024-09-13T18:29:32.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.935+0000 7efc94a6c700 1 -- 172.21.15.146:0/1166858425 wait complete. 2024-09-13T18:29:32.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc94a6c700 1 Processor -- start 2024-09-13T18:29:32.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc94a6c700 1 -- start start 2024-09-13T18:29:32.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc94a6c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc900c7530 0x7efc900ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:32.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc94a6c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc900c8f70 0x7efc9014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:32.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc94a6c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc900c9350 0x7efc90153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:32.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc94a6c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc900ae560 con 0x7efc900c7530 2024-09-13T18:29:32.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc94a6c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc900aef60 con 0x7efc900c8f70 2024-09-13T18:29:32.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc94a6c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc900af1c0 con 0x7efc900c9350 2024-09-13T18:29:32.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc900c7530 0x7efc900ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:32.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc900c7530 0x7efc900ad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55454/0 (socket says 172.21.15.146:55454) 2024-09-13T18:29:32.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8f7fe700 1 -- 172.21.15.146:0/3483349038 learned_addr learned my addr 172.21.15.146:0/3483349038 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:32.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc87fff700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc900c8f70 0x7efc9014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:32.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8ffff700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc900c9350 0x7efc90153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:32.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8f7fe700 1 -- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc900c9350 msgr2=0x7efc90153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:32.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8f7fe700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc900c9350 0x7efc90153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:32.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8f7fe700 1 -- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc900c8f70 msgr2=0x7efc9014f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:32.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8f7fe700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc900c8f70 0x7efc9014f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:32.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8f7fe700 1 -- 172.21.15.146:0/3483349038 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc78009570 con 0x7efc900c7530 2024-09-13T18:29:32.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8ffff700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc900c9350 0x7efc90153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:29:32.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8f7fe700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc900c7530 0x7efc900ad280 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efc900cac30 tx=0x7efc78004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:32.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8cff9700 1 -- 172.21.15.146:0/3483349038 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78004550 con 0x7efc900c7530 2024-09-13T18:29:32.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8cff9700 1 -- 172.21.15.146:0/3483349038 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7801f070 con 0x7efc900c7530 2024-09-13T18:29:32.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc8cff9700 1 -- 172.21.15.146:0/3483349038 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78003900 con 0x7efc900c7530 2024-09-13T18:29:32.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc94a6c700 1 -- 172.21.15.146:0/3483349038 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc900ae740 con 0x7efc900c7530 2024-09-13T18:29:32.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.939+0000 7efc94a6c700 1 -- 172.21.15.146:0/3483349038 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc90072440 con 0x7efc900c7530 2024-09-13T18:29:32.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.943+0000 7efc94a6c700 1 -- 172.21.15.146:0/3483349038 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc900af9c0 con 0x7efc900c7530 2024-09-13T18:29:32.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.943+0000 7efc8cff9700 1 -- 172.21.15.146:0/3483349038 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efc78003a60 con 0x7efc900c7530 2024-09-13T18:29:32.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.943+0000 7efc8cff9700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc70066730 0x7efc70068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:32.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.943+0000 7efc8cff9700 1 -- 172.21.15.146:0/3483349038 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7efc78021070 con 0x7efc900c7530 2024-09-13T18:29:32.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.943+0000 7efc87fff700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc70066730 0x7efc70068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:32.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.943+0000 7efc87fff700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc70066730 0x7efc70068bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efc9006fa10 tx=0x7efc80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:32.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:32.951+0000 7efc8cff9700 1 -- 172.21.15.146:0/3483349038 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efc900af9c0 con 0x7efc900c7530 2024-09-13T18:29:33.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.159+0000 7efc94a6c700 1 -- 172.21.15.146:0/3483349038 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc900af9c0 con 0x7efc900c7530 2024-09-13T18:29:33.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.159+0000 7efc8cff9700 1 -- 172.21.15.146:0/3483349038 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5529 (secure 0 0 0) 0x7efc900af9c0 con 0x7efc900c7530 2024-09-13T18:29:33.167 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:33.167 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:31.233532+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:resolve","state_seq":13,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:33.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.163+0000 7efc94a6c700 1 -- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc70066730 msgr2=0x7efc70068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:33.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.163+0000 7efc94a6c700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc70066730 0x7efc70068bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efc9006fa10 tx=0x7efc80008040 comp rx=0 tx=0).stop 2024-09-13T18:29:33.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.163+0000 7efc94a6c700 1 -- 172.21.15.146:0/3483349038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc900c7530 msgr2=0x7efc900ad280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:33.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.163+0000 7efc94a6c700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc900c7530 0x7efc900ad280 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efc900cac30 tx=0x7efc78004770 comp rx=0 tx=0).stop 2024-09-13T18:29:33.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.163+0000 7efc94a6c700 1 -- 172.21.15.146:0/3483349038 shutdown_connections 2024-09-13T18:29:33.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.163+0000 7efc94a6c700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc70066730 0x7efc70068bf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:33.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.163+0000 7efc94a6c700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc900c9350 0x7efc90153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:33.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.163+0000 7efc94a6c700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc900c8f70 0x7efc9014f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:33.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.163+0000 7efc94a6c700 1 --2- 172.21.15.146:0/3483349038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc900c7530 0x7efc900ad280 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:33.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.163+0000 7efc94a6c700 1 -- 172.21.15.146:0/3483349038 >> 172.21.15.146:0/3483349038 conn(0x7efc90066090 msgr2=0x7efc9006d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:33.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.163+0000 7efc94a6c700 1 -- 172.21.15.146:0/3483349038 shutdown_connections 2024-09-13T18:29:33.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:33.163+0000 7efc94a6c700 1 -- 172.21.15.146:0/3483349038 wait complete. 2024-09-13T18:29:33.173 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 23 2024-09-13T18:29:33.187 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:29:35.188 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:35.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdbaa285700 1 Processor -- start 2024-09-13T18:29:35.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdbaa285700 1 -- start start 2024-09-13T18:29:35.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdbaa285700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdba4153a60 0x7fdba4159700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:35.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdbaa285700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdba40cf540 0x7fdba40cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:35.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdbaa285700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba40cc2d0 con 0x7fdba40c9ed0 2024-09-13T18:29:35.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdbaa285700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba40b3990 con 0x7fdba40cf540 2024-09-13T18:29:35.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdbaa285700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba4153e40 con 0x7fdba4153a60 2024-09-13T18:29:35.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdba9283700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdba4153a60 0x7fdba4159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:35.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdba9a84700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdba40cf540 0x7fdba40cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:35.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdba9283700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdba4153a60 0x7fdba4159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:48084/0 (socket says 172.21.15.146:48084) 2024-09-13T18:29:35.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdba8a82700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fdba40c9ed0 0x7fdba40b3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:56146/0 (socket says 172.21.15.146:56146) 2024-09-13T18:29:35.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdba9283700 1 -- 172.21.15.146:0/1393905501 learned_addr learned my addr 172.21.15.146:0/1393905501 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:35.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdba9283700 1 -- 172.21.15.146:0/1393905501 >> v2:172.21.15.154:3300/0 conn(0x7fdba40cf540 msgr2=0x7fdba40cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:35.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdba9283700 1 --2- 172.21.15.146:0/1393905501 >> v2:172.21.15.154:3300/0 conn(0x7fdba40cf540 0x7fdba40cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:35.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdba9283700 1 -- 172.21.15.146:0/1393905501 >> v1:172.21.15.154:6789/0 conn(0x7fdba40c9ed0 legacy=0x7fdba40b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:35.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdba9283700 1 -- 172.21.15.146:0/1393905501 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba40cb3e0 con 0x7fdba4153a60 2024-09-13T18:29:35.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdba9283700 1 --2- 172.21.15.146:0/1393905501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdba4153a60 0x7fdba4159700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009e50 tx=0x7fdb8c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=429407dafed0a132 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:35.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdba3fff700 1 -- 172.21.15.146:0/1393905501 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00e070 con 0x7fdba4153a60 2024-09-13T18:29:35.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdba3fff700 1 -- 172.21.15.146:0/1393905501 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c00b910 con 0x7fdba4153a60 2024-09-13T18:29:35.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.307+0000 7fdba3fff700 1 -- 172.21.15.146:0/1393905501 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00bcc0 con 0x7fdba4153a60 2024-09-13T18:29:35.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 -- 172.21.15.146:0/1393905501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdba4153a60 msgr2=0x7fdba4159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:35.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 --2- 172.21.15.146:0/1393905501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdba4153a60 0x7fdba4159700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009e50 tx=0x7fdb8c00b330 comp rx=0 tx=0).stop 2024-09-13T18:29:35.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 -- 172.21.15.146:0/1393905501 shutdown_connections 2024-09-13T18:29:35.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 --2- 172.21.15.146:0/1393905501 >> v2:172.21.15.154:3300/0 conn(0x7fdba40cf540 0x7fdba40cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:35.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 --2- 172.21.15.146:0/1393905501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdba4153a60 0x7fdba4159700 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:35.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 -- 172.21.15.146:0/1393905501 >> 172.21.15.146:0/1393905501 conn(0x7fdba4071ec0 msgr2=0x7fdba40722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:35.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 -- 172.21.15.146:0/1393905501 shutdown_connections 2024-09-13T18:29:35.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 -- 172.21.15.146:0/1393905501 wait complete. 2024-09-13T18:29:35.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 Processor -- start 2024-09-13T18:29:35.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 -- start start 2024-09-13T18:29:35.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdba40cf540 0x7fdba41efb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:35.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdba4153a60 0x7fdba4147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:35.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdba40b4c50 0x7fdba41583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:35.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40c6cd0 con 0x7fdba40cf540 2024-09-13T18:29:35.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba414dee0 con 0x7fdba4153a60 2024-09-13T18:29:35.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdbaa285700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba414e180 con 0x7fdba40b4c50 2024-09-13T18:29:35.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdba9283700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdba40cf540 0x7fdba41efb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:35.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdba9283700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdba40cf540 0x7fdba41efb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55470/0 (socket says 172.21.15.146:55470) 2024-09-13T18:29:35.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdba9283700 1 -- 172.21.15.146:0/3562465519 learned_addr learned my addr 172.21.15.146:0/3562465519 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:35.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdba9a84700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdba40b4c50 0x7fdba41583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:35.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.311+0000 7fdba8a82700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdba4153a60 0x7fdba4147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:35.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.315+0000 7fdba9283700 1 -- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdba40b4c50 msgr2=0x7fdba41583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:35.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.315+0000 7fdba9283700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdba40b4c50 0x7fdba41583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:35.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.315+0000 7fdba9283700 1 -- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdba4153a60 msgr2=0x7fdba4147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:35.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.315+0000 7fdba9283700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdba4153a60 0x7fdba4147e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:35.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.315+0000 7fdba9283700 1 -- 172.21.15.146:0/3562465519 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c009b30 con 0x7fdba40cf540 2024-09-13T18:29:35.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.315+0000 7fdba9283700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdba40cf540 0x7fdba41efb30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00b370 tx=0x7fdb8c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:35.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.315+0000 7fdba1ffb700 1 -- 172.21.15.146:0/3562465519 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0048c0 con 0x7fdba40cf540 2024-09-13T18:29:35.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.315+0000 7fdba1ffb700 1 -- 172.21.15.146:0/3562465519 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c0092e0 con 0x7fdba40cf540 2024-09-13T18:29:35.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.315+0000 7fdbaa285700 1 -- 172.21.15.146:0/3562465519 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba414e460 con 0x7fdba40cf540 2024-09-13T18:29:35.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.315+0000 7fdba1ffb700 1 -- 172.21.15.146:0/3562465519 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c01c630 con 0x7fdba40cf540 2024-09-13T18:29:35.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.315+0000 7fdbaa285700 1 -- 172.21.15.146:0/3562465519 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba414ea20 con 0x7fdba40cf540 2024-09-13T18:29:35.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.315+0000 7fdbaa285700 1 -- 172.21.15.146:0/3562465519 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba40ca700 con 0x7fdba40cf540 2024-09-13T18:29:35.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.315+0000 7fdba1ffb700 1 -- 172.21.15.146:0/3562465519 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdb8c01c790 con 0x7fdba40cf540 2024-09-13T18:29:35.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.319+0000 7fdba1ffb700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdb840667b0 0x7fdb84068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:35.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.319+0000 7fdba1ffb700 1 -- 172.21.15.146:0/3562465519 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fdb8c085d80 con 0x7fdba40cf540 2024-09-13T18:29:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.319+0000 7fdba8a82700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdb840667b0 0x7fdb84068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:35.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.319+0000 7fdba8a82700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdb840667b0 0x7fdb84068c70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdba414f780 tx=0x7fdb94007ef0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:35.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.327+0000 7fdba1ffb700 1 -- 172.21.15.146:0/3562465519 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdb8c052df0 con 0x7fdba40cf540 2024-09-13T18:29:35.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.535+0000 7fdbaa285700 1 -- 172.21.15.146:0/3562465519 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdba40c9340 con 0x7fdba40cf540 2024-09-13T18:29:35.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.535+0000 7fdba1ffb700 1 -- 172.21.15.146:0/3562465519 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5529 (secure 0 0 0) 0x7fdb8c056410 con 0x7fdba40cf540 2024-09-13T18:29:35.542 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:35.542 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:31.233532+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:resolve","state_seq":13,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:35.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.539+0000 7fdbaa285700 1 -- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdb840667b0 msgr2=0x7fdb84068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:35.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.539+0000 7fdbaa285700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdb840667b0 0x7fdb84068c70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdba414f780 tx=0x7fdb94007ef0 comp rx=0 tx=0).stop 2024-09-13T18:29:35.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.539+0000 7fdbaa285700 1 -- 172.21.15.146:0/3562465519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdba40cf540 msgr2=0x7fdba41efb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:35.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.539+0000 7fdbaa285700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdba40cf540 0x7fdba41efb30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00b370 tx=0x7fdb8c004af0 comp rx=0 tx=0).stop 2024-09-13T18:29:35.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.539+0000 7fdbaa285700 1 -- 172.21.15.146:0/3562465519 shutdown_connections 2024-09-13T18:29:35.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.539+0000 7fdbaa285700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdb840667b0 0x7fdb84068c70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:35.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.539+0000 7fdbaa285700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdba40b4c50 0x7fdba41583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:35.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.539+0000 7fdbaa285700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdba4153a60 0x7fdba4147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:35.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.539+0000 7fdbaa285700 1 --2- 172.21.15.146:0/3562465519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdba40cf540 0x7fdba41efb30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:35.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.539+0000 7fdbaa285700 1 -- 172.21.15.146:0/3562465519 >> 172.21.15.146:0/3562465519 conn(0x7fdba4071ec0 msgr2=0x7fdba4159ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:35.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.539+0000 7fdbaa285700 1 -- 172.21.15.146:0/3562465519 shutdown_connections 2024-09-13T18:29:35.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:35.539+0000 7fdbaa285700 1 -- 172.21.15.146:0/3562465519 wait complete. 2024-09-13T18:29:35.546 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 23 2024-09-13T18:29:35.556 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:29:37.455 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:29:37.457 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:29:37.557 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:37.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 Processor -- start 2024-09-13T18:29:37.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 -- start start 2024-09-13T18:29:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa1800cfb70 0x7fa1800c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa180158090 0x7fa1800c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1800c9620 con 0x7fa1800cf790 2024-09-13T18:29:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18015cb00 con 0x7fa1800cfb70 2024-09-13T18:29:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801541b0 con 0x7fa180158090 2024-09-13T18:29:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa185aa4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa1800cfb70 0x7fa1800c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa185aa4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa1800cfb70 0x7fa1800c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55486/0 (socket says 172.21.15.146:55486) 2024-09-13T18:29:37.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa185aa4700 1 -- 172.21.15.146:0/1482237159 learned_addr learned my addr 172.21.15.146:0/1482237159 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:37.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa185aa4700 1 -- 172.21.15.146:0/1482237159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa180158090 msgr2=0x7fa1800c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:37.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa186aa6700 1 --2- 172.21.15.146:0/1482237159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa180158090 0x7fa1800c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:37.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa185aa4700 1 --2- 172.21.15.146:0/1482237159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa180158090 0x7fa1800c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:37.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa185aa4700 1 -- 172.21.15.146:0/1482237159 >> v1:172.21.15.154:6789/0 conn(0x7fa1800cf790 legacy=0x7fa180157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:37.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa185aa4700 1 -- 172.21.15.146:0/1482237159 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18015bba0 con 0x7fa1800cfb70 2024-09-13T18:29:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa186aa6700 1 --2- 172.21.15.146:0/1482237159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa180158090 0x7fa1800c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:29:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa185aa4700 1 --2- 172.21.15.146:0/1482237159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1800cfb70 0x7fa1800c8ba0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009e80 tx=0x7fa17c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a28bab587691f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa1852a3700 1 -- 172.21.15.146:0/1482237159 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00e070 con 0x7fa1800cfb70 2024-09-13T18:29:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa1852a3700 1 -- 172.21.15.146:0/1482237159 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c003a40 con 0x7fa1800cfb70 2024-09-13T18:29:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa1852a3700 1 -- 172.21.15.146:0/1482237159 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c003df0 con 0x7fa1800cfb70 2024-09-13T18:29:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 -- 172.21.15.146:0/1482237159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1800cfb70 msgr2=0x7fa1800c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 --2- 172.21.15.146:0/1482237159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1800cfb70 0x7fa1800c8ba0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009e80 tx=0x7fa17c00b330 comp rx=0 tx=0).stop 2024-09-13T18:29:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 -- 172.21.15.146:0/1482237159 shutdown_connections 2024-09-13T18:29:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 --2- 172.21.15.146:0/1482237159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa180158090 0x7fa1800c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 --2- 172.21.15.146:0/1482237159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1800cfb70 0x7fa1800c8ba0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 -- 172.21.15.146:0/1482237159 >> 172.21.15.146:0/1482237159 conn(0x7fa180071d90 msgr2=0x7fa1800721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 -- 172.21.15.146:0/1482237159 shutdown_connections 2024-09-13T18:29:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.670+0000 7fa188509700 1 -- 172.21.15.146:0/1482237159 wait complete. 2024-09-13T18:29:37.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.674+0000 7fa188509700 1 Processor -- start 2024-09-13T18:29:37.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.674+0000 7fa188509700 1 -- start start 2024-09-13T18:29:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.674+0000 7fa188509700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1800cf790 0x7fa1801ef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.674+0000 7fa188509700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1800cfb70 0x7fa180153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.674+0000 7fa188509700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa180158090 0x7fa1800c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.674+0000 7fa188509700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18014fc90 con 0x7fa1800cfb70 2024-09-13T18:29:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.674+0000 7fa188509700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801f2e10 con 0x7fa180158090 2024-09-13T18:29:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.674+0000 7fa188509700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18014e920 con 0x7fa1800cf790 2024-09-13T18:29:37.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.674+0000 7fa185aa4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1800cfb70 0x7fa180153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.674+0000 7fa185aa4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1800cfb70 0x7fa180153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55494/0 (socket says 172.21.15.146:55494) 2024-09-13T18:29:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.674+0000 7fa185aa4700 1 -- 172.21.15.146:0/3793560948 learned_addr learned my addr 172.21.15.146:0/3793560948 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.674+0000 7fa186aa6700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa180158090 0x7fa1800c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa1862a5700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1800cf790 0x7fa1801ef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa185aa4700 1 -- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1800cf790 msgr2=0x7fa1801ef0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa185aa4700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1800cf790 0x7fa1801ef0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa185aa4700 1 -- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa180158090 msgr2=0x7fa1800c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa185aa4700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa180158090 0x7fa1800c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa185aa4700 1 -- 172.21.15.146:0/3793560948 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c009b30 con 0x7fa1800cfb70 2024-09-13T18:29:37.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa185aa4700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1800cfb70 0x7fa180153260 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa17c004000 tx=0x7fa17c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:37.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa176ffd700 1 -- 172.21.15.146:0/3793560948 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0044c0 con 0x7fa1800cfb70 2024-09-13T18:29:37.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa176ffd700 1 -- 172.21.15.146:0/3793560948 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c004620 con 0x7fa1800cfb70 2024-09-13T18:29:37.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa176ffd700 1 -- 172.21.15.146:0/3793560948 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c012430 con 0x7fa1800cfb70 2024-09-13T18:29:37.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa188509700 1 -- 172.21.15.146:0/3793560948 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18014ebc0 con 0x7fa1800cfb70 2024-09-13T18:29:37.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa188509700 1 -- 172.21.15.146:0/3793560948 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1801f27c0 con 0x7fa1800cfb70 2024-09-13T18:29:37.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa188509700 1 -- 172.21.15.146:0/3793560948 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1801f2620 con 0x7fa1800cfb70 2024-09-13T18:29:37.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.678+0000 7fa176ffd700 1 -- 172.21.15.146:0/3793560948 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa17c012590 con 0x7fa1800cfb70 2024-09-13T18:29:37.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.682+0000 7fa176ffd700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa164066730 0x7fa164068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:37.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.682+0000 7fa176ffd700 1 -- 172.21.15.146:0/3793560948 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fa17c0859a0 con 0x7fa1800cfb70 2024-09-13T18:29:37.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.682+0000 7fa1862a5700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa164066730 0x7fa164068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:37.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.682+0000 7fa1862a5700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa164066730 0x7fa164068bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa170004f70 tx=0x7fa170009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:37.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.686+0000 7fa176ffd700 1 -- 172.21.15.146:0/3793560948 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa1801f2620 con 0x7fa1800cfb70 2024-09-13T18:29:37.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.894+0000 7fa188509700 1 -- 172.21.15.146:0/3793560948 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1801f2620 con 0x7fa1800cfb70 2024-09-13T18:29:37.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.894+0000 7fa176ffd700 1 -- 172.21.15.146:0/3793560948 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5531 (secure 0 0 0) 0x7fa1801f2620 con 0x7fa1800cfb70 2024-09-13T18:29:37.904 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:37.904 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:36.256049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:37.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.898+0000 7fa188509700 1 -- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa164066730 msgr2=0x7fa164068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:37.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.898+0000 7fa188509700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa164066730 0x7fa164068bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa170004f70 tx=0x7fa170009040 comp rx=0 tx=0).stop 2024-09-13T18:29:37.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.898+0000 7fa188509700 1 -- 172.21.15.146:0/3793560948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1800cfb70 msgr2=0x7fa180153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:37.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.898+0000 7fa188509700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1800cfb70 0x7fa180153260 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa17c004000 tx=0x7fa17c004af0 comp rx=0 tx=0).stop 2024-09-13T18:29:37.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.898+0000 7fa188509700 1 -- 172.21.15.146:0/3793560948 shutdown_connections 2024-09-13T18:29:37.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.898+0000 7fa188509700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa164066730 0x7fa164068bf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:37.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.898+0000 7fa188509700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa180158090 0x7fa1800c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:37.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.898+0000 7fa188509700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1800cfb70 0x7fa180153260 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:37.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.898+0000 7fa188509700 1 --2- 172.21.15.146:0/3793560948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1800cf790 0x7fa1801ef0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:37.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.898+0000 7fa188509700 1 -- 172.21.15.146:0/3793560948 >> 172.21.15.146:0/3793560948 conn(0x7fa180071d90 msgr2=0x7fa18015ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:37.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.898+0000 7fa188509700 1 -- 172.21.15.146:0/3793560948 shutdown_connections 2024-09-13T18:29:37.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:37.898+0000 7fa188509700 1 -- 172.21.15.146:0/3793560948 wait complete. 2024-09-13T18:29:37.908 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 24 2024-09-13T18:29:37.919 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 24, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 24, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4281": { "addr": "172.21.15.146:6835/1642500214", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 1642500214, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 1642500214, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4281, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 11 }, "gid_4290": { "addr": "172.21.15.146:6837/31041557", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 31041557, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 31041557, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4290, "incarnation": 4, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_5235": { "addr": "172.21.15.146:6833/1498985282", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 1498985282, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 1498985282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5235, "incarnation": 22, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:reconnect", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 29, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:29:36.256049+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4290, "mds_1": 4281, "mds_2": 5235 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:29:39.920 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:40.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa427fff700 1 Processor -- start 2024-09-13T18:29:40.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa427fff700 1 -- start start 2024-09-13T18:29:40.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa427fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa4280c5fc0 0x7fa4280c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa427fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4280c63a0 0x7fa4280c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa427fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4280c8c70 con 0x7fa42806f5c0 2024-09-13T18:29:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa427fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4280c9bd0 con 0x7fa4280c5fc0 2024-09-13T18:29:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa427fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280c8f50 con 0x7fa4280c63a0 2024-09-13T18:29:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa426ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa4280c5fc0 0x7fa4280c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa426ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa4280c5fc0 0x7fa4280c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55500/0 (socket says 172.21.15.146:55500) 2024-09-13T18:29:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa426ffd700 1 -- 172.21.15.146:0/2911351956 learned_addr learned my addr 172.21.15.146:0/2911351956 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa4267fc700 1 --2- 172.21.15.146:0/2911351956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4280c63a0 0x7fa4280c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa426ffd700 1 -- 172.21.15.146:0/2911351956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4280c63a0 msgr2=0x7fa4280c7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa426ffd700 1 --2- 172.21.15.146:0/2911351956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4280c63a0 0x7fa4280c7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa426ffd700 1 -- 172.21.15.146:0/2911351956 >> v1:172.21.15.154:6789/0 conn(0x7fa42806f5c0 legacy=0x7fa428159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.038+0000 7fa426ffd700 1 -- 172.21.15.146:0/2911351956 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4280c9230 con 0x7fa4280c5fc0 2024-09-13T18:29:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa426ffd700 1 --2- 172.21.15.146:0/2911351956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4280c5fc0 0x7fa4280c6c00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00bbb0 tx=0x7fa41c00bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cc90e5634b2535 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa425ffb700 1 -- 172.21.15.146:0/2911351956 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c00dd30 con 0x7fa4280c5fc0 2024-09-13T18:29:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa425ffb700 1 -- 172.21.15.146:0/2911351956 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c00de90 con 0x7fa4280c5fc0 2024-09-13T18:29:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa425ffb700 1 -- 172.21.15.146:0/2911351956 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c010690 con 0x7fa4280c5fc0 2024-09-13T18:29:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa427fff700 1 -- 172.21.15.146:0/2911351956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4280c5fc0 msgr2=0x7fa4280c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa427fff700 1 --2- 172.21.15.146:0/2911351956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4280c5fc0 0x7fa4280c6c00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00bbb0 tx=0x7fa41c00bec0 comp rx=0 tx=0).stop 2024-09-13T18:29:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa427fff700 1 -- 172.21.15.146:0/2911351956 shutdown_connections 2024-09-13T18:29:40.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa427fff700 1 --2- 172.21.15.146:0/2911351956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4280c63a0 0x7fa4280c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:40.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa427fff700 1 --2- 172.21.15.146:0/2911351956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4280c5fc0 0x7fa4280c6c00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:40.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa427fff700 1 -- 172.21.15.146:0/2911351956 >> 172.21.15.146:0/2911351956 conn(0x7fa428066090 msgr2=0x7fa428066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:40.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa427fff700 1 -- 172.21.15.146:0/2911351956 shutdown_connections 2024-09-13T18:29:40.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa427fff700 1 -- 172.21.15.146:0/2911351956 wait complete. 2024-09-13T18:29:40.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa427fff700 1 Processor -- start 2024-09-13T18:29:40.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.042+0000 7fa427fff700 1 -- start start 2024-09-13T18:29:40.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa427fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa42806f5c0 0x7fa4280adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:40.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa427fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4280c5fc0 0x7fa428152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:40.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa427fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4280c63a0 0x7fa4280c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:40.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa427fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280ae090 con 0x7fa4280c5fc0 2024-09-13T18:29:40.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa427fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4281534a0 con 0x7fa4280c63a0 2024-09-13T18:29:40.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa427fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280af7a0 con 0x7fa42806f5c0 2024-09-13T18:29:40.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa4267fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4280c5fc0 0x7fa428152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:40.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa4267fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4280c5fc0 0x7fa428152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55506/0 (socket says 172.21.15.146:55506) 2024-09-13T18:29:40.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa4267fc700 1 -- 172.21.15.146:0/483848590 learned_addr learned my addr 172.21.15.146:0/483848590 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa426ffd700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa42806f5c0 0x7fa4280adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa4277fe700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4280c63a0 0x7fa4280c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa4267fc700 1 -- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa42806f5c0 msgr2=0x7fa4280adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa4267fc700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa42806f5c0 0x7fa4280adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa4267fc700 1 -- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4280c63a0 msgr2=0x7fa4280c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa4267fc700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4280c63a0 0x7fa4280c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa4267fc700 1 -- 172.21.15.146:0/483848590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c00b810 con 0x7fa4280c5fc0 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa4277fe700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4280c63a0 0x7fa4280c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa4267fc700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4280c5fc0 0x7fa428152f60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa418009dd0 tx=0x7fa41800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa40f7fe700 1 -- 172.21.15.146:0/483848590 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418009c40 con 0x7fa4280c5fc0 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa40f7fe700 1 -- 172.21.15.146:0/483848590 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41800f040 con 0x7fa4280c5fc0 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa427fff700 1 -- 172.21.15.146:0/483848590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4280afb70 con 0x7fa4280c5fc0 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa40f7fe700 1 -- 172.21.15.146:0/483848590 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418014480 con 0x7fa4280c5fc0 2024-09-13T18:29:40.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa427fff700 1 -- 172.21.15.146:0/483848590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4280ae290 con 0x7fa4280c5fc0 2024-09-13T18:29:40.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.046+0000 7fa427fff700 1 -- 172.21.15.146:0/483848590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4280cb790 con 0x7fa4280c5fc0 2024-09-13T18:29:40.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.050+0000 7fa40f7fe700 1 -- 172.21.15.146:0/483848590 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa4180145e0 con 0x7fa4280c5fc0 2024-09-13T18:29:40.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.050+0000 7fa40f7fe700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa414066730 0x7fa414068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:40.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.050+0000 7fa40f7fe700 1 -- 172.21.15.146:0/483848590 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fa4180857e0 con 0x7fa4280c5fc0 2024-09-13T18:29:40.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.050+0000 7fa426ffd700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa414066730 0x7fa414068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:40.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.050+0000 7fa426ffd700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa414066730 0x7fa414068bf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00bfd0 tx=0x7fa41c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:40.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.054+0000 7fa40f7fe700 1 -- 172.21.15.146:0/483848590 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa418052630 con 0x7fa4280c5fc0 2024-09-13T18:29:40.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.262+0000 7fa427fff700 1 -- 172.21.15.146:0/483848590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4280ae5d0 con 0x7fa4280c5fc0 2024-09-13T18:29:40.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.262+0000 7fa40f7fe700 1 -- 172.21.15.146:0/483848590 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5531 (secure 0 0 0) 0x7fa418018090 con 0x7fa4280c5fc0 2024-09-13T18:29:40.273 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:40.273 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:36.256049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:40.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.266+0000 7fa427fff700 1 -- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa414066730 msgr2=0x7fa414068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:40.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.266+0000 7fa427fff700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa414066730 0x7fa414068bf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00bfd0 tx=0x7fa41c008040 comp rx=0 tx=0).stop 2024-09-13T18:29:40.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.266+0000 7fa427fff700 1 -- 172.21.15.146:0/483848590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4280c5fc0 msgr2=0x7fa428152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:40.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.266+0000 7fa427fff700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4280c5fc0 0x7fa428152f60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa418009dd0 tx=0x7fa41800c3a0 comp rx=0 tx=0).stop 2024-09-13T18:29:40.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.266+0000 7fa427fff700 1 -- 172.21.15.146:0/483848590 shutdown_connections 2024-09-13T18:29:40.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.266+0000 7fa427fff700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa414066730 0x7fa414068bf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:40.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.266+0000 7fa427fff700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4280c63a0 0x7fa4280c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:40.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.266+0000 7fa427fff700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4280c5fc0 0x7fa428152f60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:40.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.266+0000 7fa427fff700 1 --2- 172.21.15.146:0/483848590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa42806f5c0 0x7fa4280adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:40.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.266+0000 7fa427fff700 1 -- 172.21.15.146:0/483848590 >> 172.21.15.146:0/483848590 conn(0x7fa428066090 msgr2=0x7fa428158580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:40.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.266+0000 7fa427fff700 1 -- 172.21.15.146:0/483848590 shutdown_connections 2024-09-13T18:29:40.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:40.266+0000 7fa427fff700 1 -- 172.21.15.146:0/483848590 wait complete. 2024-09-13T18:29:40.276 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 24 2024-09-13T18:29:40.287 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:29:42.288 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:42.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.402+0000 7f40fb336700 1 Processor -- start 2024-09-13T18:29:42.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.402+0000 7f40fb336700 1 -- start start 2024-09-13T18:29:42.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.402+0000 7f40fb336700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f40f4153860 0x7f40f40c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:42.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.402+0000 7f40fb336700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40f40c7970 0x7f40f406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:42.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.402+0000 7f40fb336700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f40c98d0 con 0x7f40f406c5b0 2024-09-13T18:29:42.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.402+0000 7f40fb336700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f40c4200 con 0x7f40f4153860 2024-09-13T18:29:42.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.402+0000 7f40fb336700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40c8b20 con 0x7f40f40c7970 2024-09-13T18:29:42.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.402+0000 7f40fa334700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f40f4153860 0x7f40f40c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:42.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.402+0000 7f40fab35700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f40f406c5b0 0x7f40f4070bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35426/0 (socket says 172.21.15.146:35426) 2024-09-13T18:29:42.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.402+0000 7f40fab35700 1 -- 172.21.15.146:0/505792347 learned_addr learned my addr 172.21.15.146:0/505792347 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:29:42.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.402+0000 7f40f9b33700 1 --2- 172.21.15.146:0/505792347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40f40c7970 0x7f40f406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:42.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40f9b33700 1 -- 172.21.15.146:0/505792347 >> v2:172.21.15.146:3300/0 conn(0x7f40f4153860 msgr2=0x7f40f40c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:42.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40f9b33700 1 --2- 172.21.15.146:0/505792347 >> v2:172.21.15.146:3300/0 conn(0x7f40f4153860 0x7f40f40c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:42.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40f9b33700 1 -- 172.21.15.146:0/505792347 >> v1:172.21.15.146:6789/0 conn(0x7f40f406c5b0 legacy=0x7f40f4070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:42.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40f9b33700 1 -- 172.21.15.146:0/505792347 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f40706d0 con 0x7f40f40c7970 2024-09-13T18:29:42.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40f9b33700 1 --2- 172.21.15.146:0/505792347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40f40c7970 0x7f40f406c070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009850 tx=0x7f40e4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58859a3c748cec63 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:42.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40f9332700 1 -- 172.21.15.146:0/505792347 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4011830 con 0x7f40f40c7970 2024-09-13T18:29:42.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40f9332700 1 -- 172.21.15.146:0/505792347 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e4011990 con 0x7f40f40c7970 2024-09-13T18:29:42.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40f9332700 1 -- 172.21.15.146:0/505792347 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4011d40 con 0x7f40f40c7970 2024-09-13T18:29:42.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40fb336700 1 -- 172.21.15.146:0/505792347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40f40c7970 msgr2=0x7f40f406c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:42.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40fb336700 1 --2- 172.21.15.146:0/505792347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40f40c7970 0x7f40f406c070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009850 tx=0x7f40e4009b60 comp rx=0 tx=0).stop 2024-09-13T18:29:42.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40fb336700 1 -- 172.21.15.146:0/505792347 shutdown_connections 2024-09-13T18:29:42.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40fb336700 1 --2- 172.21.15.146:0/505792347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40f40c7970 0x7f40f406c070 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:42.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40fb336700 1 --2- 172.21.15.146:0/505792347 >> v2:172.21.15.146:3300/0 conn(0x7f40f4153860 0x7f40f40c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:42.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40fb336700 1 -- 172.21.15.146:0/505792347 >> 172.21.15.146:0/505792347 conn(0x7f40f40ccf70 msgr2=0x7f40f40cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:42.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40fb336700 1 -- 172.21.15.146:0/505792347 shutdown_connections 2024-09-13T18:29:42.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40fb336700 1 -- 172.21.15.146:0/505792347 wait complete. 2024-09-13T18:29:42.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40fb336700 1 Processor -- start 2024-09-13T18:29:42.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.406+0000 7f40fb336700 1 -- start start 2024-09-13T18:29:42.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40fb336700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40f406c5b0 0x7f40f40c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:42.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40fb336700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40f40c7970 0x7f40f41e4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:42.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40fb336700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40f4153860 0x7f40f41f2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:42.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40fb336700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f41f3380 con 0x7f40f40c7970 2024-09-13T18:29:42.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40fb336700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f414dae0 con 0x7f40f4153860 2024-09-13T18:29:42.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40fb336700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f41f30c0 con 0x7f40f406c5b0 2024-09-13T18:29:42.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40f9b33700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40f40c7970 0x7f40f41e4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:42.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40f9b33700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40f40c7970 0x7f40f41e4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46812/0 (socket says 172.21.15.146:46812) 2024-09-13T18:29:42.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40f9b33700 1 -- 172.21.15.146:0/3881590459 learned_addr learned my addr 172.21.15.146:0/3881590459 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:42.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40fa334700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40f406c5b0 0x7f40f40c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:42.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40fab35700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40f4153860 0x7f40f41f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:42.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40f9b33700 1 -- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40f406c5b0 msgr2=0x7f40f40c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:42.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40f9b33700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40f406c5b0 0x7f40f40c7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:42.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40f9b33700 1 -- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40f4153860 msgr2=0x7f40f41f2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:42.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40f9b33700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40f4153860 0x7f40f41f2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:42.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40f9b33700 1 -- 172.21.15.146:0/3881590459 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e40094b0 con 0x7f40f40c7970 2024-09-13T18:29:42.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40fab35700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40f4153860 0x7f40f41f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:29:42.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40fa334700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40f406c5b0 0x7f40f40c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:29:42.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40f9b33700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40f40c7970 0x7f40f41e4830 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f40e40090b0 tx=0x7f40e4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:42.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40eaffd700 1 -- 172.21.15.146:0/3881590459 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4004570 con 0x7f40f40c7970 2024-09-13T18:29:42.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40eaffd700 1 -- 172.21.15.146:0/3881590459 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e401f070 con 0x7f40f40c7970 2024-09-13T18:29:42.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40fb336700 1 -- 172.21.15.146:0/3881590459 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f4159440 con 0x7f40f40c7970 2024-09-13T18:29:42.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40eaffd700 1 -- 172.21.15.146:0/3881590459 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4003900 con 0x7f40f40c7970 2024-09-13T18:29:42.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40fb336700 1 -- 172.21.15.146:0/3881590459 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f4152580 con 0x7f40f40c7970 2024-09-13T18:29:42.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.410+0000 7f40eaffd700 1 -- 172.21.15.146:0/3881590459 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f40e4021030 con 0x7f40f40c7970 2024-09-13T18:29:42.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.414+0000 7f40eaffd700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f40d8066760 0x7f40d8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:42.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.414+0000 7f40fb336700 1 -- 172.21.15.146:0/3881590459 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f4067a60 con 0x7f40f40c7970 2024-09-13T18:29:42.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.414+0000 7f40eaffd700 1 -- 172.21.15.146:0/3881590459 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f40e4084bf0 con 0x7f40f40c7970 2024-09-13T18:29:42.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.414+0000 7f40fa334700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f40d8066760 0x7f40d8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:42.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.414+0000 7f40fa334700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f40d8066760 0x7f40d8068c20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009c40 tx=0x7f40f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:42.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.422+0000 7f40eaffd700 1 -- 172.21.15.146:0/3881590459 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f40e4015070 con 0x7f40f40c7970 2024-09-13T18:29:42.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.630+0000 7f40fb336700 1 -- 172.21.15.146:0/3881590459 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40f415a2b0 con 0x7f40f40c7970 2024-09-13T18:29:42.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.634+0000 7f40eaffd700 1 -- 172.21.15.146:0/3881590459 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5528 (secure 0 0 0) 0x7f40e4051a60 con 0x7f40f40c7970 2024-09-13T18:29:42.640 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:42.641 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:41.281298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:42.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.634+0000 7f40fb336700 1 -- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f40d8066760 msgr2=0x7f40d8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:42.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.634+0000 7f40fb336700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f40d8066760 0x7f40d8068c20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009c40 tx=0x7f40f0008040 comp rx=0 tx=0).stop 2024-09-13T18:29:42.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.634+0000 7f40fb336700 1 -- 172.21.15.146:0/3881590459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40f40c7970 msgr2=0x7f40f41e4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:42.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.634+0000 7f40fb336700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40f40c7970 0x7f40f41e4830 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f40e40090b0 tx=0x7f40e4004770 comp rx=0 tx=0).stop 2024-09-13T18:29:42.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.634+0000 7f40fb336700 1 -- 172.21.15.146:0/3881590459 shutdown_connections 2024-09-13T18:29:42.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.634+0000 7f40fb336700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f40d8066760 0x7f40d8068c20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:42.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.634+0000 7f40fb336700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40f4153860 0x7f40f41f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:42.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.634+0000 7f40fb336700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40f40c7970 0x7f40f41e4830 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:42.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.634+0000 7f40fb336700 1 --2- 172.21.15.146:0/3881590459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40f406c5b0 0x7f40f40c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:42.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.634+0000 7f40fb336700 1 -- 172.21.15.146:0/3881590459 >> 172.21.15.146:0/3881590459 conn(0x7f40f40ccf70 msgr2=0x7f40f41e1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:42.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.634+0000 7f40fb336700 1 -- 172.21.15.146:0/3881590459 shutdown_connections 2024-09-13T18:29:42.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:42.634+0000 7f40fb336700 1 -- 172.21.15.146:0/3881590459 wait complete. 2024-09-13T18:29:42.645 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 25 2024-09-13T18:29:42.655 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:29:44.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:44.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.770+0000 7fc101b9c700 1 Processor -- start 2024-09-13T18:29:44.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.770+0000 7fc101b9c700 1 -- start start 2024-09-13T18:29:44.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc101b9c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc0fc0caed0 0x7fc0fc0723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc101b9c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0fc0a9840 0x7fc0fc15e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc101b9c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0fc161550 con 0x7fc0fc0cb2b0 2024-09-13T18:29:44.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc101b9c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0fc072a30 con 0x7fc0fc0caed0 2024-09-13T18:29:44.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc101b9c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0fc0aa800 con 0x7fc0fc0a9840 2024-09-13T18:29:44.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc100b9a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc0fc0caed0 0x7fc0fc0723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc100b9a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc0fc0caed0 0x7fc0fc0723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46814/0 (socket says 172.21.15.146:46814) 2024-09-13T18:29:44.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc100b9a700 1 -- 172.21.15.146:0/2372830750 learned_addr learned my addr 172.21.15.146:0/2372830750 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:44.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc10139b700 1 --2- 172.21.15.146:0/2372830750 >> v2:172.21.15.154:3300/0 conn(0x7fc0fc0a9840 0x7fc0fc15e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc100b9a700 1 -- 172.21.15.146:0/2372830750 >> v2:172.21.15.154:3300/0 conn(0x7fc0fc0a9840 msgr2=0x7fc0fc15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:44.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc100b9a700 1 --2- 172.21.15.146:0/2372830750 >> v2:172.21.15.154:3300/0 conn(0x7fc0fc0a9840 0x7fc0fc15e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc100b9a700 1 -- 172.21.15.146:0/2372830750 >> v1:172.21.15.154:6789/0 conn(0x7fc0fc0cb2b0 legacy=0x7fc0fc1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:44.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc100b9a700 1 -- 172.21.15.146:0/2372830750 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc161220 con 0x7fc0fc0caed0 2024-09-13T18:29:44.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc100b9a700 1 --2- 172.21.15.146:0/2372830750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0fc0caed0 0x7fc0fc0723f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0b0180 tx=0x7fc0ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf6d16c76868308 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:44.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc0fb7fe700 1 -- 172.21.15.146:0/2372830750 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec00e070 con 0x7fc0fc0caed0 2024-09-13T18:29:44.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc0fb7fe700 1 -- 172.21.15.146:0/2372830750 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec00b910 con 0x7fc0fc0caed0 2024-09-13T18:29:44.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc0fb7fe700 1 -- 172.21.15.146:0/2372830750 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec00bcc0 con 0x7fc0fc0caed0 2024-09-13T18:29:44.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc101b9c700 1 -- 172.21.15.146:0/2372830750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0fc0caed0 msgr2=0x7fc0fc0723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:44.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc101b9c700 1 --2- 172.21.15.146:0/2372830750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0fc0caed0 0x7fc0fc0723f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0b0180 tx=0x7fc0ec00b330 comp rx=0 tx=0).stop 2024-09-13T18:29:44.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc101b9c700 1 -- 172.21.15.146:0/2372830750 shutdown_connections 2024-09-13T18:29:44.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc101b9c700 1 --2- 172.21.15.146:0/2372830750 >> v2:172.21.15.154:3300/0 conn(0x7fc0fc0a9840 0x7fc0fc15e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc101b9c700 1 --2- 172.21.15.146:0/2372830750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0fc0caed0 0x7fc0fc0723f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc101b9c700 1 -- 172.21.15.146:0/2372830750 >> 172.21.15.146:0/2372830750 conn(0x7fc0fc066090 msgr2=0x7fc0fc066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:44.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc101b9c700 1 -- 172.21.15.146:0/2372830750 shutdown_connections 2024-09-13T18:29:44.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.774+0000 7fc101b9c700 1 -- 172.21.15.146:0/2372830750 wait complete. 2024-09-13T18:29:44.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.778+0000 7fc101b9c700 1 Processor -- start 2024-09-13T18:29:44.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.778+0000 7fc101b9c700 1 -- start start 2024-09-13T18:29:44.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc101b9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0fc0a9840 0x7fc0fc1f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc101b9c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0fc0caed0 0x7fc0fc1f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc101b9c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0fc0cb2b0 0x7fc0fc1f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc101b9c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0af590 con 0x7fc0fc0a9840 2024-09-13T18:29:44.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc101b9c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc1f6c70 con 0x7fc0fc0cb2b0 2024-09-13T18:29:44.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc101b9c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc124340 con 0x7fc0fc0caed0 2024-09-13T18:29:44.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc100b9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0fc0a9840 0x7fc0fc1f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc100b9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0fc0a9840 0x7fc0fc1f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46820/0 (socket says 172.21.15.146:46820) 2024-09-13T18:29:44.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc100b9a700 1 -- 172.21.15.146:0/2220396104 learned_addr learned my addr 172.21.15.146:0/2220396104 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:44.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc0fbfff700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0fc0caed0 0x7fc0fc1f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc10139b700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0fc0cb2b0 0x7fc0fc1f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc100b9a700 1 -- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0fc0caed0 msgr2=0x7fc0fc1f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:44.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc100b9a700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0fc0caed0 0x7fc0fc1f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc100b9a700 1 -- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0fc0cb2b0 msgr2=0x7fc0fc1f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:44.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc100b9a700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0fc0cb2b0 0x7fc0fc1f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc100b9a700 1 -- 172.21.15.146:0/2220396104 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec009b30 con 0x7fc0fc0a9840 2024-09-13T18:29:44.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc100b9a700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0fc0a9840 0x7fc0fc1f4520 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0cac30 tx=0x7fc0ec004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:44.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc0f97fa700 1 -- 172.21.15.146:0/2220396104 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec0092e0 con 0x7fc0fc0a9840 2024-09-13T18:29:44.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc0f97fa700 1 -- 172.21.15.146:0/2220396104 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec01b430 con 0x7fc0fc0a9840 2024-09-13T18:29:44.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc101b9c700 1 -- 172.21.15.146:0/2220396104 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0fc0af7e0 con 0x7fc0fc0a9840 2024-09-13T18:29:44.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc0f97fa700 1 -- 172.21.15.146:0/2220396104 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec01b800 con 0x7fc0fc0a9840 2024-09-13T18:29:44.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc101b9c700 1 -- 172.21.15.146:0/2220396104 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0fc1f6620 con 0x7fc0fc0a9840 2024-09-13T18:29:44.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.782+0000 7fc101b9c700 1 -- 172.21.15.146:0/2220396104 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0fc06f5c0 con 0x7fc0fc0a9840 2024-09-13T18:29:44.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.786+0000 7fc0f97fa700 1 -- 172.21.15.146:0/2220396104 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc0ec01b960 con 0x7fc0fc0a9840 2024-09-13T18:29:44.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.786+0000 7fc0f97fa700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc0e8066730 0x7fc0e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.786+0000 7fc0f97fa700 1 -- 172.21.15.146:0/2220396104 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fc0ec085bb0 con 0x7fc0fc0a9840 2024-09-13T18:29:44.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.786+0000 7fc0fbfff700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc0e8066730 0x7fc0e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.786+0000 7fc0fbfff700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc0e8066730 0x7fc0e8068bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0aeda0 tx=0x7fc0f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:44.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.790+0000 7fc0f97fa700 1 -- 172.21.15.146:0/2220396104 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc0ec052b40 con 0x7fc0fc0a9840 2024-09-13T18:29:45.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.998+0000 7fc101b9c700 1 -- 172.21.15.146:0/2220396104 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0fc06ec10 con 0x7fc0fc0a9840 2024-09-13T18:29:45.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:44.998+0000 7fc0f97fa700 1 -- 172.21.15.146:0/2220396104 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5528 (secure 0 0 0) 0x7fc0ec01f090 con 0x7fc0fc0a9840 2024-09-13T18:29:45.008 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:45.008 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:41.281298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:45.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:45.002+0000 7fc101b9c700 1 -- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc0e8066730 msgr2=0x7fc0e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:45.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:45.002+0000 7fc101b9c700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc0e8066730 0x7fc0e8068bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0aeda0 tx=0x7fc0f0009040 comp rx=0 tx=0).stop 2024-09-13T18:29:45.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:45.002+0000 7fc101b9c700 1 -- 172.21.15.146:0/2220396104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0fc0a9840 msgr2=0x7fc0fc1f4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:45.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:45.002+0000 7fc101b9c700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0fc0a9840 0x7fc0fc1f4520 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0cac30 tx=0x7fc0ec004af0 comp rx=0 tx=0).stop 2024-09-13T18:29:45.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:45.002+0000 7fc101b9c700 1 -- 172.21.15.146:0/2220396104 shutdown_connections 2024-09-13T18:29:45.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:45.002+0000 7fc101b9c700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc0e8066730 0x7fc0e8068bf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:45.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:45.002+0000 7fc101b9c700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0fc0cb2b0 0x7fc0fc1f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:45.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:45.002+0000 7fc101b9c700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0fc0caed0 0x7fc0fc1f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:45.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:45.002+0000 7fc101b9c700 1 --2- 172.21.15.146:0/2220396104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0fc0a9840 0x7fc0fc1f4520 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:45.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:45.002+0000 7fc101b9c700 1 -- 172.21.15.146:0/2220396104 >> 172.21.15.146:0/2220396104 conn(0x7fc0fc066090 msgr2=0x7fc0fc066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:45.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:45.002+0000 7fc101b9c700 1 -- 172.21.15.146:0/2220396104 shutdown_connections 2024-09-13T18:29:45.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:45.002+0000 7fc101b9c700 1 -- 172.21.15.146:0/2220396104 wait complete. 2024-09-13T18:29:45.012 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 25 2024-09-13T18:29:45.023 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:29:46.386 INFO:tasks.workunit.client.0.smithi146.stdout:checking limits.h presence... yes 2024-09-13T18:29:46.402 INFO:tasks.workunit.client.0.smithi146.stdout:checking for limits.h... yes 2024-09-13T18:29:46.454 INFO:tasks.workunit.client.0.smithi146.stdout:checking for stdint.h... (cached) yes 2024-09-13T18:29:47.025 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:47.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.138+0000 7fab9bfff700 1 Processor -- start 2024-09-13T18:29:47.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.138+0000 7fab9bfff700 1 -- start start 2024-09-13T18:29:47.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.138+0000 7fab9bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fab9c0c7020 0x7fab9c0725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:47.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.138+0000 7fab9bfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c15e4a0 con 0x7fab9c0c1be0 2024-09-13T18:29:47.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.138+0000 7fab9bfff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c15e720 con 0x7fab9c15d970 2024-09-13T18:29:47.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.138+0000 7fab9bfff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9c0c2ba0 con 0x7fab9c0c7020 2024-09-13T18:29:47.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.138+0000 7fab9a7fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fab9c0c1be0 0x7fab9c1661f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35436/0 (socket says 172.21.15.146:35436) 2024-09-13T18:29:47.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.138+0000 7fab9a7fc700 1 -- 172.21.15.146:0/2204391733 learned_addr learned my addr 172.21.15.146:0/2204391733 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:29:47.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9b7fe700 1 --2- 172.21.15.146:0/2204391733 >> v2:172.21.15.154:3300/0 conn(0x7fab9c0c7020 0x7fab9c0725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:47.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756186835 0 0) 0x7fab9c15e4a0 con 0x7fab9c0c1be0 2024-09-13T18:29:47.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab7c0035d0 con 0x7fab9c0c1be0 2024-09-13T18:29:47.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3190843346 0 0) 0x7fab7c0035d0 con 0x7fab9c0c1be0 2024-09-13T18:29:47.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 >> v2:172.21.15.154:3300/0 conn(0x7fab9c0c7020 msgr2=0x7fab9c0725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:47.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 --2- 172.21.15.146:0/2204391733 >> v2:172.21.15.154:3300/0 conn(0x7fab9c0c7020 0x7fab9c0725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:47.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 >> v1:172.21.15.154:6789/0 conn(0x7fab9c15d970 legacy=0x7fab9c0c14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:47.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c0b18b0 con 0x7fab9c0c1be0 2024-09-13T18:29:47.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7fab84001670 con 0x7fab9c0c1be0 2024-09-13T18:29:47.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 >> v1:172.21.15.146:6789/0 conn(0x7fab9c0c1be0 legacy=0x7fab9c1661f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 --2- 172.21.15.146:0/2204391733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c003a90 0x7fab7c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 --2- 172.21.15.146:0/2204391733 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab7c004870 0x7fab7c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 --2- 172.21.15.146:0/2204391733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7c005230 0x7fab7c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0c2ba0 con 0x7fab7c003a90 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c00c050 con 0x7fab7c005230 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c00c210 con 0x7fab7c004870 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9affd700 1 --2- 172.21.15.146:0/2204391733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c003a90 0x7fab7c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9a7fc700 1 --2- 172.21.15.146:0/2204391733 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab7c004870 0x7fab7c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9b7fe700 1 --2- 172.21.15.146:0/2204391733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7c005230 0x7fab7c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9affd700 1 -- 172.21.15.146:0/2204391733 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab7c004870 msgr2=0x7fab7c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9affd700 1 --2- 172.21.15.146:0/2204391733 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab7c004870 0x7fab7c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9affd700 1 -- 172.21.15.146:0/2204391733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7c005230 msgr2=0x7fab7c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9affd700 1 --2- 172.21.15.146:0/2204391733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7c005230 0x7fab7c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:47.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9affd700 1 -- 172.21.15.146:0/2204391733 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c0035d0 con 0x7fab7c003a90 2024-09-13T18:29:47.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9b7fe700 1 --2- 172.21.15.146:0/2204391733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7c005230 0x7fab7c00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:29:47.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9b7fe700 1 -- 172.21.15.146:0/2204391733 reap_dead start 2024-09-13T18:29:47.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9affd700 1 --2- 172.21.15.146:0/2204391733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c003a90 0x7fab7c003e70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0018f0 tx=0x7fab7c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4506b6366360d79 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:47.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c001390 con 0x7fab7c003a90 2024-09-13T18:29:47.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c0052d0 con 0x7fab7c003a90 2024-09-13T18:29:47.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab99ffb700 1 -- 172.21.15.146:0/2204391733 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0036a0 con 0x7fab7c003a90 2024-09-13T18:29:47.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9bfff700 1 -- 172.21.15.146:0/2204391733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c003a90 msgr2=0x7fab7c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:47.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9bfff700 1 --2- 172.21.15.146:0/2204391733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c003a90 0x7fab7c003e70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0018f0 tx=0x7fab7c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:29:47.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9bfff700 1 -- 172.21.15.146:0/2204391733 shutdown_connections 2024-09-13T18:29:47.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9bfff700 1 --2- 172.21.15.146:0/2204391733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c003a90 0x7fab7c003e70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:47.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9bfff700 1 -- 172.21.15.146:0/2204391733 >> 172.21.15.146:0/2204391733 conn(0x7fab9c071a90 msgr2=0x7fab9c071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:47.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9bfff700 1 -- 172.21.15.146:0/2204391733 shutdown_connections 2024-09-13T18:29:47.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.142+0000 7fab9bfff700 1 -- 172.21.15.146:0/2204391733 wait complete. 2024-09-13T18:29:47.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9bfff700 1 Processor -- start 2024-09-13T18:29:47.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9bfff700 1 -- start start 2024-09-13T18:29:47.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c003a90 0x7fab9c150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:47.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab9c153110 0x7fab9c147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:47.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab9c1f7080 0x7fab9c14c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:47.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9bfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c148530 con 0x7fab7c003a90 2024-09-13T18:29:47.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9bfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c1523e0 con 0x7fab9c1f7080 2024-09-13T18:29:47.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c153570 con 0x7fab9c153110 2024-09-13T18:29:47.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c003a90 0x7fab9c150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:47.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c003a90 0x7fab9c150ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46842/0 (socket says 172.21.15.146:46842) 2024-09-13T18:29:47.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9affd700 1 -- 172.21.15.146:0/1219836695 learned_addr learned my addr 172.21.15.146:0/1219836695 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:47.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9a7fc700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab9c153110 0x7fab9c147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:47.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9b7fe700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab9c1f7080 0x7fab9c14c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:47.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9affd700 1 -- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab9c153110 msgr2=0x7fab9c147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:47.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9affd700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab9c153110 0x7fab9c147c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:47.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9affd700 1 -- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab9c1f7080 msgr2=0x7fab9c14c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:47.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9affd700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab9c1f7080 0x7fab9c14c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:47.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.146+0000 7fab9affd700 1 -- 172.21.15.146:0/1219836695 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c0cb850 con 0x7fab7c003a90 2024-09-13T18:29:47.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.150+0000 7fab9affd700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c003a90 0x7fab9c150ce0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00c0e0 tx=0x7fab8c001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:47.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.150+0000 7fab93fff700 1 -- 172.21.15.146:0/1219836695 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c00c490 con 0x7fab7c003a90 2024-09-13T18:29:47.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.150+0000 7fab93fff700 1 -- 172.21.15.146:0/1219836695 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c00c5f0 con 0x7fab7c003a90 2024-09-13T18:29:47.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.150+0000 7fab93fff700 1 -- 172.21.15.146:0/1219836695 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c00c9a0 con 0x7fab7c003a90 2024-09-13T18:29:47.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.150+0000 7fab9bfff700 1 -- 172.21.15.146:0/1219836695 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c0cbb30 con 0x7fab7c003a90 2024-09-13T18:29:47.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.150+0000 7fab9bfff700 1 -- 172.21.15.146:0/1219836695 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c152910 con 0x7fab7c003a90 2024-09-13T18:29:47.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.150+0000 7fab9bfff700 1 -- 172.21.15.146:0/1219836695 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c06f010 con 0x7fab7c003a90 2024-09-13T18:29:47.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.150+0000 7fab93fff700 1 -- 172.21.15.146:0/1219836695 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fab8c00cb00 con 0x7fab7c003a90 2024-09-13T18:29:47.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.150+0000 7fab93fff700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab80066730 0x7fab80068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:47.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.150+0000 7fab93fff700 1 -- 172.21.15.146:0/1219836695 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fab8c085330 con 0x7fab7c003a90 2024-09-13T18:29:47.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.154+0000 7fab9a7fc700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab80066730 0x7fab80068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:47.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.154+0000 7fab9a7fc700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab80066730 0x7fab80068bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab9c06d7c0 tx=0x7fab84007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:47.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.158+0000 7fab93fff700 1 -- 172.21.15.146:0/1219836695 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fab8c052220 con 0x7fab7c003a90 2024-09-13T18:29:47.251 INFO:tasks.ceph.mds.c.smithi146.stderr:2024-09-13T18:29:47.242+0000 7f6be472a700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-13T18:29:47.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.366+0000 7fab9bfff700 1 -- 172.21.15.146:0/1219836695 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab9c074480 con 0x7fab7c003a90 2024-09-13T18:29:47.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.366+0000 7fab93fff700 1 -- 172.21.15.146:0/1219836695 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5528 (secure 0 0 0) 0x7fab8c055840 con 0x7fab7c003a90 2024-09-13T18:29:47.376 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:47.376 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:46.309663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:47.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.370+0000 7fab9bfff700 1 -- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab80066730 msgr2=0x7fab80068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:47.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.370+0000 7fab9bfff700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab80066730 0x7fab80068bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab9c06d7c0 tx=0x7fab84007040 comp rx=0 tx=0).stop 2024-09-13T18:29:47.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.370+0000 7fab9bfff700 1 -- 172.21.15.146:0/1219836695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c003a90 msgr2=0x7fab9c150ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:47.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.370+0000 7fab9bfff700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c003a90 0x7fab9c150ce0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00c0e0 tx=0x7fab8c001230 comp rx=0 tx=0).stop 2024-09-13T18:29:47.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.370+0000 7fab9bfff700 1 -- 172.21.15.146:0/1219836695 shutdown_connections 2024-09-13T18:29:47.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.370+0000 7fab9bfff700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab80066730 0x7fab80068bf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:47.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.370+0000 7fab9bfff700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab9c1f7080 0x7fab9c14c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:47.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.370+0000 7fab9bfff700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab9c153110 0x7fab9c147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:47.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.370+0000 7fab9bfff700 1 --2- 172.21.15.146:0/1219836695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c003a90 0x7fab9c150ce0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:47.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.370+0000 7fab9bfff700 1 -- 172.21.15.146:0/1219836695 >> 172.21.15.146:0/1219836695 conn(0x7fab9c071a90 msgr2=0x7fab9c071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:47.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.370+0000 7fab9bfff700 1 -- 172.21.15.146:0/1219836695 shutdown_connections 2024-09-13T18:29:47.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:47.370+0000 7fab9bfff700 1 -- 172.21.15.146:0/1219836695 wait complete. 2024-09-13T18:29:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 26 2024-09-13T18:29:47.390 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=2, replacing gid=4247 2024-09-13T18:29:47.390 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.f 2024-09-13T18:29:47.609 INFO:tasks.workunit.client.0.smithi146.stdout:checking sys/time.h usability... yes 2024-09-13T18:29:47.711 INFO:tasks.workunit.client.0.smithi146.stdout:checking sys/time.h presence... yes 2024-09-13T18:29:47.727 INFO:tasks.workunit.client.0.smithi146.stdout:checking for sys/time.h... yes 2024-09-13T18:29:47.779 INFO:tasks.workunit.client.0.smithi146.stdout:checking for unistd.h... (cached) yes 2024-09-13T18:29:47.960 INFO:tasks.workunit.client.0.smithi146.stdout:checking sys/vfs.h usability... yes 2024-09-13T18:29:49.369 INFO:tasks.workunit.client.0.smithi146.stdout:checking sys/vfs.h presence... yes 2024-09-13T18:29:51.320 INFO:tasks.workunit.client.0.smithi146.stdout:checking for sys/vfs.h... yes 2024-09-13T18:29:51.391 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2024-09-13T18:29:51.391 INFO:tasks.ceph.mds.f:Restarting daemon 2024-09-13T18:29:51.392 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-09-13T18:29:51.395 INFO:tasks.ceph.mds.f:Started 2024-09-13T18:29:51.395 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:51.485 INFO:tasks.ceph.mds.f.smithi154.stdout:starting mds.f at 2024-09-13T18:29:51.497 INFO:tasks.workunit.client.0.smithi146.stdout:checking sys/limits.h usability... no 2024-09-13T18:29:51.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.510+0000 7f4caec93700 1 Processor -- start 2024-09-13T18:29:51.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.510+0000 7f4caec93700 1 -- start start 2024-09-13T18:29:51.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.510+0000 7f4caec93700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ca80c61b0 0x7f4ca815d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:51.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.510+0000 7f4caec93700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca8066780 con 0x7f4ca80c5440 2024-09-13T18:29:51.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.510+0000 7f4caec93700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca815d960 con 0x7f4ca80c6590 2024-09-13T18:29:51.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.510+0000 7f4caec93700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca81594a0 con 0x7f4ca80c61b0 2024-09-13T18:29:51.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.510+0000 7f4cadc91700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ca80c61b0 0x7f4ca815d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:51.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.510+0000 7f4cadc91700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ca80c61b0 0x7f4ca815d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51658/0 (socket says 172.21.15.146:51658) 2024-09-13T18:29:51.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.510+0000 7f4cadc91700 1 -- 172.21.15.146:0/1554712705 learned_addr learned my addr 172.21.15.146:0/1554712705 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:51.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.510+0000 7f4cae492700 1 --1- 172.21.15.146:0/1554712705 >> v1:172.21.15.146:6789/0 conn(0x7f4ca80c5440 0x7f4ca8153340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58586/0 (socket says 172.21.15.146:58586) 2024-09-13T18:29:51.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4cadc91700 1 -- 172.21.15.146:0/1554712705 >> v1:172.21.15.154:6789/0 conn(0x7f4ca80c6590 legacy=0x7f4ca806cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:51.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4cadc91700 1 -- 172.21.15.146:0/1554712705 >> v1:172.21.15.146:6789/0 conn(0x7f4ca80c5440 legacy=0x7f4ca8153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:51.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4cadc91700 1 -- 172.21.15.146:0/1554712705 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca806eec0 con 0x7f4ca80c61b0 2024-09-13T18:29:51.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4cadc91700 1 --2- 172.21.15.146:0/1554712705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ca80c61b0 0x7f4ca815d420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009910 tx=0x7f4ca4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b8c758536e61b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:51.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4cacc8f700 1 -- 172.21.15.146:0/1554712705 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca400ba60 con 0x7f4ca80c61b0 2024-09-13T18:29:51.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4cacc8f700 1 -- 172.21.15.146:0/1554712705 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca400bbc0 con 0x7f4ca80c61b0 2024-09-13T18:29:51.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4cacc8f700 1 -- 172.21.15.146:0/1554712705 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4009e60 con 0x7f4ca80c61b0 2024-09-13T18:29:51.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4caec93700 1 -- 172.21.15.146:0/1554712705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ca80c61b0 msgr2=0x7f4ca815d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:51.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4caec93700 1 --2- 172.21.15.146:0/1554712705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ca80c61b0 0x7f4ca815d420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009910 tx=0x7f4ca4009c20 comp rx=0 tx=0).stop 2024-09-13T18:29:51.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4caec93700 1 -- 172.21.15.146:0/1554712705 shutdown_connections 2024-09-13T18:29:51.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4caec93700 1 --2- 172.21.15.146:0/1554712705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ca80c61b0 0x7f4ca815d420 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:51.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4caec93700 1 -- 172.21.15.146:0/1554712705 >> 172.21.15.146:0/1554712705 conn(0x7f4ca8066b00 msgr2=0x7f4ca806aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:51.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4caec93700 1 -- 172.21.15.146:0/1554712705 shutdown_connections 2024-09-13T18:29:51.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.514+0000 7f4caec93700 1 -- 172.21.15.146:0/1554712705 wait complete. 2024-09-13T18:29:51.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4caec93700 1 Processor -- start 2024-09-13T18:29:51.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4caec93700 1 -- start start 2024-09-13T18:29:51.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4caec93700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ca80c5440 0x7f4ca80cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:51.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4caec93700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ca80c61b0 0x7f4ca81e3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:51.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4caec93700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ca80c6590 0x7f4ca81514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:51.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4caec93700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca81e2810 con 0x7f4ca80c61b0 2024-09-13T18:29:51.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4caec93700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca81e2fb0 con 0x7f4ca80c5440 2024-09-13T18:29:51.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4caec93700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca81e29f0 con 0x7f4ca80c6590 2024-09-13T18:29:51.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4cad490700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ca80c61b0 0x7f4ca81e3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4cad490700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ca80c61b0 0x7f4ca81e3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51668/0 (socket says 172.21.15.146:51668) 2024-09-13T18:29:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4cad490700 1 -- 172.21.15.146:0/1774215408 learned_addr learned my addr 172.21.15.146:0/1774215408 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4cadc91700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ca80c5440 0x7f4ca80cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4cae492700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ca80c6590 0x7f4ca81514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4cad490700 1 -- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ca80c6590 msgr2=0x7f4ca81514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4cad490700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ca80c6590 0x7f4ca81514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4cad490700 1 -- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ca80c5440 msgr2=0x7f4ca80cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4cad490700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ca80c5440 0x7f4ca80cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4cad490700 1 -- 172.21.15.146:0/1774215408 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca4009570 con 0x7f4ca80c61b0 2024-09-13T18:29:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4cad490700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ca80c61b0 0x7f4ca81e3900 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c410 tx=0x7f4c9800c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4c9e7fc700 1 -- 172.21.15.146:0/1774215408 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980052d0 con 0x7f4ca80c61b0 2024-09-13T18:29:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4c9e7fc700 1 -- 172.21.15.146:0/1774215408 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c98005450 con 0x7f4ca80c61b0 2024-09-13T18:29:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4caec93700 1 -- 172.21.15.146:0/1774215408 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca81e2cd0 con 0x7f4ca80c61b0 2024-09-13T18:29:51.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4c9e7fc700 1 -- 172.21.15.146:0/1774215408 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980038e0 con 0x7f4ca80c61b0 2024-09-13T18:29:51.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.518+0000 7f4caec93700 1 -- 172.21.15.146:0/1774215408 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca8152a90 con 0x7f4ca80c61b0 2024-09-13T18:29:51.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.522+0000 7f4caec93700 1 -- 172.21.15.146:0/1774215408 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca8159f50 con 0x7f4ca80c61b0 2024-09-13T18:29:51.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.522+0000 7f4c9e7fc700 1 -- 172.21.15.146:0/1774215408 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4c98003a40 con 0x7f4ca80c61b0 2024-09-13T18:29:51.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.522+0000 7f4c9e7fc700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c8c066730 0x7f4c8c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:51.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.522+0000 7f4c9e7fc700 1 -- 172.21.15.146:0/1774215408 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f4c98083660 con 0x7f4ca80c61b0 2024-09-13T18:29:51.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.522+0000 7f4cadc91700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c8c066730 0x7f4c8c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:51.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.522+0000 7f4cadc91700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c8c066730 0x7f4c8c068bf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4006f80 tx=0x7f4ca4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:51.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.526+0000 7f4c9e7fc700 1 -- 172.21.15.146:0/1774215408 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4c980505f0 con 0x7f4ca80c61b0 2024-09-13T18:29:51.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.742+0000 7f4caec93700 1 -- 172.21.15.146:0/1774215408 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ca815d960 con 0x7f4ca80c61b0 2024-09-13T18:29:51.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.742+0000 7f4c9e7fc700 1 -- 172.21.15.146:0/1774215408 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5528 (secure 0 0 0) 0x7f4ca815d960 con 0x7f4ca80c61b0 2024-09-13T18:29:51.750 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:51.750 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:46.309663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:51.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.742+0000 7f4caec93700 1 -- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c8c066730 msgr2=0x7f4c8c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:51.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.742+0000 7f4caec93700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c8c066730 0x7f4c8c068bf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4006f80 tx=0x7f4ca4008030 comp rx=0 tx=0).stop 2024-09-13T18:29:51.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.742+0000 7f4caec93700 1 -- 172.21.15.146:0/1774215408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ca80c61b0 msgr2=0x7f4ca81e3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:51.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.742+0000 7f4caec93700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ca80c61b0 0x7f4ca81e3900 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c410 tx=0x7f4c9800c720 comp rx=0 tx=0).stop 2024-09-13T18:29:51.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.746+0000 7f4caec93700 1 -- 172.21.15.146:0/1774215408 shutdown_connections 2024-09-13T18:29:51.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.746+0000 7f4caec93700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c8c066730 0x7f4c8c068bf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:51.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.746+0000 7f4caec93700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ca80c6590 0x7f4ca81514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:51.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.746+0000 7f4caec93700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ca80c61b0 0x7f4ca81e3900 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:51.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.746+0000 7f4caec93700 1 --2- 172.21.15.146:0/1774215408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ca80c5440 0x7f4ca80cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:51.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.746+0000 7f4caec93700 1 -- 172.21.15.146:0/1774215408 >> 172.21.15.146:0/1774215408 conn(0x7f4ca8066b00 msgr2=0x7f4ca806d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:51.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.746+0000 7f4caec93700 1 -- 172.21.15.146:0/1774215408 shutdown_connections 2024-09-13T18:29:51.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:51.746+0000 7f4caec93700 1 -- 172.21.15.146:0/1774215408 wait complete. 2024-09-13T18:29:51.754 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 26 2024-09-13T18:29:51.765 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2024-09-13T18:29:52.531 INFO:tasks.workunit.client.0.smithi146.stdout:checking sys/limits.h presence... no 2024-09-13T18:29:52.844 INFO:tasks.workunit.client.0.smithi146.stdout:checking for sys/limits.h... no 2024-09-13T18:29:52.983 INFO:tasks.workunit.client.0.smithi146.stdout:checking for an ANSI C-conforming const... yes 2024-09-13T18:29:53.223 INFO:tasks.workunit.client.0.smithi146.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-13T18:29:53.453 INFO:tasks.workunit.client.0.smithi146.stdout:checking for system... yes 2024-09-13T18:29:53.677 INFO:tasks.workunit.client.0.smithi146.stdout:checking for gettimeofday... yes 2024-09-13T18:29:53.765 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.886+0000 7f481bfff700 1 Processor -- start 2024-09-13T18:29:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.886+0000 7f481bfff700 1 -- start start 2024-09-13T18:29:53.897 INFO:tasks.workunit.client.0.smithi146.stdout:checking for mkdir... yes 2024-09-13T18:29:53.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f481c150230 0x7f481c06ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:53.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f481c150610 0x7f481c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:53.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481bfff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f481c071410 con 0x7f481c071610 2024-09-13T18:29:53.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481bfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481c1596b0 con 0x7f481c150230 2024-09-13T18:29:53.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c156090 con 0x7f481c150610 2024-09-13T18:29:53.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481affd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f481c150230 0x7f481c06ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:53.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481affd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f481c150230 0x7f481c06ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51676/0 (socket says 172.21.15.146:51676) 2024-09-13T18:29:53.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481affd700 1 -- 172.21.15.146:0/1917606916 learned_addr learned my addr 172.21.15.146:0/1917606916 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:53.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481a7fc700 1 --2- 172.21.15.146:0/1917606916 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f481c150610 0x7f481c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:53.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481affd700 1 -- 172.21.15.146:0/1917606916 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f481c150610 msgr2=0x7f481c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:53.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481affd700 1 --2- 172.21.15.146:0/1917606916 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f481c150610 0x7f481c06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:53.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481affd700 1 -- 172.21.15.146:0/1917606916 >> v1:172.21.15.154:6789/0 conn(0x7f481c071610 legacy=0x7f481c159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:53.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481affd700 1 -- 172.21.15.146:0/1917606916 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c156350 con 0x7f481c150230 2024-09-13T18:29:53.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481a7fc700 1 --2- 172.21.15.146:0/1917606916 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f481c150610 0x7f481c06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:29:53.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481affd700 1 --2- 172.21.15.146:0/1917606916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481c150230 0x7f481c06ebd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f480c009910 tx=0x7f480c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81261260d06b0169 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:53.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f4819ffb700 1 -- 172.21.15.146:0/1917606916 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c00ba60 con 0x7f481c150230 2024-09-13T18:29:53.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f4819ffb700 1 -- 172.21.15.146:0/1917606916 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c00bbc0 con 0x7f481c150230 2024-09-13T18:29:53.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f4819ffb700 1 -- 172.21.15.146:0/1917606916 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c009e60 con 0x7f481c150230 2024-09-13T18:29:53.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481bfff700 1 -- 172.21.15.146:0/1917606916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481c150230 msgr2=0x7f481c06ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:53.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481bfff700 1 --2- 172.21.15.146:0/1917606916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481c150230 0x7f481c06ebd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f480c009910 tx=0x7f480c009c20 comp rx=0 tx=0).stop 2024-09-13T18:29:53.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481bfff700 1 -- 172.21.15.146:0/1917606916 shutdown_connections 2024-09-13T18:29:53.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481bfff700 1 --2- 172.21.15.146:0/1917606916 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f481c150610 0x7f481c06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:53.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481bfff700 1 --2- 172.21.15.146:0/1917606916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481c150230 0x7f481c06ebd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:53.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481bfff700 1 -- 172.21.15.146:0/1917606916 >> 172.21.15.146:0/1917606916 conn(0x7f481c071ed0 msgr2=0x7f481c0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:53.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481bfff700 1 -- 172.21.15.146:0/1917606916 shutdown_connections 2024-09-13T18:29:53.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.890+0000 7f481bfff700 1 -- 172.21.15.146:0/1917606916 wait complete. 2024-09-13T18:29:53.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481bfff700 1 Processor -- start 2024-09-13T18:29:53.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481bfff700 1 -- start start 2024-09-13T18:29:53.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481c071610 0x7f481c158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:53.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f481c150230 0x7f481c1eb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:53.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f481c153440 0x7f481c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:53.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481bfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c1529d0 con 0x7f481c071610 2024-09-13T18:29:53.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481bfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c155770 con 0x7f481c150230 2024-09-13T18:29:53.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c1571f0 con 0x7f481c153440 2024-09-13T18:29:53.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481c071610 0x7f481c158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:53.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481c071610 0x7f481c158e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51692/0 (socket says 172.21.15.146:51692) 2024-09-13T18:29:53.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481affd700 1 -- 172.21.15.146:0/2306337495 learned_addr learned my addr 172.21.15.146:0/2306337495 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:29:53.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481a7fc700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f481c150230 0x7f481c1eb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:53.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481b7fe700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f481c153440 0x7f481c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:53.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481affd700 1 -- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f481c153440 msgr2=0x7f481c0cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:53.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481affd700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f481c153440 0x7f481c0cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:53.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481affd700 1 -- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f481c150230 msgr2=0x7f481c1eb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:53.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481affd700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f481c150230 0x7f481c1eb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:53.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481affd700 1 -- 172.21.15.146:0/2306337495 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c009570 con 0x7f481c071610 2024-09-13T18:29:53.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481affd700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481c071610 0x7f481c158e50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f480c000c00 tx=0x7f480c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:53.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f48037fe700 1 -- 172.21.15.146:0/2306337495 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c004550 con 0x7f481c071610 2024-09-13T18:29:53.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f48037fe700 1 -- 172.21.15.146:0/2306337495 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c01f070 con 0x7f481c071610 2024-09-13T18:29:53.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481bfff700 1 -- 172.21.15.146:0/2306337495 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481c155a10 con 0x7f481c071610 2024-09-13T18:29:53.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f48037fe700 1 -- 172.21.15.146:0/2306337495 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c003900 con 0x7f481c071610 2024-09-13T18:29:53.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.894+0000 7f481bfff700 1 -- 172.21.15.146:0/2306337495 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c152c40 con 0x7f481c071610 2024-09-13T18:29:53.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.898+0000 7f481bfff700 1 -- 172.21.15.146:0/2306337495 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481c06ebd0 con 0x7f481c071610 2024-09-13T18:29:53.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.898+0000 7f48037fe700 1 -- 172.21.15.146:0/2306337495 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f480c003a60 con 0x7f481c071610 2024-09-13T18:29:53.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.898+0000 7f48037fe700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4808066760 0x7f4808068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:53.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.898+0000 7f48037fe700 1 -- 172.21.15.146:0/2306337495 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f480c021070 con 0x7f481c071610 2024-09-13T18:29:53.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.898+0000 7f481a7fc700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4808066760 0x7f4808068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:53.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.898+0000 7f481a7fc700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4808066760 0x7f4808068c20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f481c0c59a0 tx=0x7f4810008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:53.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:53.906+0000 7f48037fe700 1 -- 172.21.15.146:0/2306337495 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f480c08c080 con 0x7f481c071610 2024-09-13T18:29:54.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.114+0000 7f481bfff700 1 -- 172.21.15.146:0/2306337495 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f481c14a280 con 0x7f481c071610 2024-09-13T18:29:54.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.114+0000 7f48037fe700 1 -- 172.21.15.146:0/2306337495 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f480c015020 con 0x7f481c071610 2024-09-13T18:29:54.123 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:29:54.123 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5278,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:46.309663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:54.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.118+0000 7f481bfff700 1 -- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4808066760 msgr2=0x7f4808068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:54.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.118+0000 7f481bfff700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4808066760 0x7f4808068c20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f481c0c59a0 tx=0x7f4810008040 comp rx=0 tx=0).stop 2024-09-13T18:29:54.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.118+0000 7f481bfff700 1 -- 172.21.15.146:0/2306337495 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481c071610 msgr2=0x7f481c158e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:54.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.118+0000 7f481bfff700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481c071610 0x7f481c158e50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f480c000c00 tx=0x7f480c004770 comp rx=0 tx=0).stop 2024-09-13T18:29:54.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.118+0000 7f481bfff700 1 -- 172.21.15.146:0/2306337495 shutdown_connections 2024-09-13T18:29:54.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.118+0000 7f481bfff700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4808066760 0x7f4808068c20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:54.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.118+0000 7f481bfff700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f481c153440 0x7f481c0cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:54.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.118+0000 7f481bfff700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f481c150230 0x7f481c1eb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:54.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.118+0000 7f481bfff700 1 --2- 172.21.15.146:0/2306337495 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481c071610 0x7f481c158e50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:54.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.118+0000 7f481bfff700 1 -- 172.21.15.146:0/2306337495 >> 172.21.15.146:0/2306337495 conn(0x7f481c071ed0 msgr2=0x7f481c14b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:54.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.118+0000 7f481bfff700 1 -- 172.21.15.146:0/2306337495 shutdown_connections 2024-09-13T18:29:54.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:29:54.118+0000 7f481bfff700 1 -- 172.21.15.146:0/2306337495 wait complete. 2024-09-13T18:29:54.127 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 27 2024-09-13T18:29:54.137 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2024-09-13T18:29:54.138 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 24 secs before thrashing 2024-09-13T18:29:55.735 INFO:tasks.workunit.client.0.smithi146.stdout:checking for strerror... yes 2024-09-13T18:29:56.467 INFO:tasks.workunit.client.0.smithi146.stdout:checking for open64... yes 2024-09-13T18:29:58.356 INFO:tasks.workunit.client.0.smithi146.stdout:checking for stat64... yes 2024-09-13T18:29:58.611 INFO:tasks.workunit.client.0.smithi146.stdout:checking for fseeko64... yes 2024-09-13T18:29:58.828 INFO:tasks.workunit.client.0.smithi146.stdout:checking for lrand48_r... yes 2024-09-13T18:29:59.167 INFO:tasks.workunit.client.0.smithi146.stdout:checking for srand48_r... yes 2024-09-13T18:30:00.007 INFO:tasks.workunit.client.0.smithi146.stdout:configure: creating ./config.status 2024-09-13T18:30:03.745 INFO:tasks.workunit.client.0.smithi146.stdout: /bin/bash ./config.status 2024-09-13T18:30:03.923 INFO:tasks.workunit.client.0.smithi146.stdout:config.status: creating Makefile 2024-09-13T18:30:04.862 INFO:tasks.workunit.client.0.smithi146.stdout:config.status: creating config.h 2024-09-13T18:30:05.541 INFO:tasks.workunit.client.0.smithi146.stdout:config.status: executing depfiles commands 2024-09-13T18:30:05.596 INFO:tasks.workunit.client.0.smithi146.stdout:config.status: executing default commands 2024-09-13T18:30:05.682 INFO:tasks.workunit.client.0.smithi146.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-09-13T18:30:05.934 INFO:tasks.workunit.client.0.smithi146.stdout:rm -f stamp-h1 2024-09-13T18:30:05.944 INFO:tasks.workunit.client.0.smithi146.stdout:touch config.h.in 2024-09-13T18:30:05.955 INFO:tasks.workunit.client.0.smithi146.stdout:cd . && /bin/bash ./config.status config.h 2024-09-13T18:30:06.155 INFO:tasks.workunit.client.0.smithi146.stdout:config.status: creating config.h 2024-09-13T18:30:06.235 INFO:tasks.workunit.client.0.smithi146.stdout:make all-am 2024-09-13T18:30:06.286 INFO:tasks.workunit.client.0.smithi146.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-13T18:30:06.287 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-09-13T18:30:06.589 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-09-13T18:30:06.603 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-09-13T18:30:06.962 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-09-13T18:30:06.979 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-09-13T18:30:07.510 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:30:07.514 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:30:08.461 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-09-13T18:30:08.482 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-09-13T18:30:09.102 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-09-13T18:30:09.123 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-09-13T18:30:09.611 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-09-13T18:30:09.629 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-09-13T18:30:09.943 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-09-13T18:30:09.959 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-09-13T18:30:10.956 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-09-13T18:30:10.975 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-09-13T18:30:12.268 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-09-13T18:30:12.283 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-09-13T18:30:13.330 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-09-13T18:30:13.346 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-09-13T18:30:13.673 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-09-13T18:30:13.690 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-09-13T18:30:14.883 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-09-13T18:30:14.898 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-09-13T18:30:15.286 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-09-13T18:30:15.303 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-09-13T18:30:16.523 INFO:tasks.workunit.client.0.smithi146.stderr:ffsb_op.c: In function 'print_op_results': 2024-09-13T18:30:16.523 INFO:tasks.workunit.client.0.smithi146.stderr:ffsb_op.c:83:23: warning: embedded '\0' in format [-Wformat-contains-nul] 2024-09-13T18:30:16.523 INFO:tasks.workunit.client.0.smithi146.stderr: 83 | sprintf(buf, "%s/sec\0", buf); 2024-09-13T18:30:16.524 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~ 2024-09-13T18:30:16.524 INFO:tasks.workunit.client.0.smithi146.stderr:ffsb_op.c:86:19: warning: embedded '\0' in format [-Wformat-contains-nul] 2024-09-13T18:30:16.524 INFO:tasks.workunit.client.0.smithi146.stderr: 86 | sprintf(buf, "NA\0"); 2024-09-13T18:30:16.524 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~ 2024-09-13T18:30:16.545 INFO:tasks.workunit.client.0.smithi146.stderr:ffsb_op.c: In function 'print_results': 2024-09-13T18:30:16.545 INFO:tasks.workunit.client.0.smithi146.stderr:ffsb_op.c:83:19: warning: '/sec' directive writing 4 bytes into a region of size between 1 and 256 [-Wformat-overflow=] 2024-09-13T18:30:16.545 INFO:tasks.workunit.client.0.smithi146.stderr: 83 | sprintf(buf, "%s/sec\0", buf); 2024-09-13T18:30:16.545 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~ 2024-09-13T18:30:16.545 INFO:tasks.workunit.client.0.smithi146.stderr:In file included from /usr/include/stdio.h:867, 2024-09-13T18:30:16.545 INFO:tasks.workunit.client.0.smithi146.stderr: from ffsb_op.c:20: 2024-09-13T18:30:16.545 INFO:tasks.workunit.client.0.smithi146.stderr:/usr/include/x86_64-linux-gnu/bits/stdio2.h:36:10: note: '__builtin___sprintf_chk' output between 5 and 260 bytes into a destination of size 256 2024-09-13T18:30:16.546 INFO:tasks.workunit.client.0.smithi146.stderr: 36 | return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, 2024-09-13T18:30:16.546 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:16.546 INFO:tasks.workunit.client.0.smithi146.stderr: 37 | __bos (__s), __fmt, __va_arg_pack ()); 2024-09-13T18:30:16.546 INFO:tasks.workunit.client.0.smithi146.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:16.606 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-09-13T18:30:16.625 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-09-13T18:30:18.138 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:18.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.250+0000 7f619ffff700 1 Processor -- start 2024-09-13T18:30:18.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.250+0000 7f619ffff700 1 -- start start 2024-09-13T18:30:18.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f61980dcef0 0x7f61980e7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:18.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619ffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619804b840 con 0x7f61980509b0 2024-09-13T18:30:18.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619ffff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619804c370 con 0x7f61980d6650 2024-09-13T18:30:18.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619ffff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619804c610 con 0x7f61980dcef0 2024-09-13T18:30:18.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619e7fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f61980509b0 0x7f61980dc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56398/0 (socket says 172.21.15.146:56398) 2024-09-13T18:30:18.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619e7fc700 1 -- 172.21.15.146:0/720325890 learned_addr learned my addr 172.21.15.146:0/720325890 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:30:18.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619f7fe700 1 --2- 172.21.15.146:0/720325890 >> v2:172.21.15.146:3300/0 conn(0x7f61980dcef0 0x7f61980e7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:18.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619dffb700 1 -- 172.21.15.146:0/720325890 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445931809 0 0) 0x7f619804b840 con 0x7f61980509b0 2024-09-13T18:30:18.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619dffb700 1 -- 172.21.15.146:0/720325890 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f618c0035d0 con 0x7f61980509b0 2024-09-13T18:30:18.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619dffb700 1 -- 172.21.15.146:0/720325890 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207831024 0 0) 0x7f619804c370 con 0x7f61980d6650 2024-09-13T18:30:18.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619dffb700 1 -- 172.21.15.146:0/720325890 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f619804b840 con 0x7f61980d6650 2024-09-13T18:30:18.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619dffb700 1 -- 172.21.15.146:0/720325890 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4186349908 0 0) 0x7f618c0035d0 con 0x7f61980509b0 2024-09-13T18:30:18.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619f7fe700 1 -- 172.21.15.146:0/720325890 >> v1:172.21.15.154:6789/0 conn(0x7f61980d6650 legacy=0x7f6198044be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619f7fe700 1 -- 172.21.15.146:0/720325890 >> v1:172.21.15.146:6789/0 conn(0x7f61980509b0 legacy=0x7f61980dc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619f7fe700 1 -- 172.21.15.146:0/720325890 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619804af10 con 0x7f61980dcef0 2024-09-13T18:30:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619f7fe700 1 --2- 172.21.15.146:0/720325890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61980dcef0 0x7f61980e7e20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6188009850 tx=0x7f6188009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b43d75f18d38452 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619dffb700 1 -- 172.21.15.146:0/720325890 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190001670 con 0x7f61980dcef0 2024-09-13T18:30:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619dffb700 1 -- 172.21.15.146:0/720325890 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6188004030 con 0x7f61980dcef0 2024-09-13T18:30:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619dffb700 1 -- 172.21.15.146:0/720325890 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61880043a0 con 0x7f61980dcef0 2024-09-13T18:30:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619ffff700 1 -- 172.21.15.146:0/720325890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61980dcef0 msgr2=0x7f61980e7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619ffff700 1 --2- 172.21.15.146:0/720325890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61980dcef0 0x7f61980e7e20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6188009850 tx=0x7f6188009b60 comp rx=0 tx=0).stop 2024-09-13T18:30:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619ffff700 1 -- 172.21.15.146:0/720325890 shutdown_connections 2024-09-13T18:30:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619ffff700 1 --2- 172.21.15.146:0/720325890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61980dcef0 0x7f61980e7e20 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619ffff700 1 -- 172.21.15.146:0/720325890 >> 172.21.15.146:0/720325890 conn(0x7f619804fd00 msgr2=0x7f6198050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619ffff700 1 -- 172.21.15.146:0/720325890 shutdown_connections 2024-09-13T18:30:18.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.254+0000 7f619ffff700 1 -- 172.21.15.146:0/720325890 wait complete. 2024-09-13T18:30:18.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619ffff700 1 Processor -- start 2024-09-13T18:30:18.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619ffff700 1 -- start start 2024-09-13T18:30:18.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61980509b0 0x7f61981778d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:18.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61980d6650 0x7f6198177e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:18.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61980dcef0 0x7f6198048150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:18.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6198178450 con 0x7f61980d6650 2024-09-13T18:30:18.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619ffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61980d7e80 con 0x7f61980509b0 2024-09-13T18:30:18.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61980dd490 con 0x7f61980dcef0 2024-09-13T18:30:18.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61980d6650 0x7f6198177e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:18.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61980d6650 0x7f6198177e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33650/0 (socket says 172.21.15.146:33650) 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619e7fc700 1 -- 172.21.15.146:0/951912386 learned_addr learned my addr 172.21.15.146:0/951912386 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619f7fe700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61980dcef0 0x7f6198048150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619effd700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61980509b0 0x7f61981778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619e7fc700 1 -- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61980dcef0 msgr2=0x7f6198048150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619e7fc700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61980dcef0 0x7f6198048150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619e7fc700 1 -- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61980509b0 msgr2=0x7f61981778d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619e7fc700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61980509b0 0x7f61981778d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619e7fc700 1 -- 172.21.15.146:0/951912386 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61880094b0 con 0x7f61980d6650 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619e7fc700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61980d6650 0x7f6198177e10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f619000c770 tx=0x7f619000ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f6197fff700 1 -- 172.21.15.146:0/951912386 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190004ca0 con 0x7f61980d6650 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f6197fff700 1 -- 172.21.15.146:0/951912386 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6190004e00 con 0x7f61980d6650 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619ffff700 1 -- 172.21.15.146:0/951912386 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61980dd770 con 0x7f61980d6650 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f6197fff700 1 -- 172.21.15.146:0/951912386 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61900072d0 con 0x7f61980d6650 2024-09-13T18:30:18.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.258+0000 7f619ffff700 1 -- 172.21.15.146:0/951912386 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61980d7b10 con 0x7f61980d6650 2024-09-13T18:30:18.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.262+0000 7f619ffff700 1 -- 172.21.15.146:0/951912386 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61980d7ff0 con 0x7f61980d6650 2024-09-13T18:30:18.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.262+0000 7f6197fff700 1 -- 172.21.15.146:0/951912386 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6190007430 con 0x7f61980d6650 2024-09-13T18:30:18.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.262+0000 7f6197fff700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6180066730 0x7f6180068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:18.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.262+0000 7f6197fff700 1 -- 172.21.15.146:0/951912386 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f6190084320 con 0x7f61980d6650 2024-09-13T18:30:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.262+0000 7f619effd700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6180066730 0x7f6180068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:18.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.262+0000 7f619effd700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6180066730 0x7f6180068bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f61a004ffd0 tx=0x7f61a0061e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:18.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.270+0000 7f6197fff700 1 -- 172.21.15.146:0/951912386 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f61900511a0 con 0x7f61980d6650 2024-09-13T18:30:18.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.478+0000 7f619ffff700 1 -- 172.21.15.146:0/951912386 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61980073b0 con 0x7f61980d6650 2024-09-13T18:30:18.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.478+0000 7f6197fff700 1 -- 172.21.15.146:0/951912386 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f61900547c0 con 0x7f61980d6650 2024-09-13T18:30:18.489 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:18.489 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5278,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:46.309663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:18.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.482+0000 7f619ffff700 1 -- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6180066730 msgr2=0x7f6180068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:18.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.482+0000 7f619ffff700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6180066730 0x7f6180068bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f61a004ffd0 tx=0x7f61a0061e40 comp rx=0 tx=0).stop 2024-09-13T18:30:18.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.482+0000 7f619ffff700 1 -- 172.21.15.146:0/951912386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61980d6650 msgr2=0x7f6198177e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:18.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.482+0000 7f619ffff700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61980d6650 0x7f6198177e10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f619000c770 tx=0x7f619000ca80 comp rx=0 tx=0).stop 2024-09-13T18:30:18.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.482+0000 7f619ffff700 1 -- 172.21.15.146:0/951912386 shutdown_connections 2024-09-13T18:30:18.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.482+0000 7f619ffff700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6180066730 0x7f6180068bf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.482+0000 7f619ffff700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61980dcef0 0x7f6198048150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.482+0000 7f619ffff700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61980d6650 0x7f6198177e10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.482+0000 7f619ffff700 1 --2- 172.21.15.146:0/951912386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61980509b0 0x7f61981778d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.482+0000 7f619ffff700 1 -- 172.21.15.146:0/951912386 >> 172.21.15.146:0/951912386 conn(0x7f619804fd00 msgr2=0x7f619804e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:18.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.482+0000 7f619ffff700 1 -- 172.21.15.146:0/951912386 shutdown_connections 2024-09-13T18:30:18.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.482+0000 7f619ffff700 1 -- 172.21.15.146:0/951912386 wait complete. 2024-09-13T18:30:18.493 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 27 2024-09-13T18:30:18.503 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=1) 2024-09-13T18:30:18.503 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T18:30:18.503 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:30:18.517 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T18:30:18.518 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:18.608 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-09-13T18:30:18.625 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-09-13T18:30:18.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.622+0000 7f3f848d9700 1 Processor -- start 2024-09-13T18:30:18.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.622+0000 7f3f848d9700 1 -- start start 2024-09-13T18:30:18.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.622+0000 7f3f848d9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3f800ca210 0x7f3f800b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:18.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.622+0000 7f3f848d9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3f80071390 0x7f3f800cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:18.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.622+0000 7f3f848d9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f800cc670 con 0x7f3f800cbd50 2024-09-13T18:30:18.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.622+0000 7f3f848d9700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f800caa30 con 0x7f3f800ca210 2024-09-13T18:30:18.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.622+0000 7f3f848d9700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f800cb810 con 0x7f3f80071390 2024-09-13T18:30:18.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f7effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3f800ca210 0x7f3f800b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:18.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f7effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3f800ca210 0x7f3f800b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33664/0 (socket says 172.21.15.146:33664) 2024-09-13T18:30:18.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f7effd700 1 -- 172.21.15.146:0/418763526 learned_addr learned my addr 172.21.15.146:0/418763526 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:18.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f7ffff700 1 --2- 172.21.15.146:0/418763526 >> v2:172.21.15.154:3300/0 conn(0x7f3f80071390 0x7f3f800cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:18.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f7effd700 1 -- 172.21.15.146:0/418763526 >> v2:172.21.15.154:3300/0 conn(0x7f3f80071390 msgr2=0x7f3f800cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:18.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f7effd700 1 --2- 172.21.15.146:0/418763526 >> v2:172.21.15.154:3300/0 conn(0x7f3f80071390 0x7f3f800cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f7effd700 1 -- 172.21.15.146:0/418763526 >> v1:172.21.15.154:6789/0 conn(0x7f3f800cbd50 legacy=0x7f3f800701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:18.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f7effd700 1 -- 172.21.15.146:0/418763526 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8015da20 con 0x7f3f800ca210 2024-09-13T18:30:18.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f7effd700 1 --2- 172.21.15.146:0/418763526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f800ca210 0x7f3f800b1c40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3f800cce40 tx=0x7f3f74009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d734ca1bb8b22d55 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:18.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f7e7fc700 1 -- 172.21.15.146:0/418763526 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74004230 con 0x7f3f800ca210 2024-09-13T18:30:18.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f7e7fc700 1 -- 172.21.15.146:0/418763526 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f74004390 con 0x7f3f800ca210 2024-09-13T18:30:18.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f7e7fc700 1 -- 172.21.15.146:0/418763526 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f740036c0 con 0x7f3f800ca210 2024-09-13T18:30:18.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f848d9700 1 -- 172.21.15.146:0/418763526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f800ca210 msgr2=0x7f3f800b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:18.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f848d9700 1 --2- 172.21.15.146:0/418763526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f800ca210 0x7f3f800b1c40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3f800cce40 tx=0x7f3f74009bf0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f848d9700 1 -- 172.21.15.146:0/418763526 shutdown_connections 2024-09-13T18:30:18.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f848d9700 1 --2- 172.21.15.146:0/418763526 >> v2:172.21.15.154:3300/0 conn(0x7f3f80071390 0x7f3f800cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f848d9700 1 --2- 172.21.15.146:0/418763526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f800ca210 0x7f3f800b1c40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f848d9700 1 -- 172.21.15.146:0/418763526 >> 172.21.15.146:0/418763526 conn(0x7f3f80066270 msgr2=0x7f3f80066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:18.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f848d9700 1 -- 172.21.15.146:0/418763526 shutdown_connections 2024-09-13T18:30:18.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f848d9700 1 -- 172.21.15.146:0/418763526 wait complete. 2024-09-13T18:30:18.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f848d9700 1 Processor -- start 2024-09-13T18:30:18.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.626+0000 7f3f848d9700 1 -- start start 2024-09-13T18:30:18.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f848d9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f80071390 0x7f3f800b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:18.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f848d9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f800ca210 0x7f3f801e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:18.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f848d9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f800cbd50 0x7f3f801e2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:18.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f848d9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800c7240 con 0x7f3f800ca210 2024-09-13T18:30:18.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f848d9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800c68c0 con 0x7f3f80071390 2024-09-13T18:30:18.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f848d9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800c62b0 con 0x7f3f800cbd50 2024-09-13T18:30:18.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f7effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f800ca210 0x7f3f801e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:18.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f7effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f800ca210 0x7f3f801e45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33676/0 (socket says 172.21.15.146:33676) 2024-09-13T18:30:18.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f7effd700 1 -- 172.21.15.146:0/2321631618 learned_addr learned my addr 172.21.15.146:0/2321631618 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:18.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f7f7fe700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f80071390 0x7f3f800b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:18.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f7ffff700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f800cbd50 0x7f3f801e2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:18.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f7effd700 1 -- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f800cbd50 msgr2=0x7f3f801e2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:18.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f7effd700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f800cbd50 0x7f3f801e2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f7effd700 1 -- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f80071390 msgr2=0x7f3f800b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:18.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f7effd700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f80071390 0x7f3f800b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f7effd700 1 -- 172.21.15.146:0/2321631618 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f74009570 con 0x7f3f800ca210 2024-09-13T18:30:18.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f7effd700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f800ca210 0x7f3f801e45f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3f74004000 tx=0x7f3f74004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:18.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f67fff700 1 -- 172.21.15.146:0/2321631618 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74003db0 con 0x7f3f800ca210 2024-09-13T18:30:18.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f67fff700 1 -- 172.21.15.146:0/2321631618 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7401f070 con 0x7f3f800ca210 2024-09-13T18:30:18.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f848d9700 1 -- 172.21.15.146:0/2321631618 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8015c630 con 0x7f3f800ca210 2024-09-13T18:30:18.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f67fff700 1 -- 172.21.15.146:0/2321631618 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7400b8a0 con 0x7f3f800ca210 2024-09-13T18:30:18.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f848d9700 1 -- 172.21.15.146:0/2321631618 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f800c6550 con 0x7f3f800ca210 2024-09-13T18:30:18.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.630+0000 7f3f848d9700 1 -- 172.21.15.146:0/2321631618 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8015d500 con 0x7f3f800ca210 2024-09-13T18:30:18.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.634+0000 7f3f67fff700 1 -- 172.21.15.146:0/2321631618 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3f74021030 con 0x7f3f800ca210 2024-09-13T18:30:18.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.634+0000 7f3f67fff700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f6c066730 0x7f3f6c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:18.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.634+0000 7f3f67fff700 1 -- 172.21.15.146:0/2321631618 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f3f74085670 con 0x7f3f800ca210 2024-09-13T18:30:18.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.634+0000 7f3f7f7fe700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f6c066730 0x7f3f6c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:18.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.634+0000 7f3f7f7fe700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f6c066730 0x7f3f6c068bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009980 tx=0x7f3f700090e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:18.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.638+0000 7f3f67fff700 1 -- 172.21.15.146:0/2321631618 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3f74052610 con 0x7f3f800ca210 2024-09-13T18:30:18.781 INFO:tasks.workunit.client.0.smithi146.stderr:In file included from parser.h:22, 2024-09-13T18:30:18.782 INFO:tasks.workunit.client.0.smithi146.stderr: from parser.c:25: 2024-09-13T18:30:18.782 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c: In function 'init_filesys': 2024-09-13T18:30:18.784 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T18:30:18.784 INFO:tasks.workunit.client.0.smithi146.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T18:30:18.784 INFO:tasks.workunit.client.0.smithi146.stderr: | ^ 2024-09-13T18:30:18.784 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T18:30:18.784 INFO:tasks.workunit.client.0.smithi146.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T18:30:18.784 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~ 2024-09-13T18:30:18.784 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T18:30:18.784 INFO:tasks.workunit.client.0.smithi146.stderr: 36 | container_of(ptr, type, member) 2024-09-13T18:30:18.784 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~ 2024-09-13T18:30:18.784 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2024-09-13T18:30:18.784 INFO:tasks.workunit.client.0.smithi146.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T18:30:18.784 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~ 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c:934:3: note: in expansion of macro 'list_for_each_entry' 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr: | ^ 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~ 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr: 36 | container_of(ptr, type, member) 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~ 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2024-09-13T18:30:18.785 INFO:tasks.workunit.client.0.smithi146.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~ 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c:934:3: note: in expansion of macro 'list_for_each_entry' 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr: | ^ 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~ 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr: 36 | container_of(ptr, type, member) 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~ 2024-09-13T18:30:18.786 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~ 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c:941:3: note: in expansion of macro 'list_for_each_entry' 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr: | ^ 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~ 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr: 36 | container_of(ptr, type, member) 2024-09-13T18:30:18.787 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~ 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~ 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c:941:3: note: in expansion of macro 'list_for_each_entry' 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c: In function 'init_tg_stats': 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr: | ^ 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~ 2024-09-13T18:30:18.788 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr: 36 | container_of(ptr, type, member) 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~ 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~ 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c:971:6: note: in expansion of macro 'list_for_each_entry' 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr: | ^ 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~ 2024-09-13T18:30:18.789 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T18:30:18.790 INFO:tasks.workunit.client.0.smithi146.stderr: 36 | container_of(ptr, type, member) 2024-09-13T18:30:18.790 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~ 2024-09-13T18:30:18.790 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2024-09-13T18:30:18.790 INFO:tasks.workunit.client.0.smithi146.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T18:30:18.790 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~ 2024-09-13T18:30:18.790 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c:971:6: note: in expansion of macro 'list_for_each_entry' 2024-09-13T18:30:18.790 INFO:tasks.workunit.client.0.smithi146.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-13T18:30:18.790 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:18.790 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T18:30:18.790 INFO:tasks.workunit.client.0.smithi146.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T18:30:18.790 INFO:tasks.workunit.client.0.smithi146.stderr: | ^ 2024-09-13T18:30:18.790 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T18:30:18.790 INFO:tasks.workunit.client.0.smithi146.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~ 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr: 36 | container_of(ptr, type, member) 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~ 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~ 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c:980:6: note: in expansion of macro 'list_for_each_entry' 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr: | ^ 2024-09-13T18:30:18.791 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T18:30:18.792 INFO:tasks.workunit.client.0.smithi146.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T18:30:18.792 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~ 2024-09-13T18:30:18.792 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T18:30:18.792 INFO:tasks.workunit.client.0.smithi146.stderr: 36 | container_of(ptr, type, member) 2024-09-13T18:30:18.792 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~ 2024-09-13T18:30:18.792 INFO:tasks.workunit.client.0.smithi146.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2024-09-13T18:30:18.792 INFO:tasks.workunit.client.0.smithi146.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T18:30:18.792 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~ 2024-09-13T18:30:18.792 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c:980:6: note: in expansion of macro 'list_for_each_entry' 2024-09-13T18:30:18.792 INFO:tasks.workunit.client.0.smithi146.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-13T18:30:18.792 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:18.854 INFO:tasks.workunit.client.0.smithi146.stderr:In file included from /usr/include/string.h:495, 2024-09-13T18:30:18.854 INFO:tasks.workunit.client.0.smithi146.stderr: from parser.c:19: 2024-09-13T18:30:18.854 INFO:tasks.workunit.client.0.smithi146.stderr:In function 'memcpy', 2024-09-13T18:30:18.854 INFO:tasks.workunit.client.0.smithi146.stderr: inlined from 'size64_convert' at parser.c:119:2: 2024-09-13T18:30:18.854 INFO:tasks.workunit.client.0.smithi146.stderr:/usr/include/x86_64-linux-gnu/bits/string_fortified.h:34:10: warning: '__builtin_memcpy' reading 3 bytes from a region of size 2 [-Wstringop-overflow=] 2024-09-13T18:30:18.854 INFO:tasks.workunit.client.0.smithi146.stderr: 34 | return __builtin___memcpy_chk (__dest, __src, __len, __bos0 (__dest)); 2024-09-13T18:30:18.854 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:18.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.866+0000 7f3f848d9700 1 -- 172.21.15.146:0/2321631618 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f800cc670 con 0x7f3f800ca210 2024-09-13T18:30:18.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.866+0000 7f3f67fff700 1 -- 172.21.15.146:0/2321631618 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f3f800cc670 con 0x7f3f800ca210 2024-09-13T18:30:18.875 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:18.875 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5278,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:46.309663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:18.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.870+0000 7f3f848d9700 1 -- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f6c066730 msgr2=0x7f3f6c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:18.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.870+0000 7f3f848d9700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f6c066730 0x7f3f6c068bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009980 tx=0x7f3f700090e0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.870+0000 7f3f848d9700 1 -- 172.21.15.146:0/2321631618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f800ca210 msgr2=0x7f3f801e45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:18.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.870+0000 7f3f848d9700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f800ca210 0x7f3f801e45f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3f74004000 tx=0x7f3f74004770 comp rx=0 tx=0).stop 2024-09-13T18:30:18.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.870+0000 7f3f848d9700 1 -- 172.21.15.146:0/2321631618 shutdown_connections 2024-09-13T18:30:18.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.870+0000 7f3f848d9700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f6c066730 0x7f3f6c068bf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.870+0000 7f3f848d9700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f800cbd50 0x7f3f801e2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.870+0000 7f3f848d9700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f800ca210 0x7f3f801e45f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.870+0000 7f3f848d9700 1 --2- 172.21.15.146:0/2321631618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f80071390 0x7f3f800b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:18.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.870+0000 7f3f848d9700 1 -- 172.21.15.146:0/2321631618 >> 172.21.15.146:0/2321631618 conn(0x7f3f80066270 msgr2=0x7f3f800cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:18.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.870+0000 7f3f848d9700 1 -- 172.21.15.146:0/2321631618 shutdown_connections 2024-09-13T18:30:18.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:18.870+0000 7f3f848d9700 1 -- 172.21.15.146:0/2321631618 wait complete. 2024-09-13T18:30:18.879 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 27 2024-09-13T18:30:18.889 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:30:18.924 INFO:tasks.workunit.client.0.smithi146.stderr:In function 'strncpy', 2024-09-13T18:30:18.924 INFO:tasks.workunit.client.0.smithi146.stderr: inlined from 'set_weight' at parser.c:623:4: 2024-09-13T18:30:18.924 INFO:tasks.workunit.client.0.smithi146.stderr:/usr/include/x86_64-linux-gnu/bits/string_fortified.h:106:10: warning: '__builtin_strncpy' specified bound depends on the length of the source argument [-Wstringop-overflow=] 2024-09-13T18:30:18.924 INFO:tasks.workunit.client.0.smithi146.stderr: 106 | return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest)); 2024-09-13T18:30:18.924 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:18.924 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c: In function 'set_weight': 2024-09-13T18:30:18.924 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c:620:10: note: length computed here 2024-09-13T18:30:18.924 INFO:tasks.workunit.client.0.smithi146.stderr: 620 | len = strlen(tmp_config->name); 2024-09-13T18:30:18.924 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:18.930 INFO:tasks.workunit.client.0.smithi146.stderr:In file included from /usr/include/string.h:495, 2024-09-13T18:30:18.930 INFO:tasks.workunit.client.0.smithi146.stderr: from parser.c:19: 2024-09-13T18:30:18.930 INFO:tasks.workunit.client.0.smithi146.stderr:In function 'strncpy', 2024-09-13T18:30:18.930 INFO:tasks.workunit.client.0.smithi146.stderr: inlined from 'get_weight_total' at parser.c:645:4: 2024-09-13T18:30:18.930 INFO:tasks.workunit.client.0.smithi146.stderr:/usr/include/x86_64-linux-gnu/bits/string_fortified.h:106:10: warning: '__builtin_strncpy' specified bound depends on the length of the source argument [-Wstringop-overflow=] 2024-09-13T18:30:18.930 INFO:tasks.workunit.client.0.smithi146.stderr: 106 | return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest)); 2024-09-13T18:30:18.931 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:18.931 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c: In function 'get_weight_total': 2024-09-13T18:30:18.931 INFO:tasks.workunit.client.0.smithi146.stderr:parser.c:642:10: note: length computed here 2024-09-13T18:30:18.931 INFO:tasks.workunit.client.0.smithi146.stderr: 642 | len = strlen(tmp_config->name); 2024-09-13T18:30:18.931 INFO:tasks.workunit.client.0.smithi146.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T18:30:19.102 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-09-13T18:30:19.120 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-09-13T18:30:19.363 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-09-13T18:30:19.383 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-09-13T18:30:19.702 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-09-13T18:30:19.719 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-09-13T18:30:19.846 INFO:tasks.workunit.client.0.smithi146.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-09-13T18:30:19.865 INFO:tasks.workunit.client.0.smithi146.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-09-13T18:30:20.225 INFO:tasks.workunit.client.0.smithi146.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-13T18:30:20.233 INFO:tasks.workunit.client.0.smithi146.stdout:FFSB version 6.0-RC2 started 2024-09-13T18:30:20.234 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:30:20.238 INFO:tasks.workunit.client.0.smithi146.stdout:benchmark time = 300 2024-09-13T18:30:20.238 INFO:tasks.workunit.client.0.smithi146.stdout:ThreadGroup 0 2024-09-13T18:30:20.238 INFO:tasks.workunit.client.0.smithi146.stdout:================ 2024-09-13T18:30:20.238 INFO:tasks.workunit.client.0.smithi146.stdout: num_threads = 32 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: read_random = off 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: read_size = 0 (0B) 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: read_blocksize = 0 (0B) 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: read_skip = off 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: read_skipsize = 0 (0B) 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: write_random = on 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: write_size = 5242880 (5MB) 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: fsync_file = 0 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: write_blocksize = 4096 (4KB) 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: wait time = 0 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:30:20.239 INFO:tasks.workunit.client.0.smithi146.stdout: op weights 2024-09-13T18:30:20.240 INFO:tasks.workunit.client.0.smithi146.stdout: read = 0 (0.00%) 2024-09-13T18:30:20.240 INFO:tasks.workunit.client.0.smithi146.stdout: readall = 0 (0.00%) 2024-09-13T18:30:20.240 INFO:tasks.workunit.client.0.smithi146.stdout: write = 1 (100.00%) 2024-09-13T18:30:20.240 INFO:tasks.workunit.client.0.smithi146.stdout: create = 0 (0.00%) 2024-09-13T18:30:20.240 INFO:tasks.workunit.client.0.smithi146.stdout: append = 0 (0.00%) 2024-09-13T18:30:20.240 INFO:tasks.workunit.client.0.smithi146.stdout: delete = 0 (0.00%) 2024-09-13T18:30:20.240 INFO:tasks.workunit.client.0.smithi146.stdout: metaop = 0 (0.00%) 2024-09-13T18:30:20.240 INFO:tasks.workunit.client.0.smithi146.stdout: createdir = 0 (0.00%) 2024-09-13T18:30:20.240 INFO:tasks.workunit.client.0.smithi146.stdout: stat = 0 (0.00%) 2024-09-13T18:30:20.240 INFO:tasks.workunit.client.0.smithi146.stdout: writeall = 0 (0.00%) 2024-09-13T18:30:20.240 INFO:tasks.workunit.client.0.smithi146.stdout: writeall_fsync = 0 (0.00%) 2024-09-13T18:30:20.240 INFO:tasks.workunit.client.0.smithi146.stdout: open_close = 0 (0.00%) 2024-09-13T18:30:20.240 INFO:tasks.workunit.client.0.smithi146.stdout: write_fsync = 0 (0.00%) 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout: create_fsync = 0 (0.00%) 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout: append_fsync = 0 (0.00%) 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout:FileSystem . 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout:========== 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout: num_dirs = 0 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout: starting files = 128 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout: min file size = 104857600 (100MB) 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout: max file size = 104857600 (100MB) 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout: directio = off 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout: alignedio = on 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout: bufferedio = off 2024-09-13T18:30:20.241 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:30:20.242 INFO:tasks.workunit.client.0.smithi146.stdout: aging is off 2024-09-13T18:30:20.242 INFO:tasks.workunit.client.0.smithi146.stdout: current utilization = 0.04% 2024-09-13T18:30:20.242 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:30:20.242 INFO:tasks.workunit.client.0.smithi146.stdout:checking existing fs: . 2024-09-13T18:30:20.242 INFO:tasks.workunit.client.0.smithi146.stdout:recreating new fileset 2024-09-13T18:30:20.243 INFO:tasks.workunit.client.0.smithi146.stderr:opendir: No such file or directory 2024-09-13T18:30:20.891 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:21.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.006+0000 7f0a34d1a700 1 Processor -- start 2024-09-13T18:30:21.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.006+0000 7f0a34d1a700 1 -- start start 2024-09-13T18:30:21.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a34d1a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0a280d5510 0x7f0a28046700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:21.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a34d1a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a280df330 con 0x7f0a2804dd00 2024-09-13T18:30:21.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a34d1a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a280d58f0 con 0x7f0a2804e100 2024-09-13T18:30:21.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a34d1a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a2804e6e0 con 0x7f0a280d5510 2024-09-13T18:30:21.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2effd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0a2804dd00 0x7f0a28006750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56412/0 (socket says 172.21.15.146:56412) 2024-09-13T18:30:21.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2effd700 1 -- 172.21.15.146:0/3520550955 learned_addr learned my addr 172.21.15.146:0/3520550955 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:30:21.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2f7fe700 1 --2- 172.21.15.146:0/3520550955 >> v2:172.21.15.154:3300/0 conn(0x7f0a280d5510 0x7f0a28046700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:21.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977444660 0 0) 0x7f0a280df330 con 0x7f0a2804dd00 2024-09-13T18:30:21.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a200035d0 con 0x7f0a2804dd00 2024-09-13T18:30:21.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2004850217 0 0) 0x7f0a200035d0 con 0x7f0a2804dd00 2024-09-13T18:30:21.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 >> v2:172.21.15.154:3300/0 conn(0x7f0a280d5510 msgr2=0x7f0a28046700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:21.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 --2- 172.21.15.146:0/3520550955 >> v2:172.21.15.154:3300/0 conn(0x7f0a280d5510 0x7f0a28046700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:21.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 >> v1:172.21.15.154:6789/0 conn(0x7f0a2804e100 legacy=0x7f0a280e1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:21.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a280dea00 con 0x7f0a2804dd00 2024-09-13T18:30:21.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f0a1c001670 con 0x7f0a2804dd00 2024-09-13T18:30:21.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 >> v1:172.21.15.146:6789/0 conn(0x7f0a2804dd00 legacy=0x7f0a28006750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:21.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 --2- 172.21.15.146:0/3520550955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a20003a20 0x7f0a20003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:21.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 --2- 172.21.15.146:0/3520550955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a20004870 0x7f0a20004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:21.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 --2- 172.21.15.146:0/3520550955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a20005230 0x7f0a2000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:21.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2804e6e0 con 0x7f0a20003a20 2024-09-13T18:30:21.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2000c0d0 con 0x7f0a20005230 2024-09-13T18:30:21.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2000c290 con 0x7f0a20004870 2024-09-13T18:30:21.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2f7fe700 1 --2- 172.21.15.146:0/3520550955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a20003a20 0x7f0a20003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:21.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2ffff700 1 --2- 172.21.15.146:0/3520550955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a20005230 0x7f0a2000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:21.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2effd700 1 --2- 172.21.15.146:0/3520550955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a20004870 0x7f0a20004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:21.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2f7fe700 1 -- 172.21.15.146:0/3520550955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a20004870 msgr2=0x7f0a20004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:21.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2f7fe700 1 --2- 172.21.15.146:0/3520550955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a20004870 0x7f0a20004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:21.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2f7fe700 1 -- 172.21.15.146:0/3520550955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a20005230 msgr2=0x7f0a2000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:21.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2f7fe700 1 --2- 172.21.15.146:0/3520550955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a20005230 0x7f0a2000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:21.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2f7fe700 1 -- 172.21.15.146:0/3520550955 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a200035d0 con 0x7f0a20003a20 2024-09-13T18:30:21.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2ffff700 1 -- 172.21.15.146:0/3520550955 reap_dead start 2024-09-13T18:30:21.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2effd700 1 --2- 172.21.15.146:0/3520550955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a20004870 0x7f0a20004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:30:21.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2f7fe700 1 --2- 172.21.15.146:0/3520550955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a20003a20 0x7f0a20003e40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009b70 tx=0x7f0a200058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54dd261bb2ac416 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:21.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400e6c0 con 0x7f0a20003a20 2024-09-13T18:30:21.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2400e820 con 0x7f0a20003a20 2024-09-13T18:30:21.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a2e7fc700 1 -- 172.21.15.146:0/3520550955 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400eb90 con 0x7f0a20003a20 2024-09-13T18:30:21.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a34d1a700 1 -- 172.21.15.146:0/3520550955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a20003a20 msgr2=0x7f0a20003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:21.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a34d1a700 1 --2- 172.21.15.146:0/3520550955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a20003a20 0x7f0a20003e40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009b70 tx=0x7f0a200058a0 comp rx=0 tx=0).stop 2024-09-13T18:30:21.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a34d1a700 1 -- 172.21.15.146:0/3520550955 shutdown_connections 2024-09-13T18:30:21.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a34d1a700 1 --2- 172.21.15.146:0/3520550955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a20003a20 0x7f0a20003e40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:21.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a34d1a700 1 -- 172.21.15.146:0/3520550955 >> 172.21.15.146:0/3520550955 conn(0x7f0a28007860 msgr2=0x7f0a28007c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:21.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a34d1a700 1 -- 172.21.15.146:0/3520550955 shutdown_connections 2024-09-13T18:30:21.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.010+0000 7f0a34d1a700 1 -- 172.21.15.146:0/3520550955 wait complete. 2024-09-13T18:30:21.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a34d1a700 1 Processor -- start 2024-09-13T18:30:21.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a34d1a700 1 -- start start 2024-09-13T18:30:21.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a34d1a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a20003a20 0x7f0a28032f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:21.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a34d1a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a280df850 0x7f0a280319f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:21.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a34d1a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a280dfcb0 0x7f0a28031f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:21.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a34d1a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a281742d0 con 0x7f0a20003a20 2024-09-13T18:30:21.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a34d1a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a28030990 con 0x7f0a280dfcb0 2024-09-13T18:30:21.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a34d1a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a28030c30 con 0x7f0a280df850 2024-09-13T18:30:21.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a2f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a20003a20 0x7f0a28032f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:21.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a2f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a20003a20 0x7f0a28032f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33702/0 (socket says 172.21.15.146:33702) 2024-09-13T18:30:21.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a2f7fe700 1 -- 172.21.15.146:0/1266034087 learned_addr learned my addr 172.21.15.146:0/1266034087 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:21.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a2effd700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a280df850 0x7f0a280319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:21.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a2ffff700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a280dfcb0 0x7f0a28031f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:21.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a2f7fe700 1 -- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a280df850 msgr2=0x7f0a280319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:21.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a2f7fe700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a280df850 0x7f0a280319f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:21.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a2f7fe700 1 -- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a280dfcb0 msgr2=0x7f0a28031f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:21.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a2f7fe700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a280dfcb0 0x7f0a28031f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:21.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a2f7fe700 1 -- 172.21.15.146:0/1266034087 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a28030f10 con 0x7f0a20003a20 2024-09-13T18:30:21.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a2effd700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a280df850 0x7f0a280319f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:30:21.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a2f7fe700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a20003a20 0x7f0a28032f50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000c0a0 tx=0x7f0a240047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:21.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a1bfff700 1 -- 172.21.15.146:0/1266034087 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400e6c0 con 0x7f0a20003a20 2024-09-13T18:30:21.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a1bfff700 1 -- 172.21.15.146:0/1266034087 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a24003af0 con 0x7f0a20003a20 2024-09-13T18:30:21.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a1bfff700 1 -- 172.21.15.146:0/1266034087 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24003ea0 con 0x7f0a20003a20 2024-09-13T18:30:21.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a34d1a700 1 -- 172.21.15.146:0/1266034087 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a281742d0 con 0x7f0a20003a20 2024-09-13T18:30:21.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.014+0000 7f0a34d1a700 1 -- 172.21.15.146:0/1266034087 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a280314d0 con 0x7f0a20003a20 2024-09-13T18:30:21.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.018+0000 7f0a34d1a700 1 -- 172.21.15.146:0/1266034087 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a28031690 con 0x7f0a20003a20 2024-09-13T18:30:21.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.018+0000 7f0a1bfff700 1 -- 172.21.15.146:0/1266034087 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0a24013470 con 0x7f0a20003a20 2024-09-13T18:30:21.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.018+0000 7f0a1bfff700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0a14066780 0x7f0a14068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:21.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.018+0000 7f0a1bfff700 1 -- 172.21.15.146:0/1266034087 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f0a24085ad0 con 0x7f0a20003a20 2024-09-13T18:30:21.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.018+0000 7f0a2effd700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0a14066780 0x7f0a14068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:21.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.018+0000 7f0a2effd700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0a14066780 0x7f0a14068c40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0a280e2a80 tx=0x7f0a1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:21.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.022+0000 7f0a1bfff700 1 -- 172.21.15.146:0/1266034087 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0a24052c60 con 0x7f0a20003a20 2024-09-13T18:30:21.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.266+0000 7f0a34d1a700 1 -- 172.21.15.146:0/1266034087 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a28046700 con 0x7f0a20003a20 2024-09-13T18:30:21.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.270+0000 7f0a1bfff700 1 -- 172.21.15.146:0/1266034087 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f0a24056280 con 0x7f0a20003a20 2024-09-13T18:30:21.277 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:21.277 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5278,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:46.309663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:21.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.274+0000 7f0a34d1a700 1 -- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0a14066780 msgr2=0x7f0a14068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:21.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.274+0000 7f0a34d1a700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0a14066780 0x7f0a14068c40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0a280e2a80 tx=0x7f0a1c009040 comp rx=0 tx=0).stop 2024-09-13T18:30:21.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.274+0000 7f0a34d1a700 1 -- 172.21.15.146:0/1266034087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a20003a20 msgr2=0x7f0a28032f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:21.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.274+0000 7f0a34d1a700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a20003a20 0x7f0a28032f50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000c0a0 tx=0x7f0a240047e0 comp rx=0 tx=0).stop 2024-09-13T18:30:21.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.274+0000 7f0a34d1a700 1 -- 172.21.15.146:0/1266034087 shutdown_connections 2024-09-13T18:30:21.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.274+0000 7f0a34d1a700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0a14066780 0x7f0a14068c40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:21.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.274+0000 7f0a34d1a700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a280dfcb0 0x7f0a28031f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:21.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.274+0000 7f0a34d1a700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a280df850 0x7f0a280319f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:21.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.274+0000 7f0a34d1a700 1 --2- 172.21.15.146:0/1266034087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a20003a20 0x7f0a28032f50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:21.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.274+0000 7f0a34d1a700 1 -- 172.21.15.146:0/1266034087 >> 172.21.15.146:0/1266034087 conn(0x7f0a28007860 msgr2=0x7f0a2804f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:21.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.274+0000 7f0a34d1a700 1 -- 172.21.15.146:0/1266034087 shutdown_connections 2024-09-13T18:30:21.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:21.274+0000 7f0a34d1a700 1 -- 172.21.15.146:0/1266034087 wait complete. 2024-09-13T18:30:21.283 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 27 2024-09-13T18:30:21.296 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:30:23.297 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:23.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.398+0000 7f38b4a3d700 1 Processor -- start 2024-09-13T18:30:23.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.398+0000 7f38b4a3d700 1 -- start start 2024-09-13T18:30:23.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38b4a3d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38b00c61b0 0x7f38b015d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:23.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38b4a3d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f38b00c6590 0x7f38b006cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:23.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38b4a3d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b0066780 con 0x7f38b006d2c0 2024-09-13T18:30:23.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38b4a3d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b01594a0 con 0x7f38b00c6590 2024-09-13T18:30:23.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38b4a3d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b015d960 con 0x7f38b00c61b0 2024-09-13T18:30:23.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38affff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f38b006d2c0 0x7f38b0153340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:54722/0 (socket says 172.21.15.146:54722) 2024-09-13T18:30:23.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38affff700 1 -- 172.21.15.146:0/1247361776 learned_addr learned my addr 172.21.15.146:0/1247361776 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:30:23.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38aeffd700 1 --2- 172.21.15.146:0/1247361776 >> v2:172.21.15.146:3300/0 conn(0x7f38b00c6590 0x7f38b006cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:23.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38af7fe700 1 --2- 172.21.15.146:0/1247361776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38b00c61b0 0x7f38b015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:23.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38aeffd700 1 -- 172.21.15.146:0/1247361776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38b00c61b0 msgr2=0x7f38b015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:23.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38aeffd700 1 --2- 172.21.15.146:0/1247361776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38b00c61b0 0x7f38b015d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:23.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38ae7fc700 1 -- 172.21.15.146:0/1247361776 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701153004 0 0) 0x7f38a8001110 con 0x7f38b006d2c0 2024-09-13T18:30:23.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38aeffd700 1 -- 172.21.15.146:0/1247361776 >> v1:172.21.15.146:6789/0 conn(0x7f38b006d2c0 legacy=0x7f38b0153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:23.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38aeffd700 1 -- 172.21.15.146:0/1247361776 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b006eec0 con 0x7f38b00c6590 2024-09-13T18:30:23.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38aeffd700 1 --2- 172.21.15.146:0/1247361776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b00c6590 0x7f38b006cd80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3898009850 tx=0x7f3898009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbb68b7dec3c385 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:23.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38ae7fc700 1 -- 172.21.15.146:0/1247361776 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898011830 con 0x7f38b00c6590 2024-09-13T18:30:23.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38ae7fc700 1 -- 172.21.15.146:0/1247361776 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3898011990 con 0x7f38b00c6590 2024-09-13T18:30:23.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38ae7fc700 1 -- 172.21.15.146:0/1247361776 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898011d20 con 0x7f38b00c6590 2024-09-13T18:30:23.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38b4a3d700 1 -- 172.21.15.146:0/1247361776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b00c6590 msgr2=0x7f38b006cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:23.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38b4a3d700 1 --2- 172.21.15.146:0/1247361776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b00c6590 0x7f38b006cd80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3898009850 tx=0x7f3898009b60 comp rx=0 tx=0).stop 2024-09-13T18:30:23.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38b4a3d700 1 -- 172.21.15.146:0/1247361776 shutdown_connections 2024-09-13T18:30:23.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38b4a3d700 1 --2- 172.21.15.146:0/1247361776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b00c6590 0x7f38b006cd80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:23.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38b4a3d700 1 --2- 172.21.15.146:0/1247361776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38b00c61b0 0x7f38b015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:23.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38b4a3d700 1 -- 172.21.15.146:0/1247361776 >> 172.21.15.146:0/1247361776 conn(0x7f38b0066b00 msgr2=0x7f38b006aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:23.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38b4a3d700 1 -- 172.21.15.146:0/1247361776 shutdown_connections 2024-09-13T18:30:23.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.402+0000 7f38b4a3d700 1 -- 172.21.15.146:0/1247361776 wait complete. 2024-09-13T18:30:23.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38b4a3d700 1 Processor -- start 2024-09-13T18:30:23.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38b4a3d700 1 -- start start 2024-09-13T18:30:23.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38b4a3d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38b006d2c0 0x7f38b015aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:23.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38b4a3d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38b00c61b0 0x7f38b01e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:23.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38b4a3d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b00c6590 0x7f38b014d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:23.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38b4a3d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b015b280 con 0x7f38b00c6590 2024-09-13T18:30:23.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38b4a3d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0158d80 con 0x7f38b00c61b0 2024-09-13T18:30:23.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38b4a3d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b01574d0 con 0x7f38b006d2c0 2024-09-13T18:30:23.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38affff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b00c6590 0x7f38b014d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:23.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38affff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b00c6590 0x7f38b014d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44174/0 (socket says 172.21.15.146:44174) 2024-09-13T18:30:23.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38affff700 1 -- 172.21.15.146:0/246137992 learned_addr learned my addr 172.21.15.146:0/246137992 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:23.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38aeffd700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38b00c61b0 0x7f38b01e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:23.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38af7fe700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38b006d2c0 0x7f38b015aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:23.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38affff700 1 -- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38b006d2c0 msgr2=0x7f38b015aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:23.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38affff700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38b006d2c0 0x7f38b015aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:23.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38affff700 1 -- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38b00c61b0 msgr2=0x7f38b01e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:23.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38affff700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38b00c61b0 0x7f38b01e3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:23.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38affff700 1 -- 172.21.15.146:0/246137992 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38980094b0 con 0x7f38b00c6590 2024-09-13T18:30:23.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38affff700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b00c6590 0x7f38b014d810 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f38a800c780 tx=0x7f38a800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:23.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38a7fff700 1 -- 172.21.15.146:0/246137992 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8004ca0 con 0x7f38b00c6590 2024-09-13T18:30:23.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38a7fff700 1 -- 172.21.15.146:0/246137992 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a8004e00 con 0x7f38b00c6590 2024-09-13T18:30:23.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38b4a3d700 1 -- 172.21.15.146:0/246137992 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b01587c0 con 0x7f38b00c6590 2024-09-13T18:30:23.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38a7fff700 1 -- 172.21.15.146:0/246137992 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8007310 con 0x7f38b00c6590 2024-09-13T18:30:23.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38b4a3d700 1 -- 172.21.15.146:0/246137992 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b014d2f0 con 0x7f38b00c6590 2024-09-13T18:30:23.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.406+0000 7f38b4a3d700 1 -- 172.21.15.146:0/246137992 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b0159f50 con 0x7f38b00c6590 2024-09-13T18:30:23.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.410+0000 7f38a7fff700 1 -- 172.21.15.146:0/246137992 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f38a8007470 con 0x7f38b00c6590 2024-09-13T18:30:23.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.410+0000 7f38a7fff700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3890066730 0x7f3890068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:23.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.410+0000 7f38a7fff700 1 -- 172.21.15.146:0/246137992 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f38a8083df0 con 0x7f38b00c6590 2024-09-13T18:30:23.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.410+0000 7f38af7fe700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3890066730 0x7f3890068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:23.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.410+0000 7f38af7fe700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3890066730 0x7f3890068bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f38a0003c30 tx=0x7f38a00098d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:23.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.414+0000 7f38a7fff700 1 -- 172.21.15.146:0/246137992 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f38a8050e60 con 0x7f38b00c6590 2024-09-13T18:30:23.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.638+0000 7f38b4a3d700 1 -- 172.21.15.146:0/246137992 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38b0066780 con 0x7f38b00c6590 2024-09-13T18:30:23.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.642+0000 7f38a7fff700 1 -- 172.21.15.146:0/246137992 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f38b0066780 con 0x7f38b00c6590 2024-09-13T18:30:23.649 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:23.650 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5278,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:46.309663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:23.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.642+0000 7f38b4a3d700 1 -- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3890066730 msgr2=0x7f3890068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:23.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.642+0000 7f38b4a3d700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3890066730 0x7f3890068bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f38a0003c30 tx=0x7f38a00098d0 comp rx=0 tx=0).stop 2024-09-13T18:30:23.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.642+0000 7f38b4a3d700 1 -- 172.21.15.146:0/246137992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b00c6590 msgr2=0x7f38b014d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:23.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.642+0000 7f38b4a3d700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b00c6590 0x7f38b014d810 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f38a800c780 tx=0x7f38a800ca90 comp rx=0 tx=0).stop 2024-09-13T18:30:23.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.642+0000 7f38b4a3d700 1 -- 172.21.15.146:0/246137992 shutdown_connections 2024-09-13T18:30:23.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.642+0000 7f38b4a3d700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3890066730 0x7f3890068bf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:23.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.642+0000 7f38b4a3d700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b00c6590 0x7f38b014d810 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:23.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.642+0000 7f38b4a3d700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38b00c61b0 0x7f38b01e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:23.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.642+0000 7f38b4a3d700 1 --2- 172.21.15.146:0/246137992 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38b006d2c0 0x7f38b015aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:23.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.642+0000 7f38b4a3d700 1 -- 172.21.15.146:0/246137992 >> 172.21.15.146:0/246137992 conn(0x7f38b0066b00 msgr2=0x7f38b015d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:23.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.642+0000 7f38b4a3d700 1 -- 172.21.15.146:0/246137992 shutdown_connections 2024-09-13T18:30:23.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:23.642+0000 7f38b4a3d700 1 -- 172.21.15.146:0/246137992 wait complete. 2024-09-13T18:30:23.653 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 27 2024-09-13T18:30:23.665 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:30:25.666 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:25.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.778+0000 7fe5649fd700 1 Processor -- start 2024-09-13T18:30:25.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.778+0000 7fe5649fd700 1 -- start start 2024-09-13T18:30:25.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe5649fd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe560150230 0x7fe56006ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:25.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe5649fd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe560071610 0x7fe560159b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:25.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe5649fd700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe560071410 con 0x7fe560150610 2024-09-13T18:30:25.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe5649fd700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5601596b0 con 0x7fe560150230 2024-09-13T18:30:25.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe5649fd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe560156090 con 0x7fe560071610 2024-09-13T18:30:25.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe55ed9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe560150230 0x7fe56006ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:25.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe55ed9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe560150230 0x7fe56006ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44186/0 (socket says 172.21.15.146:44186) 2024-09-13T18:30:25.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe55ed9d700 1 -- 172.21.15.146:0/175664439 learned_addr learned my addr 172.21.15.146:0/175664439 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:25.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe55f59e700 1 --2- 172.21.15.146:0/175664439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe560071610 0x7fe560159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:25.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe55ed9d700 1 -- 172.21.15.146:0/175664439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe560071610 msgr2=0x7fe560159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:25.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe55ed9d700 1 --2- 172.21.15.146:0/175664439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe560071610 0x7fe560159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:25.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe55ed9d700 1 -- 172.21.15.146:0/175664439 >> v1:172.21.15.154:6789/0 conn(0x7fe560150610 legacy=0x7fe56006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:25.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe55ed9d700 1 -- 172.21.15.146:0/175664439 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe560156350 con 0x7fe560150230 2024-09-13T18:30:25.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe55ed9d700 1 --2- 172.21.15.146:0/175664439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe560150230 0x7fe56006ebd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe55000bbb0 tx=0x7fe55000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a851de2a8c7e49a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:25.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe55dd9b700 1 -- 172.21.15.146:0/175664439 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55000dd30 con 0x7fe560150230 2024-09-13T18:30:25.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe55dd9b700 1 -- 172.21.15.146:0/175664439 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55000de90 con 0x7fe560150230 2024-09-13T18:30:25.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe55dd9b700 1 -- 172.21.15.146:0/175664439 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550010690 con 0x7fe560150230 2024-09-13T18:30:25.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe5649fd700 1 -- 172.21.15.146:0/175664439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe560150230 msgr2=0x7fe56006ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:25.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe5649fd700 1 --2- 172.21.15.146:0/175664439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe560150230 0x7fe56006ebd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe55000bbb0 tx=0x7fe55000bec0 comp rx=0 tx=0).stop 2024-09-13T18:30:25.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe5649fd700 1 -- 172.21.15.146:0/175664439 shutdown_connections 2024-09-13T18:30:25.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe5649fd700 1 --2- 172.21.15.146:0/175664439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe560071610 0x7fe560159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:25.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe5649fd700 1 --2- 172.21.15.146:0/175664439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe560150230 0x7fe56006ebd0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:25.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.782+0000 7fe5649fd700 1 -- 172.21.15.146:0/175664439 >> 172.21.15.146:0/175664439 conn(0x7fe560071ed0 msgr2=0x7fe5600722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:25.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe5649fd700 1 -- 172.21.15.146:0/175664439 shutdown_connections 2024-09-13T18:30:25.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe5649fd700 1 -- 172.21.15.146:0/175664439 wait complete. 2024-09-13T18:30:25.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe5649fd700 1 Processor -- start 2024-09-13T18:30:25.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe5649fd700 1 -- start start 2024-09-13T18:30:25.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe5649fd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe560071610 0x7fe560158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:25.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe5649fd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe560150230 0x7fe560154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:25.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe5649fd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5601588a0 0x7fe560155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:25.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe5649fd700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5601ee230 con 0x7fe5601588a0 2024-09-13T18:30:25.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe5649fd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56014aa50 con 0x7fe560071610 2024-09-13T18:30:25.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe5649fd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56014a630 con 0x7fe560150230 2024-09-13T18:30:25.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe55f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5601588a0 0x7fe560155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:25.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe55f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5601588a0 0x7fe560155500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44198/0 (socket says 172.21.15.146:44198) 2024-09-13T18:30:25.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe55f59e700 1 -- 172.21.15.146:0/3162709848 learned_addr learned my addr 172.21.15.146:0/3162709848 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:25.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe55ed9d700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe560071610 0x7fe560158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:25.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.786+0000 7fe55e59c700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe560150230 0x7fe560154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:25.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe55f59e700 1 -- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe560150230 msgr2=0x7fe560154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:25.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe55f59e700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe560150230 0x7fe560154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:25.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe55f59e700 1 -- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe560071610 msgr2=0x7fe560158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:25.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe55f59e700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe560071610 0x7fe560158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:25.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe55f59e700 1 -- 172.21.15.146:0/3162709848 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55000b810 con 0x7fe5601588a0 2024-09-13T18:30:25.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe55f59e700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5601588a0 0x7fe560155500 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe55800efc0 tx=0x7fe55800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:25.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe5477fe700 1 -- 172.21.15.146:0/3162709848 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558009ea0 con 0x7fe5601588a0 2024-09-13T18:30:25.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe5477fe700 1 -- 172.21.15.146:0/3162709848 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55800f040 con 0x7fe5601588a0 2024-09-13T18:30:25.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe5649fd700 1 -- 172.21.15.146:0/3162709848 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5601edf50 con 0x7fe5601588a0 2024-09-13T18:30:25.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe5477fe700 1 -- 172.21.15.146:0/3162709848 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800ced0 con 0x7fe5601588a0 2024-09-13T18:30:25.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe5649fd700 1 -- 172.21.15.146:0/3162709848 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe560157880 con 0x7fe5601588a0 2024-09-13T18:30:25.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe5649fd700 1 -- 172.21.15.146:0/3162709848 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56006ebd0 con 0x7fe5601588a0 2024-09-13T18:30:25.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe5477fe700 1 -- 172.21.15.146:0/3162709848 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe558014480 con 0x7fe5601588a0 2024-09-13T18:30:25.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe5477fe700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe54c066760 0x7fe54c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:25.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.790+0000 7fe5477fe700 1 -- 172.21.15.146:0/3162709848 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fe5580850a0 con 0x7fe5601588a0 2024-09-13T18:30:25.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.794+0000 7fe55ed9d700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe54c066760 0x7fe54c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:25.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.794+0000 7fe55ed9d700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe54c066760 0x7fe54c068c20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe55000bfd0 tx=0x7fe550013000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:25.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:25.798+0000 7fe5477fe700 1 -- 172.21.15.146:0/3162709848 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe558051f10 con 0x7fe5601588a0 2024-09-13T18:30:26.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.042+0000 7fe5649fd700 1 -- 172.21.15.146:0/3162709848 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe560149150 con 0x7fe5601588a0 2024-09-13T18:30:26.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.042+0000 7fe5477fe700 1 -- 172.21.15.146:0/3162709848 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6288 (secure 0 0 0) 0x7fe558018070 con 0x7fe5601588a0 2024-09-13T18:30:26.050 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:26.050 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5278,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:46.309663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:26.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.042+0000 7fe5649fd700 1 -- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe54c066760 msgr2=0x7fe54c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:26.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.042+0000 7fe5649fd700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe54c066760 0x7fe54c068c20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe55000bfd0 tx=0x7fe550013000 comp rx=0 tx=0).stop 2024-09-13T18:30:26.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.046+0000 7fe5649fd700 1 -- 172.21.15.146:0/3162709848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5601588a0 msgr2=0x7fe560155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:26.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.046+0000 7fe5649fd700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5601588a0 0x7fe560155500 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe55800efc0 tx=0x7fe55800c3a0 comp rx=0 tx=0).stop 2024-09-13T18:30:26.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.046+0000 7fe5649fd700 1 -- 172.21.15.146:0/3162709848 shutdown_connections 2024-09-13T18:30:26.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.046+0000 7fe5649fd700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe54c066760 0x7fe54c068c20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:26.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.046+0000 7fe5649fd700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5601588a0 0x7fe560155500 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:26.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.046+0000 7fe5649fd700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe560150230 0x7fe560154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:26.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.046+0000 7fe5649fd700 1 --2- 172.21.15.146:0/3162709848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe560071610 0x7fe560158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:26.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.046+0000 7fe5649fd700 1 -- 172.21.15.146:0/3162709848 >> 172.21.15.146:0/3162709848 conn(0x7fe560071ed0 msgr2=0x7fe5600c1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:26.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.046+0000 7fe5649fd700 1 -- 172.21.15.146:0/3162709848 shutdown_connections 2024-09-13T18:30:26.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:26.046+0000 7fe5649fd700 1 -- 172.21.15.146:0/3162709848 wait complete. 2024-09-13T18:30:26.054 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 27 2024-09-13T18:30:26.065 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:30:28.066 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:28.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96b151700 1 Processor -- start 2024-09-13T18:30:28.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96b151700 1 -- start start 2024-09-13T18:30:28.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96b151700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff964150550 0x7ff96406d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:28.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96b151700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96406b3c0 con 0x7ff9640c26e0 2024-09-13T18:30:28.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96b151700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96406ae40 con 0x7ff96406dc60 2024-09-13T18:30:28.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96b151700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96406b0e0 con 0x7ff964150550 2024-09-13T18:30:28.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96994e700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff9640c26e0 0x7ff964159890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:54732/0 (socket says 172.21.15.146:54732) 2024-09-13T18:30:28.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96994e700 1 -- 172.21.15.146:0/2866654240 learned_addr learned my addr 172.21.15.146:0/2866654240 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:30:28.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96a14f700 1 --2- 172.21.15.146:0/2866654240 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff964150550 0x7ff96406d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:28.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537342137 0 0) 0x7ff96406b3c0 con 0x7ff9640c26e0 2024-09-13T18:30:28.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9500035d0 con 0x7ff9640c26e0 2024-09-13T18:30:28.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2416511666 0 0) 0x7ff9500035d0 con 0x7ff9640c26e0 2024-09-13T18:30:28.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff964150550 msgr2=0x7ff96406d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:28.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96914d700 1 --2- 172.21.15.146:0/2866654240 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff964150550 0x7ff96406d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:28.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 >> v1:172.21.15.154:6789/0 conn(0x7ff96406dc60 legacy=0x7ff964160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:28.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96406cb40 con 0x7ff9640c26e0 2024-09-13T18:30:28.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.178+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7ff954001670 con 0x7ff9640c26e0 2024-09-13T18:30:28.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 >> v1:172.21.15.146:6789/0 conn(0x7ff9640c26e0 legacy=0x7ff964159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:28.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96914d700 1 --2- 172.21.15.146:0/2866654240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff950003ac0 0x7ff950003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:28.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96914d700 1 --2- 172.21.15.146:0/2866654240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff950004870 0x7ff950004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:28.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96914d700 1 --2- 172.21.15.146:0/2866654240 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff964150550 0x7ff950005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:28.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96406b0e0 con 0x7ff950003ac0 2024-09-13T18:30:28.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95000bb30 con 0x7ff950004870 2024-09-13T18:30:28.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95000bcf0 con 0x7ff964150550 2024-09-13T18:30:28.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96a14f700 1 --2- 172.21.15.146:0/2866654240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff950003ac0 0x7ff950003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:28.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96994e700 1 --2- 172.21.15.146:0/2866654240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff950004870 0x7ff950004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:28.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96a14f700 1 -- 172.21.15.146:0/2866654240 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff964150550 msgr2=0x7ff950005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:28.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96a14f700 1 --2- 172.21.15.146:0/2866654240 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff964150550 0x7ff950005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:28.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96a14f700 1 -- 172.21.15.146:0/2866654240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff950004870 msgr2=0x7ff950004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:28.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96a14f700 1 --2- 172.21.15.146:0/2866654240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff950004870 0x7ff950004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:28.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96a14f700 1 -- 172.21.15.146:0/2866654240 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500035d0 con 0x7ff950003ac0 2024-09-13T18:30:28.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96a14f700 1 --2- 172.21.15.146:0/2866654240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff950003ac0 0x7ff950003ee0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff960009700 tx=0x7ff9500058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cf3778434a0ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:28.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96000e6c0 con 0x7ff950003ac0 2024-09-13T18:30:28.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96000e820 con 0x7ff950003ac0 2024-09-13T18:30:28.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96914d700 1 -- 172.21.15.146:0/2866654240 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96000ebb0 con 0x7ff950003ac0 2024-09-13T18:30:28.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96b151700 1 -- 172.21.15.146:0/2866654240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff950003ac0 msgr2=0x7ff950003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:28.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96b151700 1 --2- 172.21.15.146:0/2866654240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff950003ac0 0x7ff950003ee0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff960009700 tx=0x7ff9500058a0 comp rx=0 tx=0).stop 2024-09-13T18:30:28.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96a950700 1 -- 172.21.15.146:0/2866654240 reap_dead start 2024-09-13T18:30:28.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96b151700 1 -- 172.21.15.146:0/2866654240 shutdown_connections 2024-09-13T18:30:28.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96b151700 1 -- 172.21.15.146:0/2866654240 >> 172.21.15.146:0/2866654240 conn(0x7ff9640c7ce0 msgr2=0x7ff9640c80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:28.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96b151700 1 -- 172.21.15.146:0/2866654240 shutdown_connections 2024-09-13T18:30:28.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96b151700 1 -- 172.21.15.146:0/2866654240 wait complete. 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96b151700 1 Processor -- start 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.182+0000 7ff96b151700 1 -- start start 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96b151700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff950003ac0 0x7ff9641f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96b151700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff964158c90 0x7ff9640ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96b151700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff964140440 0x7ff964152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96b151700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9641efe80 con 0x7ff964158c90 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96b151700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9641f3970 con 0x7ff950003ac0 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96b151700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9641417e0 con 0x7ff964140440 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96994e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff964158c90 0x7ff9640ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96a14f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff950003ac0 0x7ff9641f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96a14f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff950003ac0 0x7ff9641f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:52616/0 (socket says 172.21.15.146:52616) 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96994e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff964158c90 0x7ff9640ca2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44214/0 (socket says 172.21.15.146:44214) 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96a14f700 1 -- 172.21.15.146:0/2846059142 learned_addr learned my addr 172.21.15.146:0/2846059142 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:28.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96a950700 1 --2- 172.21.15.146:0/2846059142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff964140440 0x7ff964152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:28.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96994e700 1 -- 172.21.15.146:0/2846059142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff964140440 msgr2=0x7ff964152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:28.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96994e700 1 --2- 172.21.15.146:0/2846059142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff964140440 0x7ff964152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:28.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96994e700 1 -- 172.21.15.146:0/2846059142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff950003ac0 msgr2=0x7ff9641f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:28.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96994e700 1 --2- 172.21.15.146:0/2846059142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff950003ac0 0x7ff9641f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:28.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96994e700 1 -- 172.21.15.146:0/2846059142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9641591f0 con 0x7ff964158c90 2024-09-13T18:30:28.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96994e700 1 --2- 172.21.15.146:0/2846059142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff964158c90 0x7ff9640ca2a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff9640724c0 tx=0x7ff95400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:28.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff95affd700 1 -- 172.21.15.146:0/2846059142 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff954005e00 con 0x7ff964158c90 2024-09-13T18:30:28.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff95affd700 1 -- 172.21.15.146:0/2846059142 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95400c570 con 0x7ff964158c90 2024-09-13T18:30:28.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96b151700 1 -- 172.21.15.146:0/2846059142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9641594d0 con 0x7ff964158c90 2024-09-13T18:30:28.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff95affd700 1 -- 172.21.15.146:0/2846059142 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95400c920 con 0x7ff964158c90 2024-09-13T18:30:28.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.186+0000 7ff96b151700 1 -- 172.21.15.146:0/2846059142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9641efaa0 con 0x7ff964158c90 2024-09-13T18:30:28.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.190+0000 7ff96b151700 1 -- 172.21.15.146:0/2846059142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9641efc60 con 0x7ff964158c90 2024-09-13T18:30:28.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.190+0000 7ff95affd700 1 -- 172.21.15.146:0/2846059142 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff95400ca80 con 0x7ff964158c90 2024-09-13T18:30:28.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.190+0000 7ff95affd700 1 --2- 172.21.15.146:0/2846059142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff948066730 0x7ff948068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:28.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.190+0000 7ff95affd700 1 -- 172.21.15.146:0/2846059142 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7ff9540841b0 con 0x7ff964158c90 2024-09-13T18:30:28.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.190+0000 7ff96a14f700 1 --2- 172.21.15.146:0/2846059142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff948066730 0x7ff948068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:28.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.190+0000 7ff96a14f700 1 --2- 172.21.15.146:0/2846059142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff948066730 0x7ff948068bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff96000a010 tx=0x7ff96001f040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:28.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.194+0000 7ff95affd700 1 -- 172.21.15.146:0/2846059142 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff954051020 con 0x7ff964158c90 2024-09-13T18:30:28.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.394+0000 7ff96b151700 1 -- 172.21.15.146:0/2846059142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9640c8a00 con 0x7ff964158c90 2024-09-13T18:30:28.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.394+0000 7ff95affd700 1 -- 172.21.15.146:0/2846059142 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6288 (secure 0 0 0) 0x7ff954054640 con 0x7ff964158c90 2024-09-13T18:30:28.404 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:28.404 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5278,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:29:46.309663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":4281,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6835/1642500214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1642500214},{"type":"v1","addr":"172.21.15.146:6835","nonce":1642500214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:28.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.398+0000 7ff96b151700 1 -- 172.21.15.146:0/2846059142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff948066730 msgr2=0x7ff948068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:28.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.398+0000 7ff96b151700 1 --2- 172.21.15.146:0/2846059142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff948066730 0x7ff948068bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff96000a010 tx=0x7ff96001f040 comp rx=0 tx=0).stop 2024-09-13T18:30:28.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.398+0000 7ff96b151700 1 -- 172.21.15.146:0/2846059142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff964158c90 msgr2=0x7ff9640ca2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:28.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.398+0000 7ff96b151700 1 --2- 172.21.15.146:0/2846059142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff964158c90 0x7ff9640ca2a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff9640724c0 tx=0x7ff95400edd0 comp rx=0 tx=0).stop 2024-09-13T18:30:28.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.398+0000 7ff96b151700 1 -- 172.21.15.146:0/2846059142 shutdown_connections 2024-09-13T18:30:28.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.398+0000 7ff96b151700 1 --2- 172.21.15.146:0/2846059142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff948066730 0x7ff948068bf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:28.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.398+0000 7ff96b151700 1 --2- 172.21.15.146:0/2846059142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff964140440 0x7ff964152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:28.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.398+0000 7ff96b151700 1 --2- 172.21.15.146:0/2846059142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff964158c90 0x7ff9640ca2a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:28.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.398+0000 7ff96b151700 1 --2- 172.21.15.146:0/2846059142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff950003ac0 0x7ff9641f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:28.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.398+0000 7ff96b151700 1 -- 172.21.15.146:0/2846059142 >> 172.21.15.146:0/2846059142 conn(0x7ff9640c7ce0 msgr2=0x7ff9640c7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:28.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.398+0000 7ff96b151700 1 -- 172.21.15.146:0/2846059142 shutdown_connections 2024-09-13T18:30:28.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:28.398+0000 7ff96b151700 1 -- 172.21.15.146:0/2846059142 wait complete. 2024-09-13T18:30:28.408 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 27 2024-09-13T18:30:28.422 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:30:30.423 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:30.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.550+0000 7f6ecca02700 1 Processor -- start 2024-09-13T18:30:30.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.550+0000 7f6ecca02700 1 -- start start 2024-09-13T18:30:30.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.550+0000 7f6ecca02700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6ec8066300 0x7f6ec8156da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:30.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.550+0000 7f6ecca02700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec806d220 0x7f6ec80c4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:30.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.550+0000 7f6ecca02700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec80c4690 con 0x7f6ec81504f0 2024-09-13T18:30:30.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.550+0000 7f6ecca02700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec80c27e0 con 0x7f6ec8066300 2024-09-13T18:30:30.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.550+0000 7f6ecca02700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec806d6b0 con 0x7f6ec806d220 2024-09-13T18:30:30.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.550+0000 7f6ec6ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6ec8066300 0x7f6ec8156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:30.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.550+0000 7f6ec77fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6ec81504f0 0x7f6ec80c7fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:54738/0 (socket says 172.21.15.146:54738) 2024-09-13T18:30:30.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.550+0000 7f6ec77fe700 1 -- 172.21.15.146:0/3579710804 learned_addr learned my addr 172.21.15.146:0/3579710804 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:30:30.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ec7fff700 1 --2- 172.21.15.146:0/3579710804 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec806d220 0x7f6ec80c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:30.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ec67fc700 1 -- 172.21.15.146:0/3579710804 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338839057 0 0) 0x7f6eb8001110 con 0x7f6ec81504f0 2024-09-13T18:30:30.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ec67fc700 1 -- 172.21.15.146:0/3579710804 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6eb00035d0 con 0x7f6ec81504f0 2024-09-13T18:30:30.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ec6ffd700 1 -- 172.21.15.146:0/3579710804 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec806d220 msgr2=0x7f6ec80c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:30.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ec6ffd700 1 --2- 172.21.15.146:0/3579710804 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec806d220 0x7f6ec80c4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ec6ffd700 1 -- 172.21.15.146:0/3579710804 >> v1:172.21.15.146:6789/0 conn(0x7f6ec81504f0 legacy=0x7f6ec80c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:30.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ec6ffd700 1 -- 172.21.15.146:0/3579710804 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec8156330 con 0x7f6ec8066300 2024-09-13T18:30:30.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ec7fff700 1 --2- 172.21.15.146:0/3579710804 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec806d220 0x7f6ec80c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:30:30.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ec6ffd700 1 --2- 172.21.15.146:0/3579710804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec8066300 0x7f6ec8156da0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009850 tx=0x7f6ebc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04dff10e015368b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:30.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ec67fc700 1 -- 172.21.15.146:0/3579710804 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc011830 con 0x7f6ec8066300 2024-09-13T18:30:30.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ec67fc700 1 -- 172.21.15.146:0/3579710804 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc011990 con 0x7f6ec8066300 2024-09-13T18:30:30.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ec67fc700 1 -- 172.21.15.146:0/3579710804 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc011d40 con 0x7f6ec8066300 2024-09-13T18:30:30.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ecca02700 1 -- 172.21.15.146:0/3579710804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec8066300 msgr2=0x7f6ec8156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:30.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ecca02700 1 --2- 172.21.15.146:0/3579710804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec8066300 0x7f6ec8156da0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009850 tx=0x7f6ebc009b60 comp rx=0 tx=0).stop 2024-09-13T18:30:30.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ecca02700 1 -- 172.21.15.146:0/3579710804 shutdown_connections 2024-09-13T18:30:30.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ecca02700 1 --2- 172.21.15.146:0/3579710804 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec806d220 0x7f6ec80c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ecca02700 1 --2- 172.21.15.146:0/3579710804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec8066300 0x7f6ec8156da0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ecca02700 1 -- 172.21.15.146:0/3579710804 >> 172.21.15.146:0/3579710804 conn(0x7f6ec8066b00 msgr2=0x7f6ec806aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:30.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ecca02700 1 -- 172.21.15.146:0/3579710804 shutdown_connections 2024-09-13T18:30:30.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ecca02700 1 -- 172.21.15.146:0/3579710804 wait complete. 2024-09-13T18:30:30.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ecca02700 1 Processor -- start 2024-09-13T18:30:30.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.554+0000 7f6ecca02700 1 -- start start 2024-09-13T18:30:30.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ecca02700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec8066300 0x7f6ec81f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:30.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ecca02700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec806d220 0x7f6ec81ecb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:30.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ecca02700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec81504f0 0x7f6ec806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:30.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ecca02700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec8147b10 con 0x7f6ec8066300 2024-09-13T18:30:30.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ecca02700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec81ed050 con 0x7f6ec806d220 2024-09-13T18:30:30.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ecca02700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec81f0770 con 0x7f6ec81504f0 2024-09-13T18:30:30.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ec77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec8066300 0x7f6ec81f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:30.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ec77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec8066300 0x7f6ec81f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44228/0 (socket says 172.21.15.146:44228) 2024-09-13T18:30:30.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ec77fe700 1 -- 172.21.15.146:0/2479582281 learned_addr learned my addr 172.21.15.146:0/2479582281 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:30.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ec6ffd700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec806d220 0x7f6ec81ecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:30.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ec7fff700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec81504f0 0x7f6ec806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:30.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ec77fe700 1 -- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec81504f0 msgr2=0x7f6ec806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ec77fe700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec81504f0 0x7f6ec806da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ec77fe700 1 -- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec806d220 msgr2=0x7f6ec81ecb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ec77fe700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec806d220 0x7f6ec81ecb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ec77fe700 1 -- 172.21.15.146:0/2479582281 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc0094b0 con 0x7f6ec8066300 2024-09-13T18:30:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ec77fe700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec8066300 0x7f6ec81f0c00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8001270 tx=0x7f6eb800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6eaffff700 1 -- 172.21.15.146:0/2479582281 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8005d90 con 0x7f6ec8066300 2024-09-13T18:30:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6eaffff700 1 -- 172.21.15.146:0/2479582281 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb80070d0 con 0x7f6ec8066300 2024-09-13T18:30:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ecca02700 1 -- 172.21.15.146:0/2479582281 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec8157d90 con 0x7f6ec8066300 2024-09-13T18:30:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6eaffff700 1 -- 172.21.15.146:0/2479582281 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8007480 con 0x7f6ec8066300 2024-09-13T18:30:30.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ecca02700 1 -- 172.21.15.146:0/2479582281 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec8148020 con 0x7f6ec8066300 2024-09-13T18:30:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6eaffff700 1 -- 172.21.15.146:0/2479582281 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6eb80075e0 con 0x7f6ec8066300 2024-09-13T18:30:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6ecca02700 1 -- 172.21.15.146:0/2479582281 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec80c4e90 con 0x7f6ec8066300 2024-09-13T18:30:30.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.558+0000 7f6eaffff700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6eb4066730 0x7f6eb4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:30.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.562+0000 7f6eaffff700 1 -- 172.21.15.146:0/2479582281 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f6eb8083820 con 0x7f6ec8066300 2024-09-13T18:30:30.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.562+0000 7f6ec6ffd700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6eb4066730 0x7f6eb4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:30.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.562+0000 7f6ec6ffd700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6eb4066730 0x7f6eb4068bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0090b0 tx=0x7f6ebc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:30.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.570+0000 7f6eaffff700 1 -- 172.21.15.146:0/2479582281 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6eb80507b0 con 0x7f6ec8066300 2024-09-13T18:30:30.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.794+0000 7f6ecca02700 1 -- 172.21.15.146:0/2479582281 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ec80c4690 con 0x7f6ec8066300 2024-09-13T18:30:30.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.794+0000 7f6eaffff700 1 -- 172.21.15.146:0/2479582281 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5527 (secure 0 0 0) 0x7f6ec80c4690 con 0x7f6ec8066300 2024-09-13T18:30:30.803 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:30.803 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:30.599542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.798+0000 7f6ecca02700 1 -- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6eb4066730 msgr2=0x7f6eb4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.798+0000 7f6ecca02700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6eb4066730 0x7f6eb4068bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0090b0 tx=0x7f6ebc008030 comp rx=0 tx=0).stop 2024-09-13T18:30:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.798+0000 7f6ecca02700 1 -- 172.21.15.146:0/2479582281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec8066300 msgr2=0x7f6ec81f0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.798+0000 7f6ecca02700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec8066300 0x7f6ec81f0c00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8001270 tx=0x7f6eb800c7d0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.798+0000 7f6ecca02700 1 -- 172.21.15.146:0/2479582281 shutdown_connections 2024-09-13T18:30:30.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.798+0000 7f6ecca02700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6eb4066730 0x7f6eb4068bf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.798+0000 7f6ecca02700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec81504f0 0x7f6ec806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.798+0000 7f6ecca02700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec806d220 0x7f6ec81ecb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.798+0000 7f6ecca02700 1 --2- 172.21.15.146:0/2479582281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec8066300 0x7f6ec81f0c00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.798+0000 7f6ecca02700 1 -- 172.21.15.146:0/2479582281 >> 172.21.15.146:0/2479582281 conn(0x7f6ec8066b00 msgr2=0x7f6ec806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:30.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.798+0000 7f6ecca02700 1 -- 172.21.15.146:0/2479582281 shutdown_connections 2024-09-13T18:30:30.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.798+0000 7f6ecca02700 1 -- 172.21.15.146:0/2479582281 wait complete. 2024-09-13T18:30:30.807 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 28 2024-09-13T18:30:30.819 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-09-13T18:30:30.819 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:30:30.819 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:30.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.922+0000 7fef40b48700 1 Processor -- start 2024-09-13T18:30:30.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.922+0000 7fef40b48700 1 -- start start 2024-09-13T18:30:30.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fef3c0c6960 0x7fef3c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:30.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef3c0cc6e0 0x7fef3c06e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:30.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3c070dc0 con 0x7fef3c151080 2024-09-13T18:30:30.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef3c157470 con 0x7fef3c0c6960 2024-09-13T18:30:30.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c157750 con 0x7fef3c0cc6e0 2024-09-13T18:30:30.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef2bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fef3c0c6960 0x7fef3c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:30.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef2bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fef3c0c6960 0x7fef3c0cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44238/0 (socket says 172.21.15.146:44238) 2024-09-13T18:30:30.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef2bfff700 1 -- 172.21.15.146:0/2139284885 learned_addr learned my addr 172.21.15.146:0/2139284885 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:30.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef2bfff700 1 -- 172.21.15.146:0/2139284885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef3c0cc6e0 msgr2=0x7fef3c06e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:30.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef2bfff700 1 --2- 172.21.15.146:0/2139284885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef3c0cc6e0 0x7fef3c06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef2bfff700 1 -- 172.21.15.146:0/2139284885 >> v1:172.21.15.154:6789/0 conn(0x7fef3c151080 legacy=0x7fef3c06f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:30.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef2bfff700 1 -- 172.21.15.146:0/2139284885 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c157a30 con 0x7fef3c0c6960 2024-09-13T18:30:30.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef2bfff700 1 --2- 172.21.15.146:0/2139284885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef3c0c6960 0x7fef3c0cc1a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fef30009910 tx=0x7fef30009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc349ba95318361 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:30.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef3affd700 1 -- 172.21.15.146:0/2139284885 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3000ba60 con 0x7fef3c0c6960 2024-09-13T18:30:30.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef3affd700 1 -- 172.21.15.146:0/2139284885 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3000bbc0 con 0x7fef3c0c6960 2024-09-13T18:30:30.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef3affd700 1 -- 172.21.15.146:0/2139284885 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef30009e60 con 0x7fef3c0c6960 2024-09-13T18:30:30.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 -- 172.21.15.146:0/2139284885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef3c0c6960 msgr2=0x7fef3c0cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:30.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 --2- 172.21.15.146:0/2139284885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef3c0c6960 0x7fef3c0cc1a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fef30009910 tx=0x7fef30009c20 comp rx=0 tx=0).stop 2024-09-13T18:30:30.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 -- 172.21.15.146:0/2139284885 shutdown_connections 2024-09-13T18:30:30.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 --2- 172.21.15.146:0/2139284885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef3c0cc6e0 0x7fef3c06e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 --2- 172.21.15.146:0/2139284885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef3c0c6960 0x7fef3c0cc1a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 -- 172.21.15.146:0/2139284885 >> 172.21.15.146:0/2139284885 conn(0x7fef3c071ed0 msgr2=0x7fef3c0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:30.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 -- 172.21.15.146:0/2139284885 shutdown_connections 2024-09-13T18:30:30.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 -- 172.21.15.146:0/2139284885 wait complete. 2024-09-13T18:30:30.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 Processor -- start 2024-09-13T18:30:30.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.926+0000 7fef40b48700 1 -- start start 2024-09-13T18:30:30.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef40b48700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef3c0c6960 0x7fef3c148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:30.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef40b48700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef3c0cc6e0 0x7fef3c149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:30.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef40b48700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef3c151080 0x7fef3c1de1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:30.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef40b48700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c149210 con 0x7fef3c0cc6e0 2024-09-13T18:30:30.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef40b48700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c1ded70 con 0x7fef3c151080 2024-09-13T18:30:30.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef40b48700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c1dda60 con 0x7fef3c0c6960 2024-09-13T18:30:30.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef2bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef3c0cc6e0 0x7fef3c149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:30.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef2bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef3c0cc6e0 0x7fef3c149850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44254/0 (socket says 172.21.15.146:44254) 2024-09-13T18:30:30.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef2bfff700 1 -- 172.21.15.146:0/3199083772 learned_addr learned my addr 172.21.15.146:0/3199083772 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:30.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef3bfff700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef3c151080 0x7fef3c1de1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:30.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef3b7fe700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef3c0c6960 0x7fef3c148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:30.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef2bfff700 1 -- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef3c0c6960 msgr2=0x7fef3c148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:30.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef2bfff700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef3c0c6960 0x7fef3c148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef2bfff700 1 -- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef3c151080 msgr2=0x7fef3c1de1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:30.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef2bfff700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef3c151080 0x7fef3c1de1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:30.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef2bfff700 1 -- 172.21.15.146:0/3199083772 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef30009570 con 0x7fef3c0cc6e0 2024-09-13T18:30:30.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef2bfff700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef3c0cc6e0 0x7fef3c149850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fef30009c60 tx=0x7fef30004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:30.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef38ff9700 1 -- 172.21.15.146:0/3199083772 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef300044d0 con 0x7fef3c0cc6e0 2024-09-13T18:30:30.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef38ff9700 1 -- 172.21.15.146:0/3199083772 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3001f070 con 0x7fef3c0cc6e0 2024-09-13T18:30:30.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef40b48700 1 -- 172.21.15.146:0/3199083772 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3c147f20 con 0x7fef3c0cc6e0 2024-09-13T18:30:30.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef38ff9700 1 -- 172.21.15.146:0/3199083772 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef30003900 con 0x7fef3c0cc6e0 2024-09-13T18:30:30.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.930+0000 7fef40b48700 1 -- 172.21.15.146:0/3199083772 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3c1f1180 con 0x7fef3c0cc6e0 2024-09-13T18:30:30.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.934+0000 7fef38ff9700 1 -- 172.21.15.146:0/3199083772 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fef30003a60 con 0x7fef3c0cc6e0 2024-09-13T18:30:30.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.934+0000 7fef40b48700 1 -- 172.21.15.146:0/3199083772 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3c14eb40 con 0x7fef3c0cc6e0 2024-09-13T18:30:30.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.934+0000 7fef38ff9700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fef1c066760 0x7fef1c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:30.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.934+0000 7fef38ff9700 1 -- 172.21.15.146:0/3199083772 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fef30021070 con 0x7fef3c0cc6e0 2024-09-13T18:30:30.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.934+0000 7fef3b7fe700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fef1c066760 0x7fef1c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:30.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.934+0000 7fef3b7fe700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fef1c066760 0x7fef1c068c20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0012e0 tx=0x7fef2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:30.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:30.942+0000 7fef38ff9700 1 -- 172.21.15.146:0/3199083772 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fef30053250 con 0x7fef3c0cc6e0 2024-09-13T18:30:31.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.174+0000 7fef40b48700 1 -- 172.21.15.146:0/3199083772 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef3c151460 con 0x7fef3c0cc6e0 2024-09-13T18:30:31.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.174+0000 7fef38ff9700 1 -- 172.21.15.146:0/3199083772 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5527 (secure 0 0 0) 0x7fef30056870 con 0x7fef3c0cc6e0 2024-09-13T18:30:31.183 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:31.183 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:30.599542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:31.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.178+0000 7fef40b48700 1 -- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fef1c066760 msgr2=0x7fef1c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:31.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.178+0000 7fef40b48700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fef1c066760 0x7fef1c068c20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0012e0 tx=0x7fef2c009040 comp rx=0 tx=0).stop 2024-09-13T18:30:31.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.178+0000 7fef40b48700 1 -- 172.21.15.146:0/3199083772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef3c0cc6e0 msgr2=0x7fef3c149850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:31.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.178+0000 7fef40b48700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef3c0cc6e0 0x7fef3c149850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fef30009c60 tx=0x7fef30004770 comp rx=0 tx=0).stop 2024-09-13T18:30:31.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.178+0000 7fef40b48700 1 -- 172.21.15.146:0/3199083772 shutdown_connections 2024-09-13T18:30:31.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.178+0000 7fef40b48700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fef1c066760 0x7fef1c068c20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:31.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.178+0000 7fef40b48700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef3c151080 0x7fef3c1de1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:31.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.178+0000 7fef40b48700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef3c0cc6e0 0x7fef3c149850 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:31.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.178+0000 7fef40b48700 1 --2- 172.21.15.146:0/3199083772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef3c0c6960 0x7fef3c148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:31.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.178+0000 7fef40b48700 1 -- 172.21.15.146:0/3199083772 >> 172.21.15.146:0/3199083772 conn(0x7fef3c071ed0 msgr2=0x7fef3c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:31.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.178+0000 7fef40b48700 1 -- 172.21.15.146:0/3199083772 shutdown_connections 2024-09-13T18:30:31.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:31.178+0000 7fef40b48700 1 -- 172.21.15.146:0/3199083772 wait complete. 2024-09-13T18:30:31.187 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 28 2024-09-13T18:30:31.200 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 28, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 28, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4290": { "addr": "172.21.15.146:6837/31041557", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 31041557, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 31041557, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4290, "incarnation": 4, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_5235": { "addr": "172.21.15.146:6833/1498985282", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 1498985282, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 1498985282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5235, "incarnation": 22, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 19 }, "gid_5278": { "addr": "172.21.15.154:6839/3274220802", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 3274220802, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 3274220802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5278, "incarnation": 28, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 30, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:30:30.599542+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4290, "mds_1": 5278, "mds_2": 5235 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:30:33.201 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:33.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.318+0000 7f6c3d30c700 1 Processor -- start 2024-09-13T18:30:33.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.318+0000 7f6c3d30c700 1 -- start start 2024-09-13T18:30:33.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.318+0000 7f6c3d30c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6c380c8f70 0x7f6c380a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:33.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3d30c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c380c9350 0x7f6c380723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:33.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3d30c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c38159f80 con 0x7f6c380c7530 2024-09-13T18:30:33.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3d30c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c380ca1d0 con 0x7f6c380c8f70 2024-09-13T18:30:33.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3d30c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c38159670 con 0x7f6c380c9350 2024-09-13T18:30:33.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3cb0b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6c380c7530 0x7f6c380cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49476/0 (socket says 172.21.15.146:49476) 2024-09-13T18:30:33.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3cb0b700 1 -- 172.21.15.146:0/1377367459 learned_addr learned my addr 172.21.15.146:0/1377367459 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:30:33.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c37fff700 1 --2- 172.21.15.146:0/1377367459 >> v2:172.21.15.154:3300/0 conn(0x7f6c380c8f70 0x7f6c380a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:33.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c377fe700 1 --2- 172.21.15.146:0/1377367459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c380c9350 0x7f6c380723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:33.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c36ffd700 1 -- 172.21.15.146:0/1377367459 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593904077 0 0) 0x7f6c38159f80 con 0x7f6c380c7530 2024-09-13T18:30:33.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c36ffd700 1 -- 172.21.15.146:0/1377367459 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c200035d0 con 0x7f6c380c7530 2024-09-13T18:30:33.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c37fff700 1 -- 172.21.15.146:0/1377367459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c380c9350 msgr2=0x7f6c380723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:33.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c37fff700 1 --2- 172.21.15.146:0/1377367459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c380c9350 0x7f6c380723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:33.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c37fff700 1 -- 172.21.15.146:0/1377367459 >> v1:172.21.15.146:6789/0 conn(0x7f6c380c7530 legacy=0x7f6c380cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:33.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c37fff700 1 -- 172.21.15.146:0/1377367459 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c380aa3a0 con 0x7f6c380c8f70 2024-09-13T18:30:33.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c37fff700 1 --2- 172.21.15.146:0/1377367459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c380c8f70 0x7f6c380a9cf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009850 tx=0x7f6c28009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=990692f4991c00af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:33.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c36ffd700 1 -- 172.21.15.146:0/1377367459 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28011830 con 0x7f6c380c8f70 2024-09-13T18:30:33.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c36ffd700 1 -- 172.21.15.146:0/1377367459 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c28011990 con 0x7f6c380c8f70 2024-09-13T18:30:33.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c36ffd700 1 -- 172.21.15.146:0/1377367459 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28011d00 con 0x7f6c380c8f70 2024-09-13T18:30:33.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3d30c700 1 -- 172.21.15.146:0/1377367459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c380c8f70 msgr2=0x7f6c380a9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:33.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3d30c700 1 --2- 172.21.15.146:0/1377367459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c380c8f70 0x7f6c380a9cf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009850 tx=0x7f6c28009b60 comp rx=0 tx=0).stop 2024-09-13T18:30:33.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3d30c700 1 -- 172.21.15.146:0/1377367459 shutdown_connections 2024-09-13T18:30:33.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3d30c700 1 --2- 172.21.15.146:0/1377367459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c380c9350 0x7f6c380723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:33.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3d30c700 1 --2- 172.21.15.146:0/1377367459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c380c8f70 0x7f6c380a9cf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:33.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3d30c700 1 -- 172.21.15.146:0/1377367459 >> 172.21.15.146:0/1377367459 conn(0x7f6c38066090 msgr2=0x7f6c38066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:33.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3d30c700 1 -- 172.21.15.146:0/1377367459 shutdown_connections 2024-09-13T18:30:33.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.322+0000 7f6c3d30c700 1 -- 172.21.15.146:0/1377367459 wait complete. 2024-09-13T18:30:33.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3d30c700 1 Processor -- start 2024-09-13T18:30:33.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3d30c700 1 -- start start 2024-09-13T18:30:33.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3d30c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c380c7530 0x7f6c380af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:33.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3d30c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c380c8f70 0x7f6c380aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:33.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3d30c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c380c9350 0x7f6c38153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:33.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3d30c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380c1380 con 0x7f6c380c9350 2024-09-13T18:30:33.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3d30c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380c53b0 con 0x7f6c380c8f70 2024-09-13T18:30:33.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3d30c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380c4df0 con 0x7f6c380c7530 2024-09-13T18:30:33.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3cb0b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c380c9350 0x7f6c38153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:33.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3cb0b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c380c9350 0x7f6c38153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54038/0 (socket says 172.21.15.146:54038) 2024-09-13T18:30:33.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c37fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c380c7530 0x7f6c380af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:33.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3cb0b700 1 -- 172.21.15.146:0/3080667392 learned_addr learned my addr 172.21.15.146:0/3080667392 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:33.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c377fe700 1 --2- 172.21.15.146:0/3080667392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c380c8f70 0x7f6c380aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:33.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3cb0b700 1 -- 172.21.15.146:0/3080667392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c380c7530 msgr2=0x7f6c380af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:33.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3cb0b700 1 --2- 172.21.15.146:0/3080667392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c380c7530 0x7f6c380af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:33.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3cb0b700 1 -- 172.21.15.146:0/3080667392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c380c8f70 msgr2=0x7f6c380aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:33.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3cb0b700 1 --2- 172.21.15.146:0/3080667392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c380c8f70 0x7f6c380aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:33.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3cb0b700 1 -- 172.21.15.146:0/3080667392 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c280094b0 con 0x7f6c380c9350 2024-09-13T18:30:33.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.326+0000 7f6c3cb0b700 1 --2- 172.21.15.146:0/3080667392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c380c9350 0x7f6c38153e40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000cb10 tx=0x7f6c3000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:33.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.330+0000 7f6c34ff9700 1 -- 172.21.15.146:0/3080667392 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30007480 con 0x7f6c380c9350 2024-09-13T18:30:33.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.330+0000 7f6c34ff9700 1 -- 172.21.15.146:0/3080667392 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c300075e0 con 0x7f6c380c9350 2024-09-13T18:30:33.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.330+0000 7f6c3d30c700 1 -- 172.21.15.146:0/3080667392 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c380c50d0 con 0x7f6c380c9350 2024-09-13T18:30:33.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.330+0000 7f6c34ff9700 1 -- 172.21.15.146:0/3080667392 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30007970 con 0x7f6c380c9350 2024-09-13T18:30:33.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.330+0000 7f6c3d30c700 1 -- 172.21.15.146:0/3080667392 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c380c48e0 con 0x7f6c380c9350 2024-09-13T18:30:33.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.330+0000 7f6c3d30c700 1 -- 172.21.15.146:0/3080667392 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c18005310 con 0x7f6c380c9350 2024-09-13T18:30:33.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.330+0000 7f6c34ff9700 1 -- 172.21.15.146:0/3080667392 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6c3001b3b0 con 0x7f6c380c9350 2024-09-13T18:30:33.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.334+0000 7f6c34ff9700 1 --2- 172.21.15.146:0/3080667392 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c24066730 0x7f6c24068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:33.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.334+0000 7f6c34ff9700 1 -- 172.21.15.146:0/3080667392 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f6c30083e50 con 0x7f6c380c9350 2024-09-13T18:30:33.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.334+0000 7f6c37fff700 1 --2- 172.21.15.146:0/3080667392 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c24066730 0x7f6c24068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:33.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.334+0000 7f6c37fff700 1 --2- 172.21.15.146:0/3080667392 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c24066730 0x7f6c24068bf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6c280090b0 tx=0x7f6c28008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:33.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.338+0000 7f6c34ff9700 1 -- 172.21.15.146:0/3080667392 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6c30050d90 con 0x7f6c380c9350 2024-09-13T18:30:33.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.574+0000 7f6c3d30c700 1 -- 172.21.15.146:0/3080667392 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c18005180 con 0x7f6c380c9350 2024-09-13T18:30:33.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.574+0000 7f6c34ff9700 1 -- 172.21.15.146:0/3080667392 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5527 (secure 0 0 0) 0x7f6c30017090 con 0x7f6c380c9350 2024-09-13T18:30:33.583 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:33.583 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:30.599542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:33.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.578+0000 7f6c3d30c700 1 -- 172.21.15.146:0/3080667392 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c24066730 msgr2=0x7f6c24068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:33.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.578+0000 7f6c3d30c700 1 --2- 172.21.15.146:0/3080667392 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c24066730 0x7f6c24068bf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6c280090b0 tx=0x7f6c28008030 comp rx=0 tx=0).stop 2024-09-13T18:30:33.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.578+0000 7f6c3d30c700 1 -- 172.21.15.146:0/3080667392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c380c9350 msgr2=0x7f6c38153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:33.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.578+0000 7f6c3d30c700 1 --2- 172.21.15.146:0/3080667392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c380c9350 0x7f6c38153e40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000cb10 tx=0x7f6c3000cb40 comp rx=0 tx=0).stop 2024-09-13T18:30:33.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.578+0000 7f6c3d30c700 1 -- 172.21.15.146:0/3080667392 shutdown_connections 2024-09-13T18:30:33.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.578+0000 7f6c3d30c700 1 --2- 172.21.15.146:0/3080667392 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6c24066730 0x7f6c24068bf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:33.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.578+0000 7f6c3d30c700 1 --2- 172.21.15.146:0/3080667392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c380c9350 0x7f6c38153e40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:33.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.578+0000 7f6c3d30c700 1 --2- 172.21.15.146:0/3080667392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c380c8f70 0x7f6c380aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:33.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.578+0000 7f6c3d30c700 1 --2- 172.21.15.146:0/3080667392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c380c7530 0x7f6c380af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:33.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.578+0000 7f6c3d30c700 1 -- 172.21.15.146:0/3080667392 >> 172.21.15.146:0/3080667392 conn(0x7f6c38066090 msgr2=0x7f6c380c0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:33.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.578+0000 7f6c3d30c700 1 -- 172.21.15.146:0/3080667392 shutdown_connections 2024-09-13T18:30:33.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:33.578+0000 7f6c3d30c700 1 -- 172.21.15.146:0/3080667392 wait complete. 2024-09-13T18:30:33.589 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 28 2024-09-13T18:30:33.601 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:30:35.603 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:35.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.710+0000 7f500ffff700 1 Processor -- start 2024-09-13T18:30:35.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.710+0000 7f500ffff700 1 -- start start 2024-09-13T18:30:35.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.710+0000 7f500ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5010150230 0x7f501006ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.710+0000 7f500ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5010071610 0x7f5010159b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.710+0000 7f500ffff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5010071410 con 0x7f5010150610 2024-09-13T18:30:35.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.710+0000 7f500ffff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50101596b0 con 0x7f5010071610 2024-09-13T18:30:35.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.710+0000 7f500ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5010156090 con 0x7f5010150230 2024-09-13T18:30:35.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.710+0000 7f500effd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5010150230 0x7f501006ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.710+0000 7f500f7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5010071610 0x7f5010159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500effd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5010150230 0x7f501006ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:58292/0 (socket says 172.21.15.146:58292) 2024-09-13T18:30:35.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500effd700 1 -- 172.21.15.146:0/1147444459 learned_addr learned my addr 172.21.15.146:0/1147444459 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:35.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500f7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5010071610 0x7f5010159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:36240/0 (socket says 172.21.15.146:36240) 2024-09-13T18:30:35.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500e7fc700 1 --1- 172.21.15.146:0/1147444459 >> v1:172.21.15.154:6789/0 conn(0x7f5010150610 0x7f501006d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:48292/0 (socket says 172.21.15.146:48292) 2024-09-13T18:30:35.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500effd700 1 -- 172.21.15.146:0/1147444459 >> v2:172.21.15.154:3300/0 conn(0x7f5010071610 msgr2=0x7f5010159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:35.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500effd700 1 --2- 172.21.15.146:0/1147444459 >> v2:172.21.15.154:3300/0 conn(0x7f5010071610 0x7f5010159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500effd700 1 -- 172.21.15.146:0/1147444459 >> v1:172.21.15.154:6789/0 conn(0x7f5010150610 legacy=0x7f501006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500effd700 1 -- 172.21.15.146:0/1147444459 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5010156350 con 0x7f5010150230 2024-09-13T18:30:35.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500f7fe700 1 --2- 172.21.15.146:0/1147444459 >> v2:172.21.15.154:3300/0 conn(0x7f5010071610 0x7f5010159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:30:35.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500effd700 1 --2- 172.21.15.146:0/1147444459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5010150230 0x7f501006ebd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f500000bbb0 tx=0x7f500000bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8946a37754137dc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:35.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500dffb700 1 -- 172.21.15.146:0/1147444459 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500000dd30 con 0x7f5010150230 2024-09-13T18:30:35.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500dffb700 1 -- 172.21.15.146:0/1147444459 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500000de90 con 0x7f5010150230 2024-09-13T18:30:35.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500dffb700 1 -- 172.21.15.146:0/1147444459 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000010690 con 0x7f5010150230 2024-09-13T18:30:35.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500ffff700 1 -- 172.21.15.146:0/1147444459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5010150230 msgr2=0x7f501006ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500ffff700 1 --2- 172.21.15.146:0/1147444459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5010150230 0x7f501006ebd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f500000bbb0 tx=0x7f500000bec0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500ffff700 1 -- 172.21.15.146:0/1147444459 shutdown_connections 2024-09-13T18:30:35.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500ffff700 1 --2- 172.21.15.146:0/1147444459 >> v2:172.21.15.154:3300/0 conn(0x7f5010071610 0x7f5010159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500ffff700 1 --2- 172.21.15.146:0/1147444459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5010150230 0x7f501006ebd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500ffff700 1 -- 172.21.15.146:0/1147444459 >> 172.21.15.146:0/1147444459 conn(0x7f5010071ed0 msgr2=0x7f50100722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:35.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500ffff700 1 -- 172.21.15.146:0/1147444459 shutdown_connections 2024-09-13T18:30:35.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500ffff700 1 -- 172.21.15.146:0/1147444459 wait complete. 2024-09-13T18:30:35.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.714+0000 7f500ffff700 1 Processor -- start 2024-09-13T18:30:35.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500ffff700 1 -- start start 2024-09-13T18:30:35.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5010071610 0x7f50100cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5010150230 0x7f5010146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5010153cf0 0x7f5010146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5010158650 con 0x7f5010150230 2024-09-13T18:30:35.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500ffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50101ee7d0 con 0x7f5010071610 2024-09-13T18:30:35.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5010158ca0 con 0x7f5010153cf0 2024-09-13T18:30:35.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5010150230 0x7f5010146840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5010150230 0x7f5010146840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54040/0 (socket says 172.21.15.146:54040) 2024-09-13T18:30:35.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500effd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5010071610 0x7f50100cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500e7fc700 1 -- 172.21.15.146:0/736914813 learned_addr learned my addr 172.21.15.146:0/736914813 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:35.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500f7fe700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5010153cf0 0x7f5010146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500e7fc700 1 -- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5010153cf0 msgr2=0x7f5010146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500e7fc700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5010153cf0 0x7f5010146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500e7fc700 1 -- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5010071610 msgr2=0x7f50100cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500e7fc700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5010071610 0x7f50100cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500e7fc700 1 -- 172.21.15.146:0/736914813 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5008009a70 con 0x7f5010150230 2024-09-13T18:30:35.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500f7fe700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5010153cf0 0x7f5010146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:30:35.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500e7fc700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5010150230 0x7f5010146840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f500400c460 tx=0x7f500400c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:35.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f4ff77fe700 1 -- 172.21.15.146:0/736914813 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004005cd0 con 0x7f5010150230 2024-09-13T18:30:35.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f4ff77fe700 1 -- 172.21.15.146:0/736914813 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5004005e30 con 0x7f5010150230 2024-09-13T18:30:35.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f4ff77fe700 1 -- 172.21.15.146:0/736914813 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50040072f0 con 0x7f5010150230 2024-09-13T18:30:35.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500ffff700 1 -- 172.21.15.146:0/736914813 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500000b810 con 0x7f5010150230 2024-09-13T18:30:35.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.718+0000 7f500ffff700 1 -- 172.21.15.146:0/736914813 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5010153680 con 0x7f5010150230 2024-09-13T18:30:35.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.722+0000 7f500ffff700 1 -- 172.21.15.146:0/736914813 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f501006ebd0 con 0x7f5010150230 2024-09-13T18:30:35.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.722+0000 7f4ff77fe700 1 -- 172.21.15.146:0/736914813 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5004007450 con 0x7f5010150230 2024-09-13T18:30:35.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.722+0000 7f4ff77fe700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ffc066760 0x7f4ffc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.722+0000 7f4ff77fe700 1 -- 172.21.15.146:0/736914813 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f5004083960 con 0x7f5010150230 2024-09-13T18:30:35.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.722+0000 7f500effd700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ffc066760 0x7f4ffc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.722+0000 7f500effd700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ffc066760 0x7f4ffc068c20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f500000bfd0 tx=0x7f5000013000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:35.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.730+0000 7f4ff77fe700 1 -- 172.21.15.146:0/736914813 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5004050820 con 0x7f5010150230 2024-09-13T18:30:35.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.962+0000 7f500ffff700 1 -- 172.21.15.146:0/736914813 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f501014b150 con 0x7f5010150230 2024-09-13T18:30:35.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.962+0000 7f4ff77fe700 1 -- 172.21.15.146:0/736914813 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5529 (secure 0 0 0) 0x7f5004053e40 con 0x7f5010150230 2024-09-13T18:30:35.971 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:35.971 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:35.626909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:resolve","state_seq":13,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:35.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.966+0000 7f500ffff700 1 -- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ffc066760 msgr2=0x7f4ffc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.966+0000 7f500ffff700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ffc066760 0x7f4ffc068c20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f500000bfd0 tx=0x7f5000013000 comp rx=0 tx=0).stop 2024-09-13T18:30:35.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.966+0000 7f500ffff700 1 -- 172.21.15.146:0/736914813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5010150230 msgr2=0x7f5010146840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.966+0000 7f500ffff700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5010150230 0x7f5010146840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f500400c460 tx=0x7f500400c770 comp rx=0 tx=0).stop 2024-09-13T18:30:35.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.966+0000 7f500ffff700 1 -- 172.21.15.146:0/736914813 shutdown_connections 2024-09-13T18:30:35.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.966+0000 7f500ffff700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ffc066760 0x7f4ffc068c20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.966+0000 7f500ffff700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5010153cf0 0x7f5010146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.966+0000 7f500ffff700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5010150230 0x7f5010146840 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.966+0000 7f500ffff700 1 --2- 172.21.15.146:0/736914813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5010071610 0x7f50100cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.966+0000 7f500ffff700 1 -- 172.21.15.146:0/736914813 >> 172.21.15.146:0/736914813 conn(0x7f5010071ed0 msgr2=0x7f501006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:35.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.966+0000 7f500ffff700 1 -- 172.21.15.146:0/736914813 shutdown_connections 2024-09-13T18:30:35.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:35.966+0000 7f500ffff700 1 -- 172.21.15.146:0/736914813 wait complete. 2024-09-13T18:30:35.975 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 29 2024-09-13T18:30:35.985 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:30:37.564 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:30:37.567 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:30:37.986 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:38.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ed1a03700 1 Processor -- start 2024-09-13T18:30:38.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ed1a03700 1 -- start start 2024-09-13T18:30:38.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ed1a03700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ecc0c7970 0x7f4ecc06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:38.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ed1a03700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc0c98d0 con 0x7f4ecc153860 2024-09-13T18:30:38.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ed1a03700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc0706d0 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ed1a03700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc0c4200 con 0x7f4ecc0c7970 2024-09-13T18:30:38.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ed0a01700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4ecc153860 0x7f4ecc0c85e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49482/0 (socket says 172.21.15.146:49482) 2024-09-13T18:30:38.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ed0a01700 1 -- 172.21.15.146:0/2830558031 learned_addr learned my addr 172.21.15.146:0/2830558031 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:30:38.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ecbfff700 1 --2- 172.21.15.146:0/2830558031 >> v2:172.21.15.154:3300/0 conn(0x7f4ecc0c7970 0x7f4ecc06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:38.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ecb7fe700 1 -- 172.21.15.146:0/2830558031 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554162271 0 0) 0x7f4ec4001490 con 0x7f4ecc153860 2024-09-13T18:30:38.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ecb7fe700 1 -- 172.21.15.146:0/2830558031 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eb80035d0 con 0x7f4ecc153860 2024-09-13T18:30:38.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ecbfff700 1 -- 172.21.15.146:0/2830558031 >> v1:172.21.15.154:6789/0 conn(0x7f4ecc06c5b0 legacy=0x7f4ecc070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:38.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ecbfff700 1 -- 172.21.15.146:0/2830558031 >> v1:172.21.15.146:6789/0 conn(0x7f4ecc153860 legacy=0x7f4ecc0c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:38.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ecb7fe700 1 -- 172.21.15.146:0/2830558031 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 846583944 0 0) 0x7f4eb80035d0 con 0x7f4ecc153860 2024-09-13T18:30:38.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ecbfff700 1 -- 172.21.15.146:0/2830558031 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc159730 con 0x7f4ecc0c7970 2024-09-13T18:30:38.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ecbfff700 1 --2- 172.21.15.146:0/2830558031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c7970 0x7f4ecc06c070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009850 tx=0x7f4eb4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d2f13842698e1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:38.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.110+0000 7f4ecb7fe700 1 -- 172.21.15.146:0/2830558031 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4001b80 con 0x7f4ecc0c7970 2024-09-13T18:30:38.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.114+0000 7f4ecb7fe700 1 -- 172.21.15.146:0/2830558031 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb4004030 con 0x7f4ecc0c7970 2024-09-13T18:30:38.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.114+0000 7f4ecb7fe700 1 -- 172.21.15.146:0/2830558031 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40043a0 con 0x7f4ecc0c7970 2024-09-13T18:30:38.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.114+0000 7f4ed1a03700 1 -- 172.21.15.146:0/2830558031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c7970 msgr2=0x7f4ecc06c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:38.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.114+0000 7f4ed1a03700 1 --2- 172.21.15.146:0/2830558031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c7970 0x7f4ecc06c070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009850 tx=0x7f4eb4009b60 comp rx=0 tx=0).stop 2024-09-13T18:30:38.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.114+0000 7f4ed1a03700 1 -- 172.21.15.146:0/2830558031 shutdown_connections 2024-09-13T18:30:38.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.114+0000 7f4ed1a03700 1 --2- 172.21.15.146:0/2830558031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c7970 0x7f4ecc06c070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:38.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.114+0000 7f4ed1a03700 1 -- 172.21.15.146:0/2830558031 >> 172.21.15.146:0/2830558031 conn(0x7f4ecc0ccf70 msgr2=0x7f4ecc0cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:38.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.114+0000 7f4ed1a03700 1 -- 172.21.15.146:0/2830558031 shutdown_connections 2024-09-13T18:30:38.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.114+0000 7f4ed1a03700 1 -- 172.21.15.146:0/2830558031 wait complete. 2024-09-13T18:30:38.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.114+0000 7f4ed1a03700 1 Processor -- start 2024-09-13T18:30:38.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.114+0000 7f4ed1a03700 1 -- start start 2024-09-13T18:30:38.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed1a03700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc06c5b0 0x7f4ecc15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:38.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed1a03700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c7970 0x7f4ecc156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:38.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed1a03700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc153860 0x7f4ecc06a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:38.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed1a03700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0cc650 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed1a03700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0cae70 con 0x7f4ecc0c7970 2024-09-13T18:30:38.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed1a03700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc1ef240 con 0x7f4ecc153860 2024-09-13T18:30:38.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed0a01700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc06c5b0 0x7f4ecc15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:38.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed0a01700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc06c5b0 0x7f4ecc15bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54042/0 (socket says 172.21.15.146:54042) 2024-09-13T18:30:38.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed0a01700 1 -- 172.21.15.146:0/1966684205 learned_addr learned my addr 172.21.15.146:0/1966684205 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:38.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ecbfff700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c7970 0x7f4ecc156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:38.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed1202700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc153860 0x7f4ecc06a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:38.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed0a01700 1 -- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc153860 msgr2=0x7f4ecc06a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:38.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed0a01700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc153860 0x7f4ecc06a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:38.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed0a01700 1 -- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c7970 msgr2=0x7f4ecc156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:38.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed0a01700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c7970 0x7f4ecc156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:38.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed0a01700 1 -- 172.21.15.146:0/1966684205 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb40094b0 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ecbfff700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c7970 0x7f4ecc156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:30:38.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed0a01700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc06c5b0 0x7f4ecc15bfc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4001ad0 tx=0x7f4ec400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:38.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ec97fa700 1 -- 172.21.15.146:0/1966684205 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400ad80 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ec97fa700 1 -- 172.21.15.146:0/1966684205 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec40070d0 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed1a03700 1 -- 172.21.15.146:0/1966684205 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc0cc8f0 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ec97fa700 1 -- 172.21.15.146:0/1966684205 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4007480 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed1a03700 1 -- 172.21.15.146:0/1966684205 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc1ef500 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ed1a03700 1 -- 172.21.15.146:0/1966684205 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc067a60 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.118+0000 7f4ec97fa700 1 -- 172.21.15.146:0/1966684205 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4ec40075e0 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.122+0000 7f4ec97fa700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4eac066760 0x7f4eac068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:38.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.122+0000 7f4ec97fa700 1 -- 172.21.15.146:0/1966684205 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f4ec40843e0 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.122+0000 7f4ecbfff700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4eac066760 0x7f4eac068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:38.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.122+0000 7f4ecbfff700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4eac066760 0x7f4eac068c20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40090b0 tx=0x7f4eb4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:38.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.126+0000 7f4ec97fa700 1 -- 172.21.15.146:0/1966684205 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4ec4051320 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.374+0000 7f4ed1a03700 1 -- 172.21.15.146:0/1966684205 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ecc0c98d0 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.374+0000 7f4ec97fa700 1 -- 172.21.15.146:0/1966684205 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5529 (secure 0 0 0) 0x7f4ecc0c98d0 con 0x7f4ecc06c5b0 2024-09-13T18:30:38.382 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:38.382 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:35.626909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:resolve","state_seq":13,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:38.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.378+0000 7f4ed1a03700 1 -- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4eac066760 msgr2=0x7f4eac068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:38.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.378+0000 7f4ed1a03700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4eac066760 0x7f4eac068c20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40090b0 tx=0x7f4eb4008030 comp rx=0 tx=0).stop 2024-09-13T18:30:38.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.378+0000 7f4ed1a03700 1 -- 172.21.15.146:0/1966684205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc06c5b0 msgr2=0x7f4ecc15bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:38.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.378+0000 7f4ed1a03700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc06c5b0 0x7f4ecc15bfc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4001ad0 tx=0x7f4ec400a340 comp rx=0 tx=0).stop 2024-09-13T18:30:38.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.378+0000 7f4ed1a03700 1 -- 172.21.15.146:0/1966684205 shutdown_connections 2024-09-13T18:30:38.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.378+0000 7f4ed1a03700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4eac066760 0x7f4eac068c20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:38.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.378+0000 7f4ed1a03700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc153860 0x7f4ecc06a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:38.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.378+0000 7f4ed1a03700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c7970 0x7f4ecc156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:38.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.378+0000 7f4ed1a03700 1 --2- 172.21.15.146:0/1966684205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc06c5b0 0x7f4ecc15bfc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:38.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.378+0000 7f4ed1a03700 1 -- 172.21.15.146:0/1966684205 >> 172.21.15.146:0/1966684205 conn(0x7f4ecc0ccf70 msgr2=0x7f4ecc152830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:38.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.378+0000 7f4ed1a03700 1 -- 172.21.15.146:0/1966684205 shutdown_connections 2024-09-13T18:30:38.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:38.378+0000 7f4ed1a03700 1 -- 172.21.15.146:0/1966684205 wait complete. 2024-09-13T18:30:38.387 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 29 2024-09-13T18:30:38.401 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:30:40.401 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:40.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f388345e700 1 Processor -- start 2024-09-13T18:30:40.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f388345e700 1 -- start start 2024-09-13T18:30:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f388345e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f387c0cf170 0x7f387c1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f388345e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f387c0cb5b0 0x7f387c1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f388345e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f387c15a540 con 0x7f387c0cf550 2024-09-13T18:30:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f388345e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f387c0c5220 con 0x7f387c0cb5b0 2024-09-13T18:30:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f388345e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c159ca0 con 0x7f387c0cf170 2024-09-13T18:30:40.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f388245c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f387c0cf170 0x7f387c1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f3882c5d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f387c0cb5b0 0x7f387c1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f3882c5d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f387c0cb5b0 0x7f387c1534c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:36274/0 (socket says 172.21.15.146:36274) 2024-09-13T18:30:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f3882c5d700 1 -- 172.21.15.146:0/251407670 learned_addr learned my addr 172.21.15.146:0/251407670 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f3882c5d700 1 -- 172.21.15.146:0/251407670 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f387c0cf170 msgr2=0x7f387c1595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f3882c5d700 1 --2- 172.21.15.146:0/251407670 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f387c0cf170 0x7f387c1595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f3882c5d700 1 -- 172.21.15.146:0/251407670 >> v1:172.21.15.154:6789/0 conn(0x7f387c0cf550 legacy=0x7f387c0c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f3882c5d700 1 -- 172.21.15.146:0/251407670 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c159f80 con 0x7f387c0cb5b0 2024-09-13T18:30:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.510+0000 7f388245c700 1 --2- 172.21.15.146:0/251407670 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f387c0cf170 0x7f387c1595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:30:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f3882c5d700 1 --2- 172.21.15.146:0/251407670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f387c0cb5b0 0x7f387c1534c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3878009910 tx=0x7f3878009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c35471b4ac9acfa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f388145a700 1 -- 172.21.15.146:0/251407670 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878011830 con 0x7f387c0cb5b0 2024-09-13T18:30:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f388145a700 1 -- 172.21.15.146:0/251407670 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3878011990 con 0x7f387c0cb5b0 2024-09-13T18:30:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f388145a700 1 -- 172.21.15.146:0/251407670 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878011d40 con 0x7f387c0cb5b0 2024-09-13T18:30:40.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f388345e700 1 -- 172.21.15.146:0/251407670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f387c0cb5b0 msgr2=0x7f387c1534c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:40.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f388345e700 1 --2- 172.21.15.146:0/251407670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f387c0cb5b0 0x7f387c1534c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3878009910 tx=0x7f3878009c20 comp rx=0 tx=0).stop 2024-09-13T18:30:40.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f388345e700 1 -- 172.21.15.146:0/251407670 shutdown_connections 2024-09-13T18:30:40.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f388345e700 1 --2- 172.21.15.146:0/251407670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f387c0cb5b0 0x7f387c1534c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:40.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f388345e700 1 --2- 172.21.15.146:0/251407670 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f387c0cf170 0x7f387c1595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:40.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f388345e700 1 -- 172.21.15.146:0/251407670 >> 172.21.15.146:0/251407670 conn(0x7f387c072220 msgr2=0x7f387c066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:40.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f388345e700 1 -- 172.21.15.146:0/251407670 shutdown_connections 2024-09-13T18:30:40.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f388345e700 1 -- 172.21.15.146:0/251407670 wait complete. 2024-09-13T18:30:40.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f388345e700 1 Processor -- start 2024-09-13T18:30:40.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.514+0000 7f388345e700 1 -- start start 2024-09-13T18:30:40.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388345e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f387c0cb5b0 0x7f387c1df700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:40.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388345e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f387c0cf170 0x7f387c1f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388345e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f387c0cf550 0x7f387c0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388345e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c157660 con 0x7f387c0cb5b0 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388345e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0b5120 con 0x7f387c0cf170 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388345e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0c73e0 con 0x7f387c0cf550 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388245c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f387c0cb5b0 0x7f387c1df700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388245c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f387c0cb5b0 0x7f387c1df700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54044/0 (socket says 172.21.15.146:54044) 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388245c700 1 -- 172.21.15.146:0/3730280602 learned_addr learned my addr 172.21.15.146:0/3730280602 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f3882c5d700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f387c0cf550 0x7f387c0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f3881c5b700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f387c0cf170 0x7f387c1f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388245c700 1 -- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f387c0cf550 msgr2=0x7f387c0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388245c700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f387c0cf550 0x7f387c0b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388245c700 1 -- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f387c0cf170 msgr2=0x7f387c1f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388245c700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f387c0cf170 0x7f387c1f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:40.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388245c700 1 -- 172.21.15.146:0/3730280602 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3874009a70 con 0x7f387c0cb5b0 2024-09-13T18:30:40.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f3881c5b700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f387c0cf170 0x7f387c1f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:30:40.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f3882c5d700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f387c0cf550 0x7f387c0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:30:40.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388245c700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f387c0cb5b0 0x7f387c1df700 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f387400efc0 tx=0x7f387400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:40.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f3872ffd700 1 -- 172.21.15.146:0/3730280602 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387400cee0 con 0x7f387c0cb5b0 2024-09-13T18:30:40.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f3872ffd700 1 -- 172.21.15.146:0/3730280602 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387400f040 con 0x7f387c0cb5b0 2024-09-13T18:30:40.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388345e700 1 -- 172.21.15.146:0/3730280602 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3878009570 con 0x7f387c0cb5b0 2024-09-13T18:30:40.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f3872ffd700 1 -- 172.21.15.146:0/3730280602 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874014680 con 0x7f387c0cb5b0 2024-09-13T18:30:40.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388345e700 1 -- 172.21.15.146:0/3730280602 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c1f2a20 con 0x7f387c0cb5b0 2024-09-13T18:30:40.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.518+0000 7f388345e700 1 -- 172.21.15.146:0/3730280602 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387c1582c0 con 0x7f387c0cb5b0 2024-09-13T18:30:40.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.522+0000 7f3872ffd700 1 -- 172.21.15.146:0/3730280602 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f38740147e0 con 0x7f387c0cb5b0 2024-09-13T18:30:40.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.522+0000 7f3872ffd700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3860066730 0x7f3860068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:40.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.522+0000 7f3872ffd700 1 -- 172.21.15.146:0/3730280602 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f3874085440 con 0x7f387c0cb5b0 2024-09-13T18:30:40.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.522+0000 7f3881c5b700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3860066730 0x7f3860068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:40.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.522+0000 7f3881c5b700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3860066730 0x7f3860068bf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f387c1dfc40 tx=0x7f386c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:40.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.526+0000 7f3872ffd700 1 -- 172.21.15.146:0/3730280602 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3874052260 con 0x7f387c0cb5b0 2024-09-13T18:30:40.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.762+0000 7f388345e700 1 -- 172.21.15.146:0/3730280602 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f387c066850 con 0x7f387c0cb5b0 2024-09-13T18:30:40.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.762+0000 7f3872ffd700 1 -- 172.21.15.146:0/3730280602 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5531 (secure 0 0 0) 0x7f3874055880 con 0x7f387c0cb5b0 2024-09-13T18:30:40.772 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:40.772 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:40.646580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:40.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.766+0000 7f388345e700 1 -- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3860066730 msgr2=0x7f3860068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:40.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.766+0000 7f388345e700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3860066730 0x7f3860068bf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f387c1dfc40 tx=0x7f386c009040 comp rx=0 tx=0).stop 2024-09-13T18:30:40.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.766+0000 7f388345e700 1 -- 172.21.15.146:0/3730280602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f387c0cb5b0 msgr2=0x7f387c1df700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:40.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.766+0000 7f388345e700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f387c0cb5b0 0x7f387c1df700 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f387400efc0 tx=0x7f387400c3a0 comp rx=0 tx=0).stop 2024-09-13T18:30:40.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.766+0000 7f388345e700 1 -- 172.21.15.146:0/3730280602 shutdown_connections 2024-09-13T18:30:40.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.766+0000 7f388345e700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3860066730 0x7f3860068bf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:40.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.766+0000 7f388345e700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f387c0cf550 0x7f387c0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:40.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.766+0000 7f388345e700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f387c0cf170 0x7f387c1f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:40.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.766+0000 7f388345e700 1 --2- 172.21.15.146:0/3730280602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f387c0cb5b0 0x7f387c1df700 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:40.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.766+0000 7f388345e700 1 -- 172.21.15.146:0/3730280602 >> 172.21.15.146:0/3730280602 conn(0x7f387c072220 msgr2=0x7f387c152670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:40.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.766+0000 7f388345e700 1 -- 172.21.15.146:0/3730280602 shutdown_connections 2024-09-13T18:30:40.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:40.766+0000 7f388345e700 1 -- 172.21.15.146:0/3730280602 wait complete. 2024-09-13T18:30:40.775 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 30 2024-09-13T18:30:40.786 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:30:42.788 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:42.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.906+0000 7f106d306700 1 Processor -- start 2024-09-13T18:30:42.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.906+0000 7f106d306700 1 -- start start 2024-09-13T18:30:42.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106d306700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1068150230 0x7f106806ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:42.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106d306700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1068150610 0x7f106806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:42.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106d306700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1068071410 con 0x7f1068071610 2024-09-13T18:30:42.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106d306700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10681596b0 con 0x7f1068150230 2024-09-13T18:30:42.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106d306700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1068156090 con 0x7f1068150610 2024-09-13T18:30:42.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f1067fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1068150230 0x7f106806ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:42.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106cb05700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1068071610 0x7f1068159b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45016/0 (socket says 172.21.15.146:45016) 2024-09-13T18:30:42.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106cb05700 1 -- 172.21.15.146:0/2356460935 learned_addr learned my addr 172.21.15.146:0/2356460935 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:30:42.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f10677fe700 1 --2- 172.21.15.146:0/2356460935 >> v2:172.21.15.154:3300/0 conn(0x7f1068150610 0x7f106806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:42.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f1067fff700 1 -- 172.21.15.146:0/2356460935 >> v2:172.21.15.154:3300/0 conn(0x7f1068150610 msgr2=0x7f106806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:42.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f1067fff700 1 --2- 172.21.15.146:0/2356460935 >> v2:172.21.15.154:3300/0 conn(0x7f1068150610 0x7f106806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:42.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f1067fff700 1 -- 172.21.15.146:0/2356460935 >> v1:172.21.15.146:6789/0 conn(0x7f1068071610 legacy=0x7f1068159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:42.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f1067fff700 1 -- 172.21.15.146:0/2356460935 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1068156350 con 0x7f1068150230 2024-09-13T18:30:42.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f1067fff700 1 --2- 172.21.15.146:0/2356460935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1068150230 0x7f106806ebd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f106806e950 tx=0x7f105800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8e34f1043561ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:42.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f1066ffd700 1 -- 172.21.15.146:0/2356460935 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105800dd30 con 0x7f1068150230 2024-09-13T18:30:42.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f1066ffd700 1 -- 172.21.15.146:0/2356460935 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105800de90 con 0x7f1068150230 2024-09-13T18:30:42.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f1066ffd700 1 -- 172.21.15.146:0/2356460935 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058010690 con 0x7f1068150230 2024-09-13T18:30:42.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106d306700 1 -- 172.21.15.146:0/2356460935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1068150230 msgr2=0x7f106806ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:42.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106d306700 1 --2- 172.21.15.146:0/2356460935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1068150230 0x7f106806ebd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f106806e950 tx=0x7f105800bdd0 comp rx=0 tx=0).stop 2024-09-13T18:30:42.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106d306700 1 -- 172.21.15.146:0/2356460935 shutdown_connections 2024-09-13T18:30:42.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106d306700 1 --2- 172.21.15.146:0/2356460935 >> v2:172.21.15.154:3300/0 conn(0x7f1068150610 0x7f106806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:42.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106d306700 1 --2- 172.21.15.146:0/2356460935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1068150230 0x7f106806ebd0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:42.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106d306700 1 -- 172.21.15.146:0/2356460935 >> 172.21.15.146:0/2356460935 conn(0x7f1068071ed0 msgr2=0x7f10680722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:42.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106d306700 1 -- 172.21.15.146:0/2356460935 shutdown_connections 2024-09-13T18:30:42.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.910+0000 7f106d306700 1 -- 172.21.15.146:0/2356460935 wait complete. 2024-09-13T18:30:42.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106d306700 1 Processor -- start 2024-09-13T18:30:42.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106d306700 1 -- start start 2024-09-13T18:30:42.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106d306700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1068071610 0x7f10680cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:42.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106d306700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1068150230 0x7f10680b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:42.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106d306700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1068155a00 0x7f10680b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:42.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106d306700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681ef530 con 0x7f1068155a00 2024-09-13T18:30:42.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106d306700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681f2990 con 0x7f1068071610 2024-09-13T18:30:42.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106d306700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10681f2c50 con 0x7f1068150230 2024-09-13T18:30:42.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106cb05700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1068155a00 0x7f10680b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:42.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106cb05700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1068155a00 0x7f10680b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46918/0 (socket says 172.21.15.146:46918) 2024-09-13T18:30:42.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106cb05700 1 -- 172.21.15.146:0/1640394660 learned_addr learned my addr 172.21.15.146:0/1640394660 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:42.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f1067fff700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1068071610 0x7f10680cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:42.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f10677fe700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1068150230 0x7f10680b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:42.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106cb05700 1 -- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1068150230 msgr2=0x7f10680b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:42.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106cb05700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1068150230 0x7f10680b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:42.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106cb05700 1 -- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1068071610 msgr2=0x7f10680cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:42.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106cb05700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1068071610 0x7f10680cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:42.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106cb05700 1 -- 172.21.15.146:0/1640394660 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105800b750 con 0x7f1068155a00 2024-09-13T18:30:42.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.914+0000 7f106cb05700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1068155a00 0x7f10680b10d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f106000ca80 tx=0x7f106000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:42.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.918+0000 7f1064ff9700 1 -- 172.21.15.146:0/1640394660 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060007480 con 0x7f1068155a00 2024-09-13T18:30:42.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.918+0000 7f1064ff9700 1 -- 172.21.15.146:0/1640394660 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10600075e0 con 0x7f1068155a00 2024-09-13T18:30:42.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.918+0000 7f106d306700 1 -- 172.21.15.146:0/1640394660 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10681f2f30 con 0x7f1068155a00 2024-09-13T18:30:42.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.918+0000 7f1064ff9700 1 -- 172.21.15.146:0/1640394660 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060007950 con 0x7f1068155a00 2024-09-13T18:30:42.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.918+0000 7f106d306700 1 -- 172.21.15.146:0/1640394660 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10681f34f0 con 0x7f1068155a00 2024-09-13T18:30:42.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.918+0000 7f1064ff9700 1 -- 172.21.15.146:0/1640394660 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f106001b3b0 con 0x7f1068155a00 2024-09-13T18:30:42.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.918+0000 7f106d306700 1 -- 172.21.15.146:0/1640394660 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106804e7f0 con 0x7f1068155a00 2024-09-13T18:30:42.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.918+0000 7f1064ff9700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1054066760 0x7f1054068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:42.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.918+0000 7f1064ff9700 1 -- 172.21.15.146:0/1640394660 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f1060083db0 con 0x7f1068155a00 2024-09-13T18:30:42.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.922+0000 7f1067fff700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1054066760 0x7f1054068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:42.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.922+0000 7f1067fff700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1054066760 0x7f1054068c20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1058000c00 tx=0x7f105801b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:42.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:42.930+0000 7f1064ff9700 1 -- 172.21.15.146:0/1640394660 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1060050c90 con 0x7f1068155a00 2024-09-13T18:30:43.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.186+0000 7f106d306700 1 -- 172.21.15.146:0/1640394660 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10680c7750 con 0x7f1068155a00 2024-09-13T18:30:43.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.186+0000 7f1064ff9700 1 -- 172.21.15.146:0/1640394660 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5531 (secure 0 0 0) 0x7f10600542b0 con 0x7f1068155a00 2024-09-13T18:30:43.194 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:43.194 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:40.646580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:43.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.190+0000 7f106d306700 1 -- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1054066760 msgr2=0x7f1054068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:43.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.190+0000 7f106d306700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1054066760 0x7f1054068c20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1058000c00 tx=0x7f105801b040 comp rx=0 tx=0).stop 2024-09-13T18:30:43.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.190+0000 7f106d306700 1 -- 172.21.15.146:0/1640394660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1068155a00 msgr2=0x7f10680b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:43.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.190+0000 7f106d306700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1068155a00 0x7f10680b10d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f106000ca80 tx=0x7f106000cab0 comp rx=0 tx=0).stop 2024-09-13T18:30:43.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.190+0000 7f106d306700 1 -- 172.21.15.146:0/1640394660 shutdown_connections 2024-09-13T18:30:43.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.190+0000 7f106d306700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1054066760 0x7f1054068c20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:43.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.190+0000 7f106d306700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1068155a00 0x7f10680b10d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:43.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.190+0000 7f106d306700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1068150230 0x7f10680b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:43.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.190+0000 7f106d306700 1 --2- 172.21.15.146:0/1640394660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1068071610 0x7f10680cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:43.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.190+0000 7f106d306700 1 -- 172.21.15.146:0/1640394660 >> 172.21.15.146:0/1640394660 conn(0x7f1068071ed0 msgr2=0x7f10680c89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:43.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.190+0000 7f106d306700 1 -- 172.21.15.146:0/1640394660 shutdown_connections 2024-09-13T18:30:43.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:43.190+0000 7f106d306700 1 -- 172.21.15.146:0/1640394660 wait complete. 2024-09-13T18:30:43.198 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 30 2024-09-13T18:30:43.210 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 30, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 30, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4290": { "addr": "172.21.15.146:6837/31041557", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 31041557, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 31041557, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4290, "incarnation": 4, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_5235": { "addr": "172.21.15.146:6833/1498985282", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 1498985282, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 1498985282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5235, "incarnation": 22, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 19 }, "gid_5278": { "addr": "172.21.15.154:6839/3274220802", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 3274220802, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 3274220802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5278, "incarnation": 28, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:reconnect", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 30, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:30:40.646580+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4290, "mds_1": 5278, "mds_2": 5235 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:30:45.211 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:45.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb076543700 1 Processor -- start 2024-09-13T18:30:45.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb076543700 1 -- start start 2024-09-13T18:30:45.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb076543700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0700c63a0 0x7fb070159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:45.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb076543700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0700c8c70 con 0x7fb0700c7310 2024-09-13T18:30:45.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb076543700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0700c8f50 con 0x7fb0700c5fc0 2024-09-13T18:30:45.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb076543700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0700c9bd0 con 0x7fb0700c63a0 2024-09-13T18:30:45.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb074d40700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0700c63a0 0x7fb070159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:45.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb074d40700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0700c63a0 0x7fb070159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46932/0 (socket says 172.21.15.146:46932) 2024-09-13T18:30:45.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb074d40700 1 -- 172.21.15.146:0/966379024 learned_addr learned my addr 172.21.15.146:0/966379024 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:45.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb074d40700 1 -- 172.21.15.146:0/966379024 >> v1:172.21.15.154:6789/0 conn(0x7fb0700c5fc0 legacy=0x7fb0700c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:45.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb074d40700 1 -- 172.21.15.146:0/966379024 >> v1:172.21.15.146:6789/0 conn(0x7fb0700c7310 legacy=0x7fb070160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:45.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb074d40700 1 -- 172.21.15.146:0/966379024 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0700c9230 con 0x7fb0700c63a0 2024-09-13T18:30:45.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb074d40700 1 --2- 172.21.15.146:0/966379024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0700c63a0 0x7fb070159660 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb060009910 tx=0x7fb060009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06bc492a4c19298 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:45.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb067fff700 1 -- 172.21.15.146:0/966379024 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06000ba60 con 0x7fb0700c63a0 2024-09-13T18:30:45.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb067fff700 1 -- 172.21.15.146:0/966379024 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06000bbc0 con 0x7fb0700c63a0 2024-09-13T18:30:45.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.322+0000 7fb067fff700 1 -- 172.21.15.146:0/966379024 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060009e60 con 0x7fb0700c63a0 2024-09-13T18:30:45.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.326+0000 7fb076543700 1 -- 172.21.15.146:0/966379024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0700c63a0 msgr2=0x7fb070159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:45.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.326+0000 7fb076543700 1 --2- 172.21.15.146:0/966379024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0700c63a0 0x7fb070159660 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb060009910 tx=0x7fb060009c20 comp rx=0 tx=0).stop 2024-09-13T18:30:45.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.326+0000 7fb076543700 1 -- 172.21.15.146:0/966379024 shutdown_connections 2024-09-13T18:30:45.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.326+0000 7fb076543700 1 --2- 172.21.15.146:0/966379024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0700c63a0 0x7fb070159660 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:45.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.326+0000 7fb076543700 1 -- 172.21.15.146:0/966379024 >> 172.21.15.146:0/966379024 conn(0x7fb070066090 msgr2=0x7fb070066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:45.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.326+0000 7fb076543700 1 -- 172.21.15.146:0/966379024 shutdown_connections 2024-09-13T18:30:45.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.326+0000 7fb076543700 1 -- 172.21.15.146:0/966379024 wait complete. 2024-09-13T18:30:45.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.326+0000 7fb076543700 1 Processor -- start 2024-09-13T18:30:45.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.326+0000 7fb076543700 1 -- start start 2024-09-13T18:30:45.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb076543700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0700c5fc0 0x7fb0701ee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:45.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb076543700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0700c63a0 0x7fb0700c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:45.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb076543700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0700c7310 0x7fb0700c1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:45.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb076543700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb070150ad0 con 0x7fb0700c7310 2024-09-13T18:30:45.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb076543700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb070150d90 con 0x7fb0700c63a0 2024-09-13T18:30:45.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb076543700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb070151070 con 0x7fb0700c5fc0 2024-09-13T18:30:45.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb075d42700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0700c7310 0x7fb0700c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:45.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb075d42700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0700c7310 0x7fb0700c1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46942/0 (socket says 172.21.15.146:46942) 2024-09-13T18:30:45.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb075d42700 1 -- 172.21.15.146:0/2750563856 learned_addr learned my addr 172.21.15.146:0/2750563856 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:45.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb074d40700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0700c63a0 0x7fb0700c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:45.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb075541700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0700c5fc0 0x7fb0701ee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:45.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb075d42700 1 -- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0700c5fc0 msgr2=0x7fb0701ee6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:45.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb075d42700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0700c5fc0 0x7fb0701ee6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:45.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb075d42700 1 -- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0700c63a0 msgr2=0x7fb0700c0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:45.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb075d42700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0700c63a0 0x7fb0700c0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:45.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb075d42700 1 -- 172.21.15.146:0/2750563856 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb060009570 con 0x7fb0700c7310 2024-09-13T18:30:45.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb075d42700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0700c7310 0x7fb0700c1240 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb06800cb50 tx=0x7fb06800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:45.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb065ffb700 1 -- 172.21.15.146:0/2750563856 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb068003c20 con 0x7fb0700c7310 2024-09-13T18:30:45.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb065ffb700 1 -- 172.21.15.146:0/2750563856 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb068003d80 con 0x7fb0700c7310 2024-09-13T18:30:45.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb076543700 1 -- 172.21.15.146:0/2750563856 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0701edbe0 con 0x7fb0700c7310 2024-09-13T18:30:45.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb065ffb700 1 -- 172.21.15.146:0/2750563856 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06800aa20 con 0x7fb0700c7310 2024-09-13T18:30:45.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb076543700 1 -- 172.21.15.146:0/2750563856 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0701ee1a0 con 0x7fb0700c7310 2024-09-13T18:30:45.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.330+0000 7fb076543700 1 -- 172.21.15.146:0/2750563856 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0701eda40 con 0x7fb0700c7310 2024-09-13T18:30:45.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.334+0000 7fb065ffb700 1 -- 172.21.15.146:0/2750563856 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb06800ab80 con 0x7fb0700c7310 2024-09-13T18:30:45.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.334+0000 7fb065ffb700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb054066760 0x7fb054068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:45.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.334+0000 7fb065ffb700 1 -- 172.21.15.146:0/2750563856 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fb068084e20 con 0x7fb0700c7310 2024-09-13T18:30:45.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.334+0000 7fb075541700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb054066760 0x7fb054068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:45.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.334+0000 7fb075541700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb054066760 0x7fb054068c20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb06c000f60 tx=0x7fb06c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:45.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.338+0000 7fb065ffb700 1 -- 172.21.15.146:0/2750563856 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb0680558d0 con 0x7fb0700c7310 2024-09-13T18:30:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.570+0000 7fb076543700 1 -- 172.21.15.146:0/2750563856 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0701eda40 con 0x7fb0700c7310 2024-09-13T18:30:45.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.570+0000 7fb065ffb700 1 -- 172.21.15.146:0/2750563856 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5531 (secure 0 0 0) 0x7fb0701eda40 con 0x7fb0700c7310 2024-09-13T18:30:45.579 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:45.579 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:40.646580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:45.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.574+0000 7fb076543700 1 -- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb054066760 msgr2=0x7fb054068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:45.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.574+0000 7fb076543700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb054066760 0x7fb054068c20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb06c000f60 tx=0x7fb06c009040 comp rx=0 tx=0).stop 2024-09-13T18:30:45.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.574+0000 7fb076543700 1 -- 172.21.15.146:0/2750563856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0700c7310 msgr2=0x7fb0700c1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:45.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.574+0000 7fb076543700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0700c7310 0x7fb0700c1240 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb06800cb50 tx=0x7fb06800ce60 comp rx=0 tx=0).stop 2024-09-13T18:30:45.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.574+0000 7fb076543700 1 -- 172.21.15.146:0/2750563856 shutdown_connections 2024-09-13T18:30:45.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.574+0000 7fb076543700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb054066760 0x7fb054068c20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:45.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.574+0000 7fb076543700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0700c7310 0x7fb0700c1240 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:45.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.574+0000 7fb076543700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0700c63a0 0x7fb0700c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:45.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.574+0000 7fb076543700 1 --2- 172.21.15.146:0/2750563856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0700c5fc0 0x7fb0701ee6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:45.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.574+0000 7fb076543700 1 -- 172.21.15.146:0/2750563856 >> 172.21.15.146:0/2750563856 conn(0x7fb070066090 msgr2=0x7fb0700c6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:45.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.574+0000 7fb076543700 1 -- 172.21.15.146:0/2750563856 shutdown_connections 2024-09-13T18:30:45.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:45.574+0000 7fb076543700 1 -- 172.21.15.146:0/2750563856 wait complete. 2024-09-13T18:30:45.583 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 30 2024-09-13T18:30:45.594 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:30:47.595 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:47.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 Processor -- start 2024-09-13T18:30:47.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 -- start start 2024-09-13T18:30:47.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73801514b0 0x7f73801572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:47.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f73800b0160 0x7f73800b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:47.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f73800b27e0 0x7f73800b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:47.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738006ae30 con 0x7f73800b27e0 2024-09-13T18:30:47.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73800caa70 con 0x7f73800b0160 2024-09-13T18:30:47.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73800b0b70 con 0x7f73801514b0 2024-09-13T18:30:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73868c0700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f73800b27e0 0x7f73800b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73868c0700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f73800b27e0 0x7f73800b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46944/0 (socket says 172.21.15.146:46944) 2024-09-13T18:30:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73868c0700 1 -- 172.21.15.146:0/105681766 learned_addr learned my addr 172.21.15.146:0/105681766 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73858be700 1 --2- 172.21.15.146:0/105681766 >> v2:172.21.15.154:3300/0 conn(0x7f73800b0160 0x7f73800b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73868c0700 1 -- 172.21.15.146:0/105681766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73801514b0 msgr2=0x7f73801572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73860bf700 1 --2- 172.21.15.146:0/105681766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73801514b0 0x7f73801572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73868c0700 1 --2- 172.21.15.146:0/105681766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73801514b0 0x7f73801572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73868c0700 1 -- 172.21.15.146:0/105681766 >> v2:172.21.15.154:3300/0 conn(0x7f73800b0160 msgr2=0x7f73800b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73868c0700 1 --2- 172.21.15.146:0/105681766 >> v2:172.21.15.154:3300/0 conn(0x7f73800b0160 0x7f73800b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73868c0700 1 -- 172.21.15.146:0/105681766 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73800b0e30 con 0x7f73800b27e0 2024-09-13T18:30:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73858be700 1 --2- 172.21.15.146:0/105681766 >> v2:172.21.15.154:3300/0 conn(0x7f73800b0160 0x7f73800b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:30:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73860bf700 1 --2- 172.21.15.146:0/105681766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73801514b0 0x7f73801572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:30:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73868c0700 1 --2- 172.21.15.146:0/105681766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800b27e0 0x7f73800b2c90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7380067590 tx=0x7f737800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c35354c21a84eac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:47.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73850bd700 1 -- 172.21.15.146:0/105681766 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737800e070 con 0x7f73800b27e0 2024-09-13T18:30:47.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73850bd700 1 -- 172.21.15.146:0/105681766 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737800b9c0 con 0x7f73800b27e0 2024-09-13T18:30:47.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73850bd700 1 -- 172.21.15.146:0/105681766 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737800bd70 con 0x7f73800b27e0 2024-09-13T18:30:47.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 -- 172.21.15.146:0/105681766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800b27e0 msgr2=0x7f73800b2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:47.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 --2- 172.21.15.146:0/105681766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800b27e0 0x7f73800b2c90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7380067590 tx=0x7f737800b330 comp rx=0 tx=0).stop 2024-09-13T18:30:47.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 -- 172.21.15.146:0/105681766 shutdown_connections 2024-09-13T18:30:47.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 --2- 172.21.15.146:0/105681766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800b27e0 0x7f73800b2c90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:47.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 --2- 172.21.15.146:0/105681766 >> v2:172.21.15.154:3300/0 conn(0x7f73800b0160 0x7f73800b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:47.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 --2- 172.21.15.146:0/105681766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73801514b0 0x7f73801572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:47.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 -- 172.21.15.146:0/105681766 >> 172.21.15.146:0/105681766 conn(0x7f738006c2d0 msgr2=0x7f738006c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:47.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 -- 172.21.15.146:0/105681766 shutdown_connections 2024-09-13T18:30:47.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.706+0000 7f73870c1700 1 -- 172.21.15.146:0/105681766 wait complete. 2024-09-13T18:30:47.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73870c1700 1 Processor -- start 2024-09-13T18:30:47.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73870c1700 1 -- start start 2024-09-13T18:30:47.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73870c1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800b0160 0x7f73800cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:47.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73870c1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73800b27e0 0x7f73801eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:47.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73870c1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73801514b0 0x7f73801501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:47.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73870c1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7380154a30 con 0x7f73800b0160 2024-09-13T18:30:47.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73870c1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7380154d10 con 0x7f73801514b0 2024-09-13T18:30:47.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73870c1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7380070ed0 con 0x7f73800b27e0 2024-09-13T18:30:47.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73860bf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800b0160 0x7f73800cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:47.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73860bf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800b0160 0x7f73800cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46950/0 (socket says 172.21.15.146:46950) 2024-09-13T18:30:47.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73860bf700 1 -- 172.21.15.146:0/968783214 learned_addr learned my addr 172.21.15.146:0/968783214 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:47.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73858be700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73800b27e0 0x7f73801eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:47.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73868c0700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73801514b0 0x7f73801501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:47.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73860bf700 1 -- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73800b27e0 msgr2=0x7f73801eefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:47.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73860bf700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73800b27e0 0x7f73801eefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:47.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73860bf700 1 -- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73801514b0 msgr2=0x7f73801501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:47.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73860bf700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73801514b0 0x7f73801501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:47.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73860bf700 1 -- 172.21.15.146:0/968783214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7378009b30 con 0x7f73800b0160 2024-09-13T18:30:47.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73868c0700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73801514b0 0x7f73801501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:30:47.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73860bf700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800b0160 0x7f73800cd2a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f737000c770 tx=0x7f737000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:47.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f7376ffd700 1 -- 172.21.15.146:0/968783214 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370011800 con 0x7f73800b0160 2024-09-13T18:30:47.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f7376ffd700 1 -- 172.21.15.146:0/968783214 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7370011960 con 0x7f73800b0160 2024-09-13T18:30:47.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f7376ffd700 1 -- 172.21.15.146:0/968783214 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370011d10 con 0x7f73800b0160 2024-09-13T18:30:47.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73870c1700 1 -- 172.21.15.146:0/968783214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73800711b0 con 0x7f73800b0160 2024-09-13T18:30:47.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.710+0000 7f73870c1700 1 -- 172.21.15.146:0/968783214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7380154200 con 0x7f73800b0160 2024-09-13T18:30:47.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.714+0000 7f73870c1700 1 -- 172.21.15.146:0/968783214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73800713f0 con 0x7f73800b0160 2024-09-13T18:30:47.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.714+0000 7f7376ffd700 1 -- 172.21.15.146:0/968783214 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7370018360 con 0x7f73800b0160 2024-09-13T18:30:47.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.714+0000 7f7376ffd700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7364066730 0x7f7364068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:47.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.714+0000 7f7376ffd700 1 -- 172.21.15.146:0/968783214 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f7370084720 con 0x7f73800b0160 2024-09-13T18:30:47.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.714+0000 7f73858be700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7364066730 0x7f7364068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:47.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.714+0000 7f73858be700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7364066730 0x7f7364068bf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7380154fb0 tx=0x7f737c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:47.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.722+0000 7f7376ffd700 1 -- 172.21.15.146:0/968783214 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f73800713f0 con 0x7f73800b0160 2024-09-13T18:30:47.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.946+0000 7f73870c1700 1 -- 172.21.15.146:0/968783214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7380152c40 con 0x7f73800b0160 2024-09-13T18:30:47.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.946+0000 7f7376ffd700 1 -- 172.21.15.146:0/968783214 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5528 (secure 0 0 0) 0x7f7380152c40 con 0x7f73800b0160 2024-09-13T18:30:47.954 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:47.954 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:45.683261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:47.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.950+0000 7f73870c1700 1 -- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7364066730 msgr2=0x7f7364068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:47.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.950+0000 7f73870c1700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7364066730 0x7f7364068bf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7380154fb0 tx=0x7f737c008040 comp rx=0 tx=0).stop 2024-09-13T18:30:47.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.950+0000 7f73870c1700 1 -- 172.21.15.146:0/968783214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800b0160 msgr2=0x7f73800cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:47.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.950+0000 7f73870c1700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800b0160 0x7f73800cd2a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f737000c770 tx=0x7f737000cb30 comp rx=0 tx=0).stop 2024-09-13T18:30:47.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.950+0000 7f73870c1700 1 -- 172.21.15.146:0/968783214 shutdown_connections 2024-09-13T18:30:47.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.950+0000 7f73870c1700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7364066730 0x7f7364068bf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:47.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.950+0000 7f73870c1700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73801514b0 0x7f73801501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:47.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.950+0000 7f73870c1700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73800b27e0 0x7f73801eefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:47.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.950+0000 7f73870c1700 1 --2- 172.21.15.146:0/968783214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73800b0160 0x7f73800cd2a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:47.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.950+0000 7f73870c1700 1 -- 172.21.15.146:0/968783214 >> 172.21.15.146:0/968783214 conn(0x7f738006c2d0 msgr2=0x7f7380156df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:47.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.950+0000 7f73870c1700 1 -- 172.21.15.146:0/968783214 shutdown_connections 2024-09-13T18:30:47.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:47.950+0000 7f73870c1700 1 -- 172.21.15.146:0/968783214 wait complete. 2024-09-13T18:30:47.958 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 31 2024-09-13T18:30:47.973 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:30:49.974 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:50.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.090+0000 7f903fa5c700 1 Processor -- start 2024-09-13T18:30:50.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.090+0000 7f903fa5c700 1 -- start start 2024-09-13T18:30:50.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903fa5c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90380c73a0 0x7f903806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:50.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903fa5c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9038150ae0 0x7f903806e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:50.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903fa5c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90380c5380 con 0x7f90380c7780 2024-09-13T18:30:50.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903fa5c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90380c5940 con 0x7f9038150ae0 2024-09-13T18:30:50.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903fa5c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90380c2850 con 0x7f90380c73a0 2024-09-13T18:30:50.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903dff9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9038150ae0 0x7f903806e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:50.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903cff7700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f90380c7780 0x7f90381503d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45028/0 (socket says 172.21.15.146:45028) 2024-09-13T18:30:50.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903cff7700 1 -- 172.21.15.146:0/2739115623 learned_addr learned my addr 172.21.15.146:0/2739115623 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:30:50.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903d7f8700 1 --2- 172.21.15.146:0/2739115623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90380c73a0 0x7f903806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:50.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903dff9700 1 -- 172.21.15.146:0/2739115623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90380c73a0 msgr2=0x7f903806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:50.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903dff9700 1 --2- 172.21.15.146:0/2739115623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90380c73a0 0x7f903806d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:50.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903dff9700 1 -- 172.21.15.146:0/2739115623 >> v1:172.21.15.146:6789/0 conn(0x7f90380c7780 legacy=0x7f90381503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:50.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903dff9700 1 -- 172.21.15.146:0/2739115623 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90380c2f40 con 0x7f9038150ae0 2024-09-13T18:30:50.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f903dff9700 1 --2- 172.21.15.146:0/2739115623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9038150ae0 0x7f903806e4e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9030009d90 tx=0x7f903000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6e0f3f5d9fb0c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:50.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f902ffff700 1 -- 172.21.15.146:0/2739115623 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903000e070 con 0x7f9038150ae0 2024-09-13T18:30:50.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f902ffff700 1 -- 172.21.15.146:0/2739115623 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903000bbd0 con 0x7f9038150ae0 2024-09-13T18:30:50.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.094+0000 7f902ffff700 1 -- 172.21.15.146:0/2739115623 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030012430 con 0x7f9038150ae0 2024-09-13T18:30:50.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.098+0000 7f903fa5c700 1 -- 172.21.15.146:0/2739115623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9038150ae0 msgr2=0x7f903806e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:50.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.098+0000 7f903fa5c700 1 --2- 172.21.15.146:0/2739115623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9038150ae0 0x7f903806e4e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9030009d90 tx=0x7f903000b330 comp rx=0 tx=0).stop 2024-09-13T18:30:50.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.098+0000 7f903fa5c700 1 -- 172.21.15.146:0/2739115623 shutdown_connections 2024-09-13T18:30:50.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.098+0000 7f903fa5c700 1 --2- 172.21.15.146:0/2739115623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9038150ae0 0x7f903806e4e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:50.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.098+0000 7f903fa5c700 1 --2- 172.21.15.146:0/2739115623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90380c73a0 0x7f903806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:50.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.098+0000 7f903fa5c700 1 -- 172.21.15.146:0/2739115623 >> 172.21.15.146:0/2739115623 conn(0x7f903806ada0 msgr2=0x7f903806b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:50.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.098+0000 7f903fa5c700 1 -- 172.21.15.146:0/2739115623 shutdown_connections 2024-09-13T18:30:50.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.098+0000 7f903fa5c700 1 -- 172.21.15.146:0/2739115623 wait complete. 2024-09-13T18:30:50.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.098+0000 7f903fa5c700 1 Processor -- start 2024-09-13T18:30:50.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.098+0000 7f903fa5c700 1 -- start start 2024-09-13T18:30:50.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903fa5c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90380c73a0 0x7f9038145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:50.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903fa5c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90380c7780 0x7f9038153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:50.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903fa5c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9038150ae0 0x7f90381537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:50.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903fa5c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90380c9780 con 0x7f90380c7780 2024-09-13T18:30:50.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903fa5c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90380c9a60 con 0x7f9038150ae0 2024-09-13T18:30:50.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903fa5c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9038151f10 con 0x7f90380c73a0 2024-09-13T18:30:50.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903cff7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90380c7780 0x7f9038153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:50.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903cff7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90380c7780 0x7f9038153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46958/0 (socket says 172.21.15.146:46958) 2024-09-13T18:30:50.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903dff9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9038150ae0 0x7f90381537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:50.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903cff7700 1 -- 172.21.15.146:0/1244570289 learned_addr learned my addr 172.21.15.146:0/1244570289 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:50.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903d7f8700 1 --2- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90380c73a0 0x7f9038145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:50.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903dff9700 1 -- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90380c73a0 msgr2=0x7f9038145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:50.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903dff9700 1 --2- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90380c73a0 0x7f9038145e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:50.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903dff9700 1 -- 172.21.15.146:0/1244570289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90380c7780 msgr2=0x7f9038153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:50.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903dff9700 1 --2- 172.21.15.146:0/1244570289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90380c7780 0x7f9038153270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:50.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903dff9700 1 -- 172.21.15.146:0/1244570289 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9030009a70 con 0x7f9038150ae0 2024-09-13T18:30:50.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903dff9700 1 --2- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9038150ae0 0x7f90381537b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9030016000 tx=0x7f9030004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:50.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f902dffb700 1 -- 172.21.15.146:0/1244570289 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030012430 con 0x7f9038150ae0 2024-09-13T18:30:50.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f902dffb700 1 -- 172.21.15.146:0/1244570289 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9030020070 con 0x7f9038150ae0 2024-09-13T18:30:50.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903fa5c700 1 -- 172.21.15.146:0/1244570289 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90381521f0 con 0x7f9038150ae0 2024-09-13T18:30:50.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f902dffb700 1 -- 172.21.15.146:0/1244570289 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030004280 con 0x7f9038150ae0 2024-09-13T18:30:50.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.102+0000 7f903fa5c700 1 -- 172.21.15.146:0/1244570289 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90381527b0 con 0x7f9038150ae0 2024-09-13T18:30:50.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.106+0000 7f903fa5c700 1 -- 172.21.15.146:0/1244570289 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9038150950 con 0x7f9038150ae0 2024-09-13T18:30:50.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.106+0000 7f902dffb700 1 -- 172.21.15.146:0/1244570289 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f90300043e0 con 0x7f9038150ae0 2024-09-13T18:30:50.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.106+0000 7f902dffb700 1 --2- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f901c066760 0x7f901c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:50.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.106+0000 7f902dffb700 1 -- 172.21.15.146:0/1244570289 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f9030022070 con 0x7f9038150ae0 2024-09-13T18:30:50.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.106+0000 7f903d7f8700 1 --2- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f901c066760 0x7f901c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:50.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.106+0000 7f903d7f8700 1 --2- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f901c066760 0x7f901c068c20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9034007a60 tx=0x7f9034008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:50.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.114+0000 7f902dffb700 1 -- 172.21.15.146:0/1244570289 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9030053170 con 0x7f9038150ae0 2024-09-13T18:30:50.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.362+0000 7f903fa5c700 1 -- 172.21.15.146:0/1244570289 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f903814f390 con 0x7f9038150ae0 2024-09-13T18:30:50.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.362+0000 7f902dffb700 1 -- 172.21.15.146:0/1244570289 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5528 (secure 0 0 0) 0x7f9030056790 con 0x7f9038150ae0 2024-09-13T18:30:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:50.370 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:45.683261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.366+0000 7f903fa5c700 1 -- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f901c066760 msgr2=0x7f901c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.366+0000 7f903fa5c700 1 --2- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f901c066760 0x7f901c068c20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9034007a60 tx=0x7f9034008040 comp rx=0 tx=0).stop 2024-09-13T18:30:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.366+0000 7f903fa5c700 1 -- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9038150ae0 msgr2=0x7f90381537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.366+0000 7f903fa5c700 1 --2- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9038150ae0 0x7f90381537b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9030016000 tx=0x7f9030004af0 comp rx=0 tx=0).stop 2024-09-13T18:30:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.366+0000 7f903fa5c700 1 -- 172.21.15.146:0/1244570289 shutdown_connections 2024-09-13T18:30:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.366+0000 7f903fa5c700 1 --2- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f901c066760 0x7f901c068c20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.366+0000 7f903fa5c700 1 --2- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9038150ae0 0x7f90381537b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.366+0000 7f903fa5c700 1 --2- 172.21.15.146:0/1244570289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90380c7780 0x7f9038153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.366+0000 7f903fa5c700 1 --2- 172.21.15.146:0/1244570289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90380c73a0 0x7f9038145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.366+0000 7f903fa5c700 1 -- 172.21.15.146:0/1244570289 >> 172.21.15.146:0/1244570289 conn(0x7f903806ada0 msgr2=0x7f903814de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.366+0000 7f903fa5c700 1 -- 172.21.15.146:0/1244570289 shutdown_connections 2024-09-13T18:30:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:50.366+0000 7f903fa5c700 1 -- 172.21.15.146:0/1244570289 wait complete. 2024-09-13T18:30:50.375 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 31 2024-09-13T18:30:50.388 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:30:52.257 INFO:tasks.ceph.mds.c.smithi146.stderr:2024-09-13T18:30:52.250+0000 7f6be472a700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-13T18:30:52.389 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:52.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75aa85b700 1 Processor -- start 2024-09-13T18:30:52.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75aa85b700 1 -- start start 2024-09-13T18:30:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75aa85b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f759c0d57e0 0x7f759c047720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75aa85b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f759c0077e0 0x7f759c0de990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75aa85b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759c047290 con 0x7f759c04d2a0 2024-09-13T18:30:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75aa85b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759c0d5bc0 con 0x7f759c0077e0 2024-09-13T18:30:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75aa85b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c047c60 con 0x7f759c0d57e0 2024-09-13T18:30:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75aa05a700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f759c04d2a0 0x7f759c0deed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34130/0 (socket says 172.21.15.146:34130) 2024-09-13T18:30:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75aa05a700 1 -- 172.21.15.146:0/3558375214 learned_addr learned my addr 172.21.15.146:0/3558375214 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:30:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a9058700 1 --2- 172.21.15.146:0/3558375214 >> v2:172.21.15.154:3300/0 conn(0x7f759c0077e0 0x7f759c0de990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a9859700 1 --2- 172.21.15.146:0/3558375214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f759c0d57e0 0x7f759c047720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201276505 0 0) 0x7f759c047290 con 0x7f759c04d2a0 2024-09-13T18:30:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75900035d0 con 0x7f759c04d2a0 2024-09-13T18:30:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1663973684 0 0) 0x7f75900035d0 con 0x7f759c04d2a0 2024-09-13T18:30:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f759c0d57e0 msgr2=0x7f759c047720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a8857700 1 --2- 172.21.15.146:0/3558375214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f759c0d57e0 0x7f759c047720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 >> v2:172.21.15.154:3300/0 conn(0x7f759c0077e0 msgr2=0x7f759c0de990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a8857700 1 --2- 172.21.15.146:0/3558375214 >> v2:172.21.15.154:3300/0 conn(0x7f759c0077e0 0x7f759c0de990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c04c060 con 0x7f759c04d2a0 2024-09-13T18:30:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f7594001670 con 0x7f759c04d2a0 2024-09-13T18:30:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 >> v1:172.21.15.146:6789/0 conn(0x7f759c04d2a0 legacy=0x7f759c0deed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a8857700 1 --2- 172.21.15.146:0/3558375214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7590003a20 0x7f7590003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.506+0000 7f75a8857700 1 --2- 172.21.15.146:0/3558375214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f759c0d57e0 0x7f7590004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a8857700 1 --2- 172.21.15.146:0/3558375214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7590004db0 0x7f7590005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c047c60 con 0x7f7590003a20 2024-09-13T18:30:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759000dcd0 con 0x7f7590004db0 2024-09-13T18:30:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759000de40 con 0x7f759c0d57e0 2024-09-13T18:30:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a9058700 1 --2- 172.21.15.146:0/3558375214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f759c0d57e0 0x7f7590004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75aa05a700 1 --2- 172.21.15.146:0/3558375214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7590004db0 0x7f7590005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a9859700 1 --2- 172.21.15.146:0/3558375214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7590003a20 0x7f7590003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a9859700 1 -- 172.21.15.146:0/3558375214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f759c0d57e0 msgr2=0x7f7590004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:52.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a9859700 1 --2- 172.21.15.146:0/3558375214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f759c0d57e0 0x7f7590004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:52.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a9859700 1 -- 172.21.15.146:0/3558375214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7590004db0 msgr2=0x7f7590005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:52.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a9859700 1 --2- 172.21.15.146:0/3558375214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7590004db0 0x7f7590005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:52.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a9859700 1 -- 172.21.15.146:0/3558375214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75900035d0 con 0x7f7590003a20 2024-09-13T18:30:52.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a9859700 1 --2- 172.21.15.146:0/3558375214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7590003a20 0x7f7590003e40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f75a404f480 tx=0x7f75900058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad57efd51b281191 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:52.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4090070 con 0x7f7590003a20 2024-09-13T18:30:52.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a406f230 con 0x7f7590003a20 2024-09-13T18:30:52.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75a8857700 1 -- 172.21.15.146:0/3558375214 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4073900 con 0x7f7590003a20 2024-09-13T18:30:52.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75aa85b700 1 -- 172.21.15.146:0/3558375214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7590003a20 msgr2=0x7f7590003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:52.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75aa85b700 1 --2- 172.21.15.146:0/3558375214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7590003a20 0x7f7590003e40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f75a404f480 tx=0x7f75900058a0 comp rx=0 tx=0).stop 2024-09-13T18:30:52.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75aa05a700 1 -- 172.21.15.146:0/3558375214 reap_dead start 2024-09-13T18:30:52.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75aa85b700 1 -- 172.21.15.146:0/3558375214 shutdown_connections 2024-09-13T18:30:52.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75aa85b700 1 -- 172.21.15.146:0/3558375214 >> 172.21.15.146:0/3558375214 conn(0x7f759c04d8e0 msgr2=0x7f759c04dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:52.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75aa85b700 1 -- 172.21.15.146:0/3558375214 shutdown_connections 2024-09-13T18:30:52.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75aa85b700 1 -- 172.21.15.146:0/3558375214 wait complete. 2024-09-13T18:30:52.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75aa85b700 1 Processor -- start 2024-09-13T18:30:52.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.510+0000 7f75aa85b700 1 -- start start 2024-09-13T18:30:52.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75aa85b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7590003a20 0x7f759c177da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:52.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75aa85b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c0caf20 0x7f759c1782e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:52.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75aa85b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f759c0db650 0x7f759c0dbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:52.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75aa85b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0db230 con 0x7f759c0caf20 2024-09-13T18:30:52.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75aa85b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0da2f0 con 0x7f759c0db650 2024-09-13T18:30:52.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75aa85b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0dc010 con 0x7f7590003a20 2024-09-13T18:30:52.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75a9058700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c0caf20 0x7f759c1782e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:52.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75a9058700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c0caf20 0x7f759c1782e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:42636/0 (socket says 172.21.15.146:42636) 2024-09-13T18:30:52.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75a9058700 1 -- 172.21.15.146:0/3814044893 learned_addr learned my addr 172.21.15.146:0/3814044893 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:30:52.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75aa05a700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f759c0db650 0x7f759c0dbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:52.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75a9859700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7590003a20 0x7f759c177da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:52.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75a9058700 1 -- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7590003a20 msgr2=0x7f759c177da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:52.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75a9058700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7590003a20 0x7f759c177da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:52.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75a9058700 1 -- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f759c0db650 msgr2=0x7f759c0dbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:52.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75a9058700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f759c0db650 0x7f759c0dbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:52.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75a9058700 1 -- 172.21.15.146:0/3814044893 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c178920 con 0x7f759c0caf20 2024-09-13T18:30:52.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75a9859700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7590003a20 0x7f759c177da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:30:52.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75a9058700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c0caf20 0x7f759c1782e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f75a0011a70 tx=0x7f75a0011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:52.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f759a7fc700 1 -- 172.21.15.146:0/3814044893 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a000fa00 con 0x7f759c0caf20 2024-09-13T18:30:52.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f759a7fc700 1 -- 172.21.15.146:0/3814044893 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a000fb60 con 0x7f759c0caf20 2024-09-13T18:30:52.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75aa85b700 1 -- 172.21.15.146:0/3814044893 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759c0da5b0 con 0x7f759c0caf20 2024-09-13T18:30:52.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f759a7fc700 1 -- 172.21.15.146:0/3814044893 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0014440 con 0x7f759c0caf20 2024-09-13T18:30:52.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.514+0000 7f75aa85b700 1 -- 172.21.15.146:0/3814044893 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c0dab70 con 0x7f759c0caf20 2024-09-13T18:30:52.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.518+0000 7f75aa85b700 1 -- 172.21.15.146:0/3814044893 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759c034c60 con 0x7f759c0caf20 2024-09-13T18:30:52.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.518+0000 7f759a7fc700 1 -- 172.21.15.146:0/3814044893 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f75a001d400 con 0x7f759c0caf20 2024-09-13T18:30:52.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.518+0000 7f759a7fc700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7588066730 0x7f7588068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:52.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.518+0000 7f759a7fc700 1 -- 172.21.15.146:0/3814044893 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f75a0084cf0 con 0x7f759c0caf20 2024-09-13T18:30:52.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.518+0000 7f75a9859700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7588066730 0x7f7588068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:52.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.518+0000 7f75a9859700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7588066730 0x7f7588068bf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f75a4071000 tx=0x7f75a406b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:52.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.522+0000 7f759a7fc700 1 -- 172.21.15.146:0/3814044893 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f75a0089070 con 0x7f759c0caf20 2024-09-13T18:30:52.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.770+0000 7f75aa85b700 1 -- 172.21.15.146:0/3814044893 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f759c046bc0 con 0x7f759c0caf20 2024-09-13T18:30:52.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.770+0000 7f759a7fc700 1 -- 172.21.15.146:0/3814044893 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5528 (secure 0 0 0) 0x7f75a0051c30 con 0x7f759c0caf20 2024-09-13T18:30:52.780 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:30:52.781 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:51.700582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:52.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.774+0000 7f75aa85b700 1 -- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7588066730 msgr2=0x7f7588068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:52.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.774+0000 7f75aa85b700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7588066730 0x7f7588068bf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f75a4071000 tx=0x7f75a406b000 comp rx=0 tx=0).stop 2024-09-13T18:30:52.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.774+0000 7f75aa85b700 1 -- 172.21.15.146:0/3814044893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c0caf20 msgr2=0x7f759c1782e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:52.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.774+0000 7f75aa85b700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c0caf20 0x7f759c1782e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f75a0011a70 tx=0x7f75a0011e30 comp rx=0 tx=0).stop 2024-09-13T18:30:52.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.774+0000 7f75aa85b700 1 -- 172.21.15.146:0/3814044893 shutdown_connections 2024-09-13T18:30:52.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.774+0000 7f75aa85b700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7588066730 0x7f7588068bf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:52.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.774+0000 7f75aa85b700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f759c0db650 0x7f759c0dbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:52.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.774+0000 7f75aa85b700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c0caf20 0x7f759c1782e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:52.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.774+0000 7f75aa85b700 1 --2- 172.21.15.146:0/3814044893 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7590003a20 0x7f759c177da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:52.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.774+0000 7f75aa85b700 1 -- 172.21.15.146:0/3814044893 >> 172.21.15.146:0/3814044893 conn(0x7f759c04d8e0 msgr2=0x7f759c0d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:52.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.774+0000 7f75aa85b700 1 -- 172.21.15.146:0/3814044893 shutdown_connections 2024-09-13T18:30:52.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:30:52.774+0000 7f75aa85b700 1 -- 172.21.15.146:0/3814044893 wait complete. 2024-09-13T18:30:52.784 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 32 2024-09-13T18:30:52.796 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=1, replacing gid=4281 2024-09-13T18:30:52.797 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 8 secs before reviving mds.a 2024-09-13T18:31:00.798 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-09-13T18:31:00.798 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T18:31:00.798 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T18:31:00.801 INFO:tasks.ceph.mds.a:Started 2024-09-13T18:31:00.801 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:31:00.895 INFO:tasks.ceph.mds.a.smithi146.stdout:starting mds.a at 2024-09-13T18:31:00.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.922+0000 7f5c77fff700 1 Processor -- start 2024-09-13T18:31:00.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.922+0000 7f5c77fff700 1 -- start start 2024-09-13T18:31:00.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.922+0000 7f5c77fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c7806e7c0 0x7f5c780b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:00.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.922+0000 7f5c77fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7806eba0 0x7f5c780c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:00.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.922+0000 7f5c77fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c781598f0 0x7f5c78159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:00.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.922+0000 7f5c77fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7815a2e0 con 0x7f5c781598f0 2024-09-13T18:31:00.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.922+0000 7f5c77fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780c1020 con 0x7f5c7806e7c0 2024-09-13T18:31:00.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.922+0000 7f5c77fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c781593f0 con 0x7f5c7806eba0 2024-09-13T18:31:00.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.922+0000 7f5c777fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c781598f0 0x7f5c78159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:00.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.922+0000 7f5c777fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c781598f0 0x7f5c78159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:42668/0 (socket says 172.21.15.146:42668) 2024-09-13T18:31:00.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.922+0000 7f5c777fe700 1 -- 172.21.15.146:0/3519747416 learned_addr learned my addr 172.21.15.146:0/3519747416 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:31:00.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.922+0000 7f5c76ffd700 1 --2- 172.21.15.146:0/3519747416 >> v2:172.21.15.154:3300/0 conn(0x7f5c7806e7c0 0x7f5c780b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:00.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c767fc700 1 --2- 172.21.15.146:0/3519747416 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7806eba0 0x7f5c780c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:00.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c777fe700 1 -- 172.21.15.146:0/3519747416 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7806eba0 msgr2=0x7f5c780c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:00.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c777fe700 1 --2- 172.21.15.146:0/3519747416 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7806eba0 0x7f5c780c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:00.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c777fe700 1 -- 172.21.15.146:0/3519747416 >> v2:172.21.15.154:3300/0 conn(0x7f5c7806e7c0 msgr2=0x7f5c780b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:00.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c777fe700 1 --2- 172.21.15.146:0/3519747416 >> v2:172.21.15.154:3300/0 conn(0x7f5c7806e7c0 0x7f5c780b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:00.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c777fe700 1 -- 172.21.15.146:0/3519747416 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780c6a70 con 0x7f5c781598f0 2024-09-13T18:31:00.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c777fe700 1 --2- 172.21.15.146:0/3519747416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c781598f0 0x7f5c78159da0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5c780c9ad0 tx=0x7f5c7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3f5657c97e5b0a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:00.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c75ffb700 1 -- 172.21.15.146:0/3519747416 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000e070 con 0x7f5c781598f0 2024-09-13T18:31:00.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c75ffb700 1 -- 172.21.15.146:0/3519747416 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7000b910 con 0x7f5c781598f0 2024-09-13T18:31:00.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c75ffb700 1 -- 172.21.15.146:0/3519747416 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000bcc0 con 0x7f5c781598f0 2024-09-13T18:31:00.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c77fff700 1 -- 172.21.15.146:0/3519747416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c781598f0 msgr2=0x7f5c78159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:00.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c77fff700 1 --2- 172.21.15.146:0/3519747416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c781598f0 0x7f5c78159da0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5c780c9ad0 tx=0x7f5c7000b330 comp rx=0 tx=0).stop 2024-09-13T18:31:00.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c77fff700 1 -- 172.21.15.146:0/3519747416 shutdown_connections 2024-09-13T18:31:00.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c77fff700 1 --2- 172.21.15.146:0/3519747416 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7806eba0 0x7f5c780c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:00.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c77fff700 1 --2- 172.21.15.146:0/3519747416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c781598f0 0x7f5c78159da0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:00.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c77fff700 1 --2- 172.21.15.146:0/3519747416 >> v2:172.21.15.154:3300/0 conn(0x7f5c7806e7c0 0x7f5c780b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:00.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c77fff700 1 -- 172.21.15.146:0/3519747416 >> 172.21.15.146:0/3519747416 conn(0x7f5c78071ed0 msgr2=0x7f5c780722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:00.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c77fff700 1 -- 172.21.15.146:0/3519747416 shutdown_connections 2024-09-13T18:31:00.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.926+0000 7f5c77fff700 1 -- 172.21.15.146:0/3519747416 wait complete. 2024-09-13T18:31:00.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c77fff700 1 Processor -- start 2024-09-13T18:31:00.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c77fff700 1 -- start start 2024-09-13T18:31:00.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c77fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7806e7c0 0x7f5c7814e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:00.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c77fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7806eba0 0x7f5c780700a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:00.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c77fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c781598f0 0x7f5c780705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:00.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c77fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c781ef060 con 0x7f5c781598f0 2024-09-13T18:31:00.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c77fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c781efbc0 con 0x7f5c7806e7c0 2024-09-13T18:31:00.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c77fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c781ef320 con 0x7f5c7806eba0 2024-09-13T18:31:00.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c777fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c781598f0 0x7f5c780705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:00.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c777fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c781598f0 0x7f5c780705e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:42676/0 (socket says 172.21.15.146:42676) 2024-09-13T18:31:00.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c777fe700 1 -- 172.21.15.146:0/48486741 learned_addr learned my addr 172.21.15.146:0/48486741 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:31:00.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c76ffd700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7806e7c0 0x7f5c7814e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:00.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c767fc700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7806eba0 0x7f5c780700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:00.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c777fe700 1 -- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7806eba0 msgr2=0x7f5c780700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:00.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c777fe700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7806eba0 0x7f5c780700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:00.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c777fe700 1 -- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7806e7c0 msgr2=0x7f5c7814e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:00.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c777fe700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7806e7c0 0x7f5c7814e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:00.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c777fe700 1 -- 172.21.15.146:0/48486741 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c70009b30 con 0x7f5c781598f0 2024-09-13T18:31:00.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c777fe700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c781598f0 0x7f5c780705e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5c7806da00 tx=0x7f5c70004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:00.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c5ffff700 1 -- 172.21.15.146:0/48486741 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c700099b0 con 0x7f5c781598f0 2024-09-13T18:31:00.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c5ffff700 1 -- 172.21.15.146:0/48486741 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c700092e0 con 0x7f5c781598f0 2024-09-13T18:31:00.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c5ffff700 1 -- 172.21.15.146:0/48486741 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70004470 con 0x7f5c781598f0 2024-09-13T18:31:00.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c77fff700 1 -- 172.21.15.146:0/48486741 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c781ef600 con 0x7f5c781598f0 2024-09-13T18:31:00.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.930+0000 7f5c77fff700 1 -- 172.21.15.146:0/48486741 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7806fb80 con 0x7f5c781598f0 2024-09-13T18:31:00.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.934+0000 7f5c77fff700 1 -- 172.21.15.146:0/48486741 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c78151850 con 0x7f5c781598f0 2024-09-13T18:31:00.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.934+0000 7f5c5ffff700 1 -- 172.21.15.146:0/48486741 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5c700045d0 con 0x7f5c781598f0 2024-09-13T18:31:00.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.934+0000 7f5c5ffff700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c64066730 0x7f5c64068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:00.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.934+0000 7f5c5ffff700 1 -- 172.21.15.146:0/48486741 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f5c70085960 con 0x7f5c781598f0 2024-09-13T18:31:00.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.934+0000 7f5c76ffd700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c64066730 0x7f5c64068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:00.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.938+0000 7f5c76ffd700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c64066730 0x7f5c64068bf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009b50 tx=0x7f5c68009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:00.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:00.942+0000 7f5c5ffff700 1 -- 172.21.15.146:0/48486741 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5c700528d0 con 0x7f5c781598f0 2024-09-13T18:31:01.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.166+0000 7f5c77fff700 1 -- 172.21.15.146:0/48486741 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c780b11a0 con 0x7f5c781598f0 2024-09-13T18:31:01.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.166+0000 7f5c5ffff700 1 -- 172.21.15.146:0/48486741 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f5c70055ef0 con 0x7f5c781598f0 2024-09-13T18:31:01.174 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:31:01.174 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:51.700582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:31:01.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.170+0000 7f5c77fff700 1 -- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c64066730 msgr2=0x7f5c64068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:01.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.170+0000 7f5c77fff700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c64066730 0x7f5c64068bf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009b50 tx=0x7f5c68009240 comp rx=0 tx=0).stop 2024-09-13T18:31:01.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.170+0000 7f5c77fff700 1 -- 172.21.15.146:0/48486741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c781598f0 msgr2=0x7f5c780705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:01.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.170+0000 7f5c77fff700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c781598f0 0x7f5c780705e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5c7806da00 tx=0x7f5c70004af0 comp rx=0 tx=0).stop 2024-09-13T18:31:01.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.170+0000 7f5c77fff700 1 -- 172.21.15.146:0/48486741 shutdown_connections 2024-09-13T18:31:01.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.170+0000 7f5c77fff700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c64066730 0x7f5c64068bf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:01.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.170+0000 7f5c77fff700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c781598f0 0x7f5c780705e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:01.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.170+0000 7f5c77fff700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c7806eba0 0x7f5c780700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:01.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.170+0000 7f5c77fff700 1 --2- 172.21.15.146:0/48486741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7806e7c0 0x7f5c7814e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:01.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.170+0000 7f5c77fff700 1 -- 172.21.15.146:0/48486741 >> 172.21.15.146:0/48486741 conn(0x7f5c78071ed0 msgr2=0x7f5c78157050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:01.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.170+0000 7f5c77fff700 1 -- 172.21.15.146:0/48486741 shutdown_connections 2024-09-13T18:31:01.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:31:01.170+0000 7f5c77fff700 1 -- 172.21.15.146:0/48486741 wait complete. 2024-09-13T18:31:01.180 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 33 2024-09-13T18:31:01.190 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-09-13T18:31:01.190 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 97 secs before thrashing 2024-09-13T18:31:07.613 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:31:07.616 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:31:37.684 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:31:37.688 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:32:08.084 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:32:08.088 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:32:38.145 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:32:38.148 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:32:38.192 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:32:38.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe3fff700 1 Processor -- start 2024-09-13T18:32:38.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe3fff700 1 -- start start 2024-09-13T18:32:38.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe3fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcbdc0d4c10 0x7fcbdc04b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe3fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcbdc0d4ff0 0x7fcbdc0db500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe3fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbdc04b290 con 0x7fcbdc006ef0 2024-09-13T18:32:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe3fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbdc033fe0 con 0x7fcbdc0d4c10 2024-09-13T18:32:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe3fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbdc045990 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe37fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fcbdc006ef0 0x7fcbdc045280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:40172/0 (socket says 172.21.15.146:40172) 2024-09-13T18:32:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe2ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcbdc0d4c10 0x7fcbdc04b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe37fe700 1 -- 172.21.15.146:0/3375506945 learned_addr learned my addr 172.21.15.146:0/3375506945 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:32:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe27fc700 1 --2- 172.21.15.146:0/3375506945 >> v2:172.21.15.154:3300/0 conn(0x7fcbdc0d4ff0 0x7fcbdc0db500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe2ffd700 1 -- 172.21.15.146:0/3375506945 >> v2:172.21.15.154:3300/0 conn(0x7fcbdc0d4ff0 msgr2=0x7fcbdc0db500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe2ffd700 1 --2- 172.21.15.146:0/3375506945 >> v2:172.21.15.154:3300/0 conn(0x7fcbdc0d4ff0 0x7fcbdc0db500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe1ffb700 1 -- 172.21.15.146:0/3375506945 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471988674 0 0) 0x7fcbdc04b290 con 0x7fcbdc006ef0 2024-09-13T18:32:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe2ffd700 1 -- 172.21.15.146:0/3375506945 >> v1:172.21.15.146:6789/0 conn(0x7fcbdc006ef0 legacy=0x7fcbdc045280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe2ffd700 1 -- 172.21.15.146:0/3375506945 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc0dba40 con 0x7fcbdc0d4c10 2024-09-13T18:32:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe2ffd700 1 --2- 172.21.15.146:0/3375506945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbdc0d4c10 0x7fcbdc04b720 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc033760 tx=0x7fcbd4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc7d4ced3ad4643 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe1ffb700 1 -- 172.21.15.146:0/3375506945 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4011830 con 0x7fcbdc0d4c10 2024-09-13T18:32:38.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe1ffb700 1 -- 172.21.15.146:0/3375506945 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd4011990 con 0x7fcbdc0d4c10 2024-09-13T18:32:38.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.300+0000 7fcbe1ffb700 1 -- 172.21.15.146:0/3375506945 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4011d40 con 0x7fcbdc0d4c10 2024-09-13T18:32:38.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.304+0000 7fcbe3fff700 1 -- 172.21.15.146:0/3375506945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbdc0d4c10 msgr2=0x7fcbdc04b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.304+0000 7fcbe3fff700 1 --2- 172.21.15.146:0/3375506945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbdc0d4c10 0x7fcbdc04b720 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc033760 tx=0x7fcbd4009b30 comp rx=0 tx=0).stop 2024-09-13T18:32:38.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.304+0000 7fcbe3fff700 1 -- 172.21.15.146:0/3375506945 shutdown_connections 2024-09-13T18:32:38.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.304+0000 7fcbe3fff700 1 --2- 172.21.15.146:0/3375506945 >> v2:172.21.15.154:3300/0 conn(0x7fcbdc0d4ff0 0x7fcbdc0db500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.304+0000 7fcbe3fff700 1 --2- 172.21.15.146:0/3375506945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbdc0d4c10 0x7fcbdc04b720 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.304+0000 7fcbe3fff700 1 -- 172.21.15.146:0/3375506945 >> 172.21.15.146:0/3375506945 conn(0x7fcbdc04fd00 msgr2=0x7fcbdc050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:38.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.304+0000 7fcbe3fff700 1 -- 172.21.15.146:0/3375506945 shutdown_connections 2024-09-13T18:32:38.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.304+0000 7fcbe3fff700 1 -- 172.21.15.146:0/3375506945 wait complete. 2024-09-13T18:32:38.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.304+0000 7fcbe3fff700 1 Processor -- start 2024-09-13T18:32:38.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.304+0000 7fcbe3fff700 1 -- start start 2024-09-13T18:32:38.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbdc006ef0 0x7fcbdc0dc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbdc0d4c10 0x7fcbdc04f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbdc0d4ff0 0x7fcbdc0d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe3fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc04f060 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe3fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc032d90 con 0x7fcbdc0d4c10 2024-09-13T18:32:38.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc04fa30 con 0x7fcbdc006ef0 2024-09-13T18:32:38.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbdc0d4ff0 0x7fcbdc0d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbdc0d4ff0 0x7fcbdc0d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52326/0 (socket says 172.21.15.146:52326) 2024-09-13T18:32:38.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe37fe700 1 -- 172.21.15.146:0/1414358832 learned_addr learned my addr 172.21.15.146:0/1414358832 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:38.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe27fc700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbdc0d4c10 0x7fcbdc04f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe2ffd700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbdc006ef0 0x7fcbdc0dc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe37fe700 1 -- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbdc006ef0 msgr2=0x7fcbdc0dc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe37fe700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbdc006ef0 0x7fcbdc0dc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe37fe700 1 -- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbdc0d4c10 msgr2=0x7fcbdc04f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe37fe700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbdc0d4c10 0x7fcbdc04f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe37fe700 1 -- 172.21.15.146:0/1414358832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd40094b0 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe2ffd700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbdc006ef0 0x7fcbdc0dc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:32:38.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe37fe700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbdc0d4ff0 0x7fcbdc0d26a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4072e30 tx=0x7fcbe4075d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:38.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbcb7fe700 1 -- 172.21.15.146:0/1414358832 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4090070 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbcb7fe700 1 -- 172.21.15.146:0/1414358832 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe4075e30 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbcb7fe700 1 -- 172.21.15.146:0/1414358832 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4067e60 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe3fff700 1 -- 172.21.15.146:0/1414358832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbdc0c4c40 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe3fff700 1 -- 172.21.15.146:0/1414358832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbdc04ecf0 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.308+0000 7fcbe3fff700 1 -- 172.21.15.146:0/1414358832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbdc04f210 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.312+0000 7fcbcb7fe700 1 -- 172.21.15.146:0/1414358832 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcbe4067fc0 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.312+0000 7fcbcb7fe700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcbd0066730 0x7fcbd0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.312+0000 7fcbcb7fe700 1 -- 172.21.15.146:0/1414358832 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fcbe41008d0 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.312+0000 7fcbe2ffd700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcbd0066730 0x7fcbd0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.312+0000 7fcbe2ffd700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcbd0066730 0x7fcbd0068bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40090b0 tx=0x7fcbd4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:38.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.316+0000 7fcbcb7fe700 1 -- 172.21.15.146:0/1414358832 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcbdc04f210 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.540+0000 7fcbe3fff700 1 -- 172.21.15.146:0/1414358832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbdc04f210 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.544+0000 7fcbcb7fe700 1 -- 172.21.15.146:0/1414358832 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6288 (secure 0 0 0) 0x7fcbdc04f210 con 0x7fcbdc0d4ff0 2024-09-13T18:32:38.552 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:32:38.552 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:51.700582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:32:38.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.544+0000 7fcbe3fff700 1 -- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcbd0066730 msgr2=0x7fcbd0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.544+0000 7fcbe3fff700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcbd0066730 0x7fcbd0068bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40090b0 tx=0x7fcbd4008030 comp rx=0 tx=0).stop 2024-09-13T18:32:38.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.544+0000 7fcbe3fff700 1 -- 172.21.15.146:0/1414358832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbdc0d4ff0 msgr2=0x7fcbdc0d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.544+0000 7fcbe3fff700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbdc0d4ff0 0x7fcbdc0d26a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4072e30 tx=0x7fcbe4075d10 comp rx=0 tx=0).stop 2024-09-13T18:32:38.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.544+0000 7fcbe3fff700 1 -- 172.21.15.146:0/1414358832 shutdown_connections 2024-09-13T18:32:38.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.544+0000 7fcbe3fff700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcbd0066730 0x7fcbd0068bf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.544+0000 7fcbe3fff700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbdc0d4ff0 0x7fcbdc0d26a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.544+0000 7fcbe3fff700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbdc0d4c10 0x7fcbdc04f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.544+0000 7fcbe3fff700 1 --2- 172.21.15.146:0/1414358832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbdc006ef0 0x7fcbdc0dc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.544+0000 7fcbe3fff700 1 -- 172.21.15.146:0/1414358832 >> 172.21.15.146:0/1414358832 conn(0x7fcbdc04fd00 msgr2=0x7fcbdc0cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:38.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.544+0000 7fcbe3fff700 1 -- 172.21.15.146:0/1414358832 shutdown_connections 2024-09-13T18:32:38.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.544+0000 7fcbe3fff700 1 -- 172.21.15.146:0/1414358832 wait complete. 2024-09-13T18:32:38.557 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 33 2024-09-13T18:32:38.567 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2024-09-13T18:32:38.568 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T18:32:38.568 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:32:38.582 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T18:32:38.583 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:32:38.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.688+0000 7f8db109f700 1 Processor -- start 2024-09-13T18:32:38.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.688+0000 7f8db109f700 1 -- start start 2024-09-13T18:32:38.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.688+0000 7f8db109f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8dac071610 0x7f8dac159b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.688+0000 7f8db109f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dac071410 con 0x7f8dac150230 2024-09-13T18:32:38.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.688+0000 7f8db109f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dac156090 con 0x7f8dac150610 2024-09-13T18:32:38.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.688+0000 7f8db109f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac1596b0 con 0x7f8dac071610 2024-09-13T18:32:38.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8dabfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f8dac150230 0x7f8dac06ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:40180/0 (socket says 172.21.15.146:40180) 2024-09-13T18:32:38.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8dabfff700 1 -- 172.21.15.146:0/3371252489 learned_addr learned my addr 172.21.15.146:0/3371252489 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:32:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8db089e700 1 --2- 172.21.15.146:0/3371252489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8dac071610 0x7f8dac159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8daaffd700 1 -- 172.21.15.146:0/3371252489 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403819623 0 0) 0x7f8dac071410 con 0x7f8dac150230 2024-09-13T18:32:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8daaffd700 1 -- 172.21.15.146:0/3371252489 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d940035d0 con 0x7f8dac150230 2024-09-13T18:32:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8daaffd700 1 -- 172.21.15.146:0/3371252489 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244668565 0 0) 0x7f8dac156090 con 0x7f8dac150610 2024-09-13T18:32:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8daaffd700 1 -- 172.21.15.146:0/3371252489 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dac071410 con 0x7f8dac150610 2024-09-13T18:32:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8db089e700 1 -- 172.21.15.146:0/3371252489 >> v1:172.21.15.154:6789/0 conn(0x7f8dac150610 legacy=0x7f8dac06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8db089e700 1 -- 172.21.15.146:0/3371252489 >> v1:172.21.15.146:6789/0 conn(0x7f8dac150230 legacy=0x7f8dac06ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8db089e700 1 -- 172.21.15.146:0/3371252489 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac156350 con 0x7f8dac071610 2024-09-13T18:32:38.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8db089e700 1 --2- 172.21.15.146:0/3371252489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8dac071610 0x7f8dac159b00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8dac1597f0 tx=0x7f8da400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eed25548aa80d6b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:38.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8daaffd700 1 -- 172.21.15.146:0/3371252489 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4016070 con 0x7f8dac071610 2024-09-13T18:32:38.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8daaffd700 1 -- 172.21.15.146:0/3371252489 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da400bb20 con 0x7f8dac071610 2024-09-13T18:32:38.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8daaffd700 1 -- 172.21.15.146:0/3371252489 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400e070 con 0x7f8dac071610 2024-09-13T18:32:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8db109f700 1 -- 172.21.15.146:0/3371252489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8dac071610 msgr2=0x7f8dac159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8db109f700 1 --2- 172.21.15.146:0/3371252489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8dac071610 0x7f8dac159b00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8dac1597f0 tx=0x7f8da400b330 comp rx=0 tx=0).stop 2024-09-13T18:32:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8db109f700 1 -- 172.21.15.146:0/3371252489 shutdown_connections 2024-09-13T18:32:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8db109f700 1 --2- 172.21.15.146:0/3371252489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8dac071610 0x7f8dac159b00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8db109f700 1 -- 172.21.15.146:0/3371252489 >> 172.21.15.146:0/3371252489 conn(0x7f8dac071ed0 msgr2=0x7f8dac0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8db109f700 1 -- 172.21.15.146:0/3371252489 shutdown_connections 2024-09-13T18:32:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.692+0000 7f8db109f700 1 -- 172.21.15.146:0/3371252489 wait complete. 2024-09-13T18:32:38.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db109f700 1 Processor -- start 2024-09-13T18:32:38.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db109f700 1 -- start start 2024-09-13T18:32:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db109f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8dac071610 0x7f8dac1ebed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db109f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8dac150230 0x7f8dac0caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db109f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dac0c2330 0x7f8dac0cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db109f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac1eba40 con 0x7f8dac0c2330 2024-09-13T18:32:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db109f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac1571f0 con 0x7f8dac150230 2024-09-13T18:32:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db109f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac1547a0 con 0x7f8dac071610 2024-09-13T18:32:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db089e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dac0c2330 0x7f8dac0cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db089e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dac0c2330 0x7f8dac0cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52332/0 (socket says 172.21.15.146:52332) 2024-09-13T18:32:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8dabfff700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8dac071610 0x7f8dac1ebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db089e700 1 -- 172.21.15.146:0/229373027 learned_addr learned my addr 172.21.15.146:0/229373027 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8dab7fe700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8dac150230 0x7f8dac0caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db089e700 1 -- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8dac071610 msgr2=0x7f8dac1ebed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db089e700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8dac071610 0x7f8dac1ebed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db089e700 1 -- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8dac150230 msgr2=0x7f8dac0caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db089e700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8dac150230 0x7f8dac0caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db089e700 1 -- 172.21.15.146:0/229373027 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da4009a70 con 0x7f8dac0c2330 2024-09-13T18:32:38.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db089e700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dac0c2330 0x7f8dac0cb420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8da400b370 tx=0x7f8da4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:38.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8da8ff9700 1 -- 172.21.15.146:0/229373027 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4016040 con 0x7f8dac0c2330 2024-09-13T18:32:38.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8da8ff9700 1 -- 172.21.15.146:0/229373027 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da40048a0 con 0x7f8dac0c2330 2024-09-13T18:32:38.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8da8ff9700 1 -- 172.21.15.146:0/229373027 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da401b680 con 0x7f8dac0c2330 2024-09-13T18:32:38.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db109f700 1 -- 172.21.15.146:0/229373027 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dac154a60 con 0x7f8dac0c2330 2024-09-13T18:32:38.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.696+0000 7f8db109f700 1 -- 172.21.15.146:0/229373027 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dac157770 con 0x7f8dac0c2330 2024-09-13T18:32:38.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.700+0000 7f8db109f700 1 -- 172.21.15.146:0/229373027 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac06ebd0 con 0x7f8dac0c2330 2024-09-13T18:32:38.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.700+0000 7f8da8ff9700 1 -- 172.21.15.146:0/229373027 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8da401b7e0 con 0x7f8dac0c2330 2024-09-13T18:32:38.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.700+0000 7f8da8ff9700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8d98066730 0x7f8d98068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.700+0000 7f8da8ff9700 1 -- 172.21.15.146:0/229373027 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f8da40858b0 con 0x7f8dac0c2330 2024-09-13T18:32:38.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.700+0000 7f8dabfff700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8d98066730 0x7f8d98068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.700+0000 7f8dabfff700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8d98066730 0x7f8d98068bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8da0000f60 tx=0x7f8da0007e20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:38.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.708+0000 7f8da8ff9700 1 -- 172.21.15.146:0/229373027 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8da40526d0 con 0x7f8dac0c2330 2024-09-13T18:32:38.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.940+0000 7f8db109f700 1 -- 172.21.15.146:0/229373027 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8dac1ebb80 con 0x7f8dac0c2330 2024-09-13T18:32:38.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.944+0000 7f8da8ff9700 1 -- 172.21.15.146:0/229373027 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f8da4055cf0 con 0x7f8dac0c2330 2024-09-13T18:32:38.952 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:32:38.952 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:51.700582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:32:38.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.944+0000 7f8db109f700 1 -- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8d98066730 msgr2=0x7f8d98068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.944+0000 7f8db109f700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8d98066730 0x7f8d98068bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8da0000f60 tx=0x7f8da0007e20 comp rx=0 tx=0).stop 2024-09-13T18:32:38.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.944+0000 7f8db109f700 1 -- 172.21.15.146:0/229373027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dac0c2330 msgr2=0x7f8dac0cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.944+0000 7f8db109f700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dac0c2330 0x7f8dac0cb420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8da400b370 tx=0x7f8da4004af0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.944+0000 7f8db109f700 1 -- 172.21.15.146:0/229373027 shutdown_connections 2024-09-13T18:32:38.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.944+0000 7f8db109f700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8d98066730 0x7f8d98068bf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.944+0000 7f8db109f700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dac0c2330 0x7f8dac0cb420 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.944+0000 7f8db109f700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8dac150230 0x7f8dac0caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.944+0000 7f8db109f700 1 --2- 172.21.15.146:0/229373027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8dac071610 0x7f8dac1ebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.944+0000 7f8db109f700 1 -- 172.21.15.146:0/229373027 >> 172.21.15.146:0/229373027 conn(0x7f8dac071ed0 msgr2=0x7f8dac14be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:38.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.944+0000 7f8db109f700 1 -- 172.21.15.146:0/229373027 shutdown_connections 2024-09-13T18:32:38.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:38.944+0000 7f8db109f700 1 -- 172.21.15.146:0/229373027 wait complete. 2024-09-13T18:32:38.956 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 33 2024-09-13T18:32:38.972 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:32:40.973 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:32:41.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114ffff700 1 Processor -- start 2024-09-13T18:32:41.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114ffff700 1 -- start start 2024-09-13T18:32:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f115014fba0 0x7f11501562a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f11500c91d0 0x7f11500c9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114ffff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f115014ff80 con 0x7f11500ca730 2024-09-13T18:32:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114ffff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11500ca3a0 con 0x7f115014fba0 2024-09-13T18:32:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114ffff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11500c0360 con 0x7f11500c91d0 2024-09-13T18:32:41.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f115014fba0 0x7f11501562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:41.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f115014fba0 0x7f11501562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46274/0 (socket says 172.21.15.146:46274) 2024-09-13T18:32:41.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114effd700 1 -- 172.21.15.146:0/2010969150 learned_addr learned my addr 172.21.15.146:0/2010969150 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:41.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114f7fe700 1 --2- 172.21.15.146:0/2010969150 >> v2:172.21.15.154:3300/0 conn(0x7f11500c91d0 0x7f11500c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:41.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114effd700 1 -- 172.21.15.146:0/2010969150 >> v2:172.21.15.154:3300/0 conn(0x7f11500c91d0 msgr2=0x7f11500c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:41.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114effd700 1 --2- 172.21.15.146:0/2010969150 >> v2:172.21.15.154:3300/0 conn(0x7f11500c91d0 0x7f11500c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:41.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114effd700 1 -- 172.21.15.146:0/2010969150 >> v1:172.21.15.154:6789/0 conn(0x7f11500ca730 legacy=0x7f11500a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:41.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.088+0000 7f114effd700 1 -- 172.21.15.146:0/2010969150 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11500c8a60 con 0x7f115014fba0 2024-09-13T18:32:41.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.092+0000 7f114effd700 1 --2- 172.21.15.146:0/2010969150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f115014fba0 0x7f11501562a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f11500729a0 tx=0x7f1140009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca7315156925bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:41.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.092+0000 7f114e7fc700 1 -- 172.21.15.146:0/2010969150 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140011830 con 0x7f115014fba0 2024-09-13T18:32:41.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.092+0000 7f114e7fc700 1 -- 172.21.15.146:0/2010969150 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1140011990 con 0x7f115014fba0 2024-09-13T18:32:41.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.092+0000 7f114e7fc700 1 -- 172.21.15.146:0/2010969150 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140011d20 con 0x7f115014fba0 2024-09-13T18:32:41.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.092+0000 7f114ffff700 1 -- 172.21.15.146:0/2010969150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f115014fba0 msgr2=0x7f11501562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:41.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.092+0000 7f114ffff700 1 --2- 172.21.15.146:0/2010969150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f115014fba0 0x7f11501562a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f11500729a0 tx=0x7f1140009bf0 comp rx=0 tx=0).stop 2024-09-13T18:32:41.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.092+0000 7f114ffff700 1 -- 172.21.15.146:0/2010969150 shutdown_connections 2024-09-13T18:32:41.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.092+0000 7f114ffff700 1 --2- 172.21.15.146:0/2010969150 >> v2:172.21.15.154:3300/0 conn(0x7f11500c91d0 0x7f11500c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:41.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.092+0000 7f114ffff700 1 --2- 172.21.15.146:0/2010969150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f115014fba0 0x7f11501562a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:41.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.092+0000 7f114ffff700 1 -- 172.21.15.146:0/2010969150 >> 172.21.15.146:0/2010969150 conn(0x7f11500cac60 msgr2=0x7f11500cb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:41.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.092+0000 7f114ffff700 1 -- 172.21.15.146:0/2010969150 shutdown_connections 2024-09-13T18:32:41.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.092+0000 7f114ffff700 1 -- 172.21.15.146:0/2010969150 wait complete. 2024-09-13T18:32:41.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f114ffff700 1 Processor -- start 2024-09-13T18:32:41.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f114ffff700 1 -- start start 2024-09-13T18:32:41.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f114ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11500c91d0 0x7f1150145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:41.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f114ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11500ca730 0x7f115006e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:41.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f114ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f115014fba0 0x7f115006eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:41.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f114ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11500726a0 con 0x7f11500ca730 2024-09-13T18:32:41.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f114ffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1150144c60 con 0x7f115014fba0 2024-09-13T18:32:41.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f114ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1150144ec0 con 0x7f11500c91d0 2024-09-13T18:32:41.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f11467fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11500ca730 0x7f115006e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:41.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f11467fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11500ca730 0x7f115006e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46276/0 (socket says 172.21.15.146:46276) 2024-09-13T18:32:41.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f11467fc700 1 -- 172.21.15.146:0/2573466152 learned_addr learned my addr 172.21.15.146:0/2573466152 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:41.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f114f7fe700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f115014fba0 0x7f115006eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:41.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f114effd700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11500c91d0 0x7f1150145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:41.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f11467fc700 1 -- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11500c91d0 msgr2=0x7f1150145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:41.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f11467fc700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11500c91d0 0x7f1150145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:41.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f11467fc700 1 -- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f115014fba0 msgr2=0x7f115006eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:41.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f11467fc700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f115014fba0 0x7f115006eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:41.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.096+0000 7f11467fc700 1 -- 172.21.15.146:0/2573466152 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1140009570 con 0x7f11500ca730 2024-09-13T18:32:41.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.100+0000 7f11467fc700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11500ca730 0x7f115006e600 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f113800e940 tx=0x7f113800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:41.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.100+0000 7f1147fff700 1 -- 172.21.15.146:0/2573466152 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11380052d0 con 0x7f11500ca730 2024-09-13T18:32:41.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.100+0000 7f1147fff700 1 -- 172.21.15.146:0/2573466152 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1138005430 con 0x7f11500ca730 2024-09-13T18:32:41.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.100+0000 7f114ffff700 1 -- 172.21.15.146:0/2573466152 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1150154240 con 0x7f11500ca730 2024-09-13T18:32:41.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.100+0000 7f1147fff700 1 -- 172.21.15.146:0/2573466152 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11380055a0 con 0x7f11500ca730 2024-09-13T18:32:41.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.100+0000 7f114ffff700 1 -- 172.21.15.146:0/2573466152 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1150071d00 con 0x7f11500ca730 2024-09-13T18:32:41.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.100+0000 7f1147fff700 1 -- 172.21.15.146:0/2573466152 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f11380038e0 con 0x7f11500ca730 2024-09-13T18:32:41.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.100+0000 7f114ffff700 1 -- 172.21.15.146:0/2573466152 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115006e450 con 0x7f11500ca730 2024-09-13T18:32:41.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.100+0000 7f1147fff700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1130066730 0x7f1130068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:41.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.100+0000 7f1147fff700 1 -- 172.21.15.146:0/2573466152 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f11380865a0 con 0x7f11500ca730 2024-09-13T18:32:41.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.100+0000 7f114effd700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1130066730 0x7f1130068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:41.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.104+0000 7f114effd700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1130066730 0x7f1130068bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1140000c00 tx=0x7f1140008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:41.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.108+0000 7f1147fff700 1 -- 172.21.15.146:0/2573466152 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1138053460 con 0x7f11500ca730 2024-09-13T18:32:41.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.320+0000 7f114ffff700 1 -- 172.21.15.146:0/2573466152 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11500c17c0 con 0x7f11500ca730 2024-09-13T18:32:41.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.320+0000 7f1147fff700 1 -- 172.21.15.146:0/2573466152 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f1138056a80 con 0x7f11500ca730 2024-09-13T18:32:41.331 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:32:41.331 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:51.700582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:32:41.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.324+0000 7f114ffff700 1 -- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1130066730 msgr2=0x7f1130068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:41.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.324+0000 7f114ffff700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1130066730 0x7f1130068bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1140000c00 tx=0x7f1140008030 comp rx=0 tx=0).stop 2024-09-13T18:32:41.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.324+0000 7f114ffff700 1 -- 172.21.15.146:0/2573466152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11500ca730 msgr2=0x7f115006e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:41.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.324+0000 7f114ffff700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11500ca730 0x7f115006e600 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f113800e940 tx=0x7f113800ec50 comp rx=0 tx=0).stop 2024-09-13T18:32:41.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.324+0000 7f114ffff700 1 -- 172.21.15.146:0/2573466152 shutdown_connections 2024-09-13T18:32:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.324+0000 7f114ffff700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1130066730 0x7f1130068bf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.324+0000 7f114ffff700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f115014fba0 0x7f115006eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.324+0000 7f114ffff700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11500ca730 0x7f115006e600 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.324+0000 7f114ffff700 1 --2- 172.21.15.146:0/2573466152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11500c91d0 0x7f1150145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.324+0000 7f114ffff700 1 -- 172.21.15.146:0/2573466152 >> 172.21.15.146:0/2573466152 conn(0x7f11500cac60 msgr2=0x7f11500cb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.324+0000 7f114ffff700 1 -- 172.21.15.146:0/2573466152 shutdown_connections 2024-09-13T18:32:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:41.324+0000 7f114ffff700 1 -- 172.21.15.146:0/2573466152 wait complete. 2024-09-13T18:32:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 33 2024-09-13T18:32:41.347 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:32:43.348 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:32:43.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b55ca3700 1 Processor -- start 2024-09-13T18:32:43.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b55ca3700 1 -- start start 2024-09-13T18:32:43.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b55ca3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b50153bc0 0x7f1b500c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:43.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b55ca3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b500c6610 0x7f1b500c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:43.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b55ca3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1b500cac60 0x7f1b500cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:43.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b55ca3700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b50153fa0 con 0x7f1b500cac60 2024-09-13T18:32:43.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b55ca3700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b500aa500 con 0x7f1b50153bc0 2024-09-13T18:32:43.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b55ca3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5015a7e0 con 0x7f1b500c6610 2024-09-13T18:32:43.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b554a2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1b500cac60 0x7f1b500cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:43.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b554a2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1b500cac60 0x7f1b500cb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46282/0 (socket says 172.21.15.146:46282) 2024-09-13T18:32:43.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b554a2700 1 -- 172.21.15.146:0/2139754880 learned_addr learned my addr 172.21.15.146:0/2139754880 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:43.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b4ffff700 1 --2- 172.21.15.146:0/2139754880 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b500c6610 0x7f1b500c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:43.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b54ca1700 1 --2- 172.21.15.146:0/2139754880 >> v2:172.21.15.154:3300/0 conn(0x7f1b50153bc0 0x7f1b500c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:43.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b554a2700 1 -- 172.21.15.146:0/2139754880 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b500c6610 msgr2=0x7f1b500c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:43.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b554a2700 1 --2- 172.21.15.146:0/2139754880 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b500c6610 0x7f1b500c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:43.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b554a2700 1 -- 172.21.15.146:0/2139754880 >> v2:172.21.15.154:3300/0 conn(0x7f1b50153bc0 msgr2=0x7f1b500c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:43.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b554a2700 1 --2- 172.21.15.146:0/2139754880 >> v2:172.21.15.154:3300/0 conn(0x7f1b50153bc0 0x7f1b500c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b554a2700 1 -- 172.21.15.146:0/2139754880 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b500c9320 con 0x7f1b500cac60 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.460+0000 7f1b54ca1700 1 --2- 172.21.15.146:0/2139754880 >> v2:172.21.15.154:3300/0 conn(0x7f1b50153bc0 0x7f1b500c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b554a2700 1 --2- 172.21.15.146:0/2139754880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b500cac60 0x7f1b500cb0b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1b50072ac0 tx=0x7f1b48009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2e4c754083b10e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b4f7fe700 1 -- 172.21.15.146:0/2139754880 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48011830 con 0x7f1b500cac60 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b4f7fe700 1 -- 172.21.15.146:0/2139754880 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b48011990 con 0x7f1b500cac60 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b4f7fe700 1 -- 172.21.15.146:0/2139754880 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48011d20 con 0x7f1b500cac60 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2139754880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b500cac60 msgr2=0x7f1b500cb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b55ca3700 1 --2- 172.21.15.146:0/2139754880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b500cac60 0x7f1b500cb0b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1b50072ac0 tx=0x7f1b48009bf0 comp rx=0 tx=0).stop 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2139754880 shutdown_connections 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b55ca3700 1 --2- 172.21.15.146:0/2139754880 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b500c6610 0x7f1b500c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b55ca3700 1 --2- 172.21.15.146:0/2139754880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b500cac60 0x7f1b500cb0b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b55ca3700 1 --2- 172.21.15.146:0/2139754880 >> v2:172.21.15.154:3300/0 conn(0x7f1b50153bc0 0x7f1b500c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2139754880 >> 172.21.15.146:0/2139754880 conn(0x7f1b50066090 msgr2=0x7f1b50066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:43.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2139754880 shutdown_connections 2024-09-13T18:32:43.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2139754880 wait complete. 2024-09-13T18:32:43.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b55ca3700 1 Processor -- start 2024-09-13T18:32:43.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.464+0000 7f1b55ca3700 1 -- start start 2024-09-13T18:32:43.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b55ca3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b500c6610 0x7f1b500aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:43.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b55ca3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b500cac60 0x7f1b500c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:43.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b55ca3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b50153bc0 0x7f1b500c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:43.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b55ca3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500acea0 con 0x7f1b50153bc0 2024-09-13T18:32:43.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b55ca3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500c3bd0 con 0x7f1b500c6610 2024-09-13T18:32:43.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b55ca3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500abd90 con 0x7f1b500cac60 2024-09-13T18:32:43.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b554a2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b50153bc0 0x7f1b500c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:43.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b54ca1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b500c6610 0x7f1b500aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:43.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b4ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b500cac60 0x7f1b500c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:43.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b554a2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b50153bc0 0x7f1b500c1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46294/0 (socket says 172.21.15.146:46294) 2024-09-13T18:32:43.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b554a2700 1 -- 172.21.15.146:0/2596600051 learned_addr learned my addr 172.21.15.146:0/2596600051 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:43.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b4ffff700 1 -- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b500c6610 msgr2=0x7f1b500aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:43.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b4ffff700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b500c6610 0x7f1b500aa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:43.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b4ffff700 1 -- 172.21.15.146:0/2596600051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b50153bc0 msgr2=0x7f1b500c1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:43.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b4ffff700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b50153bc0 0x7f1b500c1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:43.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b4ffff700 1 -- 172.21.15.146:0/2596600051 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b48009570 con 0x7f1b500cac60 2024-09-13T18:32:43.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b54ca1700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b500c6610 0x7f1b500aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:32:43.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b554a2700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b50153bc0 0x7f1b500c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:32:43.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b4ffff700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b500cac60 0x7f1b500c1ab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000eed0 tx=0x7f1b4000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:43.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b4d7fa700 1 -- 172.21.15.146:0/2596600051 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b40010040 con 0x7f1b500cac60 2024-09-13T18:32:43.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b4d7fa700 1 -- 172.21.15.146:0/2596600051 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4000bb00 con 0x7f1b500cac60 2024-09-13T18:32:43.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2596600051 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b500ac070 con 0x7f1b500cac60 2024-09-13T18:32:43.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b4d7fa700 1 -- 172.21.15.146:0/2596600051 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b40014400 con 0x7f1b500cac60 2024-09-13T18:32:43.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.468+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2596600051 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b500b0120 con 0x7f1b500cac60 2024-09-13T18:32:43.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.472+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2596600051 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5015a2c0 con 0x7f1b500cac60 2024-09-13T18:32:43.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.472+0000 7f1b4d7fa700 1 -- 172.21.15.146:0/2596600051 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b40014560 con 0x7f1b500cac60 2024-09-13T18:32:43.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.472+0000 7f1b4d7fa700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1b3c066730 0x7f1b3c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:43.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.472+0000 7f1b4d7fa700 1 -- 172.21.15.146:0/2596600051 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f1b40084e30 con 0x7f1b500cac60 2024-09-13T18:32:43.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.472+0000 7f1b54ca1700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1b3c066730 0x7f1b3c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:43.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.476+0000 7f1b54ca1700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1b3c066730 0x7f1b3c068bf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400cd50 tx=0x7f1b44008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:43.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.484+0000 7f1b4d7fa700 1 -- 172.21.15.146:0/2596600051 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1b40051bc0 con 0x7f1b500cac60 2024-09-13T18:32:43.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.692+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2596600051 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b5006f690 con 0x7f1b500cac60 2024-09-13T18:32:43.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.696+0000 7f1b4d7fa700 1 -- 172.21.15.146:0/2596600051 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f1b40018090 con 0x7f1b500cac60 2024-09-13T18:32:43.704 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:32:43.704 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:51.700582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:32:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.696+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1b3c066730 msgr2=0x7f1b3c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.696+0000 7f1b55ca3700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1b3c066730 0x7f1b3c068bf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400cd50 tx=0x7f1b44008040 comp rx=0 tx=0).stop 2024-09-13T18:32:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.696+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b500cac60 msgr2=0x7f1b500c1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.696+0000 7f1b55ca3700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b500cac60 0x7f1b500c1ab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000eed0 tx=0x7f1b4000b390 comp rx=0 tx=0).stop 2024-09-13T18:32:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.696+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2596600051 shutdown_connections 2024-09-13T18:32:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.696+0000 7f1b55ca3700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1b3c066730 0x7f1b3c068bf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.696+0000 7f1b55ca3700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b50153bc0 0x7f1b500c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.696+0000 7f1b55ca3700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b500cac60 0x7f1b500c1ab0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.696+0000 7f1b55ca3700 1 --2- 172.21.15.146:0/2596600051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b500c6610 0x7f1b500aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:43.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.696+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2596600051 >> 172.21.15.146:0/2596600051 conn(0x7f1b50066090 msgr2=0x7f1b50066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:43.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.696+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2596600051 shutdown_connections 2024-09-13T18:32:43.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:43.696+0000 7f1b55ca3700 1 -- 172.21.15.146:0/2596600051 wait complete. 2024-09-13T18:32:43.708 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 33 2024-09-13T18:32:43.719 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:32:45.720 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:32:45.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b4ccea700 1 Processor -- start 2024-09-13T18:32:45.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b4ccea700 1 -- start start 2024-09-13T18:32:45.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b4ccea700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8b48154a40 0x7f8b480721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:45.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b4ccea700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8b480cac60 0x7f8b480cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b4ccea700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b480c8c70 con 0x7f8b4806f7d0 2024-09-13T18:32:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b4ccea700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b48072810 con 0x7f8b480cac60 2024-09-13T18:32:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b4ccea700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b48157910 con 0x7f8b48154a40 2024-09-13T18:32:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b47fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8b480cac60 0x7f8b480cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b46ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f8b4806f7d0 0x7f8b48157d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57000/0 (socket says 172.21.15.146:57000) 2024-09-13T18:32:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b46ffd700 1 -- 172.21.15.146:0/4226817326 learned_addr learned my addr 172.21.15.146:0/4226817326 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:32:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b477fe700 1 --2- 172.21.15.146:0/4226817326 >> v2:172.21.15.154:3300/0 conn(0x7f8b48154a40 0x7f8b480721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b47fff700 1 -- 172.21.15.146:0/4226817326 >> v2:172.21.15.154:3300/0 conn(0x7f8b48154a40 msgr2=0x7f8b480721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b47fff700 1 --2- 172.21.15.146:0/4226817326 >> v2:172.21.15.154:3300/0 conn(0x7f8b48154a40 0x7f8b480721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b47fff700 1 -- 172.21.15.146:0/4226817326 >> v1:172.21.15.146:6789/0 conn(0x7f8b4806f7d0 legacy=0x7f8b48157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.832+0000 7f8b47fff700 1 -- 172.21.15.146:0/4226817326 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b480c8f50 con 0x7f8b480cac60 2024-09-13T18:32:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b47fff700 1 --2- 172.21.15.146:0/4226817326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b480cac60 0x7f8b480cb0f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8b480cbbe0 tx=0x7f8b4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992eb0e5ac073d28 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b467fc700 1 -- 172.21.15.146:0/4226817326 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4000e070 con 0x7f8b480cac60 2024-09-13T18:32:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b467fc700 1 -- 172.21.15.146:0/4226817326 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4000bb20 con 0x7f8b480cac60 2024-09-13T18:32:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b467fc700 1 -- 172.21.15.146:0/4226817326 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40012430 con 0x7f8b480cac60 2024-09-13T18:32:45.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b4ccea700 1 -- 172.21.15.146:0/4226817326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b480cac60 msgr2=0x7f8b480cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:45.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b4ccea700 1 --2- 172.21.15.146:0/4226817326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b480cac60 0x7f8b480cb0f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8b480cbbe0 tx=0x7f8b4000b330 comp rx=0 tx=0).stop 2024-09-13T18:32:45.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b4ccea700 1 -- 172.21.15.146:0/4226817326 shutdown_connections 2024-09-13T18:32:45.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b4ccea700 1 --2- 172.21.15.146:0/4226817326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b480cac60 0x7f8b480cb0f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:45.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b4ccea700 1 --2- 172.21.15.146:0/4226817326 >> v2:172.21.15.154:3300/0 conn(0x7f8b48154a40 0x7f8b480721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:45.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b4ccea700 1 -- 172.21.15.146:0/4226817326 >> 172.21.15.146:0/4226817326 conn(0x7f8b48066090 msgr2=0x7f8b48066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:45.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b4ccea700 1 -- 172.21.15.146:0/4226817326 shutdown_connections 2024-09-13T18:32:45.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b4ccea700 1 -- 172.21.15.146:0/4226817326 wait complete. 2024-09-13T18:32:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b4ccea700 1 Processor -- start 2024-09-13T18:32:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.836+0000 7f8b4ccea700 1 -- start start 2024-09-13T18:32:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b4ccea700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b4806f7d0 0x7f8b480adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b4ccea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b480cac60 0x7f8b48156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b4ccea700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b48154a40 0x7f8b481ee950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b4ccea700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4814db30 con 0x7f8b4806f7d0 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b4ccea700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4814d500 con 0x7f8b480cac60 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b4ccea700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480aedb0 con 0x7f8b48154a40 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b477fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b4806f7d0 0x7f8b480adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b477fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b4806f7d0 0x7f8b480adb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46312/0 (socket says 172.21.15.146:46312) 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b477fe700 1 -- 172.21.15.146:0/392004982 learned_addr learned my addr 172.21.15.146:0/392004982 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b46ffd700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b480cac60 0x7f8b48156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b47fff700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b48154a40 0x7f8b481ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b477fe700 1 -- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b48154a40 msgr2=0x7f8b481ee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b477fe700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b48154a40 0x7f8b481ee950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b477fe700 1 -- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b480cac60 msgr2=0x7f8b48156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b477fe700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b480cac60 0x7f8b48156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b477fe700 1 -- 172.21.15.146:0/392004982 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40009a70 con 0x7f8b4806f7d0 2024-09-13T18:32:45.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b477fe700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b4806f7d0 0x7f8b480adb50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000ea00 tx=0x7f8b3000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:45.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b3ffff700 1 -- 172.21.15.146:0/392004982 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30009720 con 0x7f8b4806f7d0 2024-09-13T18:32:45.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b3ffff700 1 -- 172.21.15.146:0/392004982 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3000eea0 con 0x7f8b4806f7d0 2024-09-13T18:32:45.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b3ffff700 1 -- 172.21.15.146:0/392004982 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3000ca50 con 0x7f8b4806f7d0 2024-09-13T18:32:45.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b4ccea700 1 -- 172.21.15.146:0/392004982 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b480c15c0 con 0x7f8b4806f7d0 2024-09-13T18:32:45.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b4ccea700 1 -- 172.21.15.146:0/392004982 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b480c1b80 con 0x7f8b4806f7d0 2024-09-13T18:32:45.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.840+0000 7f8b4ccea700 1 -- 172.21.15.146:0/392004982 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4814dda0 con 0x7f8b4806f7d0 2024-09-13T18:32:45.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.844+0000 7f8b3ffff700 1 -- 172.21.15.146:0/392004982 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b3000cbb0 con 0x7f8b4806f7d0 2024-09-13T18:32:45.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.844+0000 7f8b3ffff700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8b28066730 0x7f8b28068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:45.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.844+0000 7f8b3ffff700 1 -- 172.21.15.146:0/392004982 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f8b30084cd0 con 0x7f8b4806f7d0 2024-09-13T18:32:45.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.844+0000 7f8b46ffd700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8b28066730 0x7f8b28068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:45.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.844+0000 7f8b46ffd700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8b28066730 0x7f8b28068bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8b480c11c0 tx=0x7f8b38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:45.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:45.848+0000 7f8b3ffff700 1 -- 172.21.15.146:0/392004982 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8b30051cf0 con 0x7f8b4806f7d0 2024-09-13T18:32:46.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.060+0000 7f8b4ccea700 1 -- 172.21.15.146:0/392004982 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b480c9e00 con 0x7f8b4806f7d0 2024-09-13T18:32:46.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.060+0000 7f8b3ffff700 1 -- 172.21.15.146:0/392004982 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f8b30055310 con 0x7f8b4806f7d0 2024-09-13T18:32:46.069 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:32:46.069 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:51.700582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:32:46.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.064+0000 7f8b4ccea700 1 -- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8b28066730 msgr2=0x7f8b28068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:46.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.064+0000 7f8b4ccea700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8b28066730 0x7f8b28068bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8b480c11c0 tx=0x7f8b38009040 comp rx=0 tx=0).stop 2024-09-13T18:32:46.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.064+0000 7f8b4ccea700 1 -- 172.21.15.146:0/392004982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b4806f7d0 msgr2=0x7f8b480adb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:46.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.064+0000 7f8b4ccea700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b4806f7d0 0x7f8b480adb50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000ea00 tx=0x7f8b3000ed10 comp rx=0 tx=0).stop 2024-09-13T18:32:46.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.064+0000 7f8b4ccea700 1 -- 172.21.15.146:0/392004982 shutdown_connections 2024-09-13T18:32:46.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.064+0000 7f8b4ccea700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8b28066730 0x7f8b28068bf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:46.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.064+0000 7f8b4ccea700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b48154a40 0x7f8b481ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.064+0000 7f8b4ccea700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b480cac60 0x7f8b48156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.064+0000 7f8b4ccea700 1 --2- 172.21.15.146:0/392004982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b4806f7d0 0x7f8b480adb50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.064+0000 7f8b4ccea700 1 -- 172.21.15.146:0/392004982 >> 172.21.15.146:0/392004982 conn(0x7f8b48066090 msgr2=0x7f8b48066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.064+0000 7f8b4ccea700 1 -- 172.21.15.146:0/392004982 shutdown_connections 2024-09-13T18:32:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:46.064+0000 7f8b4ccea700 1 -- 172.21.15.146:0/392004982 wait complete. 2024-09-13T18:32:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 33 2024-09-13T18:32:46.085 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:32:48.086 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:32:48.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.196+0000 7fdc8ce01700 1 Processor -- start 2024-09-13T18:32:48.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.196+0000 7fdc8ce01700 1 -- start start 2024-09-13T18:32:48.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc8ce01700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdc88154a40 0x7fdc880721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:48.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc8ce01700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdc8806f7d0 0x7fdc88157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:48.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc8ce01700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc880c8c70 con 0x7fdc880cac60 2024-09-13T18:32:48.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc8ce01700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc88072810 con 0x7fdc88154a40 2024-09-13T18:32:48.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc8ce01700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc88157910 con 0x7fdc8806f7d0 2024-09-13T18:32:48.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc87fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fdc880cac60 0x7fdc8815a900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57016/0 (socket says 172.21.15.146:57016) 2024-09-13T18:32:48.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc87fff700 1 -- 172.21.15.146:0/4009071381 learned_addr learned my addr 172.21.15.146:0/4009071381 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:32:48.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc877fe700 1 --2- 172.21.15.146:0/4009071381 >> v2:172.21.15.146:3300/0 conn(0x7fdc88154a40 0x7fdc880721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:48.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc86ffd700 1 --2- 172.21.15.146:0/4009071381 >> v2:172.21.15.154:3300/0 conn(0x7fdc8806f7d0 0x7fdc88157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:48.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc867fc700 1 -- 172.21.15.146:0/4009071381 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501812331 0 0) 0x7fdc880c8c70 con 0x7fdc880cac60 2024-09-13T18:32:48.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc867fc700 1 -- 172.21.15.146:0/4009071381 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc740035d0 con 0x7fdc880cac60 2024-09-13T18:32:48.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc877fe700 1 -- 172.21.15.146:0/4009071381 >> v2:172.21.15.154:3300/0 conn(0x7fdc8806f7d0 msgr2=0x7fdc88157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:48.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc877fe700 1 --2- 172.21.15.146:0/4009071381 >> v2:172.21.15.154:3300/0 conn(0x7fdc8806f7d0 0x7fdc88157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:48.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc877fe700 1 -- 172.21.15.146:0/4009071381 >> v1:172.21.15.146:6789/0 conn(0x7fdc880cac60 legacy=0x7fdc8815a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:48.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc877fe700 1 -- 172.21.15.146:0/4009071381 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc880c8f50 con 0x7fdc88154a40 2024-09-13T18:32:48.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc877fe700 1 --2- 172.21.15.146:0/4009071381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc88154a40 0x7fdc880721f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdc880ca1c0 tx=0x7fdc78009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4001183d0ff061b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:48.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc867fc700 1 -- 172.21.15.146:0/4009071381 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc78011830 con 0x7fdc88154a40 2024-09-13T18:32:48.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc867fc700 1 -- 172.21.15.146:0/4009071381 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc78011990 con 0x7fdc88154a40 2024-09-13T18:32:48.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc867fc700 1 -- 172.21.15.146:0/4009071381 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc78011d00 con 0x7fdc88154a40 2024-09-13T18:32:48.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc8ce01700 1 -- 172.21.15.146:0/4009071381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc88154a40 msgr2=0x7fdc880721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:48.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc8ce01700 1 --2- 172.21.15.146:0/4009071381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc88154a40 0x7fdc880721f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdc880ca1c0 tx=0x7fdc78009b30 comp rx=0 tx=0).stop 2024-09-13T18:32:48.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc8ce01700 1 -- 172.21.15.146:0/4009071381 shutdown_connections 2024-09-13T18:32:48.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc8ce01700 1 --2- 172.21.15.146:0/4009071381 >> v2:172.21.15.154:3300/0 conn(0x7fdc8806f7d0 0x7fdc88157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:48.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc8ce01700 1 --2- 172.21.15.146:0/4009071381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc88154a40 0x7fdc880721f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:48.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc8ce01700 1 -- 172.21.15.146:0/4009071381 >> 172.21.15.146:0/4009071381 conn(0x7fdc88066090 msgr2=0x7fdc88066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:48.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc8ce01700 1 -- 172.21.15.146:0/4009071381 shutdown_connections 2024-09-13T18:32:48.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.200+0000 7fdc8ce01700 1 -- 172.21.15.146:0/4009071381 wait complete. 2024-09-13T18:32:48.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc8ce01700 1 Processor -- start 2024-09-13T18:32:48.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc8ce01700 1 -- start start 2024-09-13T18:32:48.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc8ce01700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8806f7d0 0x7fdc8814d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:48.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc8ce01700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc880cac60 0x7fdc881f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:48.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc8ce01700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc88154a40 0x7fdc880bfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:48.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc8ce01700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8806dbf0 con 0x7fdc880cac60 2024-09-13T18:32:48.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc8ce01700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc881f21f0 con 0x7fdc88154a40 2024-09-13T18:32:48.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc8ce01700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc880c11d0 con 0x7fdc8806f7d0 2024-09-13T18:32:48.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc86ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc880cac60 0x7fdc881f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:48.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc86ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc880cac60 0x7fdc881f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46328/0 (socket says 172.21.15.146:46328) 2024-09-13T18:32:48.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc87fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc88154a40 0x7fdc880bfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:48.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc86ffd700 1 -- 172.21.15.146:0/545792754 learned_addr learned my addr 172.21.15.146:0/545792754 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:48.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc877fe700 1 --2- 172.21.15.146:0/545792754 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8806f7d0 0x7fdc8814d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:48.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc86ffd700 1 -- 172.21.15.146:0/545792754 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8806f7d0 msgr2=0x7fdc8814d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:48.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc86ffd700 1 --2- 172.21.15.146:0/545792754 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8806f7d0 0x7fdc8814d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:48.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc86ffd700 1 -- 172.21.15.146:0/545792754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc88154a40 msgr2=0x7fdc880bfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:48.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc86ffd700 1 --2- 172.21.15.146:0/545792754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc88154a40 0x7fdc880bfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:48.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc86ffd700 1 -- 172.21.15.146:0/545792754 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc780094b0 con 0x7fdc880cac60 2024-09-13T18:32:48.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc86ffd700 1 --2- 172.21.15.146:0/545792754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc880cac60 0x7fdc881f2680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000e840 tx=0x7fdc7000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:48.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc7ffff700 1 -- 172.21.15.146:0/545792754 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc700095a0 con 0x7fdc880cac60 2024-09-13T18:32:48.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc7ffff700 1 -- 172.21.15.146:0/545792754 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc70013680 con 0x7fdc880cac60 2024-09-13T18:32:48.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc7ffff700 1 -- 172.21.15.146:0/545792754 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc700139f0 con 0x7fdc880cac60 2024-09-13T18:32:48.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc8ce01700 1 -- 172.21.15.146:0/545792754 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc880c1490 con 0x7fdc880cac60 2024-09-13T18:32:48.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.204+0000 7fdc8ce01700 1 -- 172.21.15.146:0/545792754 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc880c1a50 con 0x7fdc880cac60 2024-09-13T18:32:48.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.208+0000 7fdc8ce01700 1 -- 172.21.15.146:0/545792754 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc880c9e00 con 0x7fdc880cac60 2024-09-13T18:32:48.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.208+0000 7fdc7ffff700 1 -- 172.21.15.146:0/545792754 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc70013b50 con 0x7fdc880cac60 2024-09-13T18:32:48.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.208+0000 7fdc7ffff700 1 --2- 172.21.15.146:0/545792754 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdc68066730 0x7fdc68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:48.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.208+0000 7fdc7ffff700 1 -- 172.21.15.146:0/545792754 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fdc70084dd0 con 0x7fdc880cac60 2024-09-13T18:32:48.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.208+0000 7fdc877fe700 1 --2- 172.21.15.146:0/545792754 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdc68066730 0x7fdc68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:48.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.212+0000 7fdc877fe700 1 --2- 172.21.15.146:0/545792754 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdc68066730 0x7fdc68068bf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdc78000c00 tx=0x7fdc78008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:48.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.216+0000 7fdc7ffff700 1 -- 172.21.15.146:0/545792754 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdc70051bf0 con 0x7fdc880cac60 2024-09-13T18:32:48.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.420+0000 7fdc8ce01700 1 -- 172.21.15.146:0/545792754 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc88154e20 con 0x7fdc880cac60 2024-09-13T18:32:48.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.420+0000 7fdc7ffff700 1 -- 172.21.15.146:0/545792754 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6288 (secure 0 0 0) 0x7fdc70021090 con 0x7fdc880cac60 2024-09-13T18:32:48.430 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:32:48.430 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:51.700582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:32:48.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.424+0000 7fdc8ce01700 1 -- 172.21.15.146:0/545792754 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdc68066730 msgr2=0x7fdc68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:48.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.424+0000 7fdc8ce01700 1 --2- 172.21.15.146:0/545792754 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdc68066730 0x7fdc68068bf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdc78000c00 tx=0x7fdc78008030 comp rx=0 tx=0).stop 2024-09-13T18:32:48.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.424+0000 7fdc8ce01700 1 -- 172.21.15.146:0/545792754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc880cac60 msgr2=0x7fdc881f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:48.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.424+0000 7fdc8ce01700 1 --2- 172.21.15.146:0/545792754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc880cac60 0x7fdc881f2680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000e840 tx=0x7fdc7000eb50 comp rx=0 tx=0).stop 2024-09-13T18:32:48.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.424+0000 7fdc8ce01700 1 -- 172.21.15.146:0/545792754 shutdown_connections 2024-09-13T18:32:48.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.424+0000 7fdc8ce01700 1 --2- 172.21.15.146:0/545792754 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdc68066730 0x7fdc68068bf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:48.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.424+0000 7fdc8ce01700 1 --2- 172.21.15.146:0/545792754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc88154a40 0x7fdc880bfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:48.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.424+0000 7fdc8ce01700 1 --2- 172.21.15.146:0/545792754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc880cac60 0x7fdc881f2680 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:48.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.424+0000 7fdc8ce01700 1 --2- 172.21.15.146:0/545792754 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8806f7d0 0x7fdc8814d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:48.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.424+0000 7fdc8ce01700 1 -- 172.21.15.146:0/545792754 >> 172.21.15.146:0/545792754 conn(0x7fdc88066090 msgr2=0x7fdc880700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:48.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.424+0000 7fdc8ce01700 1 -- 172.21.15.146:0/545792754 shutdown_connections 2024-09-13T18:32:48.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:48.424+0000 7fdc8ce01700 1 -- 172.21.15.146:0/545792754 wait complete. 2024-09-13T18:32:48.433 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 33 2024-09-13T18:32:48.444 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:32:50.446 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:32:50.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f57621bd700 1 Processor -- start 2024-09-13T18:32:50.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f57621bd700 1 -- start start 2024-09-13T18:32:50.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f57621bd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c070570 0x7f575c0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:50.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f57621bd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f575c0c5bd0 0x7f575c0c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:50.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f57621bd700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f575c0adfe0 con 0x7f575c157e10 2024-09-13T18:32:50.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f57621bd700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f575c15e690 con 0x7f575c0c5bd0 2024-09-13T18:32:50.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f57621bd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f575c0c7570 con 0x7f575c070570 2024-09-13T18:32:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f575bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f575c0c5bd0 0x7f575c0c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f575affd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c070570 0x7f575c0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f575affd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c070570 0x7f575c0c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:43740/0 (socket says 172.21.15.146:43740) 2024-09-13T18:32:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f575affd700 1 -- 172.21.15.146:0/1872595927 learned_addr learned my addr 172.21.15.146:0/1872595927 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f575affd700 1 -- 172.21.15.146:0/1872595927 >> v2:172.21.15.154:3300/0 conn(0x7f575c0c5bd0 msgr2=0x7f575c0c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f575affd700 1 --2- 172.21.15.146:0/1872595927 >> v2:172.21.15.154:3300/0 conn(0x7f575c0c5bd0 0x7f575c0c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f575affd700 1 -- 172.21.15.146:0/1872595927 >> v1:172.21.15.154:6789/0 conn(0x7f575c157e10 legacy=0x7f575c06fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f575a7fc700 1 -- 172.21.15.146:0/1872595927 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039132962 0 0) 0x7f575c0adfe0 con 0x7f575c157e10 2024-09-13T18:32:50.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f575affd700 1 -- 172.21.15.146:0/1872595927 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c15eb80 con 0x7f575c070570 2024-09-13T18:32:50.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f575affd700 1 --2- 172.21.15.146:0/1872595927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c070570 0x7f575c0c6df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5754009e50 tx=0x7f575400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69a773593a6f9758 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:50.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f575a7fc700 1 -- 172.21.15.146:0/1872595927 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575400e070 con 0x7f575c070570 2024-09-13T18:32:50.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f575a7fc700 1 -- 172.21.15.146:0/1872595927 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575400b910 con 0x7f575c070570 2024-09-13T18:32:50.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f575a7fc700 1 -- 172.21.15.146:0/1872595927 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575400bcc0 con 0x7f575c070570 2024-09-13T18:32:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f57621bd700 1 -- 172.21.15.146:0/1872595927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c070570 msgr2=0x7f575c0c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.560+0000 7f57621bd700 1 --2- 172.21.15.146:0/1872595927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c070570 0x7f575c0c6df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5754009e50 tx=0x7f575400b330 comp rx=0 tx=0).stop 2024-09-13T18:32:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.564+0000 7f57621bd700 1 -- 172.21.15.146:0/1872595927 shutdown_connections 2024-09-13T18:32:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.564+0000 7f57621bd700 1 --2- 172.21.15.146:0/1872595927 >> v2:172.21.15.154:3300/0 conn(0x7f575c0c5bd0 0x7f575c0c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.564+0000 7f57621bd700 1 --2- 172.21.15.146:0/1872595927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c070570 0x7f575c0c6df0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:50.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.564+0000 7f57621bd700 1 -- 172.21.15.146:0/1872595927 >> 172.21.15.146:0/1872595927 conn(0x7f575c0cb1d0 msgr2=0x7f575c0cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:50.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.564+0000 7f57621bd700 1 -- 172.21.15.146:0/1872595927 shutdown_connections 2024-09-13T18:32:50.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.564+0000 7f57621bd700 1 -- 172.21.15.146:0/1872595927 wait complete. 2024-09-13T18:32:50.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.564+0000 7f57621bd700 1 Processor -- start 2024-09-13T18:32:50.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.564+0000 7f57621bd700 1 -- start start 2024-09-13T18:32:50.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f57621bd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f575c070570 0x7f575c1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:50.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f57621bd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c0c5bd0 0x7f575c1f4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f57621bd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f575c157e10 0x7f575c1f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f57621bd700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c1f3de0 con 0x7f575c157e10 2024-09-13T18:32:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f57621bd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c160f50 con 0x7f575c070570 2024-09-13T18:32:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f57621bd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f575c160c70 con 0x7f575c0c5bd0 2024-09-13T18:32:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f575affd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c0c5bd0 0x7f575c1f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f575affd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c0c5bd0 0x7f575c1f4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:43746/0 (socket says 172.21.15.146:43746) 2024-09-13T18:32:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f575b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f575c070570 0x7f575c1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f575affd700 1 -- 172.21.15.146:0/2685924970 learned_addr learned my addr 172.21.15.146:0/2685924970 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f575bfff700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f575c157e10 0x7f575c1f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f575affd700 1 -- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f575c070570 msgr2=0x7f575c1618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f575affd700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f575c070570 0x7f575c1618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f575affd700 1 -- 172.21.15.146:0/2685924970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f575c157e10 msgr2=0x7f575c1f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f575affd700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f575c157e10 0x7f575c1f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f575affd700 1 -- 172.21.15.146:0/2685924970 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5754009b30 con 0x7f575c0c5bd0 2024-09-13T18:32:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f575b7fe700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f575c070570 0x7f575c1618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:32:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f575affd700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c0c5bd0 0x7f575c1f4200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f575c15e7d0 tx=0x7f5754004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f5743fff700 1 -- 172.21.15.146:0/2685924970 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57540048c0 con 0x7f575c0c5bd0 2024-09-13T18:32:50.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f5743fff700 1 -- 172.21.15.146:0/2685924970 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57540099b0 con 0x7f575c0c5bd0 2024-09-13T18:32:50.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f57621bd700 1 -- 172.21.15.146:0/2685924970 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f575c0c2600 con 0x7f575c0c5bd0 2024-09-13T18:32:50.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f5743fff700 1 -- 172.21.15.146:0/2685924970 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575401b650 con 0x7f575c0c5bd0 2024-09-13T18:32:50.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f57621bd700 1 -- 172.21.15.146:0/2685924970 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f575c0c2bc0 con 0x7f575c0c5bd0 2024-09-13T18:32:50.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.568+0000 7f57621bd700 1 -- 172.21.15.146:0/2685924970 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f575c1f3f20 con 0x7f575c0c5bd0 2024-09-13T18:32:50.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.572+0000 7f5743fff700 1 -- 172.21.15.146:0/2685924970 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f575401b7b0 con 0x7f575c0c5bd0 2024-09-13T18:32:50.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.572+0000 7f5743fff700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5748066730 0x7f5748068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:50.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.572+0000 7f5743fff700 1 -- 172.21.15.146:0/2685924970 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f5754086170 con 0x7f575c0c5bd0 2024-09-13T18:32:50.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.572+0000 7f575b7fe700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5748066730 0x7f5748068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:50.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.576+0000 7f575b7fe700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5748066730 0x7f5748068bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5750007fd0 tx=0x7f5750009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:50.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.584+0000 7f5743fff700 1 -- 172.21.15.146:0/2685924970 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5754053190 con 0x7f575c0c5bd0 2024-09-13T18:32:50.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.796+0000 7f57621bd700 1 -- 172.21.15.146:0/2685924970 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f575c1f3f20 con 0x7f575c0c5bd0 2024-09-13T18:32:50.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.796+0000 7f5743fff700 1 -- 172.21.15.146:0/2685924970 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f575c1f3f20 con 0x7f575c0c5bd0 2024-09-13T18:32:50.806 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:32:50.806 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:51.700582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:32:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.800+0000 7f57621bd700 1 -- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5748066730 msgr2=0x7f5748068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:50.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.800+0000 7f57621bd700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5748066730 0x7f5748068bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5750007fd0 tx=0x7f5750009040 comp rx=0 tx=0).stop 2024-09-13T18:32:50.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.800+0000 7f57621bd700 1 -- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c0c5bd0 msgr2=0x7f575c1f4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:50.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.800+0000 7f57621bd700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c0c5bd0 0x7f575c1f4200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f575c15e7d0 tx=0x7f5754004af0 comp rx=0 tx=0).stop 2024-09-13T18:32:50.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.800+0000 7f57621bd700 1 -- 172.21.15.146:0/2685924970 shutdown_connections 2024-09-13T18:32:50.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.800+0000 7f57621bd700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5748066730 0x7f5748068bf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:50.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.800+0000 7f57621bd700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f575c157e10 0x7f575c1f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:50.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.800+0000 7f57621bd700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f575c0c5bd0 0x7f575c1f4200 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.800+0000 7f57621bd700 1 --2- 172.21.15.146:0/2685924970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f575c070570 0x7f575c1618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.800+0000 7f57621bd700 1 -- 172.21.15.146:0/2685924970 >> 172.21.15.146:0/2685924970 conn(0x7f575c0cb1d0 msgr2=0x7f575c0c9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.800+0000 7f57621bd700 1 -- 172.21.15.146:0/2685924970 shutdown_connections 2024-09-13T18:32:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:50.800+0000 7f57621bd700 1 -- 172.21.15.146:0/2685924970 wait complete. 2024-09-13T18:32:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 33 2024-09-13T18:32:50.822 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:32:52.823 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:32:52.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.936+0000 7f621b59e700 1 Processor -- start 2024-09-13T18:32:52.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621b59e700 1 -- start start 2024-09-13T18:32:52.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621b59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f621c153bc0 0x7f621c0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f621c0c6610 0x7f621c0c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621b59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621c153fa0 con 0x7f621c0cac60 2024-09-13T18:32:52.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621b59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f621c0aa500 con 0x7f621c153bc0 2024-09-13T18:32:52.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621c15a7e0 con 0x7f621c0c6610 2024-09-13T18:32:52.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621ad9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f621c0cac60 0x7f621c0c7740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:47154/0 (socket says 172.21.15.146:47154) 2024-09-13T18:32:52.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621ad9d700 1 -- 172.21.15.146:0/2160398572 learned_addr learned my addr 172.21.15.146:0/2160398572 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:32:52.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6211d9b700 1 --2- 172.21.15.146:0/2160398572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f621c0c6610 0x7f621c0c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621a59c700 1 --2- 172.21.15.146:0/2160398572 >> v2:172.21.15.154:3300/0 conn(0x7f621c153bc0 0x7f621c0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208215548 0 0) 0x7f621c153fa0 con 0x7f621c0cac60 2024-09-13T18:32:52.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62080035d0 con 0x7f621c0cac60 2024-09-13T18:32:52.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3870967776 0 0) 0x7f62080035d0 con 0x7f621c0cac60 2024-09-13T18:32:52.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f621c0c6610 msgr2=0x7f621c0c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:52.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 --2- 172.21.15.146:0/2160398572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f621c0c6610 0x7f621c0c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 >> v2:172.21.15.154:3300/0 conn(0x7f621c153bc0 msgr2=0x7f621c0c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:52.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 --2- 172.21.15.146:0/2160398572 >> v2:172.21.15.154:3300/0 conn(0x7f621c153bc0 0x7f621c0c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c0c9320 con 0x7f621c0cac60 2024-09-13T18:32:52.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f62140019f0 con 0x7f621c0cac60 2024-09-13T18:32:52.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 >> v1:172.21.15.146:6789/0 conn(0x7f621c0cac60 legacy=0x7f621c0c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:52.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 --2- 172.21.15.146:0/2160398572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f621c0c6610 0x7f6208003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 --2- 172.21.15.146:0/2160398572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6208004010 0x7f6208004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 --2- 172.21.15.146:0/2160398572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6208004db0 0x7f6208005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0aa500 con 0x7f6208004db0 2024-09-13T18:32:52.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c15a7e0 con 0x7f6208004010 2024-09-13T18:32:52.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620800ba60 con 0x7f621c0c6610 2024-09-13T18:32:52.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621ad9d700 1 --2- 172.21.15.146:0/2160398572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6208004db0 0x7f6208005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621a59c700 1 --2- 172.21.15.146:0/2160398572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f621c0c6610 0x7f6208003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6211d9b700 1 --2- 172.21.15.146:0/2160398572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6208004010 0x7f6208004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621ad9d700 1 -- 172.21.15.146:0/2160398572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f621c0c6610 msgr2=0x7f6208003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:52.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621ad9d700 1 --2- 172.21.15.146:0/2160398572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f621c0c6610 0x7f6208003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621ad9d700 1 -- 172.21.15.146:0/2160398572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6208004010 msgr2=0x7f6208004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:52.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621ad9d700 1 --2- 172.21.15.146:0/2160398572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6208004010 0x7f6208004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621ad9d700 1 -- 172.21.15.146:0/2160398572 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62080035d0 con 0x7f6208004db0 2024-09-13T18:32:52.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f621ad9d700 1 --2- 172.21.15.146:0/2160398572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6208004db0 0x7f6208005230 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f621400a8d0 tx=0x7f62080058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc6b6f01f11cb85 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:52.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621400a340 con 0x7f6208004db0 2024-09-13T18:32:52.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62140072d0 con 0x7f6208004db0 2024-09-13T18:32:52.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.940+0000 7f6219d9b700 1 -- 172.21.15.146:0/2160398572 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6214007680 con 0x7f6208004db0 2024-09-13T18:32:52.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 -- 172.21.15.146:0/2160398572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6208004db0 msgr2=0x7f6208005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:52.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 --2- 172.21.15.146:0/2160398572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6208004db0 0x7f6208005230 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f621400a8d0 tx=0x7f62080058a0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621ad9d700 1 -- 172.21.15.146:0/2160398572 reap_dead start 2024-09-13T18:32:52.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 -- 172.21.15.146:0/2160398572 shutdown_connections 2024-09-13T18:32:52.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 -- 172.21.15.146:0/2160398572 >> 172.21.15.146:0/2160398572 conn(0x7f621c066090 msgr2=0x7f621c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:52.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 -- 172.21.15.146:0/2160398572 shutdown_connections 2024-09-13T18:32:52.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 -- 172.21.15.146:0/2160398572 wait complete. 2024-09-13T18:32:52.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 Processor -- start 2024-09-13T18:32:52.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 -- start start 2024-09-13T18:32:52.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6208004db0 0x7f621c1eb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f621c1579f0 0x7f621c157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621c1552d0 0x7f621c155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c157230 con 0x7f621c1552d0 2024-09-13T18:32:52.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c158390 con 0x7f6208004db0 2024-09-13T18:32:52.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621c1566b0 con 0x7f621c1579f0 2024-09-13T18:32:52.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621c1552d0 0x7f621c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621c1552d0 0x7f621c155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39758/0 (socket says 172.21.15.146:39758) 2024-09-13T18:32:52.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621ad9d700 1 -- 172.21.15.146:0/3718732706 learned_addr learned my addr 172.21.15.146:0/3718732706 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:52.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621a59c700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6208004db0 0x7f621c1eb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f6211d9b700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f621c1579f0 0x7f621c157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621ad9d700 1 -- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f621c1579f0 msgr2=0x7f621c157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:52.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621ad9d700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f621c1579f0 0x7f621c157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621ad9d700 1 -- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6208004db0 msgr2=0x7f621c1eb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:52.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621ad9d700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6208004db0 0x7f621c1eb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.944+0000 7f621ad9d700 1 -- 172.21.15.146:0/3718732706 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c156990 con 0x7f621c1552d0 2024-09-13T18:32:52.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.948+0000 7f621ad9d700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621c1552d0 0x7f621c155750 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6214001cb0 tx=0x7f6214001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:52.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.948+0000 7f62137fe700 1 -- 172.21.15.146:0/3718732706 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621400a340 con 0x7f621c1552d0 2024-09-13T18:32:52.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.948+0000 7f62137fe700 1 -- 172.21.15.146:0/3718732706 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62140019b0 con 0x7f621c1552d0 2024-09-13T18:32:52.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.948+0000 7f621b59e700 1 -- 172.21.15.146:0/3718732706 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621c156c70 con 0x7f621c1552d0 2024-09-13T18:32:52.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.948+0000 7f62137fe700 1 -- 172.21.15.146:0/3718732706 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6214012640 con 0x7f621c1552d0 2024-09-13T18:32:52.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.948+0000 7f621b59e700 1 -- 172.21.15.146:0/3718732706 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c1574d0 con 0x7f621c1552d0 2024-09-13T18:32:52.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.948+0000 7f621b59e700 1 -- 172.21.15.146:0/3718732706 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621c15a2c0 con 0x7f621c1552d0 2024-09-13T18:32:52.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.948+0000 7f62137fe700 1 -- 172.21.15.146:0/3718732706 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f621401a070 con 0x7f621c1552d0 2024-09-13T18:32:52.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.948+0000 7f62137fe700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61fc066750 0x7f61fc068c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.948+0000 7f62137fe700 1 -- 172.21.15.146:0/3718732706 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f6214085e50 con 0x7f621c1552d0 2024-09-13T18:32:52.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.948+0000 7f621a59c700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61fc066750 0x7f61fc068c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.952+0000 7f621a59c700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61fc066750 0x7f61fc068c10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f620c009cf0 tx=0x7f620c009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:52.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:52.956+0000 7f62137fe700 1 -- 172.21.15.146:0/3718732706 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6214052ef0 con 0x7f621c1552d0 2024-09-13T18:32:53.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.164+0000 7f621b59e700 1 -- 172.21.15.146:0/3718732706 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f621c06f690 con 0x7f621c1552d0 2024-09-13T18:32:53.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.164+0000 7f62137fe700 1 -- 172.21.15.146:0/3718732706 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f62140011c0 con 0x7f621c1552d0 2024-09-13T18:32:53.174 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:32:53.174 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:51.700582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:32:53.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.168+0000 7f621b59e700 1 -- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61fc066750 msgr2=0x7f61fc068c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:53.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.168+0000 7f621b59e700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61fc066750 0x7f61fc068c10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f620c009cf0 tx=0x7f620c009240 comp rx=0 tx=0).stop 2024-09-13T18:32:53.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.168+0000 7f621b59e700 1 -- 172.21.15.146:0/3718732706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621c1552d0 msgr2=0x7f621c155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:53.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.168+0000 7f621b59e700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621c1552d0 0x7f621c155750 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6214001cb0 tx=0x7f6214001360 comp rx=0 tx=0).stop 2024-09-13T18:32:53.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.168+0000 7f621b59e700 1 -- 172.21.15.146:0/3718732706 shutdown_connections 2024-09-13T18:32:53.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.168+0000 7f621b59e700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61fc066750 0x7f61fc068c10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:53.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.168+0000 7f621b59e700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621c1552d0 0x7f621c155750 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:53.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.168+0000 7f621b59e700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f621c1579f0 0x7f621c157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:53.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.168+0000 7f621b59e700 1 --2- 172.21.15.146:0/3718732706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6208004db0 0x7f621c1eb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:53.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.168+0000 7f621b59e700 1 -- 172.21.15.146:0/3718732706 >> 172.21.15.146:0/3718732706 conn(0x7f621c066090 msgr2=0x7f621c0a9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:53.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.168+0000 7f621b59e700 1 -- 172.21.15.146:0/3718732706 shutdown_connections 2024-09-13T18:32:53.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:53.168+0000 7f621b59e700 1 -- 172.21.15.146:0/3718732706 wait complete. 2024-09-13T18:32:53.181 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 33 2024-09-13T18:32:53.190 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:32:55.191 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:32:55.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.304+0000 7f32ecf6f700 1 Processor -- start 2024-09-13T18:32:55.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.304+0000 7f32ecf6f700 1 -- start start 2024-09-13T18:32:55.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.304+0000 7f32ecf6f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e8153a60 0x7f32e8159700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:55.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.304+0000 7f32ecf6f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f32e80c9ed0 0x7f32e80b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:55.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32ecf6f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f32e80cf540 0x7f32e80cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:55.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32ecf6f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e80cc2d0 con 0x7f32e80c9ed0 2024-09-13T18:32:55.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32ecf6f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e8153e40 con 0x7f32e80cf540 2024-09-13T18:32:55.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32ecf6f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e80cb3e0 con 0x7f32e8153a60 2024-09-13T18:32:55.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32deffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f32e80c9ed0 0x7f32e80b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:55.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32deffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f32e80c9ed0 0x7f32e80b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39768/0 (socket says 172.21.15.146:39768) 2024-09-13T18:32:55.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32deffd700 1 -- 172.21.15.146:0/4090413203 learned_addr learned my addr 172.21.15.146:0/4090413203 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:55.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32e7fff700 1 --2- 172.21.15.146:0/4090413203 >> v2:172.21.15.154:3300/0 conn(0x7f32e80cf540 0x7f32e80cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:55.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32e77fe700 1 --2- 172.21.15.146:0/4090413203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e8153a60 0x7f32e8159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:55.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32deffd700 1 -- 172.21.15.146:0/4090413203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e8153a60 msgr2=0x7f32e8159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:55.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32deffd700 1 --2- 172.21.15.146:0/4090413203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e8153a60 0x7f32e8159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:55.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32deffd700 1 -- 172.21.15.146:0/4090413203 >> v2:172.21.15.154:3300/0 conn(0x7f32e80cf540 msgr2=0x7f32e80cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:55.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32deffd700 1 --2- 172.21.15.146:0/4090413203 >> v2:172.21.15.154:3300/0 conn(0x7f32e80cf540 0x7f32e80cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:55.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32deffd700 1 -- 172.21.15.146:0/4090413203 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e80cb6c0 con 0x7f32e80c9ed0 2024-09-13T18:32:55.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32e7fff700 1 --2- 172.21.15.146:0/4090413203 >> v2:172.21.15.154:3300/0 conn(0x7f32e80cf540 0x7f32e80cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:32:55.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32deffd700 1 --2- 172.21.15.146:0/4090413203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e80c9ed0 0x7f32e80b3280 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f32e80b31a0 tx=0x7f32d0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d7b30704f7119a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:55.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32e6ffd700 1 -- 172.21.15.146:0/4090413203 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d000b8a0 con 0x7f32e80c9ed0 2024-09-13T18:32:55.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32e6ffd700 1 -- 172.21.15.146:0/4090413203 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d000ba00 con 0x7f32e80c9ed0 2024-09-13T18:32:55.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32e6ffd700 1 -- 172.21.15.146:0/4090413203 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d000bd90 con 0x7f32e80c9ed0 2024-09-13T18:32:55.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32ecf6f700 1 -- 172.21.15.146:0/4090413203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e80c9ed0 msgr2=0x7f32e80b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:55.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32ecf6f700 1 --2- 172.21.15.146:0/4090413203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e80c9ed0 0x7f32e80b3280 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f32e80b31a0 tx=0x7f32d0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:32:55.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32ecf6f700 1 -- 172.21.15.146:0/4090413203 shutdown_connections 2024-09-13T18:32:55.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32ecf6f700 1 --2- 172.21.15.146:0/4090413203 >> v2:172.21.15.154:3300/0 conn(0x7f32e80cf540 0x7f32e80cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:55.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32ecf6f700 1 --2- 172.21.15.146:0/4090413203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e80c9ed0 0x7f32e80b3280 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:55.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32ecf6f700 1 --2- 172.21.15.146:0/4090413203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e8153a60 0x7f32e8159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:55.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32ecf6f700 1 -- 172.21.15.146:0/4090413203 >> 172.21.15.146:0/4090413203 conn(0x7f32e8071ec0 msgr2=0x7f32e80722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:55.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32ecf6f700 1 -- 172.21.15.146:0/4090413203 shutdown_connections 2024-09-13T18:32:55.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.308+0000 7f32ecf6f700 1 -- 172.21.15.146:0/4090413203 wait complete. 2024-09-13T18:32:55.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32ecf6f700 1 Processor -- start 2024-09-13T18:32:55.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32ecf6f700 1 -- start start 2024-09-13T18:32:55.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32ecf6f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e80cf540 0x7f32e81efa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:55.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32ecf6f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e8153a60 0x7f32e80c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:55.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32ecf6f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e80c6f30 0x7f32e80c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:55.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32ecf6f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e80c6560 con 0x7f32e80c6f30 2024-09-13T18:32:55.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32ecf6f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e80c73e0 con 0x7f32e8153a60 2024-09-13T18:32:55.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32ecf6f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e814b5f0 con 0x7f32e80cf540 2024-09-13T18:32:55.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32e7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e80c6f30 0x7f32e80c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:55.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32deffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e8153a60 0x7f32e80c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:55.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32e7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e80c6f30 0x7f32e80c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39776/0 (socket says 172.21.15.146:39776) 2024-09-13T18:32:55.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32e7fff700 1 -- 172.21.15.146:0/1925899613 learned_addr learned my addr 172.21.15.146:0/1925899613 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:55.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32e77fe700 1 --2- 172.21.15.146:0/1925899613 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e80cf540 0x7f32e81efa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:55.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32e7fff700 1 -- 172.21.15.146:0/1925899613 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e80cf540 msgr2=0x7f32e81efa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:55.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32e7fff700 1 --2- 172.21.15.146:0/1925899613 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e80cf540 0x7f32e81efa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:55.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32e7fff700 1 -- 172.21.15.146:0/1925899613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e8153a60 msgr2=0x7f32e80c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:55.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32e7fff700 1 --2- 172.21.15.146:0/1925899613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e8153a60 0x7f32e80c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:55.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32e7fff700 1 -- 172.21.15.146:0/1925899613 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d0009570 con 0x7f32e80c6f30 2024-09-13T18:32:55.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32e7fff700 1 --2- 172.21.15.146:0/1925899613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e80c6f30 0x7f32e80c8840 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f32e0007e10 tx=0x7f32e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:55.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32e4ff9700 1 -- 172.21.15.146:0/1925899613 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e000dd20 con 0x7f32e80c6f30 2024-09-13T18:32:55.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32e4ff9700 1 -- 172.21.15.146:0/1925899613 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e000f040 con 0x7f32e80c6f30 2024-09-13T18:32:55.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32e4ff9700 1 -- 172.21.15.146:0/1925899613 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0003c80 con 0x7f32e80c6f30 2024-09-13T18:32:55.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32ecf6f700 1 -- 172.21.15.146:0/1925899613 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e814b8d0 con 0x7f32e80c6f30 2024-09-13T18:32:55.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.312+0000 7f32ecf6f700 1 -- 172.21.15.146:0/1925899613 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e814be90 con 0x7f32e80c6f30 2024-09-13T18:32:55.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.316+0000 7f32ecf6f700 1 -- 172.21.15.146:0/1925899613 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e814c080 con 0x7f32e80c6f30 2024-09-13T18:32:55.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.316+0000 7f32e4ff9700 1 -- 172.21.15.146:0/1925899613 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f32e0004030 con 0x7f32e80c6f30 2024-09-13T18:32:55.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.316+0000 7f32e4ff9700 1 --2- 172.21.15.146:0/1925899613 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f32c8066730 0x7f32c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:55.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.316+0000 7f32e4ff9700 1 -- 172.21.15.146:0/1925899613 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f32e0084770 con 0x7f32e80c6f30 2024-09-13T18:32:55.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.316+0000 7f32e77fe700 1 --2- 172.21.15.146:0/1925899613 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f32c8066730 0x7f32c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:55.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.316+0000 7f32e77fe700 1 --2- 172.21.15.146:0/1925899613 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f32c8066730 0x7f32c8068bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32d80099e0 tx=0x7f32d8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:55.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.324+0000 7f32e4ff9700 1 -- 172.21.15.146:0/1925899613 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f32e00514f0 con 0x7f32e80c6f30 2024-09-13T18:32:55.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.532+0000 7f32ecf6f700 1 -- 172.21.15.146:0/1925899613 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32e80c6330 con 0x7f32e80c6f30 2024-09-13T18:32:55.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.532+0000 7f32e4ff9700 1 -- 172.21.15.146:0/1925899613 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6288 (secure 0 0 0) 0x7f32e0054b10 con 0x7f32e80c6f30 2024-09-13T18:32:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:32:55.542 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:30:51.700582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4290,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.146:6837/31041557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":31041557},{"type":"v1","addr":"172.21.15.146:6837","nonce":31041557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:32:55.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.536+0000 7f32ecf6f700 1 -- 172.21.15.146:0/1925899613 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f32c8066730 msgr2=0x7f32c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:55.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.536+0000 7f32ecf6f700 1 --2- 172.21.15.146:0/1925899613 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f32c8066730 0x7f32c8068bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32d80099e0 tx=0x7f32d8008040 comp rx=0 tx=0).stop 2024-09-13T18:32:55.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.536+0000 7f32ecf6f700 1 -- 172.21.15.146:0/1925899613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e80c6f30 msgr2=0x7f32e80c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:55.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.536+0000 7f32ecf6f700 1 --2- 172.21.15.146:0/1925899613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e80c6f30 0x7f32e80c8840 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f32e0007e10 tx=0x7f32e000d3b0 comp rx=0 tx=0).stop 2024-09-13T18:32:55.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.536+0000 7f32ecf6f700 1 -- 172.21.15.146:0/1925899613 shutdown_connections 2024-09-13T18:32:55.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.536+0000 7f32ecf6f700 1 --2- 172.21.15.146:0/1925899613 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f32c8066730 0x7f32c8068bf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:55.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.536+0000 7f32ecf6f700 1 --2- 172.21.15.146:0/1925899613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e80c6f30 0x7f32e80c8840 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:55.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.536+0000 7f32ecf6f700 1 --2- 172.21.15.146:0/1925899613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e8153a60 0x7f32e80c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:55.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.536+0000 7f32ecf6f700 1 --2- 172.21.15.146:0/1925899613 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e80cf540 0x7f32e81efa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:55.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.536+0000 7f32ecf6f700 1 -- 172.21.15.146:0/1925899613 >> 172.21.15.146:0/1925899613 conn(0x7f32e8071ec0 msgr2=0x7f32e80cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:55.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.536+0000 7f32ecf6f700 1 -- 172.21.15.146:0/1925899613 shutdown_connections 2024-09-13T18:32:55.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:55.536+0000 7f32ecf6f700 1 -- 172.21.15.146:0/1925899613 wait complete. 2024-09-13T18:32:55.546 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 33 2024-09-13T18:32:55.562 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:32:57.564 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:32:57.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6f51b700 1 Processor -- start 2024-09-13T18:32:57.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6f51b700 1 -- start start 2024-09-13T18:32:57.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6f51b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb68153a60 0x7fcb68159700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6f51b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcb680c9ed0 0x7fcb680b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6f51b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcb680cf540 0x7fcb680cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6f51b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb680cc2d0 con 0x7fcb680c9ed0 2024-09-13T18:32:57.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6f51b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb68153e40 con 0x7fcb680cf540 2024-09-13T18:32:57.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6f51b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680cb3e0 con 0x7fcb68153a60 2024-09-13T18:32:57.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6dd18700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcb680c9ed0 0x7fcb680b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6dd18700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcb680c9ed0 0x7fcb680b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39782/0 (socket says 172.21.15.146:39782) 2024-09-13T18:32:57.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6dd18700 1 -- 172.21.15.146:0/1085720038 learned_addr learned my addr 172.21.15.146:0/1085720038 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:57.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6ed1a700 1 --2- 172.21.15.146:0/1085720038 >> v2:172.21.15.154:3300/0 conn(0x7fcb680cf540 0x7fcb680cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6e519700 1 --2- 172.21.15.146:0/1085720038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb68153a60 0x7fcb68159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6dd18700 1 -- 172.21.15.146:0/1085720038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb68153a60 msgr2=0x7fcb68159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:57.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6dd18700 1 --2- 172.21.15.146:0/1085720038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb68153a60 0x7fcb68159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6dd18700 1 -- 172.21.15.146:0/1085720038 >> v2:172.21.15.154:3300/0 conn(0x7fcb680cf540 msgr2=0x7fcb680cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:57.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6dd18700 1 --2- 172.21.15.146:0/1085720038 >> v2:172.21.15.154:3300/0 conn(0x7fcb680cf540 0x7fcb680cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6dd18700 1 -- 172.21.15.146:0/1085720038 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb680cb6c0 con 0x7fcb680c9ed0 2024-09-13T18:32:57.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6ed1a700 1 --2- 172.21.15.146:0/1085720038 >> v2:172.21.15.154:3300/0 conn(0x7fcb680cf540 0x7fcb680cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:32:57.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6dd18700 1 --2- 172.21.15.146:0/1085720038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb680c9ed0 0x7fcb680b3280 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcb680b31a0 tx=0x7fcb5c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9365fa05cc1edb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:57.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6d517700 1 -- 172.21.15.146:0/1085720038 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c011830 con 0x7fcb680c9ed0 2024-09-13T18:32:57.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6d517700 1 -- 172.21.15.146:0/1085720038 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c011990 con 0x7fcb680c9ed0 2024-09-13T18:32:57.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.676+0000 7fcb6d517700 1 -- 172.21.15.146:0/1085720038 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c011d20 con 0x7fcb680c9ed0 2024-09-13T18:32:57.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.680+0000 7fcb6f51b700 1 -- 172.21.15.146:0/1085720038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb680c9ed0 msgr2=0x7fcb680b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:57.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.680+0000 7fcb6f51b700 1 --2- 172.21.15.146:0/1085720038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb680c9ed0 0x7fcb680b3280 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcb680b31a0 tx=0x7fcb5c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.680+0000 7fcb6f51b700 1 -- 172.21.15.146:0/1085720038 shutdown_connections 2024-09-13T18:32:57.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.680+0000 7fcb6f51b700 1 --2- 172.21.15.146:0/1085720038 >> v2:172.21.15.154:3300/0 conn(0x7fcb680cf540 0x7fcb680cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.680+0000 7fcb6f51b700 1 --2- 172.21.15.146:0/1085720038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb680c9ed0 0x7fcb680b3280 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.680+0000 7fcb6f51b700 1 --2- 172.21.15.146:0/1085720038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb68153a60 0x7fcb68159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.680+0000 7fcb6f51b700 1 -- 172.21.15.146:0/1085720038 >> 172.21.15.146:0/1085720038 conn(0x7fcb68071ec0 msgr2=0x7fcb680722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:57.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.680+0000 7fcb6f51b700 1 -- 172.21.15.146:0/1085720038 shutdown_connections 2024-09-13T18:32:57.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.680+0000 7fcb6f51b700 1 -- 172.21.15.146:0/1085720038 wait complete. 2024-09-13T18:32:57.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.680+0000 7fcb6f51b700 1 Processor -- start 2024-09-13T18:32:57.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.680+0000 7fcb6f51b700 1 -- start start 2024-09-13T18:32:57.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6f51b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb680cf540 0x7fcb681e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6f51b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb68153a60 0x7fcb681e35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6f51b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb681e4210 0x7fcb681e46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6f51b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb68151a30 con 0x7fcb68153a60 2024-09-13T18:32:57.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6f51b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb681e4c00 con 0x7fcb680cf540 2024-09-13T18:32:57.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6f51b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6814f2c0 con 0x7fcb681e4210 2024-09-13T18:32:57.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6dd18700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb68153a60 0x7fcb681e35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6dd18700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb68153a60 0x7fcb681e35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39796/0 (socket says 172.21.15.146:39796) 2024-09-13T18:32:57.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6dd18700 1 -- 172.21.15.146:0/700392822 learned_addr learned my addr 172.21.15.146:0/700392822 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:57.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6e519700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb680cf540 0x7fcb681e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6ed1a700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb681e4210 0x7fcb681e46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6dd18700 1 -- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb681e4210 msgr2=0x7fcb681e46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:57.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6dd18700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb681e4210 0x7fcb681e46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6dd18700 1 -- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb680cf540 msgr2=0x7fcb681e65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:57.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6dd18700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb680cf540 0x7fcb681e65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6dd18700 1 -- 172.21.15.146:0/700392822 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb60009a70 con 0x7fcb68153a60 2024-09-13T18:32:57.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6dd18700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb68153a60 0x7fcb681e35e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcb68072a50 tx=0x7fcb5c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:57.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb5affd700 1 -- 172.21.15.146:0/700392822 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c004240 con 0x7fcb68153a60 2024-09-13T18:32:57.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb5affd700 1 -- 172.21.15.146:0/700392822 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c01f070 con 0x7fcb68153a60 2024-09-13T18:32:57.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6f51b700 1 -- 172.21.15.146:0/700392822 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c009570 con 0x7fcb68153a60 2024-09-13T18:32:57.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb5affd700 1 -- 172.21.15.146:0/700392822 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c004600 con 0x7fcb68153a60 2024-09-13T18:32:57.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6f51b700 1 -- 172.21.15.146:0/700392822 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb68151470 con 0x7fcb68153a60 2024-09-13T18:32:57.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.684+0000 7fcb6f51b700 1 -- 172.21.15.146:0/700392822 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb68073700 con 0x7fcb68153a60 2024-09-13T18:32:57.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.688+0000 7fcb5affd700 1 -- 172.21.15.146:0/700392822 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcb5c021030 con 0x7fcb68153a60 2024-09-13T18:32:57.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.688+0000 7fcb5affd700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcb4c066760 0x7fcb4c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.688+0000 7fcb5affd700 1 -- 172.21.15.146:0/700392822 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7fcb5c0854a0 con 0x7fcb68153a60 2024-09-13T18:32:57.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.688+0000 7fcb6e519700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcb4c066760 0x7fcb4c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.688+0000 7fcb6e519700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcb4c066760 0x7fcb4c068c20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009b50 tx=0x7fcb64009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:57.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.692+0000 7fcb5affd700 1 -- 172.21.15.146:0/700392822 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcb5c052450 con 0x7fcb68153a60 2024-09-13T18:32:57.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.900+0000 7fcb6f51b700 1 -- 172.21.15.146:0/700392822 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb68073890 con 0x7fcb68153a60 2024-09-13T18:32:57.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.904+0000 7fcb5affd700 1 -- 172.21.15.146:0/700392822 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5535 (secure 0 0 0) 0x7fcb5c05ae60 con 0x7fcb68153a60 2024-09-13T18:32:57.912 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:32:57.912 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:32:55.628409+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.904+0000 7fcb6f51b700 1 -- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcb4c066760 msgr2=0x7fcb4c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.904+0000 7fcb6f51b700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcb4c066760 0x7fcb4c068c20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009b50 tx=0x7fcb64009240 comp rx=0 tx=0).stop 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.904+0000 7fcb6f51b700 1 -- 172.21.15.146:0/700392822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb68153a60 msgr2=0x7fcb681e35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.904+0000 7fcb6f51b700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb68153a60 0x7fcb681e35e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcb68072a50 tx=0x7fcb5c004770 comp rx=0 tx=0).stop 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.904+0000 7fcb6f51b700 1 -- 172.21.15.146:0/700392822 shutdown_connections 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.904+0000 7fcb6f51b700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcb4c066760 0x7fcb4c068c20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.904+0000 7fcb6f51b700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb681e4210 0x7fcb681e46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.904+0000 7fcb6f51b700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb68153a60 0x7fcb681e35e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.904+0000 7fcb6f51b700 1 --2- 172.21.15.146:0/700392822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb680cf540 0x7fcb681e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.904+0000 7fcb6f51b700 1 -- 172.21.15.146:0/700392822 >> 172.21.15.146:0/700392822 conn(0x7fcb68071ec0 msgr2=0x7fcb680b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:57.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.904+0000 7fcb6f51b700 1 -- 172.21.15.146:0/700392822 shutdown_connections 2024-09-13T18:32:57.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:57.904+0000 7fcb6f51b700 1 -- 172.21.15.146:0/700392822 wait complete. 2024-09-13T18:32:57.916 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 34 2024-09-13T18:32:57.933 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-09-13T18:32:57.933 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:32:57.933 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:32:58.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.040+0000 7f044ffff700 1 Processor -- start 2024-09-13T18:32:58.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.040+0000 7f044ffff700 1 -- start start 2024-09-13T18:32:58.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.040+0000 7f044ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f045015cbe0 0x7f045015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.040+0000 7f044ffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04500ca9f0 con 0x7f04500c7ae0 2024-09-13T18:32:58.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.040+0000 7f044ffff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f045015d500 con 0x7f0450153530 2024-09-13T18:32:58.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.040+0000 7f044ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04500c9bb0 con 0x7f045015cbe0 2024-09-13T18:32:58.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.040+0000 7f0447fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f04500c7ae0 0x7f04500cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:47168/0 (socket says 172.21.15.146:47168) 2024-09-13T18:32:58.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.040+0000 7f0447fff700 1 -- 172.21.15.146:0/734585265 learned_addr learned my addr 172.21.15.146:0/734585265 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:32:58.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044effd700 1 --2- 172.21.15.146:0/734585265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f045015cbe0 0x7f045015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044e7fc700 1 -- 172.21.15.146:0/734585265 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402464683 0 0) 0x7f04500ca9f0 con 0x7f04500c7ae0 2024-09-13T18:32:58.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044e7fc700 1 -- 172.21.15.146:0/734585265 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f043c0035d0 con 0x7f04500c7ae0 2024-09-13T18:32:58.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044e7fc700 1 -- 172.21.15.146:0/734585265 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148078384 0 0) 0x7f045015d500 con 0x7f0450153530 2024-09-13T18:32:58.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044e7fc700 1 -- 172.21.15.146:0/734585265 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04500ca9f0 con 0x7f0450153530 2024-09-13T18:32:58.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044effd700 1 -- 172.21.15.146:0/734585265 >> v1:172.21.15.154:6789/0 conn(0x7f0450153530 legacy=0x7f04500c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044effd700 1 -- 172.21.15.146:0/734585265 >> v1:172.21.15.146:6789/0 conn(0x7f04500c7ae0 legacy=0x7f04500cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044effd700 1 -- 172.21.15.146:0/734585265 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04500c9e70 con 0x7f045015cbe0 2024-09-13T18:32:58.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044effd700 1 --2- 172.21.15.146:0/734585265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f045015cbe0 0x7f045015cfc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0440009880 tx=0x7f0440009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=628d809465b4941d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044e7fc700 1 -- 172.21.15.146:0/734585265 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440011830 con 0x7f045015cbe0 2024-09-13T18:32:58.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044e7fc700 1 -- 172.21.15.146:0/734585265 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0440011990 con 0x7f045015cbe0 2024-09-13T18:32:58.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044e7fc700 1 -- 172.21.15.146:0/734585265 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440011d20 con 0x7f045015cbe0 2024-09-13T18:32:58.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044ffff700 1 -- 172.21.15.146:0/734585265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f045015cbe0 msgr2=0x7f045015cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044ffff700 1 --2- 172.21.15.146:0/734585265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f045015cbe0 0x7f045015cfc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0440009880 tx=0x7f0440009b90 comp rx=0 tx=0).stop 2024-09-13T18:32:58.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044ffff700 1 -- 172.21.15.146:0/734585265 shutdown_connections 2024-09-13T18:32:58.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044ffff700 1 --2- 172.21.15.146:0/734585265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f045015cbe0 0x7f045015cfc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044ffff700 1 -- 172.21.15.146:0/734585265 >> 172.21.15.146:0/734585265 conn(0x7f0450066090 msgr2=0x7f04500664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044ffff700 1 -- 172.21.15.146:0/734585265 shutdown_connections 2024-09-13T18:32:58.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.044+0000 7f044ffff700 1 -- 172.21.15.146:0/734585265 wait complete. 2024-09-13T18:32:58.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044ffff700 1 Processor -- start 2024-09-13T18:32:58.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044ffff700 1 -- start start 2024-09-13T18:32:58.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04500c7ae0 0x7f04501e44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0450153530 0x7f04500cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f045015cbe0 0x7f04501e2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04500cb4d0 con 0x7f04500c7ae0 2024-09-13T18:32:58.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044ffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04500cc050 con 0x7f0450153530 2024-09-13T18:32:58.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04501f2ef0 con 0x7f045015cbe0 2024-09-13T18:32:58.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04500c7ae0 0x7f04501e44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04500c7ae0 0x7f04501e44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39800/0 (socket says 172.21.15.146:39800) 2024-09-13T18:32:58.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044effd700 1 -- 172.21.15.146:0/2853658354 learned_addr learned my addr 172.21.15.146:0/2853658354 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:32:58.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f0447fff700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0450153530 0x7f04500cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044f7fe700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f045015cbe0 0x7f04501e2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044effd700 1 -- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f045015cbe0 msgr2=0x7f04501e2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044effd700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f045015cbe0 0x7f04501e2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044effd700 1 -- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0450153530 msgr2=0x7f04500cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044effd700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0450153530 0x7f04500cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044effd700 1 -- 172.21.15.146:0/2853658354 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04400094b0 con 0x7f04500c7ae0 2024-09-13T18:32:58.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044f7fe700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f045015cbe0 0x7f04501e2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:32:58.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044effd700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04500c7ae0 0x7f04501e44b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f04400090b0 tx=0x7f0440004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044cff9700 1 -- 172.21.15.146:0/2853658354 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044001a8c0 con 0x7f04500c7ae0 2024-09-13T18:32:58.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044cff9700 1 -- 172.21.15.146:0/2853658354 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044001e070 con 0x7f04500c7ae0 2024-09-13T18:32:58.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044ffff700 1 -- 172.21.15.146:0/2853658354 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04500cb720 con 0x7f04500c7ae0 2024-09-13T18:32:58.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044cff9700 1 -- 172.21.15.146:0/2853658354 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044001ac40 con 0x7f04500c7ae0 2024-09-13T18:32:58.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.048+0000 7f044ffff700 1 -- 172.21.15.146:0/2853658354 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04500cbce0 con 0x7f04500c7ae0 2024-09-13T18:32:58.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.052+0000 7f044ffff700 1 -- 172.21.15.146:0/2853658354 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04501539e0 con 0x7f04500c7ae0 2024-09-13T18:32:58.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.052+0000 7f044cff9700 1 -- 172.21.15.146:0/2853658354 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f044002d450 con 0x7f04500c7ae0 2024-09-13T18:32:58.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.052+0000 7f044cff9700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0430066730 0x7f0430068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.052+0000 7f044cff9700 1 -- 172.21.15.146:0/2853658354 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f0440085650 con 0x7f04500c7ae0 2024-09-13T18:32:58.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.052+0000 7f0447fff700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0430066730 0x7f0430068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.052+0000 7f0447fff700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0430066730 0x7f0430068bf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f045015c8f0 tx=0x7f0438009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.056+0000 7f044cff9700 1 -- 172.21.15.146:0/2853658354 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f04400524a0 con 0x7f04500c7ae0 2024-09-13T18:32:58.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.264+0000 7f044ffff700 1 -- 172.21.15.146:0/2853658354 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04500c3620 con 0x7f04500c7ae0 2024-09-13T18:32:58.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.264+0000 7f044cff9700 1 -- 172.21.15.146:0/2853658354 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5535 (secure 0 0 0) 0x7f0440055ac0 con 0x7f04500c7ae0 2024-09-13T18:32:58.275 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:32:58.275 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:32:55.628409+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:32:58.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.268+0000 7f044ffff700 1 -- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0430066730 msgr2=0x7f0430068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.268+0000 7f044ffff700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0430066730 0x7f0430068bf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f045015c8f0 tx=0x7f0438009040 comp rx=0 tx=0).stop 2024-09-13T18:32:58.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.268+0000 7f044ffff700 1 -- 172.21.15.146:0/2853658354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04500c7ae0 msgr2=0x7f04501e44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.268+0000 7f044ffff700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04500c7ae0 0x7f04501e44b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f04400090b0 tx=0x7f0440004770 comp rx=0 tx=0).stop 2024-09-13T18:32:58.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.268+0000 7f044ffff700 1 -- 172.21.15.146:0/2853658354 shutdown_connections 2024-09-13T18:32:58.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.268+0000 7f044ffff700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0430066730 0x7f0430068bf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.268+0000 7f044ffff700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f045015cbe0 0x7f04501e2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.268+0000 7f044ffff700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0450153530 0x7f04500cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.268+0000 7f044ffff700 1 --2- 172.21.15.146:0/2853658354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04500c7ae0 0x7f04501e44b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.268+0000 7f044ffff700 1 -- 172.21.15.146:0/2853658354 >> 172.21.15.146:0/2853658354 conn(0x7f0450066090 msgr2=0x7f04500c27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.268+0000 7f044ffff700 1 -- 172.21.15.146:0/2853658354 shutdown_connections 2024-09-13T18:32:58.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:32:58.268+0000 7f044ffff700 1 -- 172.21.15.146:0/2853658354 wait complete. 2024-09-13T18:32:58.279 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 34 2024-09-13T18:32:58.291 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 34, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 34, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5235": { "addr": "172.21.15.146:6833/1498985282", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 1498985282, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 1498985282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5235, "incarnation": 22, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 19 }, "gid_5278": { "addr": "172.21.15.154:6839/3274220802", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 3274220802, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 3274220802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5278, "incarnation": 28, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 19 }, "gid_5505": { "addr": "172.21.15.146:6835/1652676664", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 1652676664, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 1652676664, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5505, "incarnation": 34, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 31, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:32:55.628409+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5505, "mds_1": 5278, "mds_2": 5235 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:33:00.292 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:33:00.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc704c01700 1 Processor -- start 2024-09-13T18:33:00.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc704c01700 1 -- start start 2024-09-13T18:33:00.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc704c01700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc700070570 0x7fc7000c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:00.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc704c01700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7000adfe0 con 0x7fc7000c5bd0 2024-09-13T18:33:00.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc704c01700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7000c7570 con 0x7fc700157e10 2024-09-13T18:33:00.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc704c01700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70015e690 con 0x7fc700070570 2024-09-13T18:33:00.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fed9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc7000c5bd0 0x7fc7000c6060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:47170/0 (socket says 172.21.15.146:47170) 2024-09-13T18:33:00.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fed9d700 1 -- 172.21.15.146:0/4026183153 learned_addr learned my addr 172.21.15.146:0/4026183153 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:33:00.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fdd9b700 1 --2- 172.21.15.146:0/4026183153 >> v2:172.21.15.154:3300/0 conn(0x7fc700070570 0x7fc7000c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:00.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fd59a700 1 -- 172.21.15.146:0/4026183153 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281146116 0 0) 0x7fc7000adfe0 con 0x7fc7000c5bd0 2024-09-13T18:33:00.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fd59a700 1 -- 172.21.15.146:0/4026183153 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6e80035d0 con 0x7fc7000c5bd0 2024-09-13T18:33:00.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fd59a700 1 -- 172.21.15.146:0/4026183153 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115420685 0 0) 0x7fc7000c7570 con 0x7fc700157e10 2024-09-13T18:33:00.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fd59a700 1 -- 172.21.15.146:0/4026183153 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7000adfe0 con 0x7fc700157e10 2024-09-13T18:33:00.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fdd9b700 1 -- 172.21.15.146:0/4026183153 >> v1:172.21.15.154:6789/0 conn(0x7fc700157e10 legacy=0x7fc70006fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:00.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fdd9b700 1 -- 172.21.15.146:0/4026183153 >> v1:172.21.15.146:6789/0 conn(0x7fc7000c5bd0 legacy=0x7fc7000c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:00.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fdd9b700 1 -- 172.21.15.146:0/4026183153 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70015eb80 con 0x7fc700070570 2024-09-13T18:33:00.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fdd9b700 1 --2- 172.21.15.146:0/4026183153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700070570 0x7fc7000c6df0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009d90 tx=0x7fc6f800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba27572ff122704 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:00.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fd59a700 1 -- 172.21.15.146:0/4026183153 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f800e070 con 0x7fc700070570 2024-09-13T18:33:00.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fd59a700 1 -- 172.21.15.146:0/4026183153 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f800bb20 con 0x7fc700070570 2024-09-13T18:33:00.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.404+0000 7fc6fd59a700 1 -- 172.21.15.146:0/4026183153 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f800be90 con 0x7fc700070570 2024-09-13T18:33:00.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.408+0000 7fc704c01700 1 -- 172.21.15.146:0/4026183153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700070570 msgr2=0x7fc7000c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:00.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.408+0000 7fc704c01700 1 --2- 172.21.15.146:0/4026183153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700070570 0x7fc7000c6df0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009d90 tx=0x7fc6f800b330 comp rx=0 tx=0).stop 2024-09-13T18:33:00.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.408+0000 7fc704c01700 1 -- 172.21.15.146:0/4026183153 shutdown_connections 2024-09-13T18:33:00.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.408+0000 7fc704c01700 1 --2- 172.21.15.146:0/4026183153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700070570 0x7fc7000c6df0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.408+0000 7fc704c01700 1 -- 172.21.15.146:0/4026183153 >> 172.21.15.146:0/4026183153 conn(0x7fc7000cb1d0 msgr2=0x7fc7000cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:00.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.408+0000 7fc704c01700 1 -- 172.21.15.146:0/4026183153 shutdown_connections 2024-09-13T18:33:00.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.408+0000 7fc704c01700 1 -- 172.21.15.146:0/4026183153 wait complete. 2024-09-13T18:33:00.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.408+0000 7fc704c01700 1 Processor -- start 2024-09-13T18:33:00.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc704c01700 1 -- start start 2024-09-13T18:33:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc704c01700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700070570 0x7fc7000ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc704c01700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7000c5bd0 0x7fc7000c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc704c01700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc700157e10 0x7fc7000af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc704c01700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7000aed50 con 0x7fc700157e10 2024-09-13T18:33:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc704c01700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc700160f30 con 0x7fc700070570 2024-09-13T18:33:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc704c01700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7001611f0 con 0x7fc7000c5bd0 2024-09-13T18:33:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6fed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc700157e10 0x7fc7000af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6fed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc700157e10 0x7fc7000af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39816/0 (socket says 172.21.15.146:39816) 2024-09-13T18:33:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6fed9d700 1 -- 172.21.15.146:0/2635222750 learned_addr learned my addr 172.21.15.146:0/2635222750 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6fdd9b700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7000c5bd0 0x7fc7000c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6fe59c700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700070570 0x7fc7000ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:00.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6fed9d700 1 -- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7000c5bd0 msgr2=0x7fc7000c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:00.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6fed9d700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7000c5bd0 0x7fc7000c3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6fed9d700 1 -- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700070570 msgr2=0x7fc7000ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:00.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6fed9d700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700070570 0x7fc7000ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6fed9d700 1 -- 172.21.15.146:0/2635222750 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f8009a70 con 0x7fc700157e10 2024-09-13T18:33:00.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6fed9d700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc700157e10 0x7fc7000af690 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000c4d0 tx=0x7fc6f000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:00.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6eeffd700 1 -- 172.21.15.146:0/2635222750 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0005d90 con 0x7fc700157e10 2024-09-13T18:33:00.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6eeffd700 1 -- 172.21.15.146:0/2635222750 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f000ce80 con 0x7fc700157e10 2024-09-13T18:33:00.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc6eeffd700 1 -- 172.21.15.146:0/2635222750 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f00072f0 con 0x7fc700157e10 2024-09-13T18:33:00.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc704c01700 1 -- 172.21.15.146:0/2635222750 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7001614d0 con 0x7fc700157e10 2024-09-13T18:33:00.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.412+0000 7fc704c01700 1 -- 172.21.15.146:0/2635222750 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc700161a90 con 0x7fc700157e10 2024-09-13T18:33:00.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.416+0000 7fc704c01700 1 -- 172.21.15.146:0/2635222750 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7001581f0 con 0x7fc700157e10 2024-09-13T18:33:00.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.416+0000 7fc6eeffd700 1 -- 172.21.15.146:0/2635222750 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc6f0007450 con 0x7fc700157e10 2024-09-13T18:33:00.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.416+0000 7fc6eeffd700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc6e0066730 0x7fc6e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:00.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.416+0000 7fc6eeffd700 1 -- 172.21.15.146:0/2635222750 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7fc6f0083a60 con 0x7fc700157e10 2024-09-13T18:33:00.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.416+0000 7fc6fe59c700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc6e0066730 0x7fc6e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:00.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.420+0000 7fc6fe59c700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc6e0066730 0x7fc6e0068bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40017b0 tx=0x7fc6f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.420+0000 7fc6eeffd700 1 -- 172.21.15.146:0/2635222750 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc6f0050830 con 0x7fc700157e10 2024-09-13T18:33:00.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.628+0000 7fc704c01700 1 -- 172.21.15.146:0/2635222750 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7000c1f70 con 0x7fc700157e10 2024-09-13T18:33:00.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.628+0000 7fc6eeffd700 1 -- 172.21.15.146:0/2635222750 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5535 (secure 0 0 0) 0x7fc6f0017090 con 0x7fc700157e10 2024-09-13T18:33:00.640 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:33:00.640 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:32:55.628409+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:33:00.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.632+0000 7fc704c01700 1 -- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc6e0066730 msgr2=0x7fc6e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:00.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.632+0000 7fc704c01700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc6e0066730 0x7fc6e0068bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40017b0 tx=0x7fc6f4009040 comp rx=0 tx=0).stop 2024-09-13T18:33:00.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.632+0000 7fc704c01700 1 -- 172.21.15.146:0/2635222750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc700157e10 msgr2=0x7fc7000af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:00.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.632+0000 7fc704c01700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc700157e10 0x7fc7000af690 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000c4d0 tx=0x7fc6f000c7e0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.632+0000 7fc704c01700 1 -- 172.21.15.146:0/2635222750 shutdown_connections 2024-09-13T18:33:00.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.632+0000 7fc704c01700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc6e0066730 0x7fc6e0068bf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.632+0000 7fc704c01700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc700157e10 0x7fc7000af690 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.632+0000 7fc704c01700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7000c5bd0 0x7fc7000c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.632+0000 7fc704c01700 1 --2- 172.21.15.146:0/2635222750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700070570 0x7fc7000ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.632+0000 7fc704c01700 1 -- 172.21.15.146:0/2635222750 >> 172.21.15.146:0/2635222750 conn(0x7fc7000cb1d0 msgr2=0x7fc7000c9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:00.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.632+0000 7fc704c01700 1 -- 172.21.15.146:0/2635222750 shutdown_connections 2024-09-13T18:33:00.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:00.632+0000 7fc704c01700 1 -- 172.21.15.146:0/2635222750 wait complete. 2024-09-13T18:33:00.644 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 34 2024-09-13T18:33:00.656 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:33:02.657 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:33:02.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.771+0000 7f3f37fff700 1 Processor -- start 2024-09-13T18:33:02.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.771+0000 7f3f37fff700 1 -- start start 2024-09-13T18:33:02.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f37fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3f38156e30 0x7f3f380c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:02.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f37fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3f3806cef0 0x7f3f380c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:02.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f37fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f38159b10 con 0x7f3f381505d0 2024-09-13T18:33:02.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f37fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f381573c0 con 0x7f3f3806cef0 2024-09-13T18:33:02.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f37fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f380c2ff0 con 0x7f3f38156e30 2024-09-13T18:33:02.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f377fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3f3806cef0 0x7f3f380c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:02.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f377fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3f3806cef0 0x7f3f380c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50536/0 (socket says 172.21.15.146:50536) 2024-09-13T18:33:02.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f377fe700 1 -- 172.21.15.146:0/393836742 learned_addr learned my addr 172.21.15.146:0/393836742 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:02.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f367fc700 1 --2- 172.21.15.146:0/393836742 >> v2:172.21.15.154:3300/0 conn(0x7f3f38156e30 0x7f3f380c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:02.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f377fe700 1 -- 172.21.15.146:0/393836742 >> v2:172.21.15.154:3300/0 conn(0x7f3f38156e30 msgr2=0x7f3f380c2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:02.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f377fe700 1 --2- 172.21.15.146:0/393836742 >> v2:172.21.15.154:3300/0 conn(0x7f3f38156e30 0x7f3f380c2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f377fe700 1 -- 172.21.15.146:0/393836742 >> v1:172.21.15.154:6789/0 conn(0x7f3f381505d0 legacy=0x7f3f3806d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:02.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f377fe700 1 -- 172.21.15.146:0/393836742 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f38159db0 con 0x7f3f3806cef0 2024-09-13T18:33:02.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f367fc700 1 --2- 172.21.15.146:0/393836742 >> v2:172.21.15.154:3300/0 conn(0x7f3f38156e30 0x7f3f380c2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:33:02.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f377fe700 1 --2- 172.21.15.146:0/393836742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f3806cef0 0x7f3f380c45f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3f380c35e0 tx=0x7f3f3000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f681fcfbc6b773 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:02.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f35ffb700 1 -- 172.21.15.146:0/393836742 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30010070 con 0x7f3f3806cef0 2024-09-13T18:33:02.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f35ffb700 1 -- 172.21.15.146:0/393836742 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3000dd80 con 0x7f3f3806cef0 2024-09-13T18:33:02.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f35ffb700 1 -- 172.21.15.146:0/393836742 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3000b310 con 0x7f3f3806cef0 2024-09-13T18:33:02.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f37fff700 1 -- 172.21.15.146:0/393836742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f3806cef0 msgr2=0x7f3f380c45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:02.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f37fff700 1 --2- 172.21.15.146:0/393836742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f3806cef0 0x7f3f380c45f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3f380c35e0 tx=0x7f3f3000d460 comp rx=0 tx=0).stop 2024-09-13T18:33:02.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f37fff700 1 -- 172.21.15.146:0/393836742 shutdown_connections 2024-09-13T18:33:02.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f37fff700 1 --2- 172.21.15.146:0/393836742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f3806cef0 0x7f3f380c45f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f37fff700 1 --2- 172.21.15.146:0/393836742 >> v2:172.21.15.154:3300/0 conn(0x7f3f38156e30 0x7f3f380c2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f37fff700 1 -- 172.21.15.146:0/393836742 >> 172.21.15.146:0/393836742 conn(0x7f3f38072270 msgr2=0x7f3f3806ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:02.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f37fff700 1 -- 172.21.15.146:0/393836742 shutdown_connections 2024-09-13T18:33:02.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.775+0000 7f3f37fff700 1 -- 172.21.15.146:0/393836742 wait complete. 2024-09-13T18:33:02.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f37fff700 1 Processor -- start 2024-09-13T18:33:02.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f37fff700 1 -- start start 2024-09-13T18:33:02.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f37fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f3806cef0 0x7f3f381eb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:02.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f37fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f381505d0 0x7f3f3814f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:02.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f37fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f38156e30 0x7f3f381ee320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:02.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f37fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f38154650 con 0x7f3f38156e30 2024-09-13T18:33:02.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f37fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f380c5fe0 con 0x7f3f381505d0 2024-09-13T18:33:02.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f37fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3814cf10 con 0x7f3f3806cef0 2024-09-13T18:33:02.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f377fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f38156e30 0x7f3f381ee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:02.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f377fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f38156e30 0x7f3f381ee320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50540/0 (socket says 172.21.15.146:50540) 2024-09-13T18:33:02.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f377fe700 1 -- 172.21.15.146:0/2285644617 learned_addr learned my addr 172.21.15.146:0/2285644617 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:02.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f367fc700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f381505d0 0x7f3f3814f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:02.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f36ffd700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f3806cef0 0x7f3f381eb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:02.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f377fe700 1 -- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f3806cef0 msgr2=0x7f3f381eb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:02.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f377fe700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f3806cef0 0x7f3f381eb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f377fe700 1 -- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f381505d0 msgr2=0x7f3f3814f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:02.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f377fe700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f381505d0 0x7f3f3814f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f377fe700 1 -- 172.21.15.146:0/2285644617 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3000bd40 con 0x7f3f38156e30 2024-09-13T18:33:02.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f377fe700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f38156e30 0x7f3f381ee320 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000d860 tx=0x7f3f30004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:02.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f1f7fe700 1 -- 172.21.15.146:0/2285644617 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3000bbc0 con 0x7f3f38156e30 2024-09-13T18:33:02.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f1f7fe700 1 -- 172.21.15.146:0/2285644617 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3000b4c0 con 0x7f3f38156e30 2024-09-13T18:33:02.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f37fff700 1 -- 172.21.15.146:0/2285644617 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f381eaec0 con 0x7f3f38156e30 2024-09-13T18:33:02.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.779+0000 7f3f1f7fe700 1 -- 172.21.15.146:0/2285644617 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30004040 con 0x7f3f38156e30 2024-09-13T18:33:02.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.783+0000 7f3f37fff700 1 -- 172.21.15.146:0/2285644617 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f381eb480 con 0x7f3f38156e30 2024-09-13T18:33:02.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.783+0000 7f3f37fff700 1 -- 172.21.15.146:0/2285644617 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f381eb6a0 con 0x7f3f38156e30 2024-09-13T18:33:02.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.783+0000 7f3f1f7fe700 1 -- 172.21.15.146:0/2285644617 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3f300041a0 con 0x7f3f38156e30 2024-09-13T18:33:02.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.783+0000 7f3f1f7fe700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f24066730 0x7f3f24068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:02.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.783+0000 7f3f1f7fe700 1 -- 172.21.15.146:0/2285644617 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f3f30087e30 con 0x7f3f38156e30 2024-09-13T18:33:02.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.783+0000 7f3f36ffd700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f24066730 0x7f3f24068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:02.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.787+0000 7f3f36ffd700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f24066730 0x7f3f24068bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f28000f60 tx=0x7f3f28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:02.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.791+0000 7f3f1f7fe700 1 -- 172.21.15.146:0/2285644617 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3f30054c90 con 0x7f3f38156e30 2024-09-13T18:33:03.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.999+0000 7f3f37fff700 1 -- 172.21.15.146:0/2285644617 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f381509b0 con 0x7f3f38156e30 2024-09-13T18:33:03.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:02.999+0000 7f3f1f7fe700 1 -- 172.21.15.146:0/2285644617 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5537 (secure 0 0 0) 0x7f3f30022090 con 0x7f3f38156e30 2024-09-13T18:33:03.010 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:33:03.011 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:01.273367+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:resolve","state_seq":32,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:33:03.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:03.003+0000 7f3f37fff700 1 -- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f24066730 msgr2=0x7f3f24068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:03.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:03.003+0000 7f3f37fff700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f24066730 0x7f3f24068bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f28000f60 tx=0x7f3f28009040 comp rx=0 tx=0).stop 2024-09-13T18:33:03.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:03.003+0000 7f3f37fff700 1 -- 172.21.15.146:0/2285644617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f38156e30 msgr2=0x7f3f381ee320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:03.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:03.003+0000 7f3f37fff700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f38156e30 0x7f3f381ee320 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000d860 tx=0x7f3f30004af0 comp rx=0 tx=0).stop 2024-09-13T18:33:03.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:03.003+0000 7f3f37fff700 1 -- 172.21.15.146:0/2285644617 shutdown_connections 2024-09-13T18:33:03.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:03.003+0000 7f3f37fff700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f24066730 0x7f3f24068bf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:03.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:03.003+0000 7f3f37fff700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f38156e30 0x7f3f381ee320 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:03.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:03.003+0000 7f3f37fff700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f381505d0 0x7f3f3814f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:03.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:03.003+0000 7f3f37fff700 1 --2- 172.21.15.146:0/2285644617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f3806cef0 0x7f3f381eb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:03.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:03.003+0000 7f3f37fff700 1 -- 172.21.15.146:0/2285644617 >> 172.21.15.146:0/2285644617 conn(0x7f3f38072270 msgr2=0x7f3f3806d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:03.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:03.003+0000 7f3f37fff700 1 -- 172.21.15.146:0/2285644617 shutdown_connections 2024-09-13T18:33:03.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:03.003+0000 7f3f37fff700 1 -- 172.21.15.146:0/2285644617 wait complete. 2024-09-13T18:33:03.014 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 35 2024-09-13T18:33:03.026 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:33:05.027 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:33:05.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff901de7700 1 Processor -- start 2024-09-13T18:33:05.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff901de7700 1 -- start start 2024-09-13T18:33:05.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff901de7700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff8fc070570 0x7ff8fc0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff901de7700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc0adfe0 con 0x7ff8fc157e10 2024-09-13T18:33:05.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff901de7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc0c7570 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff901de7700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc15e690 con 0x7ff8fc070570 2024-09-13T18:33:05.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff8f2ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff8fc070570 0x7ff8fc0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff8f2ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff8fc070570 0x7ff8fc0c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50552/0 (socket says 172.21.15.146:50552) 2024-09-13T18:33:05.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff8f2ffd700 1 -- 172.21.15.146:0/3469183130 learned_addr learned my addr 172.21.15.146:0/3469183130 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:05.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff8fb7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff8fc157e10 0x7ff8fc06fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56152/0 (socket says 172.21.15.146:56152) 2024-09-13T18:33:05.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff8f2ffd700 1 -- 172.21.15.146:0/3469183130 >> v1:172.21.15.154:6789/0 conn(0x7ff8fc0c5bd0 legacy=0x7ff8fc0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:05.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff8f2ffd700 1 -- 172.21.15.146:0/3469183130 >> v1:172.21.15.146:6789/0 conn(0x7ff8fc157e10 legacy=0x7ff8fc06fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff8f2ffd700 1 -- 172.21.15.146:0/3469183130 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc15eb80 con 0x7ff8fc070570 2024-09-13T18:33:05.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff8f2ffd700 1 --2- 172.21.15.146:0/3469183130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc070570 0x7ff8fc0c6df0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009910 tx=0x7ff8e4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c837220a1664fda server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:05.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff8faffd700 1 -- 172.21.15.146:0/3469183130 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4004230 con 0x7ff8fc070570 2024-09-13T18:33:05.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff8faffd700 1 -- 172.21.15.146:0/3469183130 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e4004390 con 0x7ff8fc070570 2024-09-13T18:33:05.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff8faffd700 1 -- 172.21.15.146:0/3469183130 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4009e60 con 0x7ff8fc070570 2024-09-13T18:33:05.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff901de7700 1 -- 172.21.15.146:0/3469183130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc070570 msgr2=0x7ff8fc0c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff901de7700 1 --2- 172.21.15.146:0/3469183130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc070570 0x7ff8fc0c6df0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009910 tx=0x7ff8e4009c20 comp rx=0 tx=0).stop 2024-09-13T18:33:05.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff901de7700 1 -- 172.21.15.146:0/3469183130 shutdown_connections 2024-09-13T18:33:05.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff901de7700 1 --2- 172.21.15.146:0/3469183130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc070570 0x7ff8fc0c6df0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff901de7700 1 -- 172.21.15.146:0/3469183130 >> 172.21.15.146:0/3469183130 conn(0x7ff8fc0cb1d0 msgr2=0x7ff8fc0cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:05.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff901de7700 1 -- 172.21.15.146:0/3469183130 shutdown_connections 2024-09-13T18:33:05.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.139+0000 7ff901de7700 1 -- 172.21.15.146:0/3469183130 wait complete. 2024-09-13T18:33:05.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff901de7700 1 Processor -- start 2024-09-13T18:33:05.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff901de7700 1 -- start start 2024-09-13T18:33:05.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff901de7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc070570 0x7ff8fc0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff901de7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0c5bd0 0x7ff8fc15edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff901de7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc157e10 0x7ff8fc158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff901de7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c4a00 con 0x7ff8fc157e10 2024-09-13T18:33:05.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff901de7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c5720 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff901de7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c4e20 con 0x7ff8fc070570 2024-09-13T18:33:05.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8fbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc157e10 0x7ff8fc158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8f2ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0c5bd0 0x7ff8fc15edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8fb7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc070570 0x7ff8fc0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8f2ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0c5bd0 0x7ff8fc15edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:53124/0 (socket says 172.21.15.146:53124) 2024-09-13T18:33:05.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8f2ffd700 1 -- 172.21.15.146:0/3596611097 learned_addr learned my addr 172.21.15.146:0/3596611097 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:05.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8f2ffd700 1 -- 172.21.15.146:0/3596611097 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc070570 msgr2=0x7ff8fc0c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8f2ffd700 1 --2- 172.21.15.146:0/3596611097 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc070570 0x7ff8fc0c2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8f2ffd700 1 -- 172.21.15.146:0/3596611097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc157e10 msgr2=0x7ff8fc158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8f2ffd700 1 --2- 172.21.15.146:0/3596611097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc157e10 0x7ff8fc158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8f2ffd700 1 -- 172.21.15.146:0/3596611097 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e4009570 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8f2ffd700 1 --2- 172.21.15.146:0/3596611097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0c5bd0 0x7ff8fc15edd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4004000 tx=0x7ff8e4004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:05.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8f8ff9700 1 -- 172.21.15.146:0/3596611097 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e400be60 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8f8ff9700 1 -- 172.21.15.146:0/3596611097 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e40036c0 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff901de7700 1 -- 172.21.15.146:0/3596611097 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc15a840 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff8f8ff9700 1 -- 172.21.15.146:0/3596611097 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4003a30 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.143+0000 7ff901de7700 1 -- 172.21.15.146:0/3596611097 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc0c4340 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.147+0000 7ff901de7700 1 -- 172.21.15.146:0/3596611097 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc1581f0 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.147+0000 7ff8f8ff9700 1 -- 172.21.15.146:0/3596611097 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff8e4003b90 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.147+0000 7ff8f8ff9700 1 --2- 172.21.15.146:0/3596611097 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff8dc066730 0x7ff8dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.147+0000 7ff8f8ff9700 1 -- 172.21.15.146:0/3596611097 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7ff8e4086360 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.147+0000 7ff8fb7fe700 1 --2- 172.21.15.146:0/3596611097 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff8dc066730 0x7ff8dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.147+0000 7ff8fb7fe700 1 --2- 172.21.15.146:0/3596611097 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff8dc066730 0x7ff8dc068bf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec000f60 tx=0x7ff8ec007f30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.159+0000 7ff8f8ff9700 1 -- 172.21.15.146:0/3596611097 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff8e4053250 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.367+0000 7ff901de7700 1 -- 172.21.15.146:0/3596611097 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8fc1581f0 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.371+0000 7ff8f8ff9700 1 -- 172.21.15.146:0/3596611097 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5537 (secure 0 0 0) 0x7ff8e4056870 con 0x7ff8fc0c5bd0 2024-09-13T18:33:05.381 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:33:05.381 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:01.273367+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:resolve","state_seq":32,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:33:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.371+0000 7ff901de7700 1 -- 172.21.15.146:0/3596611097 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff8dc066730 msgr2=0x7ff8dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.371+0000 7ff901de7700 1 --2- 172.21.15.146:0/3596611097 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff8dc066730 0x7ff8dc068bf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec000f60 tx=0x7ff8ec007f30 comp rx=0 tx=0).stop 2024-09-13T18:33:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.371+0000 7ff901de7700 1 -- 172.21.15.146:0/3596611097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0c5bd0 msgr2=0x7ff8fc15edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.371+0000 7ff901de7700 1 --2- 172.21.15.146:0/3596611097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0c5bd0 0x7ff8fc15edd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4004000 tx=0x7ff8e4004770 comp rx=0 tx=0).stop 2024-09-13T18:33:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.371+0000 7ff901de7700 1 -- 172.21.15.146:0/3596611097 shutdown_connections 2024-09-13T18:33:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.371+0000 7ff901de7700 1 --2- 172.21.15.146:0/3596611097 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff8dc066730 0x7ff8dc068bf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.371+0000 7ff901de7700 1 --2- 172.21.15.146:0/3596611097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc157e10 0x7ff8fc158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.371+0000 7ff901de7700 1 --2- 172.21.15.146:0/3596611097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0c5bd0 0x7ff8fc15edd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.371+0000 7ff901de7700 1 --2- 172.21.15.146:0/3596611097 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc070570 0x7ff8fc0c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.371+0000 7ff901de7700 1 -- 172.21.15.146:0/3596611097 >> 172.21.15.146:0/3596611097 conn(0x7ff8fc0cb1d0 msgr2=0x7ff8fc1e9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.371+0000 7ff901de7700 1 -- 172.21.15.146:0/3596611097 shutdown_connections 2024-09-13T18:33:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:05.371+0000 7ff901de7700 1 -- 172.21.15.146:0/3596611097 wait complete. 2024-09-13T18:33:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 35 2024-09-13T18:33:05.397 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:33:07.398 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:33:07.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.507+0000 7f84f34fe700 1 Processor -- start 2024-09-13T18:33:07.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.507+0000 7f84f34fe700 1 -- start start 2024-09-13T18:33:07.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.507+0000 7f84f34fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f84ec151840 0x7f84ec0c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:07.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.507+0000 7f84f34fe700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ec0c9890 con 0x7f84ec0c8380 2024-09-13T18:33:07.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.507+0000 7f84f34fe700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ec0c9510 con 0x7f84ec157fc0 2024-09-13T18:33:07.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.507+0000 7f84f34fe700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84ec0c36f0 con 0x7f84ec151840 2024-09-13T18:33:07.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.507+0000 7f84f2cfd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f84ec0c8380 0x7f84ec161b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56154/0 (socket says 172.21.15.146:56154) 2024-09-13T18:33:07.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.507+0000 7f84f2cfd700 1 -- 172.21.15.146:0/2364532277 learned_addr learned my addr 172.21.15.146:0/2364532277 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:33:07.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f24fc700 1 --2- 172.21.15.146:0/2364532277 >> v2:172.21.15.154:3300/0 conn(0x7f84ec151840 0x7f84ec0c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:07.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f14fa700 1 -- 172.21.15.146:0/2364532277 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218774693 0 0) 0x7f84ec0c9890 con 0x7f84ec0c8380 2024-09-13T18:33:07.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f14fa700 1 -- 172.21.15.146:0/2364532277 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84d80035d0 con 0x7f84ec0c8380 2024-09-13T18:33:07.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f14fa700 1 -- 172.21.15.146:0/2364532277 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511007289 0 0) 0x7f84dc001110 con 0x7f84ec157fc0 2024-09-13T18:33:07.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f14fa700 1 -- 172.21.15.146:0/2364532277 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84ec0c9890 con 0x7f84ec157fc0 2024-09-13T18:33:07.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f24fc700 1 -- 172.21.15.146:0/2364532277 >> v1:172.21.15.154:6789/0 conn(0x7f84ec157fc0 legacy=0x7f84ec06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:07.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f24fc700 1 -- 172.21.15.146:0/2364532277 >> v1:172.21.15.146:6789/0 conn(0x7f84ec0c8380 legacy=0x7f84ec161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:07.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f24fc700 1 -- 172.21.15.146:0/2364532277 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec0c7490 con 0x7f84ec151840 2024-09-13T18:33:07.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f24fc700 1 --2- 172.21.15.146:0/2364532277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84ec151840 0x7f84ec0c1f90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009850 tx=0x7f84e8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ee91c48f8dd5286 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:07.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f14fa700 1 -- 172.21.15.146:0/2364532277 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8011830 con 0x7f84ec151840 2024-09-13T18:33:07.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f14fa700 1 -- 172.21.15.146:0/2364532277 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e8011990 con 0x7f84ec151840 2024-09-13T18:33:07.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f14fa700 1 -- 172.21.15.146:0/2364532277 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8011d00 con 0x7f84ec151840 2024-09-13T18:33:07.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f34fe700 1 -- 172.21.15.146:0/2364532277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84ec151840 msgr2=0x7f84ec0c1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:07.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f34fe700 1 --2- 172.21.15.146:0/2364532277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84ec151840 0x7f84ec0c1f90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009850 tx=0x7f84e8009b60 comp rx=0 tx=0).stop 2024-09-13T18:33:07.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f34fe700 1 -- 172.21.15.146:0/2364532277 shutdown_connections 2024-09-13T18:33:07.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f34fe700 1 --2- 172.21.15.146:0/2364532277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84ec151840 0x7f84ec0c1f90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:07.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f34fe700 1 -- 172.21.15.146:0/2364532277 >> 172.21.15.146:0/2364532277 conn(0x7f84ec0ccf70 msgr2=0x7f84ec0cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:07.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f34fe700 1 -- 172.21.15.146:0/2364532277 shutdown_connections 2024-09-13T18:33:07.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.511+0000 7f84f34fe700 1 -- 172.21.15.146:0/2364532277 wait complete. 2024-09-13T18:33:07.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f34fe700 1 Processor -- start 2024-09-13T18:33:07.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f34fe700 1 -- start start 2024-09-13T18:33:07.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f34fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84ec0c8380 0x7f84ec1526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:07.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f34fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84ec151840 0x7f84ec152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:07.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f34fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84ec157fc0 0x7f84ec1f0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:07.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f34fe700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec1f1610 con 0x7f84ec0c8380 2024-09-13T18:33:07.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f34fe700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec153360 con 0x7f84ec157fc0 2024-09-13T18:33:07.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f34fe700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec1e1740 con 0x7f84ec151840 2024-09-13T18:33:07.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f24fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84ec0c8380 0x7f84ec1526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:07.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f24fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84ec0c8380 0x7f84ec1526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50564/0 (socket says 172.21.15.146:50564) 2024-09-13T18:33:07.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f24fc700 1 -- 172.21.15.146:0/2986922914 learned_addr learned my addr 172.21.15.146:0/2986922914 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:07.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f1cfb700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84ec151840 0x7f84ec152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:07.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f2cfd700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84ec157fc0 0x7f84ec1f0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:07.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f24fc700 1 -- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84ec151840 msgr2=0x7f84ec152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:07.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f24fc700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84ec151840 0x7f84ec152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:07.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f24fc700 1 -- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84ec157fc0 msgr2=0x7f84ec1f0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:07.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f24fc700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84ec157fc0 0x7f84ec1f0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:07.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f24fc700 1 -- 172.21.15.146:0/2986922914 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e80094b0 con 0x7f84ec0c8380 2024-09-13T18:33:07.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f2cfd700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84ec157fc0 0x7f84ec1f0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:33:07.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f24fc700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84ec0c8380 0x7f84ec1526b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f84e80090b0 tx=0x7f84e8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:07.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84e2ffd700 1 -- 172.21.15.146:0/2986922914 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8004380 con 0x7f84ec0c8380 2024-09-13T18:33:07.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84e2ffd700 1 -- 172.21.15.146:0/2986922914 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e801f070 con 0x7f84ec0c8380 2024-09-13T18:33:07.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f34fe700 1 -- 172.21.15.146:0/2986922914 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ec1e3570 con 0x7f84ec0c8380 2024-09-13T18:33:07.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84e2ffd700 1 -- 172.21.15.146:0/2986922914 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e80036c0 con 0x7f84ec0c8380 2024-09-13T18:33:07.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.515+0000 7f84f34fe700 1 -- 172.21.15.146:0/2986922914 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ec1e13d0 con 0x7f84ec0c8380 2024-09-13T18:33:07.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.519+0000 7f84f34fe700 1 -- 172.21.15.146:0/2986922914 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84ec067a60 con 0x7f84ec0c8380 2024-09-13T18:33:07.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.519+0000 7f84e2ffd700 1 -- 172.21.15.146:0/2986922914 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f84e8021030 con 0x7f84ec0c8380 2024-09-13T18:33:07.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.519+0000 7f84e2ffd700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f84d0066730 0x7f84d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:07.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.519+0000 7f84e2ffd700 1 -- 172.21.15.146:0/2986922914 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f84e8085950 con 0x7f84ec0c8380 2024-09-13T18:33:07.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.519+0000 7f84f1cfb700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f84d0066730 0x7f84d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:07.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.519+0000 7f84f1cfb700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f84d0066730 0x7f84d0068bf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f84ec1e30b0 tx=0x7f84dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:07.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.527+0000 7f84e2ffd700 1 -- 172.21.15.146:0/2986922914 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f84e8052840 con 0x7f84ec0c8380 2024-09-13T18:33:07.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.731+0000 7f84f34fe700 1 -- 172.21.15.146:0/2986922914 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84ec157aa0 con 0x7f84ec0c8380 2024-09-13T18:33:07.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.731+0000 7f84e2ffd700 1 -- 172.21.15.146:0/2986922914 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5539 (secure 0 0 0) 0x7f84e8015090 con 0x7f84ec0c8380 2024-09-13T18:33:07.742 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:33:07.742 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:07.299208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:reconnect","state_seq":34,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:33:07.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.735+0000 7f84f34fe700 1 -- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f84d0066730 msgr2=0x7f84d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:07.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.735+0000 7f84f34fe700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f84d0066730 0x7f84d0068bf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f84ec1e30b0 tx=0x7f84dc009040 comp rx=0 tx=0).stop 2024-09-13T18:33:07.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.735+0000 7f84f34fe700 1 -- 172.21.15.146:0/2986922914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84ec0c8380 msgr2=0x7f84ec1526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:07.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.735+0000 7f84f34fe700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84ec0c8380 0x7f84ec1526b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f84e80090b0 tx=0x7f84e8004770 comp rx=0 tx=0).stop 2024-09-13T18:33:07.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.735+0000 7f84f34fe700 1 -- 172.21.15.146:0/2986922914 shutdown_connections 2024-09-13T18:33:07.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.735+0000 7f84f34fe700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f84d0066730 0x7f84d0068bf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:07.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.735+0000 7f84f34fe700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84ec157fc0 0x7f84ec1f0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:07.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.735+0000 7f84f34fe700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84ec151840 0x7f84ec152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:07.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.735+0000 7f84f34fe700 1 --2- 172.21.15.146:0/2986922914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84ec0c8380 0x7f84ec1526b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:07.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.735+0000 7f84f34fe700 1 -- 172.21.15.146:0/2986922914 >> 172.21.15.146:0/2986922914 conn(0x7f84ec0ccf70 msgr2=0x7f84ec0c7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:07.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.735+0000 7f84f34fe700 1 -- 172.21.15.146:0/2986922914 shutdown_connections 2024-09-13T18:33:07.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:07.735+0000 7f84f34fe700 1 -- 172.21.15.146:0/2986922914 wait complete. 2024-09-13T18:33:07.747 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 36 2024-09-13T18:33:07.758 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:33:08.208 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:33:08.210 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:33:09.759 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:33:09.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff5412e9700 1 Processor -- start 2024-09-13T18:33:09.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff5412e9700 1 -- start start 2024-09-13T18:33:09.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff5412e9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53c070d20 0x7ff53c071210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:09.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff5412e9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff53c150910 0x7ff53c156740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:09.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff5412e9700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53c071830 con 0x7ff53c070940 2024-09-13T18:33:09.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff5412e9700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff53c0c24b0 con 0x7ff53c150910 2024-09-13T18:33:09.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff5412e9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0c2790 con 0x7ff53c070d20 2024-09-13T18:33:09.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff540ae8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff53c150910 0x7ff53c156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:09.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff53bfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff53c070940 0x7ff53c150200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56162/0 (socket says 172.21.15.146:56162) 2024-09-13T18:33:09.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff53bfff700 1 -- 172.21.15.146:0/2426695444 learned_addr learned my addr 172.21.15.146:0/2426695444 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:33:09.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff53b7fe700 1 --2- 172.21.15.146:0/2426695444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53c070d20 0x7ff53c071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:09.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff540ae8700 1 -- 172.21.15.146:0/2426695444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53c070d20 msgr2=0x7ff53c071210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:09.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff540ae8700 1 --2- 172.21.15.146:0/2426695444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53c070d20 0x7ff53c071210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:09.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff540ae8700 1 -- 172.21.15.146:0/2426695444 >> v1:172.21.15.146:6789/0 conn(0x7ff53c070940 legacy=0x7ff53c150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:09.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff53affd700 1 -- 172.21.15.146:0/2426695444 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282493055 0 0) 0x7ff53c071830 con 0x7ff53c070940 2024-09-13T18:33:09.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff540ae8700 1 -- 172.21.15.146:0/2426695444 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c0c2a70 con 0x7ff53c150910 2024-09-13T18:33:09.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff540ae8700 1 --2- 172.21.15.146:0/2426695444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff53c150910 0x7ff53c156740 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009850 tx=0x7ff52c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc9a6e178b26f7f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:09.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff53affd700 1 -- 172.21.15.146:0/2426695444 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c011830 con 0x7ff53c150910 2024-09-13T18:33:09.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff53affd700 1 -- 172.21.15.146:0/2426695444 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c011990 con 0x7ff53c150910 2024-09-13T18:33:09.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.871+0000 7ff53affd700 1 -- 172.21.15.146:0/2426695444 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c011d40 con 0x7ff53c150910 2024-09-13T18:33:09.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.875+0000 7ff5412e9700 1 -- 172.21.15.146:0/2426695444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff53c150910 msgr2=0x7ff53c156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:09.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.875+0000 7ff5412e9700 1 --2- 172.21.15.146:0/2426695444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff53c150910 0x7ff53c156740 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009850 tx=0x7ff52c009b60 comp rx=0 tx=0).stop 2024-09-13T18:33:09.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.875+0000 7ff5412e9700 1 -- 172.21.15.146:0/2426695444 shutdown_connections 2024-09-13T18:33:09.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.875+0000 7ff5412e9700 1 --2- 172.21.15.146:0/2426695444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff53c150910 0x7ff53c156740 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:09.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.875+0000 7ff5412e9700 1 --2- 172.21.15.146:0/2426695444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53c070d20 0x7ff53c071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:09.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.875+0000 7ff5412e9700 1 -- 172.21.15.146:0/2426695444 >> 172.21.15.146:0/2426695444 conn(0x7ff53c071a90 msgr2=0x7ff53c071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:09.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.875+0000 7ff5412e9700 1 -- 172.21.15.146:0/2426695444 shutdown_connections 2024-09-13T18:33:09.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.875+0000 7ff5412e9700 1 -- 172.21.15.146:0/2426695444 wait complete. 2024-09-13T18:33:09.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.875+0000 7ff5412e9700 1 Processor -- start 2024-09-13T18:33:09.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.875+0000 7ff5412e9700 1 -- start start 2024-09-13T18:33:09.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff5412e9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53c070940 0x7ff53c1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:09.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff5412e9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff53c070d20 0x7ff53c0c3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:09.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff5412e9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff53c150910 0x7ff53c0c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:09.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff5412e9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c149630 con 0x7ff53c150910 2024-09-13T18:33:09.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff5412e9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c149090 con 0x7ff53c070d20 2024-09-13T18:33:09.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff5412e9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c149350 con 0x7ff53c070940 2024-09-13T18:33:09.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff540ae8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff53c150910 0x7ff53c0c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:09.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff540ae8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff53c150910 0x7ff53c0c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50592/0 (socket says 172.21.15.146:50592) 2024-09-13T18:33:09.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff540ae8700 1 -- 172.21.15.146:0/1071077619 learned_addr learned my addr 172.21.15.146:0/1071077619 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:09.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff53b7fe700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff53c070d20 0x7ff53c0c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:09.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff53bfff700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53c070940 0x7ff53c1531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:09.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff540ae8700 1 -- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53c070940 msgr2=0x7ff53c1531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:09.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff540ae8700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53c070940 0x7ff53c1531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:09.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff540ae8700 1 -- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff53c070d20 msgr2=0x7ff53c0c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:09.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff540ae8700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff53c070d20 0x7ff53c0c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:09.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff540ae8700 1 -- 172.21.15.146:0/1071077619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c0094b0 con 0x7ff53c150910 2024-09-13T18:33:09.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff540ae8700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff53c150910 0x7ff53c0c3cd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0090b0 tx=0x7ff52c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:09.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff538ff9700 1 -- 172.21.15.146:0/1071077619 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c004550 con 0x7ff53c150910 2024-09-13T18:33:09.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff538ff9700 1 -- 172.21.15.146:0/1071077619 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c01e070 con 0x7ff53c150910 2024-09-13T18:33:09.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff5412e9700 1 -- 172.21.15.146:0/1071077619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53c1482c0 con 0x7ff53c150910 2024-09-13T18:33:09.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff538ff9700 1 -- 172.21.15.146:0/1071077619 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c003900 con 0x7ff53c150910 2024-09-13T18:33:09.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.879+0000 7ff5412e9700 1 -- 172.21.15.146:0/1071077619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53c148880 con 0x7ff53c150910 2024-09-13T18:33:09.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.883+0000 7ff5412e9700 1 -- 172.21.15.146:0/1071077619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff53c148a70 con 0x7ff53c150910 2024-09-13T18:33:09.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.883+0000 7ff538ff9700 1 -- 172.21.15.146:0/1071077619 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff52c003a60 con 0x7ff53c150910 2024-09-13T18:33:09.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.883+0000 7ff538ff9700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff51c066760 0x7ff51c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:09.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.883+0000 7ff538ff9700 1 -- 172.21.15.146:0/1071077619 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7ff52c0853d0 con 0x7ff53c150910 2024-09-13T18:33:09.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.883+0000 7ff53bfff700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff51c066760 0x7ff51c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:09.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.883+0000 7ff53bfff700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff51c066760 0x7ff51c068c20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff524000f60 tx=0x7ff524007e00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:09.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:09.891+0000 7ff538ff9700 1 -- 172.21.15.146:0/1071077619 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff52c052230 con 0x7ff53c150910 2024-09-13T18:33:10.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.099+0000 7ff5412e9700 1 -- 172.21.15.146:0/1071077619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff53c0b1be0 con 0x7ff53c150910 2024-09-13T18:33:10.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.099+0000 7ff538ff9700 1 -- 172.21.15.146:0/1071077619 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5539 (secure 0 0 0) 0x7ff52c006ce0 con 0x7ff53c150910 2024-09-13T18:33:10.112 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:33:10.112 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:07.299208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:reconnect","state_seq":34,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:33:10.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.107+0000 7ff5412e9700 1 -- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff51c066760 msgr2=0x7ff51c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:10.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.107+0000 7ff5412e9700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff51c066760 0x7ff51c068c20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff524000f60 tx=0x7ff524007e00 comp rx=0 tx=0).stop 2024-09-13T18:33:10.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.107+0000 7ff5412e9700 1 -- 172.21.15.146:0/1071077619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff53c150910 msgr2=0x7ff53c0c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:10.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.107+0000 7ff5412e9700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff53c150910 0x7ff53c0c3cd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0090b0 tx=0x7ff52c004770 comp rx=0 tx=0).stop 2024-09-13T18:33:10.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.107+0000 7ff5412e9700 1 -- 172.21.15.146:0/1071077619 shutdown_connections 2024-09-13T18:33:10.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.107+0000 7ff5412e9700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff51c066760 0x7ff51c068c20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:10.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.107+0000 7ff5412e9700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff53c150910 0x7ff53c0c3cd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:10.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.107+0000 7ff5412e9700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff53c070d20 0x7ff53c0c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:10.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.107+0000 7ff5412e9700 1 --2- 172.21.15.146:0/1071077619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53c070940 0x7ff53c1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:10.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.107+0000 7ff5412e9700 1 -- 172.21.15.146:0/1071077619 >> 172.21.15.146:0/1071077619 conn(0x7ff53c071a90 msgr2=0x7ff53c14ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:10.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.107+0000 7ff5412e9700 1 -- 172.21.15.146:0/1071077619 shutdown_connections 2024-09-13T18:33:10.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:10.107+0000 7ff5412e9700 1 -- 172.21.15.146:0/1071077619 wait complete. 2024-09-13T18:33:10.120 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 36 2024-09-13T18:33:10.134 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 36, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 36, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5235": { "addr": "172.21.15.146:6833/1498985282", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 1498985282, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 1498985282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5235, "incarnation": 22, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 19 }, "gid_5278": { "addr": "172.21.15.154:6839/3274220802", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 3274220802, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 3274220802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5278, "incarnation": 28, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 19 }, "gid_5505": { "addr": "172.21.15.146:6835/1652676664", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 1652676664, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 1652676664, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5505, "incarnation": 34, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 34 } }, "last_failure": 0, "last_failure_osd_epoch": 31, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:33:07.299208+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5505, "mds_1": 5278, "mds_2": 5235 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:33:12.135 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:33:12.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 Processor -- start 2024-09-13T18:33:12.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 -- start start 2024-09-13T18:33:12.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8810157e10 0x7f88100c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:12.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88100c5bd0 0x7f88100c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:12.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88100adfe0 con 0x7f881006fe60 2024-09-13T18:33:12.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f881015e690 con 0x7f8810157e10 2024-09-13T18:33:12.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88100c7570 con 0x7f88100c5bd0 2024-09-13T18:33:12.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88158a2700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f881006fe60 0x7f8810070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50934/0 (socket says 172.21.15.146:50934) 2024-09-13T18:33:12.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88158a2700 1 -- 172.21.15.146:0/1036459903 learned_addr learned my addr 172.21.15.146:0/1036459903 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:33:12.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88160a3700 1 --2- 172.21.15.146:0/1036459903 >> v2:172.21.15.146:3300/0 conn(0x7f8810157e10 0x7f88100c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:12.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88168a4700 1 --2- 172.21.15.146:0/1036459903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88100c5bd0 0x7f88100c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:12.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88150a1700 1 -- 172.21.15.146:0/1036459903 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588182144 0 0) 0x7f88100adfe0 con 0x7f881006fe60 2024-09-13T18:33:12.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88150a1700 1 -- 172.21.15.146:0/1036459903 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f80035d0 con 0x7f881006fe60 2024-09-13T18:33:12.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88160a3700 1 -- 172.21.15.146:0/1036459903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88100c5bd0 msgr2=0x7f88100c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:12.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88160a3700 1 --2- 172.21.15.146:0/1036459903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88100c5bd0 0x7f88100c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:12.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88160a3700 1 -- 172.21.15.146:0/1036459903 >> v1:172.21.15.146:6789/0 conn(0x7f881006fe60 legacy=0x7f8810070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:12.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88160a3700 1 -- 172.21.15.146:0/1036459903 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881015eb80 con 0x7f8810157e10 2024-09-13T18:33:12.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88160a3700 1 --2- 172.21.15.146:0/1036459903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8810157e10 0x7f88100c6df0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8804009850 tx=0x7f8804009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54d73256a51ee71 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:12.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88150a1700 1 -- 172.21.15.146:0/1036459903 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8804011830 con 0x7f8810157e10 2024-09-13T18:33:12.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88150a1700 1 -- 172.21.15.146:0/1036459903 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8804011990 con 0x7f8810157e10 2024-09-13T18:33:12.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f88150a1700 1 -- 172.21.15.146:0/1036459903 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8804011d40 con 0x7f8810157e10 2024-09-13T18:33:12.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 -- 172.21.15.146:0/1036459903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8810157e10 msgr2=0x7f88100c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:12.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 --2- 172.21.15.146:0/1036459903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8810157e10 0x7f88100c6df0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8804009850 tx=0x7f8804009b60 comp rx=0 tx=0).stop 2024-09-13T18:33:12.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 -- 172.21.15.146:0/1036459903 shutdown_connections 2024-09-13T18:33:12.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 --2- 172.21.15.146:0/1036459903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88100c5bd0 0x7f88100c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:12.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 --2- 172.21.15.146:0/1036459903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8810157e10 0x7f88100c6df0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:12.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 -- 172.21.15.146:0/1036459903 >> 172.21.15.146:0/1036459903 conn(0x7f88100cb1d0 msgr2=0x7f88100cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:12.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 -- 172.21.15.146:0/1036459903 shutdown_connections 2024-09-13T18:33:12.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.247+0000 7f8818307700 1 -- 172.21.15.146:0/1036459903 wait complete. 2024-09-13T18:33:12.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f8818307700 1 Processor -- start 2024-09-13T18:33:12.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f8818307700 1 -- start start 2024-09-13T18:33:12.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f8818307700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f881006fe60 0x7f88101515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:12.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f8818307700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88100c5bd0 0x7f8810151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:12.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f8818307700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8810157e10 0x7f88101526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:12.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f8818307700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88101f43e0 con 0x7f881006fe60 2024-09-13T18:33:12.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f8818307700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88101f3760 con 0x7f8810157e10 2024-09-13T18:33:12.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f8818307700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88101f3a40 con 0x7f88100c5bd0 2024-09-13T18:33:12.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f88160a3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f881006fe60 0x7f88101515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:12.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f88160a3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f881006fe60 0x7f88101515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54586/0 (socket says 172.21.15.146:54586) 2024-09-13T18:33:12.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f88160a3700 1 -- 172.21.15.146:0/3038508876 learned_addr learned my addr 172.21.15.146:0/3038508876 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:12.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f88158a2700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88100c5bd0 0x7f8810151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:12.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f88168a4700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8810157e10 0x7f88101526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:12.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f88160a3700 1 -- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88100c5bd0 msgr2=0x7f8810151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:12.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f88160a3700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88100c5bd0 0x7f8810151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:12.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f88160a3700 1 -- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8810157e10 msgr2=0x7f88101526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:12.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f88160a3700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8810157e10 0x7f88101526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:12.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f88160a3700 1 -- 172.21.15.146:0/3038508876 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88040094b0 con 0x7f881006fe60 2024-09-13T18:33:12.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f88160a3700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f881006fe60 0x7f88101515a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f88040097a0 tx=0x7f8804004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:12.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f880effd700 1 -- 172.21.15.146:0/3038508876 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8804004610 con 0x7f881006fe60 2024-09-13T18:33:12.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f880effd700 1 -- 172.21.15.146:0/3038508876 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88040036c0 con 0x7f881006fe60 2024-09-13T18:33:12.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f880effd700 1 -- 172.21.15.146:0/3038508876 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8804003a50 con 0x7f881006fe60 2024-09-13T18:33:12.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f8818307700 1 -- 172.21.15.146:0/3038508876 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88101f3d20 con 0x7f881006fe60 2024-09-13T18:33:12.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.251+0000 7f8818307700 1 -- 172.21.15.146:0/3038508876 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88101e9a60 con 0x7f881006fe60 2024-09-13T18:33:12.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.255+0000 7f8818307700 1 -- 172.21.15.146:0/3038508876 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88101581f0 con 0x7f881006fe60 2024-09-13T18:33:12.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.255+0000 7f880effd700 1 -- 172.21.15.146:0/3038508876 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8804003bb0 con 0x7f881006fe60 2024-09-13T18:33:12.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.255+0000 7f880effd700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f87fc066730 0x7f87fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:12.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.255+0000 7f880effd700 1 -- 172.21.15.146:0/3038508876 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f8804021030 con 0x7f881006fe60 2024-09-13T18:33:12.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.255+0000 7f88158a2700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f87fc066730 0x7f87fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:12.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.255+0000 7f88158a2700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f87fc066730 0x7f87fc068bf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88101f7a40 tx=0x7f8808009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:12.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.263+0000 7f880effd700 1 -- 172.21.15.146:0/3038508876 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8804052b70 con 0x7f881006fe60 2024-09-13T18:33:12.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.471+0000 7f8818307700 1 -- 172.21.15.146:0/3038508876 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88100c1f70 con 0x7f881006fe60 2024-09-13T18:33:12.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.475+0000 7f880effd700 1 -- 172.21.15.146:0/3038508876 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5536 (secure 0 0 0) 0x7f8804015090 con 0x7f881006fe60 2024-09-13T18:33:12.485 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:33:12.486 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:12.315882+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":36,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:33:12.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.475+0000 7f8818307700 1 -- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f87fc066730 msgr2=0x7f87fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:12.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.475+0000 7f8818307700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f87fc066730 0x7f87fc068bf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88101f7a40 tx=0x7f8808009040 comp rx=0 tx=0).stop 2024-09-13T18:33:12.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.475+0000 7f8818307700 1 -- 172.21.15.146:0/3038508876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f881006fe60 msgr2=0x7f88101515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:12.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.475+0000 7f8818307700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f881006fe60 0x7f88101515a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f88040097a0 tx=0x7f8804004770 comp rx=0 tx=0).stop 2024-09-13T18:33:12.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.475+0000 7f8818307700 1 -- 172.21.15.146:0/3038508876 shutdown_connections 2024-09-13T18:33:12.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.475+0000 7f8818307700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f87fc066730 0x7f87fc068bf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:12.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.475+0000 7f8818307700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8810157e10 0x7f88101526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:12.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.475+0000 7f8818307700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88100c5bd0 0x7f8810151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:12.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.475+0000 7f8818307700 1 --2- 172.21.15.146:0/3038508876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f881006fe60 0x7f88101515a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:12.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.475+0000 7f8818307700 1 -- 172.21.15.146:0/3038508876 >> 172.21.15.146:0/3038508876 conn(0x7f88100cb1d0 msgr2=0x7f8810154520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:12.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.475+0000 7f8818307700 1 -- 172.21.15.146:0/3038508876 shutdown_connections 2024-09-13T18:33:12.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:12.475+0000 7f8818307700 1 -- 172.21.15.146:0/3038508876 wait complete. 2024-09-13T18:33:12.489 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 37 2024-09-13T18:33:12.500 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:33:14.501 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:33:14.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.615+0000 7f52110f6700 1 Processor -- start 2024-09-13T18:33:14.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.615+0000 7f52110f6700 1 -- start start 2024-09-13T18:33:14.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52110f6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f520c06bb10 0x7f520c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:14.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52110f6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f520c1530d0 0x7f520c153520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:14.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52110f6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520c153a60 con 0x7f520c070940 2024-09-13T18:33:14.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52110f6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520c15a530 con 0x7f520c1530d0 2024-09-13T18:33:14.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52110f6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520c071430 con 0x7f520c06bb10 2024-09-13T18:33:14.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52108f5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f520c1530d0 0x7f520c153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:14.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f520c06bb10 0x7f520c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:14.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52108f5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f520c1530d0 0x7f520c153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:36910/0 (socket says 172.21.15.146:36910) 2024-09-13T18:33:14.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f520bfff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f520c070940 0x7f520c070d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:37894/0 (socket says 172.21.15.146:37894) 2024-09-13T18:33:14.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52108f5700 1 -- 172.21.15.146:0/3225636995 learned_addr learned my addr 172.21.15.146:0/3225636995 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:14.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52108f5700 1 -- 172.21.15.146:0/3225636995 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f520c06bb10 msgr2=0x7f520c159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:14.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f520affd700 1 -- 172.21.15.146:0/3225636995 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846296821 0 0) 0x7f520c153a60 con 0x7f520c070940 2024-09-13T18:33:14.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52108f5700 1 --2- 172.21.15.146:0/3225636995 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f520c06bb10 0x7f520c159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:14.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52108f5700 1 -- 172.21.15.146:0/3225636995 >> v1:172.21.15.154:6789/0 conn(0x7f520c070940 legacy=0x7f520c070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:14.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52108f5700 1 -- 172.21.15.146:0/3225636995 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520c15da60 con 0x7f520c1530d0 2024-09-13T18:33:14.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f520b7fe700 1 --2- 172.21.15.146:0/3225636995 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f520c06bb10 0x7f520c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:33:14.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52108f5700 1 --2- 172.21.15.146:0/3225636995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f520c1530d0 0x7f520c153520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5204009910 tx=0x7f5204009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5887b82c8a7b63c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:14.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f520affd700 1 -- 172.21.15.146:0/3225636995 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204011830 con 0x7f520c1530d0 2024-09-13T18:33:14.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f520affd700 1 -- 172.21.15.146:0/3225636995 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5204011990 con 0x7f520c1530d0 2024-09-13T18:33:14.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f520affd700 1 -- 172.21.15.146:0/3225636995 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204011d40 con 0x7f520c1530d0 2024-09-13T18:33:14.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52110f6700 1 -- 172.21.15.146:0/3225636995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f520c1530d0 msgr2=0x7f520c153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:14.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52110f6700 1 --2- 172.21.15.146:0/3225636995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f520c1530d0 0x7f520c153520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5204009910 tx=0x7f5204009c20 comp rx=0 tx=0).stop 2024-09-13T18:33:14.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52110f6700 1 -- 172.21.15.146:0/3225636995 shutdown_connections 2024-09-13T18:33:14.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52110f6700 1 --2- 172.21.15.146:0/3225636995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f520c1530d0 0x7f520c153520 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:14.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52110f6700 1 --2- 172.21.15.146:0/3225636995 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f520c06bb10 0x7f520c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:14.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52110f6700 1 -- 172.21.15.146:0/3225636995 >> 172.21.15.146:0/3225636995 conn(0x7f520c06c2d0 msgr2=0x7f520c06c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:14.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52110f6700 1 -- 172.21.15.146:0/3225636995 shutdown_connections 2024-09-13T18:33:14.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.619+0000 7f52110f6700 1 -- 172.21.15.146:0/3225636995 wait complete. 2024-09-13T18:33:14.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f52110f6700 1 Processor -- start 2024-09-13T18:33:14.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f52110f6700 1 -- start start 2024-09-13T18:33:14.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f52110f6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f520c06bb10 0x7f520c0c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:14.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f52110f6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f520c070940 0x7f520c0c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:14.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f52110f6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f520c1530d0 0x7f520c0c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:14.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f52110f6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c0c52e0 con 0x7f520c070940 2024-09-13T18:33:14.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f52110f6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c0c4660 con 0x7f520c06bb10 2024-09-13T18:33:14.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f52110f6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520c0c4940 con 0x7f520c1530d0 2024-09-13T18:33:14.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f520c070940 0x7f520c0c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:14.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f520c070940 0x7f520c0c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54608/0 (socket says 172.21.15.146:54608) 2024-09-13T18:33:14.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f520b7fe700 1 -- 172.21.15.146:0/767858231 learned_addr learned my addr 172.21.15.146:0/767858231 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:14.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f52108f5700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f520c1530d0 0x7f520c0c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:14.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f520bfff700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f520c06bb10 0x7f520c0c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:14.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f520b7fe700 1 -- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f520c1530d0 msgr2=0x7f520c0c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:14.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f520b7fe700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f520c1530d0 0x7f520c0c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:14.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f520b7fe700 1 -- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f520c06bb10 msgr2=0x7f520c0c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:14.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f520b7fe700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f520c06bb10 0x7f520c0c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:14.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f520b7fe700 1 -- 172.21.15.146:0/767858231 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc009a70 con 0x7f520c070940 2024-09-13T18:33:14.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f520b7fe700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f520c070940 0x7f520c0c5700 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00efc0 tx=0x7f51fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:14.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f5208ff9700 1 -- 172.21.15.146:0/767858231 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00cdd0 con 0x7f520c070940 2024-09-13T18:33:14.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f5208ff9700 1 -- 172.21.15.146:0/767858231 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51fc00f040 con 0x7f520c070940 2024-09-13T18:33:14.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f52110f6700 1 -- 172.21.15.146:0/767858231 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5204009570 con 0x7f520c070940 2024-09-13T18:33:14.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f5208ff9700 1 -- 172.21.15.146:0/767858231 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0146a0 con 0x7f520c070940 2024-09-13T18:33:14.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.623+0000 7f52110f6700 1 -- 172.21.15.146:0/767858231 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520c0c4f00 con 0x7f520c070940 2024-09-13T18:33:14.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.627+0000 7f5208ff9700 1 -- 172.21.15.146:0/767858231 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f51fc014800 con 0x7f520c070940 2024-09-13T18:33:14.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.627+0000 7f52110f6700 1 -- 172.21.15.146:0/767858231 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520c15d780 con 0x7f520c070940 2024-09-13T18:33:14.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.627+0000 7f5208ff9700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f51ec066730 0x7f51ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:14.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.627+0000 7f5208ff9700 1 -- 172.21.15.146:0/767858231 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f51fc085360 con 0x7f520c070940 2024-09-13T18:33:14.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.627+0000 7f520bfff700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f51ec066730 0x7f51ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:14.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.627+0000 7f520bfff700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f51ec066730 0x7f51ec068bf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f51f4000f60 tx=0x7f51f4007e00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:14.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.631+0000 7f5208ff9700 1 -- 172.21.15.146:0/767858231 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f51fc0520b0 con 0x7f520c070940 2024-09-13T18:33:14.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f52110f6700 1 -- 172.21.15.146:0/767858231 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f520c0afb70 con 0x7f520c070940 2024-09-13T18:33:14.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f5208ff9700 1 -- 172.21.15.146:0/767858231 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5536 (secure 0 0 0) 0x7f51fc0556d0 con 0x7f520c070940 2024-09-13T18:33:14.850 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:33:14.850 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:12.315882+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":36,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:33:14.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f52110f6700 1 -- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f51ec066730 msgr2=0x7f51ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:14.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f52110f6700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f51ec066730 0x7f51ec068bf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f51f4000f60 tx=0x7f51f4007e00 comp rx=0 tx=0).stop 2024-09-13T18:33:14.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f52110f6700 1 -- 172.21.15.146:0/767858231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f520c070940 msgr2=0x7f520c0c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:14.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f52110f6700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f520c070940 0x7f520c0c5700 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00efc0 tx=0x7f51fc00c3a0 comp rx=0 tx=0).stop 2024-09-13T18:33:14.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f52110f6700 1 -- 172.21.15.146:0/767858231 shutdown_connections 2024-09-13T18:33:14.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f52110f6700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f51ec066730 0x7f51ec068bf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:14.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f52110f6700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f520c1530d0 0x7f520c0c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:14.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f52110f6700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f520c070940 0x7f520c0c5700 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:14.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f52110f6700 1 --2- 172.21.15.146:0/767858231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f520c06bb10 0x7f520c0c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:14.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f52110f6700 1 -- 172.21.15.146:0/767858231 >> 172.21.15.146:0/767858231 conn(0x7f520c06c2d0 msgr2=0x7f520c0cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:14.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f52110f6700 1 -- 172.21.15.146:0/767858231 shutdown_connections 2024-09-13T18:33:14.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:14.839+0000 7f52110f6700 1 -- 172.21.15.146:0/767858231 wait complete. 2024-09-13T18:33:14.853 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 37 2024-09-13T18:33:14.865 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:33:16.866 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:33:16.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.975+0000 7f8cab59e700 1 Processor -- start 2024-09-13T18:33:16.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.975+0000 7f8cab59e700 1 -- start start 2024-09-13T18:33:16.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8cab59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8cac153bc0 0x7f8cac0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:16.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8cab59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8cac0c6a80 0x7f8cac160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:16.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8cab59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac153fa0 con 0x7f8cac0c6610 2024-09-13T18:33:16.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8cab59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0c6e60 con 0x7f8cac153bc0 2024-09-13T18:33:16.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8cab59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0aa500 con 0x7f8cac0c6a80 2024-09-13T18:33:16.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8caa59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8cac153bc0 0x7f8cac0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:16.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8caa59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8cac153bc0 0x7f8cac0c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54616/0 (socket says 172.21.15.146:54616) 2024-09-13T18:33:16.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8caa59c700 1 -- 172.21.15.146:0/3568974803 learned_addr learned my addr 172.21.15.146:0/3568974803 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:16.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8caad9d700 1 --2- 172.21.15.146:0/3568974803 >> v2:172.21.15.154:3300/0 conn(0x7f8cac0c6a80 0x7f8cac160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:16.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8caa59c700 1 -- 172.21.15.146:0/3568974803 >> v2:172.21.15.154:3300/0 conn(0x7f8cac0c6a80 msgr2=0x7f8cac160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:16.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8caa59c700 1 --2- 172.21.15.146:0/3568974803 >> v2:172.21.15.154:3300/0 conn(0x7f8cac0c6a80 0x7f8cac160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:16.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8caa59c700 1 -- 172.21.15.146:0/3568974803 >> v1:172.21.15.154:6789/0 conn(0x7f8cac0c6610 legacy=0x7f8cac0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:16.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8caa59c700 1 -- 172.21.15.146:0/3568974803 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac15a7e0 con 0x7f8cac153bc0 2024-09-13T18:33:16.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8caa59c700 1 --2- 172.21.15.146:0/3568974803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cac153bc0 0x7f8cac0c8de0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8cac06f630 tx=0x7f8c9c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8771f22e78841d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:16.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8ca959a700 1 -- 172.21.15.146:0/3568974803 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c011830 con 0x7f8cac153bc0 2024-09-13T18:33:16.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8ca959a700 1 -- 172.21.15.146:0/3568974803 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c011990 con 0x7f8cac153bc0 2024-09-13T18:33:16.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8ca959a700 1 -- 172.21.15.146:0/3568974803 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c011d20 con 0x7f8cac153bc0 2024-09-13T18:33:16.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8cab59e700 1 -- 172.21.15.146:0/3568974803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cac153bc0 msgr2=0x7f8cac0c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:16.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8cab59e700 1 --2- 172.21.15.146:0/3568974803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cac153bc0 0x7f8cac0c8de0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8cac06f630 tx=0x7f8c9c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:33:16.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8cab59e700 1 -- 172.21.15.146:0/3568974803 shutdown_connections 2024-09-13T18:33:16.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8cab59e700 1 --2- 172.21.15.146:0/3568974803 >> v2:172.21.15.154:3300/0 conn(0x7f8cac0c6a80 0x7f8cac160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:16.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8cab59e700 1 --2- 172.21.15.146:0/3568974803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cac153bc0 0x7f8cac0c8de0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:16.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8cab59e700 1 -- 172.21.15.146:0/3568974803 >> 172.21.15.146:0/3568974803 conn(0x7f8cac066090 msgr2=0x7f8cac066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:16.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8cab59e700 1 -- 172.21.15.146:0/3568974803 shutdown_connections 2024-09-13T18:33:16.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.979+0000 7f8cab59e700 1 -- 172.21.15.146:0/3568974803 wait complete. 2024-09-13T18:33:16.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8cab59e700 1 Processor -- start 2024-09-13T18:33:16.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8cab59e700 1 -- start start 2024-09-13T18:33:16.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8cab59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cac0c6610 0x7f8cac0caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:16.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8cab59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8cac0c6a80 0x7f8cac0ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:16.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8cab59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cac153bc0 0x7f8cac0ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:16.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8cab59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0c1420 con 0x7f8cac0c6610 2024-09-13T18:33:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8cab59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0aed70 con 0x7f8cac153bc0 2024-09-13T18:33:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8cab59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0abd10 con 0x7f8cac0c6a80 2024-09-13T18:33:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8caa59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cac0c6610 0x7f8cac0caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8caa59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cac0c6610 0x7f8cac0caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54618/0 (socket says 172.21.15.146:54618) 2024-09-13T18:33:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8caa59c700 1 -- 172.21.15.146:0/1417241016 learned_addr learned my addr 172.21.15.146:0/1417241016 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8caad9d700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cac153bc0 0x7f8cac0ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8ca9d9b700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8cac0c6a80 0x7f8cac0ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8caa59c700 1 -- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8cac0c6a80 msgr2=0x7f8cac0ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8caa59c700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8cac0c6a80 0x7f8cac0ac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8caa59c700 1 -- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cac153bc0 msgr2=0x7f8cac0ac6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8caa59c700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cac153bc0 0x7f8cac0ac6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8caa59c700 1 -- 172.21.15.146:0/1417241016 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c009570 con 0x7f8cac0c6610 2024-09-13T18:33:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8ca9d9b700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8cac0c6a80 0x7f8cac0ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:33:16.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8caa59c700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cac0c6610 0x7f8cac0caed0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c000c00 tx=0x7f8c9c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:16.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.983+0000 7f8c9affd700 1 -- 172.21.15.146:0/1417241016 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c004510 con 0x7f8cac0c6610 2024-09-13T18:33:16.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.987+0000 7f8c9affd700 1 -- 172.21.15.146:0/1417241016 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c01e070 con 0x7f8cac0c6610 2024-09-13T18:33:16.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.987+0000 7f8cab59e700 1 -- 172.21.15.146:0/1417241016 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cac0ab550 con 0x7f8cac0c6610 2024-09-13T18:33:16.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.987+0000 7f8c9affd700 1 -- 172.21.15.146:0/1417241016 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0038c0 con 0x7f8cac0c6610 2024-09-13T18:33:16.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.987+0000 7f8cab59e700 1 -- 172.21.15.146:0/1417241016 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac0ae860 con 0x7f8cac0c6610 2024-09-13T18:33:16.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.987+0000 7f8cab59e700 1 -- 172.21.15.146:0/1417241016 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cac159fe0 con 0x7f8cac0c6610 2024-09-13T18:33:16.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.987+0000 7f8c9affd700 1 -- 172.21.15.146:0/1417241016 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8c9c003a20 con 0x7f8cac0c6610 2024-09-13T18:33:16.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.987+0000 7f8c9affd700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8c8c066730 0x7f8c8c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:17.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.987+0000 7f8c9affd700 1 -- 172.21.15.146:0/1417241016 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f8c9c020070 con 0x7f8cac0c6610 2024-09-13T18:33:17.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.987+0000 7f8ca9d9b700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8c8c066730 0x7f8c8c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:17.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.991+0000 7f8ca9d9b700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8c8c066730 0x7f8c8c068bf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0c0ed0 tx=0x7f8ca0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:17.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:16.991+0000 7f8c9affd700 1 -- 172.21.15.146:0/1417241016 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8c9c052950 con 0x7f8cac0c6610 2024-09-13T18:33:17.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.199+0000 7f8cab59e700 1 -- 172.21.15.146:0/1417241016 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cac06f690 con 0x7f8cac0c6610 2024-09-13T18:33:17.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.199+0000 7f8c9affd700 1 -- 172.21.15.146:0/1417241016 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5536 (secure 0 0 0) 0x7f8c9c006d20 con 0x7f8cac0c6610 2024-09-13T18:33:17.212 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:33:17.212 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:12.315882+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":36,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:33:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.203+0000 7f8cab59e700 1 -- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8c8c066730 msgr2=0x7f8c8c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.203+0000 7f8cab59e700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8c8c066730 0x7f8c8c068bf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0c0ed0 tx=0x7f8ca0009040 comp rx=0 tx=0).stop 2024-09-13T18:33:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.203+0000 7f8cab59e700 1 -- 172.21.15.146:0/1417241016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cac0c6610 msgr2=0x7f8cac0caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.203+0000 7f8cab59e700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cac0c6610 0x7f8cac0caed0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c000c00 tx=0x7f8c9c004770 comp rx=0 tx=0).stop 2024-09-13T18:33:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.203+0000 7f8cab59e700 1 -- 172.21.15.146:0/1417241016 shutdown_connections 2024-09-13T18:33:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.203+0000 7f8cab59e700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8c8c066730 0x7f8c8c068bf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.203+0000 7f8cab59e700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cac153bc0 0x7f8cac0ac6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.203+0000 7f8cab59e700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8cac0c6a80 0x7f8cac0ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:17.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.203+0000 7f8cab59e700 1 --2- 172.21.15.146:0/1417241016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cac0c6610 0x7f8cac0caed0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:17.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.203+0000 7f8cab59e700 1 -- 172.21.15.146:0/1417241016 >> 172.21.15.146:0/1417241016 conn(0x7f8cac066090 msgr2=0x7f8cac066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:17.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.203+0000 7f8cab59e700 1 -- 172.21.15.146:0/1417241016 shutdown_connections 2024-09-13T18:33:17.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:17.203+0000 7f8cab59e700 1 -- 172.21.15.146:0/1417241016 wait complete. 2024-09-13T18:33:17.216 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 37 2024-09-13T18:33:17.227 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:33:19.228 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:33:19.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.339+0000 7f37f9b50700 1 Processor -- start 2024-09-13T18:33:19.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.339+0000 7f37f9b50700 1 -- start start 2024-09-13T18:33:19.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.339+0000 7f37f9b50700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40c6610 0x7f37f40c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:19.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.339+0000 7f37f9b50700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f37f40cac60 0x7f37f40cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:19.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.339+0000 7f37f9b50700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f4153fa0 con 0x7f37f4153bc0 2024-09-13T18:33:19.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.339+0000 7f37f9b50700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f40aa500 con 0x7f37f40cac60 2024-09-13T18:33:19.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.339+0000 7f37f9b50700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f415a7e0 con 0x7f37f40c6610 2024-09-13T18:33:19.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37ebfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40c6610 0x7f37f40c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:19.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37ebfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40c6610 0x7f37f40c6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:49230/0 (socket says 172.21.15.146:49230) 2024-09-13T18:33:19.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37ebfff700 1 -- 172.21.15.146:0/2458929177 learned_addr learned my addr 172.21.15.146:0/2458929177 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:19.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37f934f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f37f40cac60 0x7f37f40cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:19.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37ebfff700 1 -- 172.21.15.146:0/2458929177 >> v2:172.21.15.154:3300/0 conn(0x7f37f40cac60 msgr2=0x7f37f40cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:19.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37ebfff700 1 --2- 172.21.15.146:0/2458929177 >> v2:172.21.15.154:3300/0 conn(0x7f37f40cac60 0x7f37f40cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:19.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37ebfff700 1 -- 172.21.15.146:0/2458929177 >> v1:172.21.15.154:6789/0 conn(0x7f37f4153bc0 legacy=0x7f37f40c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:19.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37ebfff700 1 -- 172.21.15.146:0/2458929177 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f40aa220 con 0x7f37f40c6610 2024-09-13T18:33:19.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37f934f700 1 --2- 172.21.15.146:0/2458929177 >> v2:172.21.15.154:3300/0 conn(0x7f37f40cac60 0x7f37f40cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:33:19.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37ebfff700 1 --2- 172.21.15.146:0/2458929177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40c6610 0x7f37f40c6a50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009910 tx=0x7f37e4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4df9bc6155f236 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:19.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37eb7fe700 1 -- 172.21.15.146:0/2458929177 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4011830 con 0x7f37f40c6610 2024-09-13T18:33:19.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37eb7fe700 1 -- 172.21.15.146:0/2458929177 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e4011990 con 0x7f37f40c6610 2024-09-13T18:33:19.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37eb7fe700 1 -- 172.21.15.146:0/2458929177 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4011d40 con 0x7f37f40c6610 2024-09-13T18:33:19.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37f9b50700 1 -- 172.21.15.146:0/2458929177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40c6610 msgr2=0x7f37f40c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:19.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37f9b50700 1 --2- 172.21.15.146:0/2458929177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40c6610 0x7f37f40c6a50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009910 tx=0x7f37e4009c20 comp rx=0 tx=0).stop 2024-09-13T18:33:19.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37f9b50700 1 -- 172.21.15.146:0/2458929177 shutdown_connections 2024-09-13T18:33:19.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37f9b50700 1 --2- 172.21.15.146:0/2458929177 >> v2:172.21.15.154:3300/0 conn(0x7f37f40cac60 0x7f37f40cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:19.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37f9b50700 1 --2- 172.21.15.146:0/2458929177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40c6610 0x7f37f40c6a50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:19.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37f9b50700 1 -- 172.21.15.146:0/2458929177 >> 172.21.15.146:0/2458929177 conn(0x7f37f4066090 msgr2=0x7f37f4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:19.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37f9b50700 1 -- 172.21.15.146:0/2458929177 shutdown_connections 2024-09-13T18:33:19.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.343+0000 7f37f9b50700 1 -- 172.21.15.146:0/2458929177 wait complete. 2024-09-13T18:33:19.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37f9b50700 1 Processor -- start 2024-09-13T18:33:19.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37f9b50700 1 -- start start 2024-09-13T18:33:19.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37f9b50700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37f40c6610 0x7f37f40acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:19.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37f9b50700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40cac60 0x7f37f41ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:19.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37f9b50700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37f4153bc0 0x7f37f41ef700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:19.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37f9b50700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f41eed30 con 0x7f37f4153bc0 2024-09-13T18:33:19.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37f9b50700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f41f30b0 con 0x7f37f40c6610 2024-09-13T18:33:19.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37f9b50700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40ca670 con 0x7f37f40cac60 2024-09-13T18:33:19.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37f934f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37f4153bc0 0x7f37f41ef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:19.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37ebfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40cac60 0x7f37f41ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:19.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37f8b4e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37f40c6610 0x7f37f40acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:19.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37ebfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40cac60 0x7f37f41ef1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:49234/0 (socket says 172.21.15.146:49234) 2024-09-13T18:33:19.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37ebfff700 1 -- 172.21.15.146:0/995097487 learned_addr learned my addr 172.21.15.146:0/995097487 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:19.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37f8b4e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37f40c6610 0x7f37f40acbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:36948/0 (socket says 172.21.15.146:36948) 2024-09-13T18:33:19.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37ebfff700 1 -- 172.21.15.146:0/995097487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37f40c6610 msgr2=0x7f37f40acbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:19.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37ebfff700 1 --2- 172.21.15.146:0/995097487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37f40c6610 0x7f37f40acbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:19.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37ebfff700 1 -- 172.21.15.146:0/995097487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37f4153bc0 msgr2=0x7f37f41ef700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:19.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37ebfff700 1 --2- 172.21.15.146:0/995097487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37f4153bc0 0x7f37f41ef700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:19.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37ebfff700 1 -- 172.21.15.146:0/995097487 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec009a70 con 0x7f37f40cac60 2024-09-13T18:33:19.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37ebfff700 1 --2- 172.21.15.146:0/995097487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40cac60 0x7f37f41ef1c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37e4000c00 tx=0x7f37e4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:19.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37e97fa700 1 -- 172.21.15.146:0/995097487 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e40044d0 con 0x7f37f40cac60 2024-09-13T18:33:19.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37e97fa700 1 -- 172.21.15.146:0/995097487 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e40036c0 con 0x7f37f40cac60 2024-09-13T18:33:19.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37f9b50700 1 -- 172.21.15.146:0/995097487 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e4009570 con 0x7f37f40cac60 2024-09-13T18:33:19.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37e97fa700 1 -- 172.21.15.146:0/995097487 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4003a70 con 0x7f37f40cac60 2024-09-13T18:33:19.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.347+0000 7f37f9b50700 1 -- 172.21.15.146:0/995097487 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f41f2810 con 0x7f37f40cac60 2024-09-13T18:33:19.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.351+0000 7f37f9b50700 1 -- 172.21.15.146:0/995097487 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f4066470 con 0x7f37f40cac60 2024-09-13T18:33:19.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.351+0000 7f37e97fa700 1 -- 172.21.15.146:0/995097487 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f37e4003bd0 con 0x7f37f40cac60 2024-09-13T18:33:19.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.351+0000 7f37e97fa700 1 --2- 172.21.15.146:0/995097487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37e0066730 0x7f37e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:19.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.351+0000 7f37e97fa700 1 -- 172.21.15.146:0/995097487 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f37e4021030 con 0x7f37f40cac60 2024-09-13T18:33:19.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.351+0000 7f37f8b4e700 1 --2- 172.21.15.146:0/995097487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37e0066730 0x7f37e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:19.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.351+0000 7f37f8b4e700 1 --2- 172.21.15.146:0/995097487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37e0066730 0x7f37e0068bf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f37f0000f60 tx=0x7f37f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:19.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.359+0000 7f37e97fa700 1 -- 172.21.15.146:0/995097487 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f37e4052f80 con 0x7f37f40cac60 2024-09-13T18:33:19.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.567+0000 7f37f9b50700 1 -- 172.21.15.146:0/995097487 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37f4152250 con 0x7f37f40cac60 2024-09-13T18:33:19.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.571+0000 7f37e97fa700 1 -- 172.21.15.146:0/995097487 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5536 (secure 0 0 0) 0x7f37e4015090 con 0x7f37f40cac60 2024-09-13T18:33:19.581 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:33:19.581 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:17.345742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:33:19.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.571+0000 7f37f9b50700 1 -- 172.21.15.146:0/995097487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37e0066730 msgr2=0x7f37e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:19.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.571+0000 7f37f9b50700 1 --2- 172.21.15.146:0/995097487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37e0066730 0x7f37e0068bf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f37f0000f60 tx=0x7f37f0009040 comp rx=0 tx=0).stop 2024-09-13T18:33:19.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.571+0000 7f37f9b50700 1 -- 172.21.15.146:0/995097487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40cac60 msgr2=0x7f37f41ef1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:19.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.571+0000 7f37f9b50700 1 --2- 172.21.15.146:0/995097487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40cac60 0x7f37f41ef1c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37e4000c00 tx=0x7f37e4004770 comp rx=0 tx=0).stop 2024-09-13T18:33:19.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.571+0000 7f37f9b50700 1 -- 172.21.15.146:0/995097487 shutdown_connections 2024-09-13T18:33:19.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.571+0000 7f37f9b50700 1 --2- 172.21.15.146:0/995097487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37e0066730 0x7f37e0068bf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:19.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.571+0000 7f37f9b50700 1 --2- 172.21.15.146:0/995097487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37f4153bc0 0x7f37f41ef700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:19.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.571+0000 7f37f9b50700 1 --2- 172.21.15.146:0/995097487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37f40cac60 0x7f37f41ef1c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:19.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.571+0000 7f37f9b50700 1 --2- 172.21.15.146:0/995097487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37f40c6610 0x7f37f40acbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:19.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.571+0000 7f37f9b50700 1 -- 172.21.15.146:0/995097487 >> 172.21.15.146:0/995097487 conn(0x7f37f4066090 msgr2=0x7f37f40c8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:19.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.571+0000 7f37f9b50700 1 -- 172.21.15.146:0/995097487 shutdown_connections 2024-09-13T18:33:19.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:19.571+0000 7f37f9b50700 1 -- 172.21.15.146:0/995097487 wait complete. 2024-09-13T18:33:19.585 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 38 2024-09-13T18:33:19.599 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=4290 2024-09-13T18:33:19.599 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.c 2024-09-13T18:33:26.019 INFO:tasks.ceph.mds.a.smithi146.stderr:2024-09-13T18:33:26.007+0000 7f95a2b95700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:33:26.019 INFO:tasks.ceph.mds.a.smithi146.stderr:2024-09-13T18:33:26.007+0000 7f95a2b95700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-13T18:33:26.019 INFO:tasks.ceph.mds.a.smithi146.stderr:2024-09-13T18:33:26.007+0000 7f95a2b95700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-13T18:33:26.600 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-09-13T18:33:26.600 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T18:33:26.601 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T18:33:26.603 INFO:tasks.ceph.mds.c:Started 2024-09-13T18:33:26.603 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:33:26.690 INFO:tasks.ceph.mds.c.smithi146.stdout:starting mds.c at 2024-09-13T18:33:26.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.719+0000 7f9f1759e700 1 Processor -- start 2024-09-13T18:33:26.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.719+0000 7f9f1759e700 1 -- start start 2024-09-13T18:33:26.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.719+0000 7f9f1759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f10048620 0x7f9f100dd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:26.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.719+0000 7f9f1759e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9f1004c670 0x7f9f100ddb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:26.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.719+0000 7f9f1759e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f100514c0 con 0x7f9f10048a00 2024-09-13T18:33:26.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.719+0000 7f9f1759e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f100dd680 con 0x7f9f1004c670 2024-09-13T18:33:26.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.719+0000 7f9f1759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1004c1e0 con 0x7f9f10048620 2024-09-13T18:33:26.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.719+0000 7f9f16d9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9f10048a00 0x7f9f100d63e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56848/0 (socket says 172.21.15.146:56848) 2024-09-13T18:33:26.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.719+0000 7f9f16d9d700 1 -- 172.21.15.146:0/1507482279 learned_addr learned my addr 172.21.15.146:0/1507482279 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:33:26.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1659c700 1 --2- 172.21.15.146:0/1507482279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f10048620 0x7f9f100dd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:26.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f15d9b700 1 --2- 172.21.15.146:0/1507482279 >> v2:172.21.15.154:3300/0 conn(0x7f9f1004c670 0x7f9f100ddb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:26.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1559a700 1 -- 172.21.15.146:0/1507482279 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 219914072 0 0) 0x7f9f100514c0 con 0x7f9f10048a00 2024-09-13T18:33:26.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1559a700 1 -- 172.21.15.146:0/1507482279 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f000035d0 con 0x7f9f10048a00 2024-09-13T18:33:26.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1559a700 1 -- 172.21.15.146:0/1507482279 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1005030781 0 0) 0x7f9f000035d0 con 0x7f9f10048a00 2024-09-13T18:33:26.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1659c700 1 -- 172.21.15.146:0/1507482279 >> v2:172.21.15.154:3300/0 conn(0x7f9f1004c670 msgr2=0x7f9f100ddb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:26.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1659c700 1 --2- 172.21.15.146:0/1507482279 >> v2:172.21.15.154:3300/0 conn(0x7f9f1004c670 0x7f9f100ddb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:26.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1659c700 1 -- 172.21.15.146:0/1507482279 >> v1:172.21.15.146:6789/0 conn(0x7f9f10048a00 legacy=0x7f9f100d63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:26.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1659c700 1 -- 172.21.15.146:0/1507482279 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1004b2f0 con 0x7f9f10048620 2024-09-13T18:33:26.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1659c700 1 --2- 172.21.15.146:0/1507482279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f10048620 0x7f9f100dd140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009850 tx=0x7f9f08009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a747c0bc138d30 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1559a700 1 -- 172.21.15.146:0/1507482279 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18074a30 con 0x7f9f10048620 2024-09-13T18:33:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1559a700 1 -- 172.21.15.146:0/1507482279 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f08004030 con 0x7f9f10048620 2024-09-13T18:33:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1559a700 1 -- 172.21.15.146:0/1507482279 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f080043a0 con 0x7f9f10048620 2024-09-13T18:33:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1759e700 1 -- 172.21.15.146:0/1507482279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f10048620 msgr2=0x7f9f100dd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1759e700 1 --2- 172.21.15.146:0/1507482279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f10048620 0x7f9f100dd140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009850 tx=0x7f9f08009b60 comp rx=0 tx=0).stop 2024-09-13T18:33:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1759e700 1 -- 172.21.15.146:0/1507482279 shutdown_connections 2024-09-13T18:33:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1759e700 1 --2- 172.21.15.146:0/1507482279 >> v2:172.21.15.154:3300/0 conn(0x7f9f1004c670 0x7f9f100ddb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1759e700 1 --2- 172.21.15.146:0/1507482279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f10048620 0x7f9f100dd140 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1759e700 1 -- 172.21.15.146:0/1507482279 >> 172.21.15.146:0/1507482279 conn(0x7f9f10050610 msgr2=0x7f9f100509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1759e700 1 -- 172.21.15.146:0/1507482279 shutdown_connections 2024-09-13T18:33:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.723+0000 7f9f1759e700 1 -- 172.21.15.146:0/1507482279 wait complete. 2024-09-13T18:33:26.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1759e700 1 Processor -- start 2024-09-13T18:33:26.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1759e700 1 -- start start 2024-09-13T18:33:26.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f10048620 0x7f9f100de4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1759e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10048a00 0x7f9f100d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f1004c670 0x7f9f100d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1759e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f100df170 con 0x7f9f10048620 2024-09-13T18:33:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1759e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f100e02c0 con 0x7f9f10048a00 2024-09-13T18:33:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f100dda30 con 0x7f9f1004c670 2024-09-13T18:33:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f10048620 0x7f9f100de4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f10048620 0x7f9f100de4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45442/0 (socket says 172.21.15.146:45442) 2024-09-13T18:33:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1659c700 1 -- 172.21.15.146:0/1322607974 learned_addr learned my addr 172.21.15.146:0/1322607974 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f15d9b700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10048a00 0x7f9f100d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f16d9d700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f1004c670 0x7f9f100d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1659c700 1 -- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f1004c670 msgr2=0x7f9f100d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1659c700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f1004c670 0x7f9f100d2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:26.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1659c700 1 -- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10048a00 msgr2=0x7f9f100d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:26.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1659c700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10048a00 0x7f9f100d27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:26.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1659c700 1 -- 172.21.15.146:0/1322607974 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f080094b0 con 0x7f9f10048620 2024-09-13T18:33:26.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1659c700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f10048620 0x7f9f100de4a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9f1004c320 tx=0x7f9f08004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:26.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f06ffd700 1 -- 172.21.15.146:0/1322607974 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f08003e30 con 0x7f9f10048620 2024-09-13T18:33:26.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f06ffd700 1 -- 172.21.15.146:0/1322607974 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0801f070 con 0x7f9f10048620 2024-09-13T18:33:26.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1759e700 1 -- 172.21.15.146:0/1322607974 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f100ddcf0 con 0x7f9f10048620 2024-09-13T18:33:26.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f06ffd700 1 -- 172.21.15.146:0/1322607974 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f08011870 con 0x7f9f10048620 2024-09-13T18:33:26.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.727+0000 7f9f1759e700 1 -- 172.21.15.146:0/1322607974 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f100d9e50 con 0x7f9f10048620 2024-09-13T18:33:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.731+0000 7f9f1759e700 1 -- 172.21.15.146:0/1322607974 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f100d9fe0 con 0x7f9f10048620 2024-09-13T18:33:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.731+0000 7f9f06ffd700 1 -- 172.21.15.146:0/1322607974 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9f080119d0 con 0x7f9f10048620 2024-09-13T18:33:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.731+0000 7f9f06ffd700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9ef8066760 0x7f9ef8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.731+0000 7f9f06ffd700 1 -- 172.21.15.146:0/1322607974 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f9f08021070 con 0x7f9f10048620 2024-09-13T18:33:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.731+0000 7f9f15d9b700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9ef8066760 0x7f9ef8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.731+0000 7f9f15d9b700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9ef8066760 0x7f9ef8068c20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f100d4700 tx=0x7f9f0c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:26.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.735+0000 7f9f06ffd700 1 -- 172.21.15.146:0/1322607974 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9f08052e10 con 0x7f9f10048620 2024-09-13T18:33:26.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.987+0000 7f9f1759e700 1 -- 172.21.15.146:0/1322607974 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f1004af90 con 0x7f9f10048620 2024-09-13T18:33:27.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.987+0000 7f9f06ffd700 1 -- 172.21.15.146:0/1322607974 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5536 (secure 0 0 0) 0x7f9f08056430 con 0x7f9f10048620 2024-09-13T18:33:27.001 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:33:27.001 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:17.345742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:33:27.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.991+0000 7f9f1759e700 1 -- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9ef8066760 msgr2=0x7f9ef8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:27.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.991+0000 7f9f1759e700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9ef8066760 0x7f9ef8068c20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f100d4700 tx=0x7f9f0c008040 comp rx=0 tx=0).stop 2024-09-13T18:33:27.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.991+0000 7f9f1759e700 1 -- 172.21.15.146:0/1322607974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f10048620 msgr2=0x7f9f100de4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:27.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.991+0000 7f9f1759e700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f10048620 0x7f9f100de4a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9f1004c320 tx=0x7f9f08004770 comp rx=0 tx=0).stop 2024-09-13T18:33:27.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.991+0000 7f9f1759e700 1 -- 172.21.15.146:0/1322607974 shutdown_connections 2024-09-13T18:33:27.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.991+0000 7f9f1759e700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9ef8066760 0x7f9ef8068c20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:27.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.991+0000 7f9f1759e700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f1004c670 0x7f9f100d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:27.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.991+0000 7f9f1759e700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10048a00 0x7f9f100d27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:27.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.991+0000 7f9f1759e700 1 --2- 172.21.15.146:0/1322607974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f10048620 0x7f9f100de4a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:27.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.991+0000 7f9f1759e700 1 -- 172.21.15.146:0/1322607974 >> 172.21.15.146:0/1322607974 conn(0x7f9f10050610 msgr2=0x7f9f1004f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:27.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.991+0000 7f9f1759e700 1 -- 172.21.15.146:0/1322607974 shutdown_connections 2024-09-13T18:33:27.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:26.991+0000 7f9f1759e700 1 -- 172.21.15.146:0/1322607974 wait complete. 2024-09-13T18:33:27.005 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 38 2024-09-13T18:33:27.015 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-09-13T18:33:29.016 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:33:29.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.119+0000 7f15a93ea700 1 Processor -- start 2024-09-13T18:33:29.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.119+0000 7f15a93ea700 1 -- start start 2024-09-13T18:33:29.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a93ea700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f15a40c61b0 0x7f15a415d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:29.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a93ea700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a4066780 con 0x7f15a40c6590 2024-09-13T18:33:29.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a93ea700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a415d960 con 0x7f15a40c5440 2024-09-13T18:33:29.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a93ea700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a41594a0 con 0x7f15a40c61b0 2024-09-13T18:33:29.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f159b7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f15a40c6590 0x7f15a406cd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56858/0 (socket says 172.21.15.146:56858) 2024-09-13T18:33:29.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f159b7fe700 1 -- 172.21.15.146:0/2617607131 learned_addr learned my addr 172.21.15.146:0/2617607131 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:33:29.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a3fff700 1 --2- 172.21.15.146:0/2617607131 >> v2:172.21.15.154:3300/0 conn(0x7f15a40c61b0 0x7f15a415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:29.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a37fe700 1 -- 172.21.15.146:0/2617607131 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408886011 0 0) 0x7f158c001110 con 0x7f15a40c6590 2024-09-13T18:33:29.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a37fe700 1 -- 172.21.15.146:0/2617607131 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15900035d0 con 0x7f15a40c6590 2024-09-13T18:33:29.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a3fff700 1 -- 172.21.15.146:0/2617607131 >> v1:172.21.15.154:6789/0 conn(0x7f15a40c5440 legacy=0x7f15a4153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:29.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a3fff700 1 -- 172.21.15.146:0/2617607131 >> v1:172.21.15.146:6789/0 conn(0x7f15a40c6590 legacy=0x7f15a406cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:29.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a3fff700 1 -- 172.21.15.146:0/2617607131 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a406eec0 con 0x7f15a40c61b0 2024-09-13T18:33:29.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a3fff700 1 --2- 172.21.15.146:0/2617607131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a40c61b0 0x7f15a415d420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1594009850 tx=0x7f1594009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77421fe24f6fe18c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:29.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a37fe700 1 -- 172.21.15.146:0/2617607131 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594011830 con 0x7f15a40c61b0 2024-09-13T18:33:29.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a37fe700 1 -- 172.21.15.146:0/2617607131 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1594011990 con 0x7f15a40c61b0 2024-09-13T18:33:29.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a37fe700 1 -- 172.21.15.146:0/2617607131 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594011d40 con 0x7f15a40c61b0 2024-09-13T18:33:29.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a93ea700 1 -- 172.21.15.146:0/2617607131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a40c61b0 msgr2=0x7f15a415d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:29.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a93ea700 1 --2- 172.21.15.146:0/2617607131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a40c61b0 0x7f15a415d420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1594009850 tx=0x7f1594009b60 comp rx=0 tx=0).stop 2024-09-13T18:33:29.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a93ea700 1 -- 172.21.15.146:0/2617607131 shutdown_connections 2024-09-13T18:33:29.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a93ea700 1 --2- 172.21.15.146:0/2617607131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a40c61b0 0x7f15a415d420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:29.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a93ea700 1 -- 172.21.15.146:0/2617607131 >> 172.21.15.146:0/2617607131 conn(0x7f15a4066b00 msgr2=0x7f15a406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:29.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a93ea700 1 -- 172.21.15.146:0/2617607131 shutdown_connections 2024-09-13T18:33:29.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.123+0000 7f15a93ea700 1 -- 172.21.15.146:0/2617607131 wait complete. 2024-09-13T18:33:29.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a93ea700 1 Processor -- start 2024-09-13T18:33:29.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a93ea700 1 -- start start 2024-09-13T18:33:29.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a93ea700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15a40c5440 0x7f15a41f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:29.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a93ea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a40c61b0 0x7f15a40c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:29.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a93ea700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a40c6590 0x7f15a4158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:29.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a93ea700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a4152e40 con 0x7f15a40c5440 2024-09-13T18:33:29.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a93ea700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a41f35f0 con 0x7f15a40c61b0 2024-09-13T18:33:29.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a93ea700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a41ef1c0 con 0x7f15a40c6590 2024-09-13T18:33:29.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15a40c5440 0x7f15a41f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15a40c5440 0x7f15a41f30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45446/0 (socket says 172.21.15.146:45446) 2024-09-13T18:33:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a3fff700 1 -- 172.21.15.146:0/140960009 learned_addr learned my addr 172.21.15.146:0/140960009 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:33:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f159b7fe700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a40c61b0 0x7f15a40c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a8be9700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a40c6590 0x7f15a4158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a3fff700 1 -- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a40c6590 msgr2=0x7f15a4158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a3fff700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a40c6590 0x7f15a4158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a3fff700 1 -- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a40c61b0 msgr2=0x7f15a40c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a3fff700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a40c61b0 0x7f15a40c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a3fff700 1 -- 172.21.15.146:0/140960009 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940094b0 con 0x7f15a40c5440 2024-09-13T18:33:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a8be9700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a40c6590 0x7f15a4158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:33:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a3fff700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15a40c5440 0x7f15a41f30b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f1594004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a17fa700 1 -- 172.21.15.146:0/140960009 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159401aac0 con 0x7f15a40c5440 2024-09-13T18:33:29.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a17fa700 1 -- 172.21.15.146:0/140960009 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159401f070 con 0x7f15a40c5440 2024-09-13T18:33:29.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a93ea700 1 -- 172.21.15.146:0/140960009 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a41ef460 con 0x7f15a40c5440 2024-09-13T18:33:29.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a17fa700 1 -- 172.21.15.146:0/140960009 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159401ae60 con 0x7f15a40c5440 2024-09-13T18:33:29.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a93ea700 1 -- 172.21.15.146:0/140960009 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a41efa20 con 0x7f15a40c5440 2024-09-13T18:33:29.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.127+0000 7f15a17fa700 1 -- 172.21.15.146:0/140960009 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f159401b430 con 0x7f15a40c5440 2024-09-13T18:33:29.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.131+0000 7f15a93ea700 1 -- 172.21.15.146:0/140960009 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a4159f50 con 0x7f15a40c5440 2024-09-13T18:33:29.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.131+0000 7f15a17fa700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1584066760 0x7f1584068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:29.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.131+0000 7f15a17fa700 1 -- 172.21.15.146:0/140960009 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f15940858e0 con 0x7f15a40c5440 2024-09-13T18:33:29.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.131+0000 7f159b7fe700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1584066760 0x7f1584068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:29.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.131+0000 7f159b7fe700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1584066760 0x7f1584068c20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f15a40cb480 tx=0x7f158c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:29.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.135+0000 7f15a17fa700 1 -- 172.21.15.146:0/140960009 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1594052630 con 0x7f15a40c5440 2024-09-13T18:33:29.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.367+0000 7f15a93ea700 1 -- 172.21.15.146:0/140960009 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15a4150200 con 0x7f15a40c5440 2024-09-13T18:33:29.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.367+0000 7f15a17fa700 1 -- 172.21.15.146:0/140960009 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f1594021090 con 0x7f15a40c5440 2024-09-13T18:33:29.380 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:33:29.380 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:17.345742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:33:29.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.371+0000 7f15a93ea700 1 -- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1584066760 msgr2=0x7f1584068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:29.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.371+0000 7f15a93ea700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1584066760 0x7f1584068c20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f15a40cb480 tx=0x7f158c009040 comp rx=0 tx=0).stop 2024-09-13T18:33:29.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.371+0000 7f15a93ea700 1 -- 172.21.15.146:0/140960009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15a40c5440 msgr2=0x7f15a41f30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:29.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.371+0000 7f15a93ea700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15a40c5440 0x7f15a41f30b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f1594004770 comp rx=0 tx=0).stop 2024-09-13T18:33:29.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.371+0000 7f15a93ea700 1 -- 172.21.15.146:0/140960009 shutdown_connections 2024-09-13T18:33:29.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.371+0000 7f15a93ea700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1584066760 0x7f1584068c20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:29.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.371+0000 7f15a93ea700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a40c6590 0x7f15a4158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:29.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.371+0000 7f15a93ea700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a40c61b0 0x7f15a40c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:29.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.371+0000 7f15a93ea700 1 --2- 172.21.15.146:0/140960009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15a40c5440 0x7f15a41f30b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:29.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.371+0000 7f15a93ea700 1 -- 172.21.15.146:0/140960009 >> 172.21.15.146:0/140960009 conn(0x7f15a4066b00 msgr2=0x7f15a414eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:29.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.371+0000 7f15a93ea700 1 -- 172.21.15.146:0/140960009 shutdown_connections 2024-09-13T18:33:29.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:33:29.371+0000 7f15a93ea700 1 -- 172.21.15.146:0/140960009 wait complete. 2024-09-13T18:33:29.385 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 39 2024-09-13T18:33:29.395 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-09-13T18:33:29.396 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 40 secs before thrashing 2024-09-13T18:33:38.285 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:33:38.287 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:34:08.335 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:34:08.338 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:34:09.397 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:09.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.502+0000 7f4e61f7d700 1 Processor -- start 2024-09-13T18:34:09.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.502+0000 7f4e61f7d700 1 -- start start 2024-09-13T18:34:09.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e61f7d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4e5c0cbd50 0x7f4e5c0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e61f7d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4e5c071390 0x7f4e5c0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e61f7d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5c0cc670 con 0x7f4e5c0ca210 2024-09-13T18:34:09.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e61f7d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5c0caa30 con 0x7f4e5c0cbd50 2024-09-13T18:34:09.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e61f7d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5c0cb810 con 0x7f4e5c071390 2024-09-13T18:34:09.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e60f7b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4e5c0cbd50 0x7f4e5c0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e5bfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4e5c0ca210 0x7f4e5c0701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45010/0 (socket says 172.21.15.146:45010) 2024-09-13T18:34:09.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e5bfff700 1 -- 172.21.15.146:0/2300815804 learned_addr learned my addr 172.21.15.146:0/2300815804 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:34:09.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e6177c700 1 --2- 172.21.15.146:0/2300815804 >> v2:172.21.15.154:3300/0 conn(0x7f4e5c071390 0x7f4e5c0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e5b7fe700 1 -- 172.21.15.146:0/2300815804 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 533540834 0 0) 0x7f4e50001110 con 0x7f4e5c0ca210 2024-09-13T18:34:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e5b7fe700 1 -- 172.21.15.146:0/2300815804 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e440035d0 con 0x7f4e5c0ca210 2024-09-13T18:34:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e60f7b700 1 -- 172.21.15.146:0/2300815804 >> v2:172.21.15.154:3300/0 conn(0x7f4e5c071390 msgr2=0x7f4e5c0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e60f7b700 1 --2- 172.21.15.146:0/2300815804 >> v2:172.21.15.154:3300/0 conn(0x7f4e5c071390 0x7f4e5c0cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e60f7b700 1 -- 172.21.15.146:0/2300815804 >> v1:172.21.15.146:6789/0 conn(0x7f4e5c0ca210 legacy=0x7f4e5c0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e60f7b700 1 -- 172.21.15.146:0/2300815804 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c15da20 con 0x7f4e5c0cbd50 2024-09-13T18:34:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e60f7b700 1 --2- 172.21.15.146:0/2300815804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e5c0cbd50 0x7f4e5c0b1c40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0cce40 tx=0x7f4e5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5cbf3574a97c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e5b7fe700 1 -- 172.21.15.146:0/2300815804 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400bd20 con 0x7f4e5c0cbd50 2024-09-13T18:34:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e5b7fe700 1 -- 172.21.15.146:0/2300815804 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5400be80 con 0x7f4e5c0cbd50 2024-09-13T18:34:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e5b7fe700 1 -- 172.21.15.146:0/2300815804 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54014b10 con 0x7f4e5c0cbd50 2024-09-13T18:34:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e61f7d700 1 -- 172.21.15.146:0/2300815804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e5c0cbd50 msgr2=0x7f4e5c0b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e61f7d700 1 --2- 172.21.15.146:0/2300815804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e5c0cbd50 0x7f4e5c0b1c40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0cce40 tx=0x7f4e5400b330 comp rx=0 tx=0).stop 2024-09-13T18:34:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e61f7d700 1 -- 172.21.15.146:0/2300815804 shutdown_connections 2024-09-13T18:34:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e61f7d700 1 --2- 172.21.15.146:0/2300815804 >> v2:172.21.15.154:3300/0 conn(0x7f4e5c071390 0x7f4e5c0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e61f7d700 1 --2- 172.21.15.146:0/2300815804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e5c0cbd50 0x7f4e5c0b1c40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e61f7d700 1 -- 172.21.15.146:0/2300815804 >> 172.21.15.146:0/2300815804 conn(0x7f4e5c066270 msgr2=0x7f4e5c066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e61f7d700 1 -- 172.21.15.146:0/2300815804 shutdown_connections 2024-09-13T18:34:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.506+0000 7f4e61f7d700 1 -- 172.21.15.146:0/2300815804 wait complete. 2024-09-13T18:34:09.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e61f7d700 1 Processor -- start 2024-09-13T18:34:09.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e61f7d700 1 -- start start 2024-09-13T18:34:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e61f7d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c071390 0x7f4e5c1ebdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e61f7d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e5c0ca210 0x7f4e5c144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e61f7d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c0cbd50 0x7f4e5c151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e61f7d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c1ef240 con 0x7f4e5c0ca210 2024-09-13T18:34:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e61f7d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c1ec300 con 0x7f4e5c071390 2024-09-13T18:34:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e61f7d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c14fc10 con 0x7f4e5c0cbd50 2024-09-13T18:34:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e5bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e5c0ca210 0x7f4e5c144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e5bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e5c0ca210 0x7f4e5c144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35576/0 (socket says 172.21.15.146:35576) 2024-09-13T18:34:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e5bfff700 1 -- 172.21.15.146:0/1248087170 learned_addr learned my addr 172.21.15.146:0/1248087170 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e60f7b700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c071390 0x7f4e5c1ebdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e6177c700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c0cbd50 0x7f4e5c151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e5bfff700 1 -- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c0cbd50 msgr2=0x7f4e5c151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e5bfff700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c0cbd50 0x7f4e5c151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e5bfff700 1 -- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c071390 msgr2=0x7f4e5c1ebdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e5bfff700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c071390 0x7f4e5c1ebdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e5bfff700 1 -- 172.21.15.146:0/1248087170 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e54009a70 con 0x7f4e5c0ca210 2024-09-13T18:34:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.510+0000 7f4e5bfff700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e5c0ca210 0x7f4e5c144830 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4e50001270 tx=0x7f4e5000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.514+0000 7f4e597fa700 1 -- 172.21.15.146:0/1248087170 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50005d90 con 0x7f4e5c0ca210 2024-09-13T18:34:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.514+0000 7f4e597fa700 1 -- 172.21.15.146:0/1248087170 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e500070d0 con 0x7f4e5c0ca210 2024-09-13T18:34:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.514+0000 7f4e61f7d700 1 -- 172.21.15.146:0/1248087170 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c14fef0 con 0x7f4e5c0ca210 2024-09-13T18:34:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.514+0000 7f4e597fa700 1 -- 172.21.15.146:0/1248087170 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50007480 con 0x7f4e5c0ca210 2024-09-13T18:34:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.514+0000 7f4e61f7d700 1 -- 172.21.15.146:0/1248087170 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c1eebf0 con 0x7f4e5c0ca210 2024-09-13T18:34:09.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.514+0000 7f4e61f7d700 1 -- 172.21.15.146:0/1248087170 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5c0cc9b0 con 0x7f4e5c0ca210 2024-09-13T18:34:09.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.514+0000 7f4e597fa700 1 -- 172.21.15.146:0/1248087170 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4e500075e0 con 0x7f4e5c0ca210 2024-09-13T18:34:09.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.514+0000 7f4e597fa700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4e48066760 0x7f4e48068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.514+0000 7f4e597fa700 1 -- 172.21.15.146:0/1248087170 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f4e500838f0 con 0x7f4e5c0ca210 2024-09-13T18:34:09.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.514+0000 7f4e60f7b700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4e48066760 0x7f4e48068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.518+0000 7f4e60f7b700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4e48066760 0x7f4e48068c20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4e5401f040 tx=0x7f4e54018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.522+0000 7f4e597fa700 1 -- 172.21.15.146:0/1248087170 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4e50050750 con 0x7f4e5c0ca210 2024-09-13T18:34:09.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.774+0000 7f4e61f7d700 1 -- 172.21.15.146:0/1248087170 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e5c0cc670 con 0x7f4e5c0ca210 2024-09-13T18:34:09.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.774+0000 7f4e597fa700 1 -- 172.21.15.146:0/1248087170 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f4e5c0cc670 con 0x7f4e5c0ca210 2024-09-13T18:34:09.787 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:09.787 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:17.345742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:09.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.774+0000 7f4e61f7d700 1 -- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4e48066760 msgr2=0x7f4e48068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.774+0000 7f4e61f7d700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4e48066760 0x7f4e48068c20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4e5401f040 tx=0x7f4e54018000 comp rx=0 tx=0).stop 2024-09-13T18:34:09.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.774+0000 7f4e61f7d700 1 -- 172.21.15.146:0/1248087170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e5c0ca210 msgr2=0x7f4e5c144830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.774+0000 7f4e61f7d700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e5c0ca210 0x7f4e5c144830 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4e50001270 tx=0x7f4e5000c9e0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.774+0000 7f4e61f7d700 1 -- 172.21.15.146:0/1248087170 shutdown_connections 2024-09-13T18:34:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.774+0000 7f4e61f7d700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4e48066760 0x7f4e48068c20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.774+0000 7f4e61f7d700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c0cbd50 0x7f4e5c151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.774+0000 7f4e61f7d700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e5c0ca210 0x7f4e5c144830 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.774+0000 7f4e61f7d700 1 --2- 172.21.15.146:0/1248087170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c071390 0x7f4e5c1ebdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.774+0000 7f4e61f7d700 1 -- 172.21.15.146:0/1248087170 >> 172.21.15.146:0/1248087170 conn(0x7f4e5c066270 msgr2=0x7f4e5c0cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.778+0000 7f4e61f7d700 1 -- 172.21.15.146:0/1248087170 shutdown_connections 2024-09-13T18:34:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.778+0000 7f4e61f7d700 1 -- 172.21.15.146:0/1248087170 wait complete. 2024-09-13T18:34:09.791 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 39 2024-09-13T18:34:09.804 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=2) 2024-09-13T18:34:09.804 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-09-13T18:34:09.804 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:34:09.817 INFO:tasks.ceph.mds.e:Stopped 2024-09-13T18:34:09.818 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:09.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.918+0000 7f09c359e700 1 Processor -- start 2024-09-13T18:34:09.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.918+0000 7f09c359e700 1 -- start start 2024-09-13T18:34:09.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f09c40c61b0 0x7f09c415d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09c40c6590 0x7f09c406cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c359e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c4066780 con 0x7f09c406d2c0 2024-09-13T18:34:09.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c41594a0 con 0x7f09c40c61b0 2024-09-13T18:34:09.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c415d960 con 0x7f09c40c6590 2024-09-13T18:34:09.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c259c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f09c40c61b0 0x7f09c415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c259c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f09c40c61b0 0x7f09c415d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35580/0 (socket says 172.21.15.146:35580) 2024-09-13T18:34:09.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c259c700 1 -- 172.21.15.146:0/1323788933 learned_addr learned my addr 172.21.15.146:0/1323788933 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:09.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09bbfff700 1 --2- 172.21.15.146:0/1323788933 >> v2:172.21.15.154:3300/0 conn(0x7f09c40c6590 0x7f09c406cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c259c700 1 -- 172.21.15.146:0/1323788933 >> v2:172.21.15.154:3300/0 conn(0x7f09c40c6590 msgr2=0x7f09c406cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c259c700 1 --2- 172.21.15.146:0/1323788933 >> v2:172.21.15.154:3300/0 conn(0x7f09c40c6590 0x7f09c406cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c259c700 1 -- 172.21.15.146:0/1323788933 >> v1:172.21.15.154:6789/0 conn(0x7f09c406d2c0 legacy=0x7f09c4153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c259c700 1 -- 172.21.15.146:0/1323788933 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c406eec0 con 0x7f09c40c61b0 2024-09-13T18:34:09.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09bbfff700 1 --2- 172.21.15.146:0/1323788933 >> v2:172.21.15.154:3300/0 conn(0x7f09c40c6590 0x7f09c406cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:09.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c259c700 1 --2- 172.21.15.146:0/1323788933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09c40c61b0 0x7f09c415d420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f09c406ebb0 tx=0x7f09ac009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c020143c73a3e2e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c1d9b700 1 -- 172.21.15.146:0/1323788933 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac00b8a0 con 0x7f09c40c61b0 2024-09-13T18:34:09.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c1d9b700 1 -- 172.21.15.146:0/1323788933 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ac00ba00 con 0x7f09c40c61b0 2024-09-13T18:34:09.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c1d9b700 1 -- 172.21.15.146:0/1323788933 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac00bd90 con 0x7f09c40c61b0 2024-09-13T18:34:09.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c359e700 1 -- 172.21.15.146:0/1323788933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09c40c61b0 msgr2=0x7f09c415d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c359e700 1 --2- 172.21.15.146:0/1323788933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09c40c61b0 0x7f09c415d420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f09c406ebb0 tx=0x7f09ac009bf0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c359e700 1 -- 172.21.15.146:0/1323788933 shutdown_connections 2024-09-13T18:34:09.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c359e700 1 --2- 172.21.15.146:0/1323788933 >> v2:172.21.15.154:3300/0 conn(0x7f09c40c6590 0x7f09c406cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.922+0000 7f09c359e700 1 --2- 172.21.15.146:0/1323788933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09c40c61b0 0x7f09c415d420 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c359e700 1 -- 172.21.15.146:0/1323788933 >> 172.21.15.146:0/1323788933 conn(0x7f09c4066b00 msgr2=0x7f09c406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c359e700 1 -- 172.21.15.146:0/1323788933 shutdown_connections 2024-09-13T18:34:09.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c359e700 1 -- 172.21.15.146:0/1323788933 wait complete. 2024-09-13T18:34:09.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c359e700 1 Processor -- start 2024-09-13T18:34:09.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c359e700 1 -- start start 2024-09-13T18:34:09.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09c406d2c0 0x7f09c415a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09c40c61b0 0x7f09c41584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09c40c6590 0x7f09c4148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c41589f0 con 0x7f09c40c6590 2024-09-13T18:34:09.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c41f1ea0 con 0x7f09c406d2c0 2024-09-13T18:34:09.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c4157440 con 0x7f09c40c61b0 2024-09-13T18:34:09.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c2d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09c40c6590 0x7f09c4148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09bbfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09c40c61b0 0x7f09c41584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c2d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09c40c6590 0x7f09c4148250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35586/0 (socket says 172.21.15.146:35586) 2024-09-13T18:34:09.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c2d9d700 1 -- 172.21.15.146:0/3619967452 learned_addr learned my addr 172.21.15.146:0/3619967452 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:09.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.926+0000 7f09c259c700 1 --2- 172.21.15.146:0/3619967452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09c406d2c0 0x7f09c415a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.930+0000 7f09c2d9d700 1 -- 172.21.15.146:0/3619967452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09c40c61b0 msgr2=0x7f09c41584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.930+0000 7f09c2d9d700 1 --2- 172.21.15.146:0/3619967452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09c40c61b0 0x7f09c41584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.930+0000 7f09c2d9d700 1 -- 172.21.15.146:0/3619967452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09c406d2c0 msgr2=0x7f09c415a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.930+0000 7f09c2d9d700 1 --2- 172.21.15.146:0/3619967452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09c406d2c0 0x7f09c415a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.930+0000 7f09c2d9d700 1 -- 172.21.15.146:0/3619967452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac009570 con 0x7f09c40c6590 2024-09-13T18:34:09.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.930+0000 7f09c2d9d700 1 --2- 172.21.15.146:0/3619967452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09c40c6590 0x7f09c4148250 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00cb50 tx=0x7f09bc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.930+0000 7f09baffd700 1 -- 172.21.15.146:0/3619967452 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc003c20 con 0x7f09c40c6590 2024-09-13T18:34:09.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.930+0000 7f09baffd700 1 -- 172.21.15.146:0/3619967452 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc003d80 con 0x7f09c40c6590 2024-09-13T18:34:09.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.930+0000 7f09baffd700 1 -- 172.21.15.146:0/3619967452 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00aa20 con 0x7f09c40c6590 2024-09-13T18:34:09.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.930+0000 7f09c359e700 1 -- 172.21.15.146:0/3619967452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c4157720 con 0x7f09c40c6590 2024-09-13T18:34:09.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.930+0000 7f09c359e700 1 -- 172.21.15.146:0/3619967452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c41f2390 con 0x7f09c40c6590 2024-09-13T18:34:09.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.930+0000 7f09c359e700 1 -- 172.21.15.146:0/3619967452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c4159f50 con 0x7f09c40c6590 2024-09-13T18:34:09.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.930+0000 7f09baffd700 1 -- 172.21.15.146:0/3619967452 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f09bc00ab80 con 0x7f09c40c6590 2024-09-13T18:34:09.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.934+0000 7f09baffd700 1 --2- 172.21.15.146:0/3619967452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f09a4066730 0x7f09a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.934+0000 7f09baffd700 1 -- 172.21.15.146:0/3619967452 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f09bc084460 con 0x7f09c40c6590 2024-09-13T18:34:09.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.934+0000 7f09c259c700 1 --2- 172.21.15.146:0/3619967452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f09a4066730 0x7f09a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.934+0000 7f09c259c700 1 --2- 172.21.15.146:0/3619967452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f09a4066730 0x7f09a4068bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f09ac000c00 tx=0x7f09ac008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:09.938+0000 7f09baffd700 1 -- 172.21.15.146:0/3619967452 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f09bc051350 con 0x7f09c40c6590 2024-09-13T18:34:10.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.174+0000 7f09c359e700 1 -- 172.21.15.146:0/3619967452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09c4066780 con 0x7f09c40c6590 2024-09-13T18:34:10.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.178+0000 7f09baffd700 1 -- 172.21.15.146:0/3619967452 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f09c4066780 con 0x7f09c40c6590 2024-09-13T18:34:10.190 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:10.190 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:17.345742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:10.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.178+0000 7f09c359e700 1 -- 172.21.15.146:0/3619967452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f09a4066730 msgr2=0x7f09a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.178+0000 7f09c359e700 1 --2- 172.21.15.146:0/3619967452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f09a4066730 0x7f09a4068bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f09ac000c00 tx=0x7f09ac008030 comp rx=0 tx=0).stop 2024-09-13T18:34:10.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.178+0000 7f09c359e700 1 -- 172.21.15.146:0/3619967452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09c40c6590 msgr2=0x7f09c4148250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.178+0000 7f09c359e700 1 --2- 172.21.15.146:0/3619967452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09c40c6590 0x7f09c4148250 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00cb50 tx=0x7f09bc00ce60 comp rx=0 tx=0).stop 2024-09-13T18:34:10.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.178+0000 7f09c359e700 1 -- 172.21.15.146:0/3619967452 shutdown_connections 2024-09-13T18:34:10.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.178+0000 7f09c359e700 1 --2- 172.21.15.146:0/3619967452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f09a4066730 0x7f09a4068bf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.178+0000 7f09c359e700 1 --2- 172.21.15.146:0/3619967452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09c40c6590 0x7f09c4148250 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.178+0000 7f09c359e700 1 --2- 172.21.15.146:0/3619967452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09c40c61b0 0x7f09c41584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.178+0000 7f09c359e700 1 --2- 172.21.15.146:0/3619967452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09c406d2c0 0x7f09c415a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.178+0000 7f09c359e700 1 -- 172.21.15.146:0/3619967452 >> 172.21.15.146:0/3619967452 conn(0x7f09c4066b00 msgr2=0x7f09c414f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.178+0000 7f09c359e700 1 -- 172.21.15.146:0/3619967452 shutdown_connections 2024-09-13T18:34:10.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:10.178+0000 7f09c359e700 1 -- 172.21.15.146:0/3619967452 wait complete. 2024-09-13T18:34:10.195 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 39 2024-09-13T18:34:10.208 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:34:12.210 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:12.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.318+0000 7faaf96b0700 1 Processor -- start 2024-09-13T18:34:12.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.318+0000 7faaf96b0700 1 -- start start 2024-09-13T18:34:12.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf96b0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faaf40c57f0 0x7faaf40cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf96b0700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faaf40cf450 0x7faaf40cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf96b0700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf40c5bd0 con 0x7faaf40ca020 2024-09-13T18:34:12.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf96b0700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf415a160 con 0x7faaf40cf450 2024-09-13T18:34:12.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf96b0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf4159830 con 0x7faaf40c57f0 2024-09-13T18:34:12.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf37fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faaf40cf450 0x7faaf40cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf37fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faaf40cf450 0x7faaf40cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55420/0 (socket says 172.21.15.146:55420) 2024-09-13T18:34:12.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf37fe700 1 -- 172.21.15.146:0/3333313100 learned_addr learned my addr 172.21.15.146:0/3333313100 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:12.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf2ffd700 1 --2- 172.21.15.146:0/3333313100 >> v2:172.21.15.154:3300/0 conn(0x7faaf40c57f0 0x7faaf40cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf37fe700 1 -- 172.21.15.146:0/3333313100 >> v2:172.21.15.154:3300/0 conn(0x7faaf40c57f0 msgr2=0x7faaf40cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:12.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf37fe700 1 --2- 172.21.15.146:0/3333313100 >> v2:172.21.15.154:3300/0 conn(0x7faaf40c57f0 0x7faaf40cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf37fe700 1 -- 172.21.15.146:0/3333313100 >> v1:172.21.15.146:6789/0 conn(0x7faaf40ca020 legacy=0x7faaf40ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf27fc700 1 -- 172.21.15.146:0/3333313100 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537059703 0 0) 0x7faaf40c5bd0 con 0x7faaf40ca020 2024-09-13T18:34:12.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf37fe700 1 -- 172.21.15.146:0/3333313100 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf40c9bb0 con 0x7faaf40cf450 2024-09-13T18:34:12.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf37fe700 1 --2- 172.21.15.146:0/3333313100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40cf450 0x7faaf40cf8a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7faaf4072b40 tx=0x7faaec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a4bc628e074103 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:12.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf27fc700 1 -- 172.21.15.146:0/3333313100 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec00e070 con 0x7faaf40cf450 2024-09-13T18:34:12.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf27fc700 1 -- 172.21.15.146:0/3333313100 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaec00b910 con 0x7faaf40cf450 2024-09-13T18:34:12.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf27fc700 1 -- 172.21.15.146:0/3333313100 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec00bcc0 con 0x7faaf40cf450 2024-09-13T18:34:12.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf96b0700 1 -- 172.21.15.146:0/3333313100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40cf450 msgr2=0x7faaf40cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf96b0700 1 --2- 172.21.15.146:0/3333313100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40cf450 0x7faaf40cf8a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7faaf4072b40 tx=0x7faaec00b330 comp rx=0 tx=0).stop 2024-09-13T18:34:12.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf96b0700 1 -- 172.21.15.146:0/3333313100 shutdown_connections 2024-09-13T18:34:12.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf96b0700 1 --2- 172.21.15.146:0/3333313100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40cf450 0x7faaf40cf8a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf96b0700 1 --2- 172.21.15.146:0/3333313100 >> v2:172.21.15.154:3300/0 conn(0x7faaf40c57f0 0x7faaf40cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf96b0700 1 -- 172.21.15.146:0/3333313100 >> 172.21.15.146:0/3333313100 conn(0x7faaf4071d30 msgr2=0x7faaf4072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:12.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf96b0700 1 -- 172.21.15.146:0/3333313100 shutdown_connections 2024-09-13T18:34:12.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.322+0000 7faaf96b0700 1 -- 172.21.15.146:0/3333313100 wait complete. 2024-09-13T18:34:12.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaf96b0700 1 Processor -- start 2024-09-13T18:34:12.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaf96b0700 1 -- start start 2024-09-13T18:34:12.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaf96b0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf40c57f0 0x7faaf41dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaf96b0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40ca020 0x7faaf41e3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaf96b0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaf40cf450 0x7faaf41e3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaf96b0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf41f3170 con 0x7faaf40ca020 2024-09-13T18:34:12.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaf96b0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf41e03c0 con 0x7faaf40cf450 2024-09-13T18:34:12.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaf96b0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf41e05c0 con 0x7faaf40c57f0 2024-09-13T18:34:12.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaebfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40ca020 0x7faaf41e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaebfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40ca020 0x7faaf41e3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55436/0 (socket says 172.21.15.146:55436) 2024-09-13T18:34:12.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaebfff700 1 -- 172.21.15.146:0/1268661340 learned_addr learned my addr 172.21.15.146:0/1268661340 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:12.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaf2ffd700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf40c57f0 0x7faaf41dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaebfff700 1 -- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf40c57f0 msgr2=0x7faaf41dfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaebfff700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf40c57f0 0x7faaf41dfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaebfff700 1 -- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaf40cf450 msgr2=0x7faaf41e3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaf37fe700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaf40cf450 0x7faaf41e3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaebfff700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaf40cf450 0x7faaf41e3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaebfff700 1 -- 172.21.15.146:0/1268661340 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec009b30 con 0x7faaf40ca020 2024-09-13T18:34:12.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaf37fe700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaf40cf450 0x7faaf41e3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:12.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.326+0000 7faaebfff700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40ca020 0x7faaf41e3250 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7faadc0011e0 tx=0x7faadc00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:12.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.330+0000 7faaeb7fe700 1 -- 172.21.15.146:0/1268661340 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc0054e0 con 0x7faaf40ca020 2024-09-13T18:34:12.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.330+0000 7faaeb7fe700 1 -- 172.21.15.146:0/1268661340 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faadc0036a0 con 0x7faaf40ca020 2024-09-13T18:34:12.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.330+0000 7faaf96b0700 1 -- 172.21.15.146:0/1268661340 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf41e08a0 con 0x7faaf40ca020 2024-09-13T18:34:12.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.330+0000 7faaf96b0700 1 -- 172.21.15.146:0/1268661340 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf41e0e60 con 0x7faaf40ca020 2024-09-13T18:34:12.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.330+0000 7faaeb7fe700 1 -- 172.21.15.146:0/1268661340 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc003a50 con 0x7faaf40ca020 2024-09-13T18:34:12.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.330+0000 7faaf96b0700 1 -- 172.21.15.146:0/1268661340 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad8005310 con 0x7faaf40ca020 2024-09-13T18:34:12.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.330+0000 7faaeb7fe700 1 -- 172.21.15.146:0/1268661340 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faadc003bb0 con 0x7faaf40ca020 2024-09-13T18:34:12.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.330+0000 7faaeb7fe700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faad4066760 0x7faad4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.330+0000 7faaeb7fe700 1 -- 172.21.15.146:0/1268661340 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7faadc083b30 con 0x7faaf40ca020 2024-09-13T18:34:12.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.330+0000 7faaf2ffd700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faad4066760 0x7faad4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.334+0000 7faaf2ffd700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faad4066760 0x7faad4068c20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faae4009b50 tx=0x7faae4009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:12.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.338+0000 7faaeb7fe700 1 -- 172.21.15.146:0/1268661340 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faadc050980 con 0x7faaf40ca020 2024-09-13T18:34:12.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.582+0000 7faaf96b0700 1 -- 172.21.15.146:0/1268661340 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faad8005b40 con 0x7faaf40ca020 2024-09-13T18:34:12.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.586+0000 7faaeb7fe700 1 -- 172.21.15.146:0/1268661340 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6296 (secure 0 0 0) 0x7faadc003e60 con 0x7faaf40ca020 2024-09-13T18:34:12.598 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:12.599 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:17.345742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:12.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.586+0000 7faaf96b0700 1 -- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faad4066760 msgr2=0x7faad4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.586+0000 7faaf96b0700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faad4066760 0x7faad4068c20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faae4009b50 tx=0x7faae4009240 comp rx=0 tx=0).stop 2024-09-13T18:34:12.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.586+0000 7faaf96b0700 1 -- 172.21.15.146:0/1268661340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40ca020 msgr2=0x7faaf41e3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.586+0000 7faaf96b0700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40ca020 0x7faaf41e3250 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7faadc0011e0 tx=0x7faadc00ca30 comp rx=0 tx=0).stop 2024-09-13T18:34:12.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.586+0000 7faaf96b0700 1 -- 172.21.15.146:0/1268661340 shutdown_connections 2024-09-13T18:34:12.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.586+0000 7faaf96b0700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faad4066760 0x7faad4068c20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.586+0000 7faaf96b0700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaf40cf450 0x7faaf41e3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.586+0000 7faaf96b0700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40ca020 0x7faaf41e3250 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.586+0000 7faaf96b0700 1 --2- 172.21.15.146:0/1268661340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf40c57f0 0x7faaf41dfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.586+0000 7faaf96b0700 1 -- 172.21.15.146:0/1268661340 >> 172.21.15.146:0/1268661340 conn(0x7faaf4071d30 msgr2=0x7faaf40ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:12.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.586+0000 7faaf96b0700 1 -- 172.21.15.146:0/1268661340 shutdown_connections 2024-09-13T18:34:12.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:12.586+0000 7faaf96b0700 1 -- 172.21.15.146:0/1268661340 wait complete. 2024-09-13T18:34:12.602 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 39 2024-09-13T18:34:12.618 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:34:14.619 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.730+0000 7f19aab2f700 1 Processor -- start 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.730+0000 7f19aab2f700 1 -- start start 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19aab2f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f199c0d5740 0x7f199c0dbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19aab2f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199c009040 con 0x7f199c008860 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19aab2f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199c045cf0 con 0x7f199c008c40 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19aab2f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f199c036e30 con 0x7f199c0d5740 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19a932c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f199c008860 0x7f199c04cb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45716/0 (socket says 172.21.15.146:45716) 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19a932c700 1 -- 172.21.15.146:0/173288073 learned_addr learned my addr 172.21.15.146:0/173288073 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19a9b2d700 1 --2- 172.21.15.146:0/173288073 >> v2:172.21.15.146:3300/0 conn(0x7f199c0d5740 0x7f199c0dbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19a9b2d700 1 -- 172.21.15.146:0/173288073 >> v1:172.21.15.154:6789/0 conn(0x7f199c008c40 legacy=0x7f199c04bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19a9b2d700 1 -- 172.21.15.146:0/173288073 >> v1:172.21.15.146:6789/0 conn(0x7f199c008860 legacy=0x7f199c04cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19a9b2d700 1 -- 172.21.15.146:0/173288073 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c007800 con 0x7f199c0d5740 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19a9b2d700 1 --2- 172.21.15.146:0/173288073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f199c0d5740 0x7f199c0dbe80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009850 tx=0x7f19a0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fea2863ac5c524d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19a8b2b700 1 -- 172.21.15.146:0/173288073 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0011830 con 0x7f199c0d5740 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19a8b2b700 1 -- 172.21.15.146:0/173288073 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a0011990 con 0x7f199c0d5740 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19a8b2b700 1 -- 172.21.15.146:0/173288073 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0011d20 con 0x7f199c0d5740 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19aab2f700 1 -- 172.21.15.146:0/173288073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f199c0d5740 msgr2=0x7f199c0dbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19aab2f700 1 --2- 172.21.15.146:0/173288073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f199c0d5740 0x7f199c0dbe80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009850 tx=0x7f19a0009b60 comp rx=0 tx=0).stop 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19aab2f700 1 -- 172.21.15.146:0/173288073 shutdown_connections 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19aab2f700 1 --2- 172.21.15.146:0/173288073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f199c0d5740 0x7f199c0dbe80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19aab2f700 1 -- 172.21.15.146:0/173288073 >> 172.21.15.146:0/173288073 conn(0x7f199c050e10 msgr2=0x7f199c051220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19aab2f700 1 -- 172.21.15.146:0/173288073 shutdown_connections 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.734+0000 7f19aab2f700 1 -- 172.21.15.146:0/173288073 wait complete. 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aab2f700 1 Processor -- start 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aab2f700 1 -- start start 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aab2f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f199c008860 0x7f199c030f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aab2f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f199c008c40 0x7f199c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aab2f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f199c0d5740 0x7f199c0ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aab2f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c008440 con 0x7f199c0d5740 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aab2f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c1752a0 con 0x7f199c008860 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aab2f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c031b00 con 0x7f199c008c40 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aa32e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f199c0d5740 0x7f199c0ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aa32e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f199c0d5740 0x7f199c0ced60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55454/0 (socket says 172.21.15.146:55454) 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aa32e700 1 -- 172.21.15.146:0/1927670674 learned_addr learned my addr 172.21.15.146:0/1927670674 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19a9b2d700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f199c008860 0x7f199c030f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19a932c700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f199c008c40 0x7f199c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aa32e700 1 -- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f199c008c40 msgr2=0x7f199c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aa32e700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f199c008c40 0x7f199c0ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aa32e700 1 -- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f199c008860 msgr2=0x7f199c030f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aa32e700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f199c008860 0x7f199c030f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aa32e700 1 -- 172.21.15.146:0/1927670674 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a00094b0 con 0x7f199c0d5740 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19a9b2d700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f199c008860 0x7f199c030f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aa32e700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f199c0d5740 0x7f199c0ced60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f19a4068650 tx=0x7f19a4075d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f199a7fc700 1 -- 172.21.15.146:0/1927670674 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4067e50 con 0x7f199c0d5740 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aab2f700 1 -- 172.21.15.146:0/1927670674 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199c0d1c00 con 0x7f199c0d5740 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f19aab2f700 1 -- 172.21.15.146:0/1927670674 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199c0d2180 con 0x7f199c0d5740 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f199a7fc700 1 -- 172.21.15.146:0/1927670674 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4075e30 con 0x7f199c0d5740 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.738+0000 7f199a7fc700 1 -- 172.21.15.146:0/1927670674 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a404f030 con 0x7f199c0d5740 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.742+0000 7f19aab2f700 1 -- 172.21.15.146:0/1927670674 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198c005310 con 0x7f199c0d5740 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.742+0000 7f199a7fc700 1 -- 172.21.15.146:0/1927670674 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f19a4070740 con 0x7f199c0d5740 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.742+0000 7f199a7fc700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1988066730 0x7f1988068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.742+0000 7f199a7fc700 1 -- 172.21.15.146:0/1927670674 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f19a40ff020 con 0x7f199c0d5740 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.742+0000 7f19a9b2d700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1988066730 0x7f1988068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.742+0000 7f19a9b2d700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1988066730 0x7f1988068bf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009fd0 tx=0x7f19a0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.750+0000 7f199a7fc700 1 -- 172.21.15.146:0/1927670674 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f19a40cbe80 con 0x7f199c0d5740 2024-09-13T18:34:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.982+0000 7f19aab2f700 1 -- 172.21.15.146:0/1927670674 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f198c0055c0 con 0x7f199c0d5740 2024-09-13T18:34:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.982+0000 7f199a7fc700 1 -- 172.21.15.146:0/1927670674 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f19a40cf4a0 con 0x7f199c0d5740 2024-09-13T18:34:14.997 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:14.997 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:17.345742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.986+0000 7f19aab2f700 1 -- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1988066730 msgr2=0x7f1988068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.986+0000 7f19aab2f700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1988066730 0x7f1988068bf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009fd0 tx=0x7f19a0008030 comp rx=0 tx=0).stop 2024-09-13T18:34:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.986+0000 7f19aab2f700 1 -- 172.21.15.146:0/1927670674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f199c0d5740 msgr2=0x7f199c0ced60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.986+0000 7f19aab2f700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f199c0d5740 0x7f199c0ced60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f19a4068650 tx=0x7f19a4075d10 comp rx=0 tx=0).stop 2024-09-13T18:34:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.986+0000 7f19aab2f700 1 -- 172.21.15.146:0/1927670674 shutdown_connections 2024-09-13T18:34:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.986+0000 7f19aab2f700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1988066730 0x7f1988068bf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.986+0000 7f19aab2f700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f199c0d5740 0x7f199c0ced60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.986+0000 7f19aab2f700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f199c008c40 0x7f199c0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.986+0000 7f19aab2f700 1 --2- 172.21.15.146:0/1927670674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f199c008860 0x7f199c030f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.986+0000 7f19aab2f700 1 -- 172.21.15.146:0/1927670674 >> 172.21.15.146:0/1927670674 conn(0x7f199c050e10 msgr2=0x7f199c005150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.986+0000 7f19aab2f700 1 -- 172.21.15.146:0/1927670674 shutdown_connections 2024-09-13T18:34:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:14.986+0000 7f19aab2f700 1 -- 172.21.15.146:0/1927670674 wait complete. 2024-09-13T18:34:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 39 2024-09-13T18:34:15.011 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:34:17.012 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:17.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958c966700 1 Processor -- start 2024-09-13T18:34:17.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958c966700 1 -- start start 2024-09-13T18:34:17.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958c966700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9588153860 0x7f95880c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:17.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958c966700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f95880c7970 0x7f958806c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:17.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958c966700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f958806c5b0 0x7f9588070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:17.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958c966700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95880c4200 con 0x7f958806c5b0 2024-09-13T18:34:17.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958c966700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95880c98d0 con 0x7f95880c7970 2024-09-13T18:34:17.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958c966700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95880c8b20 con 0x7f9588153860 2024-09-13T18:34:17.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958759e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f958806c5b0 0x7f9588070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:17.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958759e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f958806c5b0 0x7f9588070bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55468/0 (socket says 172.21.15.146:55468) 2024-09-13T18:34:17.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958759e700 1 -- 172.21.15.146:0/152696191 learned_addr learned my addr 172.21.15.146:0/152696191 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:17.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958659c700 1 --2- 172.21.15.146:0/152696191 >> v2:172.21.15.154:3300/0 conn(0x7f95880c7970 0x7f958806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:17.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f9586d9d700 1 --2- 172.21.15.146:0/152696191 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9588153860 0x7f95880c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:17.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958759e700 1 -- 172.21.15.146:0/152696191 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9588153860 msgr2=0x7f95880c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:17.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958759e700 1 --2- 172.21.15.146:0/152696191 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9588153860 0x7f95880c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:17.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958759e700 1 -- 172.21.15.146:0/152696191 >> v2:172.21.15.154:3300/0 conn(0x7f95880c7970 msgr2=0x7f958806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:17.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958759e700 1 --2- 172.21.15.146:0/152696191 >> v2:172.21.15.154:3300/0 conn(0x7f95880c7970 0x7f958806c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:17.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958759e700 1 -- 172.21.15.146:0/152696191 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880706d0 con 0x7f958806c5b0 2024-09-13T18:34:17.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958659c700 1 --2- 172.21.15.146:0/152696191 >> v2:172.21.15.154:3300/0 conn(0x7f95880c7970 0x7f958806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:17.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f958759e700 1 --2- 172.21.15.146:0/152696191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f958806c5b0 0x7f9588070bd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9588070240 tx=0x7f958000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2507c8302fb11728 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:17.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f9585d9b700 1 -- 172.21.15.146:0/152696191 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958000e070 con 0x7f958806c5b0 2024-09-13T18:34:17.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f9585d9b700 1 -- 172.21.15.146:0/152696191 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958000b9c0 con 0x7f958806c5b0 2024-09-13T18:34:17.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.118+0000 7f9585d9b700 1 -- 172.21.15.146:0/152696191 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958000bd70 con 0x7f958806c5b0 2024-09-13T18:34:17.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 -- 172.21.15.146:0/152696191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f958806c5b0 msgr2=0x7f9588070bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:17.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 --2- 172.21.15.146:0/152696191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f958806c5b0 0x7f9588070bd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9588070240 tx=0x7f958000b330 comp rx=0 tx=0).stop 2024-09-13T18:34:17.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 -- 172.21.15.146:0/152696191 shutdown_connections 2024-09-13T18:34:17.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 --2- 172.21.15.146:0/152696191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f958806c5b0 0x7f9588070bd0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:17.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 --2- 172.21.15.146:0/152696191 >> v2:172.21.15.154:3300/0 conn(0x7f95880c7970 0x7f958806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:17.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 --2- 172.21.15.146:0/152696191 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9588153860 0x7f95880c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:17.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 -- 172.21.15.146:0/152696191 >> 172.21.15.146:0/152696191 conn(0x7f95880ccf70 msgr2=0x7f95880cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:17.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 -- 172.21.15.146:0/152696191 shutdown_connections 2024-09-13T18:34:17.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 -- 172.21.15.146:0/152696191 wait complete. 2024-09-13T18:34:17.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 Processor -- start 2024-09-13T18:34:17.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 -- start start 2024-09-13T18:34:17.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f958806c5b0 0x7f958814aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:17.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95880c7970 0x7f95881e5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:17.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9588153860 0x7f95881e38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:17.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95881e5030 con 0x7f95880c7970 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958814d910 con 0x7f9588153860 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958c966700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958814d630 con 0x7f958806c5b0 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95880c7970 0x7f95881e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95880c7970 0x7f95881e5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55470/0 (socket says 172.21.15.146:55470) 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958659c700 1 -- 172.21.15.146:0/1810122589 learned_addr learned my addr 172.21.15.146:0/1810122589 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f9586d9d700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f958806c5b0 0x7f958814aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958759e700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9588153860 0x7f95881e38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958659c700 1 -- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f958806c5b0 msgr2=0x7f958814aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958659c700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f958806c5b0 0x7f958814aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958659c700 1 -- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9588153860 msgr2=0x7f95881e38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958659c700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9588153860 0x7f95881e38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958659c700 1 -- 172.21.15.146:0/1810122589 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9580009b30 con 0x7f95880c7970 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.122+0000 7f958759e700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9588153860 0x7f95881e38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:17.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.126+0000 7f958659c700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95880c7970 0x7f95881e5450 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f957c00d800 tx=0x7f957c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:17.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.126+0000 7f956f7fe700 1 -- 172.21.15.146:0/1810122589 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c0125f0 con 0x7f95880c7970 2024-09-13T18:34:17.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.126+0000 7f956f7fe700 1 -- 172.21.15.146:0/1810122589 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957c012750 con 0x7f95880c7970 2024-09-13T18:34:17.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.126+0000 7f958c966700 1 -- 172.21.15.146:0/1810122589 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95881e4690 con 0x7f95880c7970 2024-09-13T18:34:17.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.126+0000 7f956f7fe700 1 -- 172.21.15.146:0/1810122589 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c012b00 con 0x7f95880c7970 2024-09-13T18:34:17.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.126+0000 7f958c966700 1 -- 172.21.15.146:0/1810122589 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95881e4c50 con 0x7f95880c7970 2024-09-13T18:34:17.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.126+0000 7f958c966700 1 -- 172.21.15.146:0/1810122589 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9588067a60 con 0x7f95880c7970 2024-09-13T18:34:17.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.126+0000 7f956f7fe700 1 -- 172.21.15.146:0/1810122589 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f957c012c60 con 0x7f95880c7970 2024-09-13T18:34:17.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.126+0000 7f956f7fe700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9574066730 0x7f9574068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:17.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.126+0000 7f956f7fe700 1 -- 172.21.15.146:0/1810122589 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f957c084940 con 0x7f95880c7970 2024-09-13T18:34:17.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.126+0000 7f9586d9d700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9574066730 0x7f9574068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:17.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.130+0000 7f9586d9d700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9574066730 0x7f9574068bf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9578006b40 tx=0x7f9578008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:17.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.138+0000 7f956f7fe700 1 -- 172.21.15.146:0/1810122589 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f957c051710 con 0x7f95880c7970 2024-09-13T18:34:17.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.362+0000 7f958c966700 1 -- 172.21.15.146:0/1810122589 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f958815a2b0 con 0x7f95880c7970 2024-09-13T18:34:17.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.362+0000 7f956f7fe700 1 -- 172.21.15.146:0/1810122589 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f957c054d30 con 0x7f95880c7970 2024-09-13T18:34:17.376 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:17.376 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:17.345742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:17.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.366+0000 7f958c966700 1 -- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9574066730 msgr2=0x7f9574068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:17.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.366+0000 7f958c966700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9574066730 0x7f9574068bf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9578006b40 tx=0x7f9578008040 comp rx=0 tx=0).stop 2024-09-13T18:34:17.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.366+0000 7f958c966700 1 -- 172.21.15.146:0/1810122589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95880c7970 msgr2=0x7f95881e5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:17.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.366+0000 7f958c966700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95880c7970 0x7f95881e5450 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f957c00d800 tx=0x7f957c00db10 comp rx=0 tx=0).stop 2024-09-13T18:34:17.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.366+0000 7f958c966700 1 -- 172.21.15.146:0/1810122589 shutdown_connections 2024-09-13T18:34:17.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.366+0000 7f958c966700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9574066730 0x7f9574068bf0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:17.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.366+0000 7f958c966700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9588153860 0x7f95881e38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:17.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.366+0000 7f958c966700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95880c7970 0x7f95881e5450 secure :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f957c00d800 tx=0x7f957c00db10 comp rx=0 tx=0).stop 2024-09-13T18:34:17.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.366+0000 7f958c966700 1 --2- 172.21.15.146:0/1810122589 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f958806c5b0 0x7f958814aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:17.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.366+0000 7f958c966700 1 -- 172.21.15.146:0/1810122589 >> 172.21.15.146:0/1810122589 conn(0x7f95880ccf70 msgr2=0x7f95880cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:17.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.366+0000 7f958c966700 1 -- 172.21.15.146:0/1810122589 shutdown_connections 2024-09-13T18:34:17.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:17.366+0000 7f958c966700 1 -- 172.21.15.146:0/1810122589 wait complete. 2024-09-13T18:34:17.381 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 39 2024-09-13T18:34:17.394 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:34:19.395 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:19.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0b59e700 1 Processor -- start 2024-09-13T18:34:19.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0b59e700 1 -- start start 2024-09-13T18:34:19.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14fb50 0x7f7c0c0c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:19.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0b59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7c0c0c6770 0x7f7c0c0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:19.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0b59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c0c156810 con 0x7f7c0c0c7b00 2024-09-13T18:34:19.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0b59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0c1559b0 con 0x7f7c0c0c6770 2024-09-13T18:34:19.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c155c90 con 0x7f7c0c14fb50 2024-09-13T18:34:19.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c09d9b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f7c0c0c7b00 0x7f7c0c0c6060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45724/0 (socket says 172.21.15.146:45724) 2024-09-13T18:34:19.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c09d9b700 1 -- 172.21.15.146:0/376454027 learned_addr learned my addr 172.21.15.146:0/376454027 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:34:19.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0a59c700 1 --2- 172.21.15.146:0/376454027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14fb50 0x7f7c0c0c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:19.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0ad9d700 1 --2- 172.21.15.146:0/376454027 >> v2:172.21.15.154:3300/0 conn(0x7f7c0c0c6770 0x7f7c0c0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:19.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352582468 0 0) 0x7f7c0c156810 con 0x7f7c0c0c7b00 2024-09-13T18:34:19.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bf40035d0 con 0x7f7c0c0c7b00 2024-09-13T18:34:19.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1438166772 0 0) 0x7f7bf40035d0 con 0x7f7c0c0c7b00 2024-09-13T18:34:19.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14fb50 msgr2=0x7f7c0c0c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:19.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 --2- 172.21.15.146:0/376454027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14fb50 0x7f7c0c0c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:19.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 >> v2:172.21.15.154:3300/0 conn(0x7f7c0c0c6770 msgr2=0x7f7c0c0c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:19.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 --2- 172.21.15.146:0/376454027 >> v2:172.21.15.154:3300/0 conn(0x7f7c0c0c6770 0x7f7c0c0c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:19.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c155f70 con 0x7f7c0c0c7b00 2024-09-13T18:34:19.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f7c00001670 con 0x7f7c0c0c7b00 2024-09-13T18:34:19.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 >> v1:172.21.15.146:6789/0 conn(0x7f7c0c0c7b00 legacy=0x7f7c0c0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:19.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 --2- 172.21.15.146:0/376454027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14fb50 0x7f7bf4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:19.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 --2- 172.21.15.146:0/376454027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf4004010 0x7f7bf4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:19.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 --2- 172.21.15.146:0/376454027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf4004db0 0x7f7bf4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:19.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c1559b0 con 0x7f7bf4004db0 2024-09-13T18:34:19.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c155c90 con 0x7f7bf4004010 2024-09-13T18:34:19.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.506+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf400ba60 con 0x7f7c0c14fb50 2024-09-13T18:34:19.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0ad9d700 1 --2- 172.21.15.146:0/376454027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf4004db0 0x7f7bf4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:19.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c09d9b700 1 --2- 172.21.15.146:0/376454027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf4004010 0x7f7bf4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:19.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0a59c700 1 --2- 172.21.15.146:0/376454027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14fb50 0x7f7bf4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:19.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0ad9d700 1 -- 172.21.15.146:0/376454027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14fb50 msgr2=0x7f7bf4003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:19.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0ad9d700 1 --2- 172.21.15.146:0/376454027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14fb50 0x7f7bf4003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:19.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0ad9d700 1 -- 172.21.15.146:0/376454027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf4004010 msgr2=0x7f7bf4004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:19.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0ad9d700 1 --2- 172.21.15.146:0/376454027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf4004010 0x7f7bf4004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:19.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0ad9d700 1 -- 172.21.15.146:0/376454027 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf40035d0 con 0x7f7bf4004db0 2024-09-13T18:34:19.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0a59c700 1 --2- 172.21.15.146:0/376454027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14fb50 0x7f7bf4003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:19.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0ad9d700 1 --2- 172.21.15.146:0/376454027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf4004db0 0x7f7bf4005230 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009fc0 tx=0x7f7bf40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2641136a5c7cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:19.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0400ec30 con 0x7f7bf4004db0 2024-09-13T18:34:19.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0400f040 con 0x7f7bf4004db0 2024-09-13T18:34:19.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0959a700 1 -- 172.21.15.146:0/376454027 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c04014480 con 0x7f7bf4004db0 2024-09-13T18:34:19.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0b59e700 1 -- 172.21.15.146:0/376454027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf4004db0 msgr2=0x7f7bf4005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:19.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0b59e700 1 --2- 172.21.15.146:0/376454027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf4004db0 0x7f7bf4005230 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009fc0 tx=0x7f7bf40058a0 comp rx=0 tx=0).stop 2024-09-13T18:34:19.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0ad9d700 1 -- 172.21.15.146:0/376454027 reap_dead start 2024-09-13T18:34:19.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0b59e700 1 -- 172.21.15.146:0/376454027 shutdown_connections 2024-09-13T18:34:19.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0b59e700 1 -- 172.21.15.146:0/376454027 >> 172.21.15.146:0/376454027 conn(0x7f7c0c066b00 msgr2=0x7f7c0c06aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:19.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0b59e700 1 -- 172.21.15.146:0/376454027 shutdown_connections 2024-09-13T18:34:19.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0b59e700 1 -- 172.21.15.146:0/376454027 wait complete. 2024-09-13T18:34:19.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0b59e700 1 Processor -- start 2024-09-13T18:34:19.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.510+0000 7f7c0b59e700 1 -- start start 2024-09-13T18:34:19.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf4004db0 0x7f7c0c144330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:19.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0b59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c0c14be70 0x7f7c0c14e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:19.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14eea0 0x7f7c0c153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:19.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0b59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c154d00 con 0x7f7bf4004db0 2024-09-13T18:34:19.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0b59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c153490 con 0x7f7c0c14be70 2024-09-13T18:34:19.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c153130 con 0x7f7c0c14eea0 2024-09-13T18:34:19.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf4004db0 0x7f7c0c144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:19.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf4004db0 0x7f7c0c144330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55484/0 (socket says 172.21.15.146:55484) 2024-09-13T18:34:19.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0a59c700 1 -- 172.21.15.146:0/3880287174 learned_addr learned my addr 172.21.15.146:0/3880287174 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:19.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0ad9d700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14eea0 0x7f7c0c153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:19.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c09d9b700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c0c14be70 0x7f7c0c14e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:19.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0a59c700 1 -- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14eea0 msgr2=0x7f7c0c153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:19.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0a59c700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14eea0 0x7f7c0c153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:19.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0a59c700 1 -- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c0c14be70 msgr2=0x7f7c0c14e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:19.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0a59c700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c0c14be70 0x7f7c0c14e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:19.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0a59c700 1 -- 172.21.15.146:0/3880287174 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c144870 con 0x7f7bf4004db0 2024-09-13T18:34:19.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0a59c700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf4004db0 0x7f7c0c144330 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0109c0 tx=0x7f7bfc010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:19.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7bfaffd700 1 -- 172.21.15.146:0/3880287174 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc00ebd0 con 0x7f7bf4004db0 2024-09-13T18:34:19.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7bfaffd700 1 -- 172.21.15.146:0/3880287174 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bfc00ed30 con 0x7f7bf4004db0 2024-09-13T18:34:19.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0b59e700 1 -- 172.21.15.146:0/3880287174 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0c1501a0 con 0x7f7bf4004db0 2024-09-13T18:34:19.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7bfaffd700 1 -- 172.21.15.146:0/3880287174 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc004220 con 0x7f7bf4004db0 2024-09-13T18:34:19.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0b59e700 1 -- 172.21.15.146:0/3880287174 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0c1ed6e0 con 0x7f7bf4004db0 2024-09-13T18:34:19.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.514+0000 7f7c0b59e700 1 -- 172.21.15.146:0/3880287174 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0c14f280 con 0x7f7bf4004db0 2024-09-13T18:34:19.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.518+0000 7f7bfaffd700 1 -- 172.21.15.146:0/3880287174 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7bfc004380 con 0x7f7bf4004db0 2024-09-13T18:34:19.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.518+0000 7f7bfaffd700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7bec066760 0x7f7bec068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:19.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.518+0000 7f7bfaffd700 1 -- 172.21.15.146:0/3880287174 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f7bfc016070 con 0x7f7bf4004db0 2024-09-13T18:34:19.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.518+0000 7f7c09d9b700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7bec066760 0x7f7bec068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:19.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.518+0000 7f7c09d9b700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7bec066760 0x7f7bec068c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c06e840 tx=0x7f7c00009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:19.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.522+0000 7f7bfaffd700 1 -- 172.21.15.146:0/3880287174 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7bfc086e60 con 0x7f7bf4004db0 2024-09-13T18:34:19.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.762+0000 7f7c0b59e700 1 -- 172.21.15.146:0/3880287174 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c0c0c8360 con 0x7f7bf4004db0 2024-09-13T18:34:19.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.762+0000 7f7bfaffd700 1 -- 172.21.15.146:0/3880287174 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f7bfc051a60 con 0x7f7bf4004db0 2024-09-13T18:34:19.777 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:19.777 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:17.345742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:19.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.766+0000 7f7c0b59e700 1 -- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7bec066760 msgr2=0x7f7bec068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:19.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.766+0000 7f7c0b59e700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7bec066760 0x7f7bec068c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c06e840 tx=0x7f7c00009040 comp rx=0 tx=0).stop 2024-09-13T18:34:19.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.766+0000 7f7c0b59e700 1 -- 172.21.15.146:0/3880287174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf4004db0 msgr2=0x7f7c0c144330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:19.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.766+0000 7f7c0b59e700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf4004db0 0x7f7c0c144330 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0109c0 tx=0x7f7bfc010d80 comp rx=0 tx=0).stop 2024-09-13T18:34:19.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.766+0000 7f7c0b59e700 1 -- 172.21.15.146:0/3880287174 shutdown_connections 2024-09-13T18:34:19.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.766+0000 7f7c0b59e700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7bec066760 0x7f7bec068c20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:19.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.766+0000 7f7c0b59e700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c0c14eea0 0x7f7c0c153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:19.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.766+0000 7f7c0b59e700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c0c14be70 0x7f7c0c14e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:19.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.766+0000 7f7c0b59e700 1 --2- 172.21.15.146:0/3880287174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf4004db0 0x7f7c0c144330 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:19.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.766+0000 7f7c0b59e700 1 -- 172.21.15.146:0/3880287174 >> 172.21.15.146:0/3880287174 conn(0x7f7c0c066b00 msgr2=0x7f7c0c145360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:19.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.766+0000 7f7c0b59e700 1 -- 172.21.15.146:0/3880287174 shutdown_connections 2024-09-13T18:34:19.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:19.766+0000 7f7c0b59e700 1 -- 172.21.15.146:0/3880287174 wait complete. 2024-09-13T18:34:19.780 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 39 2024-09-13T18:34:19.791 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:34:21.792 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:21.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.894+0000 7f5707fff700 1 Processor -- start 2024-09-13T18:34:21.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.894+0000 7f5707fff700 1 -- start start 2024-09-13T18:34:21.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f57000d8b80 0x7f5700007950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:21.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f570004b7e0 0x7f5700045f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:21.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000e1cc0 0x7f57000e2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:21.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5700008110 con 0x7f57000d8b80 2024-09-13T18:34:21.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57000478d0 con 0x7f570004b7e0 2024-09-13T18:34:21.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570004bc10 con 0x7f57000e1cc0 2024-09-13T18:34:21.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5706ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f57000d8b80 0x7f5700007950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:21.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5706ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f57000d8b80 0x7f5700007950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37520/0 (socket says 172.21.15.146:37520) 2024-09-13T18:34:21.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5706ffd700 1 -- 172.21.15.146:0/1302176965 learned_addr learned my addr 172.21.15.146:0/1302176965 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:21.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f57077fe700 1 --2- 172.21.15.146:0/1302176965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000e1cc0 0x7f57000e2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:21.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f57067fc700 1 --2- 172.21.15.146:0/1302176965 >> v2:172.21.15.154:3300/0 conn(0x7f570004b7e0 0x7f5700045f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:21.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5706ffd700 1 -- 172.21.15.146:0/1302176965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000e1cc0 msgr2=0x7f57000e2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:21.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5706ffd700 1 --2- 172.21.15.146:0/1302176965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000e1cc0 0x7f57000e2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:21.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5706ffd700 1 -- 172.21.15.146:0/1302176965 >> v2:172.21.15.154:3300/0 conn(0x7f570004b7e0 msgr2=0x7f5700045f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:21.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5706ffd700 1 --2- 172.21.15.146:0/1302176965 >> v2:172.21.15.154:3300/0 conn(0x7f570004b7e0 0x7f5700045f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:21.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5706ffd700 1 -- 172.21.15.146:0/1302176965 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5700047b90 con 0x7f57000d8b80 2024-09-13T18:34:21.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5706ffd700 1 --2- 172.21.15.146:0/1302176965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57000d8b80 0x7f5700007950 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5700033ba0 tx=0x7f56f8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c664e95b007743 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:21.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5705ffb700 1 -- 172.21.15.146:0/1302176965 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f800b8a0 con 0x7f57000d8b80 2024-09-13T18:34:21.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5705ffb700 1 -- 172.21.15.146:0/1302176965 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f800ba00 con 0x7f57000d8b80 2024-09-13T18:34:21.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5705ffb700 1 -- 172.21.15.146:0/1302176965 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f800bd90 con 0x7f57000d8b80 2024-09-13T18:34:21.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 -- 172.21.15.146:0/1302176965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57000d8b80 msgr2=0x7f5700007950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:21.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 --2- 172.21.15.146:0/1302176965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57000d8b80 0x7f5700007950 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5700033ba0 tx=0x7f56f8009bf0 comp rx=0 tx=0).stop 2024-09-13T18:34:21.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 -- 172.21.15.146:0/1302176965 shutdown_connections 2024-09-13T18:34:21.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 --2- 172.21.15.146:0/1302176965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000e1cc0 0x7f57000e2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:21.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 --2- 172.21.15.146:0/1302176965 >> v2:172.21.15.154:3300/0 conn(0x7f570004b7e0 0x7f5700045f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:21.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 --2- 172.21.15.146:0/1302176965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57000d8b80 0x7f5700007950 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:21.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 -- 172.21.15.146:0/1302176965 >> 172.21.15.146:0/1302176965 conn(0x7f570004d8e0 msgr2=0x7f570004dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:21.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 -- 172.21.15.146:0/1302176965 shutdown_connections 2024-09-13T18:34:21.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.898+0000 7f5707fff700 1 -- 172.21.15.146:0/1302176965 wait complete. 2024-09-13T18:34:21.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.902+0000 7f5707fff700 1 Processor -- start 2024-09-13T18:34:21.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.902+0000 7f5707fff700 1 -- start start 2024-09-13T18:34:21.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.902+0000 7f5707fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f570004b7e0 0x7f570004a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:21.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.902+0000 7f5707fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57000d8b80 0x7f5700174240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:21.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.902+0000 7f5707fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000e1cc0 0x7f57000481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:21.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.902+0000 7f5707fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5700178180 con 0x7f57000d8b80 2024-09-13T18:34:21.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.902+0000 7f5707fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5700049f90 con 0x7f570004b7e0 2024-09-13T18:34:21.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.902+0000 7f5707fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5700178610 con 0x7f57000e1cc0 2024-09-13T18:34:21.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.902+0000 7f57067fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57000d8b80 0x7f5700174240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:21.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.902+0000 7f57067fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57000d8b80 0x7f5700174240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37534/0 (socket says 172.21.15.146:37534) 2024-09-13T18:34:21.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.902+0000 7f57067fc700 1 -- 172.21.15.146:0/3151819029 learned_addr learned my addr 172.21.15.146:0/3151819029 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:21.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.902+0000 7f5706ffd700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f570004b7e0 0x7f570004a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:21.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.902+0000 7f57077fe700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000e1cc0 0x7f57000481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:21.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f57067fc700 1 -- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000e1cc0 msgr2=0x7f57000481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:21.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f57067fc700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000e1cc0 0x7f57000481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:21.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f57067fc700 1 -- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f570004b7e0 msgr2=0x7f570004a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:21.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f57067fc700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f570004b7e0 0x7f570004a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:21.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f57067fc700 1 -- 172.21.15.146:0/3151819029 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8009570 con 0x7f57000d8b80 2024-09-13T18:34:21.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f57077fe700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000e1cc0 0x7f57000481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:21.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f57067fc700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57000d8b80 0x7f5700174240 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00d800 tx=0x7f56fc00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:21.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f56effff700 1 -- 172.21.15.146:0/3151819029 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc014620 con 0x7f57000d8b80 2024-09-13T18:34:21.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f56effff700 1 -- 172.21.15.146:0/3151819029 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc014780 con 0x7f57000d8b80 2024-09-13T18:34:21.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f5707fff700 1 -- 172.21.15.146:0/3151819029 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57000519f0 con 0x7f57000d8b80 2024-09-13T18:34:21.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f56effff700 1 -- 172.21.15.146:0/3151819029 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc014b10 con 0x7f57000d8b80 2024-09-13T18:34:21.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f5707fff700 1 -- 172.21.15.146:0/3151819029 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5700177850 con 0x7f57000d8b80 2024-09-13T18:34:21.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f5707fff700 1 -- 172.21.15.146:0/3151819029 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5700007430 con 0x7f57000d8b80 2024-09-13T18:34:21.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f56effff700 1 -- 172.21.15.146:0/3151819029 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f56fc014c70 con 0x7f57000d8b80 2024-09-13T18:34:21.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f56effff700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56f4066730 0x7f56f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:21.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.906+0000 7f56effff700 1 -- 172.21.15.146:0/3151819029 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f56fc084c50 con 0x7f57000d8b80 2024-09-13T18:34:21.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.910+0000 7f5706ffd700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56f4066730 0x7f56f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:21.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.910+0000 7f5706ffd700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56f4066730 0x7f56f4068bf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f56f8000c00 tx=0x7f56f8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:21.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:21.914+0000 7f56effff700 1 -- 172.21.15.146:0/3151819029 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f56fc08a020 con 0x7f57000d8b80 2024-09-13T18:34:22.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.186+0000 7f5707fff700 1 -- 172.21.15.146:0/3151819029 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5700046a60 con 0x7f57000d8b80 2024-09-13T18:34:22.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.186+0000 7f56effff700 1 -- 172.21.15.146:0/3151819029 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f56fc051a20 con 0x7f57000d8b80 2024-09-13T18:34:22.201 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:22.201 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:17.345742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:22.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.190+0000 7f5707fff700 1 -- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56f4066730 msgr2=0x7f56f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:22.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.190+0000 7f5707fff700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56f4066730 0x7f56f4068bf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f56f8000c00 tx=0x7f56f8008030 comp rx=0 tx=0).stop 2024-09-13T18:34:22.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.190+0000 7f5707fff700 1 -- 172.21.15.146:0/3151819029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57000d8b80 msgr2=0x7f5700174240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:22.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.190+0000 7f5707fff700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57000d8b80 0x7f5700174240 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00d800 tx=0x7f56fc00db10 comp rx=0 tx=0).stop 2024-09-13T18:34:22.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.190+0000 7f5707fff700 1 -- 172.21.15.146:0/3151819029 shutdown_connections 2024-09-13T18:34:22.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.190+0000 7f5707fff700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56f4066730 0x7f56f4068bf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:22.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.190+0000 7f5707fff700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000e1cc0 0x7f57000481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:22.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.190+0000 7f5707fff700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57000d8b80 0x7f5700174240 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:22.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.190+0000 7f5707fff700 1 --2- 172.21.15.146:0/3151819029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f570004b7e0 0x7f570004a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:22.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.190+0000 7f5707fff700 1 -- 172.21.15.146:0/3151819029 >> 172.21.15.146:0/3151819029 conn(0x7f570004d8e0 msgr2=0x7f5700045e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:22.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.190+0000 7f5707fff700 1 -- 172.21.15.146:0/3151819029 shutdown_connections 2024-09-13T18:34:22.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:22.190+0000 7f5707fff700 1 -- 172.21.15.146:0/3151819029 wait complete. 2024-09-13T18:34:22.205 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 39 2024-09-13T18:34:22.218 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:34:24.219 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:24.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f80aa688700 1 Processor -- start 2024-09-13T18:34:24.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f80aa688700 1 -- start start 2024-09-13T18:34:24.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f80aa688700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f80a40c7970 0x7f80a406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:24.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f80aa688700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a40c98d0 con 0x7f80a406c5b0 2024-09-13T18:34:24.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f80aa688700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a40706d0 con 0x7f80a4153860 2024-09-13T18:34:24.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f80aa688700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a40c4200 con 0x7f80a40c7970 2024-09-13T18:34:24.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f80a8e85700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f80a40c7970 0x7f80a406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:24.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f80a8e85700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f80a40c7970 0x7f80a406c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37550/0 (socket says 172.21.15.146:37550) 2024-09-13T18:34:24.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f80a8e85700 1 -- 172.21.15.146:0/1465516496 learned_addr learned my addr 172.21.15.146:0/1465516496 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:24.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f80a8e85700 1 -- 172.21.15.146:0/1465516496 >> v1:172.21.15.154:6789/0 conn(0x7f80a4153860 legacy=0x7f80a40c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:24.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f80a8e85700 1 -- 172.21.15.146:0/1465516496 >> v1:172.21.15.146:6789/0 conn(0x7f80a406c5b0 legacy=0x7f80a4070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:24.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f80a8e85700 1 -- 172.21.15.146:0/1465516496 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a4159730 con 0x7f80a40c7970 2024-09-13T18:34:24.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f80a8e85700 1 --2- 172.21.15.146:0/1465516496 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a40c7970 0x7f80a406c070 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8094009910 tx=0x7f8094009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d25797f3a22a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:24.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f809bfff700 1 -- 172.21.15.146:0/1465516496 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809400ba60 con 0x7f80a40c7970 2024-09-13T18:34:24.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f809bfff700 1 -- 172.21.15.146:0/1465516496 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809400bbc0 con 0x7f80a40c7970 2024-09-13T18:34:24.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.330+0000 7f809bfff700 1 -- 172.21.15.146:0/1465516496 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094009e60 con 0x7f80a40c7970 2024-09-13T18:34:24.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.334+0000 7f80aa688700 1 -- 172.21.15.146:0/1465516496 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a40c7970 msgr2=0x7f80a406c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:24.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.334+0000 7f80aa688700 1 --2- 172.21.15.146:0/1465516496 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a40c7970 0x7f80a406c070 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8094009910 tx=0x7f8094009c20 comp rx=0 tx=0).stop 2024-09-13T18:34:24.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.334+0000 7f80aa688700 1 -- 172.21.15.146:0/1465516496 shutdown_connections 2024-09-13T18:34:24.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.334+0000 7f80aa688700 1 --2- 172.21.15.146:0/1465516496 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a40c7970 0x7f80a406c070 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:24.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.334+0000 7f80aa688700 1 -- 172.21.15.146:0/1465516496 >> 172.21.15.146:0/1465516496 conn(0x7f80a40ccf70 msgr2=0x7f80a40cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:24.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.334+0000 7f80aa688700 1 -- 172.21.15.146:0/1465516496 shutdown_connections 2024-09-13T18:34:24.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.334+0000 7f80aa688700 1 -- 172.21.15.146:0/1465516496 wait complete. 2024-09-13T18:34:24.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.334+0000 7f80aa688700 1 Processor -- start 2024-09-13T18:34:24.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.334+0000 7f80aa688700 1 -- start start 2024-09-13T18:34:24.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.334+0000 7f80aa688700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a406c5b0 0x7f80a414aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:24.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80aa688700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80a40c7970 0x7f80a40cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:24.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80aa688700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80a4153860 0x7f80a414ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:24.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80aa688700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40cb1f0 con 0x7f80a406c5b0 2024-09-13T18:34:24.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80aa688700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a414abe0 con 0x7f80a4153860 2024-09-13T18:34:24.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80aa688700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40caf10 con 0x7f80a40c7970 2024-09-13T18:34:24.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80a9686700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a406c5b0 0x7f80a414aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:24.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80a9686700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a406c5b0 0x7f80a414aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37564/0 (socket says 172.21.15.146:37564) 2024-09-13T18:34:24.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80a9686700 1 -- 172.21.15.146:0/2929574681 learned_addr learned my addr 172.21.15.146:0/2929574681 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:24.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80a8e85700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80a40c7970 0x7f80a40cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:24.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80a9e87700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80a4153860 0x7f80a414ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:24.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80a9686700 1 -- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80a40c7970 msgr2=0x7f80a40cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:24.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80a9686700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80a40c7970 0x7f80a40cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:24.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80a9686700 1 -- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80a4153860 msgr2=0x7f80a414ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:24.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80a9686700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80a4153860 0x7f80a414ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:24.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80a9686700 1 -- 172.21.15.146:0/2929574681 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094009570 con 0x7f80a406c5b0 2024-09-13T18:34:24.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80a9686700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a406c5b0 0x7f80a414aee0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c530 tx=0x7f80a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:24.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f8099ffb700 1 -- 172.21.15.146:0/2929574681 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a00052d0 con 0x7f80a406c5b0 2024-09-13T18:34:24.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f8099ffb700 1 -- 172.21.15.146:0/2929574681 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a0005430 con 0x7f80a406c5b0 2024-09-13T18:34:24.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80aa688700 1 -- 172.21.15.146:0/2929574681 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a40cbea0 con 0x7f80a406c5b0 2024-09-13T18:34:24.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f8099ffb700 1 -- 172.21.15.146:0/2929574681 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a00055a0 con 0x7f80a406c5b0 2024-09-13T18:34:24.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80aa688700 1 -- 172.21.15.146:0/2929574681 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a40cc460 con 0x7f80a406c5b0 2024-09-13T18:34:24.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f80aa688700 1 -- 172.21.15.146:0/2929574681 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a4067a60 con 0x7f80a406c5b0 2024-09-13T18:34:24.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.338+0000 7f8099ffb700 1 -- 172.21.15.146:0/2929574681 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f80a00036a0 con 0x7f80a406c5b0 2024-09-13T18:34:24.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.342+0000 7f8099ffb700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8088066760 0x7f8088068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:24.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.342+0000 7f8099ffb700 1 -- 172.21.15.146:0/2929574681 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f80a0083970 con 0x7f80a406c5b0 2024-09-13T18:34:24.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.342+0000 7f80a8e85700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8088066760 0x7f8088068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:24.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.342+0000 7f80a8e85700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8088066760 0x7f8088068c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8094006f80 tx=0x7f8094008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:24.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.346+0000 7f8099ffb700 1 -- 172.21.15.146:0/2929574681 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f80a00507d0 con 0x7f80a406c5b0 2024-09-13T18:34:24.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.586+0000 7f80aa688700 1 -- 172.21.15.146:0/2929574681 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80a41593f0 con 0x7f80a406c5b0 2024-09-13T18:34:24.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.590+0000 7f8099ffb700 1 -- 172.21.15.146:0/2929574681 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f80a0053df0 con 0x7f80a406c5b0 2024-09-13T18:34:24.602 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:24.602 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:33:17.345742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.146:6833/1498985282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1498985282},{"type":"v1","addr":"172.21.15.146:6833","nonce":1498985282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:24.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.590+0000 7f80aa688700 1 -- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8088066760 msgr2=0x7f8088068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:24.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.590+0000 7f80aa688700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8088066760 0x7f8088068c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8094006f80 tx=0x7f8094008030 comp rx=0 tx=0).stop 2024-09-13T18:34:24.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.590+0000 7f80aa688700 1 -- 172.21.15.146:0/2929574681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a406c5b0 msgr2=0x7f80a414aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:24.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.590+0000 7f80aa688700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a406c5b0 0x7f80a414aee0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c530 tx=0x7f80a000c8f0 comp rx=0 tx=0).stop 2024-09-13T18:34:24.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.590+0000 7f80aa688700 1 -- 172.21.15.146:0/2929574681 shutdown_connections 2024-09-13T18:34:24.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.590+0000 7f80aa688700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8088066760 0x7f8088068c20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:24.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.590+0000 7f80aa688700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80a4153860 0x7f80a414ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:24.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.590+0000 7f80aa688700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80a40c7970 0x7f80a40cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:24.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.590+0000 7f80aa688700 1 --2- 172.21.15.146:0/2929574681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a406c5b0 0x7f80a414aee0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:24.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.590+0000 7f80aa688700 1 -- 172.21.15.146:0/2929574681 >> 172.21.15.146:0/2929574681 conn(0x7f80a40ccf70 msgr2=0x7f80a4157050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:24.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.590+0000 7f80aa688700 1 -- 172.21.15.146:0/2929574681 shutdown_connections 2024-09-13T18:34:24.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:24.590+0000 7f80aa688700 1 -- 172.21.15.146:0/2929574681 wait complete. 2024-09-13T18:34:24.606 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 39 2024-09-13T18:34:24.615 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:34:26.617 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:26.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fbfff700 1 Processor -- start 2024-09-13T18:34:26.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fbfff700 1 -- start start 2024-09-13T18:34:26.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46fc153bc0 0x7f46fc0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:26.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fbfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f46fc0c6a50 0x7f46fc160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fbfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46fc153fa0 con 0x7f46fc0c6610 2024-09-13T18:34:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fbfff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc0aa500 con 0x7f46fc0c6a50 2024-09-13T18:34:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fbfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc15a7e0 con 0x7f46fc153bc0 2024-09-13T18:34:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46f3fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f46fc0c6610 0x7f46fc0cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:44298/0 (socket says 172.21.15.146:44298) 2024-09-13T18:34:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46f3fff700 1 -- 172.21.15.146:0/2298841482 learned_addr learned my addr 172.21.15.146:0/2298841482 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:34:26.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46faffd700 1 --2- 172.21.15.146:0/2298841482 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46fc153bc0 0x7f46fc0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:26.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fb7fe700 1 --2- 172.21.15.146:0/2298841482 >> v2:172.21.15.154:3300/0 conn(0x7f46fc0c6a50 0x7f46fc160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:26.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fa7fc700 1 -- 172.21.15.146:0/2298841482 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726362591 0 0) 0x7f46fc153fa0 con 0x7f46fc0c6610 2024-09-13T18:34:26.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fa7fc700 1 -- 172.21.15.146:0/2298841482 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46e80035d0 con 0x7f46fc0c6610 2024-09-13T18:34:26.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46faffd700 1 -- 172.21.15.146:0/2298841482 >> v2:172.21.15.154:3300/0 conn(0x7f46fc0c6a50 msgr2=0x7f46fc160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:26.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46faffd700 1 --2- 172.21.15.146:0/2298841482 >> v2:172.21.15.154:3300/0 conn(0x7f46fc0c6a50 0x7f46fc160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:26.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fa7fc700 1 -- 172.21.15.146:0/2298841482 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3240226868 0 0) 0x7f46e80035d0 con 0x7f46fc0c6610 2024-09-13T18:34:26.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46faffd700 1 -- 172.21.15.146:0/2298841482 >> v1:172.21.15.146:6789/0 conn(0x7f46fc0c6610 legacy=0x7f46fc0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:26.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46faffd700 1 -- 172.21.15.146:0/2298841482 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc0c9320 con 0x7f46fc153bc0 2024-09-13T18:34:26.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46faffd700 1 --2- 172.21.15.146:0/2298841482 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46fc153bc0 0x7f46fc0c8de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009850 tx=0x7f46ec009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b9738cf9bcd33b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:26.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fa7fc700 1 -- 172.21.15.146:0/2298841482 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec011830 con 0x7f46fc153bc0 2024-09-13T18:34:26.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fa7fc700 1 -- 172.21.15.146:0/2298841482 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec011990 con 0x7f46fc153bc0 2024-09-13T18:34:26.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.722+0000 7f46fa7fc700 1 -- 172.21.15.146:0/2298841482 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec011d40 con 0x7f46fc153bc0 2024-09-13T18:34:26.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 -- 172.21.15.146:0/2298841482 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46fc153bc0 msgr2=0x7f46fc0c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:26.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 --2- 172.21.15.146:0/2298841482 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46fc153bc0 0x7f46fc0c8de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009850 tx=0x7f46ec009b60 comp rx=0 tx=0).stop 2024-09-13T18:34:26.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 -- 172.21.15.146:0/2298841482 shutdown_connections 2024-09-13T18:34:26.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 --2- 172.21.15.146:0/2298841482 >> v2:172.21.15.154:3300/0 conn(0x7f46fc0c6a50 0x7f46fc160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:26.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 --2- 172.21.15.146:0/2298841482 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46fc153bc0 0x7f46fc0c8de0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:26.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 -- 172.21.15.146:0/2298841482 >> 172.21.15.146:0/2298841482 conn(0x7f46fc066090 msgr2=0x7f46fc066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:26.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 -- 172.21.15.146:0/2298841482 shutdown_connections 2024-09-13T18:34:26.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 -- 172.21.15.146:0/2298841482 wait complete. 2024-09-13T18:34:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 Processor -- start 2024-09-13T18:34:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 -- start start 2024-09-13T18:34:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46fc0c6610 0x7f46fc1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46fc0c6a50 0x7f46fc156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46fc153bc0 0x7f46fc150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0c19c0 con 0x7f46fc0c6a50 2024-09-13T18:34:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc154150 con 0x7f46fc0c6610 2024-09-13T18:34:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fbfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc150c40 con 0x7f46fc153bc0 2024-09-13T18:34:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46f3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46fc0c6a50 0x7f46fc156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46f3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46fc0c6a50 0x7f46fc156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37580/0 (socket says 172.21.15.146:37580) 2024-09-13T18:34:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46f3fff700 1 -- 172.21.15.146:0/746976093 learned_addr learned my addr 172.21.15.146:0/746976093 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46faffd700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46fc0c6610 0x7f46fc1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:26.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46fb7fe700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46fc153bc0 0x7f46fc150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:26.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46f3fff700 1 -- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46fc153bc0 msgr2=0x7f46fc150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:26.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46f3fff700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46fc153bc0 0x7f46fc150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:26.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46f3fff700 1 -- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46fc0c6610 msgr2=0x7f46fc1561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:26.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46f3fff700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46fc0c6610 0x7f46fc1561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:26.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.726+0000 7f46f3fff700 1 -- 172.21.15.146:0/746976093 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec0094b0 con 0x7f46fc0c6a50 2024-09-13T18:34:26.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.730+0000 7f46f3fff700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46fc0c6a50 0x7f46fc156730 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f46e400c750 tx=0x7f46e400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:26.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.730+0000 7f46f37fe700 1 -- 172.21.15.146:0/746976093 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4005d90 con 0x7f46fc0c6a50 2024-09-13T18:34:26.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.730+0000 7f46f37fe700 1 -- 172.21.15.146:0/746976093 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e40070d0 con 0x7f46fc0c6a50 2024-09-13T18:34:26.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.730+0000 7f46fbfff700 1 -- 172.21.15.146:0/746976093 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc150f00 con 0x7f46fc0c6a50 2024-09-13T18:34:26.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.730+0000 7f46f37fe700 1 -- 172.21.15.146:0/746976093 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4007440 con 0x7f46fc0c6a50 2024-09-13T18:34:26.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.730+0000 7f46fbfff700 1 -- 172.21.15.146:0/746976093 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc1edc00 con 0x7f46fc0c6a50 2024-09-13T18:34:26.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.730+0000 7f46f37fe700 1 -- 172.21.15.146:0/746976093 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f46e40075a0 con 0x7f46fc0c6a50 2024-09-13T18:34:26.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.730+0000 7f46fbfff700 1 -- 172.21.15.146:0/746976093 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc15a2c0 con 0x7f46fc0c6a50 2024-09-13T18:34:26.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.730+0000 7f46f37fe700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f46dc066730 0x7f46dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:26.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.730+0000 7f46f37fe700 1 -- 172.21.15.146:0/746976093 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f46e4083af0 con 0x7f46fc0c6a50 2024-09-13T18:34:26.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.730+0000 7f46faffd700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f46dc066730 0x7f46dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:26.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.734+0000 7f46faffd700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f46dc066730 0x7f46dc068bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0090b0 tx=0x7f46ec01c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:26.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.738+0000 7f46f37fe700 1 -- 172.21.15.146:0/746976093 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f46e4050890 con 0x7f46fc0c6a50 2024-09-13T18:34:26.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.958+0000 7f46fbfff700 1 -- 172.21.15.146:0/746976093 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46fc06f690 con 0x7f46fc0c6a50 2024-09-13T18:34:26.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.958+0000 7f46f37fe700 1 -- 172.21.15.146:0/746976093 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5535 (secure 0 0 0) 0x7f46e4053eb0 con 0x7f46fc0c6a50 2024-09-13T18:34:26.972 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:26.972 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:25.648844+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:26.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.962+0000 7f46fbfff700 1 -- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f46dc066730 msgr2=0x7f46dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:26.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.962+0000 7f46fbfff700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f46dc066730 0x7f46dc068bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0090b0 tx=0x7f46ec01c040 comp rx=0 tx=0).stop 2024-09-13T18:34:26.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.962+0000 7f46fbfff700 1 -- 172.21.15.146:0/746976093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46fc0c6a50 msgr2=0x7f46fc156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:26.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.962+0000 7f46fbfff700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46fc0c6a50 0x7f46fc156730 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f46e400c750 tx=0x7f46e400ca60 comp rx=0 tx=0).stop 2024-09-13T18:34:26.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.962+0000 7f46fbfff700 1 -- 172.21.15.146:0/746976093 shutdown_connections 2024-09-13T18:34:26.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.962+0000 7f46fbfff700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f46dc066730 0x7f46dc068bf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:26.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.962+0000 7f46fbfff700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46fc153bc0 0x7f46fc150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:26.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.962+0000 7f46fbfff700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46fc0c6a50 0x7f46fc156730 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:26.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.962+0000 7f46fbfff700 1 --2- 172.21.15.146:0/746976093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46fc0c6610 0x7f46fc1561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:26.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.962+0000 7f46fbfff700 1 -- 172.21.15.146:0/746976093 >> 172.21.15.146:0/746976093 conn(0x7f46fc066090 msgr2=0x7f46fc06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:26.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.962+0000 7f46fbfff700 1 -- 172.21.15.146:0/746976093 shutdown_connections 2024-09-13T18:34:26.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:26.962+0000 7f46fbfff700 1 -- 172.21.15.146:0/746976093 wait complete. 2024-09-13T18:34:26.977 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 40 2024-09-13T18:34:26.987 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2024-09-13T18:34:26.987 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:34:26.988 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:27.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d80ba7700 1 Processor -- start 2024-09-13T18:34:27.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d80ba7700 1 -- start start 2024-09-13T18:34:27.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d80ba7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d7c150610 0x7f9d7c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:27.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d80ba7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9d7c071610 0x7f9d7c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:27.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d80ba7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7c071410 con 0x7f9d7c150230 2024-09-13T18:34:27.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d80ba7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d7c1596b0 con 0x7f9d7c071610 2024-09-13T18:34:27.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d80ba7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c156090 con 0x7f9d7c150610 2024-09-13T18:34:27.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d7bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9d7c071610 0x7f9d7c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:27.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d7b7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9d7c150230 0x7f9d7c06ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:57552/0 (socket says 172.21.15.146:57552) 2024-09-13T18:34:27.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d7bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9d7c071610 0x7f9d7c06db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:51068/0 (socket says 172.21.15.146:51068) 2024-09-13T18:34:27.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d7b7fe700 1 -- 172.21.15.146:0/2479335923 learned_addr learned my addr 172.21.15.146:0/2479335923 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:34:27.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d7affd700 1 --2- 172.21.15.146:0/2479335923 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d7c150610 0x7f9d7c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:27.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d7bfff700 1 -- 172.21.15.146:0/2479335923 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d7c150610 msgr2=0x7f9d7c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:27.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d7bfff700 1 --2- 172.21.15.146:0/2479335923 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d7c150610 0x7f9d7c06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:27.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.078+0000 7f9d7bfff700 1 -- 172.21.15.146:0/2479335923 >> v1:172.21.15.154:6789/0 conn(0x7f9d7c150230 legacy=0x7f9d7c06ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:27.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d7bfff700 1 -- 172.21.15.146:0/2479335923 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c156350 con 0x7f9d7c071610 2024-09-13T18:34:27.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d7bfff700 1 --2- 172.21.15.146:0/2479335923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d7c071610 0x7f9d7c06db20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009d90 tx=0x7f9d7400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16cf7ae2c3dc07ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:27.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d7a7fc700 1 -- 172.21.15.146:0/2479335923 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7400e070 con 0x7f9d7c071610 2024-09-13T18:34:27.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d7a7fc700 1 -- 172.21.15.146:0/2479335923 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7400bbd0 con 0x7f9d7c071610 2024-09-13T18:34:27.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d7a7fc700 1 -- 172.21.15.146:0/2479335923 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74012430 con 0x7f9d7c071610 2024-09-13T18:34:27.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d80ba7700 1 -- 172.21.15.146:0/2479335923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d7c071610 msgr2=0x7f9d7c06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:27.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d80ba7700 1 --2- 172.21.15.146:0/2479335923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d7c071610 0x7f9d7c06db20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009d90 tx=0x7f9d7400b330 comp rx=0 tx=0).stop 2024-09-13T18:34:27.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d80ba7700 1 -- 172.21.15.146:0/2479335923 shutdown_connections 2024-09-13T18:34:27.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d80ba7700 1 --2- 172.21.15.146:0/2479335923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d7c071610 0x7f9d7c06db20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:27.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d80ba7700 1 --2- 172.21.15.146:0/2479335923 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d7c150610 0x7f9d7c06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:27.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d80ba7700 1 -- 172.21.15.146:0/2479335923 >> 172.21.15.146:0/2479335923 conn(0x7f9d7c071ed0 msgr2=0x7f9d7c0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:27.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d80ba7700 1 -- 172.21.15.146:0/2479335923 shutdown_connections 2024-09-13T18:34:27.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d80ba7700 1 -- 172.21.15.146:0/2479335923 wait complete. 2024-09-13T18:34:27.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d80ba7700 1 Processor -- start 2024-09-13T18:34:27.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.082+0000 7f9d80ba7700 1 -- start start 2024-09-13T18:34:27.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d80ba7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d7c071610 0x7f9d7c151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:27.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d80ba7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d7c150230 0x7f9d7c151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:27.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d80ba7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d7c1491f0 0x7f9d7c1ed700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:27.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d80ba7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c1f1070 con 0x7f9d7c071610 2024-09-13T18:34:27.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d80ba7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c1ed270 con 0x7f9d7c150230 2024-09-13T18:34:27.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d80ba7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c0caf50 con 0x7f9d7c1491f0 2024-09-13T18:34:27.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d7b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d7c071610 0x7f9d7c151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:27.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d7b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d7c071610 0x7f9d7c151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37596/0 (socket says 172.21.15.146:37596) 2024-09-13T18:34:27.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d7b7fe700 1 -- 172.21.15.146:0/959566452 learned_addr learned my addr 172.21.15.146:0/959566452 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:27.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d7affd700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d7c150230 0x7f9d7c151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:27.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d7bfff700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d7c1491f0 0x7f9d7c1ed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:27.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d7b7fe700 1 -- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d7c1491f0 msgr2=0x7f9d7c1ed700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:27.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d7b7fe700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d7c1491f0 0x7f9d7c1ed700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:27.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d7b7fe700 1 -- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d7c150230 msgr2=0x7f9d7c151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:27.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d7b7fe700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d7c150230 0x7f9d7c151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:27.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d7b7fe700 1 -- 172.21.15.146:0/959566452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d74009a70 con 0x7f9d7c071610 2024-09-13T18:34:27.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d7b7fe700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d7c071610 0x7f9d7c151340 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00c700 tx=0x7f9d6c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:27.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d63fff700 1 -- 172.21.15.146:0/959566452 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c005d90 con 0x7f9d7c071610 2024-09-13T18:34:27.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d63fff700 1 -- 172.21.15.146:0/959566452 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c0070d0 con 0x7f9d7c071610 2024-09-13T18:34:27.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d80ba7700 1 -- 172.21.15.146:0/959566452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7c0cb230 con 0x7f9d7c071610 2024-09-13T18:34:27.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d63fff700 1 -- 172.21.15.146:0/959566452 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c007440 con 0x7f9d7c071610 2024-09-13T18:34:27.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.086+0000 7f9d80ba7700 1 -- 172.21.15.146:0/959566452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7c1ecbb0 con 0x7f9d7c071610 2024-09-13T18:34:27.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.090+0000 7f9d80ba7700 1 -- 172.21.15.146:0/959566452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7c071b10 con 0x7f9d7c071610 2024-09-13T18:34:27.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.090+0000 7f9d63fff700 1 -- 172.21.15.146:0/959566452 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d6c0075a0 con 0x7f9d7c071610 2024-09-13T18:34:27.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.090+0000 7f9d63fff700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d68066760 0x7f9d68068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:27.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.090+0000 7f9d63fff700 1 -- 172.21.15.146:0/959566452 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f9d6c0838d0 con 0x7f9d7c071610 2024-09-13T18:34:27.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.090+0000 7f9d7affd700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d68066760 0x7f9d68068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:27.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.090+0000 7f9d7affd700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d68066760 0x7f9d68068c20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0c6f00 tx=0x7f9d70008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:27.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.102+0000 7f9d63fff700 1 -- 172.21.15.146:0/959566452 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9d6c050810 con 0x7f9d7c071610 2024-09-13T18:34:27.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.318+0000 7f9d80ba7700 1 -- 172.21.15.146:0/959566452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d7c06ebd0 con 0x7f9d7c071610 2024-09-13T18:34:27.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.318+0000 7f9d63fff700 1 -- 172.21.15.146:0/959566452 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5535 (secure 0 0 0) 0x7f9d6c017090 con 0x7f9d7c071610 2024-09-13T18:34:27.332 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:27.332 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:25.648844+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:27.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.322+0000 7f9d80ba7700 1 -- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d68066760 msgr2=0x7f9d68068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:27.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.322+0000 7f9d80ba7700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d68066760 0x7f9d68068c20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0c6f00 tx=0x7f9d70008040 comp rx=0 tx=0).stop 2024-09-13T18:34:27.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.322+0000 7f9d80ba7700 1 -- 172.21.15.146:0/959566452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d7c071610 msgr2=0x7f9d7c151340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:27.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.322+0000 7f9d80ba7700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d7c071610 0x7f9d7c151340 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00c700 tx=0x7f9d6c00ca10 comp rx=0 tx=0).stop 2024-09-13T18:34:27.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.322+0000 7f9d80ba7700 1 -- 172.21.15.146:0/959566452 shutdown_connections 2024-09-13T18:34:27.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.322+0000 7f9d80ba7700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d68066760 0x7f9d68068c20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:27.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.322+0000 7f9d80ba7700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d7c1491f0 0x7f9d7c1ed700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:27.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.322+0000 7f9d80ba7700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d7c150230 0x7f9d7c151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:27.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.322+0000 7f9d80ba7700 1 --2- 172.21.15.146:0/959566452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d7c071610 0x7f9d7c151340 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:27.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.322+0000 7f9d80ba7700 1 -- 172.21.15.146:0/959566452 >> 172.21.15.146:0/959566452 conn(0x7f9d7c071ed0 msgr2=0x7f9d7c06b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:27.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.322+0000 7f9d80ba7700 1 -- 172.21.15.146:0/959566452 shutdown_connections 2024-09-13T18:34:27.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:27.322+0000 7f9d80ba7700 1 -- 172.21.15.146:0/959566452 wait complete. 2024-09-13T18:34:27.336 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 40 2024-09-13T18:34:27.348 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 40, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 40, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5278": { "addr": "172.21.15.154:6839/3274220802", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 3274220802, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 3274220802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5278, "incarnation": 28, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 19 }, "gid_5505": { "addr": "172.21.15.146:6835/1652676664", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 1652676664, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 1652676664, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5505, "incarnation": 34, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 38 }, "gid_5649": { "addr": "172.21.15.146:6837/3926931671", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3926931671, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3926931671, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5649, "incarnation": 40, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 32, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:34:25.648844+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5505, "mds_1": 5278, "mds_2": 5649 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:34:29.349 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:29.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5417284700 1 Processor -- start 2024-09-13T18:34:29.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5417284700 1 -- start start 2024-09-13T18:34:29.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5417284700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f54100c6a80 0x7f5410161c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:29.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5417284700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5410153fa0 con 0x7f54100c6610 2024-09-13T18:34:29.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5417284700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54100aa500 con 0x7f5410153bc0 2024-09-13T18:34:29.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5417284700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541015a7e0 con 0x7f54100c6a80 2024-09-13T18:34:29.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5416a83700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f54100c6a80 0x7f5410161c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:29.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5416a83700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f54100c6a80 0x7f5410161c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37606/0 (socket says 172.21.15.146:37606) 2024-09-13T18:34:29.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5415a81700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f54100c6610 0x7f54100cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:44312/0 (socket says 172.21.15.146:44312) 2024-09-13T18:34:29.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5416a83700 1 -- 172.21.15.146:0/3302574916 learned_addr learned my addr 172.21.15.146:0/3302574916 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5415280700 1 -- 172.21.15.146:0/3302574916 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885014718 0 0) 0x7f5410153fa0 con 0x7f54100c6610 2024-09-13T18:34:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5415280700 1 -- 172.21.15.146:0/3302574916 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53fc0035d0 con 0x7f54100c6610 2024-09-13T18:34:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5416a83700 1 -- 172.21.15.146:0/3302574916 >> v1:172.21.15.154:6789/0 conn(0x7f5410153bc0 legacy=0x7f54100c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5415280700 1 -- 172.21.15.146:0/3302574916 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256269401 0 0) 0x7f54100aa500 con 0x7f5410153bc0 2024-09-13T18:34:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5416a83700 1 -- 172.21.15.146:0/3302574916 >> v1:172.21.15.146:6789/0 conn(0x7f54100c6610 legacy=0x7f54100cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5416a83700 1 -- 172.21.15.146:0/3302574916 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54100aa220 con 0x7f54100c6a80 2024-09-13T18:34:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5416a83700 1 --2- 172.21.15.146:0/3302574916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54100c6a80 0x7f5410161c30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5408009e50 tx=0x7f540800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43af775e5acee74 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5415280700 1 -- 172.21.15.146:0/3302574916 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540800e070 con 0x7f54100c6a80 2024-09-13T18:34:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5415280700 1 -- 172.21.15.146:0/3302574916 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540800b910 con 0x7f54100c6a80 2024-09-13T18:34:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.454+0000 7f5415280700 1 -- 172.21.15.146:0/3302574916 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540800bc80 con 0x7f54100c6a80 2024-09-13T18:34:29.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 -- 172.21.15.146:0/3302574916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54100c6a80 msgr2=0x7f5410161c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:29.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 --2- 172.21.15.146:0/3302574916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54100c6a80 0x7f5410161c30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5408009e50 tx=0x7f540800b330 comp rx=0 tx=0).stop 2024-09-13T18:34:29.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 -- 172.21.15.146:0/3302574916 shutdown_connections 2024-09-13T18:34:29.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 --2- 172.21.15.146:0/3302574916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54100c6a80 0x7f5410161c30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:29.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 -- 172.21.15.146:0/3302574916 >> 172.21.15.146:0/3302574916 conn(0x7f5410066090 msgr2=0x7f5410066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:29.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 -- 172.21.15.146:0/3302574916 shutdown_connections 2024-09-13T18:34:29.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 -- 172.21.15.146:0/3302574916 wait complete. 2024-09-13T18:34:29.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 Processor -- start 2024-09-13T18:34:29.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 -- start start 2024-09-13T18:34:29.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54100c6610 0x7f541014aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:29.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54100c6a80 0x7f54101ee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:29.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5410153bc0 0x7f541014c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:29.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410158560 con 0x7f5410153bc0 2024-09-13T18:34:29.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410158a90 con 0x7f54100c6610 2024-09-13T18:34:29.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.458+0000 7f5417284700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541006e140 con 0x7f54100c6a80 2024-09-13T18:34:29.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5416a83700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5410153bc0 0x7f541014c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:29.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5416a83700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5410153bc0 0x7f541014c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37622/0 (socket says 172.21.15.146:37622) 2024-09-13T18:34:29.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5416a83700 1 -- 172.21.15.146:0/4222391278 learned_addr learned my addr 172.21.15.146:0/4222391278 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:29.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5415a81700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54100c6a80 0x7f54101ee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:29.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5416282700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54100c6610 0x7f541014aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:29.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5416a83700 1 -- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54100c6a80 msgr2=0x7f54101ee9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:29.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5416a83700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54100c6a80 0x7f54101ee9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:29.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5416a83700 1 -- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54100c6610 msgr2=0x7f541014aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:29.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5416a83700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54100c6610 0x7f541014aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:29.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5416a83700 1 -- 172.21.15.146:0/4222391278 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5408009b30 con 0x7f5410153bc0 2024-09-13T18:34:29.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5416a83700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5410153bc0 0x7f541014c690 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f54100aa690 tx=0x7f5408004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:29.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5406ffd700 1 -- 172.21.15.146:0/4222391278 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540801c640 con 0x7f5410153bc0 2024-09-13T18:34:29.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5417284700 1 -- 172.21.15.146:0/4222391278 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54101587b0 con 0x7f5410153bc0 2024-09-13T18:34:29.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5406ffd700 1 -- 172.21.15.146:0/4222391278 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540801c7c0 con 0x7f5410153bc0 2024-09-13T18:34:29.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5406ffd700 1 -- 172.21.15.146:0/4222391278 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540801cb70 con 0x7f5410153bc0 2024-09-13T18:34:29.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5417284700 1 -- 172.21.15.146:0/4222391278 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541006ddd0 con 0x7f5410153bc0 2024-09-13T18:34:29.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5417284700 1 -- 172.21.15.146:0/4222391278 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541006df60 con 0x7f5410153bc0 2024-09-13T18:34:29.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5406ffd700 1 -- 172.21.15.146:0/4222391278 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f540801ccd0 con 0x7f5410153bc0 2024-09-13T18:34:29.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5406ffd700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53f4066730 0x7f53f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:29.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.462+0000 7f5406ffd700 1 -- 172.21.15.146:0/4222391278 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f5408086140 con 0x7f5410153bc0 2024-09-13T18:34:29.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.466+0000 7f5416282700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53f4066730 0x7f53f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:29.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.466+0000 7f5416282700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53f4066730 0x7f53f4068bf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f540c000f60 tx=0x7f540c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:29.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.470+0000 7f5406ffd700 1 -- 172.21.15.146:0/4222391278 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5408052f50 con 0x7f5410153bc0 2024-09-13T18:34:29.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.698+0000 7f5417284700 1 -- 172.21.15.146:0/4222391278 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54101518c0 con 0x7f5410153bc0 2024-09-13T18:34:29.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.702+0000 7f5406ffd700 1 -- 172.21.15.146:0/4222391278 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5535 (secure 0 0 0) 0x7f540800bc90 con 0x7f5410153bc0 2024-09-13T18:34:29.714 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:29.715 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:25.648844+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:29.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.702+0000 7f5417284700 1 -- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53f4066730 msgr2=0x7f53f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:29.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.702+0000 7f5417284700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53f4066730 0x7f53f4068bf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f540c000f60 tx=0x7f540c009040 comp rx=0 tx=0).stop 2024-09-13T18:34:29.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.702+0000 7f5417284700 1 -- 172.21.15.146:0/4222391278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5410153bc0 msgr2=0x7f541014c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:29.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.702+0000 7f5417284700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5410153bc0 0x7f541014c690 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f54100aa690 tx=0x7f5408004af0 comp rx=0 tx=0).stop 2024-09-13T18:34:29.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.702+0000 7f5417284700 1 -- 172.21.15.146:0/4222391278 shutdown_connections 2024-09-13T18:34:29.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.702+0000 7f5417284700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f53f4066730 0x7f53f4068bf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:29.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.702+0000 7f5417284700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5410153bc0 0x7f541014c690 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:29.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.702+0000 7f5417284700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54100c6a80 0x7f54101ee9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:29.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.702+0000 7f5417284700 1 --2- 172.21.15.146:0/4222391278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54100c6610 0x7f541014aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:29.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.702+0000 7f5417284700 1 -- 172.21.15.146:0/4222391278 >> 172.21.15.146:0/4222391278 conn(0x7f5410066090 msgr2=0x7f54100cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:29.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.702+0000 7f5417284700 1 -- 172.21.15.146:0/4222391278 shutdown_connections 2024-09-13T18:34:29.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:29.702+0000 7f5417284700 1 -- 172.21.15.146:0/4222391278 wait complete. 2024-09-13T18:34:29.719 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 40 2024-09-13T18:34:29.732 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:34:31.733 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:31.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.842+0000 7fdeddb5a700 1 Processor -- start 2024-09-13T18:34:31.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.842+0000 7fdeddb5a700 1 -- start start 2024-09-13T18:34:31.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdeddb5a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fded00df2a0 0x7fded0045cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:31.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdeddb5a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fded0046230 0x7fded0046a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:31.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdeddb5a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded0007830 con 0x7fded00d8a70 2024-09-13T18:34:31.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdeddb5a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded00d8e50 con 0x7fded00df2a0 2024-09-13T18:34:31.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdeddb5a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded004bea0 con 0x7fded0046230 2024-09-13T18:34:31.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fded7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fded00df2a0 0x7fded0045cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:31.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fded7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fded00df2a0 0x7fded0045cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60280/0 (socket says 172.21.15.146:60280) 2024-09-13T18:34:31.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fded7fff700 1 -- 172.21.15.146:0/424908719 learned_addr learned my addr 172.21.15.146:0/424908719 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:31.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdedd359700 1 --2- 172.21.15.146:0/424908719 >> v2:172.21.15.154:3300/0 conn(0x7fded0046230 0x7fded0046a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:31.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fded7fff700 1 -- 172.21.15.146:0/424908719 >> v2:172.21.15.154:3300/0 conn(0x7fded0046230 msgr2=0x7fded0046a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:31.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fded7fff700 1 --2- 172.21.15.146:0/424908719 >> v2:172.21.15.154:3300/0 conn(0x7fded0046230 0x7fded0046a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:31.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fded7fff700 1 -- 172.21.15.146:0/424908719 >> v1:172.21.15.154:6789/0 conn(0x7fded00d8a70 legacy=0x7fded004b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:31.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fded7fff700 1 -- 172.21.15.146:0/424908719 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded0047600 con 0x7fded00df2a0 2024-09-13T18:34:31.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fded7fff700 1 --2- 172.21.15.146:0/424908719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fded00df2a0 0x7fded0045cf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fded0008a50 tx=0x7fdec4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c444d4f1f6f393 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:31.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fded77fe700 1 -- 172.21.15.146:0/424908719 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec400b8a0 con 0x7fded00df2a0 2024-09-13T18:34:31.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fded77fe700 1 -- 172.21.15.146:0/424908719 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec400ba00 con 0x7fded00df2a0 2024-09-13T18:34:31.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fded77fe700 1 -- 172.21.15.146:0/424908719 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec400bd90 con 0x7fded00df2a0 2024-09-13T18:34:31.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdeddb5a700 1 -- 172.21.15.146:0/424908719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fded00df2a0 msgr2=0x7fded0045cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:31.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdeddb5a700 1 --2- 172.21.15.146:0/424908719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fded00df2a0 0x7fded0045cf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fded0008a50 tx=0x7fdec4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:34:31.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdeddb5a700 1 -- 172.21.15.146:0/424908719 shutdown_connections 2024-09-13T18:34:31.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdeddb5a700 1 --2- 172.21.15.146:0/424908719 >> v2:172.21.15.154:3300/0 conn(0x7fded0046230 0x7fded0046a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:31.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdeddb5a700 1 --2- 172.21.15.146:0/424908719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fded00df2a0 0x7fded0045cf0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:31.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdeddb5a700 1 -- 172.21.15.146:0/424908719 >> 172.21.15.146:0/424908719 conn(0x7fded0050e10 msgr2=0x7fded0051220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:31.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdeddb5a700 1 -- 172.21.15.146:0/424908719 shutdown_connections 2024-09-13T18:34:31.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.846+0000 7fdeddb5a700 1 -- 172.21.15.146:0/424908719 wait complete. 2024-09-13T18:34:31.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fdeddb5a700 1 Processor -- start 2024-09-13T18:34:31.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fdeddb5a700 1 -- start start 2024-09-13T18:34:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fdeddb5a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fded0046230 0x7fded00cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fdeddb5a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fded00d8a70 0x7fded00d1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fdeddb5a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fded00df2a0 0x7fded00d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fdeddb5a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded0036e50 con 0x7fded00d8a70 2024-09-13T18:34:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fdeddb5a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00cf9a0 con 0x7fded0046230 2024-09-13T18:34:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fdeddb5a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded00cf1f0 con 0x7fded00df2a0 2024-09-13T18:34:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fded7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fded00d8a70 0x7fded00d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fded7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fded00d8a70 0x7fded00d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60296/0 (socket says 172.21.15.146:60296) 2024-09-13T18:34:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fded7fff700 1 -- 172.21.15.146:0/454869890 learned_addr learned my addr 172.21.15.146:0/454869890 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fdedd359700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fded00df2a0 0x7fded00d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fdedcb58700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fded0046230 0x7fded00cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fded7fff700 1 -- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fded00df2a0 msgr2=0x7fded00d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:31.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fded7fff700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fded00df2a0 0x7fded00d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:31.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fded7fff700 1 -- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fded0046230 msgr2=0x7fded00cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:31.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fded7fff700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fded0046230 0x7fded00cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:31.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fded7fff700 1 -- 172.21.15.146:0/454869890 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec4009570 con 0x7fded00d8a70 2024-09-13T18:34:31.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.850+0000 7fded7fff700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fded00d8a70 0x7fded00d1790 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdec4014040 tx=0x7fdec4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:31.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.854+0000 7fded57fa700 1 -- 172.21.15.146:0/454869890 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec40038c0 con 0x7fded00d8a70 2024-09-13T18:34:31.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.854+0000 7fded57fa700 1 -- 172.21.15.146:0/454869890 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec4003a20 con 0x7fded00d8a70 2024-09-13T18:34:31.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.854+0000 7fdeddb5a700 1 -- 172.21.15.146:0/454869890 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded00cfbd0 con 0x7fded00d8a70 2024-09-13T18:34:31.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.854+0000 7fded57fa700 1 -- 172.21.15.146:0/454869890 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4003dd0 con 0x7fded00d8a70 2024-09-13T18:34:31.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.854+0000 7fdeddb5a700 1 -- 172.21.15.146:0/454869890 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded00d23a0 con 0x7fded00d8a70 2024-09-13T18:34:31.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.854+0000 7fdeddb5a700 1 -- 172.21.15.146:0/454869890 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded00df840 con 0x7fded00d8a70 2024-09-13T18:34:31.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.854+0000 7fded57fa700 1 -- 172.21.15.146:0/454869890 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdec4011420 con 0x7fded00d8a70 2024-09-13T18:34:31.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.854+0000 7fded57fa700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdebc066730 0x7fdebc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:31.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.854+0000 7fded57fa700 1 -- 172.21.15.146:0/454869890 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fdec4020030 con 0x7fded00d8a70 2024-09-13T18:34:31.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.854+0000 7fdedcb58700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdebc066730 0x7fdebc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:31.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.858+0000 7fdedcb58700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdebc066730 0x7fdebc068bf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdecc000f60 tx=0x7fdecc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:31.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:31.862+0000 7fded57fa700 1 -- 172.21.15.146:0/454869890 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdec4052c00 con 0x7fded00d8a70 2024-09-13T18:34:32.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.098+0000 7fdeddb5a700 1 -- 172.21.15.146:0/454869890 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fded0007cc0 con 0x7fded00d8a70 2024-09-13T18:34:32.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.102+0000 7fded57fa700 1 -- 172.21.15.146:0/454869890 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5537 (secure 0 0 0) 0x7fdec4015090 con 0x7fded00d8a70 2024-09-13T18:34:32.115 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:32.115 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:30.691240+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:resolve","state_seq":18,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:32.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.106+0000 7fdeddb5a700 1 -- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdebc066730 msgr2=0x7fdebc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:32.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.106+0000 7fdeddb5a700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdebc066730 0x7fdebc068bf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdecc000f60 tx=0x7fdecc009040 comp rx=0 tx=0).stop 2024-09-13T18:34:32.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.106+0000 7fdeddb5a700 1 -- 172.21.15.146:0/454869890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fded00d8a70 msgr2=0x7fded00d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:32.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.106+0000 7fdeddb5a700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fded00d8a70 0x7fded00d1790 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdec4014040 tx=0x7fdec4004770 comp rx=0 tx=0).stop 2024-09-13T18:34:32.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.106+0000 7fdeddb5a700 1 -- 172.21.15.146:0/454869890 shutdown_connections 2024-09-13T18:34:32.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.106+0000 7fdeddb5a700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdebc066730 0x7fdebc068bf0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:32.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.106+0000 7fdeddb5a700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fded00df2a0 0x7fded00d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:32.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.106+0000 7fdeddb5a700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fded00d8a70 0x7fded00d1790 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:32.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.106+0000 7fdeddb5a700 1 --2- 172.21.15.146:0/454869890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fded0046230 0x7fded00cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:32.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.106+0000 7fdeddb5a700 1 -- 172.21.15.146:0/454869890 >> 172.21.15.146:0/454869890 conn(0x7fded0050e10 msgr2=0x7fded0046a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:32.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.106+0000 7fdeddb5a700 1 -- 172.21.15.146:0/454869890 shutdown_connections 2024-09-13T18:34:32.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:32.106+0000 7fdeddb5a700 1 -- 172.21.15.146:0/454869890 wait complete. 2024-09-13T18:34:32.122 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 41 2024-09-13T18:34:32.134 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:34:34.135 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:34.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.254+0000 7f225e40e700 1 Processor -- start 2024-09-13T18:34:34.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.254+0000 7f225e40e700 1 -- start start 2024-09-13T18:34:34.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.254+0000 7f225e40e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2258150610 0x7f225806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:34.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.254+0000 7f225e40e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2258071610 0x7f225806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:34.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.254+0000 7f225e40e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2258071410 con 0x7f2258150230 2024-09-13T18:34:34.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.254+0000 7f225e40e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22581596b0 con 0x7f2258150610 2024-09-13T18:34:34.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.254+0000 7f225e40e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2258156090 con 0x7f2258071610 2024-09-13T18:34:34.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.254+0000 7f225cc0b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2258150610 0x7f225806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:34.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.254+0000 7f225dc0d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2258071610 0x7f225806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:34.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.254+0000 7f225cc0b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2258150610 0x7f225806d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:42034/0 (socket says 172.21.15.146:42034) 2024-09-13T18:34:34.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.254+0000 7f225cc0b700 1 -- 172.21.15.146:0/227367626 learned_addr learned my addr 172.21.15.146:0/227367626 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:34.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.254+0000 7f225d40c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2258150230 0x7f225806ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:43216/0 (socket says 172.21.15.146:43216) 2024-09-13T18:34:34.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225cc0b700 1 -- 172.21.15.146:0/227367626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2258071610 msgr2=0x7f225806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:34.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225cc0b700 1 --2- 172.21.15.146:0/227367626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2258071610 0x7f225806db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:34.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225cc0b700 1 -- 172.21.15.146:0/227367626 >> v1:172.21.15.154:6789/0 conn(0x7f2258150230 legacy=0x7f225806ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:34.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225cc0b700 1 -- 172.21.15.146:0/227367626 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2258156350 con 0x7f2258150610 2024-09-13T18:34:34.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225dc0d700 1 --2- 172.21.15.146:0/227367626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2258071610 0x7f225806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:34.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225cc0b700 1 --2- 172.21.15.146:0/227367626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2258150610 0x7f225806d5e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2248009910 tx=0x7f2248009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2e7f29e2836647f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:34.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f224ffff700 1 -- 172.21.15.146:0/227367626 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224800f8a0 con 0x7f2258150610 2024-09-13T18:34:34.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f224ffff700 1 -- 172.21.15.146:0/227367626 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224800fa00 con 0x7f2258150610 2024-09-13T18:34:34.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f224ffff700 1 -- 172.21.15.146:0/227367626 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224800fdb0 con 0x7f2258150610 2024-09-13T18:34:34.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225e40e700 1 -- 172.21.15.146:0/227367626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2258150610 msgr2=0x7f225806d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:34.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225e40e700 1 --2- 172.21.15.146:0/227367626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2258150610 0x7f225806d5e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2248009910 tx=0x7f2248009c20 comp rx=0 tx=0).stop 2024-09-13T18:34:34.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225e40e700 1 -- 172.21.15.146:0/227367626 shutdown_connections 2024-09-13T18:34:34.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225e40e700 1 --2- 172.21.15.146:0/227367626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2258071610 0x7f225806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:34.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225e40e700 1 --2- 172.21.15.146:0/227367626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2258150610 0x7f225806d5e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:34.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225e40e700 1 -- 172.21.15.146:0/227367626 >> 172.21.15.146:0/227367626 conn(0x7f2258071ed0 msgr2=0x7f22580722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:34.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225e40e700 1 -- 172.21.15.146:0/227367626 shutdown_connections 2024-09-13T18:34:34.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225e40e700 1 -- 172.21.15.146:0/227367626 wait complete. 2024-09-13T18:34:34.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225e40e700 1 Processor -- start 2024-09-13T18:34:34.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.258+0000 7f225e40e700 1 -- start start 2024-09-13T18:34:34.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225e40e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2258071610 0x7f2258154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:34.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225e40e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2258150230 0x7f2258154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:34.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225e40e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2258155510 0x7f225813e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:34.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225e40e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22581ebc10 con 0x7f2258155510 2024-09-13T18:34:34.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225e40e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22581590d0 con 0x7f2258071610 2024-09-13T18:34:34.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225e40e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22581ead10 con 0x7f2258150230 2024-09-13T18:34:34.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225dc0d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2258155510 0x7f225813e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:34.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225dc0d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2258155510 0x7f225813e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60312/0 (socket says 172.21.15.146:60312) 2024-09-13T18:34:34.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225dc0d700 1 -- 172.21.15.146:0/1025596589 learned_addr learned my addr 172.21.15.146:0/1025596589 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:34.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225cc0b700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2258150230 0x7f2258154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:34.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225d40c700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2258071610 0x7f2258154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:34.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225cc0b700 1 -- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2258071610 msgr2=0x7f2258154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:34.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225cc0b700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2258071610 0x7f2258154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:34.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225cc0b700 1 -- 172.21.15.146:0/1025596589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2258155510 msgr2=0x7f225813e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:34.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225cc0b700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2258155510 0x7f225813e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:34.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225cc0b700 1 -- 172.21.15.146:0/1025596589 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2250009a70 con 0x7f2258150230 2024-09-13T18:34:34.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225dc0d700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2258155510 0x7f225813e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:34.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225cc0b700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2258150230 0x7f2258154bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f225806ca30 tx=0x7f2248004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:34.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f224dffb700 1 -- 172.21.15.146:0/1025596589 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248004440 con 0x7f2258150230 2024-09-13T18:34:34.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f224dffb700 1 -- 172.21.15.146:0/1025596589 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22480045c0 con 0x7f2258150230 2024-09-13T18:34:34.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225e40e700 1 -- 172.21.15.146:0/1025596589 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2248009570 con 0x7f2258150230 2024-09-13T18:34:34.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f224dffb700 1 -- 172.21.15.146:0/1025596589 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248010ae0 con 0x7f2258150230 2024-09-13T18:34:34.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.262+0000 7f225e40e700 1 -- 172.21.15.146:0/1025596589 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22581eb270 con 0x7f2258150230 2024-09-13T18:34:34.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.266+0000 7f225e40e700 1 -- 172.21.15.146:0/1025596589 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2238005310 con 0x7f2258150230 2024-09-13T18:34:34.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.266+0000 7f224dffb700 1 -- 172.21.15.146:0/1025596589 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f22480036c0 con 0x7f2258150230 2024-09-13T18:34:34.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.266+0000 7f224dffb700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f223c066760 0x7f223c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:34.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.266+0000 7f224dffb700 1 -- 172.21.15.146:0/1025596589 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f2248085ea0 con 0x7f2258150230 2024-09-13T18:34:34.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.266+0000 7f225d40c700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f223c066760 0x7f223c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:34.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.266+0000 7f225d40c700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f223c066760 0x7f223c068c20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2254000f60 tx=0x7f2254007ef0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:34.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.274+0000 7f224dffb700 1 -- 172.21.15.146:0/1025596589 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2248052c10 con 0x7f2258150230 2024-09-13T18:34:34.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.506+0000 7f225e40e700 1 -- 172.21.15.146:0/1025596589 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2238005890 con 0x7f2258150230 2024-09-13T18:34:34.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.506+0000 7f224dffb700 1 -- 172.21.15.146:0/1025596589 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5537 (secure 0 0 0) 0x7f2248029070 con 0x7f2258150230 2024-09-13T18:34:34.520 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:34.520 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:30.691240+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:resolve","state_seq":18,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:34.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.510+0000 7f225e40e700 1 -- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f223c066760 msgr2=0x7f223c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:34.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.510+0000 7f225e40e700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f223c066760 0x7f223c068c20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2254000f60 tx=0x7f2254007ef0 comp rx=0 tx=0).stop 2024-09-13T18:34:34.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.510+0000 7f225e40e700 1 -- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2258150230 msgr2=0x7f2258154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:34.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.510+0000 7f225e40e700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2258150230 0x7f2258154bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f225806ca30 tx=0x7f2248004770 comp rx=0 tx=0).stop 2024-09-13T18:34:34.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.510+0000 7f225e40e700 1 -- 172.21.15.146:0/1025596589 shutdown_connections 2024-09-13T18:34:34.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.510+0000 7f225e40e700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f223c066760 0x7f223c068c20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:34.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.510+0000 7f225e40e700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2258155510 0x7f225813e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:34.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.510+0000 7f225e40e700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2258150230 0x7f2258154bf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:34.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.510+0000 7f225e40e700 1 --2- 172.21.15.146:0/1025596589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2258071610 0x7f2258154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:34.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.510+0000 7f225e40e700 1 -- 172.21.15.146:0/1025596589 >> 172.21.15.146:0/1025596589 conn(0x7f2258071ed0 msgr2=0x7f2258149540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:34.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.510+0000 7f225e40e700 1 -- 172.21.15.146:0/1025596589 shutdown_connections 2024-09-13T18:34:34.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:34.510+0000 7f225e40e700 1 -- 172.21.15.146:0/1025596589 wait complete. 2024-09-13T18:34:34.526 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 41 2024-09-13T18:34:34.541 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:34:36.542 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:36.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.642+0000 7ff848faf700 1 Processor -- start 2024-09-13T18:34:36.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.642+0000 7ff848faf700 1 -- start start 2024-09-13T18:34:36.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff848faf700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff84414fa50 0x7ff84406cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:36.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff848faf700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff8440c2460 0x7ff8440c7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:36.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff848faf700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff84406e130 con 0x7ff8440c75b0 2024-09-13T18:34:36.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff848faf700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8440c6c50 con 0x7ff8440c2460 2024-09-13T18:34:36.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff848faf700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8440c65d0 con 0x7ff84414fa50 2024-09-13T18:34:36.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff842ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff8440c2460 0x7ff8440c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:36.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff842ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff8440c2460 0x7ff8440c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60326/0 (socket says 172.21.15.146:60326) 2024-09-13T18:34:36.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff842ffd700 1 -- 172.21.15.146:0/3899720395 learned_addr learned my addr 172.21.15.146:0/3899720395 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:36.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff8437fe700 1 --2- 172.21.15.146:0/3899720395 >> v2:172.21.15.154:3300/0 conn(0x7ff84414fa50 0x7ff84406cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:36.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff842ffd700 1 -- 172.21.15.146:0/3899720395 >> v2:172.21.15.154:3300/0 conn(0x7ff84414fa50 msgr2=0x7ff84406cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:36.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff842ffd700 1 --2- 172.21.15.146:0/3899720395 >> v2:172.21.15.154:3300/0 conn(0x7ff84414fa50 0x7ff84406cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:36.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff842ffd700 1 -- 172.21.15.146:0/3899720395 >> v1:172.21.15.154:6789/0 conn(0x7ff8440c75b0 legacy=0x7ff8440c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:36.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff842ffd700 1 -- 172.21.15.146:0/3899720395 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84406e3f0 con 0x7ff8440c2460 2024-09-13T18:34:36.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff8437fe700 1 --2- 172.21.15.146:0/3899720395 >> v2:172.21.15.154:3300/0 conn(0x7ff84414fa50 0x7ff84406cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:36.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff842ffd700 1 --2- 172.21.15.146:0/3899720395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8440c2460 0x7ff8440c7070 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff8440cc4c0 tx=0x7ff83400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31126f8d9e6a5d09 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:36.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff8427fc700 1 -- 172.21.15.146:0/3899720395 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83400dd50 con 0x7ff8440c2460 2024-09-13T18:34:36.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff8427fc700 1 -- 172.21.15.146:0/3899720395 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff834004030 con 0x7ff8440c2460 2024-09-13T18:34:36.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff8427fc700 1 -- 172.21.15.146:0/3899720395 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8340043c0 con 0x7ff8440c2460 2024-09-13T18:34:36.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff848faf700 1 -- 172.21.15.146:0/3899720395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8440c2460 msgr2=0x7ff8440c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:36.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff848faf700 1 --2- 172.21.15.146:0/3899720395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8440c2460 0x7ff8440c7070 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff8440cc4c0 tx=0x7ff83400be90 comp rx=0 tx=0).stop 2024-09-13T18:34:36.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff848faf700 1 -- 172.21.15.146:0/3899720395 shutdown_connections 2024-09-13T18:34:36.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff848faf700 1 --2- 172.21.15.146:0/3899720395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8440c2460 0x7ff8440c7070 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:36.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff848faf700 1 --2- 172.21.15.146:0/3899720395 >> v2:172.21.15.154:3300/0 conn(0x7ff84414fa50 0x7ff84406cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:36.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff848faf700 1 -- 172.21.15.146:0/3899720395 >> 172.21.15.146:0/3899720395 conn(0x7ff844071080 msgr2=0x7ff844071490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:36.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff848faf700 1 -- 172.21.15.146:0/3899720395 shutdown_connections 2024-09-13T18:34:36.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.646+0000 7ff848faf700 1 -- 172.21.15.146:0/3899720395 wait complete. 2024-09-13T18:34:36.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff848faf700 1 Processor -- start 2024-09-13T18:34:36.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff848faf700 1 -- start start 2024-09-13T18:34:36.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff848faf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8440c2460 0x7ff8440c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:36.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff848faf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8440c75b0 0x7ff84406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:36.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff848faf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff84414fa50 0x7ff8441df3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:36.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff848faf700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff844154be0 con 0x7ff84414fa50 2024-09-13T18:34:36.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff848faf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff844152900 con 0x7ff8440c75b0 2024-09-13T18:34:36.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff848faf700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8441501c0 con 0x7ff8440c2460 2024-09-13T18:34:36.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff843fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff84414fa50 0x7ff8441df3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:36.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff843fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff84414fa50 0x7ff8441df3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60330/0 (socket says 172.21.15.146:60330) 2024-09-13T18:34:36.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff843fff700 1 -- 172.21.15.146:0/1099717695 learned_addr learned my addr 172.21.15.146:0/1099717695 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:36.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff842ffd700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8440c75b0 0x7ff84406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:36.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff8437fe700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8440c2460 0x7ff8440c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:36.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff843fff700 1 -- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8440c2460 msgr2=0x7ff8440c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:36.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff843fff700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8440c2460 0x7ff8440c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:36.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff843fff700 1 -- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8440c75b0 msgr2=0x7ff84406d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:36.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff843fff700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8440c75b0 0x7ff84406d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:36.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff843fff700 1 -- 172.21.15.146:0/1099717695 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83400b810 con 0x7ff84414fa50 2024-09-13T18:34:36.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff8437fe700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8440c2460 0x7ff8440c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:36.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff843fff700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff84414fa50 0x7ff8441df3f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff83800c4a0 tx=0x7ff83800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:36.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff82bfff700 1 -- 172.21.15.146:0/1099717695 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838011790 con 0x7ff84414fa50 2024-09-13T18:34:36.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff82bfff700 1 -- 172.21.15.146:0/1099717695 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8380118f0 con 0x7ff84414fa50 2024-09-13T18:34:36.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff848faf700 1 -- 172.21.15.146:0/1099717695 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8441509d0 con 0x7ff84414fa50 2024-09-13T18:34:36.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff82bfff700 1 -- 172.21.15.146:0/1099717695 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838011ca0 con 0x7ff84414fa50 2024-09-13T18:34:36.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.650+0000 7ff848faf700 1 -- 172.21.15.146:0/1099717695 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8441def60 con 0x7ff84414fa50 2024-09-13T18:34:36.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.654+0000 7ff848faf700 1 -- 172.21.15.146:0/1099717695 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8441df0f0 con 0x7ff84414fa50 2024-09-13T18:34:36.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.654+0000 7ff82bfff700 1 -- 172.21.15.146:0/1099717695 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff83801c460 con 0x7ff84414fa50 2024-09-13T18:34:36.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.654+0000 7ff82bfff700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff830066730 0x7ff830068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:36.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.654+0000 7ff82bfff700 1 -- 172.21.15.146:0/1099717695 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7ff838084360 con 0x7ff84414fa50 2024-09-13T18:34:36.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.654+0000 7ff8437fe700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff830066730 0x7ff830068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:36.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.654+0000 7ff8437fe700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff830066730 0x7ff830068bf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00e3d0 tx=0x7ff83c007a10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:36.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.662+0000 7ff82bfff700 1 -- 172.21.15.146:0/1099717695 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff838051160 con 0x7ff84414fa50 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.882+0000 7ff848faf700 1 -- 172.21.15.146:0/1099717695 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8440c67f0 con 0x7ff84414fa50 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.886+0000 7ff82bfff700 1 -- 172.21.15.146:0/1099717695 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+5539 (secure 0 0 0) 0x7ff838054780 con 0x7ff84414fa50 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:35.715618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:36.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.886+0000 7ff848faf700 1 -- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff830066730 msgr2=0x7ff830068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:36.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.886+0000 7ff848faf700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff830066730 0x7ff830068bf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00e3d0 tx=0x7ff83c007a10 comp rx=0 tx=0).stop 2024-09-13T18:34:36.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.886+0000 7ff848faf700 1 -- 172.21.15.146:0/1099717695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff84414fa50 msgr2=0x7ff8441df3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:36.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.886+0000 7ff848faf700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff84414fa50 0x7ff8441df3f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff83800c4a0 tx=0x7ff83800c7b0 comp rx=0 tx=0).stop 2024-09-13T18:34:36.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.886+0000 7ff848faf700 1 -- 172.21.15.146:0/1099717695 shutdown_connections 2024-09-13T18:34:36.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.886+0000 7ff848faf700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff830066730 0x7ff830068bf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:36.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.886+0000 7ff848faf700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff84414fa50 0x7ff8441df3f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:36.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.886+0000 7ff848faf700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8440c75b0 0x7ff84406d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:36.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.886+0000 7ff848faf700 1 --2- 172.21.15.146:0/1099717695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8440c2460 0x7ff8440c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:36.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.886+0000 7ff848faf700 1 -- 172.21.15.146:0/1099717695 >> 172.21.15.146:0/1099717695 conn(0x7ff844071080 msgr2=0x7ff84406f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:36.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.886+0000 7ff848faf700 1 -- 172.21.15.146:0/1099717695 shutdown_connections 2024-09-13T18:34:36.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:36.886+0000 7ff848faf700 1 -- 172.21.15.146:0/1099717695 wait complete. 2024-09-13T18:34:36.903 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 42 2024-09-13T18:34:36.913 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:34:38.399 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:34:38.402 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:34:38.914 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:39.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.014+0000 7f7f7759e700 1 Processor -- start 2024-09-13T18:34:39.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.014+0000 7f7f7759e700 1 -- start start 2024-09-13T18:34:39.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7759e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f78151080 0x7f7f7806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:39.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7759e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f78070dc0 con 0x7f7f780c6960 2024-09-13T18:34:39.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7759e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7806fb00 con 0x7f7f7806e650 2024-09-13T18:34:39.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7759e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f78157470 con 0x7f7f78151080 2024-09-13T18:34:39.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f75d9b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f7f780c6960 0x7f7f780cc1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:55200/0 (socket says 172.21.15.146:55200) 2024-09-13T18:34:39.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f75d9b700 1 -- 172.21.15.146:0/2190176555 learned_addr learned my addr 172.21.15.146:0/2190176555 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:34:39.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7659c700 1 --2- 172.21.15.146:0/2190176555 >> v2:172.21.15.154:3300/0 conn(0x7f7f78151080 0x7f7f7806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:39.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053378114 0 0) 0x7f7f78070dc0 con 0x7f7f780c6960 2024-09-13T18:34:39.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f600035d0 con 0x7f7f780c6960 2024-09-13T18:34:39.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987855717 0 0) 0x7f7f7806fb00 con 0x7f7f7806e650 2024-09-13T18:34:39.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f78070dc0 con 0x7f7f7806e650 2024-09-13T18:34:39.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2412446808 0 0) 0x7f7f600035d0 con 0x7f7f780c6960 2024-09-13T18:34:39.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 >> v2:172.21.15.154:3300/0 conn(0x7f7f78151080 msgr2=0x7f7f7806f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:39.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 --2- 172.21.15.146:0/2190176555 >> v2:172.21.15.154:3300/0 conn(0x7f7f78151080 0x7f7f7806f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:39.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 >> v1:172.21.15.154:6789/0 conn(0x7f7f7806e650 legacy=0x7f7f7806ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:39.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f78157750 con 0x7f7f780c6960 2024-09-13T18:34:39.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f7f6c001670 con 0x7f7f780c6960 2024-09-13T18:34:39.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 >> v1:172.21.15.146:6789/0 conn(0x7f7f780c6960 legacy=0x7f7f780cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:39.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 --2- 172.21.15.146:0/2190176555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f60003c20 0x7f7f60004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:39.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 --2- 172.21.15.146:0/2190176555 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f60004870 0x7f7f60004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:39.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 --2- 172.21.15.146:0/2190176555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f60005230 0x7f7f6000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:39.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f78157470 con 0x7f7f60003c20 2024-09-13T18:34:39.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6000c130 con 0x7f7f60005230 2024-09-13T18:34:39.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6000c2f0 con 0x7f7f60004870 2024-09-13T18:34:39.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7659c700 1 --2- 172.21.15.146:0/2190176555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f60003c20 0x7f7f60004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:39.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f76d9d700 1 --2- 172.21.15.146:0/2190176555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f60005230 0x7f7f6000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:39.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f75d9b700 1 --2- 172.21.15.146:0/2190176555 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f60004870 0x7f7f60004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:39.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7659c700 1 -- 172.21.15.146:0/2190176555 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f60004870 msgr2=0x7f7f60004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:39.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7659c700 1 --2- 172.21.15.146:0/2190176555 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f60004870 0x7f7f60004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:39.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7659c700 1 -- 172.21.15.146:0/2190176555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f60005230 msgr2=0x7f7f6000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:39.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7659c700 1 --2- 172.21.15.146:0/2190176555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f60005230 0x7f7f6000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:39.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7659c700 1 -- 172.21.15.146:0/2190176555 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f600035d0 con 0x7f7f60003c20 2024-09-13T18:34:39.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f76d9d700 1 --2- 172.21.15.146:0/2190176555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f60005230 0x7f7f6000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:39.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f76d9d700 1 -- 172.21.15.146:0/2190176555 reap_dead start 2024-09-13T18:34:39.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f75d9b700 1 --2- 172.21.15.146:0/2190176555 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f60004870 0x7f7f60004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:39.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7659c700 1 --2- 172.21.15.146:0/2190176555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f60003c20 0x7f7f60004020 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009b70 tx=0x7f7f600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e2f88086c6f036 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:39.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6800e6c0 con 0x7f7f60003c20 2024-09-13T18:34:39.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6800e820 con 0x7f7f60003c20 2024-09-13T18:34:39.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7559a700 1 -- 172.21.15.146:0/2190176555 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6800eb90 con 0x7f7f60003c20 2024-09-13T18:34:39.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7759e700 1 -- 172.21.15.146:0/2190176555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f60003c20 msgr2=0x7f7f60004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:39.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7759e700 1 --2- 172.21.15.146:0/2190176555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f60003c20 0x7f7f60004020 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009b70 tx=0x7f7f600058a0 comp rx=0 tx=0).stop 2024-09-13T18:34:39.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7759e700 1 -- 172.21.15.146:0/2190176555 shutdown_connections 2024-09-13T18:34:39.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7759e700 1 --2- 172.21.15.146:0/2190176555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f60003c20 0x7f7f60004020 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:39.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7759e700 1 -- 172.21.15.146:0/2190176555 >> 172.21.15.146:0/2190176555 conn(0x7f7f78071ed0 msgr2=0x7f7f780722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:39.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7759e700 1 -- 172.21.15.146:0/2190176555 shutdown_connections 2024-09-13T18:34:39.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.018+0000 7f7f7759e700 1 -- 172.21.15.146:0/2190176555 wait complete. 2024-09-13T18:34:39.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f7759e700 1 Processor -- start 2024-09-13T18:34:39.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f7759e700 1 -- start start 2024-09-13T18:34:39.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f7759e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f60003c20 0x7f7f780cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:39.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f7759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f780c2bc0 0x7f7f781f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:39.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f7759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f781417d0 0x7f7f78141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:39.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f7759e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f781530a0 con 0x7f7f780c2bc0 2024-09-13T18:34:39.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f7759e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f781f2e00 con 0x7f7f60003c20 2024-09-13T18:34:39.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f7759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f781ef830 con 0x7f7f781417d0 2024-09-13T18:34:39.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f75d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f780c2bc0 0x7f7f781f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:39.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f75d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f780c2bc0 0x7f7f781f3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60354/0 (socket says 172.21.15.146:60354) 2024-09-13T18:34:39.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f75d9b700 1 -- 172.21.15.146:0/2514065720 learned_addr learned my addr 172.21.15.146:0/2514065720 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:39.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f7659c700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f60003c20 0x7f7f780cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:39.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f76d9d700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f781417d0 0x7f7f78141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:39.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f75d9b700 1 -- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f781417d0 msgr2=0x7f7f78141c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:39.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f75d9b700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f781417d0 0x7f7f78141c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:39.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f75d9b700 1 -- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f60003c20 msgr2=0x7f7f780cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:39.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f75d9b700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f60003c20 0x7f7f780cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:39.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f75d9b700 1 -- 172.21.15.146:0/2514065720 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f78142190 con 0x7f7f780c2bc0 2024-09-13T18:34:39.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f76d9d700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f781417d0 0x7f7f78141c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:39.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.022+0000 7f7f75d9b700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f780c2bc0 0x7f7f781f3290 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00d7d0 tx=0x7f7f6c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:39.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.026+0000 7f7f66ffd700 1 -- 172.21.15.146:0/2514065720 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c004ca0 con 0x7f7f780c2bc0 2024-09-13T18:34:39.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.026+0000 7f7f66ffd700 1 -- 172.21.15.146:0/2514065720 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c004e00 con 0x7f7f780c2bc0 2024-09-13T18:34:39.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.026+0000 7f7f66ffd700 1 -- 172.21.15.146:0/2514065720 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c007310 con 0x7f7f780c2bc0 2024-09-13T18:34:39.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.026+0000 7f7f7759e700 1 -- 172.21.15.146:0/2514065720 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f781efaf0 con 0x7f7f780c2bc0 2024-09-13T18:34:39.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.026+0000 7f7f7759e700 1 -- 172.21.15.146:0/2514065720 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f781f00b0 con 0x7f7f780c2bc0 2024-09-13T18:34:39.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.026+0000 7f7f7759e700 1 -- 172.21.15.146:0/2514065720 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f78157140 con 0x7f7f780c2bc0 2024-09-13T18:34:39.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.026+0000 7f7f66ffd700 1 -- 172.21.15.146:0/2514065720 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7f6c007470 con 0x7f7f780c2bc0 2024-09-13T18:34:39.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.026+0000 7f7f66ffd700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7f58066730 0x7f7f58068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:39.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.026+0000 7f7f66ffd700 1 -- 172.21.15.146:0/2514065720 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f7f6c084660 con 0x7f7f780c2bc0 2024-09-13T18:34:39.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.026+0000 7f7f7659c700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7f58066730 0x7f7f58068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:39.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.030+0000 7f7f7659c700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7f58066730 0x7f7f58068bf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7f680090b0 tx=0x7f7f6801e040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:39.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.034+0000 7f7f66ffd700 1 -- 172.21.15.146:0/2514065720 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7f6c0513d0 con 0x7f7f780c2bc0 2024-09-13T18:34:39.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.266+0000 7f7f7759e700 1 -- 172.21.15.146:0/2514065720 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f780c74c0 con 0x7f7f780c2bc0 2024-09-13T18:34:39.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.266+0000 7f7f66ffd700 1 -- 172.21.15.146:0/2514065720 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+5539 (secure 0 0 0) 0x7f7f6c007720 con 0x7f7f780c2bc0 2024-09-13T18:34:39.279 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:39.280 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:35.715618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:39.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.270+0000 7f7f7759e700 1 -- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7f58066730 msgr2=0x7f7f58068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:39.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.270+0000 7f7f7759e700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7f58066730 0x7f7f58068bf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7f680090b0 tx=0x7f7f6801e040 comp rx=0 tx=0).stop 2024-09-13T18:34:39.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.270+0000 7f7f7759e700 1 -- 172.21.15.146:0/2514065720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f780c2bc0 msgr2=0x7f7f781f3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:39.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.270+0000 7f7f7759e700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f780c2bc0 0x7f7f781f3290 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00d7d0 tx=0x7f7f6c00dae0 comp rx=0 tx=0).stop 2024-09-13T18:34:39.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.270+0000 7f7f7759e700 1 -- 172.21.15.146:0/2514065720 shutdown_connections 2024-09-13T18:34:39.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.270+0000 7f7f7759e700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7f58066730 0x7f7f58068bf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:39.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.270+0000 7f7f7759e700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f781417d0 0x7f7f78141c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:39.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.270+0000 7f7f7759e700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f780c2bc0 0x7f7f781f3290 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:39.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.270+0000 7f7f7759e700 1 --2- 172.21.15.146:0/2514065720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f60003c20 0x7f7f780cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:39.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.270+0000 7f7f7759e700 1 -- 172.21.15.146:0/2514065720 >> 172.21.15.146:0/2514065720 conn(0x7f7f78071ed0 msgr2=0x7f7f7806ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:39.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.270+0000 7f7f7759e700 1 -- 172.21.15.146:0/2514065720 shutdown_connections 2024-09-13T18:34:39.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:39.270+0000 7f7f7759e700 1 -- 172.21.15.146:0/2514065720 wait complete. 2024-09-13T18:34:39.285 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 42 2024-09-13T18:34:39.299 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 42, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 42, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5278": { "addr": "172.21.15.154:6839/3274220802", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 3274220802, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 3274220802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5278, "incarnation": 28, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 19 }, "gid_5505": { "addr": "172.21.15.146:6835/1652676664", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 1652676664, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 1652676664, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5505, "incarnation": 34, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 38 }, "gid_5649": { "addr": "172.21.15.146:6837/3926931671", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3926931671, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3926931671, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5649, "incarnation": 40, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:reconnect", "state_seq": 20 } }, "last_failure": 0, "last_failure_osd_epoch": 32, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:34:35.715618+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5505, "mds_1": 5278, "mds_2": 5649 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:34:41.300 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:41.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.414+0000 7f3c8e3c9700 1 Processor -- start 2024-09-13T18:34:41.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.414+0000 7f3c8e3c9700 1 -- start start 2024-09-13T18:34:41.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.414+0000 7f3c8e3c9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c8806cef0 0x7f3c88159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:41.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.414+0000 7f3c8e3c9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3c88155bb0 0x7f3c880c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:41.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.414+0000 7f3c8e3c9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c88073c10 0x7f3c880c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:41.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.414+0000 7f3c8e3c9700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8815a0b0 con 0x7f3c88073c10 2024-09-13T18:34:41.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.414+0000 7f3c8e3c9700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c88159120 con 0x7f3c88155bb0 2024-09-13T18:34:41.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.414+0000 7f3c8e3c9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c881596e0 con 0x7f3c8806cef0 2024-09-13T18:34:41.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.414+0000 7f3c8dbc8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c88073c10 0x7f3c880c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:41.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.414+0000 7f3c8cbc6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3c88155bb0 0x7f3c880c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:41.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.414+0000 7f3c8dbc8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c88073c10 0x7f3c880c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49204/0 (socket says 172.21.15.146:49204) 2024-09-13T18:34:41.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.414+0000 7f3c8dbc8700 1 -- 172.21.15.146:0/729730704 learned_addr learned my addr 172.21.15.146:0/729730704 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:41.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.414+0000 7f3c8d3c7700 1 --2- 172.21.15.146:0/729730704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c8806cef0 0x7f3c88159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:41.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8dbc8700 1 -- 172.21.15.146:0/729730704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c8806cef0 msgr2=0x7f3c88159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:41.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8dbc8700 1 --2- 172.21.15.146:0/729730704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c8806cef0 0x7f3c88159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:41.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8dbc8700 1 -- 172.21.15.146:0/729730704 >> v2:172.21.15.154:3300/0 conn(0x7f3c88155bb0 msgr2=0x7f3c880c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:41.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8dbc8700 1 --2- 172.21.15.146:0/729730704 >> v2:172.21.15.154:3300/0 conn(0x7f3c88155bb0 0x7f3c880c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:41.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8dbc8700 1 -- 172.21.15.146:0/729730704 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c881563d0 con 0x7f3c88073c10 2024-09-13T18:34:41.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8cbc6700 1 --2- 172.21.15.146:0/729730704 >> v2:172.21.15.154:3300/0 conn(0x7f3c88155bb0 0x7f3c880c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:41.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8dbc8700 1 --2- 172.21.15.146:0/729730704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c88073c10 0x7f3c880c7760 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3c880708f0 tx=0x7f3c8000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30e944d7820c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:41.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c7ffff700 1 -- 172.21.15.146:0/729730704 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8000e070 con 0x7f3c88073c10 2024-09-13T18:34:41.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c7ffff700 1 -- 172.21.15.146:0/729730704 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c800090d0 con 0x7f3c88073c10 2024-09-13T18:34:41.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c7ffff700 1 -- 172.21.15.146:0/729730704 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8000bd60 con 0x7f3c88073c10 2024-09-13T18:34:41.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/729730704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c88073c10 msgr2=0x7f3c880c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:41.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8e3c9700 1 --2- 172.21.15.146:0/729730704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c88073c10 0x7f3c880c7760 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3c880708f0 tx=0x7f3c8000b480 comp rx=0 tx=0).stop 2024-09-13T18:34:41.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/729730704 shutdown_connections 2024-09-13T18:34:41.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8e3c9700 1 --2- 172.21.15.146:0/729730704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c88073c10 0x7f3c880c7760 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:41.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8e3c9700 1 --2- 172.21.15.146:0/729730704 >> v2:172.21.15.154:3300/0 conn(0x7f3c88155bb0 0x7f3c880c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:41.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8e3c9700 1 --2- 172.21.15.146:0/729730704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c8806cef0 0x7f3c88159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:41.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/729730704 >> 172.21.15.146:0/729730704 conn(0x7f3c88072270 msgr2=0x7f3c8806ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:41.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/729730704 shutdown_connections 2024-09-13T18:34:41.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.418+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/729730704 wait complete. 2024-09-13T18:34:41.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8e3c9700 1 Processor -- start 2024-09-13T18:34:41.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8e3c9700 1 -- start start 2024-09-13T18:34:41.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8e3c9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c8806cef0 0x7f3c880ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:41.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8e3c9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c88073c10 0x7f3c88145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:41.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8e3c9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c88155bb0 0x7f3c880b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:41.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8e3c9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8814c0d0 con 0x7f3c88073c10 2024-09-13T18:34:41.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8e3c9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c880ca560 con 0x7f3c8806cef0 2024-09-13T18:34:41.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8e3c9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c880c1f10 con 0x7f3c88155bb0 2024-09-13T18:34:41.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8cbc6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c88073c10 0x7f3c88145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:41.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8cbc6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c88073c10 0x7f3c88145940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49210/0 (socket says 172.21.15.146:49210) 2024-09-13T18:34:41.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8cbc6700 1 -- 172.21.15.146:0/430179161 learned_addr learned my addr 172.21.15.146:0/430179161 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:41.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8d3c7700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c8806cef0 0x7f3c880ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:41.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8dbc8700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c88155bb0 0x7f3c880b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:41.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8cbc6700 1 -- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c88155bb0 msgr2=0x7f3c880b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:41.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8cbc6700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c88155bb0 0x7f3c880b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:41.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8cbc6700 1 -- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c8806cef0 msgr2=0x7f3c880ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:41.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8cbc6700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c8806cef0 0x7f3c880ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:41.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8cbc6700 1 -- 172.21.15.146:0/430179161 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c780094b0 con 0x7f3c88073c10 2024-09-13T18:34:41.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8cbc6700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c88073c10 0x7f3c88145940 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800ec10 tx=0x7f3c78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:41.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c7dffb700 1 -- 172.21.15.146:0/430179161 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7800c8e0 con 0x7f3c88073c10 2024-09-13T18:34:41.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c7dffb700 1 -- 172.21.15.146:0/430179161 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7800ca40 con 0x7f3c88073c10 2024-09-13T18:34:41.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/430179161 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c80009b30 con 0x7f3c88073c10 2024-09-13T18:34:41.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c7dffb700 1 -- 172.21.15.146:0/430179161 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7800cdd0 con 0x7f3c88073c10 2024-09-13T18:34:41.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/430179161 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c880c9f10 con 0x7f3c88073c10 2024-09-13T18:34:41.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/430179161 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c880c6df0 con 0x7f3c88073c10 2024-09-13T18:34:41.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c7dffb700 1 -- 172.21.15.146:0/430179161 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3c78013470 con 0x7f3c88073c10 2024-09-13T18:34:41.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.422+0000 7f3c7dffb700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3c6c066730 0x7f3c6c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:41.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.426+0000 7f3c7dffb700 1 -- 172.21.15.146:0/430179161 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f3c78084a90 con 0x7f3c88073c10 2024-09-13T18:34:41.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.426+0000 7f3c8d3c7700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3c6c066730 0x7f3c6c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:41.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.426+0000 7f3c8d3c7700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3c6c066730 0x7f3c6c068bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009b80 tx=0x7f3c84008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:41.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.430+0000 7f3c7dffb700 1 -- 172.21.15.146:0/430179161 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3c78051920 con 0x7f3c88073c10 2024-09-13T18:34:41.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.694+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/430179161 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c880c8770 con 0x7f3c88073c10 2024-09-13T18:34:41.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.694+0000 7f3c7dffb700 1 -- 172.21.15.146:0/430179161 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+5536 (secure 0 0 0) 0x7f3c78017090 con 0x7f3c88073c10 2024-09-13T18:34:41.708 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:41.708 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:40.743519+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:41.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.698+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3c6c066730 msgr2=0x7f3c6c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:41.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.698+0000 7f3c8e3c9700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3c6c066730 0x7f3c6c068bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009b80 tx=0x7f3c84008040 comp rx=0 tx=0).stop 2024-09-13T18:34:41.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.698+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/430179161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c88073c10 msgr2=0x7f3c88145940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:41.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.698+0000 7f3c8e3c9700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c88073c10 0x7f3c88145940 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800ec10 tx=0x7f3c78009e30 comp rx=0 tx=0).stop 2024-09-13T18:34:41.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.698+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/430179161 shutdown_connections 2024-09-13T18:34:41.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.698+0000 7f3c8e3c9700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3c6c066730 0x7f3c6c068bf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:41.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.698+0000 7f3c8e3c9700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c88155bb0 0x7f3c880b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:41.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.698+0000 7f3c8e3c9700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c88073c10 0x7f3c88145940 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:41.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.698+0000 7f3c8e3c9700 1 --2- 172.21.15.146:0/430179161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c8806cef0 0x7f3c880ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:41.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.698+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/430179161 >> 172.21.15.146:0/430179161 conn(0x7f3c88072270 msgr2=0x7f3c8814ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:41.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.698+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/430179161 shutdown_connections 2024-09-13T18:34:41.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:41.698+0000 7f3c8e3c9700 1 -- 172.21.15.146:0/430179161 wait complete. 2024-09-13T18:34:41.712 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 43 2024-09-13T18:34:41.723 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:34:43.724 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:43.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f0684fad700 1 Processor -- start 2024-09-13T18:34:43.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f0684fad700 1 -- start start 2024-09-13T18:34:43.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f0684fad700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f068006f830 0x7f068006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:43.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f0684fad700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0680071810 con 0x7f06800ccc60 2024-09-13T18:34:43.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f0684fad700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0680157510 con 0x7f06800723b0 2024-09-13T18:34:43.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f0684fad700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06801577d0 con 0x7f068006f830 2024-09-13T18:34:43.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067dd9b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f06800ccc60 0x7f0680071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:52122/0 (socket says 172.21.15.146:52122) 2024-09-13T18:34:43.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067dd9b700 1 -- 172.21.15.146:0/1178161699 learned_addr learned my addr 172.21.15.146:0/1178161699 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:34:43.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067e59c700 1 --2- 172.21.15.146:0/1178161699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f068006f830 0x7f068006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:43.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024209662 0 0) 0x7f0680071810 con 0x7f06800ccc60 2024-09-13T18:34:43.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06680035d0 con 0x7f06800ccc60 2024-09-13T18:34:43.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099206866 0 0) 0x7f0680157510 con 0x7f06800723b0 2024-09-13T18:34:43.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0680071810 con 0x7f06800723b0 2024-09-13T18:34:43.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2575094933 0 0) 0x7f06680035d0 con 0x7f06800ccc60 2024-09-13T18:34:43.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f068006f830 msgr2=0x7f068006fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:43.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 --2- 172.21.15.146:0/1178161699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f068006f830 0x7f068006fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:43.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 >> v1:172.21.15.154:6789/0 conn(0x7f06800723b0 legacy=0x7f0680151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:43.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0680157ab0 con 0x7f06800ccc60 2024-09-13T18:34:43.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f0674001670 con 0x7f06800ccc60 2024-09-13T18:34:43.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 >> v1:172.21.15.146:6789/0 conn(0x7f06800ccc60 legacy=0x7f0680071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:43.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 --2- 172.21.15.146:0/1178161699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0668003be0 0x7f0668003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 --2- 172.21.15.146:0/1178161699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f068006f830 0x7f0668004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 --2- 172.21.15.146:0/1178161699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0668004db0 0x7f0668005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06801577d0 con 0x7f0668003be0 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066800bae0 con 0x7f0668004db0 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066800bca0 con 0x7f068006f830 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067e59c700 1 --2- 172.21.15.146:0/1178161699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0668003be0 0x7f0668003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.830+0000 7f067dd9b700 1 --2- 172.21.15.146:0/1178161699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f068006f830 0x7f0668004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f067e59c700 1 -- 172.21.15.146:0/1178161699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f068006f830 msgr2=0x7f0668004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f067e59c700 1 --2- 172.21.15.146:0/1178161699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f068006f830 0x7f0668004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f067e59c700 1 -- 172.21.15.146:0/1178161699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0668004db0 msgr2=0x7f0668005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f067e59c700 1 --2- 172.21.15.146:0/1178161699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0668004db0 0x7f0668005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f067e59c700 1 -- 172.21.15.146:0/1178161699 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06680035d0 con 0x7f0668003be0 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f067dd9b700 1 --2- 172.21.15.146:0/1178161699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f068006f830 0x7f0668004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:43.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f067e59c700 1 --2- 172.21.15.146:0/1178161699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0668003be0 0x7f0668003fe0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0670009b70 tx=0x7f06680058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef32798978cc45bd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:43.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067000e6c0 con 0x7f0668003be0 2024-09-13T18:34:43.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067000e820 con 0x7f0668003be0 2024-09-13T18:34:43.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f067d59a700 1 -- 172.21.15.146:0/1178161699 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067000eb90 con 0x7f0668003be0 2024-09-13T18:34:43.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 -- 172.21.15.146:0/1178161699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0668003be0 msgr2=0x7f0668003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:43.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 --2- 172.21.15.146:0/1178161699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0668003be0 0x7f0668003fe0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0670009b70 tx=0x7f06680058a0 comp rx=0 tx=0).stop 2024-09-13T18:34:43.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f067ed9d700 1 -- 172.21.15.146:0/1178161699 reap_dead start 2024-09-13T18:34:43.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 -- 172.21.15.146:0/1178161699 shutdown_connections 2024-09-13T18:34:43.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 -- 172.21.15.146:0/1178161699 >> 172.21.15.146:0/1178161699 conn(0x7f06800cc140 msgr2=0x7f06800cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:43.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 -- 172.21.15.146:0/1178161699 shutdown_connections 2024-09-13T18:34:43.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 -- 172.21.15.146:0/1178161699 wait complete. 2024-09-13T18:34:43.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 Processor -- start 2024-09-13T18:34:43.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 -- start start 2024-09-13T18:34:43.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0668003be0 0x7f0680155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:43.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06800cb9d0 0x7f06800adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06800c1160 0x7f06801444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0680156230 con 0x7f06800cb9d0 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06801f3110 con 0x7f0668003be0 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.834+0000 7f0684fad700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06801f33f0 con 0x7f06800c1160 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f067dd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06800cb9d0 0x7f06800adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f067dd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06800cb9d0 0x7f06800adb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49228/0 (socket says 172.21.15.146:49228) 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f067dd9b700 1 -- 172.21.15.146:0/843351507 learned_addr learned my addr 172.21.15.146:0/843351507 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f067e59c700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0668003be0 0x7f0680155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f067ed9d700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06800c1160 0x7f06801444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f067dd9b700 1 -- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06800c1160 msgr2=0x7f06801444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f067dd9b700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06800c1160 0x7f06801444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f067dd9b700 1 -- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0668003be0 msgr2=0x7f0680155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f067dd9b700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0668003be0 0x7f0680155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:43.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f067dd9b700 1 -- 172.21.15.146:0/843351507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0680155210 con 0x7f06800cb9d0 2024-09-13T18:34:43.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f067dd9b700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06800cb9d0 0x7f06800adb10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f067400dae0 tx=0x7f067400ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:43.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f066effd700 1 -- 172.21.15.146:0/843351507 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674004ca0 con 0x7f06800cb9d0 2024-09-13T18:34:43.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f066effd700 1 -- 172.21.15.146:0/843351507 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0674004e00 con 0x7f06800cb9d0 2024-09-13T18:34:43.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f0684fad700 1 -- 172.21.15.146:0/843351507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06801554f0 con 0x7f06800cb9d0 2024-09-13T18:34:43.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f066effd700 1 -- 172.21.15.146:0/843351507 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06740072f0 con 0x7f06800cb9d0 2024-09-13T18:34:43.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f0684fad700 1 -- 172.21.15.146:0/843351507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0680145340 con 0x7f06800cb9d0 2024-09-13T18:34:43.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f0684fad700 1 -- 172.21.15.146:0/843351507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0680071ca0 con 0x7f06800cb9d0 2024-09-13T18:34:43.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f066effd700 1 -- 172.21.15.146:0/843351507 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0674007450 con 0x7f06800cb9d0 2024-09-13T18:34:43.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f066effd700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0660066760 0x7f0660068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:43.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.838+0000 7f066effd700 1 -- 172.21.15.146:0/843351507 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f0674083f70 con 0x7f06800cb9d0 2024-09-13T18:34:43.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.842+0000 7f067e59c700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0660066760 0x7f0660068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:43.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.842+0000 7f067e59c700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0660066760 0x7f0660068c20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f06700090b0 tx=0x7f067001e040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:43.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:43.846+0000 7f066effd700 1 -- 172.21.15.146:0/843351507 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0674050c50 con 0x7f06800cb9d0 2024-09-13T18:34:44.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f0684fad700 1 -- 172.21.15.146:0/843351507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06800aeaa0 con 0x7f06800cb9d0 2024-09-13T18:34:44.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f066effd700 1 -- 172.21.15.146:0/843351507 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+5536 (secure 0 0 0) 0x7f0674019090 con 0x7f06800cb9d0 2024-09-13T18:34:44.091 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:44.091 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:40.743519+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:44.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f0684fad700 1 -- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0660066760 msgr2=0x7f0660068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:44.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f0684fad700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0660066760 0x7f0660068c20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f06700090b0 tx=0x7f067001e040 comp rx=0 tx=0).stop 2024-09-13T18:34:44.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f0684fad700 1 -- 172.21.15.146:0/843351507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06800cb9d0 msgr2=0x7f06800adb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:44.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f0684fad700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06800cb9d0 0x7f06800adb10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f067400dae0 tx=0x7f067400ddf0 comp rx=0 tx=0).stop 2024-09-13T18:34:44.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f0684fad700 1 -- 172.21.15.146:0/843351507 shutdown_connections 2024-09-13T18:34:44.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f0684fad700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0660066760 0x7f0660068c20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:44.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f0684fad700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06800c1160 0x7f06801444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:44.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f0684fad700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06800cb9d0 0x7f06800adb10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:44.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f0684fad700 1 --2- 172.21.15.146:0/843351507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0668003be0 0x7f0680155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:44.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f0684fad700 1 -- 172.21.15.146:0/843351507 >> 172.21.15.146:0/843351507 conn(0x7f06800cc140 msgr2=0x7f0680150150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:44.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f0684fad700 1 -- 172.21.15.146:0/843351507 shutdown_connections 2024-09-13T18:34:44.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:44.078+0000 7f0684fad700 1 -- 172.21.15.146:0/843351507 wait complete. 2024-09-13T18:34:44.095 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 43 2024-09-13T18:34:44.108 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:34:46.026 INFO:tasks.ceph.mds.a.smithi146.stderr:2024-09-13T18:34:46.014+0000 7f95a2b95700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-13T18:34:46.109 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:46.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.218+0000 7f07a60c1700 1 Processor -- start 2024-09-13T18:34:46.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.218+0000 7f07a60c1700 1 -- start start 2024-09-13T18:34:46.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.218+0000 7f07a60c1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00cfb70 0x7f07a00c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:46.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.218+0000 7f07a60c1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f07a0158090 0x7f07a00c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:46.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.218+0000 7f07a60c1700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a00c9620 con 0x7f07a00cf790 2024-09-13T18:34:46.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.218+0000 7f07a60c1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a015cb00 con 0x7f07a0158090 2024-09-13T18:34:46.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.218+0000 7f07a60c1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a01541b0 con 0x7f07a00cfb70 2024-09-13T18:34:46.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.218+0000 7f079f7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f07a00cf790 0x7f07a0157980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:52134/0 (socket says 172.21.15.146:52134) 2024-09-13T18:34:46.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.218+0000 7f079f7fe700 1 -- 172.21.15.146:0/3347093043 learned_addr learned my addr 172.21.15.146:0/3347093043 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:34:46.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f079ffff700 1 --2- 172.21.15.146:0/3347093043 >> v2:172.21.15.154:3300/0 conn(0x7f07a0158090 0x7f07a00c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:46.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f079e7fc700 1 -- 172.21.15.146:0/3347093043 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395150626 0 0) 0x7f0794001110 con 0x7f07a00cf790 2024-09-13T18:34:46.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f079e7fc700 1 -- 172.21.15.146:0/3347093043 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07880035d0 con 0x7f07a00cf790 2024-09-13T18:34:46.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f079effd700 1 --2- 172.21.15.146:0/3347093043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00cfb70 0x7f07a00c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:46.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f079ffff700 1 -- 172.21.15.146:0/3347093043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00cfb70 msgr2=0x7f07a00c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:46.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f079ffff700 1 --2- 172.21.15.146:0/3347093043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00cfb70 0x7f07a00c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:46.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f079ffff700 1 -- 172.21.15.146:0/3347093043 >> v1:172.21.15.146:6789/0 conn(0x7f07a00cf790 legacy=0x7f07a0157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:46.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f079ffff700 1 -- 172.21.15.146:0/3347093043 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a015bba0 con 0x7f07a0158090 2024-09-13T18:34:46.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f079ffff700 1 --2- 172.21.15.146:0/3347093043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0158090 0x7f07a00c90e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0798009d90 tx=0x7f079800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=debef4901a5c183e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:46.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f079e7fc700 1 -- 172.21.15.146:0/3347093043 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079800e070 con 0x7f07a0158090 2024-09-13T18:34:46.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f079e7fc700 1 -- 172.21.15.146:0/3347093043 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079800bae0 con 0x7f07a0158090 2024-09-13T18:34:46.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f079e7fc700 1 -- 172.21.15.146:0/3347093043 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079800be50 con 0x7f07a0158090 2024-09-13T18:34:46.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f07a60c1700 1 -- 172.21.15.146:0/3347093043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0158090 msgr2=0x7f07a00c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:46.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f07a60c1700 1 --2- 172.21.15.146:0/3347093043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0158090 0x7f07a00c90e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0798009d90 tx=0x7f079800b330 comp rx=0 tx=0).stop 2024-09-13T18:34:46.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f07a60c1700 1 -- 172.21.15.146:0/3347093043 shutdown_connections 2024-09-13T18:34:46.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f07a60c1700 1 --2- 172.21.15.146:0/3347093043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0158090 0x7f07a00c90e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:46.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f07a60c1700 1 --2- 172.21.15.146:0/3347093043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00cfb70 0x7f07a00c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:46.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f07a60c1700 1 -- 172.21.15.146:0/3347093043 >> 172.21.15.146:0/3347093043 conn(0x7f07a0071d90 msgr2=0x7f07a00721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:46.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f07a60c1700 1 -- 172.21.15.146:0/3347093043 shutdown_connections 2024-09-13T18:34:46.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.222+0000 7f07a60c1700 1 -- 172.21.15.146:0/3347093043 wait complete. 2024-09-13T18:34:46.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f07a60c1700 1 Processor -- start 2024-09-13T18:34:46.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f07a60c1700 1 -- start start 2024-09-13T18:34:46.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f07a60c1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07a00cf790 0x7f07a01f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:46.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f07a60c1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00cfb70 0x7f07a0158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:46.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f07a60c1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0158090 0x7f07a00c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:46.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f07a60c1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a0156710 con 0x7f07a00cf790 2024-09-13T18:34:46.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f07a60c1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a0154730 con 0x7f07a0158090 2024-09-13T18:34:46.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f07a60c1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a01549f0 con 0x7f07a00cfb70 2024-09-13T18:34:46.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f079f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07a00cf790 0x7f07a01f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:46.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f079f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07a00cf790 0x7f07a01f29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49230/0 (socket says 172.21.15.146:49230) 2024-09-13T18:34:46.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f079f7fe700 1 -- 172.21.15.146:0/4057450568 learned_addr learned my addr 172.21.15.146:0/4057450568 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:46.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f079ffff700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0158090 0x7f07a00c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:46.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f079effd700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00cfb70 0x7f07a0158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:46.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f079f7fe700 1 -- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00cfb70 msgr2=0x7f07a0158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:46.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f079f7fe700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00cfb70 0x7f07a0158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:46.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f079f7fe700 1 -- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0158090 msgr2=0x7f07a00c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:46.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f079f7fe700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0158090 0x7f07a00c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:46.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f079f7fe700 1 -- 172.21.15.146:0/4057450568 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0798009a70 con 0x7f07a00cf790 2024-09-13T18:34:46.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f079f7fe700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07a00cf790 0x7f07a01f29b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f079400c4a0 tx=0x7f079400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:46.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f0787fff700 1 -- 172.21.15.146:0/4057450568 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794004ca0 con 0x7f07a00cf790 2024-09-13T18:34:46.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f0787fff700 1 -- 172.21.15.146:0/4057450568 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0794004e00 con 0x7f07a00cf790 2024-09-13T18:34:46.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f07a60c1700 1 -- 172.21.15.146:0/4057450568 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a0154cd0 con 0x7f07a00cf790 2024-09-13T18:34:46.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f0787fff700 1 -- 172.21.15.146:0/4057450568 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794007310 con 0x7f07a00cf790 2024-09-13T18:34:46.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.226+0000 7f07a60c1700 1 -- 172.21.15.146:0/4057450568 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a01f2490 con 0x7f07a00cf790 2024-09-13T18:34:46.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.230+0000 7f07a60c1700 1 -- 172.21.15.146:0/4057450568 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a01f26b0 con 0x7f07a00cf790 2024-09-13T18:34:46.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.230+0000 7f0787fff700 1 -- 172.21.15.146:0/4057450568 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0794007470 con 0x7f07a00cf790 2024-09-13T18:34:46.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.230+0000 7f0787fff700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f078c066730 0x7f078c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:46.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.230+0000 7f0787fff700 1 -- 172.21.15.146:0/4057450568 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f0794084170 con 0x7f07a00cf790 2024-09-13T18:34:46.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.230+0000 7f079effd700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f078c066730 0x7f078c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.230+0000 7f079effd700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f078c066730 0x7f078c068bf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f07a00ca800 tx=0x7f0790009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:46.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.234+0000 7f0787fff700 1 -- 172.21.15.146:0/4057450568 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0794050ee0 con 0x7f07a00cf790 2024-09-13T18:34:46.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.470+0000 7f07a60c1700 1 -- 172.21.15.146:0/4057450568 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07a0159bd0 con 0x7f07a00cf790 2024-09-13T18:34:46.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.474+0000 7f0787fff700 1 -- 172.21.15.146:0/4057450568 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+5536 (secure 0 0 0) 0x7f0794007720 con 0x7f07a00cf790 2024-09-13T18:34:46.487 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:46.487 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:45.771339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:46.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.474+0000 7f07a60c1700 1 -- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f078c066730 msgr2=0x7f078c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.474+0000 7f07a60c1700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f078c066730 0x7f078c068bf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f07a00ca800 tx=0x7f0790009240 comp rx=0 tx=0).stop 2024-09-13T18:34:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.474+0000 7f07a60c1700 1 -- 172.21.15.146:0/4057450568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07a00cf790 msgr2=0x7f07a01f29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.474+0000 7f07a60c1700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07a00cf790 0x7f07a01f29b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f079400c4a0 tx=0x7f079400c7b0 comp rx=0 tx=0).stop 2024-09-13T18:34:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.474+0000 7f07a60c1700 1 -- 172.21.15.146:0/4057450568 shutdown_connections 2024-09-13T18:34:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.474+0000 7f07a60c1700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f078c066730 0x7f078c068bf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.474+0000 7f07a60c1700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0158090 0x7f07a00c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.474+0000 7f07a60c1700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00cfb70 0x7f07a0158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.474+0000 7f07a60c1700 1 --2- 172.21.15.146:0/4057450568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07a00cf790 0x7f07a01f29b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.474+0000 7f07a60c1700 1 -- 172.21.15.146:0/4057450568 >> 172.21.15.146:0/4057450568 conn(0x7f07a0071d90 msgr2=0x7f07a0072170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.474+0000 7f07a60c1700 1 -- 172.21.15.146:0/4057450568 shutdown_connections 2024-09-13T18:34:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:46.474+0000 7f07a60c1700 1 -- 172.21.15.146:0/4057450568 wait complete. 2024-09-13T18:34:46.491 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 44 2024-09-13T18:34:46.501 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=2, replacing gid=5235 2024-09-13T18:34:46.502 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.e 2024-09-13T18:34:51.503 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2024-09-13T18:34:51.503 INFO:tasks.ceph.mds.e:Restarting daemon 2024-09-13T18:34:51.503 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-09-13T18:34:51.506 INFO:tasks.ceph.mds.e:Started 2024-09-13T18:34:51.506 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:51.604 INFO:tasks.ceph.mds.e.smithi146.stdout:starting mds.e at 2024-09-13T18:34:51.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f849c4700 1 Processor -- start 2024-09-13T18:34:51.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f849c4700 1 -- start start 2024-09-13T18:34:51.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f849c4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f800cbd50 0x7f2f800b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:51.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f849c4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f800ca210 0x7f2f800cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:51.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f849c4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2f80071390 0x7f2f800701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:51.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f849c4700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f800caa30 con 0x7f2f80071390 2024-09-13T18:34:51.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f849c4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f800cb810 con 0x7f2f800ca210 2024-09-13T18:34:51.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f849c4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800cc670 con 0x7f2f800cbd50 2024-09-13T18:34:51.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f7ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2f80071390 0x7f2f800701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:51.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f7ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2f80071390 0x7f2f800701f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:47224/0 (socket says 172.21.15.146:47224) 2024-09-13T18:34:51.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f7ffff700 1 -- 172.21.15.146:0/3996345738 learned_addr learned my addr 172.21.15.146:0/3996345738 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:51.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f7effd700 1 --2- 172.21.15.146:0/3996345738 >> v2:172.21.15.154:3300/0 conn(0x7f2f800ca210 0x7f2f800cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:51.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f7f7fe700 1 --2- 172.21.15.146:0/3996345738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f800cbd50 0x7f2f800b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:51.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f7ffff700 1 -- 172.21.15.146:0/3996345738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f800cbd50 msgr2=0x7f2f800b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:51.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f7ffff700 1 --2- 172.21.15.146:0/3996345738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f800cbd50 0x7f2f800b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:51.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f7ffff700 1 -- 172.21.15.146:0/3996345738 >> v2:172.21.15.154:3300/0 conn(0x7f2f800ca210 msgr2=0x7f2f800cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:51.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f7ffff700 1 --2- 172.21.15.146:0/3996345738 >> v2:172.21.15.154:3300/0 conn(0x7f2f800ca210 0x7f2f800cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:51.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f7ffff700 1 -- 172.21.15.146:0/3996345738 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8015da20 con 0x7f2f80071390 2024-09-13T18:34:51.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.617+0000 7f2f7ffff700 1 --2- 172.21.15.146:0/3996345738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f80071390 0x7f2f800701f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2f80073aa0 tx=0x7f2f74009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9599982e57482df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:51.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7e7fc700 1 -- 172.21.15.146:0/3996345738 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74011830 con 0x7f2f80071390 2024-09-13T18:34:51.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7e7fc700 1 -- 172.21.15.146:0/3996345738 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f74011990 con 0x7f2f80071390 2024-09-13T18:34:51.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7e7fc700 1 -- 172.21.15.146:0/3996345738 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74011d40 con 0x7f2f80071390 2024-09-13T18:34:51.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 -- 172.21.15.146:0/3996345738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f80071390 msgr2=0x7f2f800701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:51.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 --2- 172.21.15.146:0/3996345738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f80071390 0x7f2f800701f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2f80073aa0 tx=0x7f2f74009c20 comp rx=0 tx=0).stop 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 -- 172.21.15.146:0/3996345738 shutdown_connections 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 --2- 172.21.15.146:0/3996345738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f80071390 0x7f2f800701f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 --2- 172.21.15.146:0/3996345738 >> v2:172.21.15.154:3300/0 conn(0x7f2f800ca210 0x7f2f800cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 --2- 172.21.15.146:0/3996345738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f800cbd50 0x7f2f800b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 -- 172.21.15.146:0/3996345738 >> 172.21.15.146:0/3996345738 conn(0x7f2f80066270 msgr2=0x7f2f80066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 -- 172.21.15.146:0/3996345738 shutdown_connections 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 -- 172.21.15.146:0/3996345738 wait complete. 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 Processor -- start 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 -- start start 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f80071390 0x7f2f801f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f800ca210 0x7f2f801f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f800cbd50 0x7f2f800cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f801efd60 con 0x7f2f800ca210 2024-09-13T18:34:51.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f801f3d10 con 0x7f2f800cbd50 2024-09-13T18:34:51.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800c8740 con 0x7f2f80071390 2024-09-13T18:34:51.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f800ca210 0x7f2f801f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:51.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f800ca210 0x7f2f801f01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:47232/0 (socket says 172.21.15.146:47232) 2024-09-13T18:34:51.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7effd700 1 -- 172.21.15.146:0/1353254026 learned_addr learned my addr 172.21.15.146:0/1353254026 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:51.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7f7fe700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f80071390 0x7f2f801f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:51.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7ffff700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f800cbd50 0x7f2f800cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:51.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7effd700 1 -- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f80071390 msgr2=0x7f2f801f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:51.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7effd700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f80071390 0x7f2f801f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:51.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7effd700 1 -- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f800cbd50 msgr2=0x7f2f800cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:51.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7effd700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f800cbd50 0x7f2f800cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:51.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7effd700 1 -- 172.21.15.146:0/1353254026 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f74009570 con 0x7f2f800ca210 2024-09-13T18:34:51.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7f7fe700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f80071390 0x7f2f801f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:51.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f7effd700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f800ca210 0x7f2f801f01f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000e910 tx=0x7f2f7000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:51.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f67fff700 1 -- 172.21.15.146:0/1353254026 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f700096a0 con 0x7f2f800ca210 2024-09-13T18:34:51.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f67fff700 1 -- 172.21.15.146:0/1353254026 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7000c850 con 0x7f2f800ca210 2024-09-13T18:34:51.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f849c4700 1 -- 172.21.15.146:0/1353254026 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f800c9230 con 0x7f2f800ca210 2024-09-13T18:34:51.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.621+0000 7f2f67fff700 1 -- 172.21.15.146:0/1353254026 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7000cc00 con 0x7f2f800ca210 2024-09-13T18:34:51.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.625+0000 7f2f849c4700 1 -- 172.21.15.146:0/1353254026 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f800c8c70 con 0x7f2f800ca210 2024-09-13T18:34:51.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.625+0000 7f2f849c4700 1 -- 172.21.15.146:0/1353254026 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8014dbd0 con 0x7f2f800ca210 2024-09-13T18:34:51.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.625+0000 7f2f67fff700 1 -- 172.21.15.146:0/1353254026 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2f70013470 con 0x7f2f800ca210 2024-09-13T18:34:51.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.625+0000 7f2f67fff700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2f6c066730 0x7f2f6c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:51.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.625+0000 7f2f67fff700 1 -- 172.21.15.146:0/1353254026 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f2f700843c0 con 0x7f2f800ca210 2024-09-13T18:34:51.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.625+0000 7f2f7f7fe700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2f6c066730 0x7f2f6c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:51.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.625+0000 7f2f7f7fe700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2f6c066730 0x7f2f6c068bf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009fd0 tx=0x7f2f7800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:51.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.633+0000 7f2f67fff700 1 -- 172.21.15.146:0/1353254026 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2f70051250 con 0x7f2f800ca210 2024-09-13T18:34:51.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f849c4700 1 -- 172.21.15.146:0/1353254026 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f8015d500 con 0x7f2f800ca210 2024-09-13T18:34:51.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f67fff700 1 -- 172.21.15.146:0/1353254026 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+5536 (secure 0 0 0) 0x7f2f70054870 con 0x7f2f800ca210 2024-09-13T18:34:51.920 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:51.920 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:45.771339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:51.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f849c4700 1 -- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2f6c066730 msgr2=0x7f2f6c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:51.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f849c4700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2f6c066730 0x7f2f6c068bf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009fd0 tx=0x7f2f7800b040 comp rx=0 tx=0).stop 2024-09-13T18:34:51.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f849c4700 1 -- 172.21.15.146:0/1353254026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f800ca210 msgr2=0x7f2f801f01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:51.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f849c4700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f800ca210 0x7f2f801f01f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000e910 tx=0x7f2f7000ecd0 comp rx=0 tx=0).stop 2024-09-13T18:34:51.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f849c4700 1 -- 172.21.15.146:0/1353254026 shutdown_connections 2024-09-13T18:34:51.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f849c4700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2f6c066730 0x7f2f6c068bf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:51.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f849c4700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f800cbd50 0x7f2f800cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:51.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f849c4700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f800ca210 0x7f2f801f01f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:51.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f849c4700 1 --2- 172.21.15.146:0/1353254026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f80071390 0x7f2f801f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:51.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f849c4700 1 -- 172.21.15.146:0/1353254026 >> 172.21.15.146:0/1353254026 conn(0x7f2f80066270 msgr2=0x7f2f800cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:51.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f849c4700 1 -- 172.21.15.146:0/1353254026 shutdown_connections 2024-09-13T18:34:51.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:51.905+0000 7f2f849c4700 1 -- 172.21.15.146:0/1353254026 wait complete. 2024-09-13T18:34:51.923 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 44 2024-09-13T18:34:51.935 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2024-09-13T18:34:53.937 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:54.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 Processor -- start 2024-09-13T18:34:54.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 -- start start 2024-09-13T18:34:54.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57140cbd50 0x7f57140b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f57140ca210 0x7f57140cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5714071390 0x7f57140701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57140caa30 con 0x7f5714071390 2024-09-13T18:34:54.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57140cb810 con 0x7f57140ca210 2024-09-13T18:34:54.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57140cc670 con 0x7f57140cbd50 2024-09-13T18:34:54.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f57199ed700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5714071390 0x7f57140701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f57199ed700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5714071390 0x7f57140701f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:47246/0 (socket says 172.21.15.146:47246) 2024-09-13T18:34:54.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f57199ed700 1 -- 172.21.15.146:0/1362951276 learned_addr learned my addr 172.21.15.146:0/1362951276 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:54.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f57189eb700 1 --2- 172.21.15.146:0/1362951276 >> v2:172.21.15.154:3300/0 conn(0x7f57140ca210 0x7f57140cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f57191ec700 1 --2- 172.21.15.146:0/1362951276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57140cbd50 0x7f57140b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f57199ed700 1 -- 172.21.15.146:0/1362951276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57140cbd50 msgr2=0x7f57140b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:54.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f57199ed700 1 --2- 172.21.15.146:0/1362951276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57140cbd50 0x7f57140b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f57199ed700 1 -- 172.21.15.146:0/1362951276 >> v2:172.21.15.154:3300/0 conn(0x7f57140ca210 msgr2=0x7f57140cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:54.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f57199ed700 1 --2- 172.21.15.146:0/1362951276 >> v2:172.21.15.154:3300/0 conn(0x7f57140ca210 0x7f57140cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f57199ed700 1 -- 172.21.15.146:0/1362951276 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571415da20 con 0x7f5714071390 2024-09-13T18:34:54.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f57191ec700 1 --2- 172.21.15.146:0/1362951276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57140cbd50 0x7f57140b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:54.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f57199ed700 1 --2- 172.21.15.146:0/1362951276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5714071390 0x7f57140701f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5714073aa0 tx=0x7f570c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108bf374e6099172 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f570bfff700 1 -- 172.21.15.146:0/1362951276 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c00e070 con 0x7f5714071390 2024-09-13T18:34:54.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f570bfff700 1 -- 172.21.15.146:0/1362951276 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c00b8d0 con 0x7f5714071390 2024-09-13T18:34:54.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f570bfff700 1 -- 172.21.15.146:0/1362951276 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c00bc80 con 0x7f5714071390 2024-09-13T18:34:54.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 -- 172.21.15.146:0/1362951276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5714071390 msgr2=0x7f57140701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 --2- 172.21.15.146:0/1362951276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5714071390 0x7f57140701f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5714073aa0 tx=0x7f570c00b330 comp rx=0 tx=0).stop 2024-09-13T18:34:54.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 -- 172.21.15.146:0/1362951276 shutdown_connections 2024-09-13T18:34:54.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 --2- 172.21.15.146:0/1362951276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5714071390 0x7f57140701f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 --2- 172.21.15.146:0/1362951276 >> v2:172.21.15.154:3300/0 conn(0x7f57140ca210 0x7f57140cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 --2- 172.21.15.146:0/1362951276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57140cbd50 0x7f57140b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 -- 172.21.15.146:0/1362951276 >> 172.21.15.146:0/1362951276 conn(0x7f5714066270 msgr2=0x7f5714066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 -- 172.21.15.146:0/1362951276 shutdown_connections 2024-09-13T18:34:54.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.053+0000 7f571a1ee700 1 -- 172.21.15.146:0/1362951276 wait complete. 2024-09-13T18:34:54.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f571a1ee700 1 Processor -- start 2024-09-13T18:34:54.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f571a1ee700 1 -- start start 2024-09-13T18:34:54.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f571a1ee700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5714071390 0x7f57141f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f571a1ee700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57140ca210 0x7f57141e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f571a1ee700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57140cbd50 0x7f5714146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f571a1ee700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571415c7a0 con 0x7f5714071390 2024-09-13T18:34:54.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f571a1ee700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57140ce410 con 0x7f57140cbd50 2024-09-13T18:34:54.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f571a1ee700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57140cd1f0 con 0x7f57140ca210 2024-09-13T18:34:54.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f57191ec700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5714071390 0x7f57141f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f57191ec700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5714071390 0x7f57141f2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:47258/0 (socket says 172.21.15.146:47258) 2024-09-13T18:34:54.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f57191ec700 1 -- 172.21.15.146:0/1560046192 learned_addr learned my addr 172.21.15.146:0/1560046192 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:34:54.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f57189eb700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57140ca210 0x7f57141e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f57199ed700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57140cbd50 0x7f5714146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f57191ec700 1 -- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57140ca210 msgr2=0x7f57141e2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f57191ec700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57140ca210 0x7f57141e2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f57191ec700 1 -- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57140cbd50 msgr2=0x7f5714146cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f57191ec700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57140cbd50 0x7f5714146cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f57191ec700 1 -- 172.21.15.146:0/1560046192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c009b30 con 0x7f5714071390 2024-09-13T18:34:54.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.057+0000 7f57191ec700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5714071390 0x7f57141f2310 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f570400e810 tx=0x7f570400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.061+0000 7f5709ffb700 1 -- 172.21.15.146:0/1560046192 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704004230 con 0x7f5714071390 2024-09-13T18:34:54.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.061+0000 7f5709ffb700 1 -- 172.21.15.146:0/1560046192 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5704004390 con 0x7f5714071390 2024-09-13T18:34:54.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.061+0000 7f571a1ee700 1 -- 172.21.15.146:0/1560046192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571415bad0 con 0x7f5714071390 2024-09-13T18:34:54.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.061+0000 7f5709ffb700 1 -- 172.21.15.146:0/1560046192 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57040036c0 con 0x7f5714071390 2024-09-13T18:34:54.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.061+0000 7f571a1ee700 1 -- 172.21.15.146:0/1560046192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571415c430 con 0x7f5714071390 2024-09-13T18:34:54.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.061+0000 7f571a1ee700 1 -- 172.21.15.146:0/1560046192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571415d500 con 0x7f5714071390 2024-09-13T18:34:54.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.061+0000 7f5709ffb700 1 -- 172.21.15.146:0/1560046192 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5704003840 con 0x7f5714071390 2024-09-13T18:34:54.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.061+0000 7f5709ffb700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56f8066730 0x7f56f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.061+0000 7f5709ffb700 1 -- 172.21.15.146:0/1560046192 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f57040840d0 con 0x7f5714071390 2024-09-13T18:34:54.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.061+0000 7f57189eb700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56f8066730 0x7f56f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.065+0000 7f57189eb700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56f8066730 0x7f56f8068bf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f571414e230 tx=0x7f5710009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.069+0000 7f5709ffb700 1 -- 172.21.15.146:0/1560046192 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5704050da0 con 0x7f5714071390 2024-09-13T18:34:54.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.321+0000 7f571a1ee700 1 -- 172.21.15.146:0/1560046192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57140c9ff0 con 0x7f5714071390 2024-09-13T18:34:54.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.321+0000 7f5709ffb700 1 -- 172.21.15.146:0/1560046192 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f5704020090 con 0x7f5714071390 2024-09-13T18:34:54.334 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:34:54.334 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5784,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:45.771339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:54.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.325+0000 7f571a1ee700 1 -- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56f8066730 msgr2=0x7f56f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.325+0000 7f571a1ee700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56f8066730 0x7f56f8068bf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f571414e230 tx=0x7f5710009240 comp rx=0 tx=0).stop 2024-09-13T18:34:54.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.325+0000 7f571a1ee700 1 -- 172.21.15.146:0/1560046192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5714071390 msgr2=0x7f57141f2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.325+0000 7f571a1ee700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5714071390 0x7f57141f2310 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f570400e810 tx=0x7f570400eb20 comp rx=0 tx=0).stop 2024-09-13T18:34:54.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.325+0000 7f571a1ee700 1 -- 172.21.15.146:0/1560046192 shutdown_connections 2024-09-13T18:34:54.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.325+0000 7f571a1ee700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56f8066730 0x7f56f8068bf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.325+0000 7f571a1ee700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57140cbd50 0x7f5714146cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.325+0000 7f571a1ee700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57140ca210 0x7f57141e2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.325+0000 7f571a1ee700 1 --2- 172.21.15.146:0/1560046192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5714071390 0x7f57141f2310 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.325+0000 7f571a1ee700 1 -- 172.21.15.146:0/1560046192 >> 172.21.15.146:0/1560046192 conn(0x7f5714066270 msgr2=0x7f5714149e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.325+0000 7f571a1ee700 1 -- 172.21.15.146:0/1560046192 shutdown_connections 2024-09-13T18:34:54.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:34:54.325+0000 7f571a1ee700 1 -- 172.21.15.146:0/1560046192 wait complete. 2024-09-13T18:34:54.338 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 45 2024-09-13T18:34:54.349 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2024-09-13T18:34:54.349 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 60 secs before thrashing 2024-09-13T18:35:08.445 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:35:08.448 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:35:38.526 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:35:38.529 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:35:54.350 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:35:54.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.460+0000 7f69f66ca700 1 Processor -- start 2024-09-13T18:35:54.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 -- start start 2024-09-13T18:35:54.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f69e80d8c60 0x7f69e804eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f69e80073f0 0x7f69e8005de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69e804f920 0x7f69e800e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e8049640 con 0x7f69e80073f0 2024-09-13T18:35:54.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e80505d0 con 0x7f69e80d8c60 2024-09-13T18:35:54.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e80508b0 con 0x7f69e804f920 2024-09-13T18:35:54.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f4ec7700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f69e80073f0 0x7f69e8005de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f4ec7700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f69e80073f0 0x7f69e8005de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57328/0 (socket says 172.21.15.146:57328) 2024-09-13T18:35:54.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f4ec7700 1 -- 172.21.15.146:0/3603581641 learned_addr learned my addr 172.21.15.146:0/3603581641 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:35:54.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f56c8700 1 --2- 172.21.15.146:0/3603581641 >> v2:172.21.15.154:3300/0 conn(0x7f69e80d8c60 0x7f69e804eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f5ec9700 1 --2- 172.21.15.146:0/3603581641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69e804f920 0x7f69e800e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f4ec7700 1 -- 172.21.15.146:0/3603581641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69e804f920 msgr2=0x7f69e800e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:54.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f4ec7700 1 --2- 172.21.15.146:0/3603581641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69e804f920 0x7f69e800e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f4ec7700 1 -- 172.21.15.146:0/3603581641 >> v2:172.21.15.154:3300/0 conn(0x7f69e80d8c60 msgr2=0x7f69e804eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:54.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f4ec7700 1 --2- 172.21.15.146:0/3603581641 >> v2:172.21.15.154:3300/0 conn(0x7f69e80d8c60 0x7f69e804eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f4ec7700 1 -- 172.21.15.146:0/3603581641 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e804f120 con 0x7f69e80073f0 2024-09-13T18:35:54.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f4ec7700 1 --2- 172.21.15.146:0/3603581641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e80073f0 0x7f69e8005de0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f69e800e450 tx=0x7f69e0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a24c7a4c0cc35b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:54.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69e7fff700 1 -- 172.21.15.146:0/3603581641 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e000b8a0 con 0x7f69e80073f0 2024-09-13T18:35:54.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69e7fff700 1 -- 172.21.15.146:0/3603581641 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e000ba00 con 0x7f69e80073f0 2024-09-13T18:35:54.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69e7fff700 1 -- 172.21.15.146:0/3603581641 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e000bd90 con 0x7f69e80073f0 2024-09-13T18:35:54.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 -- 172.21.15.146:0/3603581641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e80073f0 msgr2=0x7f69e8005de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:54.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 --2- 172.21.15.146:0/3603581641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e80073f0 0x7f69e8005de0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f69e800e450 tx=0x7f69e0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 -- 172.21.15.146:0/3603581641 shutdown_connections 2024-09-13T18:35:54.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 --2- 172.21.15.146:0/3603581641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69e804f920 0x7f69e800e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 --2- 172.21.15.146:0/3603581641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e80073f0 0x7f69e8005de0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 --2- 172.21.15.146:0/3603581641 >> v2:172.21.15.154:3300/0 conn(0x7f69e80d8c60 0x7f69e804eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 -- 172.21.15.146:0/3603581641 >> 172.21.15.146:0/3603581641 conn(0x7f69e8054520 msgr2=0x7f69e8054930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:54.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 -- 172.21.15.146:0/3603581641 shutdown_connections 2024-09-13T18:35:54.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.464+0000 7f69f66ca700 1 -- 172.21.15.146:0/3603581641 wait complete. 2024-09-13T18:35:54.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f66ca700 1 Processor -- start 2024-09-13T18:35:54.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f66ca700 1 -- start start 2024-09-13T18:35:54.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f66ca700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e80073f0 0x7f69e81744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f66ca700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69e804f920 0x7f69e80cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f66ca700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69e80d8c60 0x7f69e80ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f66ca700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e804e220 con 0x7f69e80073f0 2024-09-13T18:35:54.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f66ca700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e8166520 con 0x7f69e804f920 2024-09-13T18:35:54.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f66ca700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e80cfec0 con 0x7f69e80d8c60 2024-09-13T18:35:54.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f56c8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e80073f0 0x7f69e81744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f56c8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e80073f0 0x7f69e81744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57342/0 (socket says 172.21.15.146:57342) 2024-09-13T18:35:54.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f56c8700 1 -- 172.21.15.146:0/3419337314 learned_addr learned my addr 172.21.15.146:0/3419337314 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:35:54.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f4ec7700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69e804f920 0x7f69e80cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f5ec9700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69e80d8c60 0x7f69e80ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f56c8700 1 -- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69e80d8c60 msgr2=0x7f69e80ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:54.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f56c8700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69e80d8c60 0x7f69e80ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f56c8700 1 -- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69e804f920 msgr2=0x7f69e80cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:54.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f56c8700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69e804f920 0x7f69e80cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f56c8700 1 -- 172.21.15.146:0/3419337314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e0009570 con 0x7f69e80073f0 2024-09-13T18:35:54.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f56c8700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e80073f0 0x7f69e81744f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00e760 tx=0x7f69ec00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:54.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69e5ffb700 1 -- 172.21.15.146:0/3419337314 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec013610 con 0x7f69e80073f0 2024-09-13T18:35:54.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69e5ffb700 1 -- 172.21.15.146:0/3419337314 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec013770 con 0x7f69e80073f0 2024-09-13T18:35:54.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f66ca700 1 -- 172.21.15.146:0/3419337314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e80d0150 con 0x7f69e80073f0 2024-09-13T18:35:54.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69e5ffb700 1 -- 172.21.15.146:0/3419337314 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec013ae0 con 0x7f69e80073f0 2024-09-13T18:35:54.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f66ca700 1 -- 172.21.15.146:0/3419337314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e80d0710 con 0x7f69e80073f0 2024-09-13T18:35:54.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.468+0000 7f69f66ca700 1 -- 172.21.15.146:0/3419337314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e80d0900 con 0x7f69e80073f0 2024-09-13T18:35:54.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.472+0000 7f69e5ffb700 1 -- 172.21.15.146:0/3419337314 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69ec013c40 con 0x7f69e80073f0 2024-09-13T18:35:54.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.472+0000 7f69e5ffb700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69d4066730 0x7f69d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.472+0000 7f69e5ffb700 1 -- 172.21.15.146:0/3419337314 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f69ec084760 con 0x7f69e80073f0 2024-09-13T18:35:54.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.472+0000 7f69f4ec7700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69d4066730 0x7f69d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.472+0000 7f69f4ec7700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69d4066730 0x7f69d4068bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f69e0000c00 tx=0x7f69e0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:54.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.476+0000 7f69e5ffb700 1 -- 172.21.15.146:0/3419337314 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f69ec051560 con 0x7f69e80073f0 2024-09-13T18:35:54.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.696+0000 7f69f66ca700 1 -- 172.21.15.146:0/3419337314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69e816a0d0 con 0x7f69e80073f0 2024-09-13T18:35:54.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.696+0000 7f69e5ffb700 1 -- 172.21.15.146:0/3419337314 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f69ec054b80 con 0x7f69e80073f0 2024-09-13T18:35:54.711 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:35:54.711 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5784,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:45.771339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:35:54.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.700+0000 7f69f66ca700 1 -- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69d4066730 msgr2=0x7f69d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:54.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.700+0000 7f69f66ca700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69d4066730 0x7f69d4068bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f69e0000c00 tx=0x7f69e0008030 comp rx=0 tx=0).stop 2024-09-13T18:35:54.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.700+0000 7f69f66ca700 1 -- 172.21.15.146:0/3419337314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e80073f0 msgr2=0x7f69e81744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:54.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.700+0000 7f69f66ca700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e80073f0 0x7f69e81744f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00e760 tx=0x7f69ec00eb20 comp rx=0 tx=0).stop 2024-09-13T18:35:54.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.700+0000 7f69f66ca700 1 -- 172.21.15.146:0/3419337314 shutdown_connections 2024-09-13T18:35:54.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.700+0000 7f69f66ca700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69d4066730 0x7f69d4068bf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.700+0000 7f69f66ca700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69e80d8c60 0x7f69e80ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.700+0000 7f69f66ca700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69e804f920 0x7f69e80cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.700+0000 7f69f66ca700 1 --2- 172.21.15.146:0/3419337314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e80073f0 0x7f69e81744f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.700+0000 7f69f66ca700 1 -- 172.21.15.146:0/3419337314 >> 172.21.15.146:0/3419337314 conn(0x7f69e8054520 msgr2=0x7f69e8052030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:54.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.700+0000 7f69f66ca700 1 -- 172.21.15.146:0/3419337314 shutdown_connections 2024-09-13T18:35:54.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.700+0000 7f69f66ca700 1 -- 172.21.15.146:0/3419337314 wait complete. 2024-09-13T18:35:54.714 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 45 2024-09-13T18:35:54.728 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=1) 2024-09-13T18:35:54.728 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-09-13T18:35:54.728 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:35:54.745 INFO:tasks.ceph.mds.f:Stopped 2024-09-13T18:35:54.746 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:35:54.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.856+0000 7fa261943700 1 Processor -- start 2024-09-13T18:35:54.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.856+0000 7fa261943700 1 -- start start 2024-09-13T18:35:54.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.856+0000 7fa261943700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c06eba0 0x7fa25c0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.856+0000 7fa261943700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25c0c1020 con 0x7fa25c15a000 2024-09-13T18:35:54.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.856+0000 7fa261943700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25c1593f0 con 0x7fa25c06e7c0 2024-09-13T18:35:54.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.856+0000 7fa261943700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0c6a70 con 0x7fa25c06eba0 2024-09-13T18:35:54.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.856+0000 7fa261142700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa25c15a000 0x7fa25c1531a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37426/0 (socket says 172.21.15.146:37426) 2024-09-13T18:35:54.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.856+0000 7fa261142700 1 -- 172.21.15.146:0/1530009323 learned_addr learned my addr 172.21.15.146:0/1530009323 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:35:54.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.856+0000 7fa25bfff700 1 --2- 172.21.15.146:0/1530009323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c06eba0 0x7fa25c0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.856+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462280620 0 0) 0x7fa25c0c1020 con 0x7fa25c15a000 2024-09-13T18:35:54.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.856+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa24c0035d0 con 0x7fa25c15a000 2024-09-13T18:35:54.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.856+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739860925 0 0) 0x7fa25c1593f0 con 0x7fa25c06e7c0 2024-09-13T18:35:54.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa25c0c1020 con 0x7fa25c06e7c0 2024-09-13T18:35:54.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3786334662 0 0) 0x7fa24c0035d0 con 0x7fa25c15a000 2024-09-13T18:35:54.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c06eba0 msgr2=0x7fa25c0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:54.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 --2- 172.21.15.146:0/1530009323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c06eba0 0x7fa25c0b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 >> v1:172.21.15.154:6789/0 conn(0x7fa25c06e7c0 legacy=0x7fa25c1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:54.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c0c1290 con 0x7fa25c15a000 2024-09-13T18:35:54.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7fa2540019f0 con 0x7fa25c15a000 2024-09-13T18:35:54.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 >> v1:172.21.15.146:6789/0 conn(0x7fa25c15a000 legacy=0x7fa25c1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:54.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 --2- 172.21.15.146:0/1530009323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c06eba0 0x7fa24c003c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 --2- 172.21.15.146:0/1530009323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa24c0041b0 0x7fa24c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 --2- 172.21.15.146:0/1530009323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa24c004db0 0x7fa24c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0c6a70 con 0x7fa24c004db0 2024-09-13T18:35:54.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c00bb70 con 0x7fa24c0041b0 2024-09-13T18:35:54.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c00bd30 con 0x7fa25c06eba0 2024-09-13T18:35:54.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261142700 1 --2- 172.21.15.146:0/1530009323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa24c004db0 0x7fa24c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa260941700 1 --2- 172.21.15.146:0/1530009323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c06eba0 0x7fa24c003c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25bfff700 1 --2- 172.21.15.146:0/1530009323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa24c0041b0 0x7fa24c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261142700 1 -- 172.21.15.146:0/1530009323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c06eba0 msgr2=0x7fa24c003c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:54.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261142700 1 --2- 172.21.15.146:0/1530009323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c06eba0 0x7fa24c003c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261142700 1 -- 172.21.15.146:0/1530009323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa24c0041b0 msgr2=0x7fa24c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:54.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261142700 1 --2- 172.21.15.146:0/1530009323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa24c0041b0 0x7fa24c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261142700 1 -- 172.21.15.146:0/1530009323 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c0035d0 con 0x7fa24c004db0 2024-09-13T18:35:54.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa260941700 1 --2- 172.21.15.146:0/1530009323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa25c06eba0 0x7fa24c003c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:35:54.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261142700 1 --2- 172.21.15.146:0/1530009323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa24c004db0 0x7fa24c005230 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa25400a7d0 tx=0x7fa24c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57eef69ab37de20 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:54.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2540072d0 con 0x7fa24c004db0 2024-09-13T18:35:54.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25400ae90 con 0x7fa24c004db0 2024-09-13T18:35:54.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa25b7fe700 1 -- 172.21.15.146:0/1530009323 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254007670 con 0x7fa24c004db0 2024-09-13T18:35:54.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261943700 1 -- 172.21.15.146:0/1530009323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa24c004db0 msgr2=0x7fa24c005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:54.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261943700 1 --2- 172.21.15.146:0/1530009323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa24c004db0 0x7fa24c005230 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa25400a7d0 tx=0x7fa24c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261142700 1 -- 172.21.15.146:0/1530009323 reap_dead start 2024-09-13T18:35:54.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261943700 1 -- 172.21.15.146:0/1530009323 shutdown_connections 2024-09-13T18:35:54.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261943700 1 -- 172.21.15.146:0/1530009323 >> 172.21.15.146:0/1530009323 conn(0x7fa25c071ed0 msgr2=0x7fa25c0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:54.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261943700 1 -- 172.21.15.146:0/1530009323 shutdown_connections 2024-09-13T18:35:54.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261943700 1 -- 172.21.15.146:0/1530009323 wait complete. 2024-09-13T18:35:54.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.860+0000 7fa261943700 1 Processor -- start 2024-09-13T18:35:54.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa261943700 1 -- start start 2024-09-13T18:35:54.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa261943700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa24c004db0 0x7fa25c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa261943700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c06f5c0 0x7fa25c1e2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa261943700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa25c1f2ba0 0x7fa25c14c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa261943700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c1f2470 con 0x7fa25c06f5c0 2024-09-13T18:35:54.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa261943700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c1f2710 con 0x7fa25c1f2ba0 2024-09-13T18:35:54.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa261943700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c070f90 con 0x7fa24c004db0 2024-09-13T18:35:54.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa25bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c06f5c0 0x7fa25c1e2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa25bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c06f5c0 0x7fa25c1e2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57356/0 (socket says 172.21.15.146:57356) 2024-09-13T18:35:54.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa25bfff700 1 -- 172.21.15.146:0/4055190348 learned_addr learned my addr 172.21.15.146:0/4055190348 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:35:54.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa260941700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa24c004db0 0x7fa25c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa261142700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa25c1f2ba0 0x7fa25c14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa25bfff700 1 -- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa24c004db0 msgr2=0x7fa25c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:54.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa25bfff700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa24c004db0 0x7fa25c0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa25bfff700 1 -- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa25c1f2ba0 msgr2=0x7fa25c14c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:54.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa25bfff700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa25c1f2ba0 0x7fa25c14c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:54.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa25bfff700 1 -- 172.21.15.146:0/4055190348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c071250 con 0x7fa25c06f5c0 2024-09-13T18:35:54.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa25bfff700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c06f5c0 0x7fa25c1e2c90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa2480115d0 tx=0x7fa248011990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:54.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa2597fa700 1 -- 172.21.15.146:0/4055190348 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248004500 con 0x7fa25c06f5c0 2024-09-13T18:35:54.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa2597fa700 1 -- 172.21.15.146:0/4055190348 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2480036c0 con 0x7fa25c06f5c0 2024-09-13T18:35:54.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa261943700 1 -- 172.21.15.146:0/4055190348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25c071530 con 0x7fa25c06f5c0 2024-09-13T18:35:54.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa2597fa700 1 -- 172.21.15.146:0/4055190348 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248003a70 con 0x7fa25c06f5c0 2024-09-13T18:35:54.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.864+0000 7fa261943700 1 -- 172.21.15.146:0/4055190348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c1eef00 con 0x7fa25c06f5c0 2024-09-13T18:35:54.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.868+0000 7fa261943700 1 -- 172.21.15.146:0/4055190348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25c0b11a0 con 0x7fa25c06f5c0 2024-09-13T18:35:54.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.868+0000 7fa2597fa700 1 -- 172.21.15.146:0/4055190348 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa248003bd0 con 0x7fa25c06f5c0 2024-09-13T18:35:54.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.868+0000 7fa2597fa700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa240066730 0x7fa240068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:54.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.868+0000 7fa2597fa700 1 -- 172.21.15.146:0/4055190348 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fa248084c50 con 0x7fa25c06f5c0 2024-09-13T18:35:54.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.868+0000 7fa260941700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa240066730 0x7fa240068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:54.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.868+0000 7fa260941700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa240066730 0x7fa240068bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa250007b90 tx=0x7fa250009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:54.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:54.872+0000 7fa2597fa700 1 -- 172.21.15.146:0/4055190348 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa24808a020 con 0x7fa25c06f5c0 2024-09-13T18:35:55.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.076+0000 7fa261943700 1 -- 172.21.15.146:0/4055190348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa25c1f2850 con 0x7fa25c06f5c0 2024-09-13T18:35:55.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.076+0000 7fa2597fa700 1 -- 172.21.15.146:0/4055190348 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6296 (secure 0 0 0) 0x7fa248051ae0 con 0x7fa25c06f5c0 2024-09-13T18:35:55.090 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:35:55.090 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5784,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:45.771339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:35:55.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.080+0000 7fa261943700 1 -- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa240066730 msgr2=0x7fa240068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:55.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.080+0000 7fa261943700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa240066730 0x7fa240068bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa250007b90 tx=0x7fa250009040 comp rx=0 tx=0).stop 2024-09-13T18:35:55.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.080+0000 7fa261943700 1 -- 172.21.15.146:0/4055190348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c06f5c0 msgr2=0x7fa25c1e2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:55.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.080+0000 7fa261943700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c06f5c0 0x7fa25c1e2c90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa2480115d0 tx=0x7fa248011990 comp rx=0 tx=0).stop 2024-09-13T18:35:55.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.080+0000 7fa261943700 1 -- 172.21.15.146:0/4055190348 shutdown_connections 2024-09-13T18:35:55.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.080+0000 7fa261943700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa240066730 0x7fa240068bf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:55.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.080+0000 7fa261943700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa25c1f2ba0 0x7fa25c14c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:55.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.080+0000 7fa261943700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c06f5c0 0x7fa25c1e2c90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:55.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.080+0000 7fa261943700 1 --2- 172.21.15.146:0/4055190348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa24c004db0 0x7fa25c0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:55.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.080+0000 7fa261943700 1 -- 172.21.15.146:0/4055190348 >> 172.21.15.146:0/4055190348 conn(0x7fa25c071ed0 msgr2=0x7fa25c150c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:55.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.080+0000 7fa261943700 1 -- 172.21.15.146:0/4055190348 shutdown_connections 2024-09-13T18:35:55.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:55.080+0000 7fa261943700 1 -- 172.21.15.146:0/4055190348 wait complete. 2024-09-13T18:35:55.094 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 45 2024-09-13T18:35:55.105 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:35:57.106 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:35:57.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa594ebb700 1 Processor -- start 2024-09-13T18:35:57.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa594ebb700 1 -- start start 2024-09-13T18:35:57.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa594ebb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa5900cbd50 0x7fa5900b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:57.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa594ebb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa590071390 0x7fa5900cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:57.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa594ebb700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5900cc670 con 0x7fa5900ca210 2024-09-13T18:35:57.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa594ebb700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5900caa30 con 0x7fa590071390 2024-09-13T18:35:57.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa594ebb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5900cb810 con 0x7fa5900cbd50 2024-09-13T18:35:57.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa58ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa590071390 0x7fa5900cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:57.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa58ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa590071390 0x7fa5900cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57358/0 (socket says 172.21.15.146:57358) 2024-09-13T18:35:57.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa58effd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa5900ca210 0x7fa5900701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37440/0 (socket says 172.21.15.146:37440) 2024-09-13T18:35:57.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa58ffff700 1 -- 172.21.15.146:0/2002506129 learned_addr learned my addr 172.21.15.146:0/2002506129 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:35:57.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa58f7fe700 1 --2- 172.21.15.146:0/2002506129 >> v2:172.21.15.154:3300/0 conn(0x7fa5900cbd50 0x7fa5900b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:57.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa58ffff700 1 -- 172.21.15.146:0/2002506129 >> v2:172.21.15.154:3300/0 conn(0x7fa5900cbd50 msgr2=0x7fa5900b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:57.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa58ffff700 1 --2- 172.21.15.146:0/2002506129 >> v2:172.21.15.154:3300/0 conn(0x7fa5900cbd50 0x7fa5900b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:57.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa58ffff700 1 -- 172.21.15.146:0/2002506129 >> v1:172.21.15.146:6789/0 conn(0x7fa5900ca210 legacy=0x7fa5900701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:57.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa58ffff700 1 -- 172.21.15.146:0/2002506129 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59015da20 con 0x7fa590071390 2024-09-13T18:35:57.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa58ffff700 1 --2- 172.21.15.146:0/2002506129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa590071390 0x7fa5900cc130 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa59008c2d0 tx=0x7fa58800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f46ba473bea41a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:57.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa58e7fc700 1 -- 172.21.15.146:0/2002506129 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58800e070 con 0x7fa590071390 2024-09-13T18:35:57.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa58e7fc700 1 -- 172.21.15.146:0/2002506129 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58800b9c0 con 0x7fa590071390 2024-09-13T18:35:57.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa58e7fc700 1 -- 172.21.15.146:0/2002506129 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58800bd70 con 0x7fa590071390 2024-09-13T18:35:57.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa594ebb700 1 -- 172.21.15.146:0/2002506129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa590071390 msgr2=0x7fa5900cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:57.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa594ebb700 1 --2- 172.21.15.146:0/2002506129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa590071390 0x7fa5900cc130 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa59008c2d0 tx=0x7fa58800b330 comp rx=0 tx=0).stop 2024-09-13T18:35:57.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa594ebb700 1 -- 172.21.15.146:0/2002506129 shutdown_connections 2024-09-13T18:35:57.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa594ebb700 1 --2- 172.21.15.146:0/2002506129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa590071390 0x7fa5900cc130 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:57.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa594ebb700 1 --2- 172.21.15.146:0/2002506129 >> v2:172.21.15.154:3300/0 conn(0x7fa5900cbd50 0x7fa5900b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:57.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.224+0000 7fa594ebb700 1 -- 172.21.15.146:0/2002506129 >> 172.21.15.146:0/2002506129 conn(0x7fa590066270 msgr2=0x7fa590066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:57.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa594ebb700 1 -- 172.21.15.146:0/2002506129 shutdown_connections 2024-09-13T18:35:57.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa594ebb700 1 -- 172.21.15.146:0/2002506129 wait complete. 2024-09-13T18:35:57.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa594ebb700 1 Processor -- start 2024-09-13T18:35:57.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa594ebb700 1 -- start start 2024-09-13T18:35:57.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa594ebb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa590071390 0x7fa5901efaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:57.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa594ebb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5900ca210 0x7fa59015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:57.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa594ebb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900cbd50 0x7fa590148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:57.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa594ebb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5900b4400 con 0x7fa590071390 2024-09-13T18:35:57.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa594ebb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5901e4ce0 con 0x7fa5900ca210 2024-09-13T18:35:57.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa594ebb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5901f0030 con 0x7fa5900cbd50 2024-09-13T18:35:57.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa58f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa590071390 0x7fa5901efaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:57.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa58f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa590071390 0x7fa5901efaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57368/0 (socket says 172.21.15.146:57368) 2024-09-13T18:35:57.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa58f7fe700 1 -- 172.21.15.146:0/2198365165 learned_addr learned my addr 172.21.15.146:0/2198365165 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:35:57.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa58ffff700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900cbd50 0x7fa590148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:57.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa58effd700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5900ca210 0x7fa59015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:57.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa58f7fe700 1 -- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900cbd50 msgr2=0x7fa590148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:57.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa58f7fe700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900cbd50 0x7fa590148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:57.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa58f7fe700 1 -- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5900ca210 msgr2=0x7fa59015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:57.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa58f7fe700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5900ca210 0x7fa59015b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:57.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa58f7fe700 1 -- 172.21.15.146:0/2198365165 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa588009b30 con 0x7fa590071390 2024-09-13T18:35:57.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.228+0000 7fa58f7fe700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa590071390 0x7fa5901efaf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa58000d800 tx=0x7fa58000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:57.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.232+0000 7fa577fff700 1 -- 172.21.15.146:0/2198365165 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5800125f0 con 0x7fa590071390 2024-09-13T18:35:57.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.232+0000 7fa577fff700 1 -- 172.21.15.146:0/2198365165 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa580012750 con 0x7fa590071390 2024-09-13T18:35:57.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.232+0000 7fa594ebb700 1 -- 172.21.15.146:0/2198365165 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5901e41c0 con 0x7fa590071390 2024-09-13T18:35:57.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.232+0000 7fa577fff700 1 -- 172.21.15.146:0/2198365165 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580012ac0 con 0x7fa590071390 2024-09-13T18:35:57.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.232+0000 7fa594ebb700 1 -- 172.21.15.146:0/2198365165 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5901e4720 con 0x7fa590071390 2024-09-13T18:35:57.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.232+0000 7fa594ebb700 1 -- 172.21.15.146:0/2198365165 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa570005310 con 0x7fa590071390 2024-09-13T18:35:57.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.232+0000 7fa577fff700 1 -- 172.21.15.146:0/2198365165 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa580012c20 con 0x7fa590071390 2024-09-13T18:35:57.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.232+0000 7fa577fff700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa57c066760 0x7fa57c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:57.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.232+0000 7fa577fff700 1 -- 172.21.15.146:0/2198365165 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fa580084a20 con 0x7fa590071390 2024-09-13T18:35:57.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.232+0000 7fa58effd700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa57c066760 0x7fa57c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:57.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.236+0000 7fa58effd700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa57c066760 0x7fa57c068c20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa590149b40 tx=0x7fa584009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:57.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.240+0000 7fa577fff700 1 -- 172.21.15.146:0/2198365165 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa580051790 con 0x7fa590071390 2024-09-13T18:35:57.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.468+0000 7fa594ebb700 1 -- 172.21.15.146:0/2198365165 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5700055c0 con 0x7fa590071390 2024-09-13T18:35:57.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.468+0000 7fa577fff700 1 -- 172.21.15.146:0/2198365165 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6296 (secure 0 0 0) 0x7fa580012ed0 con 0x7fa590071390 2024-09-13T18:35:57.483 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:35:57.483 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5784,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:45.771339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:35:57.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.472+0000 7fa594ebb700 1 -- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa57c066760 msgr2=0x7fa57c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:57.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.472+0000 7fa594ebb700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa57c066760 0x7fa57c068c20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa590149b40 tx=0x7fa584009040 comp rx=0 tx=0).stop 2024-09-13T18:35:57.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.472+0000 7fa594ebb700 1 -- 172.21.15.146:0/2198365165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa590071390 msgr2=0x7fa5901efaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:57.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.472+0000 7fa594ebb700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa590071390 0x7fa5901efaf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa58000d800 tx=0x7fa58000dbc0 comp rx=0 tx=0).stop 2024-09-13T18:35:57.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.472+0000 7fa594ebb700 1 -- 172.21.15.146:0/2198365165 shutdown_connections 2024-09-13T18:35:57.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.472+0000 7fa594ebb700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa57c066760 0x7fa57c068c20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:57.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.472+0000 7fa594ebb700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900cbd50 0x7fa590148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:57.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.472+0000 7fa594ebb700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5900ca210 0x7fa59015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:57.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.472+0000 7fa594ebb700 1 --2- 172.21.15.146:0/2198365165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa590071390 0x7fa5901efaf0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:57.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.472+0000 7fa594ebb700 1 -- 172.21.15.146:0/2198365165 >> 172.21.15.146:0/2198365165 conn(0x7fa590066270 msgr2=0x7fa590149db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:57.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.472+0000 7fa594ebb700 1 -- 172.21.15.146:0/2198365165 shutdown_connections 2024-09-13T18:35:57.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:57.472+0000 7fa594ebb700 1 -- 172.21.15.146:0/2198365165 wait complete. 2024-09-13T18:35:57.487 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 45 2024-09-13T18:35:57.497 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:35:59.498 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:35:59.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.596+0000 7f2dd2c8d700 1 Processor -- start 2024-09-13T18:35:59.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.596+0000 7f2dd2c8d700 1 -- start start 2024-09-13T18:35:59.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd2c8d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2dcc06eba0 0x7f2dcc0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:59.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd2c8d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2dcc15a000 0x7f2dcc0c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:59.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd2c8d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dcc0c1020 con 0x7f2dcc06e7c0 2024-09-13T18:35:59.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd2c8d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dcc1593f0 con 0x7f2dcc15a000 2024-09-13T18:35:59.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd2c8d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0c6a70 con 0x7f2dcc06eba0 2024-09-13T18:35:59.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd1c8b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f2dcc06e7c0 0x7f2dcc1598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37450/0 (socket says 172.21.15.146:37450) 2024-09-13T18:35:59.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd1c8b700 1 -- 172.21.15.146:0/80536121 learned_addr learned my addr 172.21.15.146:0/80536121 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:35:59.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd248c700 1 --2- 172.21.15.146:0/80536121 >> v2:172.21.15.146:3300/0 conn(0x7f2dcc15a000 0x7f2dcc0c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:59.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd148a700 1 --2- 172.21.15.146:0/80536121 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2dcc06eba0 0x7f2dcc0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:59.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd0c89700 1 -- 172.21.15.146:0/80536121 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109864379 0 0) 0x7f2dcc0c1020 con 0x7f2dcc06e7c0 2024-09-13T18:35:59.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd0c89700 1 -- 172.21.15.146:0/80536121 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2db80035d0 con 0x7f2dcc06e7c0 2024-09-13T18:35:59.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd248c700 1 -- 172.21.15.146:0/80536121 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2dcc06eba0 msgr2=0x7f2dcc0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:35:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd248c700 1 --2- 172.21.15.146:0/80536121 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2dcc06eba0 0x7f2dcc0b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd248c700 1 -- 172.21.15.146:0/80536121 >> v1:172.21.15.146:6789/0 conn(0x7f2dcc06e7c0 legacy=0x7f2dcc1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd248c700 1 -- 172.21.15.146:0/80536121 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc0c1290 con 0x7f2dcc15a000 2024-09-13T18:35:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd248c700 1 --2- 172.21.15.146:0/80536121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dcc15a000 0x7f2dcc0c2ce0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009d90 tx=0x7f2dc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82918ac0b814ee9e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd0c89700 1 -- 172.21.15.146:0/80536121 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400e070 con 0x7f2dcc15a000 2024-09-13T18:35:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd0c89700 1 -- 172.21.15.146:0/80536121 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc400bb20 con 0x7f2dcc15a000 2024-09-13T18:35:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd0c89700 1 -- 172.21.15.146:0/80536121 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4012430 con 0x7f2dcc15a000 2024-09-13T18:35:59.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/80536121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dcc15a000 msgr2=0x7f2dcc0c2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:59.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd2c8d700 1 --2- 172.21.15.146:0/80536121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dcc15a000 0x7f2dcc0c2ce0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009d90 tx=0x7f2dc400b330 comp rx=0 tx=0).stop 2024-09-13T18:35:59.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/80536121 shutdown_connections 2024-09-13T18:35:59.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd2c8d700 1 --2- 172.21.15.146:0/80536121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dcc15a000 0x7f2dcc0c2ce0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:59.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd2c8d700 1 --2- 172.21.15.146:0/80536121 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2dcc06eba0 0x7f2dcc0b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:59.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/80536121 >> 172.21.15.146:0/80536121 conn(0x7f2dcc071ed0 msgr2=0x7f2dcc0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:59.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/80536121 shutdown_connections 2024-09-13T18:35:59.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.600+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/80536121 wait complete. 2024-09-13T18:35:59.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd2c8d700 1 Processor -- start 2024-09-13T18:35:59.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd2c8d700 1 -- start start 2024-09-13T18:35:59.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd2c8d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2dcc06e7c0 0x7f2dcc1f0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:59.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd2c8d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dcc06eba0 0x7f2dcc1f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:59.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd2c8d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2dcc15a000 0x7f2dcc14e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:59.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd2c8d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc1df830 con 0x7f2dcc06eba0 2024-09-13T18:35:59.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd2c8d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc1e1290 con 0x7f2dcc06e7c0 2024-09-13T18:35:59.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd2c8d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc06ba90 con 0x7f2dcc15a000 2024-09-13T18:35:59.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd148a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dcc06eba0 0x7f2dcc1f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:59.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd1c8b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2dcc06e7c0 0x7f2dcc1f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:59.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd248c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2dcc15a000 0x7f2dcc14e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:59.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd148a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dcc06eba0 0x7f2dcc1f2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57380/0 (socket says 172.21.15.146:57380) 2024-09-13T18:35:59.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd148a700 1 -- 172.21.15.146:0/793258593 learned_addr learned my addr 172.21.15.146:0/793258593 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:35:59.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd148a700 1 -- 172.21.15.146:0/793258593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2dcc15a000 msgr2=0x7f2dcc14e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:59.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd148a700 1 --2- 172.21.15.146:0/793258593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2dcc15a000 0x7f2dcc14e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:59.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd148a700 1 -- 172.21.15.146:0/793258593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2dcc06e7c0 msgr2=0x7f2dcc1f0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:59.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd148a700 1 --2- 172.21.15.146:0/793258593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2dcc06e7c0 0x7f2dcc1f0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:59.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd148a700 1 -- 172.21.15.146:0/793258593 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc4009a70 con 0x7f2dcc06eba0 2024-09-13T18:35:59.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd148a700 1 --2- 172.21.15.146:0/793258593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dcc06eba0 0x7f2dcc1f2bb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00d800 tx=0x7f2dbc00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:59.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dc27fc700 1 -- 172.21.15.146:0/793258593 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc004240 con 0x7f2dcc06eba0 2024-09-13T18:35:59.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dc27fc700 1 -- 172.21.15.146:0/793258593 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc0043a0 con 0x7f2dcc06eba0 2024-09-13T18:35:59.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/793258593 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dcc1dfa10 con 0x7f2dcc06eba0 2024-09-13T18:35:59.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dc27fc700 1 -- 172.21.15.146:0/793258593 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0036c0 con 0x7f2dcc06eba0 2024-09-13T18:35:59.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.604+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/793258593 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc1e0bd0 con 0x7f2dcc06eba0 2024-09-13T18:35:59.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.608+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/793258593 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dcc1e13d0 con 0x7f2dcc06eba0 2024-09-13T18:35:59.621 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.608+0000 7f2dc27fc700 1 -- 172.21.15.146:0/793258593 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2dbc003820 con 0x7f2dcc06eba0 2024-09-13T18:35:59.621 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.608+0000 7f2dc27fc700 1 --2- 172.21.15.146:0/793258593 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2db0066760 0x7f2db0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:59.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.608+0000 7f2dc27fc700 1 -- 172.21.15.146:0/793258593 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f2dbc084320 con 0x7f2dcc06eba0 2024-09-13T18:35:59.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.608+0000 7f2dd1c8b700 1 --2- 172.21.15.146:0/793258593 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2db0066760 0x7f2db0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:59.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.608+0000 7f2dd1c8b700 1 --2- 172.21.15.146:0/793258593 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2db0066760 0x7f2db0068c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8007fd0 tx=0x7f2dc8007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:59.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.612+0000 7f2dc27fc700 1 -- 172.21.15.146:0/793258593 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2dcc1e13d0 con 0x7f2dcc06eba0 2024-09-13T18:35:59.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.860+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/793258593 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2dcc1e13d0 con 0x7f2dcc06eba0 2024-09-13T18:35:59.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.864+0000 7f2dc27fc700 1 -- 172.21.15.146:0/793258593 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f2dcc1e13d0 con 0x7f2dcc06eba0 2024-09-13T18:35:59.877 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:35:59.877 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5784,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:45.771339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:35:59.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.864+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/793258593 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2db0066760 msgr2=0x7f2db0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:59.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.864+0000 7f2dd2c8d700 1 --2- 172.21.15.146:0/793258593 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2db0066760 0x7f2db0068c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8007fd0 tx=0x7f2dc8007de0 comp rx=0 tx=0).stop 2024-09-13T18:35:59.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.864+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/793258593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dcc06eba0 msgr2=0x7f2dcc1f2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:59.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.864+0000 7f2dd2c8d700 1 --2- 172.21.15.146:0/793258593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dcc06eba0 0x7f2dcc1f2bb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00d800 tx=0x7f2dbc00db10 comp rx=0 tx=0).stop 2024-09-13T18:35:59.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.864+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/793258593 shutdown_connections 2024-09-13T18:35:59.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.864+0000 7f2dd2c8d700 1 --2- 172.21.15.146:0/793258593 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2db0066760 0x7f2db0068c20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:59.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.864+0000 7f2dd2c8d700 1 --2- 172.21.15.146:0/793258593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2dcc15a000 0x7f2dcc14e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:59.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.864+0000 7f2dd2c8d700 1 --2- 172.21.15.146:0/793258593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dcc06eba0 0x7f2dcc1f2bb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:59.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.864+0000 7f2dd2c8d700 1 --2- 172.21.15.146:0/793258593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2dcc06e7c0 0x7f2dcc1f0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:59.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.864+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/793258593 >> 172.21.15.146:0/793258593 conn(0x7f2dcc071ed0 msgr2=0x7f2dcc157050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:59.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.864+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/793258593 shutdown_connections 2024-09-13T18:35:59.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:35:59.864+0000 7f2dd2c8d700 1 -- 172.21.15.146:0/793258593 wait complete. 2024-09-13T18:35:59.882 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 45 2024-09-13T18:35:59.897 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:36:01.897 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:02.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47c1eff700 1 Processor -- start 2024-09-13T18:36:02.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47c1eff700 1 -- start start 2024-09-13T18:36:02.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47c1eff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f47bc153bc0 0x7f47bc0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:02.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47c1eff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47bc0c6a80 0x7f47bc160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:02.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47c1eff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47bc153fa0 con 0x7f47bc0c6610 2024-09-13T18:36:02.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47c1eff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47bc0aa500 con 0x7f47bc153bc0 2024-09-13T18:36:02.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47c1eff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc15a7e0 con 0x7f47bc0c6a80 2024-09-13T18:36:02.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bbfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f47bc0c6610 0x7f47bc0cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59646/0 (socket says 172.21.15.146:59646) 2024-09-13T18:36:02.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bbfff700 1 -- 172.21.15.146:0/1424915071 learned_addr learned my addr 172.21.15.146:0/1424915071 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:36:02.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47c0efd700 1 --2- 172.21.15.146:0/1424915071 >> v2:172.21.15.154:3300/0 conn(0x7f47bc153bc0 0x7f47bc0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:02.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47c16fe700 1 --2- 172.21.15.146:0/1424915071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47bc0c6a80 0x7f47bc160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:02.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466629921 0 0) 0x7f47bc153fa0 con 0x7f47bc0c6610 2024-09-13T18:36:02.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47ac0057e0 con 0x7f47bc0c6610 2024-09-13T18:36:02.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4220368944 0 0) 0x7f47ac0057e0 con 0x7f47bc0c6610 2024-09-13T18:36:02.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47bc0c6a80 msgr2=0x7f47bc160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:02.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 --2- 172.21.15.146:0/1424915071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47bc0c6a80 0x7f47bc160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:02.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 >> v2:172.21.15.154:3300/0 conn(0x7f47bc153bc0 msgr2=0x7f47bc0c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:02.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 --2- 172.21.15.146:0/1424915071 >> v2:172.21.15.154:3300/0 conn(0x7f47bc153bc0 0x7f47bc0c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:02.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc0c9320 con 0x7f47bc0c6610 2024-09-13T18:36:02.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f47a8001670 con 0x7f47bc0c6610 2024-09-13T18:36:02.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 >> v1:172.21.15.146:6789/0 conn(0x7f47bc0c6610 legacy=0x7f47bc0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:02.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 --2- 172.21.15.146:0/1424915071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47bc0c6a80 0x7f47ac005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:02.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 --2- 172.21.15.146:0/1424915071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47ac006220 0x7f47ac006a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:02.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 --2- 172.21.15.146:0/1424915071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac006fc0 0x7f47ac007440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:02.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc0aa500 con 0x7f47ac006fc0 2024-09-13T18:36:02.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc15a7e0 con 0x7f47ac006220 2024-09-13T18:36:02.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac00dc70 con 0x7f47bc0c6a80 2024-09-13T18:36:02.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47c0efd700 1 --2- 172.21.15.146:0/1424915071 >> v2:172.21.15.154:3300/0 conn(0x7f47bc153bc0 0x7f47bc0c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:02.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47c16fe700 1 --2- 172.21.15.146:0/1424915071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac006fc0 0x7f47ac007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:02.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47c0efd700 1 --2- 172.21.15.146:0/1424915071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47bc0c6a80 0x7f47ac005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:02.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47bbfff700 1 --2- 172.21.15.146:0/1424915071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47ac006220 0x7f47ac006a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:02.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.012+0000 7f47c16fe700 1 -- 172.21.15.146:0/1424915071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47bc0c6a80 msgr2=0x7f47ac005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:02.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c16fe700 1 --2- 172.21.15.146:0/1424915071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47bc0c6a80 0x7f47ac005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:02.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c16fe700 1 -- 172.21.15.146:0/1424915071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47ac006220 msgr2=0x7f47ac006a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:02.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c16fe700 1 --2- 172.21.15.146:0/1424915071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47ac006220 0x7f47ac006a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:02.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c16fe700 1 -- 172.21.15.146:0/1424915071 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac0057e0 con 0x7f47ac006fc0 2024-09-13T18:36:02.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47bbfff700 1 --2- 172.21.15.146:0/1424915071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47ac006220 0x7f47ac006a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:02.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c16fe700 1 --2- 172.21.15.146:0/1424915071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac006fc0 0x7f47ac007440 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009fc0 tx=0x7f47ac007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f0c22accf58d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:02.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b400ec30 con 0x7f47ac006fc0 2024-09-13T18:36:02.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b400f040 con 0x7f47ac006fc0 2024-09-13T18:36:02.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47bb7fe700 1 -- 172.21.15.146:0/1424915071 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4014480 con 0x7f47ac006fc0 2024-09-13T18:36:02.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 -- 172.21.15.146:0/1424915071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac006fc0 msgr2=0x7f47ac007440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:02.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 --2- 172.21.15.146:0/1424915071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac006fc0 0x7f47ac007440 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009fc0 tx=0x7f47ac007ab0 comp rx=0 tx=0).stop 2024-09-13T18:36:02.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c16fe700 1 -- 172.21.15.146:0/1424915071 reap_dead start 2024-09-13T18:36:02.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 -- 172.21.15.146:0/1424915071 shutdown_connections 2024-09-13T18:36:02.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 -- 172.21.15.146:0/1424915071 >> 172.21.15.146:0/1424915071 conn(0x7f47bc066090 msgr2=0x7f47bc066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:02.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 -- 172.21.15.146:0/1424915071 shutdown_connections 2024-09-13T18:36:02.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 -- 172.21.15.146:0/1424915071 wait complete. 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 Processor -- start 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 -- start start 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47bc0aedf0 0x7f47bc0c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47bc0af1d0 0x7f47bc158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47bc150c90 0x7f47bc1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc1596f0 con 0x7f47bc150c90 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc159130 con 0x7f47bc0aedf0 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c1eff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc159410 con 0x7f47bc0af1d0 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c16fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47bc150c90 0x7f47bc1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c16fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47bc150c90 0x7f47bc1542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38238/0 (socket says 172.21.15.146:38238) 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c16fe700 1 -- 172.21.15.146:0/1599959370 learned_addr learned my addr 172.21.15.146:0/1599959370 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47bbfff700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47bc0af1d0 0x7f47bc158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.016+0000 7f47c0efd700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47bc0aedf0 0x7f47bc0c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:02.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47c16fe700 1 -- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47bc0af1d0 msgr2=0x7f47bc158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:02.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47c16fe700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47bc0af1d0 0x7f47bc158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:02.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47c16fe700 1 -- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47bc0aedf0 msgr2=0x7f47bc0c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:02.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47c16fe700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47bc0aedf0 0x7f47bc0c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:02.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47c16fe700 1 -- 172.21.15.146:0/1599959370 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc0c3920 con 0x7f47bc150c90 2024-09-13T18:36:02.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47c16fe700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47bc150c90 0x7f47bc1542e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f47b4016000 tx=0x7f47b4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:02.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47b97fa700 1 -- 172.21.15.146:0/1599959370 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4007000 con 0x7f47bc150c90 2024-09-13T18:36:02.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47c1eff700 1 -- 172.21.15.146:0/1599959370 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47bc0c3c00 con 0x7f47bc150c90 2024-09-13T18:36:02.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47c1eff700 1 -- 172.21.15.146:0/1599959370 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47bc1f34c0 con 0x7f47bc150c90 2024-09-13T18:36:02.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47b97fa700 1 -- 172.21.15.146:0/1599959370 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b4022070 con 0x7f47bc150c90 2024-09-13T18:36:02.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47b97fa700 1 -- 172.21.15.146:0/1599959370 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4015dd0 con 0x7f47bc150c90 2024-09-13T18:36:02.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47c1eff700 1 -- 172.21.15.146:0/1599959370 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479c005310 con 0x7f47bc150c90 2024-09-13T18:36:02.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47b97fa700 1 -- 172.21.15.146:0/1599959370 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f47b4029480 con 0x7f47bc150c90 2024-09-13T18:36:02.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47b97fa700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f47a0066760 0x7f47a0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:02.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47b97fa700 1 -- 172.21.15.146:0/1599959370 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f47b4024070 con 0x7f47bc150c90 2024-09-13T18:36:02.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.020+0000 7f47c0efd700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f47a0066760 0x7f47a0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:02.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.024+0000 7f47c0efd700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f47a0066760 0x7f47a0068c20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009e70 tx=0x7f47b0009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:02.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.028+0000 7f47b97fa700 1 -- 172.21.15.146:0/1599959370 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f47b40533b0 con 0x7f47bc150c90 2024-09-13T18:36:02.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.284+0000 7f47c1eff700 1 -- 172.21.15.146:0/1599959370 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f479c0055c0 con 0x7f47bc150c90 2024-09-13T18:36:02.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.288+0000 7f47b97fa700 1 -- 172.21.15.146:0/1599959370 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f47b40569d0 con 0x7f47bc150c90 2024-09-13T18:36:02.301 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:02.301 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5784,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:45.771339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:02.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.288+0000 7f47c1eff700 1 -- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f47a0066760 msgr2=0x7f47a0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:02.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.288+0000 7f47c1eff700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f47a0066760 0x7f47a0068c20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009e70 tx=0x7f47b0009240 comp rx=0 tx=0).stop 2024-09-13T18:36:02.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.288+0000 7f47c1eff700 1 -- 172.21.15.146:0/1599959370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47bc150c90 msgr2=0x7f47bc1542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:02.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.288+0000 7f47c1eff700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47bc150c90 0x7f47bc1542e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f47b4016000 tx=0x7f47b4009750 comp rx=0 tx=0).stop 2024-09-13T18:36:02.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.288+0000 7f47c1eff700 1 -- 172.21.15.146:0/1599959370 shutdown_connections 2024-09-13T18:36:02.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.288+0000 7f47c1eff700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f47a0066760 0x7f47a0068c20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:02.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.288+0000 7f47c1eff700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47bc150c90 0x7f47bc1542e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:02.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.288+0000 7f47c1eff700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47bc0af1d0 0x7f47bc158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:02.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.288+0000 7f47c1eff700 1 --2- 172.21.15.146:0/1599959370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47bc0aedf0 0x7f47bc0c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:02.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.288+0000 7f47c1eff700 1 -- 172.21.15.146:0/1599959370 >> 172.21.15.146:0/1599959370 conn(0x7f47bc066090 msgr2=0x7f47bc06f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:02.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.288+0000 7f47c1eff700 1 -- 172.21.15.146:0/1599959370 shutdown_connections 2024-09-13T18:36:02.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:02.288+0000 7f47c1eff700 1 -- 172.21.15.146:0/1599959370 wait complete. 2024-09-13T18:36:02.306 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 45 2024-09-13T18:36:02.315 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:36:04.316 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:04.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f74549cf700 1 Processor -- start 2024-09-13T18:36:04.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f74549cf700 1 -- start start 2024-09-13T18:36:04.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f74549cf700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f74500ca210 0x7f74500b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:04.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f74549cf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7450071390 0x7f74500cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:04.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f74549cf700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74500cc670 con 0x7f74500cbd50 2024-09-13T18:36:04.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f74549cf700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74500caa30 con 0x7f74500ca210 2024-09-13T18:36:04.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f74549cf700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74500cb810 con 0x7f7450071390 2024-09-13T18:36:04.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744f7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f74500cbd50 0x7f74500701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59650/0 (socket says 172.21.15.146:59650) 2024-09-13T18:36:04.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744f7fe700 1 -- 172.21.15.146:0/3793010432 learned_addr learned my addr 172.21.15.146:0/3793010432 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:36:04.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744ffff700 1 --2- 172.21.15.146:0/3793010432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7450071390 0x7f74500cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:04.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744effd700 1 --2- 172.21.15.146:0/3793010432 >> v2:172.21.15.154:3300/0 conn(0x7f74500ca210 0x7f74500b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:04.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427702220 0 0) 0x7f7440001110 con 0x7f74500cbd50 2024-09-13T18:36:04.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74380035d0 con 0x7f74500cbd50 2024-09-13T18:36:04.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1008965026 0 0) 0x7f74380035d0 con 0x7f74500cbd50 2024-09-13T18:36:04.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7450071390 msgr2=0x7f74500cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:04.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744e7fc700 1 --2- 172.21.15.146:0/3793010432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7450071390 0x7f74500cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:04.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 >> v2:172.21.15.154:3300/0 conn(0x7f74500ca210 msgr2=0x7f74500b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:04.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744e7fc700 1 --2- 172.21.15.146:0/3793010432 >> v2:172.21.15.154:3300/0 conn(0x7f74500ca210 0x7f74500b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:04.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745015da20 con 0x7f74500cbd50 2024-09-13T18:36:04.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f7440001800 con 0x7f74500cbd50 2024-09-13T18:36:04.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 >> v1:172.21.15.146:6789/0 conn(0x7f74500cbd50 legacy=0x7f74500701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:04.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744e7fc700 1 --2- 172.21.15.146:0/3793010432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7438003a90 0x7f7438003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744e7fc700 1 --2- 172.21.15.146:0/3793010432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7450071390 0x7f7438004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.428+0000 7f744e7fc700 1 --2- 172.21.15.146:0/3793010432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7438004db0 0x7f7438005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74500caa30 con 0x7f7438004db0 2024-09-13T18:36:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74500cb810 con 0x7f7438003a90 2024-09-13T18:36:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743800ba30 con 0x7f7450071390 2024-09-13T18:36:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744effd700 1 --2- 172.21.15.146:0/3793010432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7450071390 0x7f7438004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744f7fe700 1 --2- 172.21.15.146:0/3793010432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7438003a90 0x7f7438003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744ffff700 1 --2- 172.21.15.146:0/3793010432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7438004db0 0x7f7438005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744effd700 1 -- 172.21.15.146:0/3793010432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7438003a90 msgr2=0x7f7438003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744effd700 1 --2- 172.21.15.146:0/3793010432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7438003a90 0x7f7438003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744effd700 1 -- 172.21.15.146:0/3793010432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7438004db0 msgr2=0x7f7438005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744effd700 1 --2- 172.21.15.146:0/3793010432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7438004db0 0x7f7438005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744effd700 1 -- 172.21.15.146:0/3793010432 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74380035d0 con 0x7f7450071390 2024-09-13T18:36:04.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744effd700 1 --2- 172.21.15.146:0/3793010432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7450071390 0x7f7438004870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7444009b70 tx=0x7f74380058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b24486dc9a459a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:04.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744400e6c0 con 0x7f7450071390 2024-09-13T18:36:04.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744400e820 con 0x7f7450071390 2024-09-13T18:36:04.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744e7fc700 1 -- 172.21.15.146:0/3793010432 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744400eb90 con 0x7f7450071390 2024-09-13T18:36:04.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f74549cf700 1 -- 172.21.15.146:0/3793010432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7450071390 msgr2=0x7f7438004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:04.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f74549cf700 1 --2- 172.21.15.146:0/3793010432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7450071390 0x7f7438004870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7444009b70 tx=0x7f74380058a0 comp rx=0 tx=0).stop 2024-09-13T18:36:04.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f744ffff700 1 -- 172.21.15.146:0/3793010432 reap_dead start 2024-09-13T18:36:04.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f74549cf700 1 -- 172.21.15.146:0/3793010432 shutdown_connections 2024-09-13T18:36:04.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f74549cf700 1 -- 172.21.15.146:0/3793010432 >> 172.21.15.146:0/3793010432 conn(0x7f7450066270 msgr2=0x7f7450066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:04.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f74549cf700 1 -- 172.21.15.146:0/3793010432 shutdown_connections 2024-09-13T18:36:04.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f74549cf700 1 -- 172.21.15.146:0/3793010432 wait complete. 2024-09-13T18:36:04.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f74549cf700 1 Processor -- start 2024-09-13T18:36:04.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.432+0000 7f74549cf700 1 -- start start 2024-09-13T18:36:04.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f74549cf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7450071390 0x7f74500c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:04.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f74549cf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74500c7460 0x7f74500b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:04.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f74549cf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f745015a0f0 0x7f74500b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:04.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f74549cf700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74500c49d0 con 0x7f7450071390 2024-09-13T18:36:04.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f74549cf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74500b1910 con 0x7f745015a0f0 2024-09-13T18:36:04.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f74549cf700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74500cd490 con 0x7f74500c7460 2024-09-13T18:36:04.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f744f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7450071390 0x7f74500c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:04.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f744f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7450071390 0x7f74500c6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38262/0 (socket says 172.21.15.146:38262) 2024-09-13T18:36:04.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f744f7fe700 1 -- 172.21.15.146:0/582494018 learned_addr learned my addr 172.21.15.146:0/582494018 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:04.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f744effd700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74500c7460 0x7f74500b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:04.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f744ffff700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f745015a0f0 0x7f74500b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:04.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f744f7fe700 1 -- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74500c7460 msgr2=0x7f74500b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:04.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f744f7fe700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74500c7460 0x7f74500b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:04.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f744f7fe700 1 -- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f745015a0f0 msgr2=0x7f74500b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:04.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f744f7fe700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f745015a0f0 0x7f74500b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:04.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f744f7fe700 1 -- 172.21.15.146:0/582494018 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74500c79a0 con 0x7f7450071390 2024-09-13T18:36:04.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f744f7fe700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7450071390 0x7f74500c6a90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7440007310 tx=0x7f74400076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:04.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f7437fff700 1 -- 172.21.15.146:0/582494018 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74400077f0 con 0x7f7450071390 2024-09-13T18:36:04.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f74549cf700 1 -- 172.21.15.146:0/582494018 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74500c7c60 con 0x7f7450071390 2024-09-13T18:36:04.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f7437fff700 1 -- 172.21.15.146:0/582494018 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744000f870 con 0x7f7450071390 2024-09-13T18:36:04.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f7437fff700 1 -- 172.21.15.146:0/582494018 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744000fc40 con 0x7f7450071390 2024-09-13T18:36:04.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f74549cf700 1 -- 172.21.15.146:0/582494018 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74500c8220 con 0x7f7450071390 2024-09-13T18:36:04.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f74549cf700 1 -- 172.21.15.146:0/582494018 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74500c5d00 con 0x7f7450071390 2024-09-13T18:36:04.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f7437fff700 1 -- 172.21.15.146:0/582494018 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f744001d460 con 0x7f7450071390 2024-09-13T18:36:04.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f7437fff700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f743c066730 0x7f743c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:04.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f7437fff700 1 -- 172.21.15.146:0/582494018 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f74400845e0 con 0x7f7450071390 2024-09-13T18:36:04.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.436+0000 7f744effd700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f743c066730 0x7f743c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:04.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.440+0000 7f744effd700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f743c066730 0x7f743c068bf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f74440090b0 tx=0x7f7444008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:04.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.444+0000 7f7437fff700 1 -- 172.21.15.146:0/582494018 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f74400512d0 con 0x7f7450071390 2024-09-13T18:36:04.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.668+0000 7f74549cf700 1 -- 172.21.15.146:0/582494018 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74500cc670 con 0x7f7450071390 2024-09-13T18:36:04.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.668+0000 7f7437fff700 1 -- 172.21.15.146:0/582494018 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f74500cc670 con 0x7f7450071390 2024-09-13T18:36:04.684 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:04.684 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5784,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:45.771339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:04.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.672+0000 7f74549cf700 1 -- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f743c066730 msgr2=0x7f743c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:04.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.672+0000 7f74549cf700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f743c066730 0x7f743c068bf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f74440090b0 tx=0x7f7444008030 comp rx=0 tx=0).stop 2024-09-13T18:36:04.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.672+0000 7f74549cf700 1 -- 172.21.15.146:0/582494018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7450071390 msgr2=0x7f74500c6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:04.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.672+0000 7f74549cf700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7450071390 0x7f74500c6a90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7440007310 tx=0x7f74400076d0 comp rx=0 tx=0).stop 2024-09-13T18:36:04.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.672+0000 7f74549cf700 1 -- 172.21.15.146:0/582494018 shutdown_connections 2024-09-13T18:36:04.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.672+0000 7f74549cf700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f743c066730 0x7f743c068bf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:04.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.672+0000 7f74549cf700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f745015a0f0 0x7f74500b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:04.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.672+0000 7f74549cf700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74500c7460 0x7f74500b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:04.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.672+0000 7f74549cf700 1 --2- 172.21.15.146:0/582494018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7450071390 0x7f74500c6a90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:04.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.672+0000 7f74549cf700 1 -- 172.21.15.146:0/582494018 >> 172.21.15.146:0/582494018 conn(0x7f7450066270 msgr2=0x7f745014f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:04.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.672+0000 7f74549cf700 1 -- 172.21.15.146:0/582494018 shutdown_connections 2024-09-13T18:36:04.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:04.672+0000 7f74549cf700 1 -- 172.21.15.146:0/582494018 wait complete. 2024-09-13T18:36:04.687 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 45 2024-09-13T18:36:04.699 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:36:06.699 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:06.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.812+0000 7f06ed4c2700 1 Processor -- start 2024-09-13T18:36:06.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.812+0000 7f06ed4c2700 1 -- start start 2024-09-13T18:36:06.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.812+0000 7f06ed4c2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06e81533c0 0x7f06e81537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:06.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.812+0000 7f06ed4c2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06e806eed0 0x7f06e806af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:06.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.812+0000 7f06ed4c2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e80c2420 con 0x7f06e8159440 2024-09-13T18:36:06.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.812+0000 7f06ed4c2700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e80c1600 con 0x7f06e81533c0 2024-09-13T18:36:06.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.812+0000 7f06ed4c2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80c6da0 con 0x7f06e806eed0 2024-09-13T18:36:06.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06eccc1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06e806eed0 0x7f06e806af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:06.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06e7fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06e81533c0 0x7f06e81537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:06.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06eccc1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06e806eed0 0x7f06e806af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:38452/0 (socket says 172.21.15.146:38452) 2024-09-13T18:36:06.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06eccc1700 1 -- 172.21.15.146:0/1056969825 learned_addr learned my addr 172.21.15.146:0/1056969825 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:06.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06eccc1700 1 -- 172.21.15.146:0/1056969825 >> v2:172.21.15.154:3300/0 conn(0x7f06e81533c0 msgr2=0x7f06e81537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:06.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06eccc1700 1 --2- 172.21.15.146:0/1056969825 >> v2:172.21.15.154:3300/0 conn(0x7f06e81533c0 0x7f06e81537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:06.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06eccc1700 1 -- 172.21.15.146:0/1056969825 >> v1:172.21.15.154:6789/0 conn(0x7f06e8159440 legacy=0x7f06e80c1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:06.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06eccc1700 1 -- 172.21.15.146:0/1056969825 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80c73a0 con 0x7f06e806eed0 2024-09-13T18:36:06.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06e7fff700 1 --2- 172.21.15.146:0/1056969825 >> v2:172.21.15.154:3300/0 conn(0x7f06e81533c0 0x7f06e81537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:06.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06eccc1700 1 --2- 172.21.15.146:0/1056969825 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06e806eed0 0x7f06e806af80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009e50 tx=0x7f06e000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d9774056e27b047 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:06.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06e77fe700 1 -- 172.21.15.146:0/1056969825 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e000e070 con 0x7f06e806eed0 2024-09-13T18:36:06.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06e77fe700 1 -- 172.21.15.146:0/1056969825 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e000b910 con 0x7f06e806eed0 2024-09-13T18:36:06.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06e77fe700 1 -- 172.21.15.146:0/1056969825 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e000bcc0 con 0x7f06e806eed0 2024-09-13T18:36:06.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1056969825 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06e806eed0 msgr2=0x7f06e806af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:06.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06ed4c2700 1 --2- 172.21.15.146:0/1056969825 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06e806eed0 0x7f06e806af80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009e50 tx=0x7f06e000b330 comp rx=0 tx=0).stop 2024-09-13T18:36:06.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1056969825 shutdown_connections 2024-09-13T18:36:06.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06ed4c2700 1 --2- 172.21.15.146:0/1056969825 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06e806eed0 0x7f06e806af80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:06.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06ed4c2700 1 --2- 172.21.15.146:0/1056969825 >> v2:172.21.15.154:3300/0 conn(0x7f06e81533c0 0x7f06e81537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:06.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1056969825 >> 172.21.15.146:0/1056969825 conn(0x7f06e80cbf60 msgr2=0x7f06e80cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:06.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1056969825 shutdown_connections 2024-09-13T18:36:06.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1056969825 wait complete. 2024-09-13T18:36:06.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06ed4c2700 1 Processor -- start 2024-09-13T18:36:06.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.816+0000 7f06ed4c2700 1 -- start start 2024-09-13T18:36:06.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06ed4c2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e806eed0 0x7f06e81f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:06.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06ed4c2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06e81533c0 0x7f06e814ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:06.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06ed4c2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e8159440 0x7f06e8151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:06.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06ed4c2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e81efa90 con 0x7f06e806eed0 2024-09-13T18:36:06.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06ed4c2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e81f34f0 con 0x7f06e8159440 2024-09-13T18:36:06.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06ed4c2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80cb680 con 0x7f06e81533c0 2024-09-13T18:36:06.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06e7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e806eed0 0x7f06e81f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:06.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06e7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e806eed0 0x7f06e81f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38266/0 (socket says 172.21.15.146:38266) 2024-09-13T18:36:06.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06e7fff700 1 -- 172.21.15.146:0/1365629854 learned_addr learned my addr 172.21.15.146:0/1365629854 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:06.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06eccc1700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e8159440 0x7f06e8151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:06.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06dffff700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06e81533c0 0x7f06e814ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:06.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06e7fff700 1 -- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06e81533c0 msgr2=0x7f06e814ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:06.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06e7fff700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06e81533c0 0x7f06e814ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:06.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06e7fff700 1 -- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e8159440 msgr2=0x7f06e8151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:06.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06e7fff700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e8159440 0x7f06e8151710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:06.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06e7fff700 1 -- 172.21.15.146:0/1365629854 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d80094b0 con 0x7f06e806eed0 2024-09-13T18:36:06.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06dffff700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06e81533c0 0x7f06e814ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:06.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06eccc1700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e8159440 0x7f06e8151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:06.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06e7fff700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e806eed0 0x7f06e81f2fb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f06d800ebe0 tx=0x7f06d800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:06.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06e57fa700 1 -- 172.21.15.146:0/1365629854 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d800c850 con 0x7f06e806eed0 2024-09-13T18:36:06.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06e57fa700 1 -- 172.21.15.146:0/1365629854 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d800c9b0 con 0x7f06e806eed0 2024-09-13T18:36:06.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1365629854 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e0009b30 con 0x7f06e806eed0 2024-09-13T18:36:06.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06e57fa700 1 -- 172.21.15.146:0/1365629854 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d800cd40 con 0x7f06e806eed0 2024-09-13T18:36:06.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1365629854 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e80cb920 con 0x7f06e806eed0 2024-09-13T18:36:06.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.820+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1365629854 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e81efd00 con 0x7f06e806eed0 2024-09-13T18:36:06.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.824+0000 7f06e57fa700 1 -- 172.21.15.146:0/1365629854 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f06d8013470 con 0x7f06e806eed0 2024-09-13T18:36:06.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.824+0000 7f06e57fa700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f06c8066730 0x7f06c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:06.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.824+0000 7f06e57fa700 1 -- 172.21.15.146:0/1365629854 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f06d8084760 con 0x7f06e806eed0 2024-09-13T18:36:06.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.824+0000 7f06dffff700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f06c8066730 0x7f06c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:06.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.824+0000 7f06dffff700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f06c8066730 0x7f06c8068bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f06e814d0d0 tx=0x7f06d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:06.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:06.828+0000 7f06e57fa700 1 -- 172.21.15.146:0/1365629854 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f06d8051570 con 0x7f06e806eed0 2024-09-13T18:36:07.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.052+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1365629854 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06e8158090 con 0x7f06e806eed0 2024-09-13T18:36:07.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.052+0000 7f06e57fa700 1 -- 172.21.15.146:0/1365629854 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f06d800cea0 con 0x7f06e806eed0 2024-09-13T18:36:07.066 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:07.066 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5784,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:45.771339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:07.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.056+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f06c8066730 msgr2=0x7f06c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:07.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.056+0000 7f06ed4c2700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f06c8066730 0x7f06c8068bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f06e814d0d0 tx=0x7f06d0009040 comp rx=0 tx=0).stop 2024-09-13T18:36:07.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.056+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1365629854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e806eed0 msgr2=0x7f06e81f2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:07.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.056+0000 7f06ed4c2700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e806eed0 0x7f06e81f2fb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f06d800ebe0 tx=0x7f06d800eef0 comp rx=0 tx=0).stop 2024-09-13T18:36:07.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.056+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1365629854 shutdown_connections 2024-09-13T18:36:07.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.056+0000 7f06ed4c2700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f06c8066730 0x7f06c8068bf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:07.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.056+0000 7f06ed4c2700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e8159440 0x7f06e8151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:07.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.056+0000 7f06ed4c2700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06e81533c0 0x7f06e814ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:07.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.056+0000 7f06ed4c2700 1 --2- 172.21.15.146:0/1365629854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e806eed0 0x7f06e81f2fb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:07.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.056+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1365629854 >> 172.21.15.146:0/1365629854 conn(0x7f06e80cbf60 msgr2=0x7f06e8157650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:07.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.056+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1365629854 shutdown_connections 2024-09-13T18:36:07.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:07.056+0000 7f06ed4c2700 1 -- 172.21.15.146:0/1365629854 wait complete. 2024-09-13T18:36:07.073 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 45 2024-09-13T18:36:07.084 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:36:08.597 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:36:08.599 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:36:09.085 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:09.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.192+0000 7f1511fbc700 1 Processor -- start 2024-09-13T18:36:09.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.192+0000 7f1511fbc700 1 -- start start 2024-09-13T18:36:09.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.192+0000 7f1511fbc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f150c0cf170 0x7f150c1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:09.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.192+0000 7f1511fbc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f150c0cf550 0x7f150c0c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:09.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.192+0000 7f1511fbc700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150c15a540 con 0x7f150c0cb5b0 2024-09-13T18:36:09.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.192+0000 7f1511fbc700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c0c5220 con 0x7f150c0cf550 2024-09-13T18:36:09.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.192+0000 7f1511fbc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c159ca0 con 0x7f150c0cf170 2024-09-13T18:36:09.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.192+0000 7f150bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f150c0cf550 0x7f150c0c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:09.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.192+0000 7f15117bb700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f150c0cb5b0 0x7f150c1534c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59664/0 (socket says 172.21.15.146:59664) 2024-09-13T18:36:09.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.192+0000 7f15117bb700 1 -- 172.21.15.146:0/645285432 learned_addr learned my addr 172.21.15.146:0/645285432 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:36:09.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.192+0000 7f1510fba700 1 --2- 172.21.15.146:0/645285432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f150c0cf170 0x7f150c1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:09.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f150b7fe700 1 -- 172.21.15.146:0/645285432 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938733019 0 0) 0x7f150c15a540 con 0x7f150c0cb5b0 2024-09-13T18:36:09.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f150b7fe700 1 -- 172.21.15.146:0/645285432 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14fc0035d0 con 0x7f150c0cb5b0 2024-09-13T18:36:09.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f150bfff700 1 -- 172.21.15.146:0/645285432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f150c0cf170 msgr2=0x7f150c1595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:09.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f150bfff700 1 --2- 172.21.15.146:0/645285432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f150c0cf170 0x7f150c1595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:09.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f150bfff700 1 -- 172.21.15.146:0/645285432 >> v1:172.21.15.146:6789/0 conn(0x7f150c0cb5b0 legacy=0x7f150c1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:09.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f150bfff700 1 -- 172.21.15.146:0/645285432 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c159f80 con 0x7f150c0cf550 2024-09-13T18:36:09.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f150bfff700 1 --2- 172.21.15.146:0/645285432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0cf550 0x7f150c0c56b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009850 tx=0x7f14f8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183bd13a1d5b8346 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:09.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f150b7fe700 1 -- 172.21.15.146:0/645285432 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8011830 con 0x7f150c0cf550 2024-09-13T18:36:09.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f150b7fe700 1 -- 172.21.15.146:0/645285432 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f8011990 con 0x7f150c0cf550 2024-09-13T18:36:09.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f150b7fe700 1 -- 172.21.15.146:0/645285432 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8011d40 con 0x7f150c0cf550 2024-09-13T18:36:09.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f1511fbc700 1 -- 172.21.15.146:0/645285432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0cf550 msgr2=0x7f150c0c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:09.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f1511fbc700 1 --2- 172.21.15.146:0/645285432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0cf550 0x7f150c0c56b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009850 tx=0x7f14f8009b60 comp rx=0 tx=0).stop 2024-09-13T18:36:09.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f1511fbc700 1 -- 172.21.15.146:0/645285432 shutdown_connections 2024-09-13T18:36:09.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f1511fbc700 1 --2- 172.21.15.146:0/645285432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0cf550 0x7f150c0c56b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:09.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f1511fbc700 1 --2- 172.21.15.146:0/645285432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f150c0cf170 0x7f150c1595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:09.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f1511fbc700 1 -- 172.21.15.146:0/645285432 >> 172.21.15.146:0/645285432 conn(0x7f150c072220 msgr2=0x7f150c066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:09.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f1511fbc700 1 -- 172.21.15.146:0/645285432 shutdown_connections 2024-09-13T18:36:09.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f1511fbc700 1 -- 172.21.15.146:0/645285432 wait complete. 2024-09-13T18:36:09.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.196+0000 7f1511fbc700 1 Processor -- start 2024-09-13T18:36:09.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1511fbc700 1 -- start start 2024-09-13T18:36:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1511fbc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0cb5b0 0x7f150c0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1511fbc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f150c0cf170 0x7f150c0ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1511fbc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f150c0cf550 0x7f150c157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1511fbc700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c1ef7c0 con 0x7f150c0cb5b0 2024-09-13T18:36:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1511fbc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c1efcf0 con 0x7f150c0cf550 2024-09-13T18:36:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1511fbc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c1576f0 con 0x7f150c0cf170 2024-09-13T18:36:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1510fba700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0cb5b0 0x7f150c0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1510fba700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0cb5b0 0x7f150c0ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38288/0 (socket says 172.21.15.146:38288) 2024-09-13T18:36:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1510fba700 1 -- 172.21.15.146:0/2478310035 learned_addr learned my addr 172.21.15.146:0/2478310035 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f150bfff700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f150c0cf170 0x7f150c0ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f15117bb700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f150c0cf550 0x7f150c157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1510fba700 1 -- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f150c0cf170 msgr2=0x7f150c0ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1510fba700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f150c0cf170 0x7f150c0ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1510fba700 1 -- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f150c0cf550 msgr2=0x7f150c157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1510fba700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f150c0cf550 0x7f150c157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1510fba700 1 -- 172.21.15.146:0/2478310035 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f80094b0 con 0x7f150c0cb5b0 2024-09-13T18:36:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1510fba700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0cb5b0 0x7f150c0ce030 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f150000ea30 tx=0x7f150000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:09.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f15097fa700 1 -- 172.21.15.146:0/2478310035 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500009720 con 0x7f150c0cb5b0 2024-09-13T18:36:09.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f15097fa700 1 -- 172.21.15.146:0/2478310035 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150000c910 con 0x7f150c0cb5b0 2024-09-13T18:36:09.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1511fbc700 1 -- 172.21.15.146:0/2478310035 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c159370 con 0x7f150c0cb5b0 2024-09-13T18:36:09.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f15097fa700 1 -- 172.21.15.146:0/2478310035 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150000cc80 con 0x7f150c0cb5b0 2024-09-13T18:36:09.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.200+0000 7f1511fbc700 1 -- 172.21.15.146:0/2478310035 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c157380 con 0x7f150c0cb5b0 2024-09-13T18:36:09.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.204+0000 7f1511fbc700 1 -- 172.21.15.146:0/2478310035 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c066850 con 0x7f150c0cb5b0 2024-09-13T18:36:09.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.204+0000 7f15097fa700 1 -- 172.21.15.146:0/2478310035 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1500013470 con 0x7f150c0cb5b0 2024-09-13T18:36:09.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.204+0000 7f15097fa700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f14f0066730 0x7f14f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:09.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.204+0000 7f15097fa700 1 -- 172.21.15.146:0/2478310035 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f1500084750 con 0x7f150c0cb5b0 2024-09-13T18:36:09.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.212+0000 7f15097fa700 1 -- 172.21.15.146:0/2478310035 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f15000515e0 con 0x7f150c0cb5b0 2024-09-13T18:36:09.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.212+0000 7f150bfff700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f14f0066730 0x7f14f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:09.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.212+0000 7f150bfff700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f14f0066730 0x7f14f0068bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009850 tx=0x7f14f8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:09.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.444+0000 7f1511fbc700 1 -- 172.21.15.146:0/2478310035 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f150c0cb180 con 0x7f150c0cb5b0 2024-09-13T18:36:09.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.444+0000 7f15097fa700 1 -- 172.21.15.146:0/2478310035 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6296 (secure 0 0 0) 0x7f1500054c00 con 0x7f150c0cb5b0 2024-09-13T18:36:09.459 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:09.459 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5784,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:34:45.771339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5278,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5278":{"gid":5278,"name":"f","rank":1,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.154:6839/3274220802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":3274220802},{"type":"v1","addr":"172.21.15.154:6839","nonce":3274220802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:09.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.448+0000 7f1511fbc700 1 -- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f14f0066730 msgr2=0x7f14f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:09.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.448+0000 7f1511fbc700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f14f0066730 0x7f14f0068bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009850 tx=0x7f14f8008030 comp rx=0 tx=0).stop 2024-09-13T18:36:09.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.448+0000 7f1511fbc700 1 -- 172.21.15.146:0/2478310035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0cb5b0 msgr2=0x7f150c0ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:09.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.448+0000 7f1511fbc700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0cb5b0 0x7f150c0ce030 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f150000ea30 tx=0x7f150000edf0 comp rx=0 tx=0).stop 2024-09-13T18:36:09.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.448+0000 7f1511fbc700 1 -- 172.21.15.146:0/2478310035 shutdown_connections 2024-09-13T18:36:09.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.448+0000 7f1511fbc700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f14f0066730 0x7f14f0068bf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:09.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.448+0000 7f1511fbc700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f150c0cf550 0x7f150c157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:09.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.448+0000 7f1511fbc700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f150c0cf170 0x7f150c0ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:09.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.448+0000 7f1511fbc700 1 --2- 172.21.15.146:0/2478310035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0cb5b0 0x7f150c0ce030 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:09.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.448+0000 7f1511fbc700 1 -- 172.21.15.146:0/2478310035 >> 172.21.15.146:0/2478310035 conn(0x7f150c072220 msgr2=0x7f150c1e5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:09.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.448+0000 7f1511fbc700 1 -- 172.21.15.146:0/2478310035 shutdown_connections 2024-09-13T18:36:09.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:09.448+0000 7f1511fbc700 1 -- 172.21.15.146:0/2478310035 wait complete. 2024-09-13T18:36:09.467 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 45 2024-09-13T18:36:09.474 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:36:11.476 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:11.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd95d1a700 1 Processor -- start 2024-09-13T18:36:11.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd95d1a700 1 -- start start 2024-09-13T18:36:11.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd95d1a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbd900cbd50 0x7fbd900b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd95d1a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbd900ca210 0x7fbd900cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd95d1a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd900cc670 con 0x7fbd90071390 2024-09-13T18:36:11.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd95d1a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd900caa30 con 0x7fbd900ca210 2024-09-13T18:36:11.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd95d1a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd900cb810 con 0x7fbd900cbd50 2024-09-13T18:36:11.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd8ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbd900ca210 0x7fbd900cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd8ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbd900ca210 0x7fbd900cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39096/0 (socket says 172.21.15.146:39096) 2024-09-13T18:36:11.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd8ffff700 1 -- 172.21.15.146:0/1921874195 learned_addr learned my addr 172.21.15.146:0/1921874195 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:11.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd94d18700 1 --2- 172.21.15.146:0/1921874195 >> v2:172.21.15.154:3300/0 conn(0x7fbd900cbd50 0x7fbd900b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd8ffff700 1 -- 172.21.15.146:0/1921874195 >> v2:172.21.15.154:3300/0 conn(0x7fbd900cbd50 msgr2=0x7fbd900b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:11.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd8ffff700 1 --2- 172.21.15.146:0/1921874195 >> v2:172.21.15.154:3300/0 conn(0x7fbd900cbd50 0x7fbd900b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd8ffff700 1 -- 172.21.15.146:0/1921874195 >> v1:172.21.15.154:6789/0 conn(0x7fbd90071390 legacy=0x7fbd900701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:11.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd8ffff700 1 -- 172.21.15.146:0/1921874195 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9015da20 con 0x7fbd900ca210 2024-09-13T18:36:11.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.580+0000 7fbd8ffff700 1 --2- 172.21.15.146:0/1921874195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd900ca210 0x7fbd900cc130 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbd9008c2d0 tx=0x7fbd84009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc3337b91c5b814 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:11.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.584+0000 7fbd8f7fe700 1 -- 172.21.15.146:0/1921874195 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8400b8a0 con 0x7fbd900ca210 2024-09-13T18:36:11.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.584+0000 7fbd8f7fe700 1 -- 172.21.15.146:0/1921874195 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8400ba00 con 0x7fbd900ca210 2024-09-13T18:36:11.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.584+0000 7fbd8f7fe700 1 -- 172.21.15.146:0/1921874195 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8400bd90 con 0x7fbd900ca210 2024-09-13T18:36:11.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.584+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1921874195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd900ca210 msgr2=0x7fbd900cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:11.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.584+0000 7fbd95d1a700 1 --2- 172.21.15.146:0/1921874195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd900ca210 0x7fbd900cc130 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbd9008c2d0 tx=0x7fbd84009bf0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.584+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1921874195 shutdown_connections 2024-09-13T18:36:11.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.584+0000 7fbd95d1a700 1 --2- 172.21.15.146:0/1921874195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd900ca210 0x7fbd900cc130 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.584+0000 7fbd95d1a700 1 --2- 172.21.15.146:0/1921874195 >> v2:172.21.15.154:3300/0 conn(0x7fbd900cbd50 0x7fbd900b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.584+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1921874195 >> 172.21.15.146:0/1921874195 conn(0x7fbd90066270 msgr2=0x7fbd90066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:11.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.584+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1921874195 shutdown_connections 2024-09-13T18:36:11.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.584+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1921874195 wait complete. 2024-09-13T18:36:11.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.588+0000 7fbd95d1a700 1 Processor -- start 2024-09-13T18:36:11.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.588+0000 7fbd95d1a700 1 -- start start 2024-09-13T18:36:11.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.588+0000 7fbd95d1a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd90071390 0x7fbd900c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.588+0000 7fbd95d1a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd900cbd50 0x7fbd90140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.588+0000 7fbd95d1a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd90140da0 0x7fbd9015ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.588+0000 7fbd95d1a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900b24c0 con 0x7fbd90071390 2024-09-13T18:36:11.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.588+0000 7fbd95d1a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900c8c70 con 0x7fbd90140da0 2024-09-13T18:36:11.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.588+0000 7fbd95d1a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900cd570 con 0x7fbd900cbd50 2024-09-13T18:36:11.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd94d18700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd90071390 0x7fbd900c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd94d18700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd90071390 0x7fbd900c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39110/0 (socket says 172.21.15.146:39110) 2024-09-13T18:36:11.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd94d18700 1 -- 172.21.15.146:0/1706209999 learned_addr learned my addr 172.21.15.146:0/1706209999 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:11.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd8ffff700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd900cbd50 0x7fbd90140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd95519700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd90140da0 0x7fbd9015ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd94d18700 1 -- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd900cbd50 msgr2=0x7fbd90140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:11.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd94d18700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd900cbd50 0x7fbd90140860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd94d18700 1 -- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd90140da0 msgr2=0x7fbd9015ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:11.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd94d18700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd90140da0 0x7fbd9015ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd94d18700 1 -- 172.21.15.146:0/1706209999 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd84009570 con 0x7fbd90071390 2024-09-13T18:36:11.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd8ffff700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd900cbd50 0x7fbd90140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:11.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd95519700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd90140da0 0x7fbd9015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:11.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd94d18700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd90071390 0x7fbd900c8730 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00d830 tx=0x7fbd7c00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:11.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd8d7fa700 1 -- 172.21.15.146:0/1706209999 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0125f0 con 0x7fbd90071390 2024-09-13T18:36:11.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd8d7fa700 1 -- 172.21.15.146:0/1706209999 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c012750 con 0x7fbd90071390 2024-09-13T18:36:11.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1706209999 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd901f36b0 con 0x7fbd90071390 2024-09-13T18:36:11.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd8d7fa700 1 -- 172.21.15.146:0/1706209999 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c012ac0 con 0x7fbd90071390 2024-09-13T18:36:11.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1706209999 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd900c6470 con 0x7fbd90071390 2024-09-13T18:36:11.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1706209999 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd900c62b0 con 0x7fbd90071390 2024-09-13T18:36:11.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.592+0000 7fbd8d7fa700 1 -- 172.21.15.146:0/1706209999 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbd7c012c20 con 0x7fbd90071390 2024-09-13T18:36:11.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.596+0000 7fbd8d7fa700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbd74066760 0x7fbd74068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.596+0000 7fbd8d7fa700 1 -- 172.21.15.146:0/1706209999 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7fbd7c084a50 con 0x7fbd90071390 2024-09-13T18:36:11.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.596+0000 7fbd8ffff700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbd74066760 0x7fbd74068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.596+0000 7fbd8ffff700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbd74066760 0x7fbd74068c20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009c30 tx=0x7fbd84008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:11.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.600+0000 7fbd8d7fa700 1 -- 172.21.15.146:0/1706209999 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbd7c051890 con 0x7fbd90071390 2024-09-13T18:36:11.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1706209999 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd900cb390 con 0x7fbd90071390 2024-09-13T18:36:11.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd8d7fa700 1 -- 172.21.15.146:0/1706209999 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+5535 (secure 0 0 0) 0x7fbd900cb390 con 0x7fbd90071390 2024-09-13T18:36:11.845 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:11.845 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:10.678571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:11.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbd74066760 msgr2=0x7fbd74068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:11.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd95d1a700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbd74066760 0x7fbd74068c20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009c30 tx=0x7fbd84008030 comp rx=0 tx=0).stop 2024-09-13T18:36:11.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1706209999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd90071390 msgr2=0x7fbd900c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:11.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd95d1a700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd90071390 0x7fbd900c8730 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00d830 tx=0x7fbd7c00dbf0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1706209999 shutdown_connections 2024-09-13T18:36:11.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd95d1a700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbd74066760 0x7fbd74068c20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd95d1a700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd90140da0 0x7fbd9015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd95d1a700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd900cbd50 0x7fbd90140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd95d1a700 1 --2- 172.21.15.146:0/1706209999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd90071390 0x7fbd900c8730 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1706209999 >> 172.21.15.146:0/1706209999 conn(0x7fbd90066270 msgr2=0x7fbd90066650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:11.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1706209999 shutdown_connections 2024-09-13T18:36:11.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.832+0000 7fbd95d1a700 1 -- 172.21.15.146:0/1706209999 wait complete. 2024-09-13T18:36:11.849 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 46 2024-09-13T18:36:11.861 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2024-09-13T18:36:11.861 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:36:11.861 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:11.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.956+0000 7fe1dbfff700 1 Processor -- start 2024-09-13T18:36:11.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.956+0000 7fe1dbfff700 1 -- start start 2024-09-13T18:36:11.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.956+0000 7fe1dbfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d4007c40 0x7fe1d40db340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.956+0000 7fe1dbfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d40332a0 con 0x7fe1d40d5860 2024-09-13T18:36:11.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.956+0000 7fe1dbfff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d4033560 con 0x7fe1d4045e90 2024-09-13T18:36:11.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.956+0000 7fe1dbfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d4045510 con 0x7fe1d4007c40 2024-09-13T18:36:11.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.956+0000 7fe1daffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fe1d40d5860 0x7fe1d4005e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56864/0 (socket says 172.21.15.146:56864) 2024-09-13T18:36:11.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.956+0000 7fe1daffd700 1 -- 172.21.15.146:0/3945922035 learned_addr learned my addr 172.21.15.146:0/3945922035 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:36:11.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.956+0000 7fe1da7fc700 1 --2- 172.21.15.146:0/3945922035 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d4007c40 0x7fe1d40db340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.956+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689457890 0 0) 0x7fe1d40332a0 con 0x7fe1d40d5860 2024-09-13T18:36:11.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.956+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1c40035d0 con 0x7fe1d40d5860 2024-09-13T18:36:11.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3338865666 0 0) 0x7fe1c40035d0 con 0x7fe1d40d5860 2024-09-13T18:36:11.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d4007c40 msgr2=0x7fe1d40db340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:11.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 --2- 172.21.15.146:0/3945922035 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d4007c40 0x7fe1d40db340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 >> v1:172.21.15.154:6789/0 conn(0x7fe1d4045e90 legacy=0x7fe1d4034ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:11.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d40457f0 con 0x7fe1d40d5860 2024-09-13T18:36:11.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7fe1cc001670 con 0x7fe1d40d5860 2024-09-13T18:36:11.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1da7fc700 1 --2- 172.21.15.146:0/3945922035 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d4007c40 0x7fe1d40db340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:11.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 >> v1:172.21.15.146:6789/0 conn(0x7fe1d40d5860 legacy=0x7fe1d4005e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:11.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 --2- 172.21.15.146:0/3945922035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4003a50 0x7fe1c4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 --2- 172.21.15.146:0/3945922035 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d4007c40 0x7fe1c4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 --2- 172.21.15.146:0/3945922035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c4004db0 0x7fe1c4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d4045510 con 0x7fe1c4003a50 2024-09-13T18:36:11.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c400bac0 con 0x7fe1c4004db0 2024-09-13T18:36:11.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c400bc30 con 0x7fe1d4007c40 2024-09-13T18:36:11.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1daffd700 1 --2- 172.21.15.146:0/3945922035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4003a50 0x7fe1c4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1da7fc700 1 --2- 172.21.15.146:0/3945922035 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d4007c40 0x7fe1c4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1db7fe700 1 --2- 172.21.15.146:0/3945922035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c4004db0 0x7fe1c4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1daffd700 1 -- 172.21.15.146:0/3945922035 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d4007c40 msgr2=0x7fe1c4004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:11.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1daffd700 1 --2- 172.21.15.146:0/3945922035 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d4007c40 0x7fe1c4004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1daffd700 1 -- 172.21.15.146:0/3945922035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c4004db0 msgr2=0x7fe1c4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:11.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1daffd700 1 --2- 172.21.15.146:0/3945922035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c4004db0 0x7fe1c4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1daffd700 1 -- 172.21.15.146:0/3945922035 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d00094b0 con 0x7fe1c4003a50 2024-09-13T18:36:11.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1daffd700 1 --2- 172.21.15.146:0/3945922035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4003a50 0x7fe1c4003e30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00e500 tx=0x7fe1c40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7fc84c1e7107f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:11.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc00eea0 con 0x7fe1c4003a50 2024-09-13T18:36:11.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc005b80 con 0x7fe1c4003a50 2024-09-13T18:36:11.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1d9ffb700 1 -- 172.21.15.146:0/3945922035 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc0070d0 con 0x7fe1c4003a50 2024-09-13T18:36:11.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1dbfff700 1 -- 172.21.15.146:0/3945922035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4003a50 msgr2=0x7fe1c4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:11.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1dbfff700 1 --2- 172.21.15.146:0/3945922035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4003a50 0x7fe1c4003e30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00e500 tx=0x7fe1c40058a0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1db7fe700 1 -- 172.21.15.146:0/3945922035 reap_dead start 2024-09-13T18:36:11.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1dbfff700 1 -- 172.21.15.146:0/3945922035 shutdown_connections 2024-09-13T18:36:11.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1dbfff700 1 -- 172.21.15.146:0/3945922035 >> 172.21.15.146:0/3945922035 conn(0x7fe1d4050590 msgr2=0x7fe1d40509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:11.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1dbfff700 1 -- 172.21.15.146:0/3945922035 shutdown_connections 2024-09-13T18:36:11.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.960+0000 7fe1dbfff700 1 -- 172.21.15.146:0/3945922035 wait complete. 2024-09-13T18:36:11.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1dbfff700 1 Processor -- start 2024-09-13T18:36:11.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1dbfff700 1 -- start start 2024-09-13T18:36:11.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1dbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4003a50 0x7fe1d404b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1dbfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d40d5c40 0x7fe1d4170410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1dbfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1d40dad50 0x7fe1d40cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1dbfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40ce790 con 0x7fe1c4003a50 2024-09-13T18:36:11.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1dbfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d416fce0 con 0x7fe1d40dad50 2024-09-13T18:36:11.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1dbfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40ccf90 con 0x7fe1d40d5c40 2024-09-13T18:36:11.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1daffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4003a50 0x7fe1d404b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1daffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4003a50 0x7fe1d404b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39126/0 (socket says 172.21.15.146:39126) 2024-09-13T18:36:11.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1daffd700 1 -- 172.21.15.146:0/2376145801 learned_addr learned my addr 172.21.15.146:0/2376145801 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:11.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1da7fc700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d40d5c40 0x7fe1d4170410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1db7fe700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1d40dad50 0x7fe1d40cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1daffd700 1 -- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d40d5c40 msgr2=0x7fe1d4170410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:11.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1daffd700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d40d5c40 0x7fe1d4170410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1daffd700 1 -- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1d40dad50 msgr2=0x7fe1d40cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:11.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1daffd700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1d40dad50 0x7fe1d40cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:11.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1daffd700 1 -- 172.21.15.146:0/2376145801 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4173fc0 con 0x7fe1c4003a50 2024-09-13T18:36:11.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1daffd700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4003a50 0x7fe1d404b510 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc005f20 tx=0x7fe1cc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:11.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1c37fe700 1 -- 172.21.15.146:0/2376145801 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc022070 con 0x7fe1c4003a50 2024-09-13T18:36:11.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1c37fe700 1 -- 172.21.15.146:0/2376145801 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc00eea0 con 0x7fe1c4003a50 2024-09-13T18:36:11.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1dbfff700 1 -- 172.21.15.146:0/2376145801 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d40cd1e0 con 0x7fe1c4003a50 2024-09-13T18:36:11.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1c37fe700 1 -- 172.21.15.146:0/2376145801 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc000ed0 con 0x7fe1c4003a50 2024-09-13T18:36:11.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.964+0000 7fe1dbfff700 1 -- 172.21.15.146:0/2376145801 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d416fef0 con 0x7fe1c4003a50 2024-09-13T18:36:11.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.968+0000 7fe1c37fe700 1 -- 172.21.15.146:0/2376145801 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe1cc0072e0 con 0x7fe1c4003a50 2024-09-13T18:36:11.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.968+0000 7fe1dbfff700 1 -- 172.21.15.146:0/2376145801 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d41700b0 con 0x7fe1c4003a50 2024-09-13T18:36:11.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.968+0000 7fe1c37fe700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe1c8066760 0x7fe1c8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:11.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.968+0000 7fe1c37fe700 1 -- 172.21.15.146:0/2376145801 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7fe1cc088230 con 0x7fe1c4003a50 2024-09-13T18:36:11.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.968+0000 7fe1da7fc700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe1c8066760 0x7fe1c8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:11.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.968+0000 7fe1da7fc700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe1c8066760 0x7fe1c8068c20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40ce740 tx=0x7fe1d0009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:11.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:11.972+0000 7fe1c37fe700 1 -- 172.21.15.146:0/2376145801 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe1cc055070 con 0x7fe1c4003a50 2024-09-13T18:36:12.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.208+0000 7fe1dbfff700 1 -- 172.21.15.146:0/2376145801 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1d40cd700 con 0x7fe1c4003a50 2024-09-13T18:36:12.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.208+0000 7fe1c37fe700 1 -- 172.21.15.146:0/2376145801 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+5535 (secure 0 0 0) 0x7fe1cc058690 con 0x7fe1c4003a50 2024-09-13T18:36:12.222 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:12.222 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:10.678571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:12.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.212+0000 7fe1dbfff700 1 -- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe1c8066760 msgr2=0x7fe1c8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:12.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.212+0000 7fe1dbfff700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe1c8066760 0x7fe1c8068c20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40ce740 tx=0x7fe1d0009240 comp rx=0 tx=0).stop 2024-09-13T18:36:12.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.212+0000 7fe1dbfff700 1 -- 172.21.15.146:0/2376145801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4003a50 msgr2=0x7fe1d404b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:12.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.212+0000 7fe1dbfff700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4003a50 0x7fe1d404b510 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc005f20 tx=0x7fe1cc001a00 comp rx=0 tx=0).stop 2024-09-13T18:36:12.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.212+0000 7fe1dbfff700 1 -- 172.21.15.146:0/2376145801 shutdown_connections 2024-09-13T18:36:12.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.212+0000 7fe1dbfff700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe1c8066760 0x7fe1c8068c20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:12.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.212+0000 7fe1dbfff700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1d40dad50 0x7fe1d40cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:12.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.212+0000 7fe1dbfff700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1d40d5c40 0x7fe1d4170410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:12.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.212+0000 7fe1dbfff700 1 --2- 172.21.15.146:0/2376145801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4003a50 0x7fe1d404b510 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:12.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.212+0000 7fe1dbfff700 1 -- 172.21.15.146:0/2376145801 >> 172.21.15.146:0/2376145801 conn(0x7fe1d4050590 msgr2=0x7fe1d4006610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:12.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.212+0000 7fe1dbfff700 1 -- 172.21.15.146:0/2376145801 shutdown_connections 2024-09-13T18:36:12.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:12.212+0000 7fe1dbfff700 1 -- 172.21.15.146:0/2376145801 wait complete. 2024-09-13T18:36:12.227 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 46 2024-09-13T18:36:12.242 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 46, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 46, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5505": { "addr": "172.21.15.146:6835/1652676664", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 1652676664, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 1652676664, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5505, "incarnation": 34, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 38 }, "gid_5649": { "addr": "172.21.15.146:6837/3926931671", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3926931671, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3926931671, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5649, "incarnation": 40, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 24 }, "gid_5784": { "addr": "172.21.15.146:6833/3901685195", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3901685195, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3901685195, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5784, "incarnation": 46, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 33, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:36:10.678571+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5505, "mds_1": 5784, "mds_2": 5649 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:36:14.244 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:14.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.352+0000 7ffaad2e4700 1 Processor -- start 2024-09-13T18:36:14.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.352+0000 7ffaad2e4700 1 -- start start 2024-09-13T18:36:14.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaad2e4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa80c5fc0 0x7ffaa80c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:14.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaad2e4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffaa80c63a0 0x7ffaa80c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:14.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaad2e4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffaa806f5c0 0x7ffaa8159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:14.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaad2e4700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa80c9bd0 con 0x7ffaa806f5c0 2024-09-13T18:36:14.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaad2e4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa80c8c70 con 0x7ffaa80c63a0 2024-09-13T18:36:14.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaad2e4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa80c8f50 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaacae3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffaa806f5c0 0x7ffaa8159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:14.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaacae3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffaa806f5c0 0x7ffaa8159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39136/0 (socket says 172.21.15.146:39136) 2024-09-13T18:36:14.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaacae3700 1 -- 172.21.15.146:0/3518208647 learned_addr learned my addr 172.21.15.146:0/3518208647 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:14.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaa77fe700 1 --2- 172.21.15.146:0/3518208647 >> v2:172.21.15.154:3300/0 conn(0x7ffaa80c63a0 0x7ffaa80c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:14.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaa7fff700 1 --2- 172.21.15.146:0/3518208647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa80c5fc0 0x7ffaa80c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:14.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaacae3700 1 -- 172.21.15.146:0/3518208647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa80c5fc0 msgr2=0x7ffaa80c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:14.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaacae3700 1 --2- 172.21.15.146:0/3518208647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa80c5fc0 0x7ffaa80c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:14.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaacae3700 1 -- 172.21.15.146:0/3518208647 >> v2:172.21.15.154:3300/0 conn(0x7ffaa80c63a0 msgr2=0x7ffaa80c7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:14.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaacae3700 1 --2- 172.21.15.146:0/3518208647 >> v2:172.21.15.154:3300/0 conn(0x7ffaa80c63a0 0x7ffaa80c7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:14.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaacae3700 1 -- 172.21.15.146:0/3518208647 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa80c9230 con 0x7ffaa806f5c0 2024-09-13T18:36:14.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaacae3700 1 --2- 172.21.15.146:0/3518208647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa806f5c0 0x7ffaa8159660 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80c8490 tx=0x7ffaa000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5eb4a5b5d4aae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:14.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaa6ffd700 1 -- 172.21.15.146:0/3518208647 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa000f070 con 0x7ffaa806f5c0 2024-09-13T18:36:14.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaa6ffd700 1 -- 172.21.15.146:0/3518208647 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa000a2e0 con 0x7ffaa806f5c0 2024-09-13T18:36:14.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaa6ffd700 1 -- 172.21.15.146:0/3518208647 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa000cc10 con 0x7ffaa806f5c0 2024-09-13T18:36:14.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaad2e4700 1 -- 172.21.15.146:0/3518208647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa806f5c0 msgr2=0x7ffaa8159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:14.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.356+0000 7ffaad2e4700 1 --2- 172.21.15.146:0/3518208647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa806f5c0 0x7ffaa8159660 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80c8490 tx=0x7ffaa000c330 comp rx=0 tx=0).stop 2024-09-13T18:36:14.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 -- 172.21.15.146:0/3518208647 shutdown_connections 2024-09-13T18:36:14.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 --2- 172.21.15.146:0/3518208647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa806f5c0 0x7ffaa8159660 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:14.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 --2- 172.21.15.146:0/3518208647 >> v2:172.21.15.154:3300/0 conn(0x7ffaa80c63a0 0x7ffaa80c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:14.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 --2- 172.21.15.146:0/3518208647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa80c5fc0 0x7ffaa80c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:14.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 -- 172.21.15.146:0/3518208647 >> 172.21.15.146:0/3518208647 conn(0x7ffaa8066090 msgr2=0x7ffaa8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:14.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 -- 172.21.15.146:0/3518208647 shutdown_connections 2024-09-13T18:36:14.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 -- 172.21.15.146:0/3518208647 wait complete. 2024-09-13T18:36:14.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 Processor -- start 2024-09-13T18:36:14.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 -- start start 2024-09-13T18:36:14.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa806f5c0 0x7ffaa80c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:14.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa80c5fc0 0x7ffaa814dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:14.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaa80c63a0 0x7ffaa80c1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:14.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa814c770 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa81eaae0 con 0x7ffaa80c63a0 2024-09-13T18:36:14.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaad2e4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa81ebab0 con 0x7ffaa806f5c0 2024-09-13T18:36:14.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaa77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa80c5fc0 0x7ffaa814dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:14.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaa77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa80c5fc0 0x7ffaa814dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39138/0 (socket says 172.21.15.146:39138) 2024-09-13T18:36:14.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaa77fe700 1 -- 172.21.15.146:0/1620440228 learned_addr learned my addr 172.21.15.146:0/1620440228 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:14.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaacae3700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaa80c63a0 0x7ffaa80c1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:14.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.360+0000 7ffaa7fff700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa806f5c0 0x7ffaa80c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:14.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaa77fe700 1 -- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa806f5c0 msgr2=0x7ffaa80c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:14.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaa77fe700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa806f5c0 0x7ffaa80c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:14.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaa77fe700 1 -- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaa80c63a0 msgr2=0x7ffaa80c1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:14.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaa77fe700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaa80c63a0 0x7ffaa80c1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:14.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaa77fe700 1 -- 172.21.15.146:0/1620440228 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa000ac40 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaa77fe700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa80c5fc0 0x7ffaa814dc40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009fd0 tx=0x7ffa9c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:14.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaa4ff9700 1 -- 172.21.15.146:0/1620440228 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0095a0 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaa4ff9700 1 -- 172.21.15.146:0/1620440228 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c013680 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaa4ff9700 1 -- 172.21.15.146:0/1620440228 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c013a10 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaad2e4700 1 -- 172.21.15.146:0/1620440228 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa814f9e0 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaad2e4700 1 -- 172.21.15.146:0/1620440228 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa81eafd0 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaad2e4700 1 -- 172.21.15.146:0/1620440228 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa80cb790 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaa4ff9700 1 -- 172.21.15.146:0/1620440228 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffa9c013b70 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaa4ff9700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffa94066730 0x7ffa94068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:14.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.364+0000 7ffaa4ff9700 1 -- 172.21.15.146:0/1620440228 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7ffa9c084510 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.368+0000 7ffaa7fff700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffa94066730 0x7ffa94068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:14.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.368+0000 7ffaa7fff700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffa94066730 0x7ffa94068bf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffaa81e8710 tx=0x7ffa98009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:14.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.372+0000 7ffaa4ff9700 1 -- 172.21.15.146:0/1620440228 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ffa9c0512c0 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.612+0000 7ffaad2e4700 1 -- 172.21.15.146:0/1620440228 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaa80a9540 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.612+0000 7ffaa4ff9700 1 -- 172.21.15.146:0/1620440228 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+5535 (secure 0 0 0) 0x7ffa9c0548e0 con 0x7ffaa80c5fc0 2024-09-13T18:36:14.626 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:14.626 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:10.678571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:14.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.612+0000 7ffaad2e4700 1 -- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffa94066730 msgr2=0x7ffa94068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:14.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.612+0000 7ffaad2e4700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffa94066730 0x7ffa94068bf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffaa81e8710 tx=0x7ffa98009240 comp rx=0 tx=0).stop 2024-09-13T18:36:14.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.612+0000 7ffaad2e4700 1 -- 172.21.15.146:0/1620440228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa80c5fc0 msgr2=0x7ffaa814dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:14.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.612+0000 7ffaad2e4700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa80c5fc0 0x7ffaa814dc40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009fd0 tx=0x7ffa9c00eb20 comp rx=0 tx=0).stop 2024-09-13T18:36:14.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.616+0000 7ffaad2e4700 1 -- 172.21.15.146:0/1620440228 shutdown_connections 2024-09-13T18:36:14.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.616+0000 7ffaad2e4700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffa94066730 0x7ffa94068bf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:14.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.616+0000 7ffaad2e4700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaa80c63a0 0x7ffaa80c1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:14.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.616+0000 7ffaad2e4700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa80c5fc0 0x7ffaa814dc40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:14.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.616+0000 7ffaad2e4700 1 --2- 172.21.15.146:0/1620440228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa806f5c0 0x7ffaa80c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:14.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.616+0000 7ffaad2e4700 1 -- 172.21.15.146:0/1620440228 >> 172.21.15.146:0/1620440228 conn(0x7ffaa8066090 msgr2=0x7ffaa81564d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:14.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.616+0000 7ffaad2e4700 1 -- 172.21.15.146:0/1620440228 shutdown_connections 2024-09-13T18:36:14.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:14.616+0000 7ffaad2e4700 1 -- 172.21.15.146:0/1620440228 wait complete. 2024-09-13T18:36:14.630 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 46 2024-09-13T18:36:14.644 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:36:16.645 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:16.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.760+0000 7efdf11f0700 1 Processor -- start 2024-09-13T18:36:16.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.760+0000 7efdf11f0700 1 -- start start 2024-09-13T18:36:16.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.760+0000 7efdf11f0700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efdec06e7c0 0x7efdec0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:16.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.760+0000 7efdf11f0700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdec0c1020 con 0x7efdec15a000 2024-09-13T18:36:16.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.760+0000 7efdf11f0700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdec1593f0 con 0x7efdec06eba0 2024-09-13T18:36:16.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.760+0000 7efdf11f0700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdec0c6a70 con 0x7efdec06e7c0 2024-09-13T18:36:16.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.760+0000 7efdebfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efdec06e7c0 0x7efdec0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:16.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.760+0000 7efdf09ef700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7efdec15a000 0x7efdec1531a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56872/0 (socket says 172.21.15.146:56872) 2024-09-13T18:36:16.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.760+0000 7efdf09ef700 1 -- 172.21.15.146:0/4111929526 learned_addr learned my addr 172.21.15.146:0/4111929526 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:36:16.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdebfff700 1 -- 172.21.15.146:0/4111929526 >> v1:172.21.15.154:6789/0 conn(0x7efdec06eba0 legacy=0x7efdec1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:16.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdebfff700 1 -- 172.21.15.146:0/4111929526 >> v1:172.21.15.146:6789/0 conn(0x7efdec15a000 legacy=0x7efdec1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:16.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdebfff700 1 -- 172.21.15.146:0/4111929526 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec0c1290 con 0x7efdec06e7c0 2024-09-13T18:36:16.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdebfff700 1 --2- 172.21.15.146:0/4111929526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdec06e7c0 0x7efdec0b11a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7efddc009850 tx=0x7efddc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356a7e036f44f19a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:16.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdeb7fe700 1 -- 172.21.15.146:0/4111929526 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc004230 con 0x7efdec06e7c0 2024-09-13T18:36:16.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdeb7fe700 1 -- 172.21.15.146:0/4111929526 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc004390 con 0x7efdec06e7c0 2024-09-13T18:36:16.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdeb7fe700 1 -- 172.21.15.146:0/4111929526 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc0036c0 con 0x7efdec06e7c0 2024-09-13T18:36:16.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdf11f0700 1 -- 172.21.15.146:0/4111929526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdec06e7c0 msgr2=0x7efdec0b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:16.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdf11f0700 1 --2- 172.21.15.146:0/4111929526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdec06e7c0 0x7efdec0b11a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7efddc009850 tx=0x7efddc009b60 comp rx=0 tx=0).stop 2024-09-13T18:36:16.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdf11f0700 1 -- 172.21.15.146:0/4111929526 shutdown_connections 2024-09-13T18:36:16.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdf11f0700 1 --2- 172.21.15.146:0/4111929526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdec06e7c0 0x7efdec0b11a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:16.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdf11f0700 1 -- 172.21.15.146:0/4111929526 >> 172.21.15.146:0/4111929526 conn(0x7efdec071ed0 msgr2=0x7efdec0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:16.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdf11f0700 1 -- 172.21.15.146:0/4111929526 shutdown_connections 2024-09-13T18:36:16.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdf11f0700 1 -- 172.21.15.146:0/4111929526 wait complete. 2024-09-13T18:36:16.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdf11f0700 1 Processor -- start 2024-09-13T18:36:16.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.764+0000 7efdf11f0700 1 -- start start 2024-09-13T18:36:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdf11f0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdec06e7c0 0x7efdec1e6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdf11f0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdec06eba0 0x7efdec071480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdf11f0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdec15a000 0x7efdec070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdf11f0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec14f040 con 0x7efdec06e7c0 2024-09-13T18:36:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdf11f0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0c8e40 con 0x7efdec15a000 2024-09-13T18:36:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdf11f0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec14f2a0 con 0x7efdec06eba0 2024-09-13T18:36:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdebfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdec06e7c0 0x7efdec1e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:16.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdebfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdec06e7c0 0x7efdec1e6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39154/0 (socket says 172.21.15.146:39154) 2024-09-13T18:36:16.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdebfff700 1 -- 172.21.15.146:0/2677515050 learned_addr learned my addr 172.21.15.146:0/2677515050 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:16.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdf09ef700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdec15a000 0x7efdec070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:16.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efde3fff700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdec06eba0 0x7efdec071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:16.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdebfff700 1 -- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdec06eba0 msgr2=0x7efdec071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:16.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdebfff700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdec06eba0 0x7efdec071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:16.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdebfff700 1 -- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdec15a000 msgr2=0x7efdec070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:16.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdebfff700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdec15a000 0x7efdec070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:16.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdebfff700 1 -- 172.21.15.146:0/2677515050 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc0094b0 con 0x7efdec06e7c0 2024-09-13T18:36:16.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efde3fff700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdec06eba0 0x7efdec071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:16.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdf09ef700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdec15a000 0x7efdec070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:16.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdebfff700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdec06e7c0 0x7efdec1e6220 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efddc0090b0 tx=0x7efddc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:16.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efde97fa700 1 -- 172.21.15.146:0/2677515050 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc011830 con 0x7efdec06e7c0 2024-09-13T18:36:16.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efde97fa700 1 -- 172.21.15.146:0/2677515050 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc0119b0 con 0x7efdec06e7c0 2024-09-13T18:36:16.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdf11f0700 1 -- 172.21.15.146:0/2677515050 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdec14eaf0 con 0x7efdec06e7c0 2024-09-13T18:36:16.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efde97fa700 1 -- 172.21.15.146:0/2677515050 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc011d60 con 0x7efdec06e7c0 2024-09-13T18:36:16.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.768+0000 7efdf11f0700 1 -- 172.21.15.146:0/2677515050 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec0c8ad0 con 0x7efdec06e7c0 2024-09-13T18:36:16.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.772+0000 7efdf11f0700 1 -- 172.21.15.146:0/2677515050 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdec0c8930 con 0x7efdec06e7c0 2024-09-13T18:36:16.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.772+0000 7efde97fa700 1 -- 172.21.15.146:0/2677515050 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efddc018050 con 0x7efdec06e7c0 2024-09-13T18:36:16.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.772+0000 7efde97fa700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efdcc066730 0x7efdcc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:16.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.772+0000 7efde97fa700 1 -- 172.21.15.146:0/2677515050 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7efddc0853a0 con 0x7efdec06e7c0 2024-09-13T18:36:16.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.772+0000 7efde3fff700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efdcc066730 0x7efdcc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:16.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.772+0000 7efde3fff700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efdcc066730 0x7efdcc068bf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efdec14e9e0 tx=0x7efdd4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:16.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.776+0000 7efde97fa700 1 -- 172.21.15.146:0/2677515050 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efdec0c8930 con 0x7efdec06e7c0 2024-09-13T18:36:17.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.996+0000 7efdf11f0700 1 -- 172.21.15.146:0/2677515050 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdec0c8930 con 0x7efdec06e7c0 2024-09-13T18:36:17.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:16.996+0000 7efde97fa700 1 -- 172.21.15.146:0/2677515050 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+5537 (secure 0 0 0) 0x7efdec0c8930 con 0x7efdec06e7c0 2024-09-13T18:36:17.011 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:17.011 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:16.699218+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:resolve","state_seq":23,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:17.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:17.000+0000 7efdf11f0700 1 -- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efdcc066730 msgr2=0x7efdcc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:17.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:17.000+0000 7efdf11f0700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efdcc066730 0x7efdcc068bf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efdec14e9e0 tx=0x7efdd4009040 comp rx=0 tx=0).stop 2024-09-13T18:36:17.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:17.000+0000 7efdf11f0700 1 -- 172.21.15.146:0/2677515050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdec06e7c0 msgr2=0x7efdec1e6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:17.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:17.000+0000 7efdf11f0700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdec06e7c0 0x7efdec1e6220 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efddc0090b0 tx=0x7efddc004770 comp rx=0 tx=0).stop 2024-09-13T18:36:17.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:17.000+0000 7efdf11f0700 1 -- 172.21.15.146:0/2677515050 shutdown_connections 2024-09-13T18:36:17.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:17.000+0000 7efdf11f0700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efdcc066730 0x7efdcc068bf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:17.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:17.000+0000 7efdf11f0700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdec15a000 0x7efdec070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:17.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:17.000+0000 7efdf11f0700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdec06eba0 0x7efdec071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:17.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:17.000+0000 7efdf11f0700 1 --2- 172.21.15.146:0/2677515050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efdec06e7c0 0x7efdec1e6220 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:17.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:17.000+0000 7efdf11f0700 1 -- 172.21.15.146:0/2677515050 >> 172.21.15.146:0/2677515050 conn(0x7efdec071ed0 msgr2=0x7efdec157050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:17.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:17.000+0000 7efdf11f0700 1 -- 172.21.15.146:0/2677515050 shutdown_connections 2024-09-13T18:36:17.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:17.000+0000 7efdf11f0700 1 -- 172.21.15.146:0/2677515050 wait complete. 2024-09-13T18:36:17.016 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 47 2024-09-13T18:36:17.025 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:36:19.026 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:19.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.140+0000 7f9563044700 1 Processor -- start 2024-09-13T18:36:19.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.140+0000 7f9563044700 1 -- start start 2024-09-13T18:36:19.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9563044700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f955c066300 0x7f955c156da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:19.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9563044700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c156330 con 0x7f955c06d220 2024-09-13T18:36:19.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9563044700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c0c48d0 con 0x7f955c1504f0 2024-09-13T18:36:19.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9563044700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f955c0c27e0 con 0x7f955c066300 2024-09-13T18:36:19.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9562843700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f955c06d220 0x7f955c160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56882/0 (socket says 172.21.15.146:56882) 2024-09-13T18:36:19.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9562843700 1 -- 172.21.15.146:0/4294920560 learned_addr learned my addr 172.21.15.146:0/4294920560 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:36:19.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9561841700 1 --2- 172.21.15.146:0/4294920560 >> v2:172.21.15.154:3300/0 conn(0x7f955c066300 0x7f955c156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:19.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9561040700 1 -- 172.21.15.146:0/4294920560 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745840611 0 0) 0x7f955c156330 con 0x7f955c06d220 2024-09-13T18:36:19.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9561040700 1 -- 172.21.15.146:0/4294920560 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95480035d0 con 0x7f955c06d220 2024-09-13T18:36:19.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9561040700 1 -- 172.21.15.146:0/4294920560 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3702365650 0 0) 0x7f95480035d0 con 0x7f955c06d220 2024-09-13T18:36:19.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9561841700 1 -- 172.21.15.146:0/4294920560 >> v1:172.21.15.154:6789/0 conn(0x7f955c1504f0 legacy=0x7f955c0c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:19.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9561841700 1 -- 172.21.15.146:0/4294920560 >> v1:172.21.15.146:6789/0 conn(0x7f955c06d220 legacy=0x7f955c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:19.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9561841700 1 -- 172.21.15.146:0/4294920560 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c0c4bb0 con 0x7f955c066300 2024-09-13T18:36:19.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9561841700 1 --2- 172.21.15.146:0/4294920560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f955c066300 0x7f955c156da0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f954c009850 tx=0x7f954c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=172568f1d5848559 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:19.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9561040700 1 -- 172.21.15.146:0/4294920560 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95540019f0 con 0x7f955c066300 2024-09-13T18:36:19.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9561040700 1 -- 172.21.15.146:0/4294920560 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c004030 con 0x7f955c066300 2024-09-13T18:36:19.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.144+0000 7f9561040700 1 -- 172.21.15.146:0/4294920560 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c0043c0 con 0x7f955c066300 2024-09-13T18:36:19.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 -- 172.21.15.146:0/4294920560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f955c066300 msgr2=0x7f955c156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:19.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 --2- 172.21.15.146:0/4294920560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f955c066300 0x7f955c156da0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f954c009850 tx=0x7f954c009b60 comp rx=0 tx=0).stop 2024-09-13T18:36:19.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 -- 172.21.15.146:0/4294920560 shutdown_connections 2024-09-13T18:36:19.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 --2- 172.21.15.146:0/4294920560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f955c066300 0x7f955c156da0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:19.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 -- 172.21.15.146:0/4294920560 >> 172.21.15.146:0/4294920560 conn(0x7f955c066b00 msgr2=0x7f955c06aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:19.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 -- 172.21.15.146:0/4294920560 shutdown_connections 2024-09-13T18:36:19.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 -- 172.21.15.146:0/4294920560 wait complete. 2024-09-13T18:36:19.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 Processor -- start 2024-09-13T18:36:19.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 -- start start 2024-09-13T18:36:19.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f955c066300 0x7f955c152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:19.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f955c06d220 0x7f955c148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:19.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f955c1504f0 0x7f955c146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:19.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c151950 con 0x7f955c06d220 2024-09-13T18:36:19.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c145c20 con 0x7f955c1504f0 2024-09-13T18:36:19.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9563044700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c153190 con 0x7f955c066300 2024-09-13T18:36:19.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9561841700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f955c06d220 0x7f955c148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:19.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9561841700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f955c06d220 0x7f955c148310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39156/0 (socket says 172.21.15.146:39156) 2024-09-13T18:36:19.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9561841700 1 -- 172.21.15.146:0/2591567614 learned_addr learned my addr 172.21.15.146:0/2591567614 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:19.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9562042700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f955c066300 0x7f955c152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:19.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.148+0000 7f9562843700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f955c1504f0 0x7f955c146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:19.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9561841700 1 -- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f955c066300 msgr2=0x7f955c152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:19.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9561841700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f955c066300 0x7f955c152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:19.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9561841700 1 -- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f955c1504f0 msgr2=0x7f955c146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:19.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9561841700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f955c1504f0 0x7f955c146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:19.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9561841700 1 -- 172.21.15.146:0/2591567614 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c0094b0 con 0x7f955c06d220 2024-09-13T18:36:19.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9561841700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f955c06d220 0x7f955c148310 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f955c070790 tx=0x7f954c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:19.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9552ffd700 1 -- 172.21.15.146:0/2591567614 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c011630 con 0x7f955c06d220 2024-09-13T18:36:19.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9552ffd700 1 -- 172.21.15.146:0/2591567614 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c01f070 con 0x7f955c06d220 2024-09-13T18:36:19.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9563044700 1 -- 172.21.15.146:0/2591567614 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955c153470 con 0x7f955c06d220 2024-09-13T18:36:19.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9552ffd700 1 -- 172.21.15.146:0/2591567614 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c0119d0 con 0x7f955c06d220 2024-09-13T18:36:19.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9563044700 1 -- 172.21.15.146:0/2591567614 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c146150 con 0x7f955c06d220 2024-09-13T18:36:19.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9563044700 1 -- 172.21.15.146:0/2591567614 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955c1508d0 con 0x7f955c06d220 2024-09-13T18:36:19.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9552ffd700 1 -- 172.21.15.146:0/2591567614 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f954c011b30 con 0x7f955c06d220 2024-09-13T18:36:19.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9552ffd700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9540066730 0x7f9540068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:19.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9552ffd700 1 -- 172.21.15.146:0/2591567614 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f954c021070 con 0x7f955c06d220 2024-09-13T18:36:19.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.152+0000 7f9562042700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9540066730 0x7f9540068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:19.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.156+0000 7f9562042700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9540066730 0x7f9540068bf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9558000f60 tx=0x7f9558009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:19.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.160+0000 7f9552ffd700 1 -- 172.21.15.146:0/2591567614 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f954c052dd0 con 0x7f955c06d220 2024-09-13T18:36:19.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.388+0000 7f9563044700 1 -- 172.21.15.146:0/2591567614 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f955c148d50 con 0x7f955c06d220 2024-09-13T18:36:19.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.388+0000 7f9552ffd700 1 -- 172.21.15.146:0/2591567614 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+5537 (secure 0 0 0) 0x7f954c02b020 con 0x7f955c06d220 2024-09-13T18:36:19.402 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:19.402 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:16.699218+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:resolve","state_seq":23,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:19.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.388+0000 7f9563044700 1 -- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9540066730 msgr2=0x7f9540068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:19.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.388+0000 7f9563044700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9540066730 0x7f9540068bf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9558000f60 tx=0x7f9558009040 comp rx=0 tx=0).stop 2024-09-13T18:36:19.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.388+0000 7f9563044700 1 -- 172.21.15.146:0/2591567614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f955c06d220 msgr2=0x7f955c148310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:19.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.388+0000 7f9563044700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f955c06d220 0x7f955c148310 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f955c070790 tx=0x7f954c004770 comp rx=0 tx=0).stop 2024-09-13T18:36:19.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.392+0000 7f9563044700 1 -- 172.21.15.146:0/2591567614 shutdown_connections 2024-09-13T18:36:19.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.392+0000 7f9563044700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9540066730 0x7f9540068bf0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:19.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.392+0000 7f9563044700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f955c1504f0 0x7f955c146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:19.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.392+0000 7f9563044700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f955c06d220 0x7f955c148310 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:19.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.392+0000 7f9563044700 1 --2- 172.21.15.146:0/2591567614 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f955c066300 0x7f955c152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:19.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.392+0000 7f9563044700 1 -- 172.21.15.146:0/2591567614 >> 172.21.15.146:0/2591567614 conn(0x7f955c066b00 msgr2=0x7f955c14b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:19.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.392+0000 7f9563044700 1 -- 172.21.15.146:0/2591567614 shutdown_connections 2024-09-13T18:36:19.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:19.392+0000 7f9563044700 1 -- 172.21.15.146:0/2591567614 wait complete. 2024-09-13T18:36:19.406 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 47 2024-09-13T18:36:19.417 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:36:21.420 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:21.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.544+0000 7faf3ffff700 1 Processor -- start 2024-09-13T18:36:21.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.544+0000 7faf3ffff700 1 -- start start 2024-09-13T18:36:21.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.544+0000 7faf3ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf380dc3d0 0x7faf3804cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.544+0000 7faf3ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faf3804d240 0x7faf380480c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.544+0000 7faf3ffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf38047c30 con 0x7faf380d6670 2024-09-13T18:36:21.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.544+0000 7faf3ffff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf38035230 con 0x7faf3804d240 2024-09-13T18:36:21.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.544+0000 7faf3ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf38035a50 con 0x7faf380dc3d0 2024-09-13T18:36:21.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.544+0000 7faf3effd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7faf380d6670 0x7faf38046df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:33582/0 (socket says 172.21.15.146:33582) 2024-09-13T18:36:21.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.544+0000 7faf3effd700 1 -- 172.21.15.146:0/1006652280 learned_addr learned my addr 172.21.15.146:0/1006652280 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:36:21.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.544+0000 7faf3e7fc700 1 --2- 172.21.15.146:0/1006652280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf380dc3d0 0x7faf3804cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.544+0000 7faf3f7fe700 1 --2- 172.21.15.146:0/1006652280 >> v2:172.21.15.154:3300/0 conn(0x7faf3804d240 0x7faf380480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.544+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423655418 0 0) 0x7faf38047c30 con 0x7faf380d6670 2024-09-13T18:36:21.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.544+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf280035d0 con 0x7faf380d6670 2024-09-13T18:36:21.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3535512528 0 0) 0x7faf280035d0 con 0x7faf380d6670 2024-09-13T18:36:21.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf380dc3d0 msgr2=0x7faf3804cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 --2- 172.21.15.146:0/1006652280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf380dc3d0 0x7faf3804cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 >> v2:172.21.15.154:3300/0 conn(0x7faf3804d240 msgr2=0x7faf380480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 --2- 172.21.15.146:0/1006652280 >> v2:172.21.15.154:3300/0 conn(0x7faf3804d240 0x7faf380480c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380dc7d0 con 0x7faf380d6670 2024-09-13T18:36:21.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7faf30001670 con 0x7faf380d6670 2024-09-13T18:36:21.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 >> v1:172.21.15.146:6789/0 conn(0x7faf380d6670 legacy=0x7faf38046df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 --2- 172.21.15.146:0/1006652280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf380dc3d0 0x7faf28003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 --2- 172.21.15.146:0/1006652280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf28003fd0 0x7faf28004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 --2- 172.21.15.146:0/1006652280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf28004db0 0x7faf28005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf38035230 con 0x7faf28004db0 2024-09-13T18:36:21.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2800baf0 con 0x7faf28003fd0 2024-09-13T18:36:21.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2800bcb0 con 0x7faf380dc3d0 2024-09-13T18:36:21.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3f7fe700 1 --2- 172.21.15.146:0/1006652280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf28004db0 0x7faf28005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3effd700 1 --2- 172.21.15.146:0/1006652280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf380dc3d0 0x7faf28003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3e7fc700 1 --2- 172.21.15.146:0/1006652280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf28003fd0 0x7faf28004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3f7fe700 1 -- 172.21.15.146:0/1006652280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf380dc3d0 msgr2=0x7faf28003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3f7fe700 1 --2- 172.21.15.146:0/1006652280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf380dc3d0 0x7faf28003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3f7fe700 1 -- 172.21.15.146:0/1006652280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf28003fd0 msgr2=0x7faf28004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:21.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3f7fe700 1 --2- 172.21.15.146:0/1006652280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf28003fd0 0x7faf28004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3f7fe700 1 -- 172.21.15.146:0/1006652280 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf280035d0 con 0x7faf28004db0 2024-09-13T18:36:21.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3e7fc700 1 --2- 172.21.15.146:0/1006652280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf28003fd0 0x7faf28004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:36:21.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3f7fe700 1 --2- 172.21.15.146:0/1006652280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf28004db0 0x7faf28005230 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7faf4004f480 tx=0x7faf280058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61081b173260957 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40090070 con 0x7faf28004db0 2024-09-13T18:36:21.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4006f230 con 0x7faf28004db0 2024-09-13T18:36:21.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3dffb700 1 -- 172.21.15.146:0/1006652280 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40073900 con 0x7faf28004db0 2024-09-13T18:36:21.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3ffff700 1 -- 172.21.15.146:0/1006652280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf28004db0 msgr2=0x7faf28005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3ffff700 1 --2- 172.21.15.146:0/1006652280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf28004db0 0x7faf28005230 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7faf4004f480 tx=0x7faf280058a0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3f7fe700 1 -- 172.21.15.146:0/1006652280 reap_dead start 2024-09-13T18:36:21.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3ffff700 1 -- 172.21.15.146:0/1006652280 shutdown_connections 2024-09-13T18:36:21.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3ffff700 1 -- 172.21.15.146:0/1006652280 >> 172.21.15.146:0/1006652280 conn(0x7faf38009a30 msgr2=0x7faf38009e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3ffff700 1 -- 172.21.15.146:0/1006652280 shutdown_connections 2024-09-13T18:36:21.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3ffff700 1 -- 172.21.15.146:0/1006652280 wait complete. 2024-09-13T18:36:21.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.548+0000 7faf3ffff700 1 Processor -- start 2024-09-13T18:36:21.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3ffff700 1 -- start start 2024-09-13T18:36:21.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf28004db0 0x7faf380d7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf380c8e00 0x7faf380cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf38172720 0x7faf38052470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf38176370 con 0x7faf28004db0 2024-09-13T18:36:21.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3ffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf38172300 con 0x7faf38172720 2024-09-13T18:36:21.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf38171f60 con 0x7faf380c8e00 2024-09-13T18:36:21.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf28004db0 0x7faf380d7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf28004db0 0x7faf380d7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53614/0 (socket says 172.21.15.146:53614) 2024-09-13T18:36:21.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3effd700 1 -- 172.21.15.146:0/72337929 learned_addr learned my addr 172.21.15.146:0/72337929 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:21.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3e7fc700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf380c8e00 0x7faf380cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3f7fe700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf38172720 0x7faf38052470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3effd700 1 -- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf380c8e00 msgr2=0x7faf380cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3effd700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf380c8e00 0x7faf380cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3effd700 1 -- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf38172720 msgr2=0x7faf38052470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3effd700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf38172720 0x7faf38052470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3effd700 1 -- 172.21.15.146:0/72337929 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf38176090 con 0x7faf28004db0 2024-09-13T18:36:21.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3f7fe700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf38172720 0x7faf38052470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:21.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3effd700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf28004db0 0x7faf380d7530 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7faf3000eb20 tx=0x7faf3000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf277fe700 1 -- 172.21.15.146:0/72337929 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3000c7e0 con 0x7faf28004db0 2024-09-13T18:36:21.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf277fe700 1 -- 172.21.15.146:0/72337929 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3000c940 con 0x7faf28004db0 2024-09-13T18:36:21.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf277fe700 1 -- 172.21.15.146:0/72337929 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3000ccf0 con 0x7faf28004db0 2024-09-13T18:36:21.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3ffff700 1 -- 172.21.15.146:0/72337929 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf380d3440 con 0x7faf28004db0 2024-09-13T18:36:21.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3ffff700 1 -- 172.21.15.146:0/72337929 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf380d3720 con 0x7faf28004db0 2024-09-13T18:36:21.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.552+0000 7faf3ffff700 1 -- 172.21.15.146:0/72337929 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf38172440 con 0x7faf28004db0 2024-09-13T18:36:21.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.556+0000 7faf277fe700 1 -- 172.21.15.146:0/72337929 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faf3001d460 con 0x7faf28004db0 2024-09-13T18:36:21.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.556+0000 7faf277fe700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faf2c066760 0x7faf2c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:21.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.556+0000 7faf277fe700 1 -- 172.21.15.146:0/72337929 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7faf30084500 con 0x7faf28004db0 2024-09-13T18:36:21.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.556+0000 7faf3e7fc700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faf2c066760 0x7faf2c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:21.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.556+0000 7faf3e7fc700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faf2c066760 0x7faf2c068c20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faf380c8d90 tx=0x7faf34008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:21.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.564+0000 7faf277fe700 1 -- 172.21.15.146:0/72337929 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faf38172440 con 0x7faf28004db0 2024-09-13T18:36:21.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.800+0000 7faf3ffff700 1 -- 172.21.15.146:0/72337929 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf38172440 con 0x7faf28004db0 2024-09-13T18:36:21.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.800+0000 7faf277fe700 1 -- 172.21.15.146:0/72337929 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+5539 (secure 0 0 0) 0x7faf38172440 con 0x7faf28004db0 2024-09-13T18:36:21.815 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:21.815 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:21.729330+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:reconnect","state_seq":25,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:21.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.804+0000 7faf3ffff700 1 -- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faf2c066760 msgr2=0x7faf2c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.804+0000 7faf3ffff700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faf2c066760 0x7faf2c068c20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faf380c8d90 tx=0x7faf34008040 comp rx=0 tx=0).stop 2024-09-13T18:36:21.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.804+0000 7faf3ffff700 1 -- 172.21.15.146:0/72337929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf28004db0 msgr2=0x7faf380d7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:21.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.804+0000 7faf3ffff700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf28004db0 0x7faf380d7530 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7faf3000eb20 tx=0x7faf3000eee0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.804+0000 7faf3ffff700 1 -- 172.21.15.146:0/72337929 shutdown_connections 2024-09-13T18:36:21.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.804+0000 7faf3ffff700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faf2c066760 0x7faf2c068c20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.804+0000 7faf3ffff700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf38172720 0x7faf38052470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.804+0000 7faf3ffff700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf380c8e00 0x7faf380cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.804+0000 7faf3ffff700 1 --2- 172.21.15.146:0/72337929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf28004db0 0x7faf380d7530 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:21.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.804+0000 7faf3ffff700 1 -- 172.21.15.146:0/72337929 >> 172.21.15.146:0/72337929 conn(0x7faf38009a30 msgr2=0x7faf3804dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:21.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.804+0000 7faf3ffff700 1 -- 172.21.15.146:0/72337929 shutdown_connections 2024-09-13T18:36:21.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:21.804+0000 7faf3ffff700 1 -- 172.21.15.146:0/72337929 wait complete. 2024-09-13T18:36:21.819 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 48 2024-09-13T18:36:21.831 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:36:23.833 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:23.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb13a1c3700 1 Processor -- start 2024-09-13T18:36:23.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb13a1c3700 1 -- start start 2024-09-13T18:36:23.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb13a1c3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb134153bc0 0x7fb1340c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb13a1c3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb1340c6610 0x7fb1340cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb13a1c3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1340c6a80 0x7fb1340cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb13a1c3700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb134153fa0 con 0x7fb134153bc0 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb13a1c3700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1340aa500 con 0x7fb1340c6610 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb13a1c3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13415a7e0 con 0x7fb1340c6a80 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb1391c1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb134153bc0 0x7fb1340c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb1391c1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb134153bc0 0x7fb1340c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53620/0 (socket says 172.21.15.146:53620) 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb1391c1700 1 -- 172.21.15.146:0/1555415862 learned_addr learned my addr 172.21.15.146:0/1555415862 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb1389c0700 1 --2- 172.21.15.146:0/1555415862 >> v2:172.21.15.154:3300/0 conn(0x7fb1340c6610 0x7fb1340cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb1391c1700 1 -- 172.21.15.146:0/1555415862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1340c6a80 msgr2=0x7fb1340cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:23.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb1391c1700 1 --2- 172.21.15.146:0/1555415862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1340c6a80 0x7fb1340cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb1391c1700 1 -- 172.21.15.146:0/1555415862 >> v2:172.21.15.154:3300/0 conn(0x7fb1340c6610 msgr2=0x7fb1340cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb1391c1700 1 --2- 172.21.15.146:0/1555415862 >> v2:172.21.15.154:3300/0 conn(0x7fb1340c6610 0x7fb1340cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb1391c1700 1 -- 172.21.15.146:0/1555415862 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1340c9320 con 0x7fb134153bc0 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.936+0000 7fb1391c1700 1 --2- 172.21.15.146:0/1555415862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb134153bc0 0x7fb1340c8de0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb13406f3c0 tx=0x7fb130009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc483dac6474ee47 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb12bfff700 1 -- 172.21.15.146:0/1555415862 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13000b8a0 con 0x7fb134153bc0 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb12bfff700 1 -- 172.21.15.146:0/1555415862 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13000ba00 con 0x7fb134153bc0 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb12bfff700 1 -- 172.21.15.146:0/1555415862 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13000bd90 con 0x7fb134153bc0 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 -- 172.21.15.146:0/1555415862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb134153bc0 msgr2=0x7fb1340c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 --2- 172.21.15.146:0/1555415862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb134153bc0 0x7fb1340c8de0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb13406f3c0 tx=0x7fb130009bf0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 -- 172.21.15.146:0/1555415862 shutdown_connections 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 --2- 172.21.15.146:0/1555415862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1340c6a80 0x7fb1340cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 --2- 172.21.15.146:0/1555415862 >> v2:172.21.15.154:3300/0 conn(0x7fb1340c6610 0x7fb1340cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 --2- 172.21.15.146:0/1555415862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb134153bc0 0x7fb1340c8de0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 -- 172.21.15.146:0/1555415862 >> 172.21.15.146:0/1555415862 conn(0x7fb134066090 msgr2=0x7fb134066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 -- 172.21.15.146:0/1555415862 shutdown_connections 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 -- 172.21.15.146:0/1555415862 wait complete. 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 Processor -- start 2024-09-13T18:36:23.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 -- start start 2024-09-13T18:36:23.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1340c6610 0x7fb1340c3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1340c6a80 0x7fb134150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb134153bc0 0x7fb1341ed5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134155310 con 0x7fb1340c6610 2024-09-13T18:36:23.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340c3740 con 0x7fb134153bc0 2024-09-13T18:36:23.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.940+0000 7fb13a1c3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340c39e0 con 0x7fb1340c6a80 2024-09-13T18:36:23.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb1391c1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1340c6610 0x7fb1340c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb1391c1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1340c6610 0x7fb1340c3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53628/0 (socket says 172.21.15.146:53628) 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb1391c1700 1 -- 172.21.15.146:0/2190741289 learned_addr learned my addr 172.21.15.146:0/2190741289 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb1399c2700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb134153bc0 0x7fb1341ed5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb1389c0700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1340c6a80 0x7fb134150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb1391c1700 1 -- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1340c6a80 msgr2=0x7fb134150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb1391c1700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1340c6a80 0x7fb134150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb1391c1700 1 -- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb134153bc0 msgr2=0x7fb1341ed5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb1391c1700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb134153bc0 0x7fb1341ed5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb1391c1700 1 -- 172.21.15.146:0/2190741289 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb130009570 con 0x7fb1340c6610 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb1391c1700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1340c6610 0x7fb1340c3110 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb130000c00 tx=0x7fb130004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb129ffb700 1 -- 172.21.15.146:0/2190741289 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1300038c0 con 0x7fb1340c6610 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb129ffb700 1 -- 172.21.15.146:0/2190741289 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13001e070 con 0x7fb1340c6610 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb13a1c3700 1 -- 172.21.15.146:0/2190741289 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1340c50b0 con 0x7fb1340c6610 2024-09-13T18:36:23.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb129ffb700 1 -- 172.21.15.146:0/2190741289 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130003c40 con 0x7fb1340c6610 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb13a1c3700 1 -- 172.21.15.146:0/2190741289 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1341ecde0 con 0x7fb1340c6610 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb13a1c3700 1 -- 172.21.15.146:0/2190741289 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1340c86c0 con 0x7fb1340c6610 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb129ffb700 1 -- 172.21.15.146:0/2190741289 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb130011420 con 0x7fb1340c6610 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb129ffb700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb118066760 0x7fb118068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.944+0000 7fb129ffb700 1 -- 172.21.15.146:0/2190741289 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7fb130020070 con 0x7fb1340c6610 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.948+0000 7fb1389c0700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb118066760 0x7fb118068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:23.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.948+0000 7fb1389c0700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb118066760 0x7fb118068c20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb1340bff60 tx=0x7fb124008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:23.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:23.952+0000 7fb129ffb700 1 -- 172.21.15.146:0/2190741289 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb130052c30 con 0x7fb1340c6610 2024-09-13T18:36:24.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.188+0000 7fb13a1c3700 1 -- 172.21.15.146:0/2190741289 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1340669a0 con 0x7fb1340c6610 2024-09-13T18:36:24.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.188+0000 7fb129ffb700 1 -- 172.21.15.146:0/2190741289 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+5539 (secure 0 0 0) 0x7fb130015090 con 0x7fb1340c6610 2024-09-13T18:36:24.201 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:24.201 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:21.729330+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:reconnect","state_seq":25,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:24.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.192+0000 7fb13a1c3700 1 -- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb118066760 msgr2=0x7fb118068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.192+0000 7fb13a1c3700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb118066760 0x7fb118068c20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb1340bff60 tx=0x7fb124008040 comp rx=0 tx=0).stop 2024-09-13T18:36:24.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.192+0000 7fb13a1c3700 1 -- 172.21.15.146:0/2190741289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1340c6610 msgr2=0x7fb1340c3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:24.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.192+0000 7fb13a1c3700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1340c6610 0x7fb1340c3110 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb130000c00 tx=0x7fb130004770 comp rx=0 tx=0).stop 2024-09-13T18:36:24.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.192+0000 7fb13a1c3700 1 -- 172.21.15.146:0/2190741289 shutdown_connections 2024-09-13T18:36:24.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.192+0000 7fb13a1c3700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb118066760 0x7fb118068c20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.192+0000 7fb13a1c3700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb134153bc0 0x7fb1341ed5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.192+0000 7fb13a1c3700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1340c6a80 0x7fb134150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.192+0000 7fb13a1c3700 1 --2- 172.21.15.146:0/2190741289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1340c6610 0x7fb1340c3110 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:24.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.192+0000 7fb13a1c3700 1 -- 172.21.15.146:0/2190741289 >> 172.21.15.146:0/2190741289 conn(0x7fb134066090 msgr2=0x7fb1340afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:24.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.192+0000 7fb13a1c3700 1 -- 172.21.15.146:0/2190741289 shutdown_connections 2024-09-13T18:36:24.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:24.192+0000 7fb13a1c3700 1 -- 172.21.15.146:0/2190741289 wait complete. 2024-09-13T18:36:24.207 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 48 2024-09-13T18:36:24.221 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 48, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 48, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5505": { "addr": "172.21.15.146:6835/1652676664", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 1652676664, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 1652676664, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5505, "incarnation": 34, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 38 }, "gid_5649": { "addr": "172.21.15.146:6837/3926931671", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3926931671, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3926931671, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5649, "incarnation": 40, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 24 }, "gid_5784": { "addr": "172.21.15.146:6833/3901685195", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3901685195, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3901685195, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5784, "incarnation": 46, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:reconnect", "state_seq": 25 } }, "last_failure": 0, "last_failure_osd_epoch": 33, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:36:21.729330+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5505, "mds_1": 5784, "mds_2": 5649 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:36:26.222 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:26.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.320+0000 7f1ebd87f700 1 Processor -- start 2024-09-13T18:36:26.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.320+0000 7f1ebd87f700 1 -- start start 2024-09-13T18:36:26.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.320+0000 7f1ebd87f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1eb80c6770 0x7f1eb8159190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.320+0000 7f1ebd87f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb8156810 con 0x7f1eb80c7b00 2024-09-13T18:36:26.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.320+0000 7f1ebd87f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb81559b0 con 0x7f1eb814fb50 2024-09-13T18:36:26.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.320+0000 7f1ebd87f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb8155c90 con 0x7f1eb80c6770 2024-09-13T18:36:26.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.320+0000 7f1eb7fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1eb80c7b00 0x7f1eb80c6060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:33590/0 (socket says 172.21.15.146:33590) 2024-09-13T18:36:26.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.320+0000 7f1eb7fff700 1 -- 172.21.15.146:0/1483729002 learned_addr learned my addr 172.21.15.146:0/1483729002 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:36:26.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.320+0000 7f1ebd07e700 1 --2- 172.21.15.146:0/1483729002 >> v2:172.21.15.146:3300/0 conn(0x7f1eb80c6770 0x7f1eb8159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1eb77fe700 1 -- 172.21.15.146:0/1483729002 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317579431 0 0) 0x7f1eb8156810 con 0x7f1eb80c7b00 2024-09-13T18:36:26.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1eb77fe700 1 -- 172.21.15.146:0/1483729002 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea80035d0 con 0x7f1eb80c7b00 2024-09-13T18:36:26.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1ebd07e700 1 -- 172.21.15.146:0/1483729002 >> v1:172.21.15.154:6789/0 conn(0x7f1eb814fb50 legacy=0x7f1eb80c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1ebd07e700 1 -- 172.21.15.146:0/1483729002 >> v1:172.21.15.146:6789/0 conn(0x7f1eb80c7b00 legacy=0x7f1eb80c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1ebd07e700 1 -- 172.21.15.146:0/1483729002 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb8155f70 con 0x7f1eb80c6770 2024-09-13T18:36:26.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1ebd07e700 1 --2- 172.21.15.146:0/1483729002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eb80c6770 0x7f1eb8159190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000aed0 tx=0x7f1eb000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce88b2afa70f363 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:26.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1eb77fe700 1 -- 172.21.15.146:0/1483729002 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000f070 con 0x7f1eb80c6770 2024-09-13T18:36:26.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1eb77fe700 1 -- 172.21.15.146:0/1483729002 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb000a2e0 con 0x7f1eb80c6770 2024-09-13T18:36:26.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1eb77fe700 1 -- 172.21.15.146:0/1483729002 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000cbd0 con 0x7f1eb80c6770 2024-09-13T18:36:26.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1ebd87f700 1 -- 172.21.15.146:0/1483729002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eb80c6770 msgr2=0x7f1eb8159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1ebd87f700 1 --2- 172.21.15.146:0/1483729002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eb80c6770 0x7f1eb8159190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000aed0 tx=0x7f1eb000c330 comp rx=0 tx=0).stop 2024-09-13T18:36:26.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1ebd87f700 1 -- 172.21.15.146:0/1483729002 shutdown_connections 2024-09-13T18:36:26.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1ebd87f700 1 --2- 172.21.15.146:0/1483729002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eb80c6770 0x7f1eb8159190 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1ebd87f700 1 -- 172.21.15.146:0/1483729002 >> 172.21.15.146:0/1483729002 conn(0x7f1eb8066b00 msgr2=0x7f1eb806aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:26.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1ebd87f700 1 -- 172.21.15.146:0/1483729002 shutdown_connections 2024-09-13T18:36:26.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1ebd87f700 1 -- 172.21.15.146:0/1483729002 wait complete. 2024-09-13T18:36:26.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1ebd87f700 1 Processor -- start 2024-09-13T18:36:26.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.324+0000 7f1ebd87f700 1 -- start start 2024-09-13T18:36:26.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd87f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1eb80c6770 0x7f1eb8140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd87f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1eb80c7b00 0x7f1eb81f0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd87f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eb814fb50 0x7f1eb81f0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd87f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80ca2e0 con 0x7f1eb814fb50 2024-09-13T18:36:26.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd87f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb8153ad0 con 0x7f1eb80c6770 2024-09-13T18:36:26.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd87f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80c9e50 con 0x7f1eb80c7b00 2024-09-13T18:36:26.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd07e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eb814fb50 0x7f1eb81f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd07e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eb814fb50 0x7f1eb81f0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53642/0 (socket says 172.21.15.146:53642) 2024-09-13T18:36:26.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd07e700 1 -- 172.21.15.146:0/2732125913 learned_addr learned my addr 172.21.15.146:0/2732125913 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:26.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebc87d700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1eb80c6770 0x7f1eb8140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1eb7fff700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1eb80c7b00 0x7f1eb81f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd07e700 1 -- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1eb80c7b00 msgr2=0x7f1eb81f0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd07e700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1eb80c7b00 0x7f1eb81f0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd07e700 1 -- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1eb80c6770 msgr2=0x7f1eb8140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd07e700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1eb80c6770 0x7f1eb8140d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd07e700 1 -- 172.21.15.146:0/2732125913 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb000ab80 con 0x7f1eb814fb50 2024-09-13T18:36:26.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd07e700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eb814fb50 0x7f1eb81f0570 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000cbc0 tx=0x7f1eb0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:26.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1eb57fa700 1 -- 172.21.15.146:0/2732125913 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000aa00 con 0x7f1eb814fb50 2024-09-13T18:36:26.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1eb57fa700 1 -- 172.21.15.146:0/2732125913 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0004250 con 0x7f1eb814fb50 2024-09-13T18:36:26.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1eb57fa700 1 -- 172.21.15.146:0/2732125913 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00045c0 con 0x7f1eb814fb50 2024-09-13T18:36:26.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd87f700 1 -- 172.21.15.146:0/2732125913 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb81537f0 con 0x7f1eb814fb50 2024-09-13T18:36:26.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1ebd87f700 1 -- 172.21.15.146:0/2732125913 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb80c9800 con 0x7f1eb814fb50 2024-09-13T18:36:26.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.328+0000 7f1eb57fa700 1 -- 172.21.15.146:0/2732125913 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1eb0004720 con 0x7f1eb814fb50 2024-09-13T18:36:26.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.332+0000 7f1ebd87f700 1 -- 172.21.15.146:0/2732125913 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb814d810 con 0x7f1eb814fb50 2024-09-13T18:36:26.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.332+0000 7f1eb57fa700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1e9c066760 0x7f1e9c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:26.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.332+0000 7f1eb57fa700 1 -- 172.21.15.146:0/2732125913 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f1eb0085ac0 con 0x7f1eb814fb50 2024-09-13T18:36:26.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.332+0000 7f1ebc87d700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1e9c066760 0x7f1e9c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:26.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.332+0000 7f1ebc87d700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1e9c066760 0x7f1e9c068c20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1eac000f60 tx=0x7f1eac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:26.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.336+0000 7f1eb57fa700 1 -- 172.21.15.146:0/2732125913 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1eb0052930 con 0x7f1eb814fb50 2024-09-13T18:36:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.568+0000 7f1ebd87f700 1 -- 172.21.15.146:0/2732125913 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1eb80c9f90 con 0x7f1eb814fb50 2024-09-13T18:36:26.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.568+0000 7f1eb57fa700 1 -- 172.21.15.146:0/2732125913 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+5539 (secure 0 0 0) 0x7f1eb0055f50 con 0x7f1eb814fb50 2024-09-13T18:36:26.582 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:26.583 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:21.729330+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:reconnect","state_seq":25,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:26.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.572+0000 7f1ebd87f700 1 -- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1e9c066760 msgr2=0x7f1e9c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.572+0000 7f1ebd87f700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1e9c066760 0x7f1e9c068c20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1eac000f60 tx=0x7f1eac009040 comp rx=0 tx=0).stop 2024-09-13T18:36:26.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.572+0000 7f1ebd87f700 1 -- 172.21.15.146:0/2732125913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eb814fb50 msgr2=0x7f1eb81f0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:26.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.572+0000 7f1ebd87f700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eb814fb50 0x7f1eb81f0570 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000cbc0 tx=0x7f1eb0004af0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.572+0000 7f1ebd87f700 1 -- 172.21.15.146:0/2732125913 shutdown_connections 2024-09-13T18:36:26.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.572+0000 7f1ebd87f700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1e9c066760 0x7f1e9c068c20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.572+0000 7f1ebd87f700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eb814fb50 0x7f1eb81f0570 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.572+0000 7f1ebd87f700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1eb80c7b00 0x7f1eb81f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.572+0000 7f1ebd87f700 1 --2- 172.21.15.146:0/2732125913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1eb80c6770 0x7f1eb8140d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:26.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.572+0000 7f1ebd87f700 1 -- 172.21.15.146:0/2732125913 >> 172.21.15.146:0/2732125913 conn(0x7f1eb8066b00 msgr2=0x7f1eb806aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.572+0000 7f1ebd87f700 1 -- 172.21.15.146:0/2732125913 shutdown_connections 2024-09-13T18:36:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:26.572+0000 7f1ebd87f700 1 -- 172.21.15.146:0/2732125913 wait complete. 2024-09-13T18:36:26.587 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 48 2024-09-13T18:36:26.598 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:36:28.600 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:28.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.712+0000 7f1058800700 1 Processor -- start 2024-09-13T18:36:28.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.712+0000 7f1058800700 1 -- start start 2024-09-13T18:36:28.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1058800700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1054151360 0x7f10540cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:28.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1058800700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10540c78d0 0x7f10540cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:28.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1058800700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1054071ac0 con 0x7f10540c6960 2024-09-13T18:36:28.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1058800700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105406f760 con 0x7f1054151360 2024-09-13T18:36:28.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1058800700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105406fa40 con 0x7f10540c78d0 2024-09-13T18:36:28.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f10537fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1054151360 0x7f10540cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:28.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f10537fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1054151360 0x7f10540cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53646/0 (socket says 172.21.15.146:53646) 2024-09-13T18:36:28.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f10537fe700 1 -- 172.21.15.146:0/723013311 learned_addr learned my addr 172.21.15.146:0/723013311 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:28.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1052ffd700 1 --2- 172.21.15.146:0/723013311 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10540c78d0 0x7f10540cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:28.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f10537fe700 1 -- 172.21.15.146:0/723013311 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10540c78d0 msgr2=0x7f10540cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:28.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f10537fe700 1 --2- 172.21.15.146:0/723013311 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10540c78d0 0x7f10540cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f10537fe700 1 -- 172.21.15.146:0/723013311 >> v1:172.21.15.146:6789/0 conn(0x7f10540c6960 legacy=0x7f10541574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:28.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f10537fe700 1 -- 172.21.15.146:0/723013311 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1054071470 con 0x7f1054151360 2024-09-13T18:36:28.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f10527fc700 1 -- 172.21.15.146:0/723013311 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815942111 0 0) 0x7f1054071ac0 con 0x7f10540c6960 2024-09-13T18:36:28.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f10537fe700 1 --2- 172.21.15.146:0/723013311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1054151360 0x7f10540cc260 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1044009910 tx=0x7f1044009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853922173ef7f3d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:28.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f10527fc700 1 -- 172.21.15.146:0/723013311 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044011830 con 0x7f1054151360 2024-09-13T18:36:28.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f10527fc700 1 -- 172.21.15.146:0/723013311 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1044011990 con 0x7f1054151360 2024-09-13T18:36:28.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f10527fc700 1 -- 172.21.15.146:0/723013311 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044011d40 con 0x7f1054151360 2024-09-13T18:36:28.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1058800700 1 -- 172.21.15.146:0/723013311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1054151360 msgr2=0x7f10540cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:28.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1058800700 1 --2- 172.21.15.146:0/723013311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1054151360 0x7f10540cc260 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1044009910 tx=0x7f1044009c20 comp rx=0 tx=0).stop 2024-09-13T18:36:28.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1058800700 1 -- 172.21.15.146:0/723013311 shutdown_connections 2024-09-13T18:36:28.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1058800700 1 --2- 172.21.15.146:0/723013311 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10540c78d0 0x7f10540cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1058800700 1 --2- 172.21.15.146:0/723013311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1054151360 0x7f10540cc260 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1058800700 1 -- 172.21.15.146:0/723013311 >> 172.21.15.146:0/723013311 conn(0x7f1054071ed0 msgr2=0x7f10540722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:28.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1058800700 1 -- 172.21.15.146:0/723013311 shutdown_connections 2024-09-13T18:36:28.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.716+0000 7f1058800700 1 -- 172.21.15.146:0/723013311 wait complete. 2024-09-13T18:36:28.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f1058800700 1 Processor -- start 2024-09-13T18:36:28.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f1058800700 1 -- start start 2024-09-13T18:36:28.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f1058800700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10540c6960 0x7f105414c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:28.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f1058800700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10540c78d0 0x7f1054155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:28.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f1058800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1054151360 0x7f1054155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:28.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f1058800700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10540c2900 con 0x7f10540c6960 2024-09-13T18:36:28.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f1058800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1054154d90 con 0x7f1054151360 2024-09-13T18:36:28.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f1058800700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1054155070 con 0x7f10540c78d0 2024-09-13T18:36:28.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f10537fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10540c6960 0x7f105414c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:28.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f10537fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10540c6960 0x7f105414c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53654/0 (socket says 172.21.15.146:53654) 2024-09-13T18:36:28.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f10537fe700 1 -- 172.21.15.146:0/2691655020 learned_addr learned my addr 172.21.15.146:0/2691655020 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:28.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f1052ffd700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10540c78d0 0x7f1054155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:28.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f1053fff700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1054151360 0x7f1054155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:28.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f10537fe700 1 -- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10540c78d0 msgr2=0x7f1054155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:28.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f10537fe700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10540c78d0 0x7f1054155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f10537fe700 1 -- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1054151360 msgr2=0x7f1054155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:28.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f10537fe700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1054151360 0x7f1054155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f10537fe700 1 -- 172.21.15.146:0/2691655020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1044009570 con 0x7f10540c6960 2024-09-13T18:36:28.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f10537fe700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10540c6960 0x7f105414c8a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1044014040 tx=0x7f1044004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:28.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f103bfff700 1 -- 172.21.15.146:0/2691655020 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104400ba70 con 0x7f10540c6960 2024-09-13T18:36:28.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f103bfff700 1 -- 172.21.15.146:0/2691655020 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104401e070 con 0x7f10540c6960 2024-09-13T18:36:28.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f1058800700 1 -- 172.21.15.146:0/2691655020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1054151f10 con 0x7f10540c6960 2024-09-13T18:36:28.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f103bfff700 1 -- 172.21.15.146:0/2691655020 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104400be50 con 0x7f10540c6960 2024-09-13T18:36:28.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.720+0000 7f1058800700 1 -- 172.21.15.146:0/2691655020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10540c2340 con 0x7f10540c6960 2024-09-13T18:36:28.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.724+0000 7f1058800700 1 -- 172.21.15.146:0/2691655020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105406f0d0 con 0x7f10540c6960 2024-09-13T18:36:28.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.724+0000 7f103bfff700 1 -- 172.21.15.146:0/2691655020 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f104401a420 con 0x7f10540c6960 2024-09-13T18:36:28.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.724+0000 7f103bfff700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1040066730 0x7f1040068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:28.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.724+0000 7f103bfff700 1 -- 172.21.15.146:0/2691655020 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f1044085770 con 0x7f10540c6960 2024-09-13T18:36:28.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.724+0000 7f1052ffd700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1040066730 0x7f1040068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:28.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.724+0000 7f1052ffd700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1040066730 0x7f1040068bf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1054147ad0 tx=0x7f1048008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:28.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.732+0000 7f103bfff700 1 -- 172.21.15.146:0/2691655020 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1044052520 con 0x7f10540c6960 2024-09-13T18:36:28.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.948+0000 7f1058800700 1 -- 172.21.15.146:0/2691655020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f105406e2c0 con 0x7f10540c6960 2024-09-13T18:36:28.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.948+0000 7f103bfff700 1 -- 172.21.15.146:0/2691655020 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+5536 (secure 0 0 0) 0x7f1044055b40 con 0x7f10540c6960 2024-09-13T18:36:28.963 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:28.964 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:26.747288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:rejoin","state_seq":27,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:28.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.952+0000 7f1058800700 1 -- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1040066730 msgr2=0x7f1040068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:28.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.952+0000 7f1058800700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1040066730 0x7f1040068bf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1054147ad0 tx=0x7f1048008040 comp rx=0 tx=0).stop 2024-09-13T18:36:28.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.952+0000 7f1058800700 1 -- 172.21.15.146:0/2691655020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10540c6960 msgr2=0x7f105414c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:28.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.952+0000 7f1058800700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10540c6960 0x7f105414c8a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1044014040 tx=0x7f1044004770 comp rx=0 tx=0).stop 2024-09-13T18:36:28.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.952+0000 7f1058800700 1 -- 172.21.15.146:0/2691655020 shutdown_connections 2024-09-13T18:36:28.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.952+0000 7f1058800700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1040066730 0x7f1040068bf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.952+0000 7f1058800700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1054151360 0x7f1054155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.952+0000 7f1058800700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10540c78d0 0x7f1054155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.952+0000 7f1058800700 1 --2- 172.21.15.146:0/2691655020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10540c6960 0x7f105414c8a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:28.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.952+0000 7f1058800700 1 -- 172.21.15.146:0/2691655020 >> 172.21.15.146:0/2691655020 conn(0x7f1054071ed0 msgr2=0x7f10540c7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:28.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.952+0000 7f1058800700 1 -- 172.21.15.146:0/2691655020 shutdown_connections 2024-09-13T18:36:28.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:28.952+0000 7f1058800700 1 -- 172.21.15.146:0/2691655020 wait complete. 2024-09-13T18:36:28.967 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 49 2024-09-13T18:36:28.982 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:36:30.983 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:31.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8af4ae700 1 Processor -- start 2024-09-13T18:36:31.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8af4ae700 1 -- start start 2024-09-13T18:36:31.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8af4ae700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff8a80cf790 0x7ff8a8157980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:31.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8af4ae700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8a80cfb70 0x7ff8a8157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:31.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8af4ae700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a81541b0 con 0x7ff8a80c8ba0 2024-09-13T18:36:31.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8af4ae700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a815cb00 con 0x7ff8a80cf790 2024-09-13T18:36:31.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8af4ae700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a815bba0 con 0x7ff8a80cfb70 2024-09-13T18:36:31.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8ada4b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff8a80c8ba0 0x7ff8a80c9030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57350/0 (socket says 172.21.15.146:57350) 2024-09-13T18:36:31.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8ada4b700 1 -- 172.21.15.146:0/152359435 learned_addr learned my addr 172.21.15.146:0/152359435 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:36:31.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8aca49700 1 --2- 172.21.15.146:0/152359435 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8a80cfb70 0x7ff8a8157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:31.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8ad24a700 1 --2- 172.21.15.146:0/152359435 >> v2:172.21.15.154:3300/0 conn(0x7ff8a80cf790 0x7ff8a8157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:31.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff89ffff700 1 -- 172.21.15.146:0/152359435 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494301839 0 0) 0x7ff8a81541b0 con 0x7ff8a80c8ba0 2024-09-13T18:36:31.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff89ffff700 1 -- 172.21.15.146:0/152359435 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8940035d0 con 0x7ff8a80c8ba0 2024-09-13T18:36:31.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff89ffff700 1 -- 172.21.15.146:0/152359435 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4263241686 0 0) 0x7ff8940035d0 con 0x7ff8a80c8ba0 2024-09-13T18:36:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8aca49700 1 -- 172.21.15.146:0/152359435 >> v2:172.21.15.154:3300/0 conn(0x7ff8a80cf790 msgr2=0x7ff8a8157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8aca49700 1 --2- 172.21.15.146:0/152359435 >> v2:172.21.15.154:3300/0 conn(0x7ff8a80cf790 0x7ff8a8157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8aca49700 1 -- 172.21.15.146:0/152359435 >> v1:172.21.15.146:6789/0 conn(0x7ff8a80c8ba0 legacy=0x7ff8a80c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8aca49700 1 -- 172.21.15.146:0/152359435 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a815be80 con 0x7ff8a80cfb70 2024-09-13T18:36:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8ad24a700 1 --2- 172.21.15.146:0/152359435 >> v2:172.21.15.154:3300/0 conn(0x7ff8a80cf790 0x7ff8a8157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:36:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff8aca49700 1 --2- 172.21.15.146:0/152359435 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8a80cfb70 0x7ff8a8157ec0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff898009850 tx=0x7ff898009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64ab0cdd21087e63 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.080+0000 7ff89ffff700 1 -- 172.21.15.146:0/152359435 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a00019f0 con 0x7ff8a80cfb70 2024-09-13T18:36:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff89ffff700 1 -- 172.21.15.146:0/152359435 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff898004030 con 0x7ff8a80cfb70 2024-09-13T18:36:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff89ffff700 1 -- 172.21.15.146:0/152359435 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8980043e0 con 0x7ff8a80cfb70 2024-09-13T18:36:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 -- 172.21.15.146:0/152359435 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8a80cfb70 msgr2=0x7ff8a8157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 --2- 172.21.15.146:0/152359435 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8a80cfb70 0x7ff8a8157ec0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff898009850 tx=0x7ff898009b60 comp rx=0 tx=0).stop 2024-09-13T18:36:31.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 -- 172.21.15.146:0/152359435 shutdown_connections 2024-09-13T18:36:31.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 --2- 172.21.15.146:0/152359435 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8a80cfb70 0x7ff8a8157ec0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:31.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 --2- 172.21.15.146:0/152359435 >> v2:172.21.15.154:3300/0 conn(0x7ff8a80cf790 0x7ff8a8157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:31.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 -- 172.21.15.146:0/152359435 >> 172.21.15.146:0/152359435 conn(0x7ff8a8071d90 msgr2=0x7ff8a80721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:31.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 -- 172.21.15.146:0/152359435 shutdown_connections 2024-09-13T18:36:31.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 -- 172.21.15.146:0/152359435 wait complete. 2024-09-13T18:36:31.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 Processor -- start 2024-09-13T18:36:31.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 -- start start 2024-09-13T18:36:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8a80c8ba0 0x7ff8a80c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a80cf790 0x7ff8a81f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a80cfb70 0x7ff8a8156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a80c2de0 con 0x7ff8a80cf790 2024-09-13T18:36:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a80c30c0 con 0x7ff8a80cfb70 2024-09-13T18:36:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8af4ae700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a8155e30 con 0x7ff8a80c8ba0 2024-09-13T18:36:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8aca49700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a80cf790 0x7ff8a81f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8ad24a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8a80c8ba0 0x7ff8a80c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8aca49700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a80cf790 0x7ff8a81f27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40824/0 (socket says 172.21.15.146:40824) 2024-09-13T18:36:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.084+0000 7ff8aca49700 1 -- 172.21.15.146:0/1601725864 learned_addr learned my addr 172.21.15.146:0/1601725864 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff8ada4b700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a80cfb70 0x7ff8a8156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff8aca49700 1 -- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8a80c8ba0 msgr2=0x7ff8a80c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:31.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff8aca49700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8a80c8ba0 0x7ff8a80c33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff8aca49700 1 -- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a80cfb70 msgr2=0x7ff8a8156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff8aca49700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a80cfb70 0x7ff8a8156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff8aca49700 1 -- 172.21.15.146:0/1601725864 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a40094b0 con 0x7ff8a80cf790 2024-09-13T18:36:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff8ada4b700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a80cfb70 0x7ff8a8156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:36:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff8aca49700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a80cf790 0x7ff8a81f27c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff898000c00 tx=0x7ff898004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff89dffb700 1 -- 172.21.15.146:0/1601725864 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898003e40 con 0x7ff8a80cf790 2024-09-13T18:36:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff89dffb700 1 -- 172.21.15.146:0/1601725864 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89801f070 con 0x7ff8a80cf790 2024-09-13T18:36:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff8af4ae700 1 -- 172.21.15.146:0/1601725864 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8980094b0 con 0x7ff8a80cf790 2024-09-13T18:36:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff89dffb700 1 -- 172.21.15.146:0/1601725864 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898011870 con 0x7ff8a80cf790 2024-09-13T18:36:31.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff8af4ae700 1 -- 172.21.15.146:0/1601725864 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a81f3230 con 0x7ff8a80cf790 2024-09-13T18:36:31.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff8af4ae700 1 -- 172.21.15.146:0/1601725864 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a80c9030 con 0x7ff8a80cf790 2024-09-13T18:36:31.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff89dffb700 1 -- 172.21.15.146:0/1601725864 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff8980119d0 con 0x7ff8a80cf790 2024-09-13T18:36:31.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff89dffb700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff88c066730 0x7ff88c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:31.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff89dffb700 1 -- 172.21.15.146:0/1601725864 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7ff898021070 con 0x7ff8a80cf790 2024-09-13T18:36:31.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.088+0000 7ff8ad24a700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff88c066730 0x7ff88c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:31.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.092+0000 7ff8ad24a700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff88c066730 0x7ff88c068bf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009d80 tx=0x7ff8a4009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:31.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.096+0000 7ff89dffb700 1 -- 172.21.15.146:0/1601725864 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff898052b20 con 0x7ff8a80cf790 2024-09-13T18:36:31.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.328+0000 7ff8af4ae700 1 -- 172.21.15.146:0/1601725864 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8a80c2f20 con 0x7ff8a80cf790 2024-09-13T18:36:31.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.328+0000 7ff89dffb700 1 -- 172.21.15.146:0/1601725864 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+5536 (secure 0 0 0) 0x7ff8a80c2f20 con 0x7ff8a80cf790 2024-09-13T18:36:31.344 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:31.344 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:26.747288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:rejoin","state_seq":27,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:31.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.332+0000 7ff8af4ae700 1 -- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff88c066730 msgr2=0x7ff88c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:31.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.332+0000 7ff8af4ae700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff88c066730 0x7ff88c068bf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009d80 tx=0x7ff8a4009240 comp rx=0 tx=0).stop 2024-09-13T18:36:31.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.332+0000 7ff8af4ae700 1 -- 172.21.15.146:0/1601725864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a80cf790 msgr2=0x7ff8a81f27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:31.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.332+0000 7ff8af4ae700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a80cf790 0x7ff8a81f27c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff898000c00 tx=0x7ff898004770 comp rx=0 tx=0).stop 2024-09-13T18:36:31.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.332+0000 7ff8af4ae700 1 -- 172.21.15.146:0/1601725864 shutdown_connections 2024-09-13T18:36:31.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.332+0000 7ff8af4ae700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff88c066730 0x7ff88c068bf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:31.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.332+0000 7ff8af4ae700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a80cfb70 0x7ff8a8156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:31.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.332+0000 7ff8af4ae700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a80cf790 0x7ff8a81f27c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:31.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.332+0000 7ff8af4ae700 1 --2- 172.21.15.146:0/1601725864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8a80c8ba0 0x7ff8a80c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:31.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.332+0000 7ff8af4ae700 1 -- 172.21.15.146:0/1601725864 >> 172.21.15.146:0/1601725864 conn(0x7ff8a8071d90 msgr2=0x7ff8a80cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:31.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.332+0000 7ff8af4ae700 1 -- 172.21.15.146:0/1601725864 shutdown_connections 2024-09-13T18:36:31.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:31.332+0000 7ff8af4ae700 1 -- 172.21.15.146:0/1601725864 wait complete. 2024-09-13T18:36:31.348 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 49 2024-09-13T18:36:31.359 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:36:33.359 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:33.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd34ca08700 1 Processor -- start 2024-09-13T18:36:33.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd34ca08700 1 -- start start 2024-09-13T18:36:33.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd34ca08700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd340048620 0x7fd3400dd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:33.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd34ca08700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd34004c670 0x7fd3400ddb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:33.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd34ca08700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3400514c0 con 0x7fd340048a00 2024-09-13T18:36:33.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd34ca08700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3400dd680 con 0x7fd34004c670 2024-09-13T18:36:33.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd34ca08700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34004c1e0 con 0x7fd340048620 2024-09-13T18:36:33.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd34759e700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd340048a00 0x7fd3400d63e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57352/0 (socket says 172.21.15.146:57352) 2024-09-13T18:36:33.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd34759e700 1 -- 172.21.15.146:0/4105915902 learned_addr learned my addr 172.21.15.146:0/4105915902 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:36:33.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd346d9d700 1 --2- 172.21.15.146:0/4105915902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd340048620 0x7fd3400dd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:33.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd34659c700 1 --2- 172.21.15.146:0/4105915902 >> v2:172.21.15.154:3300/0 conn(0x7fd34004c670 0x7fd3400ddb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:33.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374080057 0 0) 0x7fd3400514c0 con 0x7fd340048a00 2024-09-13T18:36:33.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3300035d0 con 0x7fd340048a00 2024-09-13T18:36:33.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1975336527 0 0) 0x7fd3300035d0 con 0x7fd340048a00 2024-09-13T18:36:33.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd340048620 msgr2=0x7fd3400dd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:33.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd345d9b700 1 --2- 172.21.15.146:0/4105915902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd340048620 0x7fd3400dd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 >> v2:172.21.15.154:3300/0 conn(0x7fd34004c670 msgr2=0x7fd3400ddb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:33.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd345d9b700 1 --2- 172.21.15.146:0/4105915902 >> v2:172.21.15.154:3300/0 conn(0x7fd34004c670 0x7fd3400ddb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34004b2f0 con 0x7fd340048a00 2024-09-13T18:36:33.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7fd348074a30 con 0x7fd340048a00 2024-09-13T18:36:33.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 >> v1:172.21.15.146:6789/0 conn(0x7fd340048a00 legacy=0x7fd3400d63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:33.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.464+0000 7fd345d9b700 1 --2- 172.21.15.146:0/4105915902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd330003a20 0x7fd330003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:33.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd345d9b700 1 --2- 172.21.15.146:0/4105915902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd340048620 0x7fd330004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:33.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd345d9b700 1 --2- 172.21.15.146:0/4105915902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd330004db0 0x7fd330005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:33.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400dd680 con 0x7fd330004db0 2024-09-13T18:36:33.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34004c1e0 con 0x7fd330003a20 2024-09-13T18:36:33.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33000fe40 con 0x7fd340048620 2024-09-13T18:36:33.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34759e700 1 --2- 172.21.15.146:0/4105915902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd330004db0 0x7fd330005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:33.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd346d9d700 1 --2- 172.21.15.146:0/4105915902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd330003a20 0x7fd330003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:33.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34659c700 1 --2- 172.21.15.146:0/4105915902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd340048620 0x7fd330004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:33.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34759e700 1 -- 172.21.15.146:0/4105915902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd340048620 msgr2=0x7fd330004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:33.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34759e700 1 --2- 172.21.15.146:0/4105915902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd340048620 0x7fd330004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34759e700 1 -- 172.21.15.146:0/4105915902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd330003a20 msgr2=0x7fd330003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:33.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34759e700 1 --2- 172.21.15.146:0/4105915902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd330003a20 0x7fd330003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34759e700 1 -- 172.21.15.146:0/4105915902 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3300035d0 con 0x7fd330004db0 2024-09-13T18:36:33.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34759e700 1 --2- 172.21.15.146:0/4105915902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd330004db0 0x7fd330005230 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd34804f480 tx=0x7fd3300058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8427e962a0319bc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:33.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34806f270 con 0x7fd330004db0 2024-09-13T18:36:33.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd348067540 con 0x7fd330004db0 2024-09-13T18:36:33.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd345d9b700 1 -- 172.21.15.146:0/4105915902 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348067910 con 0x7fd330004db0 2024-09-13T18:36:33.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34ca08700 1 -- 172.21.15.146:0/4105915902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd330004db0 msgr2=0x7fd330005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:33.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34ca08700 1 --2- 172.21.15.146:0/4105915902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd330004db0 0x7fd330005230 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd34804f480 tx=0x7fd3300058a0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34759e700 1 -- 172.21.15.146:0/4105915902 reap_dead start 2024-09-13T18:36:33.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34ca08700 1 -- 172.21.15.146:0/4105915902 shutdown_connections 2024-09-13T18:36:33.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34ca08700 1 -- 172.21.15.146:0/4105915902 >> 172.21.15.146:0/4105915902 conn(0x7fd340050610 msgr2=0x7fd3400509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:33.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34ca08700 1 -- 172.21.15.146:0/4105915902 shutdown_connections 2024-09-13T18:36:33.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.468+0000 7fd34ca08700 1 -- 172.21.15.146:0/4105915902 wait complete. 2024-09-13T18:36:33.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.472+0000 7fd34ca08700 1 Processor -- start 2024-09-13T18:36:33.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.472+0000 7fd34ca08700 1 -- start start 2024-09-13T18:36:33.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34ca08700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3400ce6b0 0x7fd3400cff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:33.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34ca08700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3400cea90 0x7fd340173250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:33.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34ca08700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3400362c0 0x7fd3400da830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:33.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34ca08700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd340033570 con 0x7fd3400362c0 2024-09-13T18:36:33.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34ca08700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd340035b30 con 0x7fd3400cea90 2024-09-13T18:36:33.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34ca08700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400dba30 con 0x7fd3400ce6b0 2024-09-13T18:36:33.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3400362c0 0x7fd3400da830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:33.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3400362c0 0x7fd3400da830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40830/0 (socket says 172.21.15.146:40830) 2024-09-13T18:36:33.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34759e700 1 -- 172.21.15.146:0/3800196743 learned_addr learned my addr 172.21.15.146:0/3800196743 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:33.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34659c700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3400cea90 0x7fd340173250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:33.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd346d9d700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3400ce6b0 0x7fd3400cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:33.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34759e700 1 -- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3400ce6b0 msgr2=0x7fd3400cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:33.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34759e700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3400ce6b0 0x7fd3400cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34759e700 1 -- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3400cea90 msgr2=0x7fd340173250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:33.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34759e700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3400cea90 0x7fd340173250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34759e700 1 -- 172.21.15.146:0/3800196743 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400337e0 con 0x7fd3400362c0 2024-09-13T18:36:33.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd346d9d700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3400ce6b0 0x7fd3400cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:36:33.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34759e700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3400362c0 0x7fd3400da830 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd348072fe0 tx=0x7fd348068890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:33.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd32f7fe700 1 -- 172.21.15.146:0/3800196743 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348074a30 con 0x7fd3400362c0 2024-09-13T18:36:33.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd32f7fe700 1 -- 172.21.15.146:0/3800196743 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3480721b0 con 0x7fd3400362c0 2024-09-13T18:36:33.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34ca08700 1 -- 172.21.15.146:0/3800196743 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd340033ac0 con 0x7fd3400362c0 2024-09-13T18:36:33.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd32f7fe700 1 -- 172.21.15.146:0/3800196743 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3480725a0 con 0x7fd3400362c0 2024-09-13T18:36:33.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.476+0000 7fd34ca08700 1 -- 172.21.15.146:0/3800196743 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd340173790 con 0x7fd3400362c0 2024-09-13T18:36:33.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.480+0000 7fd34ca08700 1 -- 172.21.15.146:0/3800196743 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34004af90 con 0x7fd3400362c0 2024-09-13T18:36:33.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.480+0000 7fd32f7fe700 1 -- 172.21.15.146:0/3800196743 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd348072700 con 0x7fd3400362c0 2024-09-13T18:36:33.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.480+0000 7fd32f7fe700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd334066730 0x7fd334068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:33.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.480+0000 7fd32f7fe700 1 -- 172.21.15.146:0/3800196743 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7fd3480ff940 con 0x7fd3400362c0 2024-09-13T18:36:33.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.480+0000 7fd346d9d700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd334066730 0x7fd334068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:33.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.480+0000 7fd346d9d700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd334066730 0x7fd334068bf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd338009af0 tx=0x7fd338008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:33.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.484+0000 7fd32f7fe700 1 -- 172.21.15.146:0/3800196743 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd3480cc840 con 0x7fd3400362c0 2024-09-13T18:36:33.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.736+0000 7fd34ca08700 1 -- 172.21.15.146:0/3800196743 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd340033570 con 0x7fd3400362c0 2024-09-13T18:36:33.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.736+0000 7fd32f7fe700 1 -- 172.21.15.146:0/3800196743 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5536 (secure 0 0 0) 0x7fd340033570 con 0x7fd3400362c0 2024-09-13T18:36:33.753 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:33.753 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:31.783229+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:33.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.740+0000 7fd34ca08700 1 -- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd334066730 msgr2=0x7fd334068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:33.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.740+0000 7fd34ca08700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd334066730 0x7fd334068bf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd338009af0 tx=0x7fd338008040 comp rx=0 tx=0).stop 2024-09-13T18:36:33.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.740+0000 7fd34ca08700 1 -- 172.21.15.146:0/3800196743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3400362c0 msgr2=0x7fd3400da830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:33.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.740+0000 7fd34ca08700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3400362c0 0x7fd3400da830 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd348072fe0 tx=0x7fd348068890 comp rx=0 tx=0).stop 2024-09-13T18:36:33.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.740+0000 7fd34ca08700 1 -- 172.21.15.146:0/3800196743 shutdown_connections 2024-09-13T18:36:33.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.740+0000 7fd34ca08700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd334066730 0x7fd334068bf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.740+0000 7fd34ca08700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3400362c0 0x7fd3400da830 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.740+0000 7fd34ca08700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3400cea90 0x7fd340173250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.740+0000 7fd34ca08700 1 --2- 172.21.15.146:0/3800196743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3400ce6b0 0x7fd3400cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:33.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.740+0000 7fd34ca08700 1 -- 172.21.15.146:0/3800196743 >> 172.21.15.146:0/3800196743 conn(0x7fd340050610 msgr2=0x7fd34004eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:33.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.740+0000 7fd34ca08700 1 -- 172.21.15.146:0/3800196743 shutdown_connections 2024-09-13T18:36:33.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:33.740+0000 7fd34ca08700 1 -- 172.21.15.146:0/3800196743 wait complete. 2024-09-13T18:36:33.757 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 50 2024-09-13T18:36:33.773 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=1, replacing gid=5278 2024-09-13T18:36:33.773 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.f 2024-09-13T18:36:34.775 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2024-09-13T18:36:34.775 INFO:tasks.ceph.mds.f:Restarting daemon 2024-09-13T18:36:34.775 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-09-13T18:36:34.779 INFO:tasks.ceph.mds.f:Started 2024-09-13T18:36:34.779 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:36:34.863 INFO:tasks.ceph.mds.f.smithi154.stdout:starting mds.f at 2024-09-13T18:36:34.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18aac4f700 1 Processor -- start 2024-09-13T18:36:34.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18aac4f700 1 -- start start 2024-09-13T18:36:34.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18aac4f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a40cbd50 0x7f18a40b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:34.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18aac4f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f18a40ca210 0x7f18a40cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:34.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18aac4f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a40cc670 con 0x7f18a4071390 2024-09-13T18:36:34.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18aac4f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a40caa30 con 0x7f18a40ca210 2024-09-13T18:36:34.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18aac4f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a40cb810 con 0x7f18a40cbd50 2024-09-13T18:36:34.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18a944c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f18a40ca210 0x7f18a40cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:34.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18a9c4d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a40cbd50 0x7f18a40b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:34.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18a9c4d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a40cbd50 0x7f18a40b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:55438/0 (socket says 172.21.15.146:55438) 2024-09-13T18:36:34.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18a944c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f18a40ca210 0x7f18a40cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:48510/0 (socket says 172.21.15.146:48510) 2024-09-13T18:36:34.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18aa44e700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f18a4071390 0x7f18a40701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:35530/0 (socket says 172.21.15.146:35530) 2024-09-13T18:36:34.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18a9c4d700 1 -- 172.21.15.146:0/3908372291 learned_addr learned my addr 172.21.15.146:0/3908372291 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:34.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18a9c4d700 1 -- 172.21.15.146:0/3908372291 >> v2:172.21.15.154:3300/0 conn(0x7f18a40ca210 msgr2=0x7f18a40cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:36:34.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18a9c4d700 1 --2- 172.21.15.146:0/3908372291 >> v2:172.21.15.154:3300/0 conn(0x7f18a40ca210 0x7f18a40cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:34.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18a9c4d700 1 -- 172.21.15.146:0/3908372291 >> v1:172.21.15.154:6789/0 conn(0x7f18a4071390 legacy=0x7f18a40701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:34.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.880+0000 7f18a9c4d700 1 -- 172.21.15.146:0/3908372291 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a415da20 con 0x7f18a40cbd50 2024-09-13T18:36:34.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18a9c4d700 1 --2- 172.21.15.146:0/3908372291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a40cbd50 0x7f18a40b1c40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1894009910 tx=0x7f1894009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4e03aec66fa8287 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:34.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18a8c4b700 1 -- 172.21.15.146:0/3908372291 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894011830 con 0x7f18a40cbd50 2024-09-13T18:36:34.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18a8c4b700 1 -- 172.21.15.146:0/3908372291 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1894011990 con 0x7f18a40cbd50 2024-09-13T18:36:34.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18a8c4b700 1 -- 172.21.15.146:0/3908372291 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894011d40 con 0x7f18a40cbd50 2024-09-13T18:36:34.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18aac4f700 1 -- 172.21.15.146:0/3908372291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a40cbd50 msgr2=0x7f18a40b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:34.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18aac4f700 1 --2- 172.21.15.146:0/3908372291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a40cbd50 0x7f18a40b1c40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1894009910 tx=0x7f1894009c20 comp rx=0 tx=0).stop 2024-09-13T18:36:34.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18aac4f700 1 -- 172.21.15.146:0/3908372291 shutdown_connections 2024-09-13T18:36:34.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18aac4f700 1 --2- 172.21.15.146:0/3908372291 >> v2:172.21.15.154:3300/0 conn(0x7f18a40ca210 0x7f18a40cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:34.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18aac4f700 1 --2- 172.21.15.146:0/3908372291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a40cbd50 0x7f18a40b1c40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:34.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18aac4f700 1 -- 172.21.15.146:0/3908372291 >> 172.21.15.146:0/3908372291 conn(0x7f18a4066270 msgr2=0x7f18a4066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:34.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18aac4f700 1 -- 172.21.15.146:0/3908372291 shutdown_connections 2024-09-13T18:36:34.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18aac4f700 1 -- 172.21.15.146:0/3908372291 wait complete. 2024-09-13T18:36:34.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18aac4f700 1 Processor -- start 2024-09-13T18:36:34.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.884+0000 7f18aac4f700 1 -- start start 2024-09-13T18:36:34.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aac4f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a4071390 0x7f18a415c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:34.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aac4f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18a40ca210 0x7f18a4148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:34.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aac4f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a40cbd50 0x7f18a415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:34.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aac4f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a40c9090 con 0x7f18a40cbd50 2024-09-13T18:36:34.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aac4f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a41efea0 con 0x7f18a40ca210 2024-09-13T18:36:34.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aac4f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a41f03d0 con 0x7f18a4071390 2024-09-13T18:36:34.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aa44e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a40cbd50 0x7f18a415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:34.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aa44e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a40cbd50 0x7f18a415afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40846/0 (socket says 172.21.15.146:40846) 2024-09-13T18:36:34.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aa44e700 1 -- 172.21.15.146:0/1477435510 learned_addr learned my addr 172.21.15.146:0/1477435510 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:36:34.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18a9c4d700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a4071390 0x7f18a415c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:34.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18a944c700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18a40ca210 0x7f18a4148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:34.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aa44e700 1 -- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a4071390 msgr2=0x7f18a415c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:34.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aa44e700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a4071390 0x7f18a415c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:34.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aa44e700 1 -- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18a40ca210 msgr2=0x7f18a4148840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:34.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aa44e700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18a40ca210 0x7f18a4148840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:34.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aa44e700 1 -- 172.21.15.146:0/1477435510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1894009570 con 0x7f18a40cbd50 2024-09-13T18:36:34.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aa44e700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a40cbd50 0x7f18a415afc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f189c00cbd0 tx=0x7f189c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:34.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f189a7fc700 1 -- 172.21.15.146:0/1477435510 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c00aa60 con 0x7f18a40cbd50 2024-09-13T18:36:34.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f189a7fc700 1 -- 172.21.15.146:0/1477435510 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189c00abc0 con 0x7f18a40cbd50 2024-09-13T18:36:34.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aac4f700 1 -- 172.21.15.146:0/1477435510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a415c080 con 0x7f18a40cbd50 2024-09-13T18:36:34.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f189a7fc700 1 -- 172.21.15.146:0/1477435510 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c007070 con 0x7f18a40cbd50 2024-09-13T18:36:34.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f18aac4f700 1 -- 172.21.15.146:0/1477435510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a415a7c0 con 0x7f18a40cbd50 2024-09-13T18:36:34.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.888+0000 7f1887fff700 1 -- 172.21.15.146:0/1477435510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1880005310 con 0x7f18a40cbd50 2024-09-13T18:36:34.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.892+0000 7f189a7fc700 1 -- 172.21.15.146:0/1477435510 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f189c0071d0 con 0x7f18a40cbd50 2024-09-13T18:36:34.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.892+0000 7f189a7fc700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1888066730 0x7f1888068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:36:34.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.892+0000 7f189a7fc700 1 -- 172.21.15.146:0/1477435510 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f189c0840f0 con 0x7f18a40cbd50 2024-09-13T18:36:34.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.892+0000 7f18a9c4d700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1888066730 0x7f1888068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:36:34.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.892+0000 7f18a9c4d700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1888066730 0x7f1888068bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1894009150 tx=0x7f1894008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:36:34.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:34.900+0000 7f189a7fc700 1 -- 172.21.15.146:0/1477435510 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f189c050fe0 con 0x7f18a40cbd50 2024-09-13T18:36:35.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.156+0000 7f1887fff700 1 -- 172.21.15.146:0/1477435510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18800055c0 con 0x7f18a40cbd50 2024-09-13T18:36:35.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.156+0000 7f189a7fc700 1 -- 172.21.15.146:0/1477435510 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6293 (secure 0 0 0) 0x7f189c017090 con 0x7f18a40cbd50 2024-09-13T18:36:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:36:35.171 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5800,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:31.783229+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:36:35.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.160+0000 7f1887fff700 1 -- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1888066730 msgr2=0x7f1888068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:35.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.160+0000 7f1887fff700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1888066730 0x7f1888068bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1894009150 tx=0x7f1894008030 comp rx=0 tx=0).stop 2024-09-13T18:36:35.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.160+0000 7f1887fff700 1 -- 172.21.15.146:0/1477435510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a40cbd50 msgr2=0x7f18a415afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:36:35.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.160+0000 7f1887fff700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a40cbd50 0x7f18a415afc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f189c00cbd0 tx=0x7f189c00cee0 comp rx=0 tx=0).stop 2024-09-13T18:36:35.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.160+0000 7f1887fff700 1 -- 172.21.15.146:0/1477435510 shutdown_connections 2024-09-13T18:36:35.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.160+0000 7f1887fff700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1888066730 0x7f1888068bf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:35.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.160+0000 7f1887fff700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a40cbd50 0x7f18a415afc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:35.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.160+0000 7f1887fff700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18a40ca210 0x7f18a4148840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:35.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.160+0000 7f1887fff700 1 --2- 172.21.15.146:0/1477435510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a4071390 0x7f18a415c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:36:35.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.160+0000 7f1887fff700 1 -- 172.21.15.146:0/1477435510 >> 172.21.15.146:0/1477435510 conn(0x7f18a4066270 msgr2=0x7f18a4149c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:36:35.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.160+0000 7f1887fff700 1 -- 172.21.15.146:0/1477435510 shutdown_connections 2024-09-13T18:36:35.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:36:35.160+0000 7f1887fff700 1 -- 172.21.15.146:0/1477435510 wait complete. 2024-09-13T18:36:35.175 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 51 2024-09-13T18:36:35.185 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2024-09-13T18:36:35.186 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 70 secs before thrashing 2024-09-13T18:36:38.670 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:36:38.673 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:37:08.724 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:37:08.727 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:37:38.779 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:37:38.782 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:37:45.186 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:37:45.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 Processor -- start 2024-09-13T18:37:45.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 -- start start 2024-09-13T18:37:45.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faf5804d7d0 0x7faf5800d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faf5800c260 0x7faf5800c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf580df180 con 0x7faf580d89f0 2024-09-13T18:37:45.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5800d890 con 0x7faf5800c260 2024-09-13T18:37:45.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5800bdd0 con 0x7faf5804d7d0 2024-09-13T18:37:45.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf5ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faf5800c260 0x7faf5800c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf5ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faf5800c260 0x7faf5800c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35894/0 (socket says 172.21.15.146:35894) 2024-09-13T18:37:45.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf5ffff700 1 -- 172.21.15.146:0/618513814 learned_addr learned my addr 172.21.15.146:0/618513814 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:37:45.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf5effd700 1 --2- 172.21.15.146:0/618513814 >> v2:172.21.15.154:3300/0 conn(0x7faf5804d7d0 0x7faf5800d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf5ffff700 1 -- 172.21.15.146:0/618513814 >> v2:172.21.15.154:3300/0 conn(0x7faf5804d7d0 msgr2=0x7faf5800d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:45.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf5ffff700 1 --2- 172.21.15.146:0/618513814 >> v2:172.21.15.154:3300/0 conn(0x7faf5804d7d0 0x7faf5800d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf5ffff700 1 -- 172.21.15.146:0/618513814 >> v1:172.21.15.154:6789/0 conn(0x7faf580d89f0 legacy=0x7faf5804e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:45.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf5ffff700 1 -- 172.21.15.146:0/618513814 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5800cc30 con 0x7faf5800c260 2024-09-13T18:37:45.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf5ffff700 1 --2- 172.21.15.146:0/618513814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf5800c260 0x7faf5800c6f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7faf600731e0 tx=0x7faf60061ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa9b51d5ebdcb57 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:45.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf5e7fc700 1 -- 172.21.15.146:0/618513814 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf600739c0 con 0x7faf5800c260 2024-09-13T18:37:45.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf5e7fc700 1 -- 172.21.15.146:0/618513814 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf60075a30 con 0x7faf5800c260 2024-09-13T18:37:45.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf5e7fc700 1 -- 172.21.15.146:0/618513814 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60075de0 con 0x7faf5800c260 2024-09-13T18:37:45.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 -- 172.21.15.146:0/618513814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf5800c260 msgr2=0x7faf5800c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:45.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 --2- 172.21.15.146:0/618513814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf5800c260 0x7faf5800c6f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7faf600731e0 tx=0x7faf60061ef0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 -- 172.21.15.146:0/618513814 shutdown_connections 2024-09-13T18:37:45.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 --2- 172.21.15.146:0/618513814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf5800c260 0x7faf5800c6f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 --2- 172.21.15.146:0/618513814 >> v2:172.21.15.154:3300/0 conn(0x7faf5804d7d0 0x7faf5800d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 -- 172.21.15.146:0/618513814 >> 172.21.15.146:0/618513814 conn(0x7faf5800a130 msgr2=0x7faf5800dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:45.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 -- 172.21.15.146:0/618513814 shutdown_connections 2024-09-13T18:37:45.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.298+0000 7faf64f1e700 1 -- 172.21.15.146:0/618513814 wait complete. 2024-09-13T18:37:45.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf64f1e700 1 Processor -- start 2024-09-13T18:37:45.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf64f1e700 1 -- start start 2024-09-13T18:37:45.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf64f1e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf5800c260 0x7faf58177b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf64f1e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5804d7d0 0x7faf580338f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf64f1e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf580d89f0 0x7faf58033e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf64f1e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf580d2e10 con 0x7faf580d89f0 2024-09-13T18:37:45.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf64f1e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf58178630 con 0x7faf5804d7d0 2024-09-13T18:37:45.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf64f1e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf581783c0 con 0x7faf5800c260 2024-09-13T18:37:45.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf5ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf580d89f0 0x7faf58033e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf5ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf580d89f0 0x7faf58033e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35902/0 (socket says 172.21.15.146:35902) 2024-09-13T18:37:45.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf5ffff700 1 -- 172.21.15.146:0/2276483409 learned_addr learned my addr 172.21.15.146:0/2276483409 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:37:45.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf5effd700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5804d7d0 0x7faf580338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf5f7fe700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf5800c260 0x7faf58177b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf5ffff700 1 -- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf5800c260 msgr2=0x7faf58177b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:45.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf5ffff700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf5800c260 0x7faf58177b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf5ffff700 1 -- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5804d7d0 msgr2=0x7faf580338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:45.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf5ffff700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5804d7d0 0x7faf580338f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf5ffff700 1 -- 172.21.15.146:0/2276483409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6004f5c0 con 0x7faf580d89f0 2024-09-13T18:37:45.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf5ffff700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf580d89f0 0x7faf58033e30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7faf6004f480 tx=0x7faf6004f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:45.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf4bfff700 1 -- 172.21.15.146:0/2276483409 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60067e50 con 0x7faf580d89f0 2024-09-13T18:37:45.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf4bfff700 1 -- 172.21.15.146:0/2276483409 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf60067fb0 con 0x7faf580d89f0 2024-09-13T18:37:45.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf64f1e700 1 -- 172.21.15.146:0/2276483409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf580ddb10 con 0x7faf580d89f0 2024-09-13T18:37:45.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf4bfff700 1 -- 172.21.15.146:0/2276483409 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60070940 con 0x7faf580d89f0 2024-09-13T18:37:45.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.302+0000 7faf64f1e700 1 -- 172.21.15.146:0/2276483409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf580dd550 con 0x7faf580d89f0 2024-09-13T18:37:45.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.306+0000 7faf64f1e700 1 -- 172.21.15.146:0/2276483409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf580d2f80 con 0x7faf580d89f0 2024-09-13T18:37:45.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.306+0000 7faf4bfff700 1 -- 172.21.15.146:0/2276483409 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faf60070aa0 con 0x7faf580d89f0 2024-09-13T18:37:45.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.306+0000 7faf4bfff700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faf44066730 0x7faf44068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.306+0000 7faf4bfff700 1 -- 172.21.15.146:0/2276483409 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7faf60100700 con 0x7faf580d89f0 2024-09-13T18:37:45.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.306+0000 7faf5f7fe700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faf44066730 0x7faf44068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.306+0000 7faf5f7fe700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faf44066730 0x7faf44068bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faf4c005150 tx=0x7faf4c007ee0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:45.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.314+0000 7faf4bfff700 1 -- 172.21.15.146:0/2276483409 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faf580d2f80 con 0x7faf580d89f0 2024-09-13T18:37:45.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.554+0000 7faf64f1e700 1 -- 172.21.15.146:0/2276483409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf580d2f50 con 0x7faf580d89f0 2024-09-13T18:37:45.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.554+0000 7faf4bfff700 1 -- 172.21.15.146:0/2276483409 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6293 (secure 0 0 0) 0x7faf600cd540 con 0x7faf580d89f0 2024-09-13T18:37:45.573 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:37:45.573 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5800,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:31.783229+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:37:45.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.558+0000 7faf64f1e700 1 -- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faf44066730 msgr2=0x7faf44068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.558+0000 7faf64f1e700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faf44066730 0x7faf44068bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faf4c005150 tx=0x7faf4c007ee0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.558+0000 7faf64f1e700 1 -- 172.21.15.146:0/2276483409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf580d89f0 msgr2=0x7faf58033e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.558+0000 7faf64f1e700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf580d89f0 0x7faf58033e30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7faf6004f480 tx=0x7faf6004f750 comp rx=0 tx=0).stop 2024-09-13T18:37:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.558+0000 7faf64f1e700 1 -- 172.21.15.146:0/2276483409 shutdown_connections 2024-09-13T18:37:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.558+0000 7faf64f1e700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faf44066730 0x7faf44068bf0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.558+0000 7faf64f1e700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf580d89f0 0x7faf58033e30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.558+0000 7faf64f1e700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5804d7d0 0x7faf580338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.558+0000 7faf64f1e700 1 --2- 172.21.15.146:0/2276483409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf5800c260 0x7faf58177b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.558+0000 7faf64f1e700 1 -- 172.21.15.146:0/2276483409 >> 172.21.15.146:0/2276483409 conn(0x7faf5800a130 msgr2=0x7faf58051f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.558+0000 7faf64f1e700 1 -- 172.21.15.146:0/2276483409 shutdown_connections 2024-09-13T18:37:45.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.558+0000 7faf64f1e700 1 -- 172.21.15.146:0/2276483409 wait complete. 2024-09-13T18:37:45.579 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 51 2024-09-13T18:37:45.591 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2024-09-13T18:37:45.591 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T18:37:45.592 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:37:45.604 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T18:37:45.605 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:37:45.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.698+0000 7f823ea24700 1 Processor -- start 2024-09-13T18:37:45.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.698+0000 7f823ea24700 1 -- start start 2024-09-13T18:37:45.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ea24700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82300d5050 0x7f8230045470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ea24700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8230007da0 0x7f82300de1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ea24700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823004c840 con 0x7f823004b5a0 2024-09-13T18:37:45.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ea24700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823004cde0 con 0x7f8230007da0 2024-09-13T18:37:45.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ea24700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82300dbc00 con 0x7f82300d5050 2024-09-13T18:37:45.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823e223700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f823004b5a0 0x7f82300de6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37858/0 (socket says 172.21.15.146:37858) 2024-09-13T18:37:45.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823e223700 1 -- 172.21.15.146:0/3187846184 learned_addr learned my addr 172.21.15.146:0/3187846184 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:37:45.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140383934 0 0) 0x7f823004c840 con 0x7f823004b5a0 2024-09-13T18:37:45.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823da22700 1 --2- 172.21.15.146:0/3187846184 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82300d5050 0x7f8230045470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823d221700 1 --2- 172.21.15.146:0/3187846184 >> v2:172.21.15.154:3300/0 conn(0x7f8230007da0 0x7f82300de1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82240035d0 con 0x7f823004b5a0 2024-09-13T18:37:45.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 369799097 0 0) 0x7f82240035d0 con 0x7f823004b5a0 2024-09-13T18:37:45.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82300d5050 msgr2=0x7f8230045470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:45.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 --2- 172.21.15.146:0/3187846184 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82300d5050 0x7f8230045470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 >> v2:172.21.15.154:3300/0 conn(0x7f8230007da0 msgr2=0x7f82300de1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:45.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 --2- 172.21.15.146:0/3187846184 >> v2:172.21.15.154:3300/0 conn(0x7f8230007da0 0x7f82300de1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823004cb00 con 0x7f823004b5a0 2024-09-13T18:37:45.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f8238074a30 con 0x7f823004b5a0 2024-09-13T18:37:45.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 >> v1:172.21.15.146:6789/0 conn(0x7f823004b5a0 legacy=0x7f82300de6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:45.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 --2- 172.21.15.146:0/3187846184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8224003a20 0x7f8224003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 --2- 172.21.15.146:0/3187846184 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82300d5050 0x7f8224004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 --2- 172.21.15.146:0/3187846184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8224004db0 0x7f8224005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823004cde0 con 0x7f8224003a20 2024-09-13T18:37:45.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822400ba30 con 0x7f8224004db0 2024-09-13T18:37:45.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822400bba0 con 0x7f82300d5050 2024-09-13T18:37:45.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823da22700 1 --2- 172.21.15.146:0/3187846184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8224003a20 0x7f8224003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823d221700 1 --2- 172.21.15.146:0/3187846184 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82300d5050 0x7f8224004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823e223700 1 --2- 172.21.15.146:0/3187846184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8224004db0 0x7f8224005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823da22700 1 -- 172.21.15.146:0/3187846184 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82300d5050 msgr2=0x7f8224004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:45.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823da22700 1 --2- 172.21.15.146:0/3187846184 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82300d5050 0x7f8224004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823da22700 1 -- 172.21.15.146:0/3187846184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8224004db0 msgr2=0x7f8224005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:45.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823da22700 1 --2- 172.21.15.146:0/3187846184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8224004db0 0x7f8224005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823da22700 1 -- 172.21.15.146:0/3187846184 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82240035d0 con 0x7f8224003a20 2024-09-13T18:37:45.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823e223700 1 --2- 172.21.15.146:0/3187846184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8224004db0 0x7f8224005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:37:45.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823da22700 1 --2- 172.21.15.146:0/3187846184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8224003a20 0x7f8224003e40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8234009b70 tx=0x7f82240058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97704f52bf10cde0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:45.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400e6c0 con 0x7f8224003a20 2024-09-13T18:37:45.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823400e820 con 0x7f8224003a20 2024-09-13T18:37:45.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ca20700 1 -- 172.21.15.146:0/3187846184 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400eb90 con 0x7f8224003a20 2024-09-13T18:37:45.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ea24700 1 -- 172.21.15.146:0/3187846184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8224003a20 msgr2=0x7f8224003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:45.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ea24700 1 --2- 172.21.15.146:0/3187846184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8224003a20 0x7f8224003e40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8234009b70 tx=0x7f82240058a0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823e223700 1 -- 172.21.15.146:0/3187846184 reap_dead start 2024-09-13T18:37:45.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ea24700 1 -- 172.21.15.146:0/3187846184 shutdown_connections 2024-09-13T18:37:45.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.702+0000 7f823ea24700 1 -- 172.21.15.146:0/3187846184 >> 172.21.15.146:0/3187846184 conn(0x7f8230050590 msgr2=0x7f82300509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:45.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823ea24700 1 -- 172.21.15.146:0/3187846184 shutdown_connections 2024-09-13T18:37:45.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823ea24700 1 -- 172.21.15.146:0/3187846184 wait complete. 2024-09-13T18:37:45.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823ea24700 1 Processor -- start 2024-09-13T18:37:45.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823ea24700 1 -- start start 2024-09-13T18:37:45.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823ea24700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8224003a20 0x7f8230174e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823ea24700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8230036240 0x7f8230045fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823ea24700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82300d82f0 0x7f82300d7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823ea24700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8230035e10 con 0x7f8230036240 2024-09-13T18:37:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823ea24700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8230174660 con 0x7f82300d82f0 2024-09-13T18:37:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823ea24700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8230035b30 con 0x7f8224003a20 2024-09-13T18:37:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823d221700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8230036240 0x7f8230045fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823d221700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8230036240 0x7f8230045fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35922/0 (socket says 172.21.15.146:35922) 2024-09-13T18:37:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823d221700 1 -- 172.21.15.146:0/3656544487 learned_addr learned my addr 172.21.15.146:0/3656544487 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:37:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823da22700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8224003a20 0x7f8230174e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823e223700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82300d82f0 0x7f82300d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823d221700 1 -- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8224003a20 msgr2=0x7f8230174e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823d221700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8224003a20 0x7f8230174e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823d221700 1 -- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82300d82f0 msgr2=0x7f82300d7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:45.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823d221700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82300d82f0 0x7f82300d7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823d221700 1 -- 172.21.15.146:0/3656544487 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8230178670 con 0x7f8230036240 2024-09-13T18:37:45.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f823d221700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8230036240 0x7f8230045fd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8228011680 tx=0x7f8228011990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:45.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f822e7fc700 1 -- 172.21.15.146:0/3656544487 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822800c070 con 0x7f8230036240 2024-09-13T18:37:45.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f822e7fc700 1 -- 172.21.15.146:0/3656544487 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8228015930 con 0x7f8230036240 2024-09-13T18:37:45.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.706+0000 7f822e7fc700 1 -- 172.21.15.146:0/3656544487 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228015ca0 con 0x7f8230036240 2024-09-13T18:37:45.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.710+0000 7f823ea24700 1 -- 172.21.15.146:0/3656544487 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8230174900 con 0x7f8230036240 2024-09-13T18:37:45.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.710+0000 7f823ea24700 1 -- 172.21.15.146:0/3656544487 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823004f9c0 con 0x7f8230036240 2024-09-13T18:37:45.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.710+0000 7f823ea24700 1 -- 172.21.15.146:0/3656544487 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8230005d20 con 0x7f8230036240 2024-09-13T18:37:45.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.710+0000 7f822e7fc700 1 -- 172.21.15.146:0/3656544487 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f82280143e0 con 0x7f8230036240 2024-09-13T18:37:45.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.710+0000 7f822e7fc700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f821c066760 0x7f821c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:45.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.710+0000 7f822e7fc700 1 -- 172.21.15.146:0/3656544487 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f82280847b0 con 0x7f8230036240 2024-09-13T18:37:45.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.710+0000 7f823da22700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f821c066760 0x7f821c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:45.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.714+0000 7f823da22700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f821c066760 0x7f821c068c20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f82340090b0 tx=0x7f823401e040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:45.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.718+0000 7f822e7fc700 1 -- 172.21.15.146:0/3656544487 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f82280514d0 con 0x7f8230036240 2024-09-13T18:37:45.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f823ea24700 1 -- 172.21.15.146:0/3656544487 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8230174a90 con 0x7f8230036240 2024-09-13T18:37:45.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f822e7fc700 1 -- 172.21.15.146:0/3656544487 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6293 (secure 0 0 0) 0x7f8228006ce0 con 0x7f8230036240 2024-09-13T18:37:45.963 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:37:45.963 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5800,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:31.783229+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:37:45.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f823ea24700 1 -- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f821c066760 msgr2=0x7f821c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:45.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f823ea24700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f821c066760 0x7f821c068c20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f82340090b0 tx=0x7f823401e040 comp rx=0 tx=0).stop 2024-09-13T18:37:45.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f823ea24700 1 -- 172.21.15.146:0/3656544487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8230036240 msgr2=0x7f8230045fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:45.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f823ea24700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8230036240 0x7f8230045fd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8228011680 tx=0x7f8228011990 comp rx=0 tx=0).stop 2024-09-13T18:37:45.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f823ea24700 1 -- 172.21.15.146:0/3656544487 shutdown_connections 2024-09-13T18:37:45.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f823ea24700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f821c066760 0x7f821c068c20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f823ea24700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82300d82f0 0x7f82300d7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f823ea24700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8230036240 0x7f8230045fd0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f823ea24700 1 --2- 172.21.15.146:0/3656544487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8224003a20 0x7f8230174e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:45.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f823ea24700 1 -- 172.21.15.146:0/3656544487 >> 172.21.15.146:0/3656544487 conn(0x7f8230050590 msgr2=0x7f82300c47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:45.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f823ea24700 1 -- 172.21.15.146:0/3656544487 shutdown_connections 2024-09-13T18:37:45.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:45.946+0000 7f823ea24700 1 -- 172.21.15.146:0/3656544487 wait complete. 2024-09-13T18:37:45.967 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 51 2024-09-13T18:37:45.982 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:37:47.984 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:37:48.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.090+0000 7f8746624700 1 Processor -- start 2024-09-13T18:37:48.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.090+0000 7f8746624700 1 -- start start 2024-09-13T18:37:48.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.090+0000 7f8746624700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8740153bc0 0x7f87400c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:48.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.090+0000 7f8746624700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f87400c6610 0x7f87400c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:48.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.090+0000 7f8746624700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f87400cac60 0x7f87400cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.090+0000 7f8746624700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8740153fa0 con 0x7f87400cac60 2024-09-13T18:37:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.090+0000 7f8746624700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87400aa500 con 0x7f87400c6610 2024-09-13T18:37:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.090+0000 7f8746624700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874015a7e0 con 0x7f8740153bc0 2024-09-13T18:37:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8745e23700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f87400cac60 0x7f87400cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8745e23700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f87400cac60 0x7f87400cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35930/0 (socket says 172.21.15.146:35930) 2024-09-13T18:37:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8745e23700 1 -- 172.21.15.146:0/3742312761 learned_addr learned my addr 172.21.15.146:0/3742312761 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:37:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8745622700 1 --2- 172.21.15.146:0/3742312761 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8740153bc0 0x7f87400c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8744e21700 1 --2- 172.21.15.146:0/3742312761 >> v2:172.21.15.154:3300/0 conn(0x7f87400c6610 0x7f87400c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8745e23700 1 -- 172.21.15.146:0/3742312761 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8740153bc0 msgr2=0x7f87400c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8745e23700 1 --2- 172.21.15.146:0/3742312761 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8740153bc0 0x7f87400c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8745e23700 1 -- 172.21.15.146:0/3742312761 >> v2:172.21.15.154:3300/0 conn(0x7f87400c6610 msgr2=0x7f87400c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8745e23700 1 --2- 172.21.15.146:0/3742312761 >> v2:172.21.15.154:3300/0 conn(0x7f87400c6610 0x7f87400c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8745e23700 1 -- 172.21.15.146:0/3742312761 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400c9320 con 0x7f87400cac60 2024-09-13T18:37:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8745e23700 1 --2- 172.21.15.146:0/3742312761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87400cac60 0x7f87400cb0d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f87400cb9e0 tx=0x7f873800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae3d9cebe5501cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8737fff700 1 -- 172.21.15.146:0/3742312761 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738010070 con 0x7f87400cac60 2024-09-13T18:37:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8737fff700 1 -- 172.21.15.146:0/3742312761 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873800dce0 con 0x7f87400cac60 2024-09-13T18:37:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8737fff700 1 -- 172.21.15.146:0/3742312761 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873800b310 con 0x7f87400cac60 2024-09-13T18:37:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8746624700 1 -- 172.21.15.146:0/3742312761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87400cac60 msgr2=0x7f87400cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8746624700 1 --2- 172.21.15.146:0/3742312761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87400cac60 0x7f87400cb0d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f87400cb9e0 tx=0x7f873800d430 comp rx=0 tx=0).stop 2024-09-13T18:37:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8746624700 1 -- 172.21.15.146:0/3742312761 shutdown_connections 2024-09-13T18:37:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8746624700 1 --2- 172.21.15.146:0/3742312761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87400cac60 0x7f87400cb0d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8746624700 1 --2- 172.21.15.146:0/3742312761 >> v2:172.21.15.154:3300/0 conn(0x7f87400c6610 0x7f87400c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8746624700 1 --2- 172.21.15.146:0/3742312761 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8740153bc0 0x7f87400c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8746624700 1 -- 172.21.15.146:0/3742312761 >> 172.21.15.146:0/3742312761 conn(0x7f8740066090 msgr2=0x7f8740066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:48.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8746624700 1 -- 172.21.15.146:0/3742312761 shutdown_connections 2024-09-13T18:37:48.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.094+0000 7f8746624700 1 -- 172.21.15.146:0/3742312761 wait complete. 2024-09-13T18:37:48.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8746624700 1 Processor -- start 2024-09-13T18:37:48.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8746624700 1 -- start start 2024-09-13T18:37:48.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8746624700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87400c6610 0x7f8740150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:48.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8746624700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87400cac60 0x7f87400c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:48.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8746624700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8740153bc0 0x7f87400aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:48.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8746624700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87400acf50 con 0x7f8740153bc0 2024-09-13T18:37:48.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8746624700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8740154150 con 0x7f87400cac60 2024-09-13T18:37:48.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8746624700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87400ab150 con 0x7f87400c6610 2024-09-13T18:37:48.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8745e23700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8740153bc0 0x7f87400aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:48.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8745e23700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8740153bc0 0x7f87400aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35938/0 (socket says 172.21.15.146:35938) 2024-09-13T18:37:48.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8745e23700 1 -- 172.21.15.146:0/3237477819 learned_addr learned my addr 172.21.15.146:0/3237477819 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:37:48.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8745622700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87400c6610 0x7f8740150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:48.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8744e21700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87400cac60 0x7f87400c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:48.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8745e23700 1 -- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87400c6610 msgr2=0x7f8740150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:48.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8745e23700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87400c6610 0x7f8740150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:48.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8745e23700 1 -- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87400cac60 msgr2=0x7f87400c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:48.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8745e23700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87400cac60 0x7f87400c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:48.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8745e23700 1 -- 172.21.15.146:0/3237477819 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873800bd40 con 0x7f8740153bc0 2024-09-13T18:37:48.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8745e23700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8740153bc0 0x7f87400aec10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f873800d780 tx=0x7f8738004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:48.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8735ffb700 1 -- 172.21.15.146:0/3237477819 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738009a10 con 0x7f8740153bc0 2024-09-13T18:37:48.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8746624700 1 -- 172.21.15.146:0/3237477819 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87400ac620 con 0x7f8740153bc0 2024-09-13T18:37:48.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8735ffb700 1 -- 172.21.15.146:0/3237477819 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8738009b90 con 0x7f8740153bc0 2024-09-13T18:37:48.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8735ffb700 1 -- 172.21.15.146:0/3237477819 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873800b4c0 con 0x7f8740153bc0 2024-09-13T18:37:48.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.098+0000 7f8746624700 1 -- 172.21.15.146:0/3237477819 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87400ac890 con 0x7f8740153bc0 2024-09-13T18:37:48.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.102+0000 7f8746624700 1 -- 172.21.15.146:0/3237477819 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87400ad090 con 0x7f8740153bc0 2024-09-13T18:37:48.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.102+0000 7f8735ffb700 1 -- 172.21.15.146:0/3237477819 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8738004040 con 0x7f8740153bc0 2024-09-13T18:37:48.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.102+0000 7f8735ffb700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8724066730 0x7f8724068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:48.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.102+0000 7f8735ffb700 1 -- 172.21.15.146:0/3237477819 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f8738088340 con 0x7f8740153bc0 2024-09-13T18:37:48.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.102+0000 7f8745622700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8724066730 0x7f8724068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:48.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.102+0000 7f8745622700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8724066730 0x7f8724068bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8730009b50 tx=0x7f8730009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:48.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.110+0000 7f8735ffb700 1 -- 172.21.15.146:0/3237477819 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f87400ad090 con 0x7f8740153bc0 2024-09-13T18:37:48.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.318+0000 7f8746624700 1 -- 172.21.15.146:0/3237477819 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87400ad090 con 0x7f8740153bc0 2024-09-13T18:37:48.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.318+0000 7f8735ffb700 1 -- 172.21.15.146:0/3237477819 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6293 (secure 0 0 0) 0x7f87400ad090 con 0x7f8740153bc0 2024-09-13T18:37:48.338 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:37:48.338 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5800,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:31.783229+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:37:48.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.322+0000 7f8746624700 1 -- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8724066730 msgr2=0x7f8724068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:48.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.322+0000 7f8746624700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8724066730 0x7f8724068bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8730009b50 tx=0x7f8730009240 comp rx=0 tx=0).stop 2024-09-13T18:37:48.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.322+0000 7f8746624700 1 -- 172.21.15.146:0/3237477819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8740153bc0 msgr2=0x7f87400aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:48.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.322+0000 7f8746624700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8740153bc0 0x7f87400aec10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f873800d780 tx=0x7f8738004af0 comp rx=0 tx=0).stop 2024-09-13T18:37:48.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.322+0000 7f8746624700 1 -- 172.21.15.146:0/3237477819 shutdown_connections 2024-09-13T18:37:48.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.322+0000 7f8746624700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8724066730 0x7f8724068bf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:48.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.322+0000 7f8746624700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8740153bc0 0x7f87400aec10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:48.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.322+0000 7f8746624700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87400cac60 0x7f87400c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:48.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.322+0000 7f8746624700 1 --2- 172.21.15.146:0/3237477819 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87400c6610 0x7f8740150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:48.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.322+0000 7f8746624700 1 -- 172.21.15.146:0/3237477819 >> 172.21.15.146:0/3237477819 conn(0x7f8740066090 msgr2=0x7f87400c08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:48.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.322+0000 7f8746624700 1 -- 172.21.15.146:0/3237477819 shutdown_connections 2024-09-13T18:37:48.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:48.322+0000 7f8746624700 1 -- 172.21.15.146:0/3237477819 wait complete. 2024-09-13T18:37:48.342 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 51 2024-09-13T18:37:48.352 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:37:50.353 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:37:50.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.458+0000 7f4ed1b73700 1 Processor -- start 2024-09-13T18:37:50.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.458+0000 7f4ed1b73700 1 -- start start 2024-09-13T18:37:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ed1b73700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc0c6610 0x7f4ecc0c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ed1b73700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc153fa0 con 0x7f4ecc153bc0 2024-09-13T18:37:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ed1b73700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc0aa500 con 0x7f4ecc0cac60 2024-09-13T18:37:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ed1b73700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc15a7e0 con 0x7f4ecc0c6610 2024-09-13T18:37:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ed0b71700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4ecc153bc0 0x7f4ecc0c8de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37862/0 (socket says 172.21.15.146:37862) 2024-09-13T18:37:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ed0b71700 1 -- 172.21.15.146:0/2757010875 learned_addr learned my addr 172.21.15.146:0/2757010875 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:37:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ec3fff700 1 --2- 172.21.15.146:0/2757010875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc0c6610 0x7f4ecc0c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:50.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ec37fe700 1 -- 172.21.15.146:0/2757010875 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253074237 0 0) 0x7f4ecc153fa0 con 0x7f4ecc153bc0 2024-09-13T18:37:50.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ec37fe700 1 -- 172.21.15.146:0/2757010875 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eb40035d0 con 0x7f4ecc153bc0 2024-09-13T18:37:50.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ec3fff700 1 -- 172.21.15.146:0/2757010875 >> v1:172.21.15.154:6789/0 conn(0x7f4ecc0cac60 legacy=0x7f4ecc160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:50.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ec3fff700 1 -- 172.21.15.146:0/2757010875 >> v1:172.21.15.146:6789/0 conn(0x7f4ecc153bc0 legacy=0x7f4ecc0c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:50.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ec37fe700 1 -- 172.21.15.146:0/2757010875 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243895881 0 0) 0x7f4ecc0aa500 con 0x7f4ecc0cac60 2024-09-13T18:37:50.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ec3fff700 1 -- 172.21.15.146:0/2757010875 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc0aa220 con 0x7f4ecc0c6610 2024-09-13T18:37:50.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ec3fff700 1 --2- 172.21.15.146:0/2757010875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc0c6610 0x7f4ecc0c6a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009880 tx=0x7f4ebc009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=498fcb86d8b7fa6e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:50.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ec37fe700 1 -- 172.21.15.146:0/2757010875 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc00f8a0 con 0x7f4ecc0c6610 2024-09-13T18:37:50.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ec37fe700 1 -- 172.21.15.146:0/2757010875 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc00fa00 con 0x7f4ecc0c6610 2024-09-13T18:37:50.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ec37fe700 1 -- 172.21.15.146:0/2757010875 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc00fd90 con 0x7f4ecc0c6610 2024-09-13T18:37:50.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2757010875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc0c6610 msgr2=0x7f4ecc0c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:50.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ed1b73700 1 --2- 172.21.15.146:0/2757010875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc0c6610 0x7f4ecc0c6a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009880 tx=0x7f4ebc009b90 comp rx=0 tx=0).stop 2024-09-13T18:37:50.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2757010875 shutdown_connections 2024-09-13T18:37:50.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ed1b73700 1 --2- 172.21.15.146:0/2757010875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc0c6610 0x7f4ecc0c6a50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:50.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2757010875 >> 172.21.15.146:0/2757010875 conn(0x7f4ecc066090 msgr2=0x7f4ecc066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:50.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2757010875 shutdown_connections 2024-09-13T18:37:50.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.462+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2757010875 wait complete. 2024-09-13T18:37:50.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.466+0000 7f4ed1b73700 1 Processor -- start 2024-09-13T18:37:50.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.466+0000 7f4ed1b73700 1 -- start start 2024-09-13T18:37:50.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.466+0000 7f4ed1b73700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c6610 0x7f4ecc1eb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:50.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.466+0000 7f4ed1b73700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc0cac60 0x7f4ecc0c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:50.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.466+0000 7f4ed1b73700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc153bc0 0x7f4ecc0c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:50.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.466+0000 7f4ed1b73700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0acab0 con 0x7f4ecc0cac60 2024-09-13T18:37:50.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.466+0000 7f4ed1b73700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0ab510 con 0x7f4ecc0c6610 2024-09-13T18:37:50.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.466+0000 7f4ed1b73700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0ab7b0 con 0x7f4ecc153bc0 2024-09-13T18:37:50.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.466+0000 7f4ec3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc0cac60 0x7f4ecc0c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:50.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.466+0000 7f4ed0b71700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c6610 0x7f4ecc1eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:50.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.466+0000 7f4ec3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc0cac60 0x7f4ecc0c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35946/0 (socket says 172.21.15.146:35946) 2024-09-13T18:37:50.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.466+0000 7f4ec3fff700 1 -- 172.21.15.146:0/2122518800 learned_addr learned my addr 172.21.15.146:0/2122518800 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:37:50.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.466+0000 7f4ed1372700 1 --2- 172.21.15.146:0/2122518800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc153bc0 0x7f4ecc0c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:50.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.470+0000 7f4ec3fff700 1 -- 172.21.15.146:0/2122518800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc153bc0 msgr2=0x7f4ecc0c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:50.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.470+0000 7f4ec3fff700 1 --2- 172.21.15.146:0/2122518800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc153bc0 0x7f4ecc0c3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:50.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.470+0000 7f4ec3fff700 1 -- 172.21.15.146:0/2122518800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c6610 msgr2=0x7f4ecc1eb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:50.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.470+0000 7f4ec3fff700 1 --2- 172.21.15.146:0/2122518800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c6610 0x7f4ecc1eb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:50.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.470+0000 7f4ec3fff700 1 -- 172.21.15.146:0/2122518800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc0094b0 con 0x7f4ecc0cac60 2024-09-13T18:37:50.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.470+0000 7f4ec3fff700 1 --2- 172.21.15.146:0/2122518800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc0cac60 0x7f4ecc0c4330 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc000c00 tx=0x7f4ebc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:50.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.470+0000 7f4ec17fa700 1 -- 172.21.15.146:0/2122518800 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc01e070 con 0x7f4ecc0cac60 2024-09-13T18:37:50.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.470+0000 7f4ec17fa700 1 -- 172.21.15.146:0/2122518800 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc0109b0 con 0x7f4ecc0cac60 2024-09-13T18:37:50.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.470+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2122518800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc0aba90 con 0x7f4ecc0cac60 2024-09-13T18:37:50.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.470+0000 7f4ec17fa700 1 -- 172.21.15.146:0/2122518800 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc010d40 con 0x7f4ecc0cac60 2024-09-13T18:37:50.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.470+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2122518800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc0ac050 con 0x7f4ecc0cac60 2024-09-13T18:37:50.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.470+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2122518800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc0acc20 con 0x7f4ecc0cac60 2024-09-13T18:37:50.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.470+0000 7f4ec17fa700 1 -- 172.21.15.146:0/2122518800 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4ebc022070 con 0x7f4ecc0cac60 2024-09-13T18:37:50.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.474+0000 7f4ec17fa700 1 --2- 172.21.15.146:0/2122518800 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4eb8066760 0x7f4eb8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:50.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.474+0000 7f4ec17fa700 1 -- 172.21.15.146:0/2122518800 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f4ebc087260 con 0x7f4ecc0cac60 2024-09-13T18:37:50.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.474+0000 7f4ed0b71700 1 --2- 172.21.15.146:0/2122518800 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4eb8066760 0x7f4eb8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:50.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.474+0000 7f4ed0b71700 1 --2- 172.21.15.146:0/2122518800 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4eb8066760 0x7f4eb8068c20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8007fd0 tx=0x7f4ec8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:50.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.478+0000 7f4ec17fa700 1 -- 172.21.15.146:0/2122518800 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4ebc054140 con 0x7f4ecc0cac60 2024-09-13T18:37:50.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.686+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2122518800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ecc0a9de0 con 0x7f4ecc0cac60 2024-09-13T18:37:50.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.686+0000 7f4ec17fa700 1 -- 172.21.15.146:0/2122518800 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6293 (secure 0 0 0) 0x7f4ebc057760 con 0x7f4ecc0cac60 2024-09-13T18:37:50.705 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:37:50.705 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5800,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:31.783229+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:37:50.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.690+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2122518800 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4eb8066760 msgr2=0x7f4eb8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:50.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.690+0000 7f4ed1b73700 1 --2- 172.21.15.146:0/2122518800 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4eb8066760 0x7f4eb8068c20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8007fd0 tx=0x7f4ec8009040 comp rx=0 tx=0).stop 2024-09-13T18:37:50.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.690+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2122518800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc0cac60 msgr2=0x7f4ecc0c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:50.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.690+0000 7f4ed1b73700 1 --2- 172.21.15.146:0/2122518800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc0cac60 0x7f4ecc0c4330 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc000c00 tx=0x7f4ebc004770 comp rx=0 tx=0).stop 2024-09-13T18:37:50.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.690+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2122518800 shutdown_connections 2024-09-13T18:37:50.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.690+0000 7f4ed1b73700 1 --2- 172.21.15.146:0/2122518800 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4eb8066760 0x7f4eb8068c20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:50.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.690+0000 7f4ed1b73700 1 --2- 172.21.15.146:0/2122518800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ecc153bc0 0x7f4ecc0c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:50.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.690+0000 7f4ed1b73700 1 --2- 172.21.15.146:0/2122518800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ecc0cac60 0x7f4ecc0c4330 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:50.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.690+0000 7f4ed1b73700 1 --2- 172.21.15.146:0/2122518800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ecc0c6610 0x7f4ecc1eb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:50.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.690+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2122518800 >> 172.21.15.146:0/2122518800 conn(0x7f4ecc066090 msgr2=0x7f4ecc0cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:50.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.690+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2122518800 shutdown_connections 2024-09-13T18:37:50.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:50.690+0000 7f4ed1b73700 1 -- 172.21.15.146:0/2122518800 wait complete. 2024-09-13T18:37:50.710 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 51 2024-09-13T18:37:50.720 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:37:52.721 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:37:52.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 Processor -- start 2024-09-13T18:37:52.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 -- start start 2024-09-13T18:37:52.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f26d4153bc0 0x7f26d40c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:52.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f26d40c6a80 0x7f26d4160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:52.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d4153fa0 con 0x7f26d40c6610 2024-09-13T18:37:52.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d40c6e60 con 0x7f26d40c6a80 2024-09-13T18:37:52.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d40aa500 con 0x7f26d4153bc0 2024-09-13T18:37:52.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f26d40c6a80 0x7f26d4160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:52.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26ca59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f26d40c6610 0x7f26d40cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59308/0 (socket says 172.21.15.146:59308) 2024-09-13T18:37:52.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26ca59c700 1 -- 172.21.15.146:0/587326389 learned_addr learned my addr 172.21.15.146:0/587326389 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:37:52.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d2d9d700 1 --2- 172.21.15.146:0/587326389 >> v2:172.21.15.154:3300/0 conn(0x7f26d4153bc0 0x7f26d40c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:52.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d359e700 1 -- 172.21.15.146:0/587326389 >> v2:172.21.15.154:3300/0 conn(0x7f26d4153bc0 msgr2=0x7f26d40c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:52.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d359e700 1 --2- 172.21.15.146:0/587326389 >> v2:172.21.15.154:3300/0 conn(0x7f26d4153bc0 0x7f26d40c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:52.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d359e700 1 -- 172.21.15.146:0/587326389 >> v1:172.21.15.146:6789/0 conn(0x7f26d40c6610 legacy=0x7f26d40cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:52.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d359e700 1 -- 172.21.15.146:0/587326389 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d415a7e0 con 0x7f26d40c6a80 2024-09-13T18:37:52.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d259c700 1 -- 172.21.15.146:0/587326389 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156565302 0 0) 0x7f26d4153fa0 con 0x7f26d40c6610 2024-09-13T18:37:52.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d359e700 1 --2- 172.21.15.146:0/587326389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d40c6a80 0x7f26d4160c20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f26d4072ac0 tx=0x7f26cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7802d10021ff639 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:52.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d259c700 1 -- 172.21.15.146:0/587326389 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00bd20 con 0x7f26d40c6a80 2024-09-13T18:37:52.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d259c700 1 -- 172.21.15.146:0/587326389 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26cc00be80 con 0x7f26d40c6a80 2024-09-13T18:37:52.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d259c700 1 -- 172.21.15.146:0/587326389 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc0116a0 con 0x7f26d40c6a80 2024-09-13T18:37:52.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 -- 172.21.15.146:0/587326389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d40c6a80 msgr2=0x7f26d4160c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:52.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 --2- 172.21.15.146:0/587326389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d40c6a80 0x7f26d4160c20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f26d4072ac0 tx=0x7f26cc00b330 comp rx=0 tx=0).stop 2024-09-13T18:37:52.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 -- 172.21.15.146:0/587326389 shutdown_connections 2024-09-13T18:37:52.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 --2- 172.21.15.146:0/587326389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d40c6a80 0x7f26d4160c20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:52.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 --2- 172.21.15.146:0/587326389 >> v2:172.21.15.154:3300/0 conn(0x7f26d4153bc0 0x7f26d40c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:52.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 -- 172.21.15.146:0/587326389 >> 172.21.15.146:0/587326389 conn(0x7f26d4066090 msgr2=0x7f26d4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:52.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 -- 172.21.15.146:0/587326389 shutdown_connections 2024-09-13T18:37:52.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.826+0000 7f26d8971700 1 -- 172.21.15.146:0/587326389 wait complete. 2024-09-13T18:37:52.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d8971700 1 Processor -- start 2024-09-13T18:37:52.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d8971700 1 -- start start 2024-09-13T18:37:52.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d8971700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26d40c6610 0x7f26d40c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:52.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d8971700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d40c6a80 0x7f26d41ef8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:52.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d8971700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26d4153bc0 0x7f26d414f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:52.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d8971700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d40c4080 con 0x7f26d40c6a80 2024-09-13T18:37:52.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d8971700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d41ef450 con 0x7f26d4153bc0 2024-09-13T18:37:52.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d8971700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d40adc80 con 0x7f26d40c6610 2024-09-13T18:37:52.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26ca59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d40c6a80 0x7f26d41ef8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:52.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26ca59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d40c6a80 0x7f26d41ef8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50606/0 (socket says 172.21.15.146:50606) 2024-09-13T18:37:52.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26ca59c700 1 -- 172.21.15.146:0/603179385 learned_addr learned my addr 172.21.15.146:0/603179385 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:37:52.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d2d9d700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26d40c6610 0x7f26d40c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:52.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d359e700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26d4153bc0 0x7f26d414f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:52.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26ca59c700 1 -- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26d40c6610 msgr2=0x7f26d40c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:52.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26ca59c700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26d40c6610 0x7f26d40c3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:52.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26ca59c700 1 -- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26d4153bc0 msgr2=0x7f26d414f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:52.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26ca59c700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26d4153bc0 0x7f26d414f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:52.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26ca59c700 1 -- 172.21.15.146:0/603179385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc009a70 con 0x7f26d40c6a80 2024-09-13T18:37:52.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d359e700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26d4153bc0 0x7f26d414f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:37:52.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d2d9d700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26d40c6610 0x7f26d40c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:37:52.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26ca59c700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d40c6a80 0x7f26d41ef8e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00c550 tx=0x7f26bc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:52.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26cbfff700 1 -- 172.21.15.146:0/603179385 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc004ca0 con 0x7f26d40c6a80 2024-09-13T18:37:52.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26cbfff700 1 -- 172.21.15.146:0/603179385 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc004e00 con 0x7f26d40c6a80 2024-09-13T18:37:52.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d8971700 1 -- 172.21.15.146:0/603179385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d40adf20 con 0x7f26d40c6a80 2024-09-13T18:37:52.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26cbfff700 1 -- 172.21.15.146:0/603179385 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc007310 con 0x7f26d40c6a80 2024-09-13T18:37:52.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.830+0000 7f26d8971700 1 -- 172.21.15.146:0/603179385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d41ef0e0 con 0x7f26d40c6a80 2024-09-13T18:37:52.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.834+0000 7f26d8971700 1 -- 172.21.15.146:0/603179385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d4159fe0 con 0x7f26d40c6a80 2024-09-13T18:37:52.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.834+0000 7f26cbfff700 1 -- 172.21.15.146:0/603179385 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f26bc007470 con 0x7f26d40c6a80 2024-09-13T18:37:52.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.834+0000 7f26cbfff700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f26b4066730 0x7f26b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:52.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.834+0000 7f26cbfff700 1 -- 172.21.15.146:0/603179385 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f26bc084470 con 0x7f26d40c6a80 2024-09-13T18:37:52.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.834+0000 7f26d2d9d700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f26b4066730 0x7f26b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:52.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.834+0000 7f26d2d9d700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f26b4066730 0x7f26b4068bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009ad0 tx=0x7f26c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:52.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:52.842+0000 7f26cbfff700 1 -- 172.21.15.146:0/603179385 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f26bc0512b0 con 0x7f26d40c6a80 2024-09-13T18:37:53.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.050+0000 7f26d8971700 1 -- 172.21.15.146:0/603179385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26d40741f0 con 0x7f26d40c6a80 2024-09-13T18:37:53.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.050+0000 7f26cbfff700 1 -- 172.21.15.146:0/603179385 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6293 (secure 0 0 0) 0x7f26bc007720 con 0x7f26d40c6a80 2024-09-13T18:37:53.070 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:37:53.070 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5800,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:31.783229+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:37:53.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.054+0000 7f26d8971700 1 -- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f26b4066730 msgr2=0x7f26b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:53.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.054+0000 7f26d8971700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f26b4066730 0x7f26b4068bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009ad0 tx=0x7f26c4008040 comp rx=0 tx=0).stop 2024-09-13T18:37:53.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.054+0000 7f26d8971700 1 -- 172.21.15.146:0/603179385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d40c6a80 msgr2=0x7f26d41ef8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:53.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.054+0000 7f26d8971700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d40c6a80 0x7f26d41ef8e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00c550 tx=0x7f26bc00c910 comp rx=0 tx=0).stop 2024-09-13T18:37:53.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.054+0000 7f26d8971700 1 -- 172.21.15.146:0/603179385 shutdown_connections 2024-09-13T18:37:53.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.054+0000 7f26d8971700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f26b4066730 0x7f26b4068bf0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:53.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.054+0000 7f26d8971700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26d4153bc0 0x7f26d414f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:53.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.054+0000 7f26d8971700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d40c6a80 0x7f26d41ef8e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:53.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.054+0000 7f26d8971700 1 --2- 172.21.15.146:0/603179385 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26d40c6610 0x7f26d40c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:53.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.054+0000 7f26d8971700 1 -- 172.21.15.146:0/603179385 >> 172.21.15.146:0/603179385 conn(0x7f26d4066090 msgr2=0x7f26d40c09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:53.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.054+0000 7f26d8971700 1 -- 172.21.15.146:0/603179385 shutdown_connections 2024-09-13T18:37:53.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:53.054+0000 7f26d8971700 1 -- 172.21.15.146:0/603179385 wait complete. 2024-09-13T18:37:53.074 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 51 2024-09-13T18:37:53.085 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:37:55.085 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:37:55.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2395ecb700 1 Processor -- start 2024-09-13T18:37:55.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2395ecb700 1 -- start start 2024-09-13T18:37:55.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2395ecb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f23900cbd50 0x7f23900b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:55.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2395ecb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23900ca210 0x7f23900cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:55.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2395ecb700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23900cc670 con 0x7f2390071390 2024-09-13T18:37:55.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2395ecb700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23900caa30 con 0x7f23900ca210 2024-09-13T18:37:55.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2395ecb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23900cb810 con 0x7f23900cbd50 2024-09-13T18:37:55.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2387fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23900ca210 0x7f23900cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:55.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2387fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23900ca210 0x7f23900cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50616/0 (socket says 172.21.15.146:50616) 2024-09-13T18:37:55.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2387fff700 1 -- 172.21.15.146:0/658583867 learned_addr learned my addr 172.21.15.146:0/658583867 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:37:55.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2394ec9700 1 --2- 172.21.15.146:0/658583867 >> v2:172.21.15.154:3300/0 conn(0x7f23900cbd50 0x7f23900b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:55.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2387fff700 1 -- 172.21.15.146:0/658583867 >> v2:172.21.15.154:3300/0 conn(0x7f23900cbd50 msgr2=0x7f23900b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:55.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2387fff700 1 --2- 172.21.15.146:0/658583867 >> v2:172.21.15.154:3300/0 conn(0x7f23900cbd50 0x7f23900b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:55.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2387fff700 1 -- 172.21.15.146:0/658583867 >> v1:172.21.15.154:6789/0 conn(0x7f2390071390 legacy=0x7f23900701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:55.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2387fff700 1 -- 172.21.15.146:0/658583867 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239015da20 con 0x7f23900ca210 2024-09-13T18:37:55.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f2387fff700 1 --2- 172.21.15.146:0/658583867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900ca210 0x7f23900cc130 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f239008c2d0 tx=0x7f2380009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8277a424ef96716b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:55.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.190+0000 7f23877fe700 1 -- 172.21.15.146:0/658583867 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238000b8a0 con 0x7f23900ca210 2024-09-13T18:37:55.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.194+0000 7f23877fe700 1 -- 172.21.15.146:0/658583867 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238000ba00 con 0x7f23900ca210 2024-09-13T18:37:55.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.194+0000 7f23877fe700 1 -- 172.21.15.146:0/658583867 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238000bd90 con 0x7f23900ca210 2024-09-13T18:37:55.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.194+0000 7f2395ecb700 1 -- 172.21.15.146:0/658583867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900ca210 msgr2=0x7f23900cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:55.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.194+0000 7f2395ecb700 1 --2- 172.21.15.146:0/658583867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900ca210 0x7f23900cc130 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f239008c2d0 tx=0x7f2380009bf0 comp rx=0 tx=0).stop 2024-09-13T18:37:55.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.194+0000 7f2395ecb700 1 -- 172.21.15.146:0/658583867 shutdown_connections 2024-09-13T18:37:55.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.194+0000 7f2395ecb700 1 --2- 172.21.15.146:0/658583867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900ca210 0x7f23900cc130 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:55.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.194+0000 7f2395ecb700 1 --2- 172.21.15.146:0/658583867 >> v2:172.21.15.154:3300/0 conn(0x7f23900cbd50 0x7f23900b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:55.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.194+0000 7f2395ecb700 1 -- 172.21.15.146:0/658583867 >> 172.21.15.146:0/658583867 conn(0x7f2390066270 msgr2=0x7f2390066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:55.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.194+0000 7f2395ecb700 1 -- 172.21.15.146:0/658583867 shutdown_connections 2024-09-13T18:37:55.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.194+0000 7f2395ecb700 1 -- 172.21.15.146:0/658583867 wait complete. 2024-09-13T18:37:55.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.194+0000 7f2395ecb700 1 Processor -- start 2024-09-13T18:37:55.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.194+0000 7f2395ecb700 1 -- start start 2024-09-13T18:37:55.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2395ecb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2390071390 0x7f23901484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:55.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2395ecb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900ca210 0x7f2390143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:55.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2395ecb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23900cbd50 0x7f23901456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:55.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2395ecb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23901f3aa0 con 0x7f23900ca210 2024-09-13T18:37:55.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2395ecb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23901f3140 con 0x7f2390071390 2024-09-13T18:37:55.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2395ecb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239015a7c0 con 0x7f23900cbd50 2024-09-13T18:37:55.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2387fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900ca210 0x7f2390143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:55.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2387fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900ca210 0x7f2390143d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50622/0 (socket says 172.21.15.146:50622) 2024-09-13T18:37:55.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2387fff700 1 -- 172.21.15.146:0/561310541 learned_addr learned my addr 172.21.15.146:0/561310541 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:37:55.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2394ec9700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2390071390 0x7f23901484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:55.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f23956ca700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23900cbd50 0x7f23901456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:55.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2387fff700 1 -- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23900cbd50 msgr2=0x7f23901456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:55.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2387fff700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23900cbd50 0x7f23901456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:55.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2387fff700 1 -- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2390071390 msgr2=0x7f23901484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:55.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2387fff700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2390071390 0x7f23901484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:55.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2387fff700 1 -- 172.21.15.146:0/561310541 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2380009570 con 0x7f23900ca210 2024-09-13T18:37:55.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2387fff700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900ca210 0x7f2390143d70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f23900cb990 tx=0x7f2380004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:55.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f23857fa700 1 -- 172.21.15.146:0/561310541 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23800036c0 con 0x7f23900ca210 2024-09-13T18:37:55.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f23857fa700 1 -- 172.21.15.146:0/561310541 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238001f070 con 0x7f23900ca210 2024-09-13T18:37:55.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2395ecb700 1 -- 172.21.15.146:0/561310541 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239015aaa0 con 0x7f23900ca210 2024-09-13T18:37:55.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f23857fa700 1 -- 172.21.15.146:0/561310541 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380003a80 con 0x7f23900ca210 2024-09-13T18:37:55.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2395ecb700 1 -- 172.21.15.146:0/561310541 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23901f36c0 con 0x7f23900ca210 2024-09-13T18:37:55.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.198+0000 7f2395ecb700 1 -- 172.21.15.146:0/561310541 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23901f3be0 con 0x7f23900ca210 2024-09-13T18:37:55.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.202+0000 7f23857fa700 1 -- 172.21.15.146:0/561310541 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2380021030 con 0x7f23900ca210 2024-09-13T18:37:55.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.202+0000 7f23857fa700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f237c066730 0x7f237c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:55.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.202+0000 7f23857fa700 1 -- 172.21.15.146:0/561310541 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f2380085bd0 con 0x7f23900ca210 2024-09-13T18:37:55.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.202+0000 7f2394ec9700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f237c066730 0x7f237c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:55.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.202+0000 7f2394ec9700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f237c066730 0x7f237c068bf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f238c007a60 tx=0x7f238c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:55.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.210+0000 7f23857fa700 1 -- 172.21.15.146:0/561310541 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f23901f3be0 con 0x7f23900ca210 2024-09-13T18:37:55.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.418+0000 7f2395ecb700 1 -- 172.21.15.146:0/561310541 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23901f3be0 con 0x7f23900ca210 2024-09-13T18:37:55.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.418+0000 7f23857fa700 1 -- 172.21.15.146:0/561310541 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6293 (secure 0 0 0) 0x7f23901f3be0 con 0x7f23900ca210 2024-09-13T18:37:55.437 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:37:55.437 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5800,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:31.783229+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:37:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.422+0000 7f2395ecb700 1 -- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f237c066730 msgr2=0x7f237c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.422+0000 7f2395ecb700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f237c066730 0x7f237c068bf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f238c007a60 tx=0x7f238c008040 comp rx=0 tx=0).stop 2024-09-13T18:37:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.422+0000 7f2395ecb700 1 -- 172.21.15.146:0/561310541 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900ca210 msgr2=0x7f2390143d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.422+0000 7f2395ecb700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900ca210 0x7f2390143d70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f23900cb990 tx=0x7f2380004770 comp rx=0 tx=0).stop 2024-09-13T18:37:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.422+0000 7f2395ecb700 1 -- 172.21.15.146:0/561310541 shutdown_connections 2024-09-13T18:37:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.422+0000 7f2395ecb700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f237c066730 0x7f237c068bf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.422+0000 7f2395ecb700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23900cbd50 0x7f23901456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.422+0000 7f2395ecb700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900ca210 0x7f2390143d70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.422+0000 7f2395ecb700 1 --2- 172.21.15.146:0/561310541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2390071390 0x7f23901484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:55.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.422+0000 7f2395ecb700 1 -- 172.21.15.146:0/561310541 >> 172.21.15.146:0/561310541 conn(0x7f2390066270 msgr2=0x7f2390066650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:55.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.422+0000 7f2395ecb700 1 -- 172.21.15.146:0/561310541 shutdown_connections 2024-09-13T18:37:55.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:55.422+0000 7f2395ecb700 1 -- 172.21.15.146:0/561310541 wait complete. 2024-09-13T18:37:55.442 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 51 2024-09-13T18:37:55.451 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:37:57.452 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:37:57.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff993147700 1 Processor -- start 2024-09-13T18:37:57.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff993147700 1 -- start start 2024-09-13T18:37:57.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff993147700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff98c153bc0 0x7ff98c0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:57.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff993147700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff98c0c6a80 0x7ff98c160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:57.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff993147700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98c153fa0 con 0x7ff98c0c6610 2024-09-13T18:37:57.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff993147700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff98c0aa500 con 0x7ff98c153bc0 2024-09-13T18:37:57.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff993147700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c15a7e0 con 0x7ff98c0c6a80 2024-09-13T18:37:57.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff992145700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff98c153bc0 0x7ff98c0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:57.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff992946700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff98c0c6a80 0x7ff98c160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:57.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff991944700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff98c0c6610 0x7ff98c0cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:44454/0 (socket says 172.21.15.146:44454) 2024-09-13T18:37:57.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff991944700 1 -- 172.21.15.146:0/3438951711 learned_addr learned my addr 172.21.15.146:0/3438951711 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:37:57.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff991143700 1 -- 172.21.15.146:0/3438951711 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019762167 0 0) 0x7ff98c153fa0 con 0x7ff98c0c6610 2024-09-13T18:37:57.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff991143700 1 -- 172.21.15.146:0/3438951711 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9780035d0 con 0x7ff98c0c6610 2024-09-13T18:37:57.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff992946700 1 -- 172.21.15.146:0/3438951711 >> v2:172.21.15.154:3300/0 conn(0x7ff98c153bc0 msgr2=0x7ff98c0c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:57.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff992946700 1 --2- 172.21.15.146:0/3438951711 >> v2:172.21.15.154:3300/0 conn(0x7ff98c153bc0 0x7ff98c0c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:57.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff992946700 1 -- 172.21.15.146:0/3438951711 >> v1:172.21.15.154:6789/0 conn(0x7ff98c0c6610 legacy=0x7ff98c0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:57.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff992946700 1 -- 172.21.15.146:0/3438951711 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c0c9320 con 0x7ff98c0c6a80 2024-09-13T18:37:57.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.558+0000 7ff992145700 1 --2- 172.21.15.146:0/3438951711 >> v2:172.21.15.154:3300/0 conn(0x7ff98c153bc0 0x7ff98c0c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:37:57.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff992946700 1 --2- 172.21.15.146:0/3438951711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff98c0c6a80 0x7ff98c160c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff984009d90 tx=0x7ff98400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf00f3beb338d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:57.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff991143700 1 -- 172.21.15.146:0/3438951711 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98400e070 con 0x7ff98c0c6a80 2024-09-13T18:37:57.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff991143700 1 -- 172.21.15.146:0/3438951711 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98400bb20 con 0x7ff98c0c6a80 2024-09-13T18:37:57.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff991143700 1 -- 172.21.15.146:0/3438951711 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984012430 con 0x7ff98c0c6a80 2024-09-13T18:37:57.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff993147700 1 -- 172.21.15.146:0/3438951711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff98c0c6a80 msgr2=0x7ff98c160c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:57.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff993147700 1 --2- 172.21.15.146:0/3438951711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff98c0c6a80 0x7ff98c160c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff984009d90 tx=0x7ff98400b330 comp rx=0 tx=0).stop 2024-09-13T18:37:57.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff993147700 1 -- 172.21.15.146:0/3438951711 shutdown_connections 2024-09-13T18:37:57.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff993147700 1 --2- 172.21.15.146:0/3438951711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff98c0c6a80 0x7ff98c160c20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:57.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff993147700 1 --2- 172.21.15.146:0/3438951711 >> v2:172.21.15.154:3300/0 conn(0x7ff98c153bc0 0x7ff98c0c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:57.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff993147700 1 -- 172.21.15.146:0/3438951711 >> 172.21.15.146:0/3438951711 conn(0x7ff98c066090 msgr2=0x7ff98c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:57.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff993147700 1 -- 172.21.15.146:0/3438951711 shutdown_connections 2024-09-13T18:37:57.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff993147700 1 -- 172.21.15.146:0/3438951711 wait complete. 2024-09-13T18:37:57.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff993147700 1 Processor -- start 2024-09-13T18:37:57.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.562+0000 7ff993147700 1 -- start start 2024-09-13T18:37:57.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff993147700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff98c0c6610 0x7ff98c150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff993147700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff98c0c6a80 0x7ff98c0aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff993147700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff98c153bc0 0x7ff98c0ab480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff993147700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c14c9f0 con 0x7ff98c0c6610 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff993147700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c14c090 con 0x7ff98c153bc0 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff993147700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c1f2b30 con 0x7ff98c0c6a80 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff992145700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff98c0c6610 0x7ff98c150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff992145700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff98c0c6610 0x7ff98c150340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50626/0 (socket says 172.21.15.146:50626) 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff992145700 1 -- 172.21.15.146:0/674478524 learned_addr learned my addr 172.21.15.146:0/674478524 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff991944700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff98c0c6a80 0x7ff98c0aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff992946700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff98c153bc0 0x7ff98c0ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff992145700 1 -- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff98c0c6a80 msgr2=0x7ff98c0aaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff992145700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff98c0c6a80 0x7ff98c0aaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff992145700 1 -- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff98c153bc0 msgr2=0x7ff98c0ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:57.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff992145700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff98c153bc0 0x7ff98c0ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:57.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff992145700 1 -- 172.21.15.146:0/674478524 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800094b0 con 0x7ff98c0c6610 2024-09-13T18:37:57.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff992145700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff98c0c6610 0x7ff98c150340 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff98000ebe0 tx=0x7ff98000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:57.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff97effd700 1 -- 172.21.15.146:0/674478524 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980009770 con 0x7ff98c0c6610 2024-09-13T18:37:57.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff993147700 1 -- 172.21.15.146:0/674478524 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff984009a70 con 0x7ff98c0c6610 2024-09-13T18:37:57.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff97effd700 1 -- 172.21.15.146:0/674478524 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98000c850 con 0x7ff98c0c6610 2024-09-13T18:37:57.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff97effd700 1 -- 172.21.15.146:0/674478524 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98000cbe0 con 0x7ff98c0c6610 2024-09-13T18:37:57.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff993147700 1 -- 172.21.15.146:0/674478524 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98c14c610 con 0x7ff98c0c6610 2024-09-13T18:37:57.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.566+0000 7ff993147700 1 -- 172.21.15.146:0/674478524 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98c159a00 con 0x7ff98c0c6610 2024-09-13T18:37:57.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.570+0000 7ff97effd700 1 -- 172.21.15.146:0/674478524 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff98000cd40 con 0x7ff98c0c6610 2024-09-13T18:37:57.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.570+0000 7ff97effd700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff970066760 0x7ff970068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:57.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.570+0000 7ff97effd700 1 -- 172.21.15.146:0/674478524 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7ff980084a90 con 0x7ff98c0c6610 2024-09-13T18:37:57.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.570+0000 7ff991944700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff970066760 0x7ff970068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:57.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.570+0000 7ff991944700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff970066760 0x7ff970068c20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff98c1efb60 tx=0x7ff988007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:57.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.574+0000 7ff97effd700 1 -- 172.21.15.146:0/674478524 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff98c159a00 con 0x7ff98c0c6610 2024-09-13T18:37:57.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.786+0000 7ff993147700 1 -- 172.21.15.146:0/674478524 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff98c159a00 con 0x7ff98c0c6610 2024-09-13T18:37:57.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.786+0000 7ff97effd700 1 -- 172.21.15.146:0/674478524 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6293 (secure 0 0 0) 0x7ff98c159a00 con 0x7ff98c0c6610 2024-09-13T18:37:57.805 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:37:57.805 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5800,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:31.783229+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:37:57.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.790+0000 7ff993147700 1 -- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff970066760 msgr2=0x7ff970068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:57.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.790+0000 7ff993147700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff970066760 0x7ff970068c20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff98c1efb60 tx=0x7ff988007de0 comp rx=0 tx=0).stop 2024-09-13T18:37:57.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.790+0000 7ff993147700 1 -- 172.21.15.146:0/674478524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff98c0c6610 msgr2=0x7ff98c150340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:57.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.790+0000 7ff993147700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff98c0c6610 0x7ff98c150340 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff98000ebe0 tx=0x7ff98000eef0 comp rx=0 tx=0).stop 2024-09-13T18:37:57.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.790+0000 7ff993147700 1 -- 172.21.15.146:0/674478524 shutdown_connections 2024-09-13T18:37:57.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.790+0000 7ff993147700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff970066760 0x7ff970068c20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:57.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.790+0000 7ff993147700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff98c153bc0 0x7ff98c0ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:57.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.790+0000 7ff993147700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff98c0c6a80 0x7ff98c0aaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:57.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.790+0000 7ff993147700 1 --2- 172.21.15.146:0/674478524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff98c0c6610 0x7ff98c150340 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:57.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.790+0000 7ff993147700 1 -- 172.21.15.146:0/674478524 >> 172.21.15.146:0/674478524 conn(0x7ff98c066090 msgr2=0x7ff98c0c03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:57.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.790+0000 7ff993147700 1 -- 172.21.15.146:0/674478524 shutdown_connections 2024-09-13T18:37:57.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:57.790+0000 7ff993147700 1 -- 172.21.15.146:0/674478524 wait complete. 2024-09-13T18:37:57.809 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 51 2024-09-13T18:37:57.820 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:37:59.821 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:37:59.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 Processor -- start 2024-09-13T18:37:59.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 -- start start 2024-09-13T18:37:59.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f96b0153bc0 0x7f96b00c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:59.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96b00c6a80 0x7f96b0160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:59.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b0153fa0 con 0x7f96b00c6610 2024-09-13T18:37:59.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b00c6e60 con 0x7f96b00c6a80 2024-09-13T18:37:59.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b00aa500 con 0x7f96b0153bc0 2024-09-13T18:37:59.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b6ad9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96b00c6a80 0x7f96b0160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:59.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b6ad9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96b00c6a80 0x7f96b0160c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50640/0 (socket says 172.21.15.146:50640) 2024-09-13T18:37:59.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b6ad9700 1 -- 172.21.15.146:0/1236624866 learned_addr learned my addr 172.21.15.146:0/1236624866 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:37:59.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b5ad7700 1 --1- 172.21.15.146:0/1236624866 >> v1:172.21.15.146:6789/0 conn(0x7f96b00c6610 0x7f96b00cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59322/0 (socket says 172.21.15.146:59322) 2024-09-13T18:37:59.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b62d8700 1 --2- 172.21.15.146:0/1236624866 >> v2:172.21.15.154:3300/0 conn(0x7f96b0153bc0 0x7f96b00c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:59.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b6ad9700 1 -- 172.21.15.146:0/1236624866 >> v2:172.21.15.154:3300/0 conn(0x7f96b0153bc0 msgr2=0x7f96b00c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:37:59.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b6ad9700 1 --2- 172.21.15.146:0/1236624866 >> v2:172.21.15.154:3300/0 conn(0x7f96b0153bc0 0x7f96b00c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:59.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b6ad9700 1 -- 172.21.15.146:0/1236624866 >> v1:172.21.15.146:6789/0 conn(0x7f96b00c6610 legacy=0x7f96b00cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:59.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b6ad9700 1 -- 172.21.15.146:0/1236624866 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b015a7e0 con 0x7f96b00c6a80 2024-09-13T18:37:59.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b6ad9700 1 --2- 172.21.15.146:0/1236624866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b00c6a80 0x7f96b0160c20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f96b0072ac0 tx=0x7f96a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f0580f5e8e391 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:59.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b52d6700 1 -- 172.21.15.146:0/1236624866 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a800e070 con 0x7f96b00c6a80 2024-09-13T18:37:59.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b52d6700 1 -- 172.21.15.146:0/1236624866 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a800b910 con 0x7f96b00c6a80 2024-09-13T18:37:59.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b52d6700 1 -- 172.21.15.146:0/1236624866 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a800bcc0 con 0x7f96b00c6a80 2024-09-13T18:37:59.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 -- 172.21.15.146:0/1236624866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b00c6a80 msgr2=0x7f96b0160c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:59.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 --2- 172.21.15.146:0/1236624866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b00c6a80 0x7f96b0160c20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f96b0072ac0 tx=0x7f96a800b330 comp rx=0 tx=0).stop 2024-09-13T18:37:59.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 -- 172.21.15.146:0/1236624866 shutdown_connections 2024-09-13T18:37:59.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 --2- 172.21.15.146:0/1236624866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b00c6a80 0x7f96b0160c20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:59.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 --2- 172.21.15.146:0/1236624866 >> v2:172.21.15.154:3300/0 conn(0x7f96b0153bc0 0x7f96b00c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:59.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 -- 172.21.15.146:0/1236624866 >> 172.21.15.146:0/1236624866 conn(0x7f96b0066090 msgr2=0x7f96b0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:37:59.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 -- 172.21.15.146:0/1236624866 shutdown_connections 2024-09-13T18:37:59.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.926+0000 7f96b72da700 1 -- 172.21.15.146:0/1236624866 wait complete. 2024-09-13T18:37:59.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b72da700 1 Processor -- start 2024-09-13T18:37:59.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b72da700 1 -- start start 2024-09-13T18:37:59.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b72da700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96b00c6610 0x7f96b006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:59.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b72da700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b00c6a80 0x7f96b00aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:59.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b72da700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96b0153bc0 0x7f96b00acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:59.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b72da700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b01eed70 con 0x7f96b00c6a80 2024-09-13T18:37:59.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b72da700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b01eef60 con 0x7f96b0153bc0 2024-09-13T18:37:59.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b72da700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b01ef240 con 0x7f96b00c6610 2024-09-13T18:37:59.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b5ad7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b00c6a80 0x7f96b00aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:59.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b5ad7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b00c6a80 0x7f96b00aca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50652/0 (socket says 172.21.15.146:50652) 2024-09-13T18:37:59.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b5ad7700 1 -- 172.21.15.146:0/1663629516 learned_addr learned my addr 172.21.15.146:0/1663629516 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:37:59.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b6ad9700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96b0153bc0 0x7f96b00acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:59.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b62d8700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96b00c6610 0x7f96b006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:59.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b5ad7700 1 -- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96b00c6610 msgr2=0x7f96b006d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:59.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b5ad7700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96b00c6610 0x7f96b006d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:59.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b5ad7700 1 -- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96b0153bc0 msgr2=0x7f96b00acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:37:59.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b5ad7700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96b0153bc0 0x7f96b00acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:37:59.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b5ad7700 1 -- 172.21.15.146:0/1663629516 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a8009b30 con 0x7f96b00c6a80 2024-09-13T18:37:59.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b62d8700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96b00c6610 0x7f96b006d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:37:59.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b5ad7700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b00c6a80 0x7f96b00aca60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00c700 tx=0x7f96ac00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:59.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96a6ffd700 1 -- 172.21.15.146:0/1663629516 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac005cd0 con 0x7f96b00c6a80 2024-09-13T18:37:59.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96a6ffd700 1 -- 172.21.15.146:0/1663629516 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac005e30 con 0x7f96b00c6a80 2024-09-13T18:37:59.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b72da700 1 -- 172.21.15.146:0/1663629516 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b01ef520 con 0x7f96b00c6a80 2024-09-13T18:37:59.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96a6ffd700 1 -- 172.21.15.146:0/1663629516 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac0072d0 con 0x7f96b00c6a80 2024-09-13T18:37:59.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.930+0000 7f96b72da700 1 -- 172.21.15.146:0/1663629516 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b01efae0 con 0x7f96b00c6a80 2024-09-13T18:37:59.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.934+0000 7f96b72da700 1 -- 172.21.15.146:0/1663629516 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b01efc70 con 0x7f96b00c6a80 2024-09-13T18:37:59.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.934+0000 7f96a6ffd700 1 -- 172.21.15.146:0/1663629516 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f96ac007430 con 0x7f96b00c6a80 2024-09-13T18:37:59.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.934+0000 7f96a6ffd700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9694066730 0x7f9694068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:37:59.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.934+0000 7f96a6ffd700 1 -- 172.21.15.146:0/1663629516 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f96ac083a10 con 0x7f96b00c6a80 2024-09-13T18:37:59.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.934+0000 7f96b62d8700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9694066730 0x7f9694068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:37:59.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.938+0000 7f96b62d8700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9694066730 0x7f9694068bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009ba0 tx=0x7f96a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:37:59.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:37:59.942+0000 7f96a6ffd700 1 -- 172.21.15.146:0/1663629516 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f96ac050900 con 0x7f96b00c6a80 2024-09-13T18:38:00.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.158+0000 7f96b72da700 1 -- 172.21.15.146:0/1663629516 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96b0159fe0 con 0x7f96b00c6a80 2024-09-13T18:38:00.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.158+0000 7f96a6ffd700 1 -- 172.21.15.146:0/1663629516 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6293 (secure 0 0 0) 0x7f96ac017090 con 0x7f96b00c6a80 2024-09-13T18:38:00.179 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:00.179 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5800,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:36:31.783229+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5505,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6835/1652676664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1652676664},{"type":"v1","addr":"172.21.15.146:6835","nonce":1652676664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:00.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.162+0000 7f96b72da700 1 -- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9694066730 msgr2=0x7f9694068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:00.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.162+0000 7f96b72da700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9694066730 0x7f9694068bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009ba0 tx=0x7f96a0008040 comp rx=0 tx=0).stop 2024-09-13T18:38:00.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.162+0000 7f96b72da700 1 -- 172.21.15.146:0/1663629516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b00c6a80 msgr2=0x7f96b00aca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:00.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.162+0000 7f96b72da700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b00c6a80 0x7f96b00aca60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00c700 tx=0x7f96ac00ca10 comp rx=0 tx=0).stop 2024-09-13T18:38:00.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.162+0000 7f96b72da700 1 -- 172.21.15.146:0/1663629516 shutdown_connections 2024-09-13T18:38:00.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.162+0000 7f96b72da700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9694066730 0x7f9694068bf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:00.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.162+0000 7f96b72da700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96b0153bc0 0x7f96b00acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:00.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.162+0000 7f96b72da700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b00c6a80 0x7f96b00aca60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:00.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.162+0000 7f96b72da700 1 --2- 172.21.15.146:0/1663629516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96b00c6610 0x7f96b006d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:00.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.162+0000 7f96b72da700 1 -- 172.21.15.146:0/1663629516 >> 172.21.15.146:0/1663629516 conn(0x7f96b0066090 msgr2=0x7f96b00c7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:00.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.162+0000 7f96b72da700 1 -- 172.21.15.146:0/1663629516 shutdown_connections 2024-09-13T18:38:00.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:00.162+0000 7f96b72da700 1 -- 172.21.15.146:0/1663629516 wait complete. 2024-09-13T18:38:00.182 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 51 2024-09-13T18:38:00.193 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:38:02.194 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:38:02.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.298+0000 7fc538f1b700 1 Processor -- start 2024-09-13T18:38:02.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.298+0000 7fc538f1b700 1 -- start start 2024-09-13T18:38:02.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.298+0000 7fc538f1b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc534153bc0 0x7fc5340c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:02.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.298+0000 7fc538f1b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5340c6a80 0x7fc534160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:02.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.298+0000 7fc538f1b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc534153fa0 con 0x7fc5340c6610 2024-09-13T18:38:02.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.298+0000 7fc538f1b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5340aa500 con 0x7fc534153bc0 2024-09-13T18:38:02.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.298+0000 7fc538f1b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53415a7e0 con 0x7fc5340c6a80 2024-09-13T18:38:02.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.298+0000 7fc532ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc5340c6610 0x7fc5340cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58402/0 (socket says 172.21.15.146:58402) 2024-09-13T18:38:02.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.298+0000 7fc532ffd700 1 -- 172.21.15.146:0/2674611731 learned_addr learned my addr 172.21.15.146:0/2674611731 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:38:02.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc5337fe700 1 --2- 172.21.15.146:0/2674611731 >> v2:172.21.15.146:3300/0 conn(0x7fc534153bc0 0x7fc5340c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:02.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc533fff700 1 --2- 172.21.15.146:0/2674611731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5340c6a80 0x7fc534160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:02.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc5327fc700 1 -- 172.21.15.146:0/2674611731 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 543991628 0 0) 0x7fc534153fa0 con 0x7fc5340c6610 2024-09-13T18:38:02.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc5327fc700 1 -- 172.21.15.146:0/2674611731 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc51c0035d0 con 0x7fc5340c6610 2024-09-13T18:38:02.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc5337fe700 1 -- 172.21.15.146:0/2674611731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5340c6a80 msgr2=0x7fc534160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:02.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc5337fe700 1 --2- 172.21.15.146:0/2674611731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5340c6a80 0x7fc534160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc5337fe700 1 -- 172.21.15.146:0/2674611731 >> v1:172.21.15.146:6789/0 conn(0x7fc5340c6610 legacy=0x7fc5340cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:02.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc5337fe700 1 -- 172.21.15.146:0/2674611731 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5340c9320 con 0x7fc534153bc0 2024-09-13T18:38:02.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc5337fe700 1 --2- 172.21.15.146:0/2674611731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc534153bc0 0x7fc5340c8de0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc524009850 tx=0x7fc524009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff4908b62ab9a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:02.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc5327fc700 1 -- 172.21.15.146:0/2674611731 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524011830 con 0x7fc534153bc0 2024-09-13T18:38:02.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc5327fc700 1 -- 172.21.15.146:0/2674611731 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc524011990 con 0x7fc534153bc0 2024-09-13T18:38:02.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc5327fc700 1 -- 172.21.15.146:0/2674611731 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524011d00 con 0x7fc534153bc0 2024-09-13T18:38:02.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc538f1b700 1 -- 172.21.15.146:0/2674611731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc534153bc0 msgr2=0x7fc5340c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:02.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc538f1b700 1 --2- 172.21.15.146:0/2674611731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc534153bc0 0x7fc5340c8de0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc524009850 tx=0x7fc524009b60 comp rx=0 tx=0).stop 2024-09-13T18:38:02.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc538f1b700 1 -- 172.21.15.146:0/2674611731 shutdown_connections 2024-09-13T18:38:02.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc538f1b700 1 --2- 172.21.15.146:0/2674611731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5340c6a80 0x7fc534160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc538f1b700 1 --2- 172.21.15.146:0/2674611731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc534153bc0 0x7fc5340c8de0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc538f1b700 1 -- 172.21.15.146:0/2674611731 >> 172.21.15.146:0/2674611731 conn(0x7fc534066090 msgr2=0x7fc534066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:02.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc538f1b700 1 -- 172.21.15.146:0/2674611731 shutdown_connections 2024-09-13T18:38:02.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.302+0000 7fc538f1b700 1 -- 172.21.15.146:0/2674611731 wait complete. 2024-09-13T18:38:02.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.306+0000 7fc538f1b700 1 Processor -- start 2024-09-13T18:38:02.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.306+0000 7fc538f1b700 1 -- start start 2024-09-13T18:38:02.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.306+0000 7fc538f1b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5340c6610 0x7fc5341ec3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:02.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.306+0000 7fc538f1b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5340c6a80 0x7fc5340c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:02.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.306+0000 7fc538f1b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc534153bc0 0x7fc5340ac620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:02.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.306+0000 7fc538f1b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5341ec910 con 0x7fc5340c6610 2024-09-13T18:38:02.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.306+0000 7fc538f1b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340c2f90 con 0x7fc534153bc0 2024-09-13T18:38:02.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.306+0000 7fc538f1b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340c3270 con 0x7fc5340c6a80 2024-09-13T18:38:02.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc5337fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5340c6610 0x7fc5341ec3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:02.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc5337fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5340c6610 0x7fc5341ec3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51816/0 (socket says 172.21.15.146:51816) 2024-09-13T18:38:02.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc5337fe700 1 -- 172.21.15.146:0/179403624 learned_addr learned my addr 172.21.15.146:0/179403624 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:02.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc532ffd700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5340c6a80 0x7fc5340c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:02.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc533fff700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc534153bc0 0x7fc5340ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:02.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc5337fe700 1 -- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5340c6a80 msgr2=0x7fc5340c1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:02.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc5337fe700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5340c6a80 0x7fc5340c1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc5337fe700 1 -- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc534153bc0 msgr2=0x7fc5340ac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:02.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc5337fe700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc534153bc0 0x7fc5340ac620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc5337fe700 1 -- 172.21.15.146:0/179403624 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240094b0 con 0x7fc5340c6610 2024-09-13T18:38:02.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc533fff700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc534153bc0 0x7fc5340ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:38:02.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc5337fe700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5340c6610 0x7fc5341ec3d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc5240090b0 tx=0x7fc524004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:02.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc51bfff700 1 -- 172.21.15.146:0/179403624 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524004550 con 0x7fc5340c6610 2024-09-13T18:38:02.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc51bfff700 1 -- 172.21.15.146:0/179403624 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52401e070 con 0x7fc5340c6610 2024-09-13T18:38:02.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc538f1b700 1 -- 172.21.15.146:0/179403624 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5340ab860 con 0x7fc5340c6610 2024-09-13T18:38:02.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc51bfff700 1 -- 172.21.15.146:0/179403624 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524003900 con 0x7fc5340c6610 2024-09-13T18:38:02.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc538f1b700 1 -- 172.21.15.146:0/179403624 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5340abe20 con 0x7fc5340c6610 2024-09-13T18:38:02.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc538f1b700 1 -- 172.21.15.146:0/179403624 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc514005310 con 0x7fc5340c6610 2024-09-13T18:38:02.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.310+0000 7fc51bfff700 1 -- 172.21.15.146:0/179403624 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc524003a60 con 0x7fc5340c6610 2024-09-13T18:38:02.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.314+0000 7fc51bfff700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc520066730 0x7fc520068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:02.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.314+0000 7fc51bfff700 1 -- 172.21.15.146:0/179403624 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fc5240856e0 con 0x7fc5340c6610 2024-09-13T18:38:02.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.314+0000 7fc532ffd700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc520066730 0x7fc520068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:02.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.314+0000 7fc532ffd700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc520066730 0x7fc520068bf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc5340ad330 tx=0x7fc528009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:02.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.318+0000 7fc51bfff700 1 -- 172.21.15.146:0/179403624 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc5240523b0 con 0x7fc5340c6610 2024-09-13T18:38:02.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.526+0000 7fc538f1b700 1 -- 172.21.15.146:0/179403624 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5140055c0 con 0x7fc5340c6610 2024-09-13T18:38:02.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.526+0000 7fc51bfff700 1 -- 172.21.15.146:0/179403624 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5532 (secure 0 0 0) 0x7fc5240559d0 con 0x7fc5340c6610 2024-09-13T18:38:02.545 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:02.545 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:00.700447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:02.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.530+0000 7fc538f1b700 1 -- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc520066730 msgr2=0x7fc520068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:02.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.530+0000 7fc538f1b700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc520066730 0x7fc520068bf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc5340ad330 tx=0x7fc528009040 comp rx=0 tx=0).stop 2024-09-13T18:38:02.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.530+0000 7fc538f1b700 1 -- 172.21.15.146:0/179403624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5340c6610 msgr2=0x7fc5341ec3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:02.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.530+0000 7fc538f1b700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5340c6610 0x7fc5341ec3d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc5240090b0 tx=0x7fc524004770 comp rx=0 tx=0).stop 2024-09-13T18:38:02.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.530+0000 7fc538f1b700 1 -- 172.21.15.146:0/179403624 shutdown_connections 2024-09-13T18:38:02.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.530+0000 7fc538f1b700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc520066730 0x7fc520068bf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.530+0000 7fc538f1b700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc534153bc0 0x7fc5340ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.530+0000 7fc538f1b700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5340c6a80 0x7fc5340c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.530+0000 7fc538f1b700 1 --2- 172.21.15.146:0/179403624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5340c6610 0x7fc5341ec3d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.530+0000 7fc538f1b700 1 -- 172.21.15.146:0/179403624 >> 172.21.15.146:0/179403624 conn(0x7fc534066090 msgr2=0x7fc5340c7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:02.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.530+0000 7fc538f1b700 1 -- 172.21.15.146:0/179403624 shutdown_connections 2024-09-13T18:38:02.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.530+0000 7fc538f1b700 1 -- 172.21.15.146:0/179403624 wait complete. 2024-09-13T18:38:02.549 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 52 2024-09-13T18:38:02.560 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-09-13T18:38:02.560 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:38:02.560 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:38:02.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.650+0000 7fbc784b6700 1 Processor -- start 2024-09-13T18:38:02.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.650+0000 7fbc784b6700 1 -- start start 2024-09-13T18:38:02.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc784b6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbc7006df50 0x7fbc700c0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:02.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc784b6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700c1440 0x7fbc700c1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:02.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc784b6700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc70156bf0 con 0x7fbc7006e330 2024-09-13T18:38:02.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc784b6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc700cc020 con 0x7fbc7006df50 2024-09-13T18:38:02.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc784b6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc700c1dd0 con 0x7fbc700c1440 2024-09-13T18:38:02.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc75a51700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fbc7006e330 0x7fbc7006fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58418/0 (socket says 172.21.15.146:58418) 2024-09-13T18:38:02.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc75a51700 1 -- 172.21.15.146:0/3364291889 learned_addr learned my addr 172.21.15.146:0/3364291889 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:38:02.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc76a53700 1 --2- 172.21.15.146:0/3364291889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700c1440 0x7fbc700c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:02.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc76252700 1 --2- 172.21.15.146:0/3364291889 >> v2:172.21.15.154:3300/0 conn(0x7fbc7006df50 0x7fbc700c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:02.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc75250700 1 -- 172.21.15.146:0/3364291889 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678784410 0 0) 0x7fbc70156bf0 con 0x7fbc7006e330 2024-09-13T18:38:02.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc75250700 1 -- 172.21.15.146:0/3364291889 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc5c0035d0 con 0x7fbc7006e330 2024-09-13T18:38:02.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc75250700 1 -- 172.21.15.146:0/3364291889 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1879068605 0 0) 0x7fbc5c0035d0 con 0x7fbc7006e330 2024-09-13T18:38:02.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc76a53700 1 -- 172.21.15.146:0/3364291889 >> v2:172.21.15.154:3300/0 conn(0x7fbc7006df50 msgr2=0x7fbc700c0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:02.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc76a53700 1 --2- 172.21.15.146:0/3364291889 >> v2:172.21.15.154:3300/0 conn(0x7fbc7006df50 0x7fbc700c0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc76a53700 1 -- 172.21.15.146:0/3364291889 >> v1:172.21.15.146:6789/0 conn(0x7fbc7006e330 legacy=0x7fbc7006fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:02.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc76a53700 1 -- 172.21.15.146:0/3364291889 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc70070560 con 0x7fbc700c1440 2024-09-13T18:38:02.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc76a53700 1 --2- 172.21.15.146:0/3364291889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700c1440 0x7fbc700c1890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009d90 tx=0x7fbc6800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9022ff659306fd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:02.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc75250700 1 -- 172.21.15.146:0/3364291889 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6800e070 con 0x7fbc700c1440 2024-09-13T18:38:02.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc75250700 1 -- 172.21.15.146:0/3364291889 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc68003a40 con 0x7fbc700c1440 2024-09-13T18:38:02.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc75250700 1 -- 172.21.15.146:0/3364291889 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c001670 con 0x7fbc700c1440 2024-09-13T18:38:02.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc784b6700 1 -- 172.21.15.146:0/3364291889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700c1440 msgr2=0x7fbc700c1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:02.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc784b6700 1 --2- 172.21.15.146:0/3364291889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700c1440 0x7fbc700c1890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009d90 tx=0x7fbc6800b330 comp rx=0 tx=0).stop 2024-09-13T18:38:02.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc784b6700 1 -- 172.21.15.146:0/3364291889 shutdown_connections 2024-09-13T18:38:02.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc784b6700 1 --2- 172.21.15.146:0/3364291889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc700c1440 0x7fbc700c1890 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc784b6700 1 --2- 172.21.15.146:0/3364291889 >> v2:172.21.15.154:3300/0 conn(0x7fbc7006df50 0x7fbc700c0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc784b6700 1 -- 172.21.15.146:0/3364291889 >> 172.21.15.146:0/3364291889 conn(0x7fbc700cb1d0 msgr2=0x7fbc700cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:02.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc784b6700 1 -- 172.21.15.146:0/3364291889 shutdown_connections 2024-09-13T18:38:02.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.654+0000 7fbc784b6700 1 -- 172.21.15.146:0/3364291889 wait complete. 2024-09-13T18:38:02.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.658+0000 7fbc784b6700 1 Processor -- start 2024-09-13T18:38:02.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.658+0000 7fbc784b6700 1 -- start start 2024-09-13T18:38:02.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc784b6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc7006df50 0x7fbc700c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:02.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc784b6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc7006e330 0x7fbc701ebf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:02.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc784b6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc700c1440 0x7fbc701ec460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:02.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc784b6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70151880 con 0x7fbc7006df50 2024-09-13T18:38:02.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc784b6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc701ebb00 con 0x7fbc700c1440 2024-09-13T18:38:02.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc784b6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70152fd0 con 0x7fbc7006e330 2024-09-13T18:38:02.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc76252700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc7006df50 0x7fbc700c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:02.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc76252700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc7006df50 0x7fbc700c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51828/0 (socket says 172.21.15.146:51828) 2024-09-13T18:38:02.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc76252700 1 -- 172.21.15.146:0/980851800 learned_addr learned my addr 172.21.15.146:0/980851800 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:02.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc75a51700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc7006e330 0x7fbc701ebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:02.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc76a53700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc700c1440 0x7fbc701ec460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:02.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc76252700 1 -- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc7006e330 msgr2=0x7fbc701ebf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:02.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc76252700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc7006e330 0x7fbc701ebf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc76252700 1 -- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc700c1440 msgr2=0x7fbc701ec460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:02.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc76252700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc700c1440 0x7fbc701ec460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc76252700 1 -- 172.21.15.146:0/980851800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc68009a70 con 0x7fbc7006df50 2024-09-13T18:38:02.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc76252700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc7006df50 0x7fbc700c6c90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000e9d0 tx=0x7fbc6000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:02.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc66ffd700 1 -- 172.21.15.146:0/980851800 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc60009720 con 0x7fbc7006df50 2024-09-13T18:38:02.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc66ffd700 1 -- 172.21.15.146:0/980851800 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6000ee70 con 0x7fbc7006df50 2024-09-13T18:38:02.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc784b6700 1 -- 172.21.15.146:0/980851800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc70151320 con 0x7fbc7006df50 2024-09-13T18:38:02.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc66ffd700 1 -- 172.21.15.146:0/980851800 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6000ca50 con 0x7fbc7006df50 2024-09-13T18:38:02.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.662+0000 7fbc784b6700 1 -- 172.21.15.146:0/980851800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc701483c0 con 0x7fbc7006df50 2024-09-13T18:38:02.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.666+0000 7fbc784b6700 1 -- 172.21.15.146:0/980851800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc701ebd70 con 0x7fbc7006df50 2024-09-13T18:38:02.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.666+0000 7fbc66ffd700 1 -- 172.21.15.146:0/980851800 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbc6000cbb0 con 0x7fbc7006df50 2024-09-13T18:38:02.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.666+0000 7fbc66ffd700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbc54066730 0x7fbc54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:02.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.666+0000 7fbc66ffd700 1 -- 172.21.15.146:0/980851800 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fbc600847c0 con 0x7fbc7006df50 2024-09-13T18:38:02.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.666+0000 7fbc75a51700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbc54066730 0x7fbc54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:02.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.666+0000 7fbc75a51700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbc54066730 0x7fbc54068bf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0011b0 tx=0x7fbc6c001140 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:02.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.674+0000 7fbc66ffd700 1 -- 172.21.15.146:0/980851800 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbc60051480 con 0x7fbc7006df50 2024-09-13T18:38:02.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.882+0000 7fbc784b6700 1 -- 172.21.15.146:0/980851800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc70148220 con 0x7fbc7006df50 2024-09-13T18:38:02.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.882+0000 7fbc66ffd700 1 -- 172.21.15.146:0/980851800 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5532 (secure 0 0 0) 0x7fbc70148220 con 0x7fbc7006df50 2024-09-13T18:38:02.901 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:02.901 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:00.700447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:02.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.886+0000 7fbc784b6700 1 -- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbc54066730 msgr2=0x7fbc54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:02.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.886+0000 7fbc784b6700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbc54066730 0x7fbc54068bf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0011b0 tx=0x7fbc6c001140 comp rx=0 tx=0).stop 2024-09-13T18:38:02.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.886+0000 7fbc784b6700 1 -- 172.21.15.146:0/980851800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc7006df50 msgr2=0x7fbc700c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:02.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.886+0000 7fbc784b6700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc7006df50 0x7fbc700c6c90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000e9d0 tx=0x7fbc6000ece0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.886+0000 7fbc784b6700 1 -- 172.21.15.146:0/980851800 shutdown_connections 2024-09-13T18:38:02.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.886+0000 7fbc784b6700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbc54066730 0x7fbc54068bf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.886+0000 7fbc784b6700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc700c1440 0x7fbc701ec460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.886+0000 7fbc784b6700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc7006e330 0x7fbc701ebf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.886+0000 7fbc784b6700 1 --2- 172.21.15.146:0/980851800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc7006df50 0x7fbc700c6c90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:02.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.886+0000 7fbc784b6700 1 -- 172.21.15.146:0/980851800 >> 172.21.15.146:0/980851800 conn(0x7fbc700cb1d0 msgr2=0x7fbc7014ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:02.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.886+0000 7fbc784b6700 1 -- 172.21.15.146:0/980851800 shutdown_connections 2024-09-13T18:38:02.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:02.886+0000 7fbc784b6700 1 -- 172.21.15.146:0/980851800 wait complete. 2024-09-13T18:38:02.905 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 52 2024-09-13T18:38:02.917 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 52, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 52, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5649": { "addr": "172.21.15.146:6837/3926931671", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3926931671, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3926931671, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5649, "incarnation": 40, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 24 }, "gid_5784": { "addr": "172.21.15.146:6833/3901685195", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3901685195, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3901685195, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5784, "incarnation": 46, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 29 }, "gid_5800": { "addr": "172.21.15.154:6839/776638146", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 776638146, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 776638146, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5800, "incarnation": 52, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 34, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:38:00.700447+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5800, "mds_1": 5784, "mds_2": 5649 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:38:04.918 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:38:05.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.018+0000 7fc653083700 1 Processor -- start 2024-09-13T18:38:05.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.018+0000 7fc653083700 1 -- start start 2024-09-13T18:38:05.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc653083700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0c63a0 0x7fc64c159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:05.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc653083700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc64c0c7310 0x7fc64c159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:05.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc653083700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64c0c8c70 con 0x7fc64c0c5fc0 2024-09-13T18:38:05.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc653083700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64c0c9bd0 con 0x7fc64c0c7310 2024-09-13T18:38:05.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc653083700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c15a0e0 con 0x7fc64c0c63a0 2024-09-13T18:38:05.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc652081700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc64c0c5fc0 0x7fc64c0c6c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58434/0 (socket says 172.21.15.146:58434) 2024-09-13T18:38:05.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc652081700 1 -- 172.21.15.146:0/510766818 learned_addr learned my addr 172.21.15.146:0/510766818 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:38:05.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc652882700 1 --2- 172.21.15.146:0/510766818 >> v2:172.21.15.154:3300/0 conn(0x7fc64c0c7310 0x7fc64c159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:05.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc651880700 1 --2- 172.21.15.146:0/510766818 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0c63a0 0x7fc64c159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:05.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc65107f700 1 -- 172.21.15.146:0/510766818 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196200744 0 0) 0x7fc64c0c8c70 con 0x7fc64c0c5fc0 2024-09-13T18:38:05.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc65107f700 1 -- 172.21.15.146:0/510766818 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6380035d0 con 0x7fc64c0c5fc0 2024-09-13T18:38:05.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc652882700 1 -- 172.21.15.146:0/510766818 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0c63a0 msgr2=0x7fc64c159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:05.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc652882700 1 --2- 172.21.15.146:0/510766818 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0c63a0 0x7fc64c159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:05.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc652882700 1 -- 172.21.15.146:0/510766818 >> v1:172.21.15.146:6789/0 conn(0x7fc64c0c5fc0 legacy=0x7fc64c0c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:05.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc652882700 1 -- 172.21.15.146:0/510766818 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c0c8f50 con 0x7fc64c0c7310 2024-09-13T18:38:05.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc651880700 1 --2- 172.21.15.146:0/510766818 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0c63a0 0x7fc64c159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:38:05.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc652882700 1 --2- 172.21.15.146:0/510766818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64c0c7310 0x7fc64c159ba0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc644009d90 tx=0x7fc64400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3d1acccc60b9fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:05.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc65107f700 1 -- 172.21.15.146:0/510766818 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64400e070 con 0x7fc64c0c7310 2024-09-13T18:38:05.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc65107f700 1 -- 172.21.15.146:0/510766818 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64400bb20 con 0x7fc64c0c7310 2024-09-13T18:38:05.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc65107f700 1 -- 172.21.15.146:0/510766818 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64400be90 con 0x7fc64c0c7310 2024-09-13T18:38:05.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc653083700 1 -- 172.21.15.146:0/510766818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64c0c7310 msgr2=0x7fc64c159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:05.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc653083700 1 --2- 172.21.15.146:0/510766818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64c0c7310 0x7fc64c159ba0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc644009d90 tx=0x7fc64400b330 comp rx=0 tx=0).stop 2024-09-13T18:38:05.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc653083700 1 -- 172.21.15.146:0/510766818 shutdown_connections 2024-09-13T18:38:05.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc653083700 1 --2- 172.21.15.146:0/510766818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64c0c7310 0x7fc64c159ba0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:05.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc653083700 1 --2- 172.21.15.146:0/510766818 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0c63a0 0x7fc64c159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:05.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc653083700 1 -- 172.21.15.146:0/510766818 >> 172.21.15.146:0/510766818 conn(0x7fc64c066090 msgr2=0x7fc64c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:05.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc653083700 1 -- 172.21.15.146:0/510766818 shutdown_connections 2024-09-13T18:38:05.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.022+0000 7fc653083700 1 -- 172.21.15.146:0/510766818 wait complete. 2024-09-13T18:38:05.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc653083700 1 Processor -- start 2024-09-13T18:38:05.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc653083700 1 -- start start 2024-09-13T18:38:05.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc653083700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0c5fc0 0x7fc64c1f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:05.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc653083700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64c0c63a0 0x7fc64c14e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:05.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc653083700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc64c0c7310 0x7fc64c14ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:05.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc653083700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c162140 con 0x7fc64c0c7310 2024-09-13T18:38:05.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc653083700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c1f2e70 con 0x7fc64c0c63a0 2024-09-13T18:38:05.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc653083700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c0c0930 con 0x7fc64c0c5fc0 2024-09-13T18:38:05.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc652882700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc64c0c7310 0x7fc64c14ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:05.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc652882700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc64c0c7310 0x7fc64c14ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51840/0 (socket says 172.21.15.146:51840) 2024-09-13T18:38:05.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc652882700 1 -- 172.21.15.146:0/3227554940 learned_addr learned my addr 172.21.15.146:0/3227554940 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:05.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc651880700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64c0c63a0 0x7fc64c14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:05.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc652081700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0c5fc0 0x7fc64c1f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:05.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc652882700 1 -- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0c5fc0 msgr2=0x7fc64c1f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:05.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc652882700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0c5fc0 0x7fc64c1f2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:05.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc652882700 1 -- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64c0c63a0 msgr2=0x7fc64c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:05.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc652882700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64c0c63a0 0x7fc64c14e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:05.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.026+0000 7fc652882700 1 -- 172.21.15.146:0/3227554940 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c0094b0 con 0x7fc64c0c7310 2024-09-13T18:38:05.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.030+0000 7fc652882700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc64c0c7310 0x7fc64c14ed40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc644014040 tx=0x7fc644004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:05.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.030+0000 7fc642ffd700 1 -- 172.21.15.146:0/3227554940 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644007000 con 0x7fc64c0c7310 2024-09-13T18:38:05.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.030+0000 7fc642ffd700 1 -- 172.21.15.146:0/3227554940 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644012650 con 0x7fc64c0c7310 2024-09-13T18:38:05.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.030+0000 7fc653083700 1 -- 172.21.15.146:0/3227554940 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc644009a70 con 0x7fc64c0c7310 2024-09-13T18:38:05.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.030+0000 7fc642ffd700 1 -- 172.21.15.146:0/3227554940 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644012a00 con 0x7fc64c0c7310 2024-09-13T18:38:05.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.030+0000 7fc653083700 1 -- 172.21.15.146:0/3227554940 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c1f2b90 con 0x7fc64c0c7310 2024-09-13T18:38:05.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.030+0000 7fc653083700 1 -- 172.21.15.146:0/3227554940 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64c0cb790 con 0x7fc64c0c7310 2024-09-13T18:38:05.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.030+0000 7fc642ffd700 1 -- 172.21.15.146:0/3227554940 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc644012b60 con 0x7fc64c0c7310 2024-09-13T18:38:05.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.034+0000 7fc642ffd700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc630066760 0x7fc630068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:05.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.034+0000 7fc642ffd700 1 -- 172.21.15.146:0/3227554940 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fc644086420 con 0x7fc64c0c7310 2024-09-13T18:38:05.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.034+0000 7fc652081700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc630066760 0x7fc630068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:05.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.034+0000 7fc652081700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc630066760 0x7fc630068c20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc648007fd0 tx=0x7fc648009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:05.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.038+0000 7fc642ffd700 1 -- 172.21.15.146:0/3227554940 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc644053190 con 0x7fc64c0c7310 2024-09-13T18:38:05.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.250+0000 7fc653083700 1 -- 172.21.15.146:0/3227554940 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc64c0c0700 con 0x7fc64c0c7310 2024-09-13T18:38:05.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.250+0000 7fc642ffd700 1 -- 172.21.15.146:0/3227554940 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5532 (secure 0 0 0) 0x7fc64c0c0700 con 0x7fc64c0c7310 2024-09-13T18:38:05.270 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:05.270 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:00.700447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:05.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.254+0000 7fc653083700 1 -- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc630066760 msgr2=0x7fc630068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:05.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.254+0000 7fc653083700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc630066760 0x7fc630068c20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc648007fd0 tx=0x7fc648009040 comp rx=0 tx=0).stop 2024-09-13T18:38:05.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.254+0000 7fc653083700 1 -- 172.21.15.146:0/3227554940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc64c0c7310 msgr2=0x7fc64c14ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:05.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.254+0000 7fc653083700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc64c0c7310 0x7fc64c14ed40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc644014040 tx=0x7fc644004af0 comp rx=0 tx=0).stop 2024-09-13T18:38:05.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.254+0000 7fc653083700 1 -- 172.21.15.146:0/3227554940 shutdown_connections 2024-09-13T18:38:05.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.254+0000 7fc653083700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc630066760 0x7fc630068c20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:05.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.254+0000 7fc653083700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc64c0c7310 0x7fc64c14ed40 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:05.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.254+0000 7fc653083700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64c0c63a0 0x7fc64c14e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:05.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.254+0000 7fc653083700 1 --2- 172.21.15.146:0/3227554940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0c5fc0 0x7fc64c1f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:05.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.254+0000 7fc653083700 1 -- 172.21.15.146:0/3227554940 >> 172.21.15.146:0/3227554940 conn(0x7fc64c066090 msgr2=0x7fc64c0c6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:05.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.254+0000 7fc653083700 1 -- 172.21.15.146:0/3227554940 shutdown_connections 2024-09-13T18:38:05.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:05.254+0000 7fc653083700 1 -- 172.21.15.146:0/3227554940 wait complete. 2024-09-13T18:38:05.274 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 52 2024-09-13T18:38:05.286 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:38:07.287 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:38:07.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.386+0000 7fa6a7430700 1 Processor -- start 2024-09-13T18:38:07.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.386+0000 7fa6a7430700 1 -- start start 2024-09-13T18:38:07.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a7430700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6a00cbd50 0x7fa6a00b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:07.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a7430700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6a00ca210 0x7fa6a00cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:07.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a7430700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a00cc670 con 0x7fa6a0071390 2024-09-13T18:38:07.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a7430700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a00caa30 con 0x7fa6a00ca210 2024-09-13T18:38:07.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a7430700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00cb810 con 0x7fa6a00cbd50 2024-09-13T18:38:07.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a6c2f700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa6a0071390 0x7fa6a00701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58446/0 (socket says 172.21.15.146:58446) 2024-09-13T18:38:07.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a6c2f700 1 -- 172.21.15.146:0/2967850487 learned_addr learned my addr 172.21.15.146:0/2967850487 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:38:07.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a642e700 1 --2- 172.21.15.146:0/2967850487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6a00cbd50 0x7fa6a00b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:07.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a5c2d700 1 --2- 172.21.15.146:0/2967850487 >> v2:172.21.15.154:3300/0 conn(0x7fa6a00ca210 0x7fa6a00cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:07.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a542c700 1 -- 172.21.15.146:0/2967850487 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902420356 0 0) 0x7fa698001490 con 0x7fa6a0071390 2024-09-13T18:38:07.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a542c700 1 -- 172.21.15.146:0/2967850487 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa68c0035d0 con 0x7fa6a0071390 2024-09-13T18:38:07.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a5c2d700 1 -- 172.21.15.146:0/2967850487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6a00cbd50 msgr2=0x7fa6a00b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:07.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a5c2d700 1 --2- 172.21.15.146:0/2967850487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6a00cbd50 0x7fa6a00b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:07.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a5c2d700 1 -- 172.21.15.146:0/2967850487 >> v1:172.21.15.146:6789/0 conn(0x7fa6a0071390 legacy=0x7fa6a00701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:07.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a5c2d700 1 -- 172.21.15.146:0/2967850487 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a015da20 con 0x7fa6a00ca210 2024-09-13T18:38:07.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a642e700 1 --2- 172.21.15.146:0/2967850487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6a00cbd50 0x7fa6a00b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:38:07.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a5c2d700 1 --2- 172.21.15.146:0/2967850487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6a00ca210 0x7fa6a00cc130 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009880 tx=0x7fa69c009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a513a266c3d41bf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:07.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a542c700 1 -- 172.21.15.146:0/2967850487 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c011830 con 0x7fa6a00ca210 2024-09-13T18:38:07.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a542c700 1 -- 172.21.15.146:0/2967850487 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c011990 con 0x7fa6a00ca210 2024-09-13T18:38:07.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a542c700 1 -- 172.21.15.146:0/2967850487 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c011d20 con 0x7fa6a00ca210 2024-09-13T18:38:07.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a7430700 1 -- 172.21.15.146:0/2967850487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6a00ca210 msgr2=0x7fa6a00cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:07.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a7430700 1 --2- 172.21.15.146:0/2967850487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6a00ca210 0x7fa6a00cc130 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009880 tx=0x7fa69c009b90 comp rx=0 tx=0).stop 2024-09-13T18:38:07.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a7430700 1 -- 172.21.15.146:0/2967850487 shutdown_connections 2024-09-13T18:38:07.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a7430700 1 --2- 172.21.15.146:0/2967850487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6a00ca210 0x7fa6a00cc130 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:07.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a7430700 1 --2- 172.21.15.146:0/2967850487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6a00cbd50 0x7fa6a00b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:07.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a7430700 1 -- 172.21.15.146:0/2967850487 >> 172.21.15.146:0/2967850487 conn(0x7fa6a0066270 msgr2=0x7fa6a0066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:07.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a7430700 1 -- 172.21.15.146:0/2967850487 shutdown_connections 2024-09-13T18:38:07.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.390+0000 7fa6a7430700 1 -- 172.21.15.146:0/2967850487 wait complete. 2024-09-13T18:38:07.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a7430700 1 Processor -- start 2024-09-13T18:38:07.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a7430700 1 -- start start 2024-09-13T18:38:07.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a7430700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6a0071390 0x7fa6a00cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:07.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a7430700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6a00ca210 0x7fa6a00b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:07.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a7430700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6a00cbd50 0x7fa6a01efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:07.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a7430700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a01f3670 con 0x7fa6a00ca210 2024-09-13T18:38:07.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a7430700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a01f3950 con 0x7fa6a00cbd50 2024-09-13T18:38:07.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a7430700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00c7220 con 0x7fa6a0071390 2024-09-13T18:38:07.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a5c2d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6a00ca210 0x7fa6a00b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:07.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a5c2d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6a00ca210 0x7fa6a00b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51846/0 (socket says 172.21.15.146:51846) 2024-09-13T18:38:07.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a5c2d700 1 -- 172.21.15.146:0/2554443207 learned_addr learned my addr 172.21.15.146:0/2554443207 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:07.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a6c2f700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6a00cbd50 0x7fa6a01efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:07.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a642e700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6a0071390 0x7fa6a00cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:07.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a5c2d700 1 -- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6a0071390 msgr2=0x7fa6a00cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:07.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a5c2d700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6a0071390 0x7fa6a00cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:07.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a5c2d700 1 -- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6a00cbd50 msgr2=0x7fa6a01efbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:07.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a5c2d700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6a00cbd50 0x7fa6a01efbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:07.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a5c2d700 1 -- 172.21.15.146:0/2554443207 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6940094b0 con 0x7fa6a00ca210 2024-09-13T18:38:07.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa6a5c2d700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6a00ca210 0x7fa6a00b3180 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0090b0 tx=0x7fa69c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:07.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.394+0000 7fa692ffd700 1 -- 172.21.15.146:0/2554443207 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c004400 con 0x7fa6a00ca210 2024-09-13T18:38:07.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.398+0000 7fa692ffd700 1 -- 172.21.15.146:0/2554443207 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c01f070 con 0x7fa6a00ca210 2024-09-13T18:38:07.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.398+0000 7fa6a7430700 1 -- 172.21.15.146:0/2554443207 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69c0094b0 con 0x7fa6a00ca210 2024-09-13T18:38:07.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.398+0000 7fa692ffd700 1 -- 172.21.15.146:0/2554443207 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c0036c0 con 0x7fa6a00ca210 2024-09-13T18:38:07.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.398+0000 7fa6a7430700 1 -- 172.21.15.146:0/2554443207 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a01ef650 con 0x7fa6a00ca210 2024-09-13T18:38:07.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.398+0000 7fa6a7430700 1 -- 172.21.15.146:0/2554443207 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a00c7080 con 0x7fa6a00ca210 2024-09-13T18:38:07.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.398+0000 7fa692ffd700 1 -- 172.21.15.146:0/2554443207 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa69c021030 con 0x7fa6a00ca210 2024-09-13T18:38:07.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.398+0000 7fa692ffd700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa684066730 0x7fa684068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:07.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.398+0000 7fa692ffd700 1 -- 172.21.15.146:0/2554443207 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fa69c0856e0 con 0x7fa6a00ca210 2024-09-13T18:38:07.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.398+0000 7fa6a642e700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa684066730 0x7fa684068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:07.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.402+0000 7fa6a642e700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa684066730 0x7fa684068bf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa694009ce0 tx=0x7fa694009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:07.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.406+0000 7fa692ffd700 1 -- 172.21.15.146:0/2554443207 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa6a00c7080 con 0x7fa6a00ca210 2024-09-13T18:38:07.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.618+0000 7fa6a7430700 1 -- 172.21.15.146:0/2554443207 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6a00c7080 con 0x7fa6a00ca210 2024-09-13T18:38:07.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.618+0000 7fa692ffd700 1 -- 172.21.15.146:0/2554443207 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+5534 (secure 0 0 0) 0x7fa6a00c7080 con 0x7fa6a00ca210 2024-09-13T18:38:07.638 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:07.638 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:06.741351+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:resolve","state_seq":24,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:07.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.622+0000 7fa6a7430700 1 -- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa684066730 msgr2=0x7fa684068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:07.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.622+0000 7fa6a7430700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa684066730 0x7fa684068bf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa694009ce0 tx=0x7fa694009240 comp rx=0 tx=0).stop 2024-09-13T18:38:07.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.622+0000 7fa6a7430700 1 -- 172.21.15.146:0/2554443207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6a00ca210 msgr2=0x7fa6a00b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:07.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.622+0000 7fa6a7430700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6a00ca210 0x7fa6a00b3180 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0090b0 tx=0x7fa69c004770 comp rx=0 tx=0).stop 2024-09-13T18:38:07.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.622+0000 7fa6a7430700 1 -- 172.21.15.146:0/2554443207 shutdown_connections 2024-09-13T18:38:07.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.622+0000 7fa6a7430700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa684066730 0x7fa684068bf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:07.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.622+0000 7fa6a7430700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6a00cbd50 0x7fa6a01efbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:07.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.622+0000 7fa6a7430700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6a00ca210 0x7fa6a00b3180 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:07.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.622+0000 7fa6a7430700 1 --2- 172.21.15.146:0/2554443207 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6a0071390 0x7fa6a00cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:07.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.622+0000 7fa6a7430700 1 -- 172.21.15.146:0/2554443207 >> 172.21.15.146:0/2554443207 conn(0x7fa6a0066270 msgr2=0x7fa6a0066650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:07.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.622+0000 7fa6a7430700 1 -- 172.21.15.146:0/2554443207 shutdown_connections 2024-09-13T18:38:07.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:07.622+0000 7fa6a7430700 1 -- 172.21.15.146:0/2554443207 wait complete. 2024-09-13T18:38:07.643 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 53 2024-09-13T18:38:07.652 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:38:09.167 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:38:09.169 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:38:09.653 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:38:09.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.758+0000 7fa3988d3700 1 Processor -- start 2024-09-13T18:38:09.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.758+0000 7fa3988d3700 1 -- start start 2024-09-13T18:38:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa3988d3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3940c5fc0 0x7fa3940c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa3988d3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa3940c63a0 0x7fa3940c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa3988d3700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3940cb380 con 0x7fa39406f5c0 2024-09-13T18:38:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa3988d3700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3940c9bd0 con 0x7fa3940c63a0 2024-09-13T18:38:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa3988d3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3940c8c70 con 0x7fa3940c5fc0 2024-09-13T18:38:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa39359e700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa39406f5c0 0x7fa3940cac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58448/0 (socket says 172.21.15.146:58448) 2024-09-13T18:38:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa39359e700 1 -- 172.21.15.146:0/8081075 learned_addr learned my addr 172.21.15.146:0/8081075 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:38:09.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa39259c700 1 --2- 172.21.15.146:0/8081075 >> v2:172.21.15.146:3300/0 conn(0x7fa3940c63a0 0x7fa3940c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:09.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa392d9d700 1 --2- 172.21.15.146:0/8081075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3940c5fc0 0x7fa3940c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:09.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa391d9b700 1 -- 172.21.15.146:0/8081075 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663986783 0 0) 0x7fa3940cb380 con 0x7fa39406f5c0 2024-09-13T18:38:09.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa391d9b700 1 -- 172.21.15.146:0/8081075 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa37c0035d0 con 0x7fa39406f5c0 2024-09-13T18:38:09.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa39259c700 1 -- 172.21.15.146:0/8081075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3940c5fc0 msgr2=0x7fa3940c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:09.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa39259c700 1 --2- 172.21.15.146:0/8081075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3940c5fc0 0x7fa3940c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:09.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa39259c700 1 -- 172.21.15.146:0/8081075 >> v1:172.21.15.146:6789/0 conn(0x7fa39406f5c0 legacy=0x7fa3940cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:09.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa39259c700 1 -- 172.21.15.146:0/8081075 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3940c8f50 con 0x7fa3940c63a0 2024-09-13T18:38:09.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa39259c700 1 --2- 172.21.15.146:0/8081075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3940c63a0 0x7fa3940c7140 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa388009850 tx=0x7fa388009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc9b43749874032 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:09.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa391d9b700 1 -- 172.21.15.146:0/8081075 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388011830 con 0x7fa3940c63a0 2024-09-13T18:38:09.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa391d9b700 1 -- 172.21.15.146:0/8081075 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa388011990 con 0x7fa3940c63a0 2024-09-13T18:38:09.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa391d9b700 1 -- 172.21.15.146:0/8081075 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388011d40 con 0x7fa3940c63a0 2024-09-13T18:38:09.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa3988d3700 1 -- 172.21.15.146:0/8081075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3940c63a0 msgr2=0x7fa3940c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:09.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.762+0000 7fa3988d3700 1 --2- 172.21.15.146:0/8081075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3940c63a0 0x7fa3940c7140 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa388009850 tx=0x7fa388009b60 comp rx=0 tx=0).stop 2024-09-13T18:38:09.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.766+0000 7fa3988d3700 1 -- 172.21.15.146:0/8081075 shutdown_connections 2024-09-13T18:38:09.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.766+0000 7fa3988d3700 1 --2- 172.21.15.146:0/8081075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3940c63a0 0x7fa3940c7140 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:09.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.766+0000 7fa3988d3700 1 --2- 172.21.15.146:0/8081075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3940c5fc0 0x7fa3940c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:09.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.766+0000 7fa3988d3700 1 -- 172.21.15.146:0/8081075 >> 172.21.15.146:0/8081075 conn(0x7fa394066090 msgr2=0x7fa394066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:09.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.766+0000 7fa3988d3700 1 -- 172.21.15.146:0/8081075 shutdown_connections 2024-09-13T18:38:09.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.766+0000 7fa3988d3700 1 -- 172.21.15.146:0/8081075 wait complete. 2024-09-13T18:38:09.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.766+0000 7fa3988d3700 1 Processor -- start 2024-09-13T18:38:09.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.766+0000 7fa3988d3700 1 -- start start 2024-09-13T18:38:09.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa3988d3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa39406f5c0 0x7fa3940ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:09.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa3988d3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3940c5fc0 0x7fa3940c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa3988d3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3940c63a0 0x7fa3940c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa3988d3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3940c3b60 con 0x7fa3940c63a0 2024-09-13T18:38:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa3988d3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3940c2200 con 0x7fa39406f5c0 2024-09-13T18:38:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa3988d3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa394152040 con 0x7fa3940c5fc0 2024-09-13T18:38:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa39359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3940c63a0 0x7fa3940c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa39359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3940c63a0 0x7fa3940c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51858/0 (socket says 172.21.15.146:51858) 2024-09-13T18:38:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa39259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3940c5fc0 0x7fa3940c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa39359e700 1 -- 172.21.15.146:0/3175289369 learned_addr learned my addr 172.21.15.146:0/3175289369 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa392d9d700 1 --2- 172.21.15.146:0/3175289369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa39406f5c0 0x7fa3940ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa39359e700 1 -- 172.21.15.146:0/3175289369 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3940c5fc0 msgr2=0x7fa3940c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa39359e700 1 --2- 172.21.15.146:0/3175289369 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3940c5fc0 0x7fa3940c40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:09.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa39359e700 1 -- 172.21.15.146:0/3175289369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa39406f5c0 msgr2=0x7fa3940ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:09.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa39359e700 1 --2- 172.21.15.146:0/3175289369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa39406f5c0 0x7fa3940ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:09.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa39359e700 1 -- 172.21.15.146:0/3175289369 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3880094b0 con 0x7fa3940c63a0 2024-09-13T18:38:09.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa39359e700 1 --2- 172.21.15.146:0/3175289369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3940c63a0 0x7fa3940c45f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00cb40 tx=0x7fa38c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:09.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa37b7fe700 1 -- 172.21.15.146:0/3175289369 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c007500 con 0x7fa3940c63a0 2024-09-13T18:38:09.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa37b7fe700 1 -- 172.21.15.146:0/3175289369 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c007660 con 0x7fa3940c63a0 2024-09-13T18:38:09.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa3988d3700 1 -- 172.21.15.146:0/3175289369 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3940afe90 con 0x7fa3940c63a0 2024-09-13T18:38:09.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa37b7fe700 1 -- 172.21.15.146:0/3175289369 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c0079f0 con 0x7fa3940c63a0 2024-09-13T18:38:09.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.770+0000 7fa3988d3700 1 -- 172.21.15.146:0/3175289369 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3940af8d0 con 0x7fa3940c63a0 2024-09-13T18:38:09.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.774+0000 7fa3988d3700 1 -- 172.21.15.146:0/3175289369 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3940cb790 con 0x7fa3940c63a0 2024-09-13T18:38:09.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.774+0000 7fa37b7fe700 1 -- 172.21.15.146:0/3175289369 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa38c01b3b0 con 0x7fa3940c63a0 2024-09-13T18:38:09.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.774+0000 7fa37b7fe700 1 --2- 172.21.15.146:0/3175289369 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa380066730 0x7fa380068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:09.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.774+0000 7fa37b7fe700 1 -- 172.21.15.146:0/3175289369 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fa38c084100 con 0x7fa3940c63a0 2024-09-13T18:38:09.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.774+0000 7fa392d9d700 1 --2- 172.21.15.146:0/3175289369 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa380066730 0x7fa380068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:09.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.774+0000 7fa392d9d700 1 --2- 172.21.15.146:0/3175289369 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa380066730 0x7fa380068bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa384009bf0 tx=0x7fa384009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:09.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.782+0000 7fa37b7fe700 1 -- 172.21.15.146:0/3175289369 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa38c050ec0 con 0x7fa3940c63a0 2024-09-13T18:38:10.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.982+0000 7fa3988d3700 1 -- 172.21.15.146:0/3175289369 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3940cb560 con 0x7fa3940c63a0 2024-09-13T18:38:10.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.982+0000 7fa37b7fe700 1 -- 172.21.15.146:0/3175289369 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+5534 (secure 0 0 0) 0x7fa38c0544e0 con 0x7fa3940c63a0 2024-09-13T18:38:10.005 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:10.005 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:06.741351+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:resolve","state_seq":24,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:10.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.986+0000 7fa3988d3700 1 -- 172.21.15.146:0/3175289369 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa380066730 msgr2=0x7fa380068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:10.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.986+0000 7fa3988d3700 1 --2- 172.21.15.146:0/3175289369 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa380066730 0x7fa380068bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa384009bf0 tx=0x7fa384009240 comp rx=0 tx=0).stop 2024-09-13T18:38:10.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.986+0000 7fa3988d3700 1 -- 172.21.15.146:0/3175289369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3940c63a0 msgr2=0x7fa3940c45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:10.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.986+0000 7fa3988d3700 1 --2- 172.21.15.146:0/3175289369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3940c63a0 0x7fa3940c45f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00cb40 tx=0x7fa38c00cb70 comp rx=0 tx=0).stop 2024-09-13T18:38:10.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.986+0000 7fa3988d3700 1 -- 172.21.15.146:0/3175289369 shutdown_connections 2024-09-13T18:38:10.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.986+0000 7fa3988d3700 1 --2- 172.21.15.146:0/3175289369 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa380066730 0x7fa380068bf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:10.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.986+0000 7fa3988d3700 1 --2- 172.21.15.146:0/3175289369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3940c63a0 0x7fa3940c45f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:10.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.986+0000 7fa3988d3700 1 --2- 172.21.15.146:0/3175289369 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3940c5fc0 0x7fa3940c40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:10.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.986+0000 7fa3988d3700 1 --2- 172.21.15.146:0/3175289369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa39406f5c0 0x7fa3940ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:10.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.986+0000 7fa3988d3700 1 -- 172.21.15.146:0/3175289369 >> 172.21.15.146:0/3175289369 conn(0x7fa394066090 msgr2=0x7fa394071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:10.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.986+0000 7fa3988d3700 1 -- 172.21.15.146:0/3175289369 shutdown_connections 2024-09-13T18:38:10.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:09.986+0000 7fa3988d3700 1 -- 172.21.15.146:0/3175289369 wait complete. 2024-09-13T18:38:10.008 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 53 2024-09-13T18:38:10.018 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:38:12.019 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:38:12.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.126+0000 7f239484b700 1 Processor -- start 2024-09-13T18:38:12.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.126+0000 7f239484b700 1 -- start start 2024-09-13T18:38:12.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f239484b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f239014fb50 0x7f23900c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:12.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f239484b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23900c7b00 0x7f23900c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:12.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f239484b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2390156810 con 0x7f23900c6060 2024-09-13T18:38:12.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f239484b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23901559b0 con 0x7f23900c7b00 2024-09-13T18:38:12.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f239484b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2390155c90 con 0x7f239014fb50 2024-09-13T18:38:12.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f238e59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23900c7b00 0x7f23900c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:12.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f238f59e700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f23900c6060 0x7f2390159190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:33004/0 (socket says 172.21.15.146:33004) 2024-09-13T18:38:12.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f238f59e700 1 -- 172.21.15.146:0/2180381037 learned_addr learned my addr 172.21.15.146:0/2180381037 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:38:12.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f238ed9d700 1 --2- 172.21.15.146:0/2180381037 >> v2:172.21.15.154:3300/0 conn(0x7f239014fb50 0x7f23900c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:12.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f238e59c700 1 -- 172.21.15.146:0/2180381037 >> v2:172.21.15.154:3300/0 conn(0x7f239014fb50 msgr2=0x7f23900c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:12.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f238e59c700 1 --2- 172.21.15.146:0/2180381037 >> v2:172.21.15.154:3300/0 conn(0x7f239014fb50 0x7f23900c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:12.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f238e59c700 1 -- 172.21.15.146:0/2180381037 >> v1:172.21.15.146:6789/0 conn(0x7f23900c6060 legacy=0x7f2390159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:12.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f238dd9b700 1 -- 172.21.15.146:0/2180381037 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749122813 0 0) 0x7f2390156810 con 0x7f23900c6060 2024-09-13T18:38:12.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f238e59c700 1 -- 172.21.15.146:0/2180381037 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2390155f70 con 0x7f23900c7b00 2024-09-13T18:38:12.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f238e59c700 1 --2- 172.21.15.146:0/2180381037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900c7b00 0x7f23900c8510 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2384009850 tx=0x7f2384009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bb315890604df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:12.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f238dd9b700 1 -- 172.21.15.146:0/2180381037 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384010800 con 0x7f23900c7b00 2024-09-13T18:38:12.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f238dd9b700 1 -- 172.21.15.146:0/2180381037 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2384010960 con 0x7f23900c7b00 2024-09-13T18:38:12.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f238dd9b700 1 -- 172.21.15.146:0/2180381037 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384010cf0 con 0x7f23900c7b00 2024-09-13T18:38:12.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f239484b700 1 -- 172.21.15.146:0/2180381037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900c7b00 msgr2=0x7f23900c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:12.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f239484b700 1 --2- 172.21.15.146:0/2180381037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900c7b00 0x7f23900c8510 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2384009850 tx=0x7f2384009b60 comp rx=0 tx=0).stop 2024-09-13T18:38:12.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f239484b700 1 -- 172.21.15.146:0/2180381037 shutdown_connections 2024-09-13T18:38:12.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f239484b700 1 --2- 172.21.15.146:0/2180381037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23900c7b00 0x7f23900c8510 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:12.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f239484b700 1 --2- 172.21.15.146:0/2180381037 >> v2:172.21.15.154:3300/0 conn(0x7f239014fb50 0x7f23900c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:12.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f239484b700 1 -- 172.21.15.146:0/2180381037 >> 172.21.15.146:0/2180381037 conn(0x7f2390066b00 msgr2=0x7f239006aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:12.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f239484b700 1 -- 172.21.15.146:0/2180381037 shutdown_connections 2024-09-13T18:38:12.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.130+0000 7f239484b700 1 -- 172.21.15.146:0/2180381037 wait complete. 2024-09-13T18:38:12.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.134+0000 7f239484b700 1 Processor -- start 2024-09-13T18:38:12.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.134+0000 7f239484b700 1 -- start start 2024-09-13T18:38:12.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.134+0000 7f239484b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23900c6060 0x7f2390154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:12.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.134+0000 7f239484b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23900c7b00 0x7f239014bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:12.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.134+0000 7f239484b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f239014fb50 0x7f23901eef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:12.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.134+0000 7f239484b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23900c5500 con 0x7f239014fb50 2024-09-13T18:38:12.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.134+0000 7f239484b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23901eea70 con 0x7f23900c6060 2024-09-13T18:38:12.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.134+0000 7f239484b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239014c110 con 0x7f23900c7b00 2024-09-13T18:38:12.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.134+0000 7f238f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f239014fb50 0x7f23901eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:12.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.134+0000 7f238f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f239014fb50 0x7f23901eef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32904/0 (socket says 172.21.15.146:32904) 2024-09-13T18:38:12.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.134+0000 7f238f59e700 1 -- 172.21.15.146:0/3658259229 learned_addr learned my addr 172.21.15.146:0/3658259229 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:12.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.134+0000 7f238e59c700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23900c7b00 0x7f239014bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:12.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.134+0000 7f238ed9d700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23900c6060 0x7f2390154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:12.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.138+0000 7f238f59e700 1 -- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23900c7b00 msgr2=0x7f239014bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:12.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.138+0000 7f238f59e700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23900c7b00 0x7f239014bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:12.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.138+0000 7f238f59e700 1 -- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23900c6060 msgr2=0x7f2390154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:12.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.138+0000 7f238f59e700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23900c6060 0x7f2390154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:12.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.138+0000 7f238f59e700 1 -- 172.21.15.146:0/3658259229 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23840094b0 con 0x7f239014fb50 2024-09-13T18:38:12.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.138+0000 7f238f59e700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f239014fb50 0x7f23901eef70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2388002230 tx=0x7f238800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:12.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.138+0000 7f23777fe700 1 -- 172.21.15.146:0/3658259229 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238800aac0 con 0x7f239014fb50 2024-09-13T18:38:12.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.138+0000 7f23777fe700 1 -- 172.21.15.146:0/3658259229 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238800ac40 con 0x7f239014fb50 2024-09-13T18:38:12.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.138+0000 7f23777fe700 1 -- 172.21.15.146:0/3658259229 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23880070d0 con 0x7f239014fb50 2024-09-13T18:38:12.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.138+0000 7f239484b700 1 -- 172.21.15.146:0/3658259229 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23901f2180 con 0x7f239014fb50 2024-09-13T18:38:12.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.138+0000 7f239484b700 1 -- 172.21.15.146:0/3658259229 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23901f2740 con 0x7f239014fb50 2024-09-13T18:38:12.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.138+0000 7f239484b700 1 -- 172.21.15.146:0/3658259229 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23900c64f0 con 0x7f239014fb50 2024-09-13T18:38:12.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.138+0000 7f23777fe700 1 -- 172.21.15.146:0/3658259229 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2388007250 con 0x7f239014fb50 2024-09-13T18:38:12.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.142+0000 7f23777fe700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f237c066760 0x7f237c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:12.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.142+0000 7f23777fe700 1 -- 172.21.15.146:0/3658259229 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f23880841d0 con 0x7f239014fb50 2024-09-13T18:38:12.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.142+0000 7f238ed9d700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f237c066760 0x7f237c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:12.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.142+0000 7f238ed9d700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f237c066760 0x7f237c068c20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2380009b50 tx=0x7f2380009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:12.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.150+0000 7f23777fe700 1 -- 172.21.15.146:0/3658259229 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2388050e90 con 0x7f239014fb50 2024-09-13T18:38:12.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.358+0000 7f239484b700 1 -- 172.21.15.146:0/3658259229 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23900c6c30 con 0x7f239014fb50 2024-09-13T18:38:12.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.362+0000 7f23777fe700 1 -- 172.21.15.146:0/3658259229 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+5536 (secure 0 0 0) 0x7f2388018090 con 0x7f239014fb50 2024-09-13T18:38:12.380 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:12.380 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:11.769644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:reconnect","state_seq":26,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:12.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.362+0000 7f239484b700 1 -- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f237c066760 msgr2=0x7f237c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:12.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.362+0000 7f239484b700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f237c066760 0x7f237c068c20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2380009b50 tx=0x7f2380009240 comp rx=0 tx=0).stop 2024-09-13T18:38:12.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.362+0000 7f239484b700 1 -- 172.21.15.146:0/3658259229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f239014fb50 msgr2=0x7f23901eef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:12.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.362+0000 7f239484b700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f239014fb50 0x7f23901eef70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2388002230 tx=0x7f238800cf50 comp rx=0 tx=0).stop 2024-09-13T18:38:12.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.362+0000 7f239484b700 1 -- 172.21.15.146:0/3658259229 shutdown_connections 2024-09-13T18:38:12.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.362+0000 7f239484b700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f237c066760 0x7f237c068c20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:12.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.362+0000 7f239484b700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f239014fb50 0x7f23901eef70 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:12.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.362+0000 7f239484b700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23900c7b00 0x7f239014bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:12.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.362+0000 7f239484b700 1 --2- 172.21.15.146:0/3658259229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23900c6060 0x7f2390154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:12.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.362+0000 7f239484b700 1 -- 172.21.15.146:0/3658259229 >> 172.21.15.146:0/3658259229 conn(0x7f2390066b00 msgr2=0x7f239014d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:12.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.362+0000 7f239484b700 1 -- 172.21.15.146:0/3658259229 shutdown_connections 2024-09-13T18:38:12.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:12.362+0000 7f239484b700 1 -- 172.21.15.146:0/3658259229 wait complete. 2024-09-13T18:38:12.384 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 54 2024-09-13T18:38:12.395 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:38:14.396 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:38:14.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 Processor -- start 2024-09-13T18:38:14.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 -- start start 2024-09-13T18:38:14.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f049c06de80 0x7f049c06ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:14.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f049c070420 0x7f049c0c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:14.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f049c1575e0 0x7f049c157a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:14.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f049c06edb0 con 0x7f049c070420 2024-09-13T18:38:14.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f049c0c7910 con 0x7f049c06de80 2024-09-13T18:38:14.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049c0c7630 con 0x7f049c1575e0 2024-09-13T18:38:14.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a118d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f049c070420 0x7f049c0c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a118d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f049c070420 0x7f049c0c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32918/0 (socket says 172.21.15.146:32918) 2024-09-13T18:38:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a118d700 1 -- 172.21.15.146:0/1205006220 learned_addr learned my addr 172.21.15.146:0/1205006220 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a218f700 1 --2- 172.21.15.146:0/1205006220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f049c1575e0 0x7f049c157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a198e700 1 --2- 172.21.15.146:0/1205006220 >> v2:172.21.15.154:3300/0 conn(0x7f049c06de80 0x7f049c06ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a118d700 1 -- 172.21.15.146:0/1205006220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f049c1575e0 msgr2=0x7f049c157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a118d700 1 --2- 172.21.15.146:0/1205006220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f049c1575e0 0x7f049c157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a118d700 1 -- 172.21.15.146:0/1205006220 >> v2:172.21.15.154:3300/0 conn(0x7f049c06de80 msgr2=0x7f049c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a118d700 1 --2- 172.21.15.146:0/1205006220 >> v2:172.21.15.154:3300/0 conn(0x7f049c06de80 0x7f049c06ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a118d700 1 -- 172.21.15.146:0/1205006220 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c06f660 con 0x7f049c070420 2024-09-13T18:38:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a118d700 1 --2- 172.21.15.146:0/1205006220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c070420 0x7f049c0c65e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f049c06c170 tx=0x7f0498009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a1e5d94f6b1430 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a098c700 1 -- 172.21.15.146:0/1205006220 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049800b8a0 con 0x7f049c070420 2024-09-13T18:38:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a098c700 1 -- 172.21.15.146:0/1205006220 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049800ba00 con 0x7f049c070420 2024-09-13T18:38:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a098c700 1 -- 172.21.15.146:0/1205006220 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049800bd90 con 0x7f049c070420 2024-09-13T18:38:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 -- 172.21.15.146:0/1205006220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c070420 msgr2=0x7f049c0c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 --2- 172.21.15.146:0/1205006220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c070420 0x7f049c0c65e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f049c06c170 tx=0x7f0498009bf0 comp rx=0 tx=0).stop 2024-09-13T18:38:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 -- 172.21.15.146:0/1205006220 shutdown_connections 2024-09-13T18:38:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 --2- 172.21.15.146:0/1205006220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f049c1575e0 0x7f049c157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 --2- 172.21.15.146:0/1205006220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c070420 0x7f049c0c65e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:14.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 --2- 172.21.15.146:0/1205006220 >> v2:172.21.15.154:3300/0 conn(0x7f049c06de80 0x7f049c06ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:14.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 -- 172.21.15.146:0/1205006220 >> 172.21.15.146:0/1205006220 conn(0x7f049c0cb1d0 msgr2=0x7f049c0cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:14.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 -- 172.21.15.146:0/1205006220 shutdown_connections 2024-09-13T18:38:14.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.502+0000 7f04a3bf2700 1 -- 172.21.15.146:0/1205006220 wait complete. 2024-09-13T18:38:14.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a3bf2700 1 Processor -- start 2024-09-13T18:38:14.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a3bf2700 1 -- start start 2024-09-13T18:38:14.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a3bf2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f049c06de80 0x7f049c1f10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:14.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a3bf2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c070420 0x7f049c1f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:14.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a3bf2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f049c1575e0 0x7f049c0ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:14.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a3bf2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049c14dcc0 con 0x7f049c070420 2024-09-13T18:38:14.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a3bf2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049c14bd70 con 0x7f049c06de80 2024-09-13T18:38:14.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a3bf2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049c14c100 con 0x7f049c1575e0 2024-09-13T18:38:14.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a118d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c070420 0x7f049c1f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:14.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a118d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c070420 0x7f049c1f1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32922/0 (socket says 172.21.15.146:32922) 2024-09-13T18:38:14.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a118d700 1 -- 172.21.15.146:0/3433629755 learned_addr learned my addr 172.21.15.146:0/3433629755 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:14.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a198e700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f049c06de80 0x7f049c1f10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:14.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a218f700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f049c1575e0 0x7f049c0ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:14.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a118d700 1 -- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f049c1575e0 msgr2=0x7f049c0ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:14.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a118d700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f049c1575e0 0x7f049c0ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:14.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a118d700 1 -- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f049c06de80 msgr2=0x7f049c1f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:14.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a118d700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f049c06de80 0x7f049c1f10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:14.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a118d700 1 -- 172.21.15.146:0/3433629755 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0498009570 con 0x7f049c070420 2024-09-13T18:38:14.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a218f700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f049c1575e0 0x7f049c0ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:38:14.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a198e700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f049c06de80 0x7f049c1f10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:38:14.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04a118d700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c070420 0x7f049c1f1610 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0498013000 tx=0x7f0498004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:14.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.506+0000 7f04927fc700 1 -- 172.21.15.146:0/3433629755 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04980036c0 con 0x7f049c070420 2024-09-13T18:38:14.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.510+0000 7f04927fc700 1 -- 172.21.15.146:0/3433629755 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049801f070 con 0x7f049c070420 2024-09-13T18:38:14.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.510+0000 7f04a3bf2700 1 -- 172.21.15.146:0/3433629755 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049c14cda0 con 0x7f049c070420 2024-09-13T18:38:14.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.510+0000 7f04927fc700 1 -- 172.21.15.146:0/3433629755 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498003a40 con 0x7f049c070420 2024-09-13T18:38:14.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.510+0000 7f04a3bf2700 1 -- 172.21.15.146:0/3433629755 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049c14d320 con 0x7f049c070420 2024-09-13T18:38:14.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.510+0000 7f04a3bf2700 1 -- 172.21.15.146:0/3433629755 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049c0c72d0 con 0x7f049c070420 2024-09-13T18:38:14.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.510+0000 7f04927fc700 1 -- 172.21.15.146:0/3433629755 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0498021030 con 0x7f049c070420 2024-09-13T18:38:14.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.510+0000 7f04927fc700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0480066730 0x7f0480068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:14.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.510+0000 7f04927fc700 1 -- 172.21.15.146:0/3433629755 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f0498085d60 con 0x7f049c070420 2024-09-13T18:38:14.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.510+0000 7f04a198e700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0480066730 0x7f0480068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:14.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.514+0000 7f04a198e700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0480066730 0x7f0480068bf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f048c003950 tx=0x7f048c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:14.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.522+0000 7f04927fc700 1 -- 172.21.15.146:0/3433629755 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0498052ab0 con 0x7f049c070420 2024-09-13T18:38:14.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.726+0000 7f04a3bf2700 1 -- 172.21.15.146:0/3433629755 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f049c157360 con 0x7f049c070420 2024-09-13T18:38:14.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.726+0000 7f04927fc700 1 -- 172.21.15.146:0/3433629755 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+5536 (secure 0 0 0) 0x7f04980560d0 con 0x7f049c070420 2024-09-13T18:38:14.746 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:14.746 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:11.769644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:reconnect","state_seq":26,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:14.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.730+0000 7f04a3bf2700 1 -- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0480066730 msgr2=0x7f0480068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:14.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.730+0000 7f04a3bf2700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0480066730 0x7f0480068bf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f048c003950 tx=0x7f048c008040 comp rx=0 tx=0).stop 2024-09-13T18:38:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.730+0000 7f04a3bf2700 1 -- 172.21.15.146:0/3433629755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c070420 msgr2=0x7f049c1f1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.730+0000 7f04a3bf2700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c070420 0x7f049c1f1610 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0498013000 tx=0x7f0498004770 comp rx=0 tx=0).stop 2024-09-13T18:38:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.730+0000 7f04a3bf2700 1 -- 172.21.15.146:0/3433629755 shutdown_connections 2024-09-13T18:38:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.730+0000 7f04a3bf2700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0480066730 0x7f0480068bf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.730+0000 7f04a3bf2700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f049c1575e0 0x7f049c0ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.730+0000 7f04a3bf2700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c070420 0x7f049c1f1610 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.730+0000 7f04a3bf2700 1 --2- 172.21.15.146:0/3433629755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f049c06de80 0x7f049c1f10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.730+0000 7f04a3bf2700 1 -- 172.21.15.146:0/3433629755 >> 172.21.15.146:0/3433629755 conn(0x7f049c0cb1d0 msgr2=0x7f049c0cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.730+0000 7f04a3bf2700 1 -- 172.21.15.146:0/3433629755 shutdown_connections 2024-09-13T18:38:14.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:14.730+0000 7f04a3bf2700 1 -- 172.21.15.146:0/3433629755 wait complete. 2024-09-13T18:38:14.750 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 54 2024-09-13T18:38:14.762 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 54, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 54, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5649": { "addr": "172.21.15.146:6837/3926931671", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3926931671, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3926931671, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5649, "incarnation": 40, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 24 }, "gid_5784": { "addr": "172.21.15.146:6833/3901685195", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3901685195, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3901685195, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5784, "incarnation": 46, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 29 }, "gid_5800": { "addr": "172.21.15.154:6839/776638146", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 776638146, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 776638146, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5800, "incarnation": 52, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:reconnect", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 34, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:38:11.769644+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5800, "mds_1": 5784, "mds_2": 5649 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:38:16.763 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:38:16.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.866+0000 7f01684ea700 1 Processor -- start 2024-09-13T18:38:16.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.866+0000 7f01684ea700 1 -- start start 2024-09-13T18:38:16.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f01684ea700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f01601506f0 0x7f01600cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:16.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f01684ea700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016006ec90 con 0x7f0160150ad0 2024-09-13T18:38:16.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f01684ea700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01600c1da0 con 0x7f01600adca0 2024-09-13T18:38:16.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f01684ea700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016006d040 con 0x7f01601506f0 2024-09-13T18:38:16.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165a85700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0160150ad0 0x7f016006e580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:33020/0 (socket says 172.21.15.146:33020) 2024-09-13T18:38:16.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165a85700 1 -- 172.21.15.146:0/234606342 learned_addr learned my addr 172.21.15.146:0/234606342 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:38:16.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0166286700 1 --2- 172.21.15.146:0/234606342 >> v2:172.21.15.154:3300/0 conn(0x7f01601506f0 0x7f01600cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:16.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518578704 0 0) 0x7f015c001110 con 0x7f0160150ad0 2024-09-13T18:38:16.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f014c0057e0 con 0x7f0160150ad0 2024-09-13T18:38:16.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2023436865 0 0) 0x7f014c0057e0 con 0x7f0160150ad0 2024-09-13T18:38:16.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 >> v2:172.21.15.154:3300/0 conn(0x7f01601506f0 msgr2=0x7f01600cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:16.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 --2- 172.21.15.146:0/234606342 >> v2:172.21.15.154:3300/0 conn(0x7f01601506f0 0x7f01600cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:16.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 >> v1:172.21.15.154:6789/0 conn(0x7f01600adca0 legacy=0x7f01600c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:16.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016006c7e0 con 0x7f0160150ad0 2024-09-13T18:38:16.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f015c001800 con 0x7f0160150ad0 2024-09-13T18:38:16.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 >> v1:172.21.15.146:6789/0 conn(0x7f0160150ad0 legacy=0x7f016006e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:16.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 --2- 172.21.15.146:0/234606342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f014c005c30 0x7f014c006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:16.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 --2- 172.21.15.146:0/234606342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f014c006a80 0x7f014c006f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:16.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 --2- 172.21.15.146:0/234606342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c007440 0x7f014c00dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:16.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016006d040 con 0x7f014c007440 2024-09-13T18:38:16.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c00e2e0 con 0x7f014c006a80 2024-09-13T18:38:16.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c00e4a0 con 0x7f014c005c30 2024-09-13T18:38:16.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0166a87700 1 --2- 172.21.15.146:0/234606342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c007440 0x7f014c00dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:16.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165a85700 1 --2- 172.21.15.146:0/234606342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f014c006a80 0x7f014c006f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:16.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0166286700 1 --2- 172.21.15.146:0/234606342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f014c005c30 0x7f014c006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:16.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0166a87700 1 -- 172.21.15.146:0/234606342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f014c005c30 msgr2=0x7f014c006050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:16.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0166a87700 1 --2- 172.21.15.146:0/234606342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f014c005c30 0x7f014c006050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:16.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0166a87700 1 -- 172.21.15.146:0/234606342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f014c006a80 msgr2=0x7f014c006f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:16.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0166a87700 1 --2- 172.21.15.146:0/234606342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f014c006a80 0x7f014c006f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:16.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0166a87700 1 -- 172.21.15.146:0/234606342 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c0057e0 con 0x7f014c007440 2024-09-13T18:38:16.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165a85700 1 --2- 172.21.15.146:0/234606342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f014c006a80 0x7f014c006f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:38:16.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0166a87700 1 -- 172.21.15.146:0/234606342 reap_dead start 2024-09-13T18:38:16.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0166a87700 1 --2- 172.21.15.146:0/234606342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c007440 0x7f014c00dc40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f015800c160 tx=0x7f014c007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d25be467d47d5e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:16.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158001db0 con 0x7f014c007440 2024-09-13T18:38:16.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015800ccc0 con 0x7f014c007440 2024-09-13T18:38:16.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.870+0000 7f0165284700 1 -- 172.21.15.146:0/234606342 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158003c60 con 0x7f014c007440 2024-09-13T18:38:16.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.874+0000 7f01684ea700 1 -- 172.21.15.146:0/234606342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c007440 msgr2=0x7f014c00dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:16.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.874+0000 7f01684ea700 1 --2- 172.21.15.146:0/234606342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c007440 0x7f014c00dc40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f015800c160 tx=0x7f014c007ab0 comp rx=0 tx=0).stop 2024-09-13T18:38:16.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.874+0000 7f01684ea700 1 -- 172.21.15.146:0/234606342 shutdown_connections 2024-09-13T18:38:16.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.874+0000 7f01684ea700 1 --2- 172.21.15.146:0/234606342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c007440 0x7f014c00dc40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:16.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.874+0000 7f01684ea700 1 -- 172.21.15.146:0/234606342 >> 172.21.15.146:0/234606342 conn(0x7f01600cb1d0 msgr2=0x7f01600cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:16.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.874+0000 7f01684ea700 1 -- 172.21.15.146:0/234606342 shutdown_connections 2024-09-13T18:38:16.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.874+0000 7f01684ea700 1 -- 172.21.15.146:0/234606342 wait complete. 2024-09-13T18:38:16.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.874+0000 7f01684ea700 1 Processor -- start 2024-09-13T18:38:16.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.874+0000 7f01684ea700 1 -- start start 2024-09-13T18:38:16.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f01684ea700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f014c007440 0x7f01600b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:16.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f01684ea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01601eb8b0 0x7f01601ebd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f01684ea700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0160149b90 0x7f016014a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f01684ea700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01600b1480 con 0x7f0160149b90 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f01684ea700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01601ec250 con 0x7f01601eb8b0 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f01684ea700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01600afdc0 con 0x7f014c007440 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0166a87700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0160149b90 0x7f016014a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0166a87700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0160149b90 0x7f016014a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32940/0 (socket says 172.21.15.146:32940) 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0166a87700 1 -- 172.21.15.146:0/1890435942 learned_addr learned my addr 172.21.15.146:0/1890435942 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0165a85700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01601eb8b0 0x7f01601ebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0166286700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f014c007440 0x7f01600b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0166a87700 1 -- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f014c007440 msgr2=0x7f01600b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0166a87700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f014c007440 0x7f01600b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0166a87700 1 -- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01601eb8b0 msgr2=0x7f01601ebd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0166a87700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01601eb8b0 0x7f01601ebd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0166a87700 1 -- 172.21.15.146:0/1890435942 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01600b00a0 con 0x7f0160149b90 2024-09-13T18:38:16.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0166a87700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0160149b90 0x7f016014a040 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0158005040 tx=0x7f01580014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:16.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0156ffd700 1 -- 172.21.15.146:0/1890435942 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158003e30 con 0x7f0160149b90 2024-09-13T18:38:16.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0156ffd700 1 -- 172.21.15.146:0/1890435942 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0158005070 con 0x7f0160149b90 2024-09-13T18:38:16.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f01684ea700 1 -- 172.21.15.146:0/1890435942 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01600b0380 con 0x7f0160149b90 2024-09-13T18:38:16.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f0156ffd700 1 -- 172.21.15.146:0/1890435942 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158005420 con 0x7f0160149b90 2024-09-13T18:38:16.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f01684ea700 1 -- 172.21.15.146:0/1890435942 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01600b0940 con 0x7f0160149b90 2024-09-13T18:38:16.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.878+0000 7f01684ea700 1 -- 172.21.15.146:0/1890435942 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01600cbcf0 con 0x7f0160149b90 2024-09-13T18:38:16.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.882+0000 7f0156ffd700 1 -- 172.21.15.146:0/1890435942 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0158005580 con 0x7f0160149b90 2024-09-13T18:38:16.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.882+0000 7f0156ffd700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0144066760 0x7f0144068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:16.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.882+0000 7f0156ffd700 1 -- 172.21.15.146:0/1890435942 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f0158085b00 con 0x7f0160149b90 2024-09-13T18:38:16.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.882+0000 7f0166286700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0144066760 0x7f0144068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:16.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.882+0000 7f0166286700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0144066760 0x7f0144068c20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0150009bf0 tx=0x7f0150009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:16.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:16.890+0000 7f0156ffd700 1 -- 172.21.15.146:0/1890435942 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0158087e60 con 0x7f0160149b90 2024-09-13T18:38:17.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.098+0000 7f01684ea700 1 -- 172.21.15.146:0/1890435942 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f016006ec90 con 0x7f0160149b90 2024-09-13T18:38:17.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.102+0000 7f0156ffd700 1 -- 172.21.15.146:0/1890435942 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5533 (secure 0 0 0) 0x7f016006ec90 con 0x7f0160149b90 2024-09-13T18:38:17.119 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:17.119 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:16.788778+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:17.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.102+0000 7f01684ea700 1 -- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0144066760 msgr2=0x7f0144068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:17.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.102+0000 7f01684ea700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0144066760 0x7f0144068c20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0150009bf0 tx=0x7f0150009240 comp rx=0 tx=0).stop 2024-09-13T18:38:17.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.102+0000 7f01684ea700 1 -- 172.21.15.146:0/1890435942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0160149b90 msgr2=0x7f016014a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:17.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.102+0000 7f01684ea700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0160149b90 0x7f016014a040 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0158005040 tx=0x7f01580014a0 comp rx=0 tx=0).stop 2024-09-13T18:38:17.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.102+0000 7f01684ea700 1 -- 172.21.15.146:0/1890435942 shutdown_connections 2024-09-13T18:38:17.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.102+0000 7f01684ea700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0144066760 0x7f0144068c20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:17.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.102+0000 7f01684ea700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0160149b90 0x7f016014a040 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:17.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.102+0000 7f01684ea700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01601eb8b0 0x7f01601ebd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:17.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.102+0000 7f01684ea700 1 --2- 172.21.15.146:0/1890435942 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f014c007440 0x7f01600b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:17.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.102+0000 7f01684ea700 1 -- 172.21.15.146:0/1890435942 >> 172.21.15.146:0/1890435942 conn(0x7f01600cb1d0 msgr2=0x7f016014e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:17.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.102+0000 7f01684ea700 1 -- 172.21.15.146:0/1890435942 shutdown_connections 2024-09-13T18:38:17.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:17.102+0000 7f01684ea700 1 -- 172.21.15.146:0/1890435942 wait complete. 2024-09-13T18:38:17.122 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 55 2024-09-13T18:38:17.133 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:38:19.134 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:38:19.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac2d129700 1 Processor -- start 2024-09-13T18:38:19.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac2d129700 1 -- start start 2024-09-13T18:38:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac2d129700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac280c6610 0x7fac280cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac2d129700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac280c6a80 0x7fac280cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac2d129700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac28153fa0 con 0x7fac28153bc0 2024-09-13T18:38:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac2d129700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac280aa500 con 0x7fac280c6610 2024-09-13T18:38:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac2d129700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2815a7e0 con 0x7fac280c6a80 2024-09-13T18:38:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac277fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac280c6610 0x7fac280cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac2c928700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac280c6a80 0x7fac280cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac277fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac280c6610 0x7fac280cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:58418/0 (socket says 172.21.15.146:58418) 2024-09-13T18:38:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac277fe700 1 -- 172.21.15.146:0/781427373 learned_addr learned my addr 172.21.15.146:0/781427373 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac2c928700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac280c6a80 0x7fac280cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:42408/0 (socket says 172.21.15.146:42408) 2024-09-13T18:38:19.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac277fe700 1 -- 172.21.15.146:0/781427373 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac280c6a80 msgr2=0x7fac280cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac277fe700 1 --2- 172.21.15.146:0/781427373 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac280c6a80 0x7fac280cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac277fe700 1 -- 172.21.15.146:0/781427373 >> v1:172.21.15.154:6789/0 conn(0x7fac28153bc0 legacy=0x7fac280c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac277fe700 1 -- 172.21.15.146:0/781427373 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac280aa220 con 0x7fac280c6610 2024-09-13T18:38:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.238+0000 7fac277fe700 1 --2- 172.21.15.146:0/781427373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac280c6610 0x7fac280cac60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fac18009910 tx=0x7fac18009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5131b443e79f483 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac26ffd700 1 -- 172.21.15.146:0/781427373 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18011830 con 0x7fac280c6610 2024-09-13T18:38:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac26ffd700 1 -- 172.21.15.146:0/781427373 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac18011990 con 0x7fac280c6610 2024-09-13T18:38:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac26ffd700 1 -- 172.21.15.146:0/781427373 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18011d40 con 0x7fac280c6610 2024-09-13T18:38:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 -- 172.21.15.146:0/781427373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac280c6610 msgr2=0x7fac280cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 --2- 172.21.15.146:0/781427373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac280c6610 0x7fac280cac60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fac18009910 tx=0x7fac18009c20 comp rx=0 tx=0).stop 2024-09-13T18:38:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 -- 172.21.15.146:0/781427373 shutdown_connections 2024-09-13T18:38:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 --2- 172.21.15.146:0/781427373 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac280c6a80 0x7fac280cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 --2- 172.21.15.146:0/781427373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac280c6610 0x7fac280cac60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 -- 172.21.15.146:0/781427373 >> 172.21.15.146:0/781427373 conn(0x7fac28066090 msgr2=0x7fac28066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:19.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 -- 172.21.15.146:0/781427373 shutdown_connections 2024-09-13T18:38:19.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 -- 172.21.15.146:0/781427373 wait complete. 2024-09-13T18:38:19.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 Processor -- start 2024-09-13T18:38:19.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 -- start start 2024-09-13T18:38:19.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac280c6610 0x7fac281efc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:19.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac280c6a80 0x7fac280ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:19.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac28153bc0 0x7fac28158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:19.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac280ae2b0 con 0x7fac28153bc0 2024-09-13T18:38:19.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281f0140 con 0x7fac280c6a80 2024-09-13T18:38:19.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2d129700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac280ab220 con 0x7fac280c6610 2024-09-13T18:38:19.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2c928700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac28153bc0 0x7fac28158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:19.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2c928700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac28153bc0 0x7fac28158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32954/0 (socket says 172.21.15.146:32954) 2024-09-13T18:38:19.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2c928700 1 -- 172.21.15.146:0/2988503083 learned_addr learned my addr 172.21.15.146:0/2988503083 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:19.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac27fff700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac280c6610 0x7fac281efc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:19.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac277fe700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac280c6a80 0x7fac280ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:19.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2c928700 1 -- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac280c6610 msgr2=0x7fac281efc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:19.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2c928700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac280c6610 0x7fac281efc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:19.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2c928700 1 -- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac280c6a80 msgr2=0x7fac280ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:19.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.242+0000 7fac2c928700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac280c6a80 0x7fac280ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:19.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.246+0000 7fac2c928700 1 -- 172.21.15.146:0/2988503083 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac18009570 con 0x7fac28153bc0 2024-09-13T18:38:19.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.246+0000 7fac2c928700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac28153bc0 0x7fac28158a20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fac200092e0 tx=0x7fac2000e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:19.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.246+0000 7fac24ff9700 1 -- 172.21.15.146:0/2988503083 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20010040 con 0x7fac28153bc0 2024-09-13T18:38:19.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.246+0000 7fac24ff9700 1 -- 172.21.15.146:0/2988503083 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac20003a40 con 0x7fac28153bc0 2024-09-13T18:38:19.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.246+0000 7fac2d129700 1 -- 172.21.15.146:0/2988503083 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac280af980 con 0x7fac28153bc0 2024-09-13T18:38:19.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.246+0000 7fac24ff9700 1 -- 172.21.15.146:0/2988503083 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20003df0 con 0x7fac28153bc0 2024-09-13T18:38:19.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.246+0000 7fac2d129700 1 -- 172.21.15.146:0/2988503083 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac280aff00 con 0x7fac28153bc0 2024-09-13T18:38:19.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.246+0000 7fac2d129700 1 -- 172.21.15.146:0/2988503083 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac280ae3f0 con 0x7fac28153bc0 2024-09-13T18:38:19.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.246+0000 7fac24ff9700 1 -- 172.21.15.146:0/2988503083 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fac2000e820 con 0x7fac28153bc0 2024-09-13T18:38:19.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.246+0000 7fac24ff9700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fac14066730 0x7fac14068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:19.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.246+0000 7fac24ff9700 1 -- 172.21.15.146:0/2988503083 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fac20084740 con 0x7fac28153bc0 2024-09-13T18:38:19.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.246+0000 7fac27fff700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fac14066730 0x7fac14068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:19.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.250+0000 7fac27fff700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fac14066730 0x7fac14068bf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000f60 tx=0x7fac1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:19.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.254+0000 7fac24ff9700 1 -- 172.21.15.146:0/2988503083 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fac280ae3f0 con 0x7fac28153bc0 2024-09-13T18:38:19.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.462+0000 7fac2d129700 1 -- 172.21.15.146:0/2988503083 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac280ae3f0 con 0x7fac28153bc0 2024-09-13T18:38:19.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.462+0000 7fac24ff9700 1 -- 172.21.15.146:0/2988503083 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5533 (secure 0 0 0) 0x7fac280ae3f0 con 0x7fac28153bc0 2024-09-13T18:38:19.481 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:19.481 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:16.788778+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:19.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.466+0000 7fac2d129700 1 -- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fac14066730 msgr2=0x7fac14068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:19.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.466+0000 7fac2d129700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fac14066730 0x7fac14068bf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000f60 tx=0x7fac1c009040 comp rx=0 tx=0).stop 2024-09-13T18:38:19.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.466+0000 7fac2d129700 1 -- 172.21.15.146:0/2988503083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac28153bc0 msgr2=0x7fac28158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:19.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.466+0000 7fac2d129700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac28153bc0 0x7fac28158a20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fac200092e0 tx=0x7fac2000e3a0 comp rx=0 tx=0).stop 2024-09-13T18:38:19.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.466+0000 7fac2d129700 1 -- 172.21.15.146:0/2988503083 shutdown_connections 2024-09-13T18:38:19.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.466+0000 7fac2d129700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fac14066730 0x7fac14068bf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:19.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.466+0000 7fac2d129700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac28153bc0 0x7fac28158a20 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:19.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.466+0000 7fac2d129700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac280c6a80 0x7fac280ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:19.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.466+0000 7fac2d129700 1 --2- 172.21.15.146:0/2988503083 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac280c6610 0x7fac281efc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:19.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.466+0000 7fac2d129700 1 -- 172.21.15.146:0/2988503083 >> 172.21.15.146:0/2988503083 conn(0x7fac28066090 msgr2=0x7fac280c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:19.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.466+0000 7fac2d129700 1 -- 172.21.15.146:0/2988503083 shutdown_connections 2024-09-13T18:38:19.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:19.466+0000 7fac2d129700 1 -- 172.21.15.146:0/2988503083 wait complete. 2024-09-13T18:38:19.487 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 55 2024-09-13T18:38:19.502 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:38:21.503 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:38:21.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d9366700 1 Processor -- start 2024-09-13T18:38:21.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d9366700 1 -- start start 2024-09-13T18:38:21.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d9366700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0d40c9350 0x7fb0d40a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:21.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d9366700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb0d40c7530 0x7fb0d40cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:21.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d9366700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d4159f80 con 0x7fb0d40c8f70 2024-09-13T18:38:21.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d9366700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d40ca1d0 con 0x7fb0d40c9350 2024-09-13T18:38:21.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d9366700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d4159670 con 0x7fb0d40c7530 2024-09-13T18:38:21.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d37fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0d40c9350 0x7fb0d40a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:21.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d37fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0d40c9350 0x7fb0d40a9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48270/0 (socket says 172.21.15.146:48270) 2024-09-13T18:38:21.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d37fe700 1 -- 172.21.15.146:0/3798125292 learned_addr learned my addr 172.21.15.146:0/3798125292 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:21.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d8b65700 1 --2- 172.21.15.146:0/3798125292 >> v2:172.21.15.154:3300/0 conn(0x7fb0d40c7530 0x7fb0d40cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:21.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d37fe700 1 -- 172.21.15.146:0/3798125292 >> v2:172.21.15.154:3300/0 conn(0x7fb0d40c7530 msgr2=0x7fb0d40cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:21.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d37fe700 1 --2- 172.21.15.146:0/3798125292 >> v2:172.21.15.154:3300/0 conn(0x7fb0d40c7530 0x7fb0d40cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:21.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d37fe700 1 -- 172.21.15.146:0/3798125292 >> v1:172.21.15.154:6789/0 conn(0x7fb0d40c8f70 legacy=0x7fb0d40723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:21.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d37fe700 1 -- 172.21.15.146:0/3798125292 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d40aa3a0 con 0x7fb0d40c9350 2024-09-13T18:38:21.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.606+0000 7fb0d37fe700 1 --2- 172.21.15.146:0/3798125292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0d40c9350 0x7fb0d40a9cf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb0d406edb0 tx=0x7fb0c4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983039b72b7bfd32 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:21.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.610+0000 7fb0d2ffd700 1 -- 172.21.15.146:0/3798125292 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4011830 con 0x7fb0d40c9350 2024-09-13T18:38:21.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.610+0000 7fb0d2ffd700 1 -- 172.21.15.146:0/3798125292 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c4011990 con 0x7fb0d40c9350 2024-09-13T18:38:21.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.610+0000 7fb0d2ffd700 1 -- 172.21.15.146:0/3798125292 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4011d20 con 0x7fb0d40c9350 2024-09-13T18:38:21.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.610+0000 7fb0d9366700 1 -- 172.21.15.146:0/3798125292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0d40c9350 msgr2=0x7fb0d40a9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:21.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.610+0000 7fb0d9366700 1 --2- 172.21.15.146:0/3798125292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0d40c9350 0x7fb0d40a9cf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb0d406edb0 tx=0x7fb0c4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:38:21.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.610+0000 7fb0d9366700 1 -- 172.21.15.146:0/3798125292 shutdown_connections 2024-09-13T18:38:21.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.610+0000 7fb0d9366700 1 --2- 172.21.15.146:0/3798125292 >> v2:172.21.15.154:3300/0 conn(0x7fb0d40c7530 0x7fb0d40cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:21.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.610+0000 7fb0d9366700 1 --2- 172.21.15.146:0/3798125292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0d40c9350 0x7fb0d40a9cf0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:21.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.610+0000 7fb0d9366700 1 -- 172.21.15.146:0/3798125292 >> 172.21.15.146:0/3798125292 conn(0x7fb0d4066090 msgr2=0x7fb0d4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:21.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.610+0000 7fb0d9366700 1 -- 172.21.15.146:0/3798125292 shutdown_connections 2024-09-13T18:38:21.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.610+0000 7fb0d9366700 1 -- 172.21.15.146:0/3798125292 wait complete. 2024-09-13T18:38:21.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.610+0000 7fb0d9366700 1 Processor -- start 2024-09-13T18:38:21.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.610+0000 7fb0d9366700 1 -- start start 2024-09-13T18:38:21.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d9366700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0d40c7530 0x7fb0d41efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:21.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d9366700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0d40c8f70 0x7fb0d40abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:21.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d9366700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0d40c9350 0x7fb0d4153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:21.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d9366700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d41ef250 con 0x7fb0d40c9350 2024-09-13T18:38:21.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d9366700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d40ad690 con 0x7fb0d40c7530 2024-09-13T18:38:21.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d9366700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d40ab140 con 0x7fb0d40c8f70 2024-09-13T18:38:21.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d8b65700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0d40c9350 0x7fb0d4153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:21.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d8b65700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0d40c9350 0x7fb0d4153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48274/0 (socket says 172.21.15.146:48274) 2024-09-13T18:38:21.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d8b65700 1 -- 172.21.15.146:0/3574883447 learned_addr learned my addr 172.21.15.146:0/3574883447 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:21.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d3fff700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0d40c7530 0x7fb0d41efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:21.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d37fe700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0d40c8f70 0x7fb0d40abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:21.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d8b65700 1 -- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0d40c8f70 msgr2=0x7fb0d40abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:21.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d8b65700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0d40c8f70 0x7fb0d40abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:21.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d8b65700 1 -- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0d40c7530 msgr2=0x7fb0d41efcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:21.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d8b65700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0d40c7530 0x7fb0d41efcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:21.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d8b65700 1 -- 172.21.15.146:0/3574883447 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c4009570 con 0x7fb0d40c9350 2024-09-13T18:38:21.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d8b65700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0d40c9350 0x7fb0d4153610 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00efc0 tx=0x7fb0cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:21.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d0ff9700 1 -- 172.21.15.146:0/3574883447 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc009ea0 con 0x7fb0d40c9350 2024-09-13T18:38:21.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d0ff9700 1 -- 172.21.15.146:0/3574883447 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0cc00f040 con 0x7fb0d40c9350 2024-09-13T18:38:21.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d9366700 1 -- 172.21.15.146:0/3574883447 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d40ab420 con 0x7fb0d40c9350 2024-09-13T18:38:21.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d0ff9700 1 -- 172.21.15.146:0/3574883447 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc00ced0 con 0x7fb0d40c9350 2024-09-13T18:38:21.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d9366700 1 -- 172.21.15.146:0/3574883447 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d41ef7d0 con 0x7fb0d40c9350 2024-09-13T18:38:21.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.614+0000 7fb0d9366700 1 -- 172.21.15.146:0/3574883447 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d40c4e40 con 0x7fb0d40c9350 2024-09-13T18:38:21.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.618+0000 7fb0d0ff9700 1 -- 172.21.15.146:0/3574883447 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb0cc014480 con 0x7fb0d40c9350 2024-09-13T18:38:21.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.618+0000 7fb0d0ff9700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb0c0066730 0x7fb0c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:21.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.618+0000 7fb0d0ff9700 1 -- 172.21.15.146:0/3574883447 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fb0cc085100 con 0x7fb0d40c9350 2024-09-13T18:38:21.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.618+0000 7fb0d3fff700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb0c0066730 0x7fb0c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:21.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.618+0000 7fb0d3fff700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb0c0066730 0x7fb0c0068bf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80018a0 tx=0x7fb0c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:21.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.622+0000 7fb0d0ff9700 1 -- 172.21.15.146:0/3574883447 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb0cc051dd0 con 0x7fb0d40c9350 2024-09-13T18:38:21.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.834+0000 7fb0d9366700 1 -- 172.21.15.146:0/3574883447 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0d415a410 con 0x7fb0d40c9350 2024-09-13T18:38:21.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.834+0000 7fb0d0ff9700 1 -- 172.21.15.146:0/3574883447 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5533 (secure 0 0 0) 0x7fb0cc018070 con 0x7fb0d40c9350 2024-09-13T18:38:21.854 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:21.854 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:16.788778+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:21.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.838+0000 7fb0d9366700 1 -- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb0c0066730 msgr2=0x7fb0c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:21.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.838+0000 7fb0d9366700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb0c0066730 0x7fb0c0068bf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80018a0 tx=0x7fb0c8009040 comp rx=0 tx=0).stop 2024-09-13T18:38:21.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.838+0000 7fb0d9366700 1 -- 172.21.15.146:0/3574883447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0d40c9350 msgr2=0x7fb0d4153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:21.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.838+0000 7fb0d9366700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0d40c9350 0x7fb0d4153610 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00efc0 tx=0x7fb0cc00c3a0 comp rx=0 tx=0).stop 2024-09-13T18:38:21.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.838+0000 7fb0d9366700 1 -- 172.21.15.146:0/3574883447 shutdown_connections 2024-09-13T18:38:21.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.838+0000 7fb0d9366700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb0c0066730 0x7fb0c0068bf0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:21.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.838+0000 7fb0d9366700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0d40c9350 0x7fb0d4153610 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:21.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.838+0000 7fb0d9366700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0d40c8f70 0x7fb0d40abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:21.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.838+0000 7fb0d9366700 1 --2- 172.21.15.146:0/3574883447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0d40c7530 0x7fb0d41efcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:21.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.838+0000 7fb0d9366700 1 -- 172.21.15.146:0/3574883447 >> 172.21.15.146:0/3574883447 conn(0x7fb0d4066090 msgr2=0x7fb0d406c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:21.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.838+0000 7fb0d9366700 1 -- 172.21.15.146:0/3574883447 shutdown_connections 2024-09-13T18:38:21.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:21.838+0000 7fb0d9366700 1 -- 172.21.15.146:0/3574883447 wait complete. 2024-09-13T18:38:21.858 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 55 2024-09-13T18:38:21.870 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:38:23.871 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:38:23.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.978+0000 7f2884402700 1 Processor -- start 2024-09-13T18:38:23.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.978+0000 7f2884402700 1 -- start start 2024-09-13T18:38:23.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.978+0000 7f2884402700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f287c1506c0 0x7f287c15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:23.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.978+0000 7f2884402700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f287c0c10f0 0x7f287c160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:23.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f2884402700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f287c0c15c0 con 0x7f287c0c1fd0 2024-09-13T18:38:23.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f2884402700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287c0c1e60 con 0x7f287c0c10f0 2024-09-13T18:38:24.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f2884402700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287c0c18a0 con 0x7f287c1506c0 2024-09-13T18:38:24.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f288299f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f287c0c10f0 0x7f287c160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:24.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f288299f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f287c0c10f0 0x7f287c160ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48286/0 (socket says 172.21.15.146:48286) 2024-09-13T18:38:24.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f288299f700 1 -- 172.21.15.146:0/2557307325 learned_addr learned my addr 172.21.15.146:0/2557307325 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:24.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f288219e700 1 --2- 172.21.15.146:0/2557307325 >> v2:172.21.15.154:3300/0 conn(0x7f287c1506c0 0x7f287c15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:24.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f288299f700 1 -- 172.21.15.146:0/2557307325 >> v2:172.21.15.154:3300/0 conn(0x7f287c1506c0 msgr2=0x7f287c15a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:24.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f288299f700 1 --2- 172.21.15.146:0/2557307325 >> v2:172.21.15.154:3300/0 conn(0x7f287c1506c0 0x7f287c15a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:24.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f288299f700 1 -- 172.21.15.146:0/2557307325 >> v1:172.21.15.154:6789/0 conn(0x7f287c0c1fd0 legacy=0x7f287c0c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:24.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f288299f700 1 -- 172.21.15.146:0/2557307325 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c0c1b80 con 0x7f287c0c10f0 2024-09-13T18:38:24.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f288299f700 1 --2- 172.21.15.146:0/2557307325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c0c10f0 0x7f287c160ca0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f287c06f200 tx=0x7f287400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabc0ba93456eed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:24.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f288119c700 1 -- 172.21.15.146:0/2557307325 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874010070 con 0x7f287c0c10f0 2024-09-13T18:38:24.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f288119c700 1 -- 172.21.15.146:0/2557307325 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287400dce0 con 0x7f287c0c10f0 2024-09-13T18:38:24.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f288119c700 1 -- 172.21.15.146:0/2557307325 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287400b310 con 0x7f287c0c10f0 2024-09-13T18:38:24.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f2884402700 1 -- 172.21.15.146:0/2557307325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c0c10f0 msgr2=0x7f287c160ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:24.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f2884402700 1 --2- 172.21.15.146:0/2557307325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c0c10f0 0x7f287c160ca0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f287c06f200 tx=0x7f287400d430 comp rx=0 tx=0).stop 2024-09-13T18:38:24.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f2884402700 1 -- 172.21.15.146:0/2557307325 shutdown_connections 2024-09-13T18:38:24.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f2884402700 1 --2- 172.21.15.146:0/2557307325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c0c10f0 0x7f287c160ca0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:24.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f2884402700 1 --2- 172.21.15.146:0/2557307325 >> v2:172.21.15.154:3300/0 conn(0x7f287c1506c0 0x7f287c15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:24.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f2884402700 1 -- 172.21.15.146:0/2557307325 >> 172.21.15.146:0/2557307325 conn(0x7f287c0cc0d0 msgr2=0x7f287c0cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:24.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f2884402700 1 -- 172.21.15.146:0/2557307325 shutdown_connections 2024-09-13T18:38:24.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.982+0000 7f2884402700 1 -- 172.21.15.146:0/2557307325 wait complete. 2024-09-13T18:38:24.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.986+0000 7f2884402700 1 Processor -- start 2024-09-13T18:38:24.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.986+0000 7f2884402700 1 -- start start 2024-09-13T18:38:24.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.986+0000 7f2884402700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f287c0c10f0 0x7f287c0adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:24.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.986+0000 7f2884402700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c0c1fd0 0x7f287c14c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:24.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.986+0000 7f2884402700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f287c1506c0 0x7f287c14cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:24.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.986+0000 7f2884402700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c14b7c0 con 0x7f287c0c1fd0 2024-09-13T18:38:24.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.986+0000 7f2884402700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c159280 con 0x7f287c0c10f0 2024-09-13T18:38:24.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.986+0000 7f2884402700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0b1790 con 0x7f287c1506c0 2024-09-13T18:38:24.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.986+0000 7f288199d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c0c1fd0 0x7f287c14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:24.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.986+0000 7f288219e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f287c0c10f0 0x7f287c0adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:24.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.986+0000 7f288199d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c0c1fd0 0x7f287c14c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48300/0 (socket says 172.21.15.146:48300) 2024-09-13T18:38:24.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.986+0000 7f288199d700 1 -- 172.21.15.146:0/334691086 learned_addr learned my addr 172.21.15.146:0/334691086 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:24.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.986+0000 7f288299f700 1 --2- 172.21.15.146:0/334691086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f287c1506c0 0x7f287c14cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:24.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.990+0000 7f288199d700 1 -- 172.21.15.146:0/334691086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f287c1506c0 msgr2=0x7f287c14cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:24.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.990+0000 7f288199d700 1 --2- 172.21.15.146:0/334691086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f287c1506c0 0x7f287c14cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:24.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.990+0000 7f288199d700 1 -- 172.21.15.146:0/334691086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f287c0c10f0 msgr2=0x7f287c0adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:24.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.990+0000 7f288199d700 1 --2- 172.21.15.146:0/334691086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f287c0c10f0 0x7f287c0adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:24.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.990+0000 7f288199d700 1 -- 172.21.15.146:0/334691086 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287400bd40 con 0x7f287c0c1fd0 2024-09-13T18:38:24.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.990+0000 7f288199d700 1 --2- 172.21.15.146:0/334691086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c0c1fd0 0x7f287c14c580 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f287000c500 tx=0x7f287000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:24.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.990+0000 7f287affd700 1 -- 172.21.15.146:0/334691086 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28700052d0 con 0x7f287c0c1fd0 2024-09-13T18:38:24.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.990+0000 7f287affd700 1 -- 172.21.15.146:0/334691086 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2870005430 con 0x7f287c0c1fd0 2024-09-13T18:38:24.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.990+0000 7f2884402700 1 -- 172.21.15.146:0/334691086 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c1583d0 con 0x7f287c0c1fd0 2024-09-13T18:38:24.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.990+0000 7f287affd700 1 -- 172.21.15.146:0/334691086 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28700055a0 con 0x7f287c0c1fd0 2024-09-13T18:38:24.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.990+0000 7f2884402700 1 -- 172.21.15.146:0/334691086 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287c1597b0 con 0x7f287c0c1fd0 2024-09-13T18:38:24.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.990+0000 7f2884402700 1 -- 172.21.15.146:0/334691086 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287c070a10 con 0x7f287c0c1fd0 2024-09-13T18:38:24.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.990+0000 7f287affd700 1 -- 172.21.15.146:0/334691086 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f28700038e0 con 0x7f287c0c1fd0 2024-09-13T18:38:24.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.994+0000 7f287affd700 1 --2- 172.21.15.146:0/334691086 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2868066780 0x7f2868068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:24.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.994+0000 7f287affd700 1 -- 172.21.15.146:0/334691086 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f2870083aa0 con 0x7f287c0c1fd0 2024-09-13T18:38:24.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.994+0000 7f288219e700 1 --2- 172.21.15.146:0/334691086 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2868066780 0x7f2868068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:24.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.994+0000 7f288219e700 1 --2- 172.21.15.146:0/334691086 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2868066780 0x7f2868068c40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f286c009b50 tx=0x7f286c009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:24.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:23.998+0000 7f287affd700 1 -- 172.21.15.146:0/334691086 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f28700508d0 con 0x7f287c0c1fd0 2024-09-13T18:38:24.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.206+0000 7f2884402700 1 -- 172.21.15.146:0/334691086 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f287c06ee60 con 0x7f287c0c1fd0 2024-09-13T18:38:24.224 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:24.224 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:22.169554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:24.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.206+0000 7f287affd700 1 -- 172.21.15.146:0/334691086 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5533 (secure 0 0 0) 0x7f2870017020 con 0x7f287c0c1fd0 2024-09-13T18:38:24.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.206+0000 7f2884402700 1 -- 172.21.15.146:0/334691086 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2868066780 msgr2=0x7f2868068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:24.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.210+0000 7f2884402700 1 --2- 172.21.15.146:0/334691086 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2868066780 0x7f2868068c40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f286c009b50 tx=0x7f286c009240 comp rx=0 tx=0).stop 2024-09-13T18:38:24.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.210+0000 7f2884402700 1 -- 172.21.15.146:0/334691086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c0c1fd0 msgr2=0x7f287c14c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:24.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.210+0000 7f2884402700 1 --2- 172.21.15.146:0/334691086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c0c1fd0 0x7f287c14c580 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f287000c500 tx=0x7f287000c810 comp rx=0 tx=0).stop 2024-09-13T18:38:24.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.210+0000 7f2884402700 1 -- 172.21.15.146:0/334691086 shutdown_connections 2024-09-13T18:38:24.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.210+0000 7f2884402700 1 --2- 172.21.15.146:0/334691086 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2868066780 0x7f2868068c40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:24.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.210+0000 7f2884402700 1 --2- 172.21.15.146:0/334691086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f287c1506c0 0x7f287c14cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:24.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.210+0000 7f2884402700 1 --2- 172.21.15.146:0/334691086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c0c1fd0 0x7f287c14c580 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:24.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.210+0000 7f2884402700 1 --2- 172.21.15.146:0/334691086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f287c0c10f0 0x7f287c0adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:24.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.210+0000 7f2884402700 1 -- 172.21.15.146:0/334691086 >> 172.21.15.146:0/334691086 conn(0x7f287c0cc0d0 msgr2=0x7f287c0c6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:24.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.210+0000 7f2884402700 1 -- 172.21.15.146:0/334691086 shutdown_connections 2024-09-13T18:38:24.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:24.210+0000 7f2884402700 1 -- 172.21.15.146:0/334691086 wait complete. 2024-09-13T18:38:24.237 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 56 2024-09-13T18:38:24.248 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=0, replacing gid=5505 2024-09-13T18:38:24.249 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.a 2024-09-13T18:38:29.940 INFO:tasks.ceph.mds.f.smithi154.stderr:2024-09-13T18:38:29.937+0000 7fcbfff40700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:38:29.940 INFO:tasks.ceph.mds.f.smithi154.stderr:2024-09-13T18:38:29.937+0000 7fcbfff40700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-13T18:38:29.940 INFO:tasks.ceph.mds.f.smithi154.stderr:2024-09-13T18:38:29.937+0000 7fcbfff40700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-13T18:38:30.250 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-09-13T18:38:30.250 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T18:38:30.250 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T18:38:30.253 INFO:tasks.ceph.mds.a:Started 2024-09-13T18:38:30.253 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:38:30.342 INFO:tasks.ceph.mds.a.smithi146.stdout:starting mds.a at 2024-09-13T18:38:30.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 Processor -- start 2024-09-13T18:38:30.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 -- start start 2024-09-13T18:38:30.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9f0151590 0x7ff9f01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:30.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff9f006db40 0x7ff9f0161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:30.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f0158d10 con 0x7ff9f0157fd0 2024-09-13T18:38:30.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f00c69b0 con 0x7ff9f006db40 2024-09-13T18:38:30.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f01572e0 con 0x7ff9f0151590 2024-09-13T18:38:30.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f4d55700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff9f006db40 0x7ff9f0161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:30.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f4d55700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff9f006db40 0x7ff9f0161c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48318/0 (socket says 172.21.15.146:48318) 2024-09-13T18:38:30.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f4d55700 1 -- 172.21.15.146:0/3259102779 learned_addr learned my addr 172.21.15.146:0/3259102779 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:30.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9effff700 1 --2- 172.21.15.146:0/3259102779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9f0151590 0x7ff9f01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:30.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f4d55700 1 -- 172.21.15.146:0/3259102779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9f0151590 msgr2=0x7ff9f01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:30.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f4d55700 1 --2- 172.21.15.146:0/3259102779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9f0151590 0x7ff9f01587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:30.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f4d55700 1 -- 172.21.15.146:0/3259102779 >> v1:172.21.15.154:6789/0 conn(0x7ff9f0157fd0 legacy=0x7ff9f00c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:30.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f4d55700 1 -- 172.21.15.146:0/3259102779 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f0066500 con 0x7ff9f006db40 2024-09-13T18:38:30.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9effff700 1 --2- 172.21.15.146:0/3259102779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9f0151590 0x7ff9f01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:38:30.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f4d55700 1 --2- 172.21.15.146:0/3259102779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9f006db40 0x7ff9f0161c60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009e50 tx=0x7ff9e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172b19bf40a2c946 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:30.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9eeffd700 1 -- 172.21.15.146:0/3259102779 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e800e070 con 0x7ff9f006db40 2024-09-13T18:38:30.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9eeffd700 1 -- 172.21.15.146:0/3259102779 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e800b9c0 con 0x7ff9f006db40 2024-09-13T18:38:30.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9eeffd700 1 -- 172.21.15.146:0/3259102779 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e800bd70 con 0x7ff9f006db40 2024-09-13T18:38:30.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 -- 172.21.15.146:0/3259102779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9f006db40 msgr2=0x7ff9f0161c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:30.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 --2- 172.21.15.146:0/3259102779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9f006db40 0x7ff9f0161c60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009e50 tx=0x7ff9e800b330 comp rx=0 tx=0).stop 2024-09-13T18:38:30.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 -- 172.21.15.146:0/3259102779 shutdown_connections 2024-09-13T18:38:30.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 --2- 172.21.15.146:0/3259102779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9f006db40 0x7ff9f0161c60 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:30.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 --2- 172.21.15.146:0/3259102779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9f0151590 0x7ff9f01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:30.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 -- 172.21.15.146:0/3259102779 >> 172.21.15.146:0/3259102779 conn(0x7ff9f0066b00 msgr2=0x7ff9f006aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:30.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 -- 172.21.15.146:0/3259102779 shutdown_connections 2024-09-13T18:38:30.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.354+0000 7ff9f5556700 1 -- 172.21.15.146:0/3259102779 wait complete. 2024-09-13T18:38:30.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f5556700 1 Processor -- start 2024-09-13T18:38:30.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f5556700 1 -- start start 2024-09-13T18:38:30.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f5556700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9f006db40 0x7ff9f00ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:30.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f5556700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9f0151590 0x7ff9f01f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:30.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f5556700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9f0157fd0 0x7ff9f01ed4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:30.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f5556700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f01f11b0 con 0x7ff9f0157fd0 2024-09-13T18:38:30.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f5556700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f0152130 con 0x7ff9f006db40 2024-09-13T18:38:30.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f5556700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f01536f0 con 0x7ff9f0151590 2024-09-13T18:38:30.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f4d55700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9f0157fd0 0x7ff9f01ed4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:30.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f4d55700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9f0157fd0 0x7ff9f01ed4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48332/0 (socket says 172.21.15.146:48332) 2024-09-13T18:38:30.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f4d55700 1 -- 172.21.15.146:0/2322212855 learned_addr learned my addr 172.21.15.146:0/2322212855 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:30.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9effff700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9f006db40 0x7ff9f00ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:30.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9ef7fe700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9f0151590 0x7ff9f01f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:30.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f4d55700 1 -- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9f0151590 msgr2=0x7ff9f01f0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:30.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f4d55700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9f0151590 0x7ff9f01f0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:30.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f4d55700 1 -- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9f006db40 msgr2=0x7ff9f00ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:30.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f4d55700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9f006db40 0x7ff9f00ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:30.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f4d55700 1 -- 172.21.15.146:0/2322212855 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e8009b30 con 0x7ff9f0157fd0 2024-09-13T18:38:30.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9effff700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9f006db40 0x7ff9f00ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:38:30.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f4d55700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9f0157fd0 0x7ff9f01ed4d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00c90e0 tx=0x7ff9e8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:30.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9ecff9700 1 -- 172.21.15.146:0/2322212855 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80048b0 con 0x7ff9f0157fd0 2024-09-13T18:38:30.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9ecff9700 1 -- 172.21.15.146:0/2322212855 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e80099b0 con 0x7ff9f0157fd0 2024-09-13T18:38:30.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f5556700 1 -- 172.21.15.146:0/2322212855 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f01eccf0 con 0x7ff9f0157fd0 2024-09-13T18:38:30.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9ecff9700 1 -- 172.21.15.146:0/2322212855 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e801c680 con 0x7ff9f0157fd0 2024-09-13T18:38:30.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.358+0000 7ff9f5556700 1 -- 172.21.15.146:0/2322212855 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f0152660 con 0x7ff9f0157fd0 2024-09-13T18:38:30.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.362+0000 7ff9f5556700 1 -- 172.21.15.146:0/2322212855 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f015b630 con 0x7ff9f0157fd0 2024-09-13T18:38:30.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.362+0000 7ff9ecff9700 1 -- 172.21.15.146:0/2322212855 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9e801c7e0 con 0x7ff9f0157fd0 2024-09-13T18:38:30.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.362+0000 7ff9ecff9700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff9dc066760 0x7ff9dc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:30.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.362+0000 7ff9ecff9700 1 -- 172.21.15.146:0/2322212855 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7ff9e8086000 con 0x7ff9f0157fd0 2024-09-13T18:38:30.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.362+0000 7ff9effff700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff9dc066760 0x7ff9dc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:30.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.362+0000 7ff9effff700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff9dc066760 0x7ff9dc068c20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009a10 tx=0x7ff9e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:30.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.370+0000 7ff9ecff9700 1 -- 172.21.15.146:0/2322212855 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff9e8052d50 con 0x7ff9f0157fd0 2024-09-13T18:38:30.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.610+0000 7ff9f5556700 1 -- 172.21.15.146:0/2322212855 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9f0157ab0 con 0x7ff9f0157fd0 2024-09-13T18:38:30.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.614+0000 7ff9ecff9700 1 -- 172.21.15.146:0/2322212855 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5533 (secure 0 0 0) 0x7ff9e8056370 con 0x7ff9f0157fd0 2024-09-13T18:38:30.632 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:30.632 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:22.169554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:30.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.614+0000 7ff9f5556700 1 -- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff9dc066760 msgr2=0x7ff9dc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:30.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.614+0000 7ff9f5556700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff9dc066760 0x7ff9dc068c20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009a10 tx=0x7ff9e0008040 comp rx=0 tx=0).stop 2024-09-13T18:38:30.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.614+0000 7ff9f5556700 1 -- 172.21.15.146:0/2322212855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9f0157fd0 msgr2=0x7ff9f01ed4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:30.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.614+0000 7ff9f5556700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9f0157fd0 0x7ff9f01ed4d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00c90e0 tx=0x7ff9e8004af0 comp rx=0 tx=0).stop 2024-09-13T18:38:30.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.614+0000 7ff9f5556700 1 -- 172.21.15.146:0/2322212855 shutdown_connections 2024-09-13T18:38:30.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.614+0000 7ff9f5556700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff9dc066760 0x7ff9dc068c20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:30.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.614+0000 7ff9f5556700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9f0157fd0 0x7ff9f01ed4d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:30.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.614+0000 7ff9f5556700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9f0151590 0x7ff9f01f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:30.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.614+0000 7ff9f5556700 1 --2- 172.21.15.146:0/2322212855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9f006db40 0x7ff9f00ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:30.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.614+0000 7ff9f5556700 1 -- 172.21.15.146:0/2322212855 >> 172.21.15.146:0/2322212855 conn(0x7ff9f0066b00 msgr2=0x7ff9f015a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:30.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.614+0000 7ff9f5556700 1 -- 172.21.15.146:0/2322212855 shutdown_connections 2024-09-13T18:38:30.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:30.614+0000 7ff9f5556700 1 -- 172.21.15.146:0/2322212855 wait complete. 2024-09-13T18:38:30.636 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 56 2024-09-13T18:38:30.652 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-09-13T18:38:32.653 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:38:32.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea9b2e700 1 Processor -- start 2024-09-13T18:38:32.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea9b2e700 1 -- start start 2024-09-13T18:38:32.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea9b2e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ea40ae4c0 0x7f7ea40ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:32.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea9b2e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7ea40c7780 0x7f7ea40c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:32.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea9b2e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea40ccd80 con 0x7f7ea4150410 2024-09-13T18:38:32.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea9b2e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea406f8b0 con 0x7f7ea40c7780 2024-09-13T18:38:32.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea9b2e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea406fb90 con 0x7f7ea40ae4c0 2024-09-13T18:38:32.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea3fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7ea40c7780 0x7f7ea40c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:32.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea3fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7ea40c7780 0x7f7ea40c7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37218/0 (socket says 172.21.15.146:37218) 2024-09-13T18:38:32.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea3fff700 1 -- 172.21.15.146:0/903934864 learned_addr learned my addr 172.21.15.146:0/903934864 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:32.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea2ffd700 1 --2- 172.21.15.146:0/903934864 >> v2:172.21.15.154:3300/0 conn(0x7f7ea40ae4c0 0x7f7ea40ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:32.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea3fff700 1 -- 172.21.15.146:0/903934864 >> v2:172.21.15.154:3300/0 conn(0x7f7ea40ae4c0 msgr2=0x7f7ea40ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:32.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea3fff700 1 --2- 172.21.15.146:0/903934864 >> v2:172.21.15.154:3300/0 conn(0x7f7ea40ae4c0 0x7f7ea40ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:32.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea3fff700 1 -- 172.21.15.146:0/903934864 >> v1:172.21.15.154:6789/0 conn(0x7f7ea4150410 legacy=0x7f7ea415a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:38:32.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea3fff700 1 -- 172.21.15.146:0/903934864 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea406fe70 con 0x7f7ea40c7780 2024-09-13T18:38:32.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.746+0000 7f7ea3fff700 1 --2- 172.21.15.146:0/903934864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea40c7780 0x7f7ea40c7bb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4157310 tx=0x7f7e9c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f57861f390a63c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:32.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea27fc700 1 -- 172.21.15.146:0/903934864 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c004230 con 0x7f7ea40c7780 2024-09-13T18:38:32.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea27fc700 1 -- 172.21.15.146:0/903934864 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c004390 con 0x7f7ea40c7780 2024-09-13T18:38:32.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea27fc700 1 -- 172.21.15.146:0/903934864 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0036c0 con 0x7f7ea40c7780 2024-09-13T18:38:32.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/903934864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea40c7780 msgr2=0x7f7ea40c7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:32.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 --2- 172.21.15.146:0/903934864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea40c7780 0x7f7ea40c7bb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4157310 tx=0x7f7e9c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:38:32.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/903934864 shutdown_connections 2024-09-13T18:38:32.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 --2- 172.21.15.146:0/903934864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea40c7780 0x7f7ea40c7bb0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:32.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 --2- 172.21.15.146:0/903934864 >> v2:172.21.15.154:3300/0 conn(0x7f7ea40ae4c0 0x7f7ea40ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:32.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/903934864 >> 172.21.15.146:0/903934864 conn(0x7f7ea40cc140 msgr2=0x7f7ea40cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:32.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/903934864 shutdown_connections 2024-09-13T18:38:32.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/903934864 wait complete. 2024-09-13T18:38:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 Processor -- start 2024-09-13T18:38:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 -- start start 2024-09-13T18:38:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ea40ae4c0 0x7f7ea40afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ea40c7780 0x7f7ea40cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea4150410 0x7f7ea41f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40cbfc0 con 0x7f7ea4150410 2024-09-13T18:38:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea41f3820 con 0x7f7ea40c7780 2024-09-13T18:38:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.750+0000 7f7ea9b2e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea4159840 con 0x7f7ea40ae4c0 2024-09-13T18:38:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea4150410 0x7f7ea41f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:32.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea4150410 0x7f7ea41f32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37232/0 (socket says 172.21.15.146:37232) 2024-09-13T18:38:32.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea3fff700 1 -- 172.21.15.146:0/3960786632 learned_addr learned my addr 172.21.15.146:0/3960786632 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:38:32.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea2ffd700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ea40c7780 0x7f7ea40cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:32.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea37fe700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ea40ae4c0 0x7f7ea40afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:32.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea3fff700 1 -- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ea40ae4c0 msgr2=0x7f7ea40afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:32.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea3fff700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ea40ae4c0 0x7f7ea40afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:32.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea3fff700 1 -- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ea40c7780 msgr2=0x7f7ea40cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:32.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea3fff700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ea40c7780 0x7f7ea40cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:32.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea3fff700 1 -- 172.21.15.146:0/3960786632 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c009570 con 0x7f7ea4150410 2024-09-13T18:38:32.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea3fff700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea4150410 0x7f7ea41f32e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00b0e0 tx=0x7f7e9c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:32.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7e9bfff700 1 -- 172.21.15.146:0/3960786632 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00b660 con 0x7f7ea4150410 2024-09-13T18:38:32.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7e9bfff700 1 -- 172.21.15.146:0/3960786632 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c00b7c0 con 0x7f7ea4150410 2024-09-13T18:38:32.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/3960786632 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea40ae960 con 0x7f7ea4150410 2024-09-13T18:38:32.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7e9bfff700 1 -- 172.21.15.146:0/3960786632 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00bb70 con 0x7f7ea4150410 2024-09-13T18:38:32.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/3960786632 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea40aec00 con 0x7f7ea4150410 2024-09-13T18:38:32.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/3960786632 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea40cb7a0 con 0x7f7ea4150410 2024-09-13T18:38:32.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7e9bfff700 1 -- 172.21.15.146:0/3960786632 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7e9c00bcd0 con 0x7f7ea4150410 2024-09-13T18:38:32.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7e9bfff700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7e84066760 0x7f7e84068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:38:32.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7e9bfff700 1 -- 172.21.15.146:0/3960786632 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f7e9c085a40 con 0x7f7ea4150410 2024-09-13T18:38:32.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.754+0000 7f7ea37fe700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7e84066760 0x7f7e84068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:38:32.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.758+0000 7f7ea37fe700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7e84066760 0x7f7e84068c20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400f300 tx=0x7f7e940042e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:38:32.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:32.762+0000 7f7e9bfff700 1 -- 172.21.15.146:0/3960786632 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7ea40cb7a0 con 0x7f7ea4150410 2024-09-13T18:38:33.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.018+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/3960786632 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ea40cb7a0 con 0x7f7ea4150410 2024-09-13T18:38:33.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.018+0000 7f7e9bfff700 1 -- 172.21.15.146:0/3960786632 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6290 (secure 0 0 0) 0x7f7ea40cb7a0 con 0x7f7ea4150410 2024-09-13T18:38:33.038 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:38:33.038 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6063,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:22.169554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:38:33.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.022+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7e84066760 msgr2=0x7f7e84068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:33.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.022+0000 7f7ea9b2e700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7e84066760 0x7f7e84068c20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400f300 tx=0x7f7e940042e0 comp rx=0 tx=0).stop 2024-09-13T18:38:33.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.022+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/3960786632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea4150410 msgr2=0x7f7ea41f32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:38:33.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.022+0000 7f7ea9b2e700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea4150410 0x7f7ea41f32e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00b0e0 tx=0x7f7e9c004770 comp rx=0 tx=0).stop 2024-09-13T18:38:33.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.022+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/3960786632 shutdown_connections 2024-09-13T18:38:33.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.022+0000 7f7ea9b2e700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7e84066760 0x7f7e84068c20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:33.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.022+0000 7f7ea9b2e700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea4150410 0x7f7ea41f32e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:33.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.022+0000 7f7ea9b2e700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ea40c7780 0x7f7ea40cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:33.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.022+0000 7f7ea9b2e700 1 --2- 172.21.15.146:0/3960786632 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ea40ae4c0 0x7f7ea40afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:38:33.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.022+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/3960786632 >> 172.21.15.146:0/3960786632 conn(0x7f7ea40cc140 msgr2=0x7f7ea414b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:38:33.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.022+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/3960786632 shutdown_connections 2024-09-13T18:38:33.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:38:33.022+0000 7f7ea9b2e700 1 -- 172.21.15.146:0/3960786632 wait complete. 2024-09-13T18:38:33.042 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 57 2024-09-13T18:38:33.056 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-09-13T18:38:33.056 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 116 secs before thrashing 2024-09-13T18:38:39.247 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:38:39.249 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:39:09.302 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:39:09.305 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:39:39.357 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:39:39.361 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:40:09.404 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:40:09.407 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:40:29.057 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:29.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feaff374700 1 Processor -- start 2024-09-13T18:40:29.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feaff374700 1 -- start start 2024-09-13T18:40:29.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feaff374700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7feaf80cd280 0x7feaf80c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:29.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feaff374700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feaf806b130 0x7feaf806ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:29.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feaff374700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7feaf80cd660 0x7feaf80c2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:29.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feaff374700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf80c34c0 con 0x7feaf80cd280 2024-09-13T18:40:29.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feaff374700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf81580b0 con 0x7feaf80cd660 2024-09-13T18:40:29.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feaff374700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf81585e0 con 0x7feaf806b130 2024-09-13T18:40:29.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feafd110700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7feaf80cd280 0x7feaf80c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:29.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feafd110700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7feaf80cd280 0x7feaf80c7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55806/0 (socket says 172.21.15.146:55806) 2024-09-13T18:40:29.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feafd110700 1 -- 172.21.15.146:0/297401709 learned_addr learned my addr 172.21.15.146:0/297401709 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:29.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feafd911700 1 --2- 172.21.15.146:0/297401709 >> v2:172.21.15.154:3300/0 conn(0x7feaf80cd660 0x7feaf80c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:29.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feafc90f700 1 --2- 172.21.15.146:0/297401709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feaf806b130 0x7feaf806ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:29.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feafd110700 1 -- 172.21.15.146:0/297401709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feaf806b130 msgr2=0x7feaf806ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:29.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feafd110700 1 --2- 172.21.15.146:0/297401709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feaf806b130 0x7feaf806ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feafd110700 1 -- 172.21.15.146:0/297401709 >> v2:172.21.15.154:3300/0 conn(0x7feaf80cd660 msgr2=0x7feaf80c2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:29.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feafd110700 1 --2- 172.21.15.146:0/297401709 >> v2:172.21.15.154:3300/0 conn(0x7feaf80cd660 0x7feaf80c2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feafd110700 1 -- 172.21.15.146:0/297401709 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf806a700 con 0x7feaf80cd280 2024-09-13T18:40:29.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feafc90f700 1 --2- 172.21.15.146:0/297401709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feaf806b130 0x7feaf806ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:40:29.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feafd911700 1 --2- 172.21.15.146:0/297401709 >> v2:172.21.15.154:3300/0 conn(0x7feaf80cd660 0x7feaf80c2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:40:29.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.161+0000 7feafd110700 1 --2- 172.21.15.146:0/297401709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feaf80cd280 0x7feaf80c7ac0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7feaf806b070 tx=0x7feaf4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240fcf8d83c80fb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:29.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaeffff700 1 -- 172.21.15.146:0/297401709 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf400b8a0 con 0x7feaf80cd280 2024-09-13T18:40:29.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaeffff700 1 -- 172.21.15.146:0/297401709 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf400ba00 con 0x7feaf80cd280 2024-09-13T18:40:29.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaeffff700 1 -- 172.21.15.146:0/297401709 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf400bd90 con 0x7feaf80cd280 2024-09-13T18:40:29.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 -- 172.21.15.146:0/297401709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feaf80cd280 msgr2=0x7feaf80c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:29.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 --2- 172.21.15.146:0/297401709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feaf80cd280 0x7feaf80c7ac0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7feaf806b070 tx=0x7feaf4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 -- 172.21.15.146:0/297401709 shutdown_connections 2024-09-13T18:40:29.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 --2- 172.21.15.146:0/297401709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feaf806b130 0x7feaf806ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 --2- 172.21.15.146:0/297401709 >> v2:172.21.15.154:3300/0 conn(0x7feaf80cd660 0x7feaf80c2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 --2- 172.21.15.146:0/297401709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feaf80cd280 0x7feaf80c7ac0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 -- 172.21.15.146:0/297401709 >> 172.21.15.146:0/297401709 conn(0x7feaf8066090 msgr2=0x7feaf80664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:29.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 -- 172.21.15.146:0/297401709 shutdown_connections 2024-09-13T18:40:29.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 -- 172.21.15.146:0/297401709 wait complete. 2024-09-13T18:40:29.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 Processor -- start 2024-09-13T18:40:29.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 -- start start 2024-09-13T18:40:29.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feaf806b130 0x7feaf80c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:29.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feaf80cd280 0x7feaf8155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:29.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feaf80cd660 0x7feaf81563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:29.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf81568e0 con 0x7feaf80cd280 2024-09-13T18:40:29.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf81efba0 con 0x7feaf80cd660 2024-09-13T18:40:29.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feaff374700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf81f0100 con 0x7feaf806b130 2024-09-13T18:40:29.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feafc90f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feaf80cd280 0x7feaf8155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:29.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feafc90f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feaf80cd280 0x7feaf8155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55812/0 (socket says 172.21.15.146:55812) 2024-09-13T18:40:29.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feafc90f700 1 -- 172.21.15.146:0/3471765046 learned_addr learned my addr 172.21.15.146:0/3471765046 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:29.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feafd911700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feaf80cd660 0x7feaf81563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:29.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feafd110700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feaf806b130 0x7feaf80c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:29.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feafc90f700 1 -- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feaf806b130 msgr2=0x7feaf80c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:29.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feafc90f700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feaf806b130 0x7feaf80c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feafc90f700 1 -- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feaf80cd660 msgr2=0x7feaf81563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:29.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feafc90f700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feaf80cd660 0x7feaf81563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.165+0000 7feafc90f700 1 -- 172.21.15.146:0/3471765046 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf4009570 con 0x7feaf80cd280 2024-09-13T18:40:29.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.169+0000 7feafc90f700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feaf80cd280 0x7feaf8155e60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7feae800d800 tx=0x7feae800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:29.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.169+0000 7feaedffb700 1 -- 172.21.15.146:0/3471765046 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae80125f0 con 0x7feaf80cd280 2024-09-13T18:40:29.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.169+0000 7feaedffb700 1 -- 172.21.15.146:0/3471765046 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae8012750 con 0x7feaf80cd280 2024-09-13T18:40:29.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.169+0000 7feaff374700 1 -- 172.21.15.146:0/3471765046 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf81553c0 con 0x7feaf80cd280 2024-09-13T18:40:29.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.169+0000 7feaedffb700 1 -- 172.21.15.146:0/3471765046 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8012b00 con 0x7feaf80cd280 2024-09-13T18:40:29.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.169+0000 7feaff374700 1 -- 172.21.15.146:0/3471765046 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf8155660 con 0x7feaf80cd280 2024-09-13T18:40:29.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.169+0000 7feaff374700 1 -- 172.21.15.146:0/3471765046 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf8073890 con 0x7feaf80cd280 2024-09-13T18:40:29.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.169+0000 7feaedffb700 1 -- 172.21.15.146:0/3471765046 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feae8012c60 con 0x7feaf80cd280 2024-09-13T18:40:29.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.169+0000 7feaedffb700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7feadc066760 0x7feadc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:29.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.169+0000 7feaedffb700 1 -- 172.21.15.146:0/3471765046 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7feae8084620 con 0x7feaf80cd280 2024-09-13T18:40:29.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.173+0000 7feafd110700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7feadc066760 0x7feadc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:29.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.173+0000 7feafd110700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7feadc066760 0x7feadc068c20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000c00 tx=0x7feaf4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:29.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.177+0000 7feaedffb700 1 -- 172.21.15.146:0/3471765046 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feae8051400 con 0x7feaf80cd280 2024-09-13T18:40:29.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.405+0000 7feaff374700 1 -- 172.21.15.146:0/3471765046 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feaf80c2740 con 0x7feaf80cd280 2024-09-13T18:40:29.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.405+0000 7feaedffb700 1 -- 172.21.15.146:0/3471765046 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6290 (secure 0 0 0) 0x7feae8054a20 con 0x7feaf80cd280 2024-09-13T18:40:29.423 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:29.423 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6063,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:22.169554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:29.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.409+0000 7feaff374700 1 -- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7feadc066760 msgr2=0x7feadc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:29.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.409+0000 7feaff374700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7feadc066760 0x7feadc068c20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000c00 tx=0x7feaf4008030 comp rx=0 tx=0).stop 2024-09-13T18:40:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.409+0000 7feaff374700 1 -- 172.21.15.146:0/3471765046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feaf80cd280 msgr2=0x7feaf8155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.409+0000 7feaff374700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feaf80cd280 0x7feaf8155e60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7feae800d800 tx=0x7feae800db10 comp rx=0 tx=0).stop 2024-09-13T18:40:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.409+0000 7feaff374700 1 -- 172.21.15.146:0/3471765046 shutdown_connections 2024-09-13T18:40:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.409+0000 7feaff374700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7feadc066760 0x7feadc068c20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.409+0000 7feaff374700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feaf80cd660 0x7feaf81563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.409+0000 7feaff374700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feaf80cd280 0x7feaf8155e60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.409+0000 7feaff374700 1 --2- 172.21.15.146:0/3471765046 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feaf806b130 0x7feaf80c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.409+0000 7feaff374700 1 -- 172.21.15.146:0/3471765046 >> 172.21.15.146:0/3471765046 conn(0x7feaf8066090 msgr2=0x7feaf8066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:29.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.409+0000 7feaff374700 1 -- 172.21.15.146:0/3471765046 shutdown_connections 2024-09-13T18:40:29.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.409+0000 7feaff374700 1 -- 172.21.15.146:0/3471765046 wait complete. 2024-09-13T18:40:29.471 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 57 2024-09-13T18:40:29.487 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=2) 2024-09-13T18:40:29.488 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T18:40:29.488 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:40:29.500 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T18:40:29.500 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:29.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.601+0000 7ffb52fee700 1 Processor -- start 2024-09-13T18:40:29.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.601+0000 7ffb52fee700 1 -- start start 2024-09-13T18:40:29.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb52fee700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffb4c0c63a0 0x7ffb4c159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:29.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb52fee700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb4c0c7310 0x7ffb4c159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:29.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb52fee700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4c0c8c70 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb52fee700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c0c9bd0 con 0x7ffb4c0c63a0 2024-09-13T18:40:29.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb52fee700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c15a0e0 con 0x7ffb4c0c7310 2024-09-13T18:40:29.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb517eb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffb4c0c63a0 0x7ffb4c159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:29.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb517eb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffb4c0c63a0 0x7ffb4c159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55822/0 (socket says 172.21.15.146:55822) 2024-09-13T18:40:29.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb517eb700 1 -- 172.21.15.146:0/1320817730 learned_addr learned my addr 172.21.15.146:0/1320817730 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:29.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb527ed700 1 --2- 172.21.15.146:0/1320817730 >> v2:172.21.15.154:3300/0 conn(0x7ffb4c0c7310 0x7ffb4c159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:29.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb517eb700 1 -- 172.21.15.146:0/1320817730 >> v2:172.21.15.154:3300/0 conn(0x7ffb4c0c7310 msgr2=0x7ffb4c159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:29.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb517eb700 1 --2- 172.21.15.146:0/1320817730 >> v2:172.21.15.154:3300/0 conn(0x7ffb4c0c7310 0x7ffb4c159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb517eb700 1 -- 172.21.15.146:0/1320817730 >> v1:172.21.15.154:6789/0 conn(0x7ffb4c0c5fc0 legacy=0x7ffb4c0c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:29.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb517eb700 1 -- 172.21.15.146:0/1320817730 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c0c8f50 con 0x7ffb4c0c63a0 2024-09-13T18:40:29.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb527ed700 1 --2- 172.21.15.146:0/1320817730 >> v2:172.21.15.154:3300/0 conn(0x7ffb4c0c7310 0x7ffb4c159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:40:29.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb517eb700 1 --2- 172.21.15.146:0/1320817730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4c0c63a0 0x7ffb4c159660 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0c82b0 tx=0x7ffb3c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab48dd9fcc7dba6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:29.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb50fea700 1 -- 172.21.15.146:0/1320817730 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00dc10 con 0x7ffb4c0c63a0 2024-09-13T18:40:29.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb50fea700 1 -- 172.21.15.146:0/1320817730 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c00dd70 con 0x7ffb4c0c63a0 2024-09-13T18:40:29.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb50fea700 1 -- 172.21.15.146:0/1320817730 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c004250 con 0x7ffb4c0c63a0 2024-09-13T18:40:29.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb52fee700 1 -- 172.21.15.146:0/1320817730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4c0c63a0 msgr2=0x7ffb4c159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:29.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb52fee700 1 --2- 172.21.15.146:0/1320817730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4c0c63a0 0x7ffb4c159660 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0c82b0 tx=0x7ffb3c00bea0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb52fee700 1 -- 172.21.15.146:0/1320817730 shutdown_connections 2024-09-13T18:40:29.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb52fee700 1 --2- 172.21.15.146:0/1320817730 >> v2:172.21.15.154:3300/0 conn(0x7ffb4c0c7310 0x7ffb4c159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb52fee700 1 --2- 172.21.15.146:0/1320817730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4c0c63a0 0x7ffb4c159660 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb52fee700 1 -- 172.21.15.146:0/1320817730 >> 172.21.15.146:0/1320817730 conn(0x7ffb4c066090 msgr2=0x7ffb4c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:29.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb52fee700 1 -- 172.21.15.146:0/1320817730 shutdown_connections 2024-09-13T18:40:29.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.605+0000 7ffb52fee700 1 -- 172.21.15.146:0/1320817730 wait complete. 2024-09-13T18:40:29.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.609+0000 7ffb52fee700 1 Processor -- start 2024-09-13T18:40:29.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.609+0000 7ffb52fee700 1 -- start start 2024-09-13T18:40:29.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.609+0000 7ffb52fee700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4c0c5fc0 0x7ffb4c0c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:29.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.609+0000 7ffb52fee700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb4c0c63a0 0x7ffb4c0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:29.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.609+0000 7ffb52fee700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb4c0c7310 0x7ffb4c0af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:29.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.609+0000 7ffb52fee700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0ad540 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.609+0000 7ffb52fee700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0afbf0 con 0x7ffb4c0c63a0 2024-09-13T18:40:29.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.609+0000 7ffb52fee700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0c7dc0 con 0x7ffb4c0c7310 2024-09-13T18:40:29.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.609+0000 7ffb51fec700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4c0c5fc0 0x7ffb4c0c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:29.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.609+0000 7ffb51fec700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4c0c5fc0 0x7ffb4c0c49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55836/0 (socket says 172.21.15.146:55836) 2024-09-13T18:40:29.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.609+0000 7ffb51fec700 1 -- 172.21.15.146:0/2514158846 learned_addr learned my addr 172.21.15.146:0/2514158846 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:29.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.609+0000 7ffb517eb700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb4c0c63a0 0x7ffb4c0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:29.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb51fec700 1 -- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb4c0c7310 msgr2=0x7ffb4c0af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:29.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb527ed700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb4c0c7310 0x7ffb4c0af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:29.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb51fec700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb4c0c7310 0x7ffb4c0af210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb51fec700 1 -- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb4c0c63a0 msgr2=0x7ffb4c0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:29.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb51fec700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb4c0c63a0 0x7ffb4c0aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb51fec700 1 -- 172.21.15.146:0/2514158846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c00b820 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb517eb700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb4c0c63a0 0x7ffb4c0aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:40:29.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb51fec700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4c0c5fc0 0x7ffb4c0c49e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ffb48001790 tx=0x7ffb48001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:29.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb427fc700 1 -- 172.21.15.146:0/2514158846 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48013070 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb52fee700 1 -- 172.21.15.146:0/2514158846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4c0aa8c0 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb427fc700 1 -- 172.21.15.146:0/2514158846 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4800e2e0 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb427fc700 1 -- 172.21.15.146:0/2514158846 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb480052f0 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb52fee700 1 -- 172.21.15.146:0/2514158846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c0aaba0 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb52fee700 1 -- 172.21.15.146:0/2514158846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c0af750 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb427fc700 1 -- 172.21.15.146:0/2514158846 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffb48005450 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb427fc700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb30066730 0x7ffb30068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:29.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb427fc700 1 -- 172.21.15.146:0/2514158846 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7ffb48087f90 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.613+0000 7ffb517eb700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb30066730 0x7ffb30068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:29.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.617+0000 7ffb517eb700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb30066730 0x7ffb30068bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0c82b0 tx=0x7ffb3c014000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:29.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.621+0000 7ffb427fc700 1 -- 172.21.15.146:0/2514158846 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ffb480b9db0 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.841+0000 7ffb52fee700 1 -- 172.21.15.146:0/2514158846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb4c0cb790 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.841+0000 7ffb427fc700 1 -- 172.21.15.146:0/2514158846 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6290 (secure 0 0 0) 0x7ffb48054c60 con 0x7ffb4c0c5fc0 2024-09-13T18:40:29.858 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:29.858 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6063,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:22.169554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:29.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.845+0000 7ffb52fee700 1 -- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb30066730 msgr2=0x7ffb30068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:29.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.845+0000 7ffb52fee700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb30066730 0x7ffb30068bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0c82b0 tx=0x7ffb3c014000 comp rx=0 tx=0).stop 2024-09-13T18:40:29.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.845+0000 7ffb52fee700 1 -- 172.21.15.146:0/2514158846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4c0c5fc0 msgr2=0x7ffb4c0c49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:29.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.845+0000 7ffb52fee700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4c0c5fc0 0x7ffb4c0c49e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ffb48001790 tx=0x7ffb48001b50 comp rx=0 tx=0).stop 2024-09-13T18:40:29.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.845+0000 7ffb52fee700 1 -- 172.21.15.146:0/2514158846 shutdown_connections 2024-09-13T18:40:29.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.845+0000 7ffb52fee700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb30066730 0x7ffb30068bf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.845+0000 7ffb52fee700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb4c0c7310 0x7ffb4c0af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.845+0000 7ffb52fee700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb4c0c63a0 0x7ffb4c0aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.845+0000 7ffb52fee700 1 --2- 172.21.15.146:0/2514158846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4c0c5fc0 0x7ffb4c0c49e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:29.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.845+0000 7ffb52fee700 1 -- 172.21.15.146:0/2514158846 >> 172.21.15.146:0/2514158846 conn(0x7ffb4c066090 msgr2=0x7ffb4c156870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:29.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.845+0000 7ffb52fee700 1 -- 172.21.15.146:0/2514158846 shutdown_connections 2024-09-13T18:40:29.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:29.845+0000 7ffb52fee700 1 -- 172.21.15.146:0/2514158846 wait complete. 2024-09-13T18:40:29.861 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 57 2024-09-13T18:40:29.874 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:40:31.875 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:32.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.985+0000 7fd366987700 1 Processor -- start 2024-09-13T18:40:32.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.985+0000 7fd366987700 1 -- start start 2024-09-13T18:40:32.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.985+0000 7fd366987700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd360150610 0x7fd36006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:32.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.985+0000 7fd366987700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd360071610 0x7fd36006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:32.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.985+0000 7fd366987700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd360071410 con 0x7fd360150230 2024-09-13T18:40:32.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.985+0000 7fd366987700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3601596b0 con 0x7fd360071610 2024-09-13T18:40:32.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.985+0000 7fd366987700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd360156090 con 0x7fd360150610 2024-09-13T18:40:32.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.985+0000 7fd365985700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd360150230 0x7fd36006ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:53166/0 (socket says 172.21.15.146:53166) 2024-09-13T18:40:32.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.985+0000 7fd365985700 1 -- 172.21.15.146:0/3417465091 learned_addr learned my addr 172.21.15.146:0/3417465091 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:40:32.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.985+0000 7fd366186700 1 --2- 172.21.15.146:0/3417465091 >> v2:172.21.15.154:3300/0 conn(0x7fd360071610 0x7fd36006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:32.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.985+0000 7fd365184700 1 --2- 172.21.15.146:0/3417465091 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd360150610 0x7fd36006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:32.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.985+0000 7fd364983700 1 -- 172.21.15.146:0/3417465091 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540422301 0 0) 0x7fd360071410 con 0x7fd360150230 2024-09-13T18:40:32.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd364983700 1 -- 172.21.15.146:0/3417465091 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd34c0035d0 con 0x7fd360150230 2024-09-13T18:40:32.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd365184700 1 -- 172.21.15.146:0/3417465091 >> v2:172.21.15.154:3300/0 conn(0x7fd360071610 msgr2=0x7fd36006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:32.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd365184700 1 --2- 172.21.15.146:0/3417465091 >> v2:172.21.15.154:3300/0 conn(0x7fd360071610 0x7fd36006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:32.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd365184700 1 -- 172.21.15.146:0/3417465091 >> v1:172.21.15.154:6789/0 conn(0x7fd360150230 legacy=0x7fd36006ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:32.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd365184700 1 -- 172.21.15.146:0/3417465091 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd360156350 con 0x7fd360150610 2024-09-13T18:40:32.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd366186700 1 --2- 172.21.15.146:0/3417465091 >> v2:172.21.15.154:3300/0 conn(0x7fd360071610 0x7fd36006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:40:32.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd365184700 1 --2- 172.21.15.146:0/3417465091 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd360150610 0x7fd36006d5e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd350009850 tx=0x7fd350009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e113a8941f425a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:32.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd364983700 1 -- 172.21.15.146:0/3417465091 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350011830 con 0x7fd360150610 2024-09-13T18:40:32.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd364983700 1 -- 172.21.15.146:0/3417465091 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd350011990 con 0x7fd360150610 2024-09-13T18:40:32.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd364983700 1 -- 172.21.15.146:0/3417465091 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350011d00 con 0x7fd360150610 2024-09-13T18:40:32.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd366987700 1 -- 172.21.15.146:0/3417465091 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd360150610 msgr2=0x7fd36006d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:32.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd366987700 1 --2- 172.21.15.146:0/3417465091 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd360150610 0x7fd36006d5e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd350009850 tx=0x7fd350009b60 comp rx=0 tx=0).stop 2024-09-13T18:40:32.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd366987700 1 -- 172.21.15.146:0/3417465091 shutdown_connections 2024-09-13T18:40:32.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd366987700 1 --2- 172.21.15.146:0/3417465091 >> v2:172.21.15.154:3300/0 conn(0x7fd360071610 0x7fd36006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:32.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd366987700 1 --2- 172.21.15.146:0/3417465091 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd360150610 0x7fd36006d5e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:32.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd366987700 1 -- 172.21.15.146:0/3417465091 >> 172.21.15.146:0/3417465091 conn(0x7fd360071ed0 msgr2=0x7fd3600722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:32.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd366987700 1 -- 172.21.15.146:0/3417465091 shutdown_connections 2024-09-13T18:40:32.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.989+0000 7fd366987700 1 -- 172.21.15.146:0/3417465091 wait complete. 2024-09-13T18:40:32.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd366987700 1 Processor -- start 2024-09-13T18:40:32.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd366987700 1 -- start start 2024-09-13T18:40:32.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd366987700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd360071610 0x7fd3601ed110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:32.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd366987700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd360150230 0x7fd3601449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:32.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd366987700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd360150d40 0x7fd360144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:32.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd366987700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd360155920 con 0x7fd360150230 2024-09-13T18:40:32.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd366987700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd360153d90 con 0x7fd360071610 2024-09-13T18:40:32.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd366987700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3601edab0 con 0x7fd360150d40 2024-09-13T18:40:32.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd365184700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd360150230 0x7fd3601449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:32.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd365184700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd360150230 0x7fd3601449e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32998/0 (socket says 172.21.15.146:32998) 2024-09-13T18:40:32.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd365184700 1 -- 172.21.15.146:0/2427710881 learned_addr learned my addr 172.21.15.146:0/2427710881 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:32.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd366186700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd360150d40 0x7fd360144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:32.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd365985700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd360071610 0x7fd3601ed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:32.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd365184700 1 -- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd360150d40 msgr2=0x7fd360144f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:32.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd365184700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd360150d40 0x7fd360144f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:32.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd365184700 1 -- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd360071610 msgr2=0x7fd3601ed110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:32.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd365184700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd360071610 0x7fd3601ed110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:32.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd365184700 1 -- 172.21.15.146:0/2427710881 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd358009a70 con 0x7fd360150230 2024-09-13T18:40:32.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd365184700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd360150230 0x7fd3601449e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd350014040 tx=0x7fd350004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:32.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd3567fc700 1 -- 172.21.15.146:0/2427710881 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350004550 con 0x7fd360150230 2024-09-13T18:40:32.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd3567fc700 1 -- 172.21.15.146:0/2427710881 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35001f070 con 0x7fd360150230 2024-09-13T18:40:32.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.993+0000 7fd366987700 1 -- 172.21.15.146:0/2427710881 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3500094b0 con 0x7fd360150230 2024-09-13T18:40:32.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.997+0000 7fd3567fc700 1 -- 172.21.15.146:0/2427710881 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350003900 con 0x7fd360150230 2024-09-13T18:40:32.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.997+0000 7fd366987700 1 -- 172.21.15.146:0/2427710881 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd360154030 con 0x7fd360150230 2024-09-13T18:40:32.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.997+0000 7fd366987700 1 -- 172.21.15.146:0/2427710881 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd360153be0 con 0x7fd360150230 2024-09-13T18:40:32.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.997+0000 7fd3567fc700 1 -- 172.21.15.146:0/2427710881 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd350003a60 con 0x7fd360150230 2024-09-13T18:40:32.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.997+0000 7fd3567fc700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd344066760 0x7fd344068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:32.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.997+0000 7fd3567fc700 1 -- 172.21.15.146:0/2427710881 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fd350021070 con 0x7fd360150230 2024-09-13T18:40:32.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:31.997+0000 7fd365985700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd344066760 0x7fd344068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:32.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.001+0000 7fd365985700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd344066760 0x7fd344068c20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd35c007fd0 tx=0x7fd35c007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:32.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.005+0000 7fd3567fc700 1 -- 172.21.15.146:0/2427710881 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd350052a90 con 0x7fd360150230 2024-09-13T18:40:32.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.261+0000 7fd366987700 1 -- 172.21.15.146:0/2427710881 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3600713d0 con 0x7fd360150230 2024-09-13T18:40:32.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.261+0000 7fd3567fc700 1 -- 172.21.15.146:0/2427710881 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6290 (secure 0 0 0) 0x7fd350015090 con 0x7fd360150230 2024-09-13T18:40:32.278 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:32.278 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6063,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:22.169554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:32.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.261+0000 7fd366987700 1 -- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd344066760 msgr2=0x7fd344068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:32.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.261+0000 7fd366987700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd344066760 0x7fd344068c20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd35c007fd0 tx=0x7fd35c007de0 comp rx=0 tx=0).stop 2024-09-13T18:40:32.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.261+0000 7fd366987700 1 -- 172.21.15.146:0/2427710881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd360150230 msgr2=0x7fd3601449e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:32.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.261+0000 7fd366987700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd360150230 0x7fd3601449e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd350014040 tx=0x7fd350004770 comp rx=0 tx=0).stop 2024-09-13T18:40:32.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.265+0000 7fd366987700 1 -- 172.21.15.146:0/2427710881 shutdown_connections 2024-09-13T18:40:32.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.265+0000 7fd366987700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd344066760 0x7fd344068c20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:32.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.265+0000 7fd366987700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd360150d40 0x7fd360144f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:32.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.265+0000 7fd366987700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd360150230 0x7fd3601449e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:32.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.265+0000 7fd366987700 1 --2- 172.21.15.146:0/2427710881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd360071610 0x7fd3601ed110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:32.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.265+0000 7fd366987700 1 -- 172.21.15.146:0/2427710881 >> 172.21.15.146:0/2427710881 conn(0x7fd360071ed0 msgr2=0x7fd3600c6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:32.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.265+0000 7fd366987700 1 -- 172.21.15.146:0/2427710881 shutdown_connections 2024-09-13T18:40:32.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:32.265+0000 7fd366987700 1 -- 172.21.15.146:0/2427710881 wait complete. 2024-09-13T18:40:32.282 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 57 2024-09-13T18:40:32.293 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:40:34.294 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:34.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1eb59e700 1 Processor -- start 2024-09-13T18:40:34.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1eb59e700 1 -- start start 2024-09-13T18:40:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1eb59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd1ec0c7970 0x7fd1ec06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1eb59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd1ec06c5b0 0x7fd1ec070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1eb59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec0c98d0 con 0x7fd1ec153860 2024-09-13T18:40:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1eb59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec0c4200 con 0x7fd1ec06c5b0 2024-09-13T18:40:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1eb59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec0706d0 con 0x7fd1ec0c7970 2024-09-13T18:40:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1ead9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd1ec06c5b0 0x7fd1ec070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1ea59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd1ec153860 0x7fd1ec0c85e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59998/0 (socket says 172.21.15.146:59998) 2024-09-13T18:40:34.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1ea59c700 1 -- 172.21.15.146:0/559889546 learned_addr learned my addr 172.21.15.146:0/559889546 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:40:34.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1e3fff700 1 --2- 172.21.15.146:0/559889546 >> v2:172.21.15.154:3300/0 conn(0x7fd1ec0c7970 0x7fd1ec06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:34.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1ead9d700 1 -- 172.21.15.146:0/559889546 >> v2:172.21.15.154:3300/0 conn(0x7fd1ec0c7970 msgr2=0x7fd1ec06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:34.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1ead9d700 1 --2- 172.21.15.146:0/559889546 >> v2:172.21.15.154:3300/0 conn(0x7fd1ec0c7970 0x7fd1ec06c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:34.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1ead9d700 1 -- 172.21.15.146:0/559889546 >> v1:172.21.15.146:6789/0 conn(0x7fd1ec153860 legacy=0x7fd1ec0c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:34.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1e9d9b700 1 -- 172.21.15.146:0/559889546 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997559222 0 0) 0x7fd1dc001110 con 0x7fd1ec153860 2024-09-13T18:40:34.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1ead9d700 1 -- 172.21.15.146:0/559889546 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec159730 con 0x7fd1ec06c5b0 2024-09-13T18:40:34.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.393+0000 7fd1ead9d700 1 --2- 172.21.15.146:0/559889546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec06c5b0 0x7fd1ec070bd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec06d190 tx=0x7fd1e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31269858f558c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:34.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.397+0000 7fd1e9d9b700 1 -- 172.21.15.146:0/559889546 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e400e070 con 0x7fd1ec06c5b0 2024-09-13T18:40:34.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.397+0000 7fd1e9d9b700 1 -- 172.21.15.146:0/559889546 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e400bb20 con 0x7fd1ec06c5b0 2024-09-13T18:40:34.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.397+0000 7fd1e9d9b700 1 -- 172.21.15.146:0/559889546 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4012430 con 0x7fd1ec06c5b0 2024-09-13T18:40:34.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.397+0000 7fd1eb59e700 1 -- 172.21.15.146:0/559889546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec06c5b0 msgr2=0x7fd1ec070bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:34.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.397+0000 7fd1eb59e700 1 --2- 172.21.15.146:0/559889546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec06c5b0 0x7fd1ec070bd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec06d190 tx=0x7fd1e400b330 comp rx=0 tx=0).stop 2024-09-13T18:40:34.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.397+0000 7fd1eb59e700 1 -- 172.21.15.146:0/559889546 shutdown_connections 2024-09-13T18:40:34.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.397+0000 7fd1eb59e700 1 --2- 172.21.15.146:0/559889546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec06c5b0 0x7fd1ec070bd0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:34.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.397+0000 7fd1eb59e700 1 --2- 172.21.15.146:0/559889546 >> v2:172.21.15.154:3300/0 conn(0x7fd1ec0c7970 0x7fd1ec06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:34.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.397+0000 7fd1eb59e700 1 -- 172.21.15.146:0/559889546 >> 172.21.15.146:0/559889546 conn(0x7fd1ec0ccf70 msgr2=0x7fd1ec0cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:34.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.397+0000 7fd1eb59e700 1 -- 172.21.15.146:0/559889546 shutdown_connections 2024-09-13T18:40:34.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.397+0000 7fd1eb59e700 1 -- 172.21.15.146:0/559889546 wait complete. 2024-09-13T18:40:34.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.397+0000 7fd1eb59e700 1 Processor -- start 2024-09-13T18:40:34.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.397+0000 7fd1eb59e700 1 -- start start 2024-09-13T18:40:34.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1eb59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec06c5b0 0x7fd1ec150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:34.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1eb59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1ec0c7970 0x7fd1ec14dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:34.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1eb59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec153860 0x7fd1ec14e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:34.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1eb59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec14da50 con 0x7fd1ec153860 2024-09-13T18:40:34.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1eb59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0cc270 con 0x7fd1ec0c7970 2024-09-13T18:40:34.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1eb59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0cc4d0 con 0x7fd1ec06c5b0 2024-09-13T18:40:34.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1ead9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec153860 0x7fd1ec14e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:34.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1ead9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec153860 0x7fd1ec14e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33008/0 (socket says 172.21.15.146:33008) 2024-09-13T18:40:34.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1ead9d700 1 -- 172.21.15.146:0/4243624601 learned_addr learned my addr 172.21.15.146:0/4243624601 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:34.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1e3fff700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1ec0c7970 0x7fd1ec14dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:34.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1ea59c700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec06c5b0 0x7fd1ec150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:34.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1ead9d700 1 -- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec06c5b0 msgr2=0x7fd1ec150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:34.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1ead9d700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec06c5b0 0x7fd1ec150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:34.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1ead9d700 1 -- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1ec0c7970 msgr2=0x7fd1ec14dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:34.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1ead9d700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1ec0c7970 0x7fd1ec14dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:34.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1ead9d700 1 -- 172.21.15.146:0/4243624601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e4009a70 con 0x7fd1ec153860 2024-09-13T18:40:34.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1ead9d700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec153860 0x7fd1ec14e420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4014040 tx=0x7fd1e4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:34.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1e2ffd700 1 -- 172.21.15.146:0/4243624601 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4012430 con 0x7fd1ec153860 2024-09-13T18:40:34.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1e2ffd700 1 -- 172.21.15.146:0/4243624601 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e4020070 con 0x7fd1ec153860 2024-09-13T18:40:34.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1e2ffd700 1 -- 172.21.15.146:0/4243624601 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4004280 con 0x7fd1ec153860 2024-09-13T18:40:34.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1eb59e700 1 -- 172.21.15.146:0/4243624601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec1efe60 con 0x7fd1ec153860 2024-09-13T18:40:34.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1eb59e700 1 -- 172.21.15.146:0/4243624601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec14f4c0 con 0x7fd1ec153860 2024-09-13T18:40:34.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1eb59e700 1 -- 172.21.15.146:0/4243624601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec067a60 con 0x7fd1ec153860 2024-09-13T18:40:34.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.401+0000 7fd1e2ffd700 1 -- 172.21.15.146:0/4243624601 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd1e40043e0 con 0x7fd1ec153860 2024-09-13T18:40:34.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.405+0000 7fd1e2ffd700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd1cc066730 0x7fd1cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:34.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.405+0000 7fd1e2ffd700 1 -- 172.21.15.146:0/4243624601 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fd1e4022070 con 0x7fd1ec153860 2024-09-13T18:40:34.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.405+0000 7fd1ea59c700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd1cc066730 0x7fd1cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:34.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.405+0000 7fd1ea59c700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd1cc066730 0x7fd1cc068bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc000f60 tx=0x7fd1dc007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:34.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.409+0000 7fd1e2ffd700 1 -- 172.21.15.146:0/4243624601 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd1e4053080 con 0x7fd1ec153860 2024-09-13T18:40:34.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.645+0000 7fd1eb59e700 1 -- 172.21.15.146:0/4243624601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1ec157670 con 0x7fd1ec153860 2024-09-13T18:40:34.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.645+0000 7fd1e2ffd700 1 -- 172.21.15.146:0/4243624601 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6290 (secure 0 0 0) 0x7fd1e40566a0 con 0x7fd1ec153860 2024-09-13T18:40:34.662 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:34.662 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6063,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:22.169554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:34.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.645+0000 7fd1eb59e700 1 -- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd1cc066730 msgr2=0x7fd1cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:34.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.645+0000 7fd1eb59e700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd1cc066730 0x7fd1cc068bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc000f60 tx=0x7fd1dc007e90 comp rx=0 tx=0).stop 2024-09-13T18:40:34.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.645+0000 7fd1eb59e700 1 -- 172.21.15.146:0/4243624601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec153860 msgr2=0x7fd1ec14e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:34.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.645+0000 7fd1eb59e700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec153860 0x7fd1ec14e420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4014040 tx=0x7fd1e4004af0 comp rx=0 tx=0).stop 2024-09-13T18:40:34.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.649+0000 7fd1eb59e700 1 -- 172.21.15.146:0/4243624601 shutdown_connections 2024-09-13T18:40:34.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.649+0000 7fd1eb59e700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd1cc066730 0x7fd1cc068bf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:34.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.649+0000 7fd1eb59e700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec153860 0x7fd1ec14e420 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:34.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.649+0000 7fd1eb59e700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1ec0c7970 0x7fd1ec14dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:34.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.649+0000 7fd1eb59e700 1 --2- 172.21.15.146:0/4243624601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec06c5b0 0x7fd1ec150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:34.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.649+0000 7fd1eb59e700 1 -- 172.21.15.146:0/4243624601 >> 172.21.15.146:0/4243624601 conn(0x7fd1ec0ccf70 msgr2=0x7fd1ec0c85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:34.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.649+0000 7fd1eb59e700 1 -- 172.21.15.146:0/4243624601 shutdown_connections 2024-09-13T18:40:34.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:34.649+0000 7fd1eb59e700 1 -- 172.21.15.146:0/4243624601 wait complete. 2024-09-13T18:40:34.665 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 57 2024-09-13T18:40:34.675 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:40:36.676 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:36.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f1226631700 1 Processor -- start 2024-09-13T18:40:36.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f1226631700 1 -- start start 2024-09-13T18:40:36.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f1226631700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f122006e7c0 0x7f12200b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:36.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f1226631700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12200c1020 con 0x7f122006eba0 2024-09-13T18:40:36.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f1226631700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12201593f0 con 0x7f122015a000 2024-09-13T18:40:36.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f1226631700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12200c6a70 con 0x7f122006e7c0 2024-09-13T18:40:36.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f122562f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f122006e7c0 0x7f12200b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:36.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f122562f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f122006e7c0 0x7f12200b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33022/0 (socket says 172.21.15.146:33022) 2024-09-13T18:40:36.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f122562f700 1 -- 172.21.15.146:0/4057537140 learned_addr learned my addr 172.21.15.146:0/4057537140 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:36.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f1224e2e700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f122006eba0 0x7f12201598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60010/0 (socket says 172.21.15.146:60010) 2024-09-13T18:40:36.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f122562f700 1 -- 172.21.15.146:0/4057537140 >> v1:172.21.15.154:6789/0 conn(0x7f122015a000 legacy=0x7f12201531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:36.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f1217fff700 1 -- 172.21.15.146:0/4057537140 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364458212 0 0) 0x7f12200c1020 con 0x7f122006eba0 2024-09-13T18:40:36.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f122562f700 1 -- 172.21.15.146:0/4057537140 >> v1:172.21.15.146:6789/0 conn(0x7f122006eba0 legacy=0x7f12201598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:36.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f122562f700 1 -- 172.21.15.146:0/4057537140 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12200c1290 con 0x7f122006e7c0 2024-09-13T18:40:36.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f122562f700 1 --2- 172.21.15.146:0/4057537140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f122006e7c0 0x7f12200b11a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f121c009910 tx=0x7f121c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b597cda2ac1ea194 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:36.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f1217fff700 1 -- 172.21.15.146:0/4057537140 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c00ba60 con 0x7f122006e7c0 2024-09-13T18:40:36.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f1217fff700 1 -- 172.21.15.146:0/4057537140 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121c00bbc0 con 0x7f122006e7c0 2024-09-13T18:40:36.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.785+0000 7f1217fff700 1 -- 172.21.15.146:0/4057537140 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c009e60 con 0x7f122006e7c0 2024-09-13T18:40:36.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 -- 172.21.15.146:0/4057537140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f122006e7c0 msgr2=0x7f12200b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:36.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 --2- 172.21.15.146:0/4057537140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f122006e7c0 0x7f12200b11a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f121c009910 tx=0x7f121c009c20 comp rx=0 tx=0).stop 2024-09-13T18:40:36.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 -- 172.21.15.146:0/4057537140 shutdown_connections 2024-09-13T18:40:36.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 --2- 172.21.15.146:0/4057537140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f122006e7c0 0x7f12200b11a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:36.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 -- 172.21.15.146:0/4057537140 >> 172.21.15.146:0/4057537140 conn(0x7f1220071ed0 msgr2=0x7f12200722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:36.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 -- 172.21.15.146:0/4057537140 shutdown_connections 2024-09-13T18:40:36.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 -- 172.21.15.146:0/4057537140 wait complete. 2024-09-13T18:40:36.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 Processor -- start 2024-09-13T18:40:36.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 -- start start 2024-09-13T18:40:36.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122006e7c0 0x7f12200c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:36.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f122006eba0 0x7f122006b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:36.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f122015a000 0x7f12201ef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:36.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12200c94f0 con 0x7f122015a000 2024-09-13T18:40:36.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12201eeec0 con 0x7f122006e7c0 2024-09-13T18:40:36.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1226631700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12201f2320 con 0x7f122006eba0 2024-09-13T18:40:36.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1225e30700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f122015a000 0x7f12201ef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:36.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1225e30700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f122015a000 0x7f12201ef3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33036/0 (socket says 172.21.15.146:33036) 2024-09-13T18:40:36.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f122562f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122006e7c0 0x7f12200c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:36.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1225e30700 1 -- 172.21.15.146:0/3648094519 learned_addr learned my addr 172.21.15.146:0/3648094519 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:36.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.789+0000 7f1224e2e700 1 --2- 172.21.15.146:0/3648094519 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f122006eba0 0x7f122006b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:36.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1225e30700 1 -- 172.21.15.146:0/3648094519 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f122006eba0 msgr2=0x7f122006b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:36.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1225e30700 1 --2- 172.21.15.146:0/3648094519 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f122006eba0 0x7f122006b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:36.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1225e30700 1 -- 172.21.15.146:0/3648094519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122006e7c0 msgr2=0x7f12200c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:36.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1225e30700 1 --2- 172.21.15.146:0/3648094519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122006e7c0 0x7f12200c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:36.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1225e30700 1 -- 172.21.15.146:0/3648094519 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121c009570 con 0x7f122015a000 2024-09-13T18:40:36.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1225e30700 1 --2- 172.21.15.146:0/3648094519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f122015a000 0x7f12201ef3a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f121800cb50 tx=0x7f121800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:36.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1215ffb700 1 -- 172.21.15.146:0/3648094519 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218003c20 con 0x7f122015a000 2024-09-13T18:40:36.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1215ffb700 1 -- 172.21.15.146:0/3648094519 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1218003d80 con 0x7f122015a000 2024-09-13T18:40:36.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1226631700 1 -- 172.21.15.146:0/3648094519 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12201f25e0 con 0x7f122015a000 2024-09-13T18:40:36.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1215ffb700 1 -- 172.21.15.146:0/3648094519 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121800aa20 con 0x7f122015a000 2024-09-13T18:40:36.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1226631700 1 -- 172.21.15.146:0/3648094519 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12201f2ba0 con 0x7f122015a000 2024-09-13T18:40:36.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1215ffb700 1 -- 172.21.15.146:0/3648094519 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f121800ab80 con 0x7f122015a000 2024-09-13T18:40:36.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1226631700 1 -- 172.21.15.146:0/3648094519 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1220152c60 con 0x7f122015a000 2024-09-13T18:40:36.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1215ffb700 1 --2- 172.21.15.146:0/3648094519 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1204066730 0x7f1204068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:36.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f1215ffb700 1 -- 172.21.15.146:0/3648094519 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f12180847e0 con 0x7f122015a000 2024-09-13T18:40:36.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.793+0000 7f122562f700 1 --2- 172.21.15.146:0/3648094519 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1204066730 0x7f1204068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:36.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.797+0000 7f122562f700 1 --2- 172.21.15.146:0/3648094519 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1204066730 0x7f1204068bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f121c014040 tx=0x7f121c01c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:36.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:36.801+0000 7f1215ffb700 1 -- 172.21.15.146:0/3648094519 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f12180514a0 con 0x7f122015a000 2024-09-13T18:40:37.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.025+0000 7f1226631700 1 -- 172.21.15.146:0/3648094519 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12200b11a0 con 0x7f122015a000 2024-09-13T18:40:37.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.025+0000 7f1215ffb700 1 -- 172.21.15.146:0/3648094519 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6290 (secure 0 0 0) 0x7f12180588b0 con 0x7f122015a000 2024-09-13T18:40:37.043 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:37.044 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6063,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:22.169554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:37.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.029+0000 7f1226631700 1 -- 172.21.15.146:0/3648094519 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1204066730 msgr2=0x7f1204068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:37.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.029+0000 7f1226631700 1 --2- 172.21.15.146:0/3648094519 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1204066730 0x7f1204068bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f121c014040 tx=0x7f121c01c040 comp rx=0 tx=0).stop 2024-09-13T18:40:37.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.029+0000 7f1226631700 1 -- 172.21.15.146:0/3648094519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f122015a000 msgr2=0x7f12201ef3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:37.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.029+0000 7f1226631700 1 --2- 172.21.15.146:0/3648094519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f122015a000 0x7f12201ef3a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f121800cb50 tx=0x7f121800ce60 comp rx=0 tx=0).stop 2024-09-13T18:40:37.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.029+0000 7f1226631700 1 -- 172.21.15.146:0/3648094519 shutdown_connections 2024-09-13T18:40:37.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.029+0000 7f1226631700 1 --2- 172.21.15.146:0/3648094519 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1204066730 0x7f1204068bf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:37.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.029+0000 7f1226631700 1 --2- 172.21.15.146:0/3648094519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f122015a000 0x7f12201ef3a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:37.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.029+0000 7f1226631700 1 --2- 172.21.15.146:0/3648094519 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f122006eba0 0x7f122006b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:37.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.029+0000 7f1226631700 1 --2- 172.21.15.146:0/3648094519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122006e7c0 0x7f12200c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:37.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.029+0000 7f1226631700 1 -- 172.21.15.146:0/3648094519 >> 172.21.15.146:0/3648094519 conn(0x7f1220071ed0 msgr2=0x7f12200722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:37.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.029+0000 7f1226631700 1 -- 172.21.15.146:0/3648094519 shutdown_connections 2024-09-13T18:40:37.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:37.029+0000 7f1226631700 1 -- 172.21.15.146:0/3648094519 wait complete. 2024-09-13T18:40:37.050 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 57 2024-09-13T18:40:37.076 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:40:39.077 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.185+0000 7ff54c99f700 1 Processor -- start 2024-09-13T18:40:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.185+0000 7ff54c99f700 1 -- start start 2024-09-13T18:40:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.185+0000 7ff54c99f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff54806e7c0 0x7ff5480b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:39.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.185+0000 7ff54c99f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff54806eba0 0x7ff5480c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:39.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.185+0000 7ff54c99f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5480c1020 con 0x7ff5481598f0 2024-09-13T18:40:39.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.185+0000 7ff54c99f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5481593f0 con 0x7ff54806eba0 2024-09-13T18:40:39.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.185+0000 7ff54c99f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5480c6a70 con 0x7ff54806e7c0 2024-09-13T18:40:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.185+0000 7ff5477fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff54806e7c0 0x7ff5480b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.185+0000 7ff547fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff5481598f0 0x7ff5481531a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:53182/0 (socket says 172.21.15.146:53182) 2024-09-13T18:40:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.185+0000 7ff547fff700 1 -- 172.21.15.146:0/169425403 learned_addr learned my addr 172.21.15.146:0/169425403 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:40:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff546ffd700 1 --2- 172.21.15.146:0/169425403 >> v2:172.21.15.146:3300/0 conn(0x7ff54806eba0 0x7ff5480c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff546ffd700 1 -- 172.21.15.146:0/169425403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff54806e7c0 msgr2=0x7ff5480b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff546ffd700 1 --2- 172.21.15.146:0/169425403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff54806e7c0 0x7ff5480b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff546ffd700 1 -- 172.21.15.146:0/169425403 >> v1:172.21.15.154:6789/0 conn(0x7ff5481598f0 legacy=0x7ff5481531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff546ffd700 1 -- 172.21.15.146:0/169425403 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5480c1290 con 0x7ff54806eba0 2024-09-13T18:40:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff546ffd700 1 --2- 172.21.15.146:0/169425403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54806eba0 0x7ff5480c2ce0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff538009850 tx=0x7ff538009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34108559812ee46 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:39.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff5467fc700 1 -- 172.21.15.146:0/169425403 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff538011830 con 0x7ff54806eba0 2024-09-13T18:40:39.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff5467fc700 1 -- 172.21.15.146:0/169425403 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff538011990 con 0x7ff54806eba0 2024-09-13T18:40:39.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff5467fc700 1 -- 172.21.15.146:0/169425403 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff538011d40 con 0x7ff54806eba0 2024-09-13T18:40:39.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff54c99f700 1 -- 172.21.15.146:0/169425403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54806eba0 msgr2=0x7ff5480c2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:39.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff54c99f700 1 --2- 172.21.15.146:0/169425403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54806eba0 0x7ff5480c2ce0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff538009850 tx=0x7ff538009b60 comp rx=0 tx=0).stop 2024-09-13T18:40:39.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff54c99f700 1 -- 172.21.15.146:0/169425403 shutdown_connections 2024-09-13T18:40:39.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff54c99f700 1 --2- 172.21.15.146:0/169425403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54806eba0 0x7ff5480c2ce0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:39.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff54c99f700 1 --2- 172.21.15.146:0/169425403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff54806e7c0 0x7ff5480b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:39.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff54c99f700 1 -- 172.21.15.146:0/169425403 >> 172.21.15.146:0/169425403 conn(0x7ff548071ed0 msgr2=0x7ff5480722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:39.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff54c99f700 1 -- 172.21.15.146:0/169425403 shutdown_connections 2024-09-13T18:40:39.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.189+0000 7ff54c99f700 1 -- 172.21.15.146:0/169425403 wait complete. 2024-09-13T18:40:39.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff54c99f700 1 Processor -- start 2024-09-13T18:40:39.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff54c99f700 1 -- start start 2024-09-13T18:40:39.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff54c99f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54806e7c0 0x7ff5481f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:39.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff54c99f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff54806eba0 0x7ff54814c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff54c99f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5481598f0 0x7ff5481437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff54c99f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54814dcc0 con 0x7ff54806e7c0 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff54c99f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5481f2530 con 0x7ff5481598f0 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff54c99f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5481f3480 con 0x7ff54806eba0 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff5477fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54806e7c0 0x7ff5481f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff5477fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54806e7c0 0x7ff5481f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33052/0 (socket says 172.21.15.146:33052) 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff5477fe700 1 -- 172.21.15.146:0/3148711452 learned_addr learned my addr 172.21.15.146:0/3148711452 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff546ffd700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff54806eba0 0x7ff54814c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff547fff700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5481598f0 0x7ff5481437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff5477fe700 1 -- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff54806eba0 msgr2=0x7ff54814c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff5477fe700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff54806eba0 0x7ff54814c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff5477fe700 1 -- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5481598f0 msgr2=0x7ff5481437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff5477fe700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5481598f0 0x7ff5481437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff5477fe700 1 -- 172.21.15.146:0/3148711452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5380094b0 con 0x7ff54806e7c0 2024-09-13T18:40:39.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff5477fe700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54806e7c0 0x7ff5481f2f40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00e840 tx=0x7ff53c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:39.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff52ffff700 1 -- 172.21.15.146:0/3148711452 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0095a0 con 0x7ff54806e7c0 2024-09-13T18:40:39.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff52ffff700 1 -- 172.21.15.146:0/3148711452 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c013680 con 0x7ff54806e7c0 2024-09-13T18:40:39.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff54c99f700 1 -- 172.21.15.146:0/3148711452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5480c8c10 con 0x7ff54806e7c0 2024-09-13T18:40:39.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff52ffff700 1 -- 172.21.15.146:0/3148711452 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0139f0 con 0x7ff54806e7c0 2024-09-13T18:40:39.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff54c99f700 1 -- 172.21.15.146:0/3148711452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54814f240 con 0x7ff54806e7c0 2024-09-13T18:40:39.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.193+0000 7ff54c99f700 1 -- 172.21.15.146:0/3148711452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5480b11a0 con 0x7ff54806e7c0 2024-09-13T18:40:39.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.197+0000 7ff52ffff700 1 -- 172.21.15.146:0/3148711452 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff53c013b50 con 0x7ff54806e7c0 2024-09-13T18:40:39.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.197+0000 7ff52ffff700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff534066730 0x7ff534068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:39.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.197+0000 7ff52ffff700 1 -- 172.21.15.146:0/3148711452 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7ff53c084550 con 0x7ff54806e7c0 2024-09-13T18:40:39.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.197+0000 7ff546ffd700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff534066730 0x7ff534068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:39.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.197+0000 7ff546ffd700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff534066730 0x7ff534068bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff538009850 tx=0x7ff53801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:39.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.201+0000 7ff52ffff700 1 -- 172.21.15.146:0/3148711452 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff53c0512a0 con 0x7ff54806e7c0 2024-09-13T18:40:39.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.421+0000 7ff54c99f700 1 -- 172.21.15.146:0/3148711452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff54806e310 con 0x7ff54806e7c0 2024-09-13T18:40:39.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.425+0000 7ff52ffff700 1 -- 172.21.15.146:0/3148711452 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6290 (secure 0 0 0) 0x7ff53c0548c0 con 0x7ff54806e7c0 2024-09-13T18:40:39.440 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:39.441 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6063,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:22.169554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:39.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.425+0000 7ff54c99f700 1 -- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff534066730 msgr2=0x7ff534068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:39.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.425+0000 7ff54c99f700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff534066730 0x7ff534068bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff538009850 tx=0x7ff53801c040 comp rx=0 tx=0).stop 2024-09-13T18:40:39.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.425+0000 7ff54c99f700 1 -- 172.21.15.146:0/3148711452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54806e7c0 msgr2=0x7ff5481f2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:39.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.425+0000 7ff54c99f700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54806e7c0 0x7ff5481f2f40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00e840 tx=0x7ff53c00eb50 comp rx=0 tx=0).stop 2024-09-13T18:40:39.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.425+0000 7ff54c99f700 1 -- 172.21.15.146:0/3148711452 shutdown_connections 2024-09-13T18:40:39.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.425+0000 7ff54c99f700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff534066730 0x7ff534068bf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:39.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.425+0000 7ff54c99f700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5481598f0 0x7ff5481437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:39.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.425+0000 7ff54c99f700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff54806eba0 0x7ff54814c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:39.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.425+0000 7ff54c99f700 1 --2- 172.21.15.146:0/3148711452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54806e7c0 0x7ff5481f2f40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:39.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.425+0000 7ff54c99f700 1 -- 172.21.15.146:0/3148711452 >> 172.21.15.146:0/3148711452 conn(0x7ff548071ed0 msgr2=0x7ff548158b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:39.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.425+0000 7ff54c99f700 1 -- 172.21.15.146:0/3148711452 shutdown_connections 2024-09-13T18:40:39.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:39.425+0000 7ff54c99f700 1 -- 172.21.15.146:0/3148711452 wait complete. 2024-09-13T18:40:39.444 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 57 2024-09-13T18:40:39.455 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:40:39.475 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:40:39.479 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:40:41.456 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:41.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb636fb1700 1 Processor -- start 2024-09-13T18:40:41.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb636fb1700 1 -- start start 2024-09-13T18:40:41.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb636fb1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6280d6670 0x7fb6280dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:41.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb636fb1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb628046df0 0x7fb6280471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:41.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb636fb1700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb628047c30 con 0x7fb62804cd00 2024-09-13T18:40:41.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb636fb1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb628035230 con 0x7fb628046df0 2024-09-13T18:40:41.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb636fb1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb628035a50 con 0x7fb6280d6670 2024-09-13T18:40:41.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb635faf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6280d6670 0x7fb6280dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:41.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb6357ae700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb628046df0 0x7fb6280471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:41.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb635faf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6280d6670 0x7fb6280dc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:34158/0 (socket says 172.21.15.146:34158) 2024-09-13T18:40:41.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb635faf700 1 -- 172.21.15.146:0/3586958422 learned_addr learned my addr 172.21.15.146:0/3586958422 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:41.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb635faf700 1 -- 172.21.15.146:0/3586958422 >> v2:172.21.15.154:3300/0 conn(0x7fb628046df0 msgr2=0x7fb6280471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:41.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb635faf700 1 --2- 172.21.15.146:0/3586958422 >> v2:172.21.15.154:3300/0 conn(0x7fb628046df0 0x7fb6280471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:41.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb635faf700 1 -- 172.21.15.146:0/3586958422 >> v1:172.21.15.154:6789/0 conn(0x7fb62804cd00 legacy=0x7fb6280480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:41.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb635faf700 1 -- 172.21.15.146:0/3586958422 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6280dca60 con 0x7fb6280d6670 2024-09-13T18:40:41.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.553+0000 7fb635faf700 1 --2- 172.21.15.146:0/3586958422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6280d6670 0x7fb6280dc3d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009910 tx=0x7fb62c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61a30f13a9db0288 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:41.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb634fad700 1 -- 172.21.15.146:0/3586958422 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c011830 con 0x7fb6280d6670 2024-09-13T18:40:41.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb634fad700 1 -- 172.21.15.146:0/3586958422 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c011990 con 0x7fb6280d6670 2024-09-13T18:40:41.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb634fad700 1 -- 172.21.15.146:0/3586958422 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c011d40 con 0x7fb6280d6670 2024-09-13T18:40:41.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb636fb1700 1 -- 172.21.15.146:0/3586958422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6280d6670 msgr2=0x7fb6280dc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:41.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb636fb1700 1 --2- 172.21.15.146:0/3586958422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6280d6670 0x7fb6280dc3d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009910 tx=0x7fb62c009c20 comp rx=0 tx=0).stop 2024-09-13T18:40:41.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb636fb1700 1 -- 172.21.15.146:0/3586958422 shutdown_connections 2024-09-13T18:40:41.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb636fb1700 1 --2- 172.21.15.146:0/3586958422 >> v2:172.21.15.154:3300/0 conn(0x7fb628046df0 0x7fb6280471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:41.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb636fb1700 1 --2- 172.21.15.146:0/3586958422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6280d6670 0x7fb6280dc3d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:41.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb636fb1700 1 -- 172.21.15.146:0/3586958422 >> 172.21.15.146:0/3586958422 conn(0x7fb628009a30 msgr2=0x7fb628009e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:41.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb636fb1700 1 -- 172.21.15.146:0/3586958422 shutdown_connections 2024-09-13T18:40:41.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb636fb1700 1 -- 172.21.15.146:0/3586958422 wait complete. 2024-09-13T18:40:41.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb636fb1700 1 Processor -- start 2024-09-13T18:40:41.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb636fb1700 1 -- start start 2024-09-13T18:40:41.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.557+0000 7fb636fb1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb628046df0 0x7fb628172060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:41.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb636fb1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb62804cd00 0x7fb6280cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:41.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb636fb1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6280d6670 0x7fb6280dec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:41.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb636fb1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6280ccbc0 con 0x7fb62804cd00 2024-09-13T18:40:41.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb636fb1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb628176520 con 0x7fb628046df0 2024-09-13T18:40:41.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb636fb1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb628175700 con 0x7fb6280d6670 2024-09-13T18:40:41.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb6357ae700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb62804cd00 0x7fb6280cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:41.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb6357ae700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb62804cd00 0x7fb6280cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45516/0 (socket says 172.21.15.146:45516) 2024-09-13T18:40:41.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb6357ae700 1 -- 172.21.15.146:0/141331925 learned_addr learned my addr 172.21.15.146:0/141331925 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:41.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb635faf700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb628046df0 0x7fb628172060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:41.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb6367b0700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6280d6670 0x7fb6280dec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:41.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb6357ae700 1 -- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6280d6670 msgr2=0x7fb6280dec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:41.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb6357ae700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6280d6670 0x7fb6280dec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:41.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb6357ae700 1 -- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb628046df0 msgr2=0x7fb628172060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:41.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb6357ae700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb628046df0 0x7fb628172060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:41.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb6357ae700 1 -- 172.21.15.146:0/141331925 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c009570 con 0x7fb62804cd00 2024-09-13T18:40:41.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb6357ae700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb62804cd00 0x7fb6280cc680 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb62000e9a0 tx=0x7fb62000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:41.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb6267fc700 1 -- 172.21.15.146:0/141331925 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620009720 con 0x7fb62804cd00 2024-09-13T18:40:41.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb6267fc700 1 -- 172.21.15.146:0/141331925 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62000c690 con 0x7fb62804cd00 2024-09-13T18:40:41.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb636fb1700 1 -- 172.21.15.146:0/141331925 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6281759a0 con 0x7fb62804cd00 2024-09-13T18:40:41.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb6267fc700 1 -- 172.21.15.146:0/141331925 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62000ca20 con 0x7fb62804cd00 2024-09-13T18:40:41.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb636fb1700 1 -- 172.21.15.146:0/141331925 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb628175f60 con 0x7fb62804cd00 2024-09-13T18:40:41.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.561+0000 7fb636fb1700 1 -- 172.21.15.146:0/141331925 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6280d6a50 con 0x7fb62804cd00 2024-09-13T18:40:41.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.565+0000 7fb6267fc700 1 -- 172.21.15.146:0/141331925 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb62000cb80 con 0x7fb62804cd00 2024-09-13T18:40:41.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.565+0000 7fb6267fc700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb614066730 0x7fb614068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:41.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.565+0000 7fb6267fc700 1 -- 172.21.15.146:0/141331925 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fb6200846e0 con 0x7fb62804cd00 2024-09-13T18:40:41.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.565+0000 7fb635faf700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb614066730 0x7fb614068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:41.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.565+0000 7fb635faf700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb614066730 0x7fb614068bf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb62c000c00 tx=0x7fb62c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:41.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.569+0000 7fb6267fc700 1 -- 172.21.15.146:0/141331925 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb620051440 con 0x7fb62804cd00 2024-09-13T18:40:41.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.829+0000 7fb636fb1700 1 -- 172.21.15.146:0/141331925 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb62804d890 con 0x7fb62804cd00 2024-09-13T18:40:41.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.829+0000 7fb6267fc700 1 -- 172.21.15.146:0/141331925 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6290 (secure 0 0 0) 0x7fb620054a60 con 0x7fb62804cd00 2024-09-13T18:40:41.847 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:41.847 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6063,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:22.169554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:41.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.833+0000 7fb636fb1700 1 -- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb614066730 msgr2=0x7fb614068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:41.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.833+0000 7fb636fb1700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb614066730 0x7fb614068bf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb62c000c00 tx=0x7fb62c008030 comp rx=0 tx=0).stop 2024-09-13T18:40:41.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.833+0000 7fb636fb1700 1 -- 172.21.15.146:0/141331925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb62804cd00 msgr2=0x7fb6280cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:41.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.833+0000 7fb636fb1700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb62804cd00 0x7fb6280cc680 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb62000e9a0 tx=0x7fb62000ecb0 comp rx=0 tx=0).stop 2024-09-13T18:40:41.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.833+0000 7fb636fb1700 1 -- 172.21.15.146:0/141331925 shutdown_connections 2024-09-13T18:40:41.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.833+0000 7fb636fb1700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb614066730 0x7fb614068bf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:41.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.833+0000 7fb636fb1700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6280d6670 0x7fb6280dec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:41.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.833+0000 7fb636fb1700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb62804cd00 0x7fb6280cc680 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:41.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.833+0000 7fb636fb1700 1 --2- 172.21.15.146:0/141331925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb628046df0 0x7fb628172060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:41.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.833+0000 7fb636fb1700 1 -- 172.21.15.146:0/141331925 >> 172.21.15.146:0/141331925 conn(0x7fb628009a30 msgr2=0x7fb6280d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:41.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.833+0000 7fb636fb1700 1 -- 172.21.15.146:0/141331925 shutdown_connections 2024-09-13T18:40:41.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:41.833+0000 7fb636fb1700 1 -- 172.21.15.146:0/141331925 wait complete. 2024-09-13T18:40:41.852 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 57 2024-09-13T18:40:41.866 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:40:43.867 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:43.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.973+0000 7f96c359e700 1 Processor -- start 2024-09-13T18:40:43.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.973+0000 7f96c359e700 1 -- start start 2024-09-13T18:40:43.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.973+0000 7f96c359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96c40cf170 0x7f96c41595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:43.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.973+0000 7f96c359e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c415a540 con 0x7f96c40cf550 2024-09-13T18:40:43.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.973+0000 7f96c359e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c40c5220 con 0x7f96c40cb5b0 2024-09-13T18:40:43.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.973+0000 7f96c359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c4159ca0 con 0x7f96c40cf170 2024-09-13T18:40:43.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.973+0000 7f96c1d9b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f96c40cf550 0x7f96c40c56b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56884/0 (socket says 172.21.15.146:56884) 2024-09-13T18:40:43.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.973+0000 7f96c1d9b700 1 -- 172.21.15.146:0/3916636005 learned_addr learned my addr 172.21.15.146:0/3916636005 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:40:43.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.973+0000 7f96c259c700 1 --2- 172.21.15.146:0/3916636005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96c40cf170 0x7f96c41595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:43.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.973+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195525247 0 0) 0x7f96c415a540 con 0x7f96c40cf550 2024-09-13T18:40:43.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.973+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96b00035d0 con 0x7f96c40cf550 2024-09-13T18:40:43.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732842021 0 0) 0x7f96c40c5220 con 0x7f96c40cb5b0 2024-09-13T18:40:43.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96c415a540 con 0x7f96c40cb5b0 2024-09-13T18:40:43.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1982859048 0 0) 0x7f96b00035d0 con 0x7f96c40cf550 2024-09-13T18:40:43.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96c40cf170 msgr2=0x7f96c41595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:43.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 --2- 172.21.15.146:0/3916636005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96c40cf170 0x7f96c41595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:43.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 >> v1:172.21.15.154:6789/0 conn(0x7f96c40cb5b0 legacy=0x7f96c41534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:43.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c4159f80 con 0x7f96c40cf550 2024-09-13T18:40:43.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f96ac001670 con 0x7f96c40cf550 2024-09-13T18:40:43.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 >> v1:172.21.15.146:6789/0 conn(0x7f96c40cf550 legacy=0x7f96c40c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:43.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 --2- 172.21.15.146:0/3916636005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96c40cf170 0x7f96b0003c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:43.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 --2- 172.21.15.146:0/3916636005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96b00041b0 0x7f96b0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:43.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 --2- 172.21.15.146:0/3916636005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b0004db0 0x7f96b0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:43.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c4159ca0 con 0x7f96b0004db0 2024-09-13T18:40:43.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b000bb70 con 0x7f96b00041b0 2024-09-13T18:40:43.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b000bd30 con 0x7f96c40cf170 2024-09-13T18:40:43.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c2d9d700 1 --2- 172.21.15.146:0/3916636005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b0004db0 0x7f96b0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:43.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c259c700 1 --2- 172.21.15.146:0/3916636005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96c40cf170 0x7f96b0003c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:43.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c1d9b700 1 --2- 172.21.15.146:0/3916636005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96b00041b0 0x7f96b0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:43.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c2d9d700 1 -- 172.21.15.146:0/3916636005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96c40cf170 msgr2=0x7f96b0003c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:43.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c2d9d700 1 --2- 172.21.15.146:0/3916636005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96c40cf170 0x7f96b0003c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:43.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c2d9d700 1 -- 172.21.15.146:0/3916636005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96b00041b0 msgr2=0x7f96b0004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:43.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c2d9d700 1 --2- 172.21.15.146:0/3916636005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96b00041b0 0x7f96b0004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:43.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c2d9d700 1 -- 172.21.15.146:0/3916636005 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00035d0 con 0x7f96b0004db0 2024-09-13T18:40:43.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c2d9d700 1 --2- 172.21.15.146:0/3916636005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b0004db0 0x7f96b0005230 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f96bc002230 tx=0x7f96b00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efde7b3d2f639ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:43.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc001f30 con 0x7f96b0004db0 2024-09-13T18:40:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96bc00ce60 con 0x7f96b0004db0 2024-09-13T18:40:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c159a700 1 -- 172.21.15.146:0/3916636005 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc003c60 con 0x7f96b0004db0 2024-09-13T18:40:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 -- 172.21.15.146:0/3916636005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b0004db0 msgr2=0x7f96b0005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 --2- 172.21.15.146:0/3916636005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b0004db0 0x7f96b0005230 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f96bc002230 tx=0x7f96b00058a0 comp rx=0 tx=0).stop 2024-09-13T18:40:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c2d9d700 1 -- 172.21.15.146:0/3916636005 reap_dead start 2024-09-13T18:40:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 -- 172.21.15.146:0/3916636005 shutdown_connections 2024-09-13T18:40:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 -- 172.21.15.146:0/3916636005 >> 172.21.15.146:0/3916636005 conn(0x7f96c4072220 msgr2=0x7f96c4066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 -- 172.21.15.146:0/3916636005 shutdown_connections 2024-09-13T18:40:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 -- 172.21.15.146:0/3916636005 wait complete. 2024-09-13T18:40:43.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 Processor -- start 2024-09-13T18:40:44.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 -- start start 2024-09-13T18:40:44.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96c40ce1f0 0x7f96c4159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:44.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96c40ce5d0 0x7f96c4153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:44.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96c4157b60 0x7f96c41f2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:44.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c4158a30 con 0x7f96c40ce5d0 2024-09-13T18:40:44.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c414ed60 con 0x7f96c4157b60 2024-09-13T18:40:44.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.977+0000 7f96c359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c4158220 con 0x7f96c40ce1f0 2024-09-13T18:40:44.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96c40ce5d0 0x7f96c4153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:44.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96c40ce1f0 0x7f96c4159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96c40ce5d0 0x7f96c4153f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45526/0 (socket says 172.21.15.146:45526) 2024-09-13T18:40:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c1d9b700 1 -- 172.21.15.146:0/1296037167 learned_addr learned my addr 172.21.15.146:0/1296037167 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c2d9d700 1 --2- 172.21.15.146:0/1296037167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96c4157b60 0x7f96c41f2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c1d9b700 1 -- 172.21.15.146:0/1296037167 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96c40ce1f0 msgr2=0x7f96c4159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c1d9b700 1 --2- 172.21.15.146:0/1296037167 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96c40ce1f0 0x7f96c4159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c1d9b700 1 -- 172.21.15.146:0/1296037167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96c4157b60 msgr2=0x7f96c41f2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c1d9b700 1 --2- 172.21.15.146:0/1296037167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96c4157b60 0x7f96c41f2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c1d9b700 1 -- 172.21.15.146:0/1296037167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c40cecb0 con 0x7f96c40ce5d0 2024-09-13T18:40:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c1d9b700 1 --2- 172.21.15.146:0/1296037167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96c40ce5d0 0x7f96c4153f60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f96ac001c10 tx=0x7f96ac00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96baffd700 1 -- 172.21.15.146:0/1296037167 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac00c5d0 con 0x7f96c40ce5d0 2024-09-13T18:40:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96baffd700 1 -- 172.21.15.146:0/1296037167 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac00c730 con 0x7f96c40ce5d0 2024-09-13T18:40:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c359e700 1 -- 172.21.15.146:0/1296037167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c4158470 con 0x7f96c40ce5d0 2024-09-13T18:40:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c359e700 1 -- 172.21.15.146:0/1296037167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c4158d40 con 0x7f96c40ce5d0 2024-09-13T18:40:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96baffd700 1 -- 172.21.15.146:0/1296037167 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac00cae0 con 0x7f96c40ce5d0 2024-09-13T18:40:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96baffd700 1 -- 172.21.15.146:0/1296037167 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f96ac00cc40 con 0x7f96c40ce5d0 2024-09-13T18:40:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96baffd700 1 --2- 172.21.15.146:0/1296037167 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f96a4066730 0x7f96a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96baffd700 1 -- 172.21.15.146:0/1296037167 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f96ac0844c0 con 0x7f96c40ce5d0 2024-09-13T18:40:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.981+0000 7f96c259c700 1 --2- 172.21.15.146:0/1296037167 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f96a4066730 0x7f96a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.985+0000 7f96c359e700 1 -- 172.21.15.146:0/1296037167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a8005310 con 0x7f96c40ce5d0 2024-09-13T18:40:44.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.985+0000 7f96c259c700 1 --2- 172.21.15.146:0/1296037167 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f96a4066730 0x7f96a4068bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009b70 tx=0x7f96b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:44.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:43.989+0000 7f96baffd700 1 -- 172.21.15.146:0/1296037167 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f96ac051170 con 0x7f96c40ce5d0 2024-09-13T18:40:44.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.205+0000 7f96c359e700 1 -- 172.21.15.146:0/1296037167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96a8005b40 con 0x7f96c40ce5d0 2024-09-13T18:40:44.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.209+0000 7f96baffd700 1 -- 172.21.15.146:0/1296037167 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6290 (secure 0 0 0) 0x7f96ac054790 con 0x7f96c40ce5d0 2024-09-13T18:40:44.225 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:44.225 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6063,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:38:22.169554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"c","rank":2,"incarnation":40,"state":"up:active","state_seq":24,"addr":"172.21.15.146:6837/3926931671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3926931671},{"type":"v1","addr":"172.21.15.146:6837","nonce":3926931671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:44.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.209+0000 7f96c359e700 1 -- 172.21.15.146:0/1296037167 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f96a4066730 msgr2=0x7f96a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:44.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.209+0000 7f96c359e700 1 --2- 172.21.15.146:0/1296037167 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f96a4066730 0x7f96a4068bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009b70 tx=0x7f96b4008040 comp rx=0 tx=0).stop 2024-09-13T18:40:44.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.209+0000 7f96c359e700 1 -- 172.21.15.146:0/1296037167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96c40ce5d0 msgr2=0x7f96c4153f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:44.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.209+0000 7f96c359e700 1 --2- 172.21.15.146:0/1296037167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96c40ce5d0 0x7f96c4153f60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f96ac001c10 tx=0x7f96ac00edd0 comp rx=0 tx=0).stop 2024-09-13T18:40:44.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.209+0000 7f96c359e700 1 -- 172.21.15.146:0/1296037167 shutdown_connections 2024-09-13T18:40:44.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.209+0000 7f96c359e700 1 --2- 172.21.15.146:0/1296037167 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f96a4066730 0x7f96a4068bf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:44.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.209+0000 7f96c359e700 1 --2- 172.21.15.146:0/1296037167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96c4157b60 0x7f96c41f2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:44.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.209+0000 7f96c359e700 1 --2- 172.21.15.146:0/1296037167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96c40ce5d0 0x7f96c4153f60 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:44.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.209+0000 7f96c359e700 1 --2- 172.21.15.146:0/1296037167 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96c40ce1f0 0x7f96c4159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:44.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.209+0000 7f96c359e700 1 -- 172.21.15.146:0/1296037167 >> 172.21.15.146:0/1296037167 conn(0x7f96c4072220 msgr2=0x7f96c414f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:44.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.209+0000 7f96c359e700 1 -- 172.21.15.146:0/1296037167 shutdown_connections 2024-09-13T18:40:44.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:44.209+0000 7f96c359e700 1 -- 172.21.15.146:0/1296037167 wait complete. 2024-09-13T18:40:44.229 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 57 2024-09-13T18:40:44.239 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:40:46.241 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:46.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6f59e700 1 Processor -- start 2024-09-13T18:40:46.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6f59e700 1 -- start start 2024-09-13T18:40:46.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6f59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4f680d69b0 0x7f4f6804e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:46.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6f59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4f6804ccb0 0x7f4f6804eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:46.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6f59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f680470f0 con 0x7f4f6800d530 2024-09-13T18:40:46.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6f59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f68048050 con 0x7f4f680d69b0 2024-09-13T18:40:46.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6f59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f68046ee0 con 0x7f4f6804ccb0 2024-09-13T18:40:46.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6e59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4f680d69b0 0x7f4f6804e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:46.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6e59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4f680d69b0 0x7f4f6804e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45528/0 (socket says 172.21.15.146:45528) 2024-09-13T18:40:46.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6e59c700 1 -- 172.21.15.146:0/1661777069 learned_addr learned my addr 172.21.15.146:0/1661777069 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:46.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6ed9d700 1 --1- 172.21.15.146:0/1661777069 >> v1:172.21.15.146:6789/0 conn(0x7f4f6800d530 0x7f4f6800be60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56888/0 (socket says 172.21.15.146:56888) 2024-09-13T18:40:46.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6dd9b700 1 --2- 172.21.15.146:0/1661777069 >> v2:172.21.15.154:3300/0 conn(0x7f4f6804ccb0 0x7f4f6804eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:46.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6e59c700 1 -- 172.21.15.146:0/1661777069 >> v2:172.21.15.154:3300/0 conn(0x7f4f6804ccb0 msgr2=0x7f4f6804eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:46.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6e59c700 1 --2- 172.21.15.146:0/1661777069 >> v2:172.21.15.154:3300/0 conn(0x7f4f6804ccb0 0x7f4f6804eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6e59c700 1 -- 172.21.15.146:0/1661777069 >> v1:172.21.15.146:6789/0 conn(0x7f4f6800d530 legacy=0x7f4f6800be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:46.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6e59c700 1 -- 172.21.15.146:0/1661777069 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f680473d0 con 0x7f4f680d69b0 2024-09-13T18:40:46.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6e59c700 1 --2- 172.21.15.146:0/1661777069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f680d69b0 0x7f4f6804e990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4f680da180 tx=0x7f4f60009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1024ba297616b5a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:46.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6d59a700 1 -- 172.21.15.146:0/1661777069 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6000b8a0 con 0x7f4f680d69b0 2024-09-13T18:40:46.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6d59a700 1 -- 172.21.15.146:0/1661777069 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6000ba00 con 0x7f4f680d69b0 2024-09-13T18:40:46.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.337+0000 7f4f6d59a700 1 -- 172.21.15.146:0/1661777069 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6000bd90 con 0x7f4f680d69b0 2024-09-13T18:40:46.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1661777069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f680d69b0 msgr2=0x7f4f6804e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:46.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 --2- 172.21.15.146:0/1661777069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f680d69b0 0x7f4f6804e990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4f680da180 tx=0x7f4f60009bf0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1661777069 shutdown_connections 2024-09-13T18:40:46.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 --2- 172.21.15.146:0/1661777069 >> v2:172.21.15.154:3300/0 conn(0x7f4f6804ccb0 0x7f4f6804eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 --2- 172.21.15.146:0/1661777069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f680d69b0 0x7f4f6804e990 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1661777069 >> 172.21.15.146:0/1661777069 conn(0x7f4f6800a130 msgr2=0x7f4f6800dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:46.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1661777069 shutdown_connections 2024-09-13T18:40:46.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1661777069 wait complete. 2024-09-13T18:40:46.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 Processor -- start 2024-09-13T18:40:46.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 -- start start 2024-09-13T18:40:46.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f6800d530 0x7f4f68171a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:46.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f6804ccb0 0x7f4f6804bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:46.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f680d69b0 0x7f4f680d9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:46.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f68171fd0 con 0x7f4f6800d530 2024-09-13T18:40:46.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f68172a50 con 0x7f4f6804ccb0 2024-09-13T18:40:46.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f681721b0 con 0x7f4f680d69b0 2024-09-13T18:40:46.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6e59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f6800d530 0x7f4f68171a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:46.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6e59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f6800d530 0x7f4f68171a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45530/0 (socket says 172.21.15.146:45530) 2024-09-13T18:40:46.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6e59c700 1 -- 172.21.15.146:0/1031728071 learned_addr learned my addr 172.21.15.146:0/1031728071 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:46.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6ed9d700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f680d69b0 0x7f4f680d9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:46.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6dd9b700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f6804ccb0 0x7f4f6804bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:46.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6e59c700 1 -- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f680d69b0 msgr2=0x7f4f680d9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:46.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6e59c700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f680d69b0 0x7f4f680d9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6e59c700 1 -- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f6804ccb0 msgr2=0x7f4f6804bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:46.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6e59c700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f6804ccb0 0x7f4f6804bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6e59c700 1 -- 172.21.15.146:0/1031728071 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f60009570 con 0x7f4f6800d530 2024-09-13T18:40:46.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6ed9d700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f680d69b0 0x7f4f680d9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:40:46.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f6e59c700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f6800d530 0x7f4f68171a90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4f60000c00 tx=0x7f4f60004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:46.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.341+0000 7f4f5effd700 1 -- 172.21.15.146:0/1031728071 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60003ad0 con 0x7f4f6800d530 2024-09-13T18:40:46.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.345+0000 7f4f5effd700 1 -- 172.21.15.146:0/1031728071 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6001e070 con 0x7f4f6800d530 2024-09-13T18:40:46.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.345+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1031728071 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f68172490 con 0x7f4f6800d530 2024-09-13T18:40:46.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.345+0000 7f4f5effd700 1 -- 172.21.15.146:0/1031728071 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60003e90 con 0x7f4f6800d530 2024-09-13T18:40:46.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.345+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1031728071 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6804b7a0 con 0x7f4f6800d530 2024-09-13T18:40:46.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.345+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1031728071 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6804b600 con 0x7f4f6800d530 2024-09-13T18:40:46.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.345+0000 7f4f5effd700 1 -- 172.21.15.146:0/1031728071 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4f6002f430 con 0x7f4f6800d530 2024-09-13T18:40:46.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.345+0000 7f4f5effd700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4f50066760 0x7f4f50068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:46.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.345+0000 7f4f5effd700 1 -- 172.21.15.146:0/1031728071 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f4f60020070 con 0x7f4f6800d530 2024-09-13T18:40:46.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.345+0000 7f4f6dd9b700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4f50066760 0x7f4f50068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:46.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.349+0000 7f4f6dd9b700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4f50066760 0x7f4f50068c20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4f680370b0 tx=0x7f4f64008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:46.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.349+0000 7f4f5effd700 1 -- 172.21.15.146:0/1031728071 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4f60052cb0 con 0x7f4f6800d530 2024-09-13T18:40:46.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.557+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1031728071 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f6804b600 con 0x7f4f6800d530 2024-09-13T18:40:46.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.557+0000 7f4f5effd700 1 -- 172.21.15.146:0/1031728071 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5529 (secure 0 0 0) 0x7f4f6804b600 con 0x7f4f6800d530 2024-09-13T18:40:46.574 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:46.574 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:40:45.737261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:46.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.561+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4f50066760 msgr2=0x7f4f50068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:46.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.561+0000 7f4f6f59e700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4f50066760 0x7f4f50068c20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4f680370b0 tx=0x7f4f64008040 comp rx=0 tx=0).stop 2024-09-13T18:40:46.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.561+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1031728071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f6800d530 msgr2=0x7f4f68171a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:46.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.561+0000 7f4f6f59e700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f6800d530 0x7f4f68171a90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4f60000c00 tx=0x7f4f60004770 comp rx=0 tx=0).stop 2024-09-13T18:40:46.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.561+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1031728071 shutdown_connections 2024-09-13T18:40:46.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.561+0000 7f4f6f59e700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4f50066760 0x7f4f50068c20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.561+0000 7f4f6f59e700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f680d69b0 0x7f4f680d9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.561+0000 7f4f6f59e700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f6804ccb0 0x7f4f6804bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.561+0000 7f4f6f59e700 1 --2- 172.21.15.146:0/1031728071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f6800d530 0x7f4f68171a90 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.561+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1031728071 >> 172.21.15.146:0/1031728071 conn(0x7f4f6800a130 msgr2=0x7f4f6800dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:46.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.561+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1031728071 shutdown_connections 2024-09-13T18:40:46.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.561+0000 7f4f6f59e700 1 -- 172.21.15.146:0/1031728071 wait complete. 2024-09-13T18:40:46.580 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 58 2024-09-13T18:40:46.594 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-09-13T18:40:46.594 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:40:46.594 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:46.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.685+0000 7f048d83b700 1 Processor -- start 2024-09-13T18:40:46.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.685+0000 7f048d83b700 1 -- start start 2024-09-13T18:40:46.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.685+0000 7f048d83b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f048815a190 0x7f04880c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:46.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.685+0000 7f048d83b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f04880c3f60 0x7f04880706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:46.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.685+0000 7f048d83b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f048806c950 0x7f048806cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:46.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.685+0000 7f048d83b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04880c4380 con 0x7f048806c950 2024-09-13T18:40:46.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.685+0000 7f048d83b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048815d700 con 0x7f04880c3f60 2024-09-13T18:40:46.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.685+0000 7f048d83b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048806b3c0 con 0x7f048815a190 2024-09-13T18:40:46.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f04877fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f048806c950 0x7f048806cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:46.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f04877fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f048806c950 0x7f048806cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45532/0 (socket says 172.21.15.146:45532) 2024-09-13T18:40:46.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f04877fe700 1 -- 172.21.15.146:0/2434818071 learned_addr learned my addr 172.21.15.146:0/2434818071 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:46.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f04867fc700 1 --2- 172.21.15.146:0/2434818071 >> v2:172.21.15.154:3300/0 conn(0x7f04880c3f60 0x7f04880706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:46.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f04877fe700 1 -- 172.21.15.146:0/2434818071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f048815a190 msgr2=0x7f04880c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:46.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f04877fe700 1 --2- 172.21.15.146:0/2434818071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f048815a190 0x7f04880c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f04877fe700 1 -- 172.21.15.146:0/2434818071 >> v2:172.21.15.154:3300/0 conn(0x7f04880c3f60 msgr2=0x7f04880706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:46.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f04877fe700 1 --2- 172.21.15.146:0/2434818071 >> v2:172.21.15.154:3300/0 conn(0x7f04880c3f60 0x7f04880706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f04877fe700 1 -- 172.21.15.146:0/2434818071 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0488072390 con 0x7f048806c950 2024-09-13T18:40:46.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f04877fe700 1 --2- 172.21.15.146:0/2434818071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f048806c950 0x7f048806cdd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f04880c7780 tx=0x7f048000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197bc5a0b3e8df55 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:46.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f0485ffb700 1 -- 172.21.15.146:0/2434818071 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048000e070 con 0x7f048806c950 2024-09-13T18:40:46.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f0485ffb700 1 -- 172.21.15.146:0/2434818071 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048000b910 con 0x7f048806c950 2024-09-13T18:40:46.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f0485ffb700 1 -- 172.21.15.146:0/2434818071 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048000bcc0 con 0x7f048806c950 2024-09-13T18:40:46.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f048d83b700 1 -- 172.21.15.146:0/2434818071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f048806c950 msgr2=0x7f048806cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:46.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f048d83b700 1 --2- 172.21.15.146:0/2434818071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f048806c950 0x7f048806cdd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f04880c7780 tx=0x7f048000b330 comp rx=0 tx=0).stop 2024-09-13T18:40:46.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f048d83b700 1 -- 172.21.15.146:0/2434818071 shutdown_connections 2024-09-13T18:40:46.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f048d83b700 1 --2- 172.21.15.146:0/2434818071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f048806c950 0x7f048806cdd0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f048d83b700 1 --2- 172.21.15.146:0/2434818071 >> v2:172.21.15.154:3300/0 conn(0x7f04880c3f60 0x7f04880706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f048d83b700 1 --2- 172.21.15.146:0/2434818071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f048815a190 0x7f04880c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f048d83b700 1 -- 172.21.15.146:0/2434818071 >> 172.21.15.146:0/2434818071 conn(0x7f048806f7a0 msgr2=0x7f048806fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:46.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f048d83b700 1 -- 172.21.15.146:0/2434818071 shutdown_connections 2024-09-13T18:40:46.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.689+0000 7f048d83b700 1 -- 172.21.15.146:0/2434818071 wait complete. 2024-09-13T18:40:46.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f048d83b700 1 Processor -- start 2024-09-13T18:40:46.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f048d83b700 1 -- start start 2024-09-13T18:40:46.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f048d83b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f048806c950 0x7f048815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:46.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f048d83b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04880c3f60 0x7f04881ef890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:46.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f048d83b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f048815a190 0x7f04880c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:46.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f048d83b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04881533c0 con 0x7f048806c950 2024-09-13T18:40:46.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f048d83b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04881ef400 con 0x7f04880c3f60 2024-09-13T18:40:46.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f048d83b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04880c58d0 con 0x7f048815a190 2024-09-13T18:40:46.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f0486ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f048806c950 0x7f048815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:46.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f0486ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f048806c950 0x7f048815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45538/0 (socket says 172.21.15.146:45538) 2024-09-13T18:40:46.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f0486ffd700 1 -- 172.21.15.146:0/3201065668 learned_addr learned my addr 172.21.15.146:0/3201065668 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:46.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f04877fe700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f048815a190 0x7f04880c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:46.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f04867fc700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04880c3f60 0x7f04881ef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:46.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f0486ffd700 1 -- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f048815a190 msgr2=0x7f04880c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:46.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f0486ffd700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f048815a190 0x7f04880c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f0486ffd700 1 -- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04880c3f60 msgr2=0x7f04881ef890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:46.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f0486ffd700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04880c3f60 0x7f04881ef890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f0486ffd700 1 -- 172.21.15.146:0/3201065668 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0480009b30 con 0x7f048806c950 2024-09-13T18:40:46.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f0486ffd700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f048806c950 0x7f048815bcd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f047800b550 tx=0x7f047800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:46.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f046ffff700 1 -- 172.21.15.146:0/3201065668 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478011830 con 0x7f048806c950 2024-09-13T18:40:46.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f046ffff700 1 -- 172.21.15.146:0/3201065668 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0478011990 con 0x7f048806c950 2024-09-13T18:40:46.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f048d83b700 1 -- 172.21.15.146:0/3201065668 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04880c6440 con 0x7f048806c950 2024-09-13T18:40:46.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f046ffff700 1 -- 172.21.15.146:0/3201065668 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478011d40 con 0x7f048806c950 2024-09-13T18:40:46.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f048d83b700 1 -- 172.21.15.146:0/3201065668 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04880c5e80 con 0x7f048806c950 2024-09-13T18:40:46.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.693+0000 7f048d83b700 1 -- 172.21.15.146:0/3201065668 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04880c86d0 con 0x7f048806c950 2024-09-13T18:40:46.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.697+0000 7f046ffff700 1 -- 172.21.15.146:0/3201065668 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f047801a400 con 0x7f048806c950 2024-09-13T18:40:46.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.697+0000 7f046ffff700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0474066730 0x7f0474068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:46.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.697+0000 7f046ffff700 1 -- 172.21.15.146:0/3201065668 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f0478084640 con 0x7f048806c950 2024-09-13T18:40:46.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.697+0000 7f04867fc700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0474066730 0x7f0474068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:46.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.697+0000 7f04867fc700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0474066730 0x7f0474068bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f04880c6a90 tx=0x7f047c0091c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:46.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.705+0000 7f046ffff700 1 -- 172.21.15.146:0/3201065668 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f04780513d0 con 0x7f048806c950 2024-09-13T18:40:46.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.933+0000 7f048d83b700 1 -- 172.21.15.146:0/3201065668 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f048815db70 con 0x7f048806c950 2024-09-13T18:40:46.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.933+0000 7f046ffff700 1 -- 172.21.15.146:0/3201065668 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5529 (secure 0 0 0) 0x7f048815db70 con 0x7f048806c950 2024-09-13T18:40:46.952 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:46.952 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:40:45.737261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:46.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.937+0000 7f048d83b700 1 -- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0474066730 msgr2=0x7f0474068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:46.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.937+0000 7f048d83b700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0474066730 0x7f0474068bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f04880c6a90 tx=0x7f047c0091c0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.937+0000 7f048d83b700 1 -- 172.21.15.146:0/3201065668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f048806c950 msgr2=0x7f048815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:46.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.937+0000 7f048d83b700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f048806c950 0x7f048815bcd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f047800b550 tx=0x7f047800b860 comp rx=0 tx=0).stop 2024-09-13T18:40:46.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.937+0000 7f048d83b700 1 -- 172.21.15.146:0/3201065668 shutdown_connections 2024-09-13T18:40:46.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.937+0000 7f048d83b700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0474066730 0x7f0474068bf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.937+0000 7f048d83b700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f048815a190 0x7f04880c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.937+0000 7f048d83b700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04880c3f60 0x7f04881ef890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.937+0000 7f048d83b700 1 --2- 172.21.15.146:0/3201065668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f048806c950 0x7f048815bcd0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:46.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.937+0000 7f048d83b700 1 -- 172.21.15.146:0/3201065668 >> 172.21.15.146:0/3201065668 conn(0x7f048806f7a0 msgr2=0x7f0488157e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:46.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.937+0000 7f048d83b700 1 -- 172.21.15.146:0/3201065668 shutdown_connections 2024-09-13T18:40:46.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:46.937+0000 7f048d83b700 1 -- 172.21.15.146:0/3201065668 wait complete. 2024-09-13T18:40:46.957 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 58 2024-09-13T18:40:46.973 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 58, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 58, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5784": { "addr": "172.21.15.146:6833/3901685195", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3901685195, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3901685195, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5784, "incarnation": 46, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 29 }, "gid_5800": { "addr": "172.21.15.154:6839/776638146", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 776638146, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 776638146, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5800, "incarnation": 52, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 30 }, "gid_6063": { "addr": "172.21.15.146:6835/832794630", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 832794630, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 832794630, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6063, "incarnation": 58, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 35, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:40:45.737261+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5800, "mds_1": 5784, "mds_2": 6063 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:40:48.974 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:49.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.077+0000 7ff0b083d700 1 Processor -- start 2024-09-13T18:40:49.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.077+0000 7ff0b083d700 1 -- start start 2024-09-13T18:40:49.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0b083d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac15a000 0x7ff0ac0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:49.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0b083d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ac0c1020 con 0x7ff0ac06eba0 2024-09-13T18:40:49.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0b083d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ac1593f0 con 0x7ff0ac06e7c0 2024-09-13T18:40:49.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0b083d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac0c6a70 con 0x7ff0ac15a000 2024-09-13T18:40:49.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aaffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff0ac06eba0 0x7ff0ac1531a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56902/0 (socket says 172.21.15.146:56902) 2024-09-13T18:40:49.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aaffd700 1 -- 172.21.15.146:0/1351259877 learned_addr learned my addr 172.21.15.146:0/1351259877 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:40:49.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0abfff700 1 --2- 172.21.15.146:0/1351259877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac15a000 0x7ff0ac0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:49.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11079034 0 0) 0x7ff0ac0c1020 con 0x7ff0ac06eba0 2024-09-13T18:40:49.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0940035d0 con 0x7ff0ac06eba0 2024-09-13T18:40:49.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3987408013 0 0) 0x7ff0940035d0 con 0x7ff0ac06eba0 2024-09-13T18:40:49.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac15a000 msgr2=0x7ff0ac0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:49.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 --2- 172.21.15.146:0/1351259877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac15a000 0x7ff0ac0b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:49.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 >> v1:172.21.15.154:6789/0 conn(0x7ff0ac06e7c0 legacy=0x7ff0ac1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:49.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac0c1290 con 0x7ff0ac06eba0 2024-09-13T18:40:49.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7ff0a0001670 con 0x7ff0ac06eba0 2024-09-13T18:40:49.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 >> v1:172.21.15.146:6789/0 conn(0x7ff0ac06eba0 legacy=0x7ff0ac1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:49.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 --2- 172.21.15.146:0/1351259877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff094003ac0 0x7ff094003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:49.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 --2- 172.21.15.146:0/1351259877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac15a000 0x7ff094004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 --2- 172.21.15.146:0/1351259877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff094004db0 0x7ff094005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac0c6a70 con 0x7ff094003ac0 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09400bae0 con 0x7ff094004db0 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09400bc50 con 0x7ff0ac15a000 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0ab7fe700 1 --2- 172.21.15.146:0/1351259877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff094003ac0 0x7ff094003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0abfff700 1 --2- 172.21.15.146:0/1351259877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff094004db0 0x7ff094005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aaffd700 1 --2- 172.21.15.146:0/1351259877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac15a000 0x7ff094004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0ab7fe700 1 -- 172.21.15.146:0/1351259877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac15a000 msgr2=0x7ff094004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0ab7fe700 1 --2- 172.21.15.146:0/1351259877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac15a000 0x7ff094004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0ab7fe700 1 -- 172.21.15.146:0/1351259877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff094004db0 msgr2=0x7ff094005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0ab7fe700 1 --2- 172.21.15.146:0/1351259877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff094004db0 0x7ff094005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0ab7fe700 1 -- 172.21.15.146:0/1351259877 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0940035d0 con 0x7ff094003ac0 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0abfff700 1 --2- 172.21.15.146:0/1351259877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff094004db0 0x7ff094005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:40:49.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0ab7fe700 1 --2- 172.21.15.146:0/1351259877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff094003ac0 0x7ff094003ea0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0018f0 tx=0x7ff0940058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610edc56a90da04e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:49.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c001390 con 0x7ff094003ac0 2024-09-13T18:40:49.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c00c690 con 0x7ff094003ac0 2024-09-13T18:40:49.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0aa7fc700 1 -- 172.21.15.146:0/1351259877 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00ca40 con 0x7ff094003ac0 2024-09-13T18:40:49.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0b083d700 1 -- 172.21.15.146:0/1351259877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff094003ac0 msgr2=0x7ff094003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:49.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0b083d700 1 --2- 172.21.15.146:0/1351259877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff094003ac0 0x7ff094003ea0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0018f0 tx=0x7ff0940058a0 comp rx=0 tx=0).stop 2024-09-13T18:40:49.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0abfff700 1 -- 172.21.15.146:0/1351259877 reap_dead start 2024-09-13T18:40:49.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0b083d700 1 -- 172.21.15.146:0/1351259877 shutdown_connections 2024-09-13T18:40:49.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0b083d700 1 -- 172.21.15.146:0/1351259877 >> 172.21.15.146:0/1351259877 conn(0x7ff0ac071ed0 msgr2=0x7ff0ac0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:49.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0b083d700 1 -- 172.21.15.146:0/1351259877 shutdown_connections 2024-09-13T18:40:49.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.081+0000 7ff0b083d700 1 -- 172.21.15.146:0/1351259877 wait complete. 2024-09-13T18:40:49.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0b083d700 1 Processor -- start 2024-09-13T18:40:49.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0b083d700 1 -- start start 2024-09-13T18:40:49.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0b083d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0ac1f2340 0x7ff0ac158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:49.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0b083d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1f2720 0x7ff0ac0c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:49.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0b083d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0ac14d6b0 0x7ff0ac14db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:49.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0b083d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac1f2fc0 con 0x7ff0ac1f2340 2024-09-13T18:40:49.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0b083d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac14e070 con 0x7ff0ac14d6b0 2024-09-13T18:40:49.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0b083d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac1f2ce0 con 0x7ff0ac1f2720 2024-09-13T18:40:49.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0ab7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0ac1f2340 0x7ff0ac158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:49.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0ab7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0ac1f2340 0x7ff0ac158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45550/0 (socket says 172.21.15.146:45550) 2024-09-13T18:40:49.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0ab7fe700 1 -- 172.21.15.146:0/373131193 learned_addr learned my addr 172.21.15.146:0/373131193 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:49.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0aaffd700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1f2720 0x7ff0ac0c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:49.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0abfff700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0ac14d6b0 0x7ff0ac14db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:49.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0ab7fe700 1 -- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1f2720 msgr2=0x7ff0ac0c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:49.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0ab7fe700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1f2720 0x7ff0ac0c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:49.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0ab7fe700 1 -- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0ac14d6b0 msgr2=0x7ff0ac14db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:49.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0ab7fe700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0ac14d6b0 0x7ff0ac14db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:49.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0ab7fe700 1 -- 172.21.15.146:0/373131193 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac0c7aa0 con 0x7ff0ac1f2340 2024-09-13T18:40:49.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0abfff700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0ac14d6b0 0x7ff0ac14db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:40:49.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.085+0000 7ff0aaffd700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1f2720 0x7ff0ac0c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:40:49.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.089+0000 7ff0ab7fe700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0ac1f2340 0x7ff0ac158eb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff09c013000 tx=0x7ff09c001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:49.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.089+0000 7ff093fff700 1 -- 172.21.15.146:0/373131193 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00ca10 con 0x7ff0ac1f2340 2024-09-13T18:40:49.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.089+0000 7ff093fff700 1 -- 172.21.15.146:0/373131193 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c020070 con 0x7ff0ac1f2340 2024-09-13T18:40:49.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.089+0000 7ff0b083d700 1 -- 172.21.15.146:0/373131193 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ac0c7d80 con 0x7ff0ac1f2340 2024-09-13T18:40:49.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.089+0000 7ff093fff700 1 -- 172.21.15.146:0/373131193 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00cd70 con 0x7ff0ac1f2340 2024-09-13T18:40:49.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.089+0000 7ff0b083d700 1 -- 172.21.15.146:0/373131193 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ac0c8340 con 0x7ff0ac1f2340 2024-09-13T18:40:49.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.089+0000 7ff0b083d700 1 -- 172.21.15.146:0/373131193 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ac0c8500 con 0x7ff0ac1f2340 2024-09-13T18:40:49.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.089+0000 7ff093fff700 1 -- 172.21.15.146:0/373131193 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff09c0036a0 con 0x7ff0ac1f2340 2024-09-13T18:40:49.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.089+0000 7ff093fff700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff098066730 0x7ff098068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:49.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.089+0000 7ff093fff700 1 -- 172.21.15.146:0/373131193 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7ff09c022070 con 0x7ff0ac1f2340 2024-09-13T18:40:49.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.089+0000 7ff0aaffd700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff098066730 0x7ff098068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:49.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.093+0000 7ff0aaffd700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff098066730 0x7ff098068bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0001eb0 tx=0x7ff0a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:49.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.097+0000 7ff093fff700 1 -- 172.21.15.146:0/373131193 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff09c0527b0 con 0x7ff0ac1f2340 2024-09-13T18:40:49.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.341+0000 7ff0b083d700 1 -- 172.21.15.146:0/373131193 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0ac0b11a0 con 0x7ff0ac1f2340 2024-09-13T18:40:49.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.341+0000 7ff093fff700 1 -- 172.21.15.146:0/373131193 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5529 (secure 0 0 0) 0x7ff09c055dd0 con 0x7ff0ac1f2340 2024-09-13T18:40:49.360 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:49.360 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:40:45.737261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:49.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.345+0000 7ff0b083d700 1 -- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff098066730 msgr2=0x7ff098068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:49.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.345+0000 7ff0b083d700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff098066730 0x7ff098068bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0001eb0 tx=0x7ff0a0009040 comp rx=0 tx=0).stop 2024-09-13T18:40:49.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.345+0000 7ff0b083d700 1 -- 172.21.15.146:0/373131193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0ac1f2340 msgr2=0x7ff0ac158eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:49.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.345+0000 7ff0b083d700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0ac1f2340 0x7ff0ac158eb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff09c013000 tx=0x7ff09c001230 comp rx=0 tx=0).stop 2024-09-13T18:40:49.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.345+0000 7ff0b083d700 1 -- 172.21.15.146:0/373131193 shutdown_connections 2024-09-13T18:40:49.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.345+0000 7ff0b083d700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff098066730 0x7ff098068bf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:49.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.345+0000 7ff0b083d700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0ac14d6b0 0x7ff0ac14db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:49.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.345+0000 7ff0b083d700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1f2720 0x7ff0ac0c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:49.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.345+0000 7ff0b083d700 1 --2- 172.21.15.146:0/373131193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0ac1f2340 0x7ff0ac158eb0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:49.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.345+0000 7ff0b083d700 1 -- 172.21.15.146:0/373131193 >> 172.21.15.146:0/373131193 conn(0x7ff0ac071ed0 msgr2=0x7ff0ac0c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:49.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.345+0000 7ff0b083d700 1 -- 172.21.15.146:0/373131193 shutdown_connections 2024-09-13T18:40:49.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:49.345+0000 7ff0b083d700 1 -- 172.21.15.146:0/373131193 wait complete. 2024-09-13T18:40:49.364 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 58 2024-09-13T18:40:49.375 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:40:51.376 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f716a55b700 1 Processor -- start 2024-09-13T18:40:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f716a55b700 1 -- start start 2024-09-13T18:40:51.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f716a55b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f71640c63a0 0x7f7164159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:51.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f716a55b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71640c7310 0x7f7164159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:51.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f716a55b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71640c8c70 con 0x7f71640c5fc0 2024-09-13T18:40:51.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f716a55b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71640c9bd0 con 0x7f71640c63a0 2024-09-13T18:40:51.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f716a55b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716415a0e0 con 0x7f71640c7310 2024-09-13T18:40:51.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f7168d58700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f71640c63a0 0x7f7164159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:51.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f7168d58700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f71640c63a0 0x7f7164159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54892/0 (socket says 172.21.15.146:54892) 2024-09-13T18:40:51.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f7169559700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f71640c5fc0 0x7f71640c6c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:44612/0 (socket says 172.21.15.146:44612) 2024-09-13T18:40:51.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f7168d58700 1 -- 172.21.15.146:0/1696672346 learned_addr learned my addr 172.21.15.146:0/1696672346 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:51.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f7169d5a700 1 --2- 172.21.15.146:0/1696672346 >> v2:172.21.15.154:3300/0 conn(0x7f71640c7310 0x7f7164159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:51.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f7168d58700 1 -- 172.21.15.146:0/1696672346 >> v2:172.21.15.154:3300/0 conn(0x7f71640c7310 msgr2=0x7f7164159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:51.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f7168d58700 1 --2- 172.21.15.146:0/1696672346 >> v2:172.21.15.154:3300/0 conn(0x7f71640c7310 0x7f7164159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:51.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f7168d58700 1 -- 172.21.15.146:0/1696672346 >> v1:172.21.15.146:6789/0 conn(0x7f71640c5fc0 legacy=0x7f71640c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:51.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f715bfff700 1 -- 172.21.15.146:0/1696672346 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164524622 0 0) 0x7f71640c8c70 con 0x7f71640c5fc0 2024-09-13T18:40:51.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f7168d58700 1 -- 172.21.15.146:0/1696672346 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71640c8f50 con 0x7f71640c63a0 2024-09-13T18:40:51.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.489+0000 7f7168d58700 1 --2- 172.21.15.146:0/1696672346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71640c63a0 0x7f7164159660 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f71640c82b0 tx=0x7f715400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b44f6246be49425 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:51.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f715bfff700 1 -- 172.21.15.146:0/1696672346 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400dd90 con 0x7f71640c63a0 2024-09-13T18:40:51.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f715bfff700 1 -- 172.21.15.146:0/1696672346 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7154010470 con 0x7f71640c63a0 2024-09-13T18:40:51.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f715bfff700 1 -- 172.21.15.146:0/1696672346 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154010800 con 0x7f71640c63a0 2024-09-13T18:40:51.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 -- 172.21.15.146:0/1696672346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71640c63a0 msgr2=0x7f7164159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:51.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 --2- 172.21.15.146:0/1696672346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71640c63a0 0x7f7164159660 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f71640c82b0 tx=0x7f715400be90 comp rx=0 tx=0).stop 2024-09-13T18:40:51.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 -- 172.21.15.146:0/1696672346 shutdown_connections 2024-09-13T18:40:51.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 --2- 172.21.15.146:0/1696672346 >> v2:172.21.15.154:3300/0 conn(0x7f71640c7310 0x7f7164159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:51.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 --2- 172.21.15.146:0/1696672346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71640c63a0 0x7f7164159660 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:51.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 -- 172.21.15.146:0/1696672346 >> 172.21.15.146:0/1696672346 conn(0x7f7164066090 msgr2=0x7f7164066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:51.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 -- 172.21.15.146:0/1696672346 shutdown_connections 2024-09-13T18:40:51.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 -- 172.21.15.146:0/1696672346 wait complete. 2024-09-13T18:40:51.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 Processor -- start 2024-09-13T18:40:51.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 -- start start 2024-09-13T18:40:51.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71640c5fc0 0x7f716406cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:51.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71640c63a0 0x7f71640adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:51.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71640c7310 0x7f7164152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:51.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71641eb740 con 0x7f71640c7310 2024-09-13T18:40:51.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71640ad800 con 0x7f71640c63a0 2024-09-13T18:40:51.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.493+0000 7f716a55b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71641ebbd0 con 0x7f71640c5fc0 2024-09-13T18:40:51.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7169d5a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71640c7310 0x7f7164152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:51.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7169d5a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71640c7310 0x7f7164152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54894/0 (socket says 172.21.15.146:54894) 2024-09-13T18:40:51.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7169d5a700 1 -- 172.21.15.146:0/2892820147 learned_addr learned my addr 172.21.15.146:0/2892820147 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:51.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7168d58700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71640c63a0 0x7f71640adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:51.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7169559700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71640c5fc0 0x7f716406cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:51.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7169d5a700 1 -- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71640c5fc0 msgr2=0x7f716406cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:51.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7169d5a700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71640c5fc0 0x7f716406cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:51.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7169d5a700 1 -- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71640c63a0 msgr2=0x7f71640adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:51.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7169d5a700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71640c63a0 0x7f71640adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:51.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7169d5a700 1 -- 172.21.15.146:0/2892820147 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715400b810 con 0x7f71640c7310 2024-09-13T18:40:51.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7169559700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71640c5fc0 0x7f716406cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:40:51.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7169d5a700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71640c7310 0x7f7164152510 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f715c007e60 tx=0x7f715c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:51.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7159ffb700 1 -- 172.21.15.146:0/2892820147 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c00ede0 con 0x7f71640c7310 2024-09-13T18:40:51.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7159ffb700 1 -- 172.21.15.146:0/2892820147 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c014420 con 0x7f71640c7310 2024-09-13T18:40:51.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f716a55b700 1 -- 172.21.15.146:0/2892820147 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71640ad520 con 0x7f71640c7310 2024-09-13T18:40:51.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7159ffb700 1 -- 172.21.15.146:0/2892820147 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c0147d0 con 0x7f71640c7310 2024-09-13T18:40:51.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f716a55b700 1 -- 172.21.15.146:0/2892820147 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71641eb0f0 con 0x7f71640c7310 2024-09-13T18:40:51.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f716a55b700 1 -- 172.21.15.146:0/2892820147 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7144005310 con 0x7f71640c7310 2024-09-13T18:40:51.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.497+0000 7f7159ffb700 1 -- 172.21.15.146:0/2892820147 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f715c014930 con 0x7f71640c7310 2024-09-13T18:40:51.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.501+0000 7f7159ffb700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7148066760 0x7f7148068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:51.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.501+0000 7f7159ffb700 1 -- 172.21.15.146:0/2892820147 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f715c084e00 con 0x7f71640c7310 2024-09-13T18:40:51.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.501+0000 7f7169559700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7148066760 0x7f7148068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:51.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.501+0000 7f7169559700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7148066760 0x7f7148068c20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7160000f60 tx=0x7f7160007cc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:51.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.505+0000 7f7159ffb700 1 -- 172.21.15.146:0/2892820147 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f715c051b00 con 0x7f71640c7310 2024-09-13T18:40:51.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.725+0000 7f716a55b700 1 -- 172.21.15.146:0/2892820147 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71440055c0 con 0x7f71640c7310 2024-09-13T18:40:51.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.725+0000 7f7159ffb700 1 -- 172.21.15.146:0/2892820147 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5531 (secure 0 0 0) 0x7f715c009960 con 0x7f71640c7310 2024-09-13T18:40:51.744 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:51.744 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:40:51.549958+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:resolve","state_seq":37,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:51.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.729+0000 7f716a55b700 1 -- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7148066760 msgr2=0x7f7148068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:51.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.729+0000 7f716a55b700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7148066760 0x7f7148068c20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7160000f60 tx=0x7f7160007cc0 comp rx=0 tx=0).stop 2024-09-13T18:40:51.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.729+0000 7f716a55b700 1 -- 172.21.15.146:0/2892820147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71640c7310 msgr2=0x7f7164152510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:51.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.729+0000 7f716a55b700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71640c7310 0x7f7164152510 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f715c007e60 tx=0x7f715c00e3c0 comp rx=0 tx=0).stop 2024-09-13T18:40:51.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.729+0000 7f716a55b700 1 -- 172.21.15.146:0/2892820147 shutdown_connections 2024-09-13T18:40:51.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.729+0000 7f716a55b700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7148066760 0x7f7148068c20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:51.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.729+0000 7f716a55b700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71640c7310 0x7f7164152510 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:51.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.729+0000 7f716a55b700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71640c63a0 0x7f71640adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:51.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.729+0000 7f716a55b700 1 --2- 172.21.15.146:0/2892820147 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71640c5fc0 0x7f716406cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:51.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.729+0000 7f716a55b700 1 -- 172.21.15.146:0/2892820147 >> 172.21.15.146:0/2892820147 conn(0x7f7164066090 msgr2=0x7f7164066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:51.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.729+0000 7f716a55b700 1 -- 172.21.15.146:0/2892820147 shutdown_connections 2024-09-13T18:40:51.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:51.729+0000 7f716a55b700 1 -- 172.21.15.146:0/2892820147 wait complete. 2024-09-13T18:40:51.748 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 59 2024-09-13T18:40:51.759 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:40:53.760 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:53.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea6f25700 1 Processor -- start 2024-09-13T18:40:53.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea6f25700 1 -- start start 2024-09-13T18:40:53.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea6f25700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8ea006eba0 0x7f8ea00b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:53.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea6f25700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ea015a000 0x7f8ea00c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:53.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea6f25700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea00c1020 con 0x7f8ea006e7c0 2024-09-13T18:40:53.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea6f25700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea01593f0 con 0x7f8ea006eba0 2024-09-13T18:40:53.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea6f25700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea00c6a70 con 0x7f8ea015a000 2024-09-13T18:40:53.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea5722700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8ea006eba0 0x7f8ea00b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:53.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea5722700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8ea006eba0 0x7f8ea00b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54896/0 (socket says 172.21.15.146:54896) 2024-09-13T18:40:53.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea5722700 1 -- 172.21.15.146:0/1424193405 learned_addr learned my addr 172.21.15.146:0/1424193405 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:53.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea6724700 1 --2- 172.21.15.146:0/1424193405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ea015a000 0x7f8ea00c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:53.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea5722700 1 -- 172.21.15.146:0/1424193405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ea015a000 msgr2=0x7f8ea00c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:53.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea5722700 1 --2- 172.21.15.146:0/1424193405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ea015a000 0x7f8ea00c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:53.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea5722700 1 -- 172.21.15.146:0/1424193405 >> v1:172.21.15.154:6789/0 conn(0x7f8ea006e7c0 legacy=0x7f8ea01598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:53.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.869+0000 7f8ea5722700 1 -- 172.21.15.146:0/1424193405 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea00c1290 con 0x7f8ea006eba0 2024-09-13T18:40:53.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea5722700 1 --2- 172.21.15.146:0/1424193405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ea006eba0 0x7f8ea00b11a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009910 tx=0x7f8e90009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7599749d88ac6c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:53.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea4f21700 1 -- 172.21.15.146:0/1424193405 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90011830 con 0x7f8ea006eba0 2024-09-13T18:40:53.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea4f21700 1 -- 172.21.15.146:0/1424193405 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e90011990 con 0x7f8ea006eba0 2024-09-13T18:40:53.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea4f21700 1 -- 172.21.15.146:0/1424193405 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90011d40 con 0x7f8ea006eba0 2024-09-13T18:40:53.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea6f25700 1 -- 172.21.15.146:0/1424193405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ea006eba0 msgr2=0x7f8ea00b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:53.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea6f25700 1 --2- 172.21.15.146:0/1424193405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ea006eba0 0x7f8ea00b11a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009910 tx=0x7f8e90009c20 comp rx=0 tx=0).stop 2024-09-13T18:40:53.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea6f25700 1 -- 172.21.15.146:0/1424193405 shutdown_connections 2024-09-13T18:40:53.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea6f25700 1 --2- 172.21.15.146:0/1424193405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ea015a000 0x7f8ea00c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:53.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea6f25700 1 --2- 172.21.15.146:0/1424193405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ea006eba0 0x7f8ea00b11a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:53.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea6f25700 1 -- 172.21.15.146:0/1424193405 >> 172.21.15.146:0/1424193405 conn(0x7f8ea0071ed0 msgr2=0x7f8ea00722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:53.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea6f25700 1 -- 172.21.15.146:0/1424193405 shutdown_connections 2024-09-13T18:40:53.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea6f25700 1 -- 172.21.15.146:0/1424193405 wait complete. 2024-09-13T18:40:53.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea6f25700 1 Processor -- start 2024-09-13T18:40:53.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.873+0000 7f8ea6f25700 1 -- start start 2024-09-13T18:40:53.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea6f25700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ea006e7c0 0x7f8ea01e4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:53.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea6f25700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ea006eba0 0x7f8ea01f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:53.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea6f25700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ea015a000 0x7f8ea00cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:53.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea6f25700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0151630 con 0x7f8ea006eba0 2024-09-13T18:40:53.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea6f25700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea014c670 con 0x7f8ea006e7c0 2024-09-13T18:40:53.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea6f25700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0152950 con 0x7f8ea015a000 2024-09-13T18:40:53.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea5722700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ea006eba0 0x7f8ea01f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:53.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea5722700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ea006eba0 0x7f8ea01f00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54904/0 (socket says 172.21.15.146:54904) 2024-09-13T18:40:53.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea5722700 1 -- 172.21.15.146:0/2976950237 learned_addr learned my addr 172.21.15.146:0/2976950237 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea5f23700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ea006e7c0 0x7f8ea01e4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea6724700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ea015a000 0x7f8ea00cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea5722700 1 -- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ea015a000 msgr2=0x7f8ea00cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea5722700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ea015a000 0x7f8ea00cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea5722700 1 -- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ea006e7c0 msgr2=0x7f8ea01e4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea5722700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ea006e7c0 0x7f8ea01e4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea5722700 1 -- 172.21.15.146:0/2976950237 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e90009570 con 0x7f8ea006eba0 2024-09-13T18:40:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea5722700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ea006eba0 0x7f8ea01f00e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009f70 tx=0x7f8e90004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8e967fc700 1 -- 172.21.15.146:0/2976950237 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90004510 con 0x7f8ea006eba0 2024-09-13T18:40:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8e967fc700 1 -- 172.21.15.146:0/2976950237 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9001e070 con 0x7f8ea006eba0 2024-09-13T18:40:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea6f25700 1 -- 172.21.15.146:0/2976950237 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea0152bf0 con 0x7f8ea006eba0 2024-09-13T18:40:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8e967fc700 1 -- 172.21.15.146:0/2976950237 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90003900 con 0x7f8ea006eba0 2024-09-13T18:40:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea6f25700 1 -- 172.21.15.146:0/2976950237 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea01e51f0 con 0x7f8ea006eba0 2024-09-13T18:40:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.877+0000 7f8ea6f25700 1 -- 172.21.15.146:0/2976950237 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea00b11a0 con 0x7f8ea006eba0 2024-09-13T18:40:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.881+0000 7f8e967fc700 1 -- 172.21.15.146:0/2976950237 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8e90003a60 con 0x7f8ea006eba0 2024-09-13T18:40:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.881+0000 7f8e967fc700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e84066730 0x7f8e84068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:53.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.881+0000 7f8e967fc700 1 -- 172.21.15.146:0/2976950237 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f8e90085580 con 0x7f8ea006eba0 2024-09-13T18:40:53.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.881+0000 7f8ea5f23700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e84066730 0x7f8e84068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:53.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.881+0000 7f8ea5f23700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e84066730 0x7f8e84068bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c000f60 tx=0x7f8e9c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:53.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:53.885+0000 7f8e967fc700 1 -- 172.21.15.146:0/2976950237 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8e90052310 con 0x7f8ea006eba0 2024-09-13T18:40:54.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8ea6f25700 1 -- 172.21.15.146:0/2976950237 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ea006e310 con 0x7f8ea006eba0 2024-09-13T18:40:54.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8e967fc700 1 -- 172.21.15.146:0/2976950237 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5531 (secure 0 0 0) 0x7f8e90055930 con 0x7f8ea006eba0 2024-09-13T18:40:54.145 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:54.145 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:40:51.549958+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:resolve","state_seq":37,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:54.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8ea6f25700 1 -- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e84066730 msgr2=0x7f8e84068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:54.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8ea6f25700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e84066730 0x7f8e84068bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c000f60 tx=0x7f8e9c009040 comp rx=0 tx=0).stop 2024-09-13T18:40:54.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8ea6f25700 1 -- 172.21.15.146:0/2976950237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ea006eba0 msgr2=0x7f8ea01f00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:54.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8ea6f25700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ea006eba0 0x7f8ea01f00e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009f70 tx=0x7f8e90004770 comp rx=0 tx=0).stop 2024-09-13T18:40:54.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8ea6f25700 1 -- 172.21.15.146:0/2976950237 shutdown_connections 2024-09-13T18:40:54.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8ea6f25700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e84066730 0x7f8e84068bf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:54.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8ea6f25700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ea015a000 0x7f8ea00cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:54.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8ea6f25700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ea006eba0 0x7f8ea01f00e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:54.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8ea6f25700 1 --2- 172.21.15.146:0/2976950237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ea006e7c0 0x7f8ea01e4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:54.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8ea6f25700 1 -- 172.21.15.146:0/2976950237 >> 172.21.15.146:0/2976950237 conn(0x7f8ea0071ed0 msgr2=0x7f8ea006f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:54.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8ea6f25700 1 -- 172.21.15.146:0/2976950237 shutdown_connections 2024-09-13T18:40:54.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:54.129+0000 7f8ea6f25700 1 -- 172.21.15.146:0/2976950237 wait complete. 2024-09-13T18:40:54.149 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 59 2024-09-13T18:40:54.161 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:40:56.161 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:56.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab30f8f700 1 Processor -- start 2024-09-13T18:40:56.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab30f8f700 1 -- start start 2024-09-13T18:40:56.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab30f8f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fab2c0c7970 0x7fab2c06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:56.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab30f8f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab2c06c5b0 0x7fab2c070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:56.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab30f8f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2c0c98d0 con 0x7fab2c153860 2024-09-13T18:40:56.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab30f8f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2c0c4200 con 0x7fab2c0c7970 2024-09-13T18:40:56.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab30f8f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c0706d0 con 0x7fab2c06c5b0 2024-09-13T18:40:56.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab2affd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fab2c0c7970 0x7fab2c06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:56.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab2b7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fab2c153860 0x7fab2c0c85e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:44620/0 (socket says 172.21.15.146:44620) 2024-09-13T18:40:56.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab2affd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fab2c0c7970 0x7fab2c06c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54918/0 (socket says 172.21.15.146:54918) 2024-09-13T18:40:56.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab2b7fe700 1 -- 172.21.15.146:0/887384697 learned_addr learned my addr 172.21.15.146:0/887384697 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:40:56.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab2affd700 1 -- 172.21.15.146:0/887384697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab2c06c5b0 msgr2=0x7fab2c070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:40:56.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab2bfff700 1 --2- 172.21.15.146:0/887384697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab2c06c5b0 0x7fab2c070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:56.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab2affd700 1 --2- 172.21.15.146:0/887384697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab2c06c5b0 0x7fab2c070bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:56.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab2affd700 1 -- 172.21.15.146:0/887384697 >> v1:172.21.15.146:6789/0 conn(0x7fab2c153860 legacy=0x7fab2c0c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:56.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab2a7fc700 1 -- 172.21.15.146:0/887384697 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155747943 0 0) 0x7fab1c001110 con 0x7fab2c153860 2024-09-13T18:40:56.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.265+0000 7fab2affd700 1 -- 172.21.15.146:0/887384697 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c159730 con 0x7fab2c0c7970 2024-09-13T18:40:56.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab2affd700 1 --2- 172.21.15.146:0/887384697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c0c7970 0x7fab2c06c070 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fab14009850 tx=0x7fab14009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7ddf6ed1bb2f92 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:56.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab2a7fc700 1 -- 172.21.15.146:0/887384697 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14011830 con 0x7fab2c0c7970 2024-09-13T18:40:56.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab2a7fc700 1 -- 172.21.15.146:0/887384697 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab14011990 con 0x7fab2c0c7970 2024-09-13T18:40:56.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab2a7fc700 1 -- 172.21.15.146:0/887384697 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14011d20 con 0x7fab2c0c7970 2024-09-13T18:40:56.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab30f8f700 1 -- 172.21.15.146:0/887384697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c0c7970 msgr2=0x7fab2c06c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:56.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab30f8f700 1 --2- 172.21.15.146:0/887384697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c0c7970 0x7fab2c06c070 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fab14009850 tx=0x7fab14009b60 comp rx=0 tx=0).stop 2024-09-13T18:40:56.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab30f8f700 1 -- 172.21.15.146:0/887384697 shutdown_connections 2024-09-13T18:40:56.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab30f8f700 1 --2- 172.21.15.146:0/887384697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab2c06c5b0 0x7fab2c070bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:56.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab30f8f700 1 --2- 172.21.15.146:0/887384697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c0c7970 0x7fab2c06c070 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:56.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab30f8f700 1 -- 172.21.15.146:0/887384697 >> 172.21.15.146:0/887384697 conn(0x7fab2c0ccf70 msgr2=0x7fab2c0cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:56.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab30f8f700 1 -- 172.21.15.146:0/887384697 shutdown_connections 2024-09-13T18:40:56.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab30f8f700 1 -- 172.21.15.146:0/887384697 wait complete. 2024-09-13T18:40:56.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab30f8f700 1 Processor -- start 2024-09-13T18:40:56.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.269+0000 7fab30f8f700 1 -- start start 2024-09-13T18:40:56.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab30f8f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c06c5b0 0x7fab2c14bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:56.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab30f8f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c0c7970 0x7fab2c15c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:56.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab30f8f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab2c153860 0x7fab2c1e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:56.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab30f8f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c14c9d0 con 0x7fab2c06c5b0 2024-09-13T18:40:56.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab30f8f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1ef4f0 con 0x7fab2c0c7970 2024-09-13T18:40:56.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab30f8f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1ef7d0 con 0x7fab2c153860 2024-09-13T18:40:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab2b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c06c5b0 0x7fab2c14bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab2b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c06c5b0 0x7fab2c14bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54932/0 (socket says 172.21.15.146:54932) 2024-09-13T18:40:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab2b7fe700 1 -- 172.21.15.146:0/742860610 learned_addr learned my addr 172.21.15.146:0/742860610 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab2affd700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c0c7970 0x7fab2c15c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab2bfff700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab2c153860 0x7fab2c1e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab2b7fe700 1 -- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab2c153860 msgr2=0x7fab2c1e3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab2b7fe700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab2c153860 0x7fab2c1e3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab2b7fe700 1 -- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c0c7970 msgr2=0x7fab2c15c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:56.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab2b7fe700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c0c7970 0x7fab2c15c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:56.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab2b7fe700 1 -- 172.21.15.146:0/742860610 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab140094b0 con 0x7fab2c06c5b0 2024-09-13T18:40:56.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab2affd700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c0c7970 0x7fab2c15c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:40:56.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab2b7fe700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c06c5b0 0x7fab2c14bc10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c4b0 tx=0x7fab1c00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:56.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab23fff700 1 -- 172.21.15.146:0/742860610 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c005d90 con 0x7fab2c06c5b0 2024-09-13T18:40:56.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab23fff700 1 -- 172.21.15.146:0/742860610 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c0070d0 con 0x7fab2c06c5b0 2024-09-13T18:40:56.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab30f8f700 1 -- 172.21.15.146:0/742860610 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c1efab0 con 0x7fab2c06c5b0 2024-09-13T18:40:56.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab23fff700 1 -- 172.21.15.146:0/742860610 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c007460 con 0x7fab2c06c5b0 2024-09-13T18:40:56.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.273+0000 7fab30f8f700 1 -- 172.21.15.146:0/742860610 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c1f27f0 con 0x7fab2c06c5b0 2024-09-13T18:40:56.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.277+0000 7fab30f8f700 1 -- 172.21.15.146:0/742860610 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2c067a60 con 0x7fab2c06c5b0 2024-09-13T18:40:56.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.277+0000 7fab23fff700 1 -- 172.21.15.146:0/742860610 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fab1c0075c0 con 0x7fab2c06c5b0 2024-09-13T18:40:56.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.277+0000 7fab23fff700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab0c066760 0x7fab0c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:56.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.277+0000 7fab23fff700 1 -- 172.21.15.146:0/742860610 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7fab1c083b50 con 0x7fab2c06c5b0 2024-09-13T18:40:56.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.277+0000 7fab2affd700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab0c066760 0x7fab0c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:56.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.277+0000 7fab2affd700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab0c066760 0x7fab0c068c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fab140090b0 tx=0x7fab14008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:56.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.285+0000 7fab23fff700 1 -- 172.21.15.146:0/742860610 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fab1c0507d0 con 0x7fab2c06c5b0 2024-09-13T18:40:56.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.529+0000 7fab30f8f700 1 -- 172.21.15.146:0/742860610 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab2c0c98d0 con 0x7fab2c06c5b0 2024-09-13T18:40:56.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.529+0000 7fab23fff700 1 -- 172.21.15.146:0/742860610 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5531 (secure 0 0 0) 0x7fab2c0c98d0 con 0x7fab2c06c5b0 2024-09-13T18:40:56.546 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:56.547 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:40:51.549958+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:resolve","state_seq":37,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:56.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.533+0000 7fab30f8f700 1 -- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab0c066760 msgr2=0x7fab0c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.533+0000 7fab30f8f700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab0c066760 0x7fab0c068c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fab140090b0 tx=0x7fab14008030 comp rx=0 tx=0).stop 2024-09-13T18:40:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.533+0000 7fab30f8f700 1 -- 172.21.15.146:0/742860610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c06c5b0 msgr2=0x7fab2c14bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.533+0000 7fab30f8f700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c06c5b0 0x7fab2c14bc10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c4b0 tx=0x7fab1c00c870 comp rx=0 tx=0).stop 2024-09-13T18:40:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.533+0000 7fab30f8f700 1 -- 172.21.15.146:0/742860610 shutdown_connections 2024-09-13T18:40:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.533+0000 7fab30f8f700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab0c066760 0x7fab0c068c20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.533+0000 7fab30f8f700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab2c153860 0x7fab2c1e3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.533+0000 7fab30f8f700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c0c7970 0x7fab2c15c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.533+0000 7fab30f8f700 1 --2- 172.21.15.146:0/742860610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c06c5b0 0x7fab2c14bc10 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.533+0000 7fab30f8f700 1 -- 172.21.15.146:0/742860610 >> 172.21.15.146:0/742860610 conn(0x7fab2c0ccf70 msgr2=0x7fab2c070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.533+0000 7fab30f8f700 1 -- 172.21.15.146:0/742860610 shutdown_connections 2024-09-13T18:40:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:56.533+0000 7fab30f8f700 1 -- 172.21.15.146:0/742860610 wait complete. 2024-09-13T18:40:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 59 2024-09-13T18:40:56.563 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:40:58.564 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:40:58.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17233e4700 1 Processor -- start 2024-09-13T18:40:58.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17233e4700 1 -- start start 2024-09-13T18:40:58.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17233e4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f171c0cf170 0x7f171c1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:58.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17233e4700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171c15a540 con 0x7f171c0cb5b0 2024-09-13T18:40:58.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17233e4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171c0c5220 con 0x7f171c0cf550 2024-09-13T18:40:58.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17233e4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c159ca0 con 0x7f171c0cf170 2024-09-13T18:40:58.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f1722be3700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f171c0cb5b0 0x7f171c1534c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:44626/0 (socket says 172.21.15.146:44626) 2024-09-13T18:40:58.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f1722be3700 1 -- 172.21.15.146:0/3323777079 learned_addr learned my addr 172.21.15.146:0/3323777079 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:40:58.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17223e2700 1 --2- 172.21.15.146:0/3323777079 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f171c0cf170 0x7f171c1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:58.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17213e0700 1 -- 172.21.15.146:0/3323777079 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584235006 0 0) 0x7f171c15a540 con 0x7f171c0cb5b0 2024-09-13T18:40:58.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17213e0700 1 -- 172.21.15.146:0/3323777079 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17080035d0 con 0x7f171c0cb5b0 2024-09-13T18:40:58.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17213e0700 1 -- 172.21.15.146:0/3323777079 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108160001 0 0) 0x7f171c0c5220 con 0x7f171c0cf550 2024-09-13T18:40:58.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17213e0700 1 -- 172.21.15.146:0/3323777079 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f171c15a540 con 0x7f171c0cf550 2024-09-13T18:40:58.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17223e2700 1 -- 172.21.15.146:0/3323777079 >> v1:172.21.15.154:6789/0 conn(0x7f171c0cf550 legacy=0x7f171c0c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:58.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17223e2700 1 -- 172.21.15.146:0/3323777079 >> v1:172.21.15.146:6789/0 conn(0x7f171c0cb5b0 legacy=0x7f171c1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:58.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17223e2700 1 -- 172.21.15.146:0/3323777079 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c159f80 con 0x7f171c0cf170 2024-09-13T18:40:58.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17223e2700 1 --2- 172.21.15.146:0/3323777079 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f171c0cf170 0x7f171c1595f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1714009dc0 tx=0x7f171400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a37678c9601079f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:58.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17213e0700 1 -- 172.21.15.146:0/3323777079 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171400e070 con 0x7f171c0cf170 2024-09-13T18:40:58.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17213e0700 1 -- 172.21.15.146:0/3323777079 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171400bb20 con 0x7f171c0cf170 2024-09-13T18:40:58.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.673+0000 7f17213e0700 1 -- 172.21.15.146:0/3323777079 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714012430 con 0x7f171c0cf170 2024-09-13T18:40:58.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.677+0000 7f17233e4700 1 -- 172.21.15.146:0/3323777079 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f171c0cf170 msgr2=0x7f171c1595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:58.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.677+0000 7f17233e4700 1 --2- 172.21.15.146:0/3323777079 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f171c0cf170 0x7f171c1595f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1714009dc0 tx=0x7f171400b330 comp rx=0 tx=0).stop 2024-09-13T18:40:58.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.677+0000 7f17233e4700 1 -- 172.21.15.146:0/3323777079 shutdown_connections 2024-09-13T18:40:58.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.677+0000 7f17233e4700 1 --2- 172.21.15.146:0/3323777079 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f171c0cf170 0x7f171c1595f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:58.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.677+0000 7f17233e4700 1 -- 172.21.15.146:0/3323777079 >> 172.21.15.146:0/3323777079 conn(0x7f171c072220 msgr2=0x7f171c066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:58.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.677+0000 7f17233e4700 1 -- 172.21.15.146:0/3323777079 shutdown_connections 2024-09-13T18:40:58.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.677+0000 7f17233e4700 1 -- 172.21.15.146:0/3323777079 wait complete. 2024-09-13T18:40:58.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.677+0000 7f17233e4700 1 Processor -- start 2024-09-13T18:40:58.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.677+0000 7f17233e4700 1 -- start start 2024-09-13T18:40:58.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17233e4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f171c0cb5b0 0x7f171c0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:58.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17233e4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f171c0cf170 0x7f171c1e2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:58.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17233e4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f171c0cf550 0x7f171c1e0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:58.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17233e4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c1dfd20 con 0x7f171c0cb5b0 2024-09-13T18:40:58.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17233e4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c1ef5c0 con 0x7f171c0cf170 2024-09-13T18:40:58.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17233e4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c1eefd0 con 0x7f171c0cf550 2024-09-13T18:40:58.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17223e2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f171c0cb5b0 0x7f171c0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:58.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17223e2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f171c0cb5b0 0x7f171c0ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54948/0 (socket says 172.21.15.146:54948) 2024-09-13T18:40:58.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17223e2700 1 -- 172.21.15.146:0/3039645943 learned_addr learned my addr 172.21.15.146:0/3039645943 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:40:58.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f1721be1700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f171c0cf170 0x7f171c1e2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:58.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f1722be3700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f171c0cf550 0x7f171c1e0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:58.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17223e2700 1 -- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f171c0cf550 msgr2=0x7f171c1e0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:58.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17223e2700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f171c0cf550 0x7f171c1e0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:58.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17223e2700 1 -- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f171c0cf170 msgr2=0x7f171c1e2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:58.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17223e2700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f171c0cf170 0x7f171c1e2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:58.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17223e2700 1 -- 172.21.15.146:0/3039645943 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1714009a70 con 0x7f171c0cb5b0 2024-09-13T18:40:58.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17223e2700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f171c0cb5b0 0x7f171c0ce7c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1714014040 tx=0x7f1714004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:58.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f170effd700 1 -- 172.21.15.146:0/3039645943 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714012430 con 0x7f171c0cb5b0 2024-09-13T18:40:58.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f170effd700 1 -- 172.21.15.146:0/3039645943 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1714020070 con 0x7f171c0cb5b0 2024-09-13T18:40:58.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f170effd700 1 -- 172.21.15.146:0/3039645943 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171401ba70 con 0x7f171c0cb5b0 2024-09-13T18:40:58.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17233e4700 1 -- 172.21.15.146:0/3039645943 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c1e26c0 con 0x7f171c0cb5b0 2024-09-13T18:40:58.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17233e4700 1 -- 172.21.15.146:0/3039645943 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c1df7f0 con 0x7f171c0cb5b0 2024-09-13T18:40:58.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f17233e4700 1 -- 172.21.15.146:0/3039645943 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c066850 con 0x7f171c0cb5b0 2024-09-13T18:40:58.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.681+0000 7f170effd700 1 -- 172.21.15.146:0/3039645943 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f171401bbd0 con 0x7f171c0cb5b0 2024-09-13T18:40:58.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.685+0000 7f170effd700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1700066730 0x7f1700068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:40:58.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.685+0000 7f170effd700 1 -- 172.21.15.146:0/3039645943 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f1714022070 con 0x7f171c0cb5b0 2024-09-13T18:40:58.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.685+0000 7f1721be1700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1700066730 0x7f1700068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:40:58.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.685+0000 7f1721be1700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1700066730 0x7f1700068bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f171c0c94b0 tx=0x7f1710009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:40:58.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.689+0000 7f170effd700 1 -- 172.21.15.146:0/3039645943 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f17140537d0 con 0x7f171c0cb5b0 2024-09-13T18:40:58.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.921+0000 7f17233e4700 1 -- 172.21.15.146:0/3039645943 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f171c0cb180 con 0x7f171c0cb5b0 2024-09-13T18:40:58.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.921+0000 7f170effd700 1 -- 172.21.15.146:0/3039645943 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5533 (secure 0 0 0) 0x7f171402c090 con 0x7f171c0cb5b0 2024-09-13T18:40:58.938 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:40:58.938 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:40:56.567768+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:reconnect","state_seq":39,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:40:58.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.925+0000 7f17233e4700 1 -- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1700066730 msgr2=0x7f1700068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:58.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.925+0000 7f17233e4700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1700066730 0x7f1700068bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f171c0c94b0 tx=0x7f1710009040 comp rx=0 tx=0).stop 2024-09-13T18:40:58.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.925+0000 7f17233e4700 1 -- 172.21.15.146:0/3039645943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f171c0cb5b0 msgr2=0x7f171c0ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:40:58.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.925+0000 7f17233e4700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f171c0cb5b0 0x7f171c0ce7c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1714014040 tx=0x7f1714004af0 comp rx=0 tx=0).stop 2024-09-13T18:40:58.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.925+0000 7f17233e4700 1 -- 172.21.15.146:0/3039645943 shutdown_connections 2024-09-13T18:40:58.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.925+0000 7f17233e4700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1700066730 0x7f1700068bf0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:58.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.925+0000 7f17233e4700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f171c0cf550 0x7f171c1e0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:58.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.925+0000 7f17233e4700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f171c0cf170 0x7f171c1e2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:58.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.925+0000 7f17233e4700 1 --2- 172.21.15.146:0/3039645943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f171c0cb5b0 0x7f171c0ce7c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:40:58.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.925+0000 7f17233e4700 1 -- 172.21.15.146:0/3039645943 >> 172.21.15.146:0/3039645943 conn(0x7f171c072220 msgr2=0x7f171c0c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:40:58.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.925+0000 7f17233e4700 1 -- 172.21.15.146:0/3039645943 shutdown_connections 2024-09-13T18:40:58.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:40:58.925+0000 7f17233e4700 1 -- 172.21.15.146:0/3039645943 wait complete. 2024-09-13T18:40:58.942 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 60 2024-09-13T18:40:58.954 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 60, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 60, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5784": { "addr": "172.21.15.146:6833/3901685195", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3901685195, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3901685195, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5784, "incarnation": 46, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 29 }, "gid_5800": { "addr": "172.21.15.154:6839/776638146", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 776638146, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 776638146, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5800, "incarnation": 52, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 30 }, "gid_6063": { "addr": "172.21.15.146:6835/832794630", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 832794630, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 832794630, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6063, "incarnation": 58, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:reconnect", "state_seq": 39 } }, "last_failure": 0, "last_failure_osd_epoch": 35, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:40:56.567768+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5800, "mds_1": 5784, "mds_2": 6063 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:41:00.956 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:41:01.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f8587247700 1 Processor -- start 2024-09-13T18:41:01.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f8587247700 1 -- start start 2024-09-13T18:41:01.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f8587247700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f858006cb00 0x7f85801513b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:01.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f8587247700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f85800afbe0 0x7f85801518f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:01.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f8587247700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85801584a0 con 0x7f858006c720 2024-09-13T18:41:01.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f8587247700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85800c3dd0 con 0x7f85800afbe0 2024-09-13T18:41:01.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f8587247700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85800c9150 con 0x7f858006cb00 2024-09-13T18:41:01.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f8584fe3700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f858006c720 0x7f8580157d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:47286/0 (socket says 172.21.15.146:47286) 2024-09-13T18:41:01.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f8584fe3700 1 -- 172.21.15.146:0/1439056399 learned_addr learned my addr 172.21.15.146:0/1439056399 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:41:01.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f85857e4700 1 --2- 172.21.15.146:0/1439056399 >> v2:172.21.15.146:3300/0 conn(0x7f85800afbe0 0x7f85801518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:01.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f857ffff700 1 --2- 172.21.15.146:0/1439056399 >> v2:172.21.15.154:3300/0 conn(0x7f858006cb00 0x7f85801513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:01.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f85857e4700 1 -- 172.21.15.146:0/1439056399 >> v2:172.21.15.154:3300/0 conn(0x7f858006cb00 msgr2=0x7f85801513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:41:01.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f85857e4700 1 --2- 172.21.15.146:0/1439056399 >> v2:172.21.15.154:3300/0 conn(0x7f858006cb00 0x7f85801513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:01.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f85857e4700 1 -- 172.21.15.146:0/1439056399 >> v1:172.21.15.146:6789/0 conn(0x7f858006c720 legacy=0x7f8580157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:01.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f857f7fe700 1 -- 172.21.15.146:0/1439056399 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341256257 0 0) 0x7f85801584a0 con 0x7f858006c720 2024-09-13T18:41:01.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f85857e4700 1 -- 172.21.15.146:0/1439056399 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85800c9430 con 0x7f85800afbe0 2024-09-13T18:41:01.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f85857e4700 1 --2- 172.21.15.146:0/1439056399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85800afbe0 0x7f85801518f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f85800c8cb0 tx=0x7f857800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f688ceb546253 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:01.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f857f7fe700 1 -- 172.21.15.146:0/1439056399 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857800e070 con 0x7f85800afbe0 2024-09-13T18:41:01.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f857f7fe700 1 -- 172.21.15.146:0/1439056399 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857800bbd0 con 0x7f85800afbe0 2024-09-13T18:41:01.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.069+0000 7f857f7fe700 1 -- 172.21.15.146:0/1439056399 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578012430 con 0x7f85800afbe0 2024-09-13T18:41:01.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 -- 172.21.15.146:0/1439056399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85800afbe0 msgr2=0x7f85801518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:01.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 --2- 172.21.15.146:0/1439056399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85800afbe0 0x7f85801518f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f85800c8cb0 tx=0x7f857800b330 comp rx=0 tx=0).stop 2024-09-13T18:41:01.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 -- 172.21.15.146:0/1439056399 shutdown_connections 2024-09-13T18:41:01.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 --2- 172.21.15.146:0/1439056399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85800afbe0 0x7f85801518f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:01.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 --2- 172.21.15.146:0/1439056399 >> v2:172.21.15.154:3300/0 conn(0x7f858006cb00 0x7f85801513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:01.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 -- 172.21.15.146:0/1439056399 >> 172.21.15.146:0/1439056399 conn(0x7f858006a9a0 msgr2=0x7f858006ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:41:01.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 -- 172.21.15.146:0/1439056399 shutdown_connections 2024-09-13T18:41:01.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 -- 172.21.15.146:0/1439056399 wait complete. 2024-09-13T18:41:01.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 Processor -- start 2024-09-13T18:41:01.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 -- start start 2024-09-13T18:41:01.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f858006c720 0x7f85800cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:01.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f858006cb00 0x7f8580159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:01.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85800afbe0 0x7f85800cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:01.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85800cb840 con 0x7f858006c720 2024-09-13T18:41:01.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85800b1fe0 con 0x7f85800afbe0 2024-09-13T18:41:01.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85800cca20 con 0x7f858006cb00 2024-09-13T18:41:01.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8584fe3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f858006c720 0x7f85800cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:01.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8584fe3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f858006c720 0x7f85800cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46316/0 (socket says 172.21.15.146:46316) 2024-09-13T18:41:01.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8584fe3700 1 -- 172.21.15.146:0/3675548731 learned_addr learned my addr 172.21.15.146:0/3675548731 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:41:01.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f85857e4700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85800afbe0 0x7f85800cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:01.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f857ffff700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f858006cb00 0x7f8580159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:01.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8584fe3700 1 -- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f858006cb00 msgr2=0x7f8580159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:01.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8584fe3700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f858006cb00 0x7f8580159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:01.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8584fe3700 1 -- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85800afbe0 msgr2=0x7f85800cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:01.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8584fe3700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85800afbe0 0x7f85800cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:01.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8584fe3700 1 -- 172.21.15.146:0/3675548731 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8578009a70 con 0x7f858006c720 2024-09-13T18:41:01.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f857ffff700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f858006cb00 0x7f8580159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:41:01.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8584fe3700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f858006c720 0x7f85800cc4e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f856c00c5b0 tx=0x7f856c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:01.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f857d7fa700 1 -- 172.21.15.146:0/3675548731 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c004ca0 con 0x7f858006c720 2024-09-13T18:41:01.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f857d7fa700 1 -- 172.21.15.146:0/3675548731 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c004e20 con 0x7f858006c720 2024-09-13T18:41:01.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 -- 172.21.15.146:0/3675548731 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8580140430 con 0x7f858006c720 2024-09-13T18:41:01.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f8587247700 1 -- 172.21.15.146:0/3675548731 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85800cb280 con 0x7f858006c720 2024-09-13T18:41:01.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.073+0000 7f857d7fa700 1 -- 172.21.15.146:0/3675548731 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c007310 con 0x7f858006c720 2024-09-13T18:41:01.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.077+0000 7f8587247700 1 -- 172.21.15.146:0/3675548731 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858015aa90 con 0x7f858006c720 2024-09-13T18:41:01.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.077+0000 7f857d7fa700 1 -- 172.21.15.146:0/3675548731 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f856c007470 con 0x7f858006c720 2024-09-13T18:41:01.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.077+0000 7f857d7fa700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8564066730 0x7f8564068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:01.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.077+0000 7f857d7fa700 1 -- 172.21.15.146:0/3675548731 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f856c0844b0 con 0x7f858006c720 2024-09-13T18:41:01.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.077+0000 7f857ffff700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8564066730 0x7f8564068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:01.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.077+0000 7f857ffff700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8564066730 0x7f8564068bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f858015a4f0 tx=0x7f8574008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:01.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.085+0000 7f857d7fa700 1 -- 172.21.15.146:0/3675548731 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f856c0511c0 con 0x7f858006c720 2024-09-13T18:41:01.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.301+0000 7f8587247700 1 -- 172.21.15.146:0/3675548731 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85800c8360 con 0x7f858006c720 2024-09-13T18:41:01.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.305+0000 7f857d7fa700 1 -- 172.21.15.146:0/3675548731 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5533 (secure 0 0 0) 0x7f856c007720 con 0x7f858006c720 2024-09-13T18:41:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:41:01.321 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:40:56.567768+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:reconnect","state_seq":39,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:41:01.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.305+0000 7f8587247700 1 -- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8564066730 msgr2=0x7f8564068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:01.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.305+0000 7f8587247700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8564066730 0x7f8564068bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f858015a4f0 tx=0x7f8574008040 comp rx=0 tx=0).stop 2024-09-13T18:41:01.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.305+0000 7f8587247700 1 -- 172.21.15.146:0/3675548731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f858006c720 msgr2=0x7f85800cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:01.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.305+0000 7f8587247700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f858006c720 0x7f85800cc4e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f856c00c5b0 tx=0x7f856c00c8c0 comp rx=0 tx=0).stop 2024-09-13T18:41:01.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.305+0000 7f8587247700 1 -- 172.21.15.146:0/3675548731 shutdown_connections 2024-09-13T18:41:01.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.305+0000 7f8587247700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8564066730 0x7f8564068bf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:01.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.305+0000 7f8587247700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85800afbe0 0x7f85800cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:01.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.305+0000 7f8587247700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f858006cb00 0x7f8580159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:01.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.305+0000 7f8587247700 1 --2- 172.21.15.146:0/3675548731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f858006c720 0x7f85800cc4e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:01.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.305+0000 7f8587247700 1 -- 172.21.15.146:0/3675548731 >> 172.21.15.146:0/3675548731 conn(0x7f858006a9a0 msgr2=0x7f8580155180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:41:01.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.305+0000 7f8587247700 1 -- 172.21.15.146:0/3675548731 shutdown_connections 2024-09-13T18:41:01.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:01.305+0000 7f8587247700 1 -- 172.21.15.146:0/3675548731 wait complete. 2024-09-13T18:41:01.325 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 60 2024-09-13T18:41:01.337 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:41:03.338 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:41:03.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 Processor -- start 2024-09-13T18:41:03.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 -- start start 2024-09-13T18:41:03.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1f380721f0 0x7f1f38157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:03.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f380cac60 0x7f1f380cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:03.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f380c8c70 con 0x7f1f38154a40 2024-09-13T18:41:03.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f38072810 con 0x7f1f380721f0 2024-09-13T18:41:03.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f38157910 con 0x7f1f380cac60 2024-09-13T18:41:03.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3c8e6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1f380721f0 0x7f1f38157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:03.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3c8e6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1f380721f0 0x7f1f38157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46322/0 (socket says 172.21.15.146:46322) 2024-09-13T18:41:03.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3c8e6700 1 -- 172.21.15.146:0/3347661710 learned_addr learned my addr 172.21.15.146:0/3347661710 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:41:03.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3d8e8700 1 --2- 172.21.15.146:0/3347661710 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f380cac60 0x7f1f380cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:03.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3c8e6700 1 -- 172.21.15.146:0/3347661710 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f380cac60 msgr2=0x7f1f380cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:41:03.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3c8e6700 1 --2- 172.21.15.146:0/3347661710 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f380cac60 0x7f1f380cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:03.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3c8e6700 1 -- 172.21.15.146:0/3347661710 >> v1:172.21.15.154:6789/0 conn(0x7f1f38154a40 legacy=0x7f1f3806f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:03.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3c8e6700 1 -- 172.21.15.146:0/3347661710 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f380c8f50 con 0x7f1f380721f0 2024-09-13T18:41:03.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3d8e8700 1 --2- 172.21.15.146:0/3347661710 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f380cac60 0x7f1f380cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:41:03.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3c8e6700 1 --2- 172.21.15.146:0/3347661710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f380721f0 0x7f1f38157d30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009910 tx=0x7f1f28009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cd08f876e0267 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:03.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f2ffff700 1 -- 172.21.15.146:0/3347661710 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800ba60 con 0x7f1f380721f0 2024-09-13T18:41:03.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f2ffff700 1 -- 172.21.15.146:0/3347661710 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2800bbc0 con 0x7f1f380721f0 2024-09-13T18:41:03.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f2ffff700 1 -- 172.21.15.146:0/3347661710 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28009e60 con 0x7f1f380721f0 2024-09-13T18:41:03.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3347661710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f380721f0 msgr2=0x7f1f38157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:03.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 --2- 172.21.15.146:0/3347661710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f380721f0 0x7f1f38157d30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009910 tx=0x7f1f28009c20 comp rx=0 tx=0).stop 2024-09-13T18:41:03.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3347661710 shutdown_connections 2024-09-13T18:41:03.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 --2- 172.21.15.146:0/3347661710 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f380cac60 0x7f1f380cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:03.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 --2- 172.21.15.146:0/3347661710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f380721f0 0x7f1f38157d30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:03.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3347661710 >> 172.21.15.146:0/3347661710 conn(0x7f1f38066090 msgr2=0x7f1f38066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:41:03.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3347661710 shutdown_connections 2024-09-13T18:41:03.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.445+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3347661710 wait complete. 2024-09-13T18:41:03.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3e0e9700 1 Processor -- start 2024-09-13T18:41:03.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3e0e9700 1 -- start start 2024-09-13T18:41:03.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3e0e9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f380721f0 0x7f1f380c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:03.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3e0e9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f380cac60 0x7f1f381edf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:03.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3e0e9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f38154a40 0x7f1f380ab280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:03.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3e0e9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380a9ea0 con 0x7f1f380721f0 2024-09-13T18:41:03.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3e0e9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380aa460 con 0x7f1f380cac60 2024-09-13T18:41:03.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3e0e9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380ac310 con 0x7f1f38154a40 2024-09-13T18:41:03.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3d0e7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f380721f0 0x7f1f380c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:03.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3d0e7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f380721f0 0x7f1f380c5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46328/0 (socket says 172.21.15.146:46328) 2024-09-13T18:41:03.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3d0e7700 1 -- 172.21.15.146:0/3115771054 learned_addr learned my addr 172.21.15.146:0/3115771054 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:41:03.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3d8e8700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f38154a40 0x7f1f380ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:03.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3c8e6700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f380cac60 0x7f1f381edf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:03.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3d0e7700 1 -- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f38154a40 msgr2=0x7f1f380ab280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:03.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3d0e7700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f38154a40 0x7f1f380ab280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:03.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3d0e7700 1 -- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f380cac60 msgr2=0x7f1f381edf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:03.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3d0e7700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f380cac60 0x7f1f381edf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:03.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3d0e7700 1 -- 172.21.15.146:0/3115771054 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f28009570 con 0x7f1f380721f0 2024-09-13T18:41:03.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3d0e7700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f380721f0 0x7f1f380c5630 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400c530 tx=0x7f1f3400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:03.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f2dffb700 1 -- 172.21.15.146:0/3115771054 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f340052d0 con 0x7f1f380721f0 2024-09-13T18:41:03.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f2dffb700 1 -- 172.21.15.146:0/3115771054 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f34005430 con 0x7f1f380721f0 2024-09-13T18:41:03.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f2dffb700 1 -- 172.21.15.146:0/3115771054 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f340055a0 con 0x7f1f380721f0 2024-09-13T18:41:03.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3115771054 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f380ac5f0 con 0x7f1f380721f0 2024-09-13T18:41:03.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.449+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3115771054 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f380ac980 con 0x7f1f380721f0 2024-09-13T18:41:03.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.453+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3115771054 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f380c9e00 con 0x7f1f380721f0 2024-09-13T18:41:03.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.453+0000 7f1f2dffb700 1 -- 172.21.15.146:0/3115771054 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f340038e0 con 0x7f1f380721f0 2024-09-13T18:41:03.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.453+0000 7f1f2dffb700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1f1c066760 0x7f1f1c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:03.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.453+0000 7f1f2dffb700 1 -- 172.21.15.146:0/3115771054 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f1f34083bc0 con 0x7f1f380721f0 2024-09-13T18:41:03.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.453+0000 7f1f3c8e6700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1f1c066760 0x7f1f1c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:03.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.453+0000 7f1f3c8e6700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1f1c066760 0x7f1f1c068c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1f28000c00 tx=0x7f1f28008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:03.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.461+0000 7f1f2dffb700 1 -- 172.21.15.146:0/3115771054 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1f34050950 con 0x7f1f380721f0 2024-09-13T18:41:03.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.693+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3115771054 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f380ad490 con 0x7f1f380721f0 2024-09-13T18:41:03.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.697+0000 7f1f2dffb700 1 -- 172.21.15.146:0/3115771054 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5530 (secure 0 0 0) 0x7f1f34053f70 con 0x7f1f380721f0 2024-09-13T18:41:03.713 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:41:03.713 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:41:01.600269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:rejoin","state_seq":41,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:41:03.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.697+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1f1c066760 msgr2=0x7f1f1c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:03.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.697+0000 7f1f3e0e9700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1f1c066760 0x7f1f1c068c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1f28000c00 tx=0x7f1f28008030 comp rx=0 tx=0).stop 2024-09-13T18:41:03.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.697+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3115771054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f380721f0 msgr2=0x7f1f380c5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.697+0000 7f1f3e0e9700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f380721f0 0x7f1f380c5630 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400c530 tx=0x7f1f3400c840 comp rx=0 tx=0).stop 2024-09-13T18:41:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.697+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3115771054 shutdown_connections 2024-09-13T18:41:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.697+0000 7f1f3e0e9700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1f1c066760 0x7f1f1c068c20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.697+0000 7f1f3e0e9700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f38154a40 0x7f1f380ab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.697+0000 7f1f3e0e9700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f380cac60 0x7f1f381edf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.697+0000 7f1f3e0e9700 1 --2- 172.21.15.146:0/3115771054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f380721f0 0x7f1f380c5630 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.697+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3115771054 >> 172.21.15.146:0/3115771054 conn(0x7f1f38066090 msgr2=0x7f1f38070160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:41:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.697+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3115771054 shutdown_connections 2024-09-13T18:41:03.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:03.697+0000 7f1f3e0e9700 1 -- 172.21.15.146:0/3115771054 wait complete. 2024-09-13T18:41:03.717 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 61 2024-09-13T18:41:03.730 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:41:05.730 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:41:05.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.837+0000 7f5d1d53d700 1 Processor -- start 2024-09-13T18:41:05.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.837+0000 7f5d1d53d700 1 -- start start 2024-09-13T18:41:05.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5d180cbd50 0x7f5d180b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:05.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5d180ca210 0x7f5d180cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:05.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d18071390 0x7f5d180701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:05.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d180caa30 con 0x7f5d180cbd50 2024-09-13T18:41:05.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d180cb810 con 0x7f5d180ca210 2024-09-13T18:41:05.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d180cc670 con 0x7f5d18071390 2024-09-13T18:41:05.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d17fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5d180cbd50 0x7f5d180b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:05.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d17fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5d180cbd50 0x7f5d180b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46340/0 (socket says 172.21.15.146:46340) 2024-09-13T18:41:05.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d17fff700 1 -- 172.21.15.146:0/656765399 learned_addr learned my addr 172.21.15.146:0/656765399 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:41:05.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d0f7fe700 1 --2- 172.21.15.146:0/656765399 >> v2:172.21.15.154:3300/0 conn(0x7f5d180ca210 0x7f5d180cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:05.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1cd3c700 1 --2- 172.21.15.146:0/656765399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d18071390 0x7f5d180701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:05.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d17fff700 1 -- 172.21.15.146:0/656765399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d18071390 msgr2=0x7f5d180701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:41:05.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d17fff700 1 --2- 172.21.15.146:0/656765399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d18071390 0x7f5d180701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:05.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d17fff700 1 -- 172.21.15.146:0/656765399 >> v2:172.21.15.154:3300/0 conn(0x7f5d180ca210 msgr2=0x7f5d180cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:41:05.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d17fff700 1 --2- 172.21.15.146:0/656765399 >> v2:172.21.15.154:3300/0 conn(0x7f5d180ca210 0x7f5d180cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d17fff700 1 -- 172.21.15.146:0/656765399 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1815da20 con 0x7f5d180cbd50 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d17fff700 1 --2- 172.21.15.146:0/656765399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d180cbd50 0x7f5d180b1c40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5d180b18a0 tx=0x7f5d08009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57f559e7fa19244 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d177fe700 1 -- 172.21.15.146:0/656765399 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0800ba20 con 0x7f5d180cbd50 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d177fe700 1 -- 172.21.15.146:0/656765399 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0800bb80 con 0x7f5d180cbd50 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d177fe700 1 -- 172.21.15.146:0/656765399 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08009e60 con 0x7f5d180cbd50 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 -- 172.21.15.146:0/656765399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d180cbd50 msgr2=0x7f5d180b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 --2- 172.21.15.146:0/656765399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d180cbd50 0x7f5d180b1c40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5d180b18a0 tx=0x7f5d08009c20 comp rx=0 tx=0).stop 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 -- 172.21.15.146:0/656765399 shutdown_connections 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 --2- 172.21.15.146:0/656765399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d18071390 0x7f5d180701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 --2- 172.21.15.146:0/656765399 >> v2:172.21.15.154:3300/0 conn(0x7f5d180ca210 0x7f5d180cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 --2- 172.21.15.146:0/656765399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d180cbd50 0x7f5d180b1c40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 -- 172.21.15.146:0/656765399 >> 172.21.15.146:0/656765399 conn(0x7f5d18066270 msgr2=0x7f5d18066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 -- 172.21.15.146:0/656765399 shutdown_connections 2024-09-13T18:41:05.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 -- 172.21.15.146:0/656765399 wait complete. 2024-09-13T18:41:05.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 Processor -- start 2024-09-13T18:41:05.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.841+0000 7f5d1d53d700 1 -- start start 2024-09-13T18:41:05.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d1d53d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d18071390 0x7f5d1815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:05.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d1d53d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d180ca210 0x7f5d181515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:05.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d1d53d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d180cbd50 0x7f5d180c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:05.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d1d53d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1815c790 con 0x7f5d180ca210 2024-09-13T18:41:05.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d1d53d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d181ef360 con 0x7f5d180cbd50 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d1d53d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d181ef640 con 0x7f5d18071390 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d0f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d180ca210 0x7f5d181515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d0f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d180ca210 0x7f5d181515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46346/0 (socket says 172.21.15.146:46346) 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d0f7fe700 1 -- 172.21.15.146:0/2876490019 learned_addr learned my addr 172.21.15.146:0/2876490019 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d17fff700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d18071390 0x7f5d1815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d1cd3c700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d180cbd50 0x7f5d180c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d0f7fe700 1 -- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d18071390 msgr2=0x7f5d1815c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d0f7fe700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d18071390 0x7f5d1815c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d0f7fe700 1 -- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d180cbd50 msgr2=0x7f5d180c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d0f7fe700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d180cbd50 0x7f5d180c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d0f7fe700 1 -- 172.21.15.146:0/2876490019 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d08009570 con 0x7f5d180ca210 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d17fff700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d18071390 0x7f5d1815c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d0f7fe700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d180ca210 0x7f5d181515e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000e760 tx=0x7f5d0000ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:05.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d157fa700 1 -- 172.21.15.146:0/2876490019 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00013610 con 0x7f5d180ca210 2024-09-13T18:41:05.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d157fa700 1 -- 172.21.15.146:0/2876490019 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d00013770 con 0x7f5d180ca210 2024-09-13T18:41:05.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d1d53d700 1 -- 172.21.15.146:0/2876490019 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d181ef920 con 0x7f5d180ca210 2024-09-13T18:41:05.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d157fa700 1 -- 172.21.15.146:0/2876490019 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00013ae0 con 0x7f5d180ca210 2024-09-13T18:41:05.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d1d53d700 1 -- 172.21.15.146:0/2876490019 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d181efee0 con 0x7f5d180ca210 2024-09-13T18:41:05.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.845+0000 7f5d1d53d700 1 -- 172.21.15.146:0/2876490019 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1815d500 con 0x7f5d180ca210 2024-09-13T18:41:05.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.849+0000 7f5d157fa700 1 -- 172.21.15.146:0/2876490019 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5d00013c40 con 0x7f5d180ca210 2024-09-13T18:41:05.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.849+0000 7f5d157fa700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cf8066730 0x7f5cf8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:05.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.849+0000 7f5d157fa700 1 -- 172.21.15.146:0/2876490019 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f5d00084580 con 0x7f5d180ca210 2024-09-13T18:41:05.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.849+0000 7f5d17fff700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cf8066730 0x7f5cf8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:05.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.849+0000 7f5d17fff700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cf8066730 0x7f5cf8068bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5d08000c00 tx=0x7f5d08008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:05.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:05.853+0000 7f5d157fa700 1 -- 172.21.15.146:0/2876490019 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5d00051170 con 0x7f5d180ca210 2024-09-13T18:41:06.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.093+0000 7f5d1d53d700 1 -- 172.21.15.146:0/2876490019 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d180c9ff0 con 0x7f5d180ca210 2024-09-13T18:41:06.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.093+0000 7f5d157fa700 1 -- 172.21.15.146:0/2876490019 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5530 (secure 0 0 0) 0x7f5d00054790 con 0x7f5d180ca210 2024-09-13T18:41:06.111 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:41:06.112 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:41:01.600269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:rejoin","state_seq":41,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:41:06.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.097+0000 7f5d1d53d700 1 -- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cf8066730 msgr2=0x7f5cf8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:06.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.097+0000 7f5d1d53d700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cf8066730 0x7f5cf8068bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5d08000c00 tx=0x7f5d08008030 comp rx=0 tx=0).stop 2024-09-13T18:41:06.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.097+0000 7f5d1d53d700 1 -- 172.21.15.146:0/2876490019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d180ca210 msgr2=0x7f5d181515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.097+0000 7f5d1d53d700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d180ca210 0x7f5d181515e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000e760 tx=0x7f5d0000ea70 comp rx=0 tx=0).stop 2024-09-13T18:41:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.097+0000 7f5d1d53d700 1 -- 172.21.15.146:0/2876490019 shutdown_connections 2024-09-13T18:41:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.097+0000 7f5d1d53d700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cf8066730 0x7f5cf8068bf0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.097+0000 7f5d1d53d700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d180cbd50 0x7f5d180c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.097+0000 7f5d1d53d700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d180ca210 0x7f5d181515e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.097+0000 7f5d1d53d700 1 --2- 172.21.15.146:0/2876490019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d18071390 0x7f5d1815c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.097+0000 7f5d1d53d700 1 -- 172.21.15.146:0/2876490019 >> 172.21.15.146:0/2876490019 conn(0x7f5d18066270 msgr2=0x7f5d180c4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:41:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.097+0000 7f5d1d53d700 1 -- 172.21.15.146:0/2876490019 shutdown_connections 2024-09-13T18:41:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:06.097+0000 7f5d1d53d700 1 -- 172.21.15.146:0/2876490019 wait complete. 2024-09-13T18:41:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 61 2024-09-13T18:41:06.126 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:41:08.127 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:41:08.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.225+0000 7fc363fff700 1 Processor -- start 2024-09-13T18:41:08.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.225+0000 7fc363fff700 1 -- start start 2024-09-13T18:41:08.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.225+0000 7fc363fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc36406e7c0 0x7fc3640b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:08.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.225+0000 7fc363fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc36406eba0 0x7fc3640c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:08.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.225+0000 7fc363fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3640c1020 con 0x7fc3641598f0 2024-09-13T18:41:08.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.225+0000 7fc363fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3641593f0 con 0x7fc36406eba0 2024-09-13T18:41:08.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.225+0000 7fc363fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640c6a70 con 0x7fc36406e7c0 2024-09-13T18:41:08.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc362ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc36406e7c0 0x7fc3640b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:08.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc362ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc36406e7c0 0x7fc3640b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:39154/0 (socket says 172.21.15.146:39154) 2024-09-13T18:41:08.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc362ffd700 1 -- 172.21.15.146:0/3733184579 learned_addr learned my addr 172.21.15.146:0/3733184579 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:41:08.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc3627fc700 1 --2- 172.21.15.146:0/3733184579 >> v2:172.21.15.154:3300/0 conn(0x7fc36406eba0 0x7fc3640c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:08.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc362ffd700 1 -- 172.21.15.146:0/3733184579 >> v2:172.21.15.154:3300/0 conn(0x7fc36406eba0 msgr2=0x7fc3640c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:41:08.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc362ffd700 1 --2- 172.21.15.146:0/3733184579 >> v2:172.21.15.154:3300/0 conn(0x7fc36406eba0 0x7fc3640c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:08.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc362ffd700 1 -- 172.21.15.146:0/3733184579 >> v1:172.21.15.154:6789/0 conn(0x7fc3641598f0 legacy=0x7fc3641531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:08.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc362ffd700 1 -- 172.21.15.146:0/3733184579 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3640c1290 con 0x7fc36406e7c0 2024-09-13T18:41:08.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc362ffd700 1 --2- 172.21.15.146:0/3733184579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc36406e7c0 0x7fc3640b11a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc358009910 tx=0x7fc358009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feed7565b3baf91d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:08.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc361ffb700 1 -- 172.21.15.146:0/3733184579 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358011830 con 0x7fc36406e7c0 2024-09-13T18:41:08.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc361ffb700 1 -- 172.21.15.146:0/3733184579 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc358011990 con 0x7fc36406e7c0 2024-09-13T18:41:08.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc361ffb700 1 -- 172.21.15.146:0/3733184579 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358011d40 con 0x7fc36406e7c0 2024-09-13T18:41:08.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc363fff700 1 -- 172.21.15.146:0/3733184579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc36406e7c0 msgr2=0x7fc3640b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:08.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc363fff700 1 --2- 172.21.15.146:0/3733184579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc36406e7c0 0x7fc3640b11a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc358009910 tx=0x7fc358009c20 comp rx=0 tx=0).stop 2024-09-13T18:41:08.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc363fff700 1 -- 172.21.15.146:0/3733184579 shutdown_connections 2024-09-13T18:41:08.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc363fff700 1 --2- 172.21.15.146:0/3733184579 >> v2:172.21.15.154:3300/0 conn(0x7fc36406eba0 0x7fc3640c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:08.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc363fff700 1 --2- 172.21.15.146:0/3733184579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc36406e7c0 0x7fc3640b11a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:08.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc363fff700 1 -- 172.21.15.146:0/3733184579 >> 172.21.15.146:0/3733184579 conn(0x7fc364071ed0 msgr2=0x7fc3640722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:41:08.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc363fff700 1 -- 172.21.15.146:0/3733184579 shutdown_connections 2024-09-13T18:41:08.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.229+0000 7fc363fff700 1 -- 172.21.15.146:0/3733184579 wait complete. 2024-09-13T18:41:08.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc363fff700 1 Processor -- start 2024-09-13T18:41:08.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc363fff700 1 -- start start 2024-09-13T18:41:08.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc363fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc36406e7c0 0x7fc3640cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:08.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc363fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc36406eba0 0x7fc3640c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:08.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc363fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3641598f0 0x7fc36414a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:08.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc363fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc364071710 con 0x7fc36406e7c0 2024-09-13T18:41:08.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc363fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640aff00 con 0x7fc3641598f0 2024-09-13T18:41:08.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc363fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640af740 con 0x7fc36406eba0 2024-09-13T18:41:08.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc362ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc36406e7c0 0x7fc3640cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:08.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc362ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc36406e7c0 0x7fc3640cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46352/0 (socket says 172.21.15.146:46352) 2024-09-13T18:41:08.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc362ffd700 1 -- 172.21.15.146:0/1157792885 learned_addr learned my addr 172.21.15.146:0/1157792885 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:41:08.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc3627fc700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc36406eba0 0x7fc3640c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:08.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc3637fe700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3641598f0 0x7fc36414a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:08.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc362ffd700 1 -- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc36406eba0 msgr2=0x7fc3640c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:08.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc362ffd700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc36406eba0 0x7fc3640c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:08.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc362ffd700 1 -- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3641598f0 msgr2=0x7fc36414a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:08.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc362ffd700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3641598f0 0x7fc36414a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:08.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc362ffd700 1 -- 172.21.15.146:0/1157792885 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc358009570 con 0x7fc36406e7c0 2024-09-13T18:41:08.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc362ffd700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc36406e7c0 0x7fc3640cb230 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc3640c6bb0 tx=0x7fc358004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:08.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc34b7fe700 1 -- 172.21.15.146:0/1157792885 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358004510 con 0x7fc36406e7c0 2024-09-13T18:41:08.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc34b7fe700 1 -- 172.21.15.146:0/1157792885 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35801e070 con 0x7fc36406e7c0 2024-09-13T18:41:08.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc363fff700 1 -- 172.21.15.146:0/1157792885 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3640ae000 con 0x7fc36406e7c0 2024-09-13T18:41:08.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc34b7fe700 1 -- 172.21.15.146:0/1157792885 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358003900 con 0x7fc36406e7c0 2024-09-13T18:41:08.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.233+0000 7fc363fff700 1 -- 172.21.15.146:0/1157792885 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3640add20 con 0x7fc36406e7c0 2024-09-13T18:41:08.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.237+0000 7fc363fff700 1 -- 172.21.15.146:0/1157792885 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3640af580 con 0x7fc36406e7c0 2024-09-13T18:41:08.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.237+0000 7fc34b7fe700 1 -- 172.21.15.146:0/1157792885 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc358003a60 con 0x7fc36406e7c0 2024-09-13T18:41:08.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.237+0000 7fc34b7fe700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc350066760 0x7fc350068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:08.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.237+0000 7fc34b7fe700 1 -- 172.21.15.146:0/1157792885 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7fc358020070 con 0x7fc36406e7c0 2024-09-13T18:41:08.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.237+0000 7fc3627fc700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc350066760 0x7fc350068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:08.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.237+0000 7fc3627fc700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc350066760 0x7fc350068c20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc364151b50 tx=0x7fc354009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:08.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.245+0000 7fc34b7fe700 1 -- 172.21.15.146:0/1157792885 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc358052890 con 0x7fc36406e7c0 2024-09-13T18:41:08.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.477+0000 7fc363fff700 1 -- 172.21.15.146:0/1157792885 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3640b11a0 con 0x7fc36406e7c0 2024-09-13T18:41:08.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.477+0000 7fc34b7fe700 1 -- 172.21.15.146:0/1157792885 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+5530 (secure 0 0 0) 0x7fc358006da0 con 0x7fc36406e7c0 2024-09-13T18:41:08.496 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:41:08.496 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:41:06.615397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:41:08.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.481+0000 7fc363fff700 1 -- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc350066760 msgr2=0x7fc350068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:08.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.481+0000 7fc363fff700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc350066760 0x7fc350068c20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc364151b50 tx=0x7fc354009240 comp rx=0 tx=0).stop 2024-09-13T18:41:08.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.481+0000 7fc363fff700 1 -- 172.21.15.146:0/1157792885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc36406e7c0 msgr2=0x7fc3640cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:08.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.481+0000 7fc363fff700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc36406e7c0 0x7fc3640cb230 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc3640c6bb0 tx=0x7fc358004770 comp rx=0 tx=0).stop 2024-09-13T18:41:08.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.481+0000 7fc363fff700 1 -- 172.21.15.146:0/1157792885 shutdown_connections 2024-09-13T18:41:08.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.481+0000 7fc363fff700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc350066760 0x7fc350068c20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:08.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.481+0000 7fc363fff700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3641598f0 0x7fc36414a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:08.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.481+0000 7fc363fff700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc36406eba0 0x7fc3640c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:08.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.481+0000 7fc363fff700 1 --2- 172.21.15.146:0/1157792885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc36406e7c0 0x7fc3640cb230 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:08.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.481+0000 7fc363fff700 1 -- 172.21.15.146:0/1157792885 >> 172.21.15.146:0/1157792885 conn(0x7fc364071ed0 msgr2=0x7fc36414ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:41:08.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.481+0000 7fc363fff700 1 -- 172.21.15.146:0/1157792885 shutdown_connections 2024-09-13T18:41:08.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:08.481+0000 7fc363fff700 1 -- 172.21.15.146:0/1157792885 wait complete. 2024-09-13T18:41:08.502 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 62 2024-09-13T18:41:08.514 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=2, replacing gid=5649 2024-09-13T18:41:08.514 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.c 2024-09-13T18:41:09.537 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:41:09.539 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:41:13.514 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-09-13T18:41:13.515 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T18:41:13.515 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T18:41:13.517 INFO:tasks.ceph.mds.c:Started 2024-09-13T18:41:13.517 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:41:13.607 INFO:tasks.ceph.mds.c.smithi146.stdout:starting mds.c at 2024-09-13T18:41:13.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.621+0000 7f2e11d14700 1 Processor -- start 2024-09-13T18:41:13.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.621+0000 7f2e11d14700 1 -- start start 2024-09-13T18:41:13.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.621+0000 7f2e11d14700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e0c0c7b00 0x7f2e0c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:13.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.621+0000 7f2e11d14700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2e0c0c65a0 0x7f2e0c159190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:13.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.621+0000 7f2e11d14700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0c156810 con 0x7f2e0c14fb50 2024-09-13T18:41:13.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.621+0000 7f2e11d14700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0c0c69f0 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.621+0000 7f2e11d14700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c1559b0 con 0x7f2e0c0c7b00 2024-09-13T18:41:13.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.621+0000 7f2e11513700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2e0c0c65a0 0x7f2e0c159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:13.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.621+0000 7f2e11513700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2e0c0c65a0 0x7f2e0c159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:36960/0 (socket says 172.21.15.146:36960) 2024-09-13T18:41:13.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.621+0000 7f2e11513700 1 -- 172.21.15.146:0/3607610741 learned_addr learned my addr 172.21.15.146:0/3607610741 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:41:13.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.621+0000 7f2e0bfff700 1 --2- 172.21.15.146:0/3607610741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e0c0c7b00 0x7f2e0c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:13.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.621+0000 7f2e10d12700 1 --1- 172.21.15.146:0/3607610741 >> v1:172.21.15.154:6789/0 conn(0x7f2e0c14fb50 0x7f2e0c0c7fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:35902/0 (socket says 172.21.15.146:35902) 2024-09-13T18:41:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11513700 1 -- 172.21.15.146:0/3607610741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e0c0c7b00 msgr2=0x7f2e0c0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:41:13.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11513700 1 --2- 172.21.15.146:0/3607610741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e0c0c7b00 0x7f2e0c0c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e0b7fe700 1 -- 172.21.15.146:0/3607610741 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340160202 0 0) 0x7f2e0c156810 con 0x7f2e0c14fb50 2024-09-13T18:41:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11513700 1 -- 172.21.15.146:0/3607610741 >> v1:172.21.15.154:6789/0 conn(0x7f2e0c14fb50 legacy=0x7f2e0c0c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11513700 1 -- 172.21.15.146:0/3607610741 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c155c90 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e0bfff700 1 --2- 172.21.15.146:0/3607610741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e0c0c7b00 0x7f2e0c0c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:41:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11513700 1 --2- 172.21.15.146:0/3607610741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e0c0c65a0 0x7f2e0c159190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400d150 tx=0x7f2e0400d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=923d6899b2492a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e0b7fe700 1 -- 172.21.15.146:0/3607610741 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04010070 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e0b7fe700 1 -- 172.21.15.146:0/3607610741 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0400dd10 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e0b7fe700 1 -- 172.21.15.146:0/3607610741 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0400b310 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11d14700 1 -- 172.21.15.146:0/3607610741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e0c0c65a0 msgr2=0x7f2e0c159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11d14700 1 --2- 172.21.15.146:0/3607610741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e0c0c65a0 0x7f2e0c159190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400d150 tx=0x7f2e0400d460 comp rx=0 tx=0).stop 2024-09-13T18:41:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11d14700 1 -- 172.21.15.146:0/3607610741 shutdown_connections 2024-09-13T18:41:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11d14700 1 --2- 172.21.15.146:0/3607610741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e0c0c65a0 0x7f2e0c159190 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11d14700 1 --2- 172.21.15.146:0/3607610741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e0c0c7b00 0x7f2e0c0c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11d14700 1 -- 172.21.15.146:0/3607610741 >> 172.21.15.146:0/3607610741 conn(0x7f2e0c066b00 msgr2=0x7f2e0c06aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:41:13.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11d14700 1 -- 172.21.15.146:0/3607610741 shutdown_connections 2024-09-13T18:41:13.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11d14700 1 -- 172.21.15.146:0/3607610741 wait complete. 2024-09-13T18:41:13.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11d14700 1 Processor -- start 2024-09-13T18:41:13.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.625+0000 7f2e11d14700 1 -- start start 2024-09-13T18:41:13.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e11d14700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e0c0c65a0 0x7f2e0c06ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:13.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e11d14700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e0c0c7b00 0x7f2e0c1f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:13.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e11d14700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e0c14fb50 0x7f2e0c141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:13.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e11d14700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c0c5a30 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e11d14700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c0c4a60 con 0x7f2e0c14fb50 2024-09-13T18:41:13.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e11d14700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c154430 con 0x7f2e0c0c7b00 2024-09-13T18:41:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e10d12700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e0c0c65a0 0x7f2e0c06ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e10d12700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e0c0c65a0 0x7f2e0c06ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59498/0 (socket says 172.21.15.146:59498) 2024-09-13T18:41:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e10d12700 1 -- 172.21.15.146:0/1662014121 learned_addr learned my addr 172.21.15.146:0/1662014121 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:41:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e11513700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e0c14fb50 0x7f2e0c141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e0bfff700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e0c0c7b00 0x7f2e0c1f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e10d12700 1 -- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e0c0c7b00 msgr2=0x7f2e0c1f1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e10d12700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e0c0c7b00 0x7f2e0c1f1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e10d12700 1 -- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e0c14fb50 msgr2=0x7f2e0c141de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e10d12700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e0c14fb50 0x7f2e0c141de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e10d12700 1 -- 172.21.15.146:0/1662014121 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e000094b0 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e10d12700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e0c0c65a0 0x7f2e0c06ac70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2df800c4d0 tx=0x7f2df800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:13.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e097fa700 1 -- 172.21.15.146:0/1662014121 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df800cea0 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e097fa700 1 -- 172.21.15.146:0/1662014121 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df8004ca0 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e11d14700 1 -- 172.21.15.146:0/1662014121 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0400bd40 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e097fa700 1 -- 172.21.15.146:0/1662014121 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8007310 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e11d14700 1 -- 172.21.15.146:0/1662014121 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0c0c4c70 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.629+0000 7f2e11d14700 1 -- 172.21.15.146:0/1662014121 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0c0c6060 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.633+0000 7f2e097fa700 1 -- 172.21.15.146:0/1662014121 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2df8007470 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.633+0000 7f2e097fa700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2df0066730 0x7f2df0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:41:13.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.633+0000 7f2e097fa700 1 -- 172.21.15.146:0/1662014121 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f2df8084370 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.633+0000 7f2e0bfff700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2df0066730 0x7f2df0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:41:13.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.633+0000 7f2e0bfff700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2df0066730 0x7f2df0068bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0c2150 tx=0x7f2e00009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:41:13.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.637+0000 7f2e097fa700 1 -- 172.21.15.146:0/1662014121 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2df8051070 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.869+0000 7f2e11d14700 1 -- 172.21.15.146:0/1662014121 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e0c0c6c30 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.873+0000 7f2e097fa700 1 -- 172.21.15.146:0/1662014121 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6290 (secure 0 0 0) 0x7f2df8007720 con 0x7f2e0c0c65a0 2024-09-13T18:41:13.889 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:41:13.889 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:41:06.615397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:41:13.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.873+0000 7f2e11d14700 1 -- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2df0066730 msgr2=0x7f2df0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:13.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.873+0000 7f2e11d14700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2df0066730 0x7f2df0068bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0c2150 tx=0x7f2e00009240 comp rx=0 tx=0).stop 2024-09-13T18:41:13.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.873+0000 7f2e11d14700 1 -- 172.21.15.146:0/1662014121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e0c0c65a0 msgr2=0x7f2e0c06ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:41:13.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.873+0000 7f2e11d14700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e0c0c65a0 0x7f2e0c06ac70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2df800c4d0 tx=0x7f2df800c7e0 comp rx=0 tx=0).stop 2024-09-13T18:41:13.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.873+0000 7f2e11d14700 1 -- 172.21.15.146:0/1662014121 shutdown_connections 2024-09-13T18:41:13.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.873+0000 7f2e11d14700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2df0066730 0x7f2df0068bf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:13.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.873+0000 7f2e11d14700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e0c14fb50 0x7f2e0c141de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:13.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.873+0000 7f2e11d14700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e0c0c7b00 0x7f2e0c1f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:13.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.873+0000 7f2e11d14700 1 --2- 172.21.15.146:0/1662014121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e0c0c65a0 0x7f2e0c06ac70 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:41:13.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.873+0000 7f2e11d14700 1 -- 172.21.15.146:0/1662014121 >> 172.21.15.146:0/1662014121 conn(0x7f2e0c066b00 msgr2=0x7f2e0c144b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:41:13.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.873+0000 7f2e11d14700 1 -- 172.21.15.146:0/1662014121 shutdown_connections 2024-09-13T18:41:13.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:41:13.873+0000 7f2e11d14700 1 -- 172.21.15.146:0/1662014121 wait complete. 2024-09-13T18:41:13.893 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 63 2024-09-13T18:41:13.905 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-09-13T18:41:13.905 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 106 secs before thrashing 2024-09-13T18:41:39.644 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:41:39.646 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:42:09.698 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:42:09.701 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:42:39.794 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:42:39.798 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:42:59.906 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:00.019 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc560d73700 1 Processor -- start 2024-09-13T18:43:00.019 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc560d73700 1 -- start start 2024-09-13T18:43:00.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc560d73700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc55c150230 0x7fc55c06ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:00.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc560d73700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc55c150610 0x7fc55c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:00.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc560d73700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55c071410 con 0x7fc55c071610 2024-09-13T18:43:00.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc560d73700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55c1596b0 con 0x7fc55c150610 2024-09-13T18:43:00.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc560d73700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55c156090 con 0x7fc55c150230 2024-09-13T18:43:00.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc553fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc55c150610 0x7fc55c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:00.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc553fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc55c150610 0x7fc55c06d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60254/0 (socket says 172.21.15.146:60254) 2024-09-13T18:43:00.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc553fff700 1 -- 172.21.15.146:0/561496162 learned_addr learned my addr 172.21.15.146:0/561496162 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:00.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc55b7fe700 1 --2- 172.21.15.146:0/561496162 >> v2:172.21.15.154:3300/0 conn(0x7fc55c150230 0x7fc55c06ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:00.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc553fff700 1 -- 172.21.15.146:0/561496162 >> v2:172.21.15.154:3300/0 conn(0x7fc55c150230 msgr2=0x7fc55c06ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:00.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc553fff700 1 --2- 172.21.15.146:0/561496162 >> v2:172.21.15.154:3300/0 conn(0x7fc55c150230 0x7fc55c06ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc553fff700 1 -- 172.21.15.146:0/561496162 >> v1:172.21.15.154:6789/0 conn(0x7fc55c071610 legacy=0x7fc55c159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:00.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc553fff700 1 -- 172.21.15.146:0/561496162 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c156350 con 0x7fc55c150610 2024-09-13T18:43:00.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc55b7fe700 1 --2- 172.21.15.146:0/561496162 >> v2:172.21.15.154:3300/0 conn(0x7fc55c150230 0x7fc55c06ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:43:00.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:42:59.999+0000 7fc553fff700 1 --2- 172.21.15.146:0/561496162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc55c150610 0x7fc55c06d5e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc55c06ca30 tx=0x7fc54c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8fa246f593045d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:00.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc55affd700 1 -- 172.21.15.146:0/561496162 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c00b8a0 con 0x7fc55c150610 2024-09-13T18:43:00.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc55affd700 1 -- 172.21.15.146:0/561496162 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c00ba00 con 0x7fc55c150610 2024-09-13T18:43:00.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc55affd700 1 -- 172.21.15.146:0/561496162 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c00bd90 con 0x7fc55c150610 2024-09-13T18:43:00.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 -- 172.21.15.146:0/561496162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc55c150610 msgr2=0x7fc55c06d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:00.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 --2- 172.21.15.146:0/561496162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc55c150610 0x7fc55c06d5e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc55c06ca30 tx=0x7fc54c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 -- 172.21.15.146:0/561496162 shutdown_connections 2024-09-13T18:43:00.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 --2- 172.21.15.146:0/561496162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc55c150610 0x7fc55c06d5e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 --2- 172.21.15.146:0/561496162 >> v2:172.21.15.154:3300/0 conn(0x7fc55c150230 0x7fc55c06ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 -- 172.21.15.146:0/561496162 >> 172.21.15.146:0/561496162 conn(0x7fc55c071ed0 msgr2=0x7fc55c0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:00.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 -- 172.21.15.146:0/561496162 shutdown_connections 2024-09-13T18:43:00.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 -- 172.21.15.146:0/561496162 wait complete. 2024-09-13T18:43:00.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 Processor -- start 2024-09-13T18:43:00.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 -- start start 2024-09-13T18:43:00.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc55c071610 0x7fc55c0cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:00.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc55c150230 0x7fc55c0cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:00.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc55c1623b0 0x7fc55c13db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:00.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c13f960 con 0x7fc55c150230 2024-09-13T18:43:00.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c13f6a0 con 0x7fc55c071610 2024-09-13T18:43:00.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc560d73700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c1eedf0 con 0x7fc55c1623b0 2024-09-13T18:43:00.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc553fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc55c150230 0x7fc55c0cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:00.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc553fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc55c150230 0x7fc55c0cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60262/0 (socket says 172.21.15.146:60262) 2024-09-13T18:43:00.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc553fff700 1 -- 172.21.15.146:0/3409204401 learned_addr learned my addr 172.21.15.146:0/3409204401 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:00.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc55b7fe700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc55c071610 0x7fc55c0cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:00.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.003+0000 7fc55bfff700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc55c1623b0 0x7fc55c13db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:00.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc553fff700 1 -- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc55c1623b0 msgr2=0x7fc55c13db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:00.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc553fff700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc55c1623b0 0x7fc55c13db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc553fff700 1 -- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc55c071610 msgr2=0x7fc55c0cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:00.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc553fff700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc55c071610 0x7fc55c0cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc553fff700 1 -- 172.21.15.146:0/3409204401 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c009570 con 0x7fc55c150230 2024-09-13T18:43:00.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc553fff700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc55c150230 0x7fc55c0cb640 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0c89c0 tx=0x7fc54c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:00.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc558ff9700 1 -- 172.21.15.146:0/3409204401 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c004440 con 0x7fc55c150230 2024-09-13T18:43:00.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc558ff9700 1 -- 172.21.15.146:0/3409204401 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c0045a0 con 0x7fc55c150230 2024-09-13T18:43:00.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc560d73700 1 -- 172.21.15.146:0/3409204401 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55c1ef0d0 con 0x7fc55c150230 2024-09-13T18:43:00.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc558ff9700 1 -- 172.21.15.146:0/3409204401 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c01ba70 con 0x7fc55c150230 2024-09-13T18:43:00.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc560d73700 1 -- 172.21.15.146:0/3409204401 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55c1ef690 con 0x7fc55c150230 2024-09-13T18:43:00.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc560d73700 1 -- 172.21.15.146:0/3409204401 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc540005310 con 0x7fc55c150230 2024-09-13T18:43:00.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc558ff9700 1 -- 172.21.15.146:0/3409204401 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc54c01bbd0 con 0x7fc55c150230 2024-09-13T18:43:00.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.007+0000 7fc558ff9700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc53c066760 0x7fc53c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:00.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.011+0000 7fc558ff9700 1 -- 172.21.15.146:0/3409204401 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7fc54c086000 con 0x7fc55c150230 2024-09-13T18:43:00.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.011+0000 7fc55b7fe700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc53c066760 0x7fc53c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:00.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.011+0000 7fc55b7fe700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc53c066760 0x7fc53c068c20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc5440099f0 tx=0x7fc544008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:00.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.019+0000 7fc558ff9700 1 -- 172.21.15.146:0/3409204401 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc54c052e20 con 0x7fc55c150230 2024-09-13T18:43:00.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc560d73700 1 -- 172.21.15.146:0/3409204401 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5400055c0 con 0x7fc55c150230 2024-09-13T18:43:00.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc558ff9700 1 -- 172.21.15.146:0/3409204401 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6290 (secure 0 0 0) 0x7fc54c056440 con 0x7fc55c150230 2024-09-13T18:43:00.276 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:00.276 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:41:06.615397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:00.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc560d73700 1 -- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc53c066760 msgr2=0x7fc53c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:00.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc560d73700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc53c066760 0x7fc53c068c20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc5440099f0 tx=0x7fc544008040 comp rx=0 tx=0).stop 2024-09-13T18:43:00.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc560d73700 1 -- 172.21.15.146:0/3409204401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc55c150230 msgr2=0x7fc55c0cb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:00.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc560d73700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc55c150230 0x7fc55c0cb640 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0c89c0 tx=0x7fc54c004770 comp rx=0 tx=0).stop 2024-09-13T18:43:00.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc560d73700 1 -- 172.21.15.146:0/3409204401 shutdown_connections 2024-09-13T18:43:00.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc560d73700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc53c066760 0x7fc53c068c20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc560d73700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc55c1623b0 0x7fc55c13db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc560d73700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc55c150230 0x7fc55c0cb640 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc560d73700 1 --2- 172.21.15.146:0/3409204401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc55c071610 0x7fc55c0cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc560d73700 1 -- 172.21.15.146:0/3409204401 >> 172.21.15.146:0/3409204401 conn(0x7fc55c071ed0 msgr2=0x7fc55c06d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:00.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc560d73700 1 -- 172.21.15.146:0/3409204401 shutdown_connections 2024-09-13T18:43:00.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.255+0000 7fc560d73700 1 -- 172.21.15.146:0/3409204401 wait complete. 2024-09-13T18:43:00.279 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 63 2024-09-13T18:43:00.290 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=1) 2024-09-13T18:43:00.291 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-09-13T18:43:00.291 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:43:00.302 INFO:tasks.ceph.mds.e:Stopped 2024-09-13T18:43:00.303 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:00.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.395+0000 7f065c843700 1 Processor -- start 2024-09-13T18:43:00.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.395+0000 7f065c843700 1 -- start start 2024-09-13T18:43:00.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.395+0000 7f065c843700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06500db370 0x7f065004bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:00.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.395+0000 7f065c843700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f065004c0d0 0x7f0650046ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:00.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.395+0000 7f065c843700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0650046a30 con 0x7f06500d5740 2024-09-13T18:43:00.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.395+0000 7f065c843700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06500d5b20 con 0x7f065004c0d0 2024-09-13T18:43:00.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.395+0000 7f065c843700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0650036d90 con 0x7f06500db370 2024-09-13T18:43:00.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.395+0000 7f065759e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f065004c0d0 0x7f0650046ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:00.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.395+0000 7f065759e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f065004c0d0 0x7f0650046ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60268/0 (socket says 172.21.15.146:60268) 2024-09-13T18:43:00.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.395+0000 7f065759e700 1 -- 172.21.15.146:0/3808240817 learned_addr learned my addr 172.21.15.146:0/3808240817 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:00.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065659c700 1 --2- 172.21.15.146:0/3808240817 >> v2:172.21.15.154:3300/0 conn(0x7f06500db370 0x7f065004bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:00.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065759e700 1 -- 172.21.15.146:0/3808240817 >> v2:172.21.15.154:3300/0 conn(0x7f06500db370 msgr2=0x7f065004bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:00.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065759e700 1 --2- 172.21.15.146:0/3808240817 >> v2:172.21.15.154:3300/0 conn(0x7f06500db370 0x7f065004bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065759e700 1 -- 172.21.15.146:0/3808240817 >> v1:172.21.15.146:6789/0 conn(0x7f06500d5740 legacy=0x7f0650045cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:00.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065759e700 1 -- 172.21.15.146:0/3808240817 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0650046400 con 0x7f065004c0d0 2024-09-13T18:43:00.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065759e700 1 --2- 172.21.15.146:0/3808240817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f065004c0d0 0x7f0650046ec0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f06580731e0 tx=0x7f0658061ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3ec1223cf93b8b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:00.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f0655d9b700 1 -- 172.21.15.146:0/3808240817 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658090070 con 0x7f065004c0d0 2024-09-13T18:43:00.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f0655d9b700 1 -- 172.21.15.146:0/3808240817 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065806f230 con 0x7f065004c0d0 2024-09-13T18:43:00.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f0655d9b700 1 -- 172.21.15.146:0/3808240817 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658075a30 con 0x7f065004c0d0 2024-09-13T18:43:00.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065c843700 1 -- 172.21.15.146:0/3808240817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f065004c0d0 msgr2=0x7f0650046ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:00.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065c843700 1 --2- 172.21.15.146:0/3808240817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f065004c0d0 0x7f0650046ec0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f06580731e0 tx=0x7f0658061ef0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065c843700 1 -- 172.21.15.146:0/3808240817 shutdown_connections 2024-09-13T18:43:00.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065c843700 1 --2- 172.21.15.146:0/3808240817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f065004c0d0 0x7f0650046ec0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065c843700 1 --2- 172.21.15.146:0/3808240817 >> v2:172.21.15.154:3300/0 conn(0x7f06500db370 0x7f065004bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065c843700 1 -- 172.21.15.146:0/3808240817 >> 172.21.15.146:0/3808240817 conn(0x7f0650050e10 msgr2=0x7f0650051220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065c843700 1 -- 172.21.15.146:0/3808240817 shutdown_connections 2024-09-13T18:43:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.399+0000 7f065c843700 1 -- 172.21.15.146:0/3808240817 wait complete. 2024-09-13T18:43:00.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f065c843700 1 Processor -- start 2024-09-13T18:43:00.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f065c843700 1 -- start start 2024-09-13T18:43:00.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f065c843700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f065004c0d0 0x7f06500d8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:00.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f065c843700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06500d5740 0x7f0650177cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:00.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f065c843700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06500db370 0x7f0650178210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:00.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f065c843700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0650048730 con 0x7f065004c0d0 2024-09-13T18:43:00.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f065c843700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065002cdf0 con 0x7f06500d5740 2024-09-13T18:43:00.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f065c843700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06500481b0 con 0x7f06500db370 2024-09-13T18:43:00.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f0656d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f065004c0d0 0x7f06500d8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:00.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f0656d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f065004c0d0 0x7f06500d8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60280/0 (socket says 172.21.15.146:60280) 2024-09-13T18:43:00.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f0656d9d700 1 -- 172.21.15.146:0/2882531505 learned_addr learned my addr 172.21.15.146:0/2882531505 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:00.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f065659c700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06500d5740 0x7f0650177cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:00.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f065759e700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06500db370 0x7f0650178210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:00.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f0656d9d700 1 -- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06500db370 msgr2=0x7f0650178210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:00.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f0656d9d700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06500db370 0x7f0650178210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f0656d9d700 1 -- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06500d5740 msgr2=0x7f0650177cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:00.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f0656d9d700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06500d5740 0x7f0650177cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f0656d9d700 1 -- 172.21.15.146:0/2882531505 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065804f5c0 con 0x7f065004c0d0 2024-09-13T18:43:00.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f0656d9d700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f065004c0d0 0x7f06500d8120 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f064800c500 tx=0x7f064800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:00.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f063f7fe700 1 -- 172.21.15.146:0/2882531505 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06480052d0 con 0x7f065004c0d0 2024-09-13T18:43:00.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f063f7fe700 1 -- 172.21.15.146:0/2882531505 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0648005430 con 0x7f065004c0d0 2024-09-13T18:43:00.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f065c843700 1 -- 172.21.15.146:0/2882531505 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0650048450 con 0x7f065004c0d0 2024-09-13T18:43:00.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f065c843700 1 -- 172.21.15.146:0/2882531505 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06500d7c00 con 0x7f065004c0d0 2024-09-13T18:43:00.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.403+0000 7f063f7fe700 1 -- 172.21.15.146:0/2882531505 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06480055a0 con 0x7f065004c0d0 2024-09-13T18:43:00.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.407+0000 7f065c843700 1 -- 172.21.15.146:0/2882531505 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0638005310 con 0x7f065004c0d0 2024-09-13T18:43:00.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.407+0000 7f063f7fe700 1 -- 172.21.15.146:0/2882531505 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f06480038e0 con 0x7f065004c0d0 2024-09-13T18:43:00.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.407+0000 7f063f7fe700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0644066730 0x7f0644068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:00.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.407+0000 7f063f7fe700 1 -- 172.21.15.146:0/2882531505 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f0648083cf0 con 0x7f065004c0d0 2024-09-13T18:43:00.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.407+0000 7f065659c700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0644066730 0x7f0644068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.411+0000 7f065659c700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0644066730 0x7f0644068bf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0650048b00 tx=0x7f064c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:00.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.415+0000 7f063f7fe700 1 -- 172.21.15.146:0/2882531505 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0648088070 con 0x7f065004c0d0 2024-09-13T18:43:00.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.627+0000 7f065c843700 1 -- 172.21.15.146:0/2882531505 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06380055c0 con 0x7f065004c0d0 2024-09-13T18:43:00.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.627+0000 7f063f7fe700 1 -- 172.21.15.146:0/2882531505 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6290 (secure 0 0 0) 0x7f06480509f0 con 0x7f065004c0d0 2024-09-13T18:43:00.648 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:00.648 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:41:06.615397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.627+0000 7f065c843700 1 -- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0644066730 msgr2=0x7f0644068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.627+0000 7f065c843700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0644066730 0x7f0644068bf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0650048b00 tx=0x7f064c008040 comp rx=0 tx=0).stop 2024-09-13T18:43:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.627+0000 7f065c843700 1 -- 172.21.15.146:0/2882531505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f065004c0d0 msgr2=0x7f06500d8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.627+0000 7f065c843700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f065004c0d0 0x7f06500d8120 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f064800c500 tx=0x7f064800c8c0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.627+0000 7f065c843700 1 -- 172.21.15.146:0/2882531505 shutdown_connections 2024-09-13T18:43:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.627+0000 7f065c843700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0644066730 0x7f0644068bf0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.627+0000 7f065c843700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06500db370 0x7f0650178210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.627+0000 7f065c843700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06500d5740 0x7f0650177cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.627+0000 7f065c843700 1 --2- 172.21.15.146:0/2882531505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f065004c0d0 0x7f06500d8120 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:00.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.627+0000 7f065c843700 1 -- 172.21.15.146:0/2882531505 >> 172.21.15.146:0/2882531505 conn(0x7f0650050e10 msgr2=0x7f065004fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:00.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.631+0000 7f065c843700 1 -- 172.21.15.146:0/2882531505 shutdown_connections 2024-09-13T18:43:00.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:00.631+0000 7f065c843700 1 -- 172.21.15.146:0/2882531505 wait complete. 2024-09-13T18:43:00.652 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 63 2024-09-13T18:43:00.670 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:43:02.671 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:02.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85359e700 1 Processor -- start 2024-09-13T18:43:02.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85359e700 1 -- start start 2024-09-13T18:43:02.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd84c04d7d0 0x7fd84c00d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:02.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85359e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84c0df180 con 0x7fd84c00c260 2024-09-13T18:43:02.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85359e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84c00bdd0 con 0x7fd84c0d89f0 2024-09-13T18:43:02.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd84c00d890 con 0x7fd84c04d7d0 2024-09-13T18:43:02.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd851d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd84c04d7d0 0x7fd84c00d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:02.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd852d9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd84c00c260 0x7fd84c00c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57316/0 (socket says 172.21.15.146:57316) 2024-09-13T18:43:02.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd852d9d700 1 -- 172.21.15.146:0/1822688251 learned_addr learned my addr 172.21.15.146:0/1822688251 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:43:02.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd851d9b700 1 -- 172.21.15.146:0/1822688251 >> v1:172.21.15.154:6789/0 conn(0x7fd84c0d89f0 legacy=0x7fd84c04e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:02.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd851d9b700 1 -- 172.21.15.146:0/1822688251 >> v1:172.21.15.146:6789/0 conn(0x7fd84c00c260 legacy=0x7fd84c00c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:02.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd851d9b700 1 -- 172.21.15.146:0/1822688251 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c0df6b0 con 0x7fd84c04d7d0 2024-09-13T18:43:02.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd851d9b700 1 --2- 172.21.15.146:0/1822688251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd84c04d7d0 0x7fd84c00d350 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd848009850 tx=0x7fd848009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be56efea01e8e812 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:02.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85159a700 1 -- 172.21.15.146:0/1822688251 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848004230 con 0x7fd84c04d7d0 2024-09-13T18:43:02.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85159a700 1 -- 172.21.15.146:0/1822688251 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd848004390 con 0x7fd84c04d7d0 2024-09-13T18:43:02.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85159a700 1 -- 172.21.15.146:0/1822688251 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480036c0 con 0x7fd84c04d7d0 2024-09-13T18:43:02.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85359e700 1 -- 172.21.15.146:0/1822688251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd84c04d7d0 msgr2=0x7fd84c00d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:02.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85359e700 1 --2- 172.21.15.146:0/1822688251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd84c04d7d0 0x7fd84c00d350 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd848009850 tx=0x7fd848009b60 comp rx=0 tx=0).stop 2024-09-13T18:43:02.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85359e700 1 -- 172.21.15.146:0/1822688251 shutdown_connections 2024-09-13T18:43:02.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85359e700 1 --2- 172.21.15.146:0/1822688251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd84c04d7d0 0x7fd84c00d350 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:02.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85359e700 1 -- 172.21.15.146:0/1822688251 >> 172.21.15.146:0/1822688251 conn(0x7fd84c00a130 msgr2=0x7fd84c00dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:02.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85359e700 1 -- 172.21.15.146:0/1822688251 shutdown_connections 2024-09-13T18:43:02.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.779+0000 7fd85359e700 1 -- 172.21.15.146:0/1822688251 wait complete. 2024-09-13T18:43:02.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd85359e700 1 Processor -- start 2024-09-13T18:43:02.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd85359e700 1 -- start start 2024-09-13T18:43:02.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd85359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd84c00c260 0x7fd84c031960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:02.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd85359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd84c0d89f0 0x7fd84c0da0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:02.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd85359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd84c1782c0 0x7fd84c0ddb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:02.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd85359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c178750 con 0x7fd84c1782c0 2024-09-13T18:43:02.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd85359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0db6f0 con 0x7fd84c00c260 2024-09-13T18:43:02.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd85359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c177870 con 0x7fd84c0d89f0 2024-09-13T18:43:02.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd852d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd84c1782c0 0x7fd84c0ddb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:02.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd852d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd84c1782c0 0x7fd84c0ddb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54706/0 (socket says 172.21.15.146:54706) 2024-09-13T18:43:02.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd852d9d700 1 -- 172.21.15.146:0/3440061309 learned_addr learned my addr 172.21.15.146:0/3440061309 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:02.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd85259c700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd84c00c260 0x7fd84c031960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:02.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd851d9b700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd84c0d89f0 0x7fd84c0da0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:02.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd852d9d700 1 -- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd84c0d89f0 msgr2=0x7fd84c0da0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:02.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd852d9d700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd84c0d89f0 0x7fd84c0da0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:02.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd852d9d700 1 -- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd84c00c260 msgr2=0x7fd84c031960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:02.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd852d9d700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd84c00c260 0x7fd84c031960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:02.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd852d9d700 1 -- 172.21.15.146:0/3440061309 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8480094b0 con 0x7fd84c1782c0 2024-09-13T18:43:02.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd851d9b700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd84c0d89f0 0x7fd84c0da0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:43:02.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd852d9d700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd84c1782c0 0x7fd84c0ddb10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd854068770 tx=0x7fd854075d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:02.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd842ffd700 1 -- 172.21.15.146:0/3440061309 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854067e50 con 0x7fd84c1782c0 2024-09-13T18:43:02.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd842ffd700 1 -- 172.21.15.146:0/3440061309 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd854075e30 con 0x7fd84c1782c0 2024-09-13T18:43:02.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd85359e700 1 -- 172.21.15.146:0/3440061309 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84c0db9b0 con 0x7fd84c1782c0 2024-09-13T18:43:02.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd842ffd700 1 -- 172.21.15.146:0/3440061309 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85406f280 con 0x7fd84c1782c0 2024-09-13T18:43:02.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.783+0000 7fd85359e700 1 -- 172.21.15.146:0/3440061309 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84c0dbf50 con 0x7fd84c1782c0 2024-09-13T18:43:02.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.787+0000 7fd85359e700 1 -- 172.21.15.146:0/3440061309 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84c000d10 con 0x7fd84c1782c0 2024-09-13T18:43:02.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.787+0000 7fd842ffd700 1 -- 172.21.15.146:0/3440061309 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd85406f5c0 con 0x7fd84c1782c0 2024-09-13T18:43:02.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.787+0000 7fd842ffd700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd834066760 0x7fd834068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:02.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.787+0000 7fd842ffd700 1 -- 172.21.15.146:0/3440061309 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7fd8540fef40 con 0x7fd84c1782c0 2024-09-13T18:43:02.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.787+0000 7fd85259c700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd834066760 0x7fd834068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:02.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.787+0000 7fd85259c700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd834066760 0x7fd834068c20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd844005150 tx=0x7fd844007ee0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:02.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:02.795+0000 7fd842ffd700 1 -- 172.21.15.146:0/3440061309 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd8540cbc40 con 0x7fd84c1782c0 2024-09-13T18:43:03.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.003+0000 7fd85359e700 1 -- 172.21.15.146:0/3440061309 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd84c0d8dd0 con 0x7fd84c1782c0 2024-09-13T18:43:03.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.003+0000 7fd842ffd700 1 -- 172.21.15.146:0/3440061309 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6290 (secure 0 0 0) 0x7fd8540cf260 con 0x7fd84c1782c0 2024-09-13T18:43:03.026 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:03.026 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:41:06.615397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:03.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.007+0000 7fd85359e700 1 -- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd834066760 msgr2=0x7fd834068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:03.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.007+0000 7fd85359e700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd834066760 0x7fd834068c20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd844005150 tx=0x7fd844007ee0 comp rx=0 tx=0).stop 2024-09-13T18:43:03.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.007+0000 7fd85359e700 1 -- 172.21.15.146:0/3440061309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd84c1782c0 msgr2=0x7fd84c0ddb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:03.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.007+0000 7fd85359e700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd84c1782c0 0x7fd84c0ddb10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd854068770 tx=0x7fd854075d10 comp rx=0 tx=0).stop 2024-09-13T18:43:03.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.007+0000 7fd85359e700 1 -- 172.21.15.146:0/3440061309 shutdown_connections 2024-09-13T18:43:03.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.007+0000 7fd85359e700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd834066760 0x7fd834068c20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:03.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.007+0000 7fd85359e700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd84c1782c0 0x7fd84c0ddb10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:03.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.007+0000 7fd85359e700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd84c0d89f0 0x7fd84c0da0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:03.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.007+0000 7fd85359e700 1 --2- 172.21.15.146:0/3440061309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd84c00c260 0x7fd84c031960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:03.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.007+0000 7fd85359e700 1 -- 172.21.15.146:0/3440061309 >> 172.21.15.146:0/3440061309 conn(0x7fd84c00a130 msgr2=0x7fd84c04fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:03.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.007+0000 7fd85359e700 1 -- 172.21.15.146:0/3440061309 shutdown_connections 2024-09-13T18:43:03.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:03.007+0000 7fd85359e700 1 -- 172.21.15.146:0/3440061309 wait complete. 2024-09-13T18:43:03.031 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 63 2024-09-13T18:43:03.045 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:43:05.046 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:05.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.143+0000 7f3c34ce4700 1 Processor -- start 2024-09-13T18:43:05.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c34ce4700 1 -- start start 2024-09-13T18:43:05.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c34ce4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c300725b0 0x7f3c30159f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:05.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c34ce4700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c300c2100 con 0x7f3c300c7630 2024-09-13T18:43:05.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c34ce4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c30153bd0 con 0x7f3c301537f0 2024-09-13T18:43:05.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c34ce4700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c300c30c0 con 0x7f3c300725b0 2024-09-13T18:43:05.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c2effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c300725b0 0x7f3c30159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:05.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c2ffff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f3c300c7630 0x7f3c300c7a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57328/0 (socket says 172.21.15.146:57328) 2024-09-13T18:43:05.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c2ffff700 1 -- 172.21.15.146:0/556417223 learned_addr learned my addr 172.21.15.146:0/556417223 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:43:05.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c2e7fc700 1 -- 172.21.15.146:0/556417223 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076730239 0 0) 0x7f3c300c2100 con 0x7f3c300c7630 2024-09-13T18:43:05.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c2e7fc700 1 -- 172.21.15.146:0/556417223 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c180035d0 con 0x7f3c300c7630 2024-09-13T18:43:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c2effd700 1 -- 172.21.15.146:0/556417223 >> v1:172.21.15.154:6789/0 conn(0x7f3c301537f0 legacy=0x7f3c300c19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c2effd700 1 -- 172.21.15.146:0/556417223 >> v1:172.21.15.146:6789/0 conn(0x7f3c300c7630 legacy=0x7f3c300c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c2effd700 1 -- 172.21.15.146:0/556417223 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c300c8240 con 0x7f3c300725b0 2024-09-13T18:43:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c2effd700 1 --2- 172.21.15.146:0/556417223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c300725b0 0x7f3c30159f10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009850 tx=0x7f3c24009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c4bba7aee63efa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c2e7fc700 1 -- 172.21.15.146:0/556417223 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24011830 con 0x7f3c300725b0 2024-09-13T18:43:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c2e7fc700 1 -- 172.21.15.146:0/556417223 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c24011990 con 0x7f3c300725b0 2024-09-13T18:43:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c2e7fc700 1 -- 172.21.15.146:0/556417223 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24011d40 con 0x7f3c300725b0 2024-09-13T18:43:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c34ce4700 1 -- 172.21.15.146:0/556417223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c300725b0 msgr2=0x7f3c30159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.147+0000 7f3c34ce4700 1 --2- 172.21.15.146:0/556417223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c300725b0 0x7f3c30159f10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009850 tx=0x7f3c24009b60 comp rx=0 tx=0).stop 2024-09-13T18:43:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c34ce4700 1 -- 172.21.15.146:0/556417223 shutdown_connections 2024-09-13T18:43:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c34ce4700 1 --2- 172.21.15.146:0/556417223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c300725b0 0x7f3c30159f10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c34ce4700 1 -- 172.21.15.146:0/556417223 >> 172.21.15.146:0/556417223 conn(0x7f3c30071a90 msgr2=0x7f3c30071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c34ce4700 1 -- 172.21.15.146:0/556417223 shutdown_connections 2024-09-13T18:43:05.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c34ce4700 1 -- 172.21.15.146:0/556417223 wait complete. 2024-09-13T18:43:05.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c34ce4700 1 Processor -- start 2024-09-13T18:43:05.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c34ce4700 1 -- start start 2024-09-13T18:43:05.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c34ce4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c300725b0 0x7f3c3014c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:05.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c34ce4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c300c7630 0x7f3c30151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:05.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c34ce4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c301537f0 0x7f3c3014aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:05.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c34ce4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c301e28c0 con 0x7f3c300725b0 2024-09-13T18:43:05.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c34ce4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3014cd00 con 0x7f3c301537f0 2024-09-13T18:43:05.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c34ce4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3006e040 con 0x7f3c300c7630 2024-09-13T18:43:05.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c2f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c300725b0 0x7f3c3014c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:05.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c2f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c300725b0 0x7f3c3014c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54722/0 (socket says 172.21.15.146:54722) 2024-09-13T18:43:05.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c2f7fe700 1 -- 172.21.15.146:0/508763509 learned_addr learned my addr 172.21.15.146:0/508763509 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:05.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c2effd700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c300c7630 0x7f3c30151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:05.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.151+0000 7f3c2ffff700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c301537f0 0x7f3c3014aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c2f7fe700 1 -- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c300c7630 msgr2=0x7f3c30151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c2f7fe700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c300c7630 0x7f3c30151100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c2f7fe700 1 -- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c301537f0 msgr2=0x7f3c3014aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c2f7fe700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c301537f0 0x7f3c3014aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c2f7fe700 1 -- 172.21.15.146:0/508763509 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c240094b0 con 0x7f3c300725b0 2024-09-13T18:43:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c2effd700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c300c7630 0x7f3c30151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:43:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c2f7fe700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c300725b0 0x7f3c3014c7c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c500 tx=0x7f3c2000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c17fff700 1 -- 172.21.15.146:0/508763509 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c200052d0 con 0x7f3c300725b0 2024-09-13T18:43:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c17fff700 1 -- 172.21.15.146:0/508763509 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c20005430 con 0x7f3c300725b0 2024-09-13T18:43:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c17fff700 1 -- 172.21.15.146:0/508763509 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c200055a0 con 0x7f3c300725b0 2024-09-13T18:43:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c34ce4700 1 -- 172.21.15.146:0/508763509 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c30157a10 con 0x7f3c300725b0 2024-09-13T18:43:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c34ce4700 1 -- 172.21.15.146:0/508763509 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3006e2e0 con 0x7f3c300725b0 2024-09-13T18:43:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c34ce4700 1 -- 172.21.15.146:0/508763509 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3006a8e0 con 0x7f3c300725b0 2024-09-13T18:43:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c17fff700 1 -- 172.21.15.146:0/508763509 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3c200038e0 con 0x7f3c300725b0 2024-09-13T18:43:05.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c17fff700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3c1c066730 0x7f3c1c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:05.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c17fff700 1 -- 172.21.15.146:0/508763509 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f3c20083ba0 con 0x7f3c300725b0 2024-09-13T18:43:05.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.155+0000 7f3c2effd700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3c1c066730 0x7f3c1c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:05.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.159+0000 7f3c2effd700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3c1c066730 0x7f3c1c068bf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3c240090b0 tx=0x7f3c24008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:05.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.163+0000 7f3c17fff700 1 -- 172.21.15.146:0/508763509 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3c200508a0 con 0x7f3c300725b0 2024-09-13T18:43:05.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.403+0000 7f3c34ce4700 1 -- 172.21.15.146:0/508763509 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c300c6e00 con 0x7f3c300725b0 2024-09-13T18:43:05.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.403+0000 7f3c17fff700 1 -- 172.21.15.146:0/508763509 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6290 (secure 0 0 0) 0x7f3c20053ec0 con 0x7f3c300725b0 2024-09-13T18:43:05.424 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:05.424 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:41:06.615397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:05.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.407+0000 7f3c34ce4700 1 -- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3c1c066730 msgr2=0x7f3c1c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:05.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.407+0000 7f3c34ce4700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3c1c066730 0x7f3c1c068bf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3c240090b0 tx=0x7f3c24008030 comp rx=0 tx=0).stop 2024-09-13T18:43:05.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.407+0000 7f3c34ce4700 1 -- 172.21.15.146:0/508763509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c300725b0 msgr2=0x7f3c3014c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:05.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.407+0000 7f3c34ce4700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c300725b0 0x7f3c3014c7c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c500 tx=0x7f3c2000c810 comp rx=0 tx=0).stop 2024-09-13T18:43:05.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.407+0000 7f3c34ce4700 1 -- 172.21.15.146:0/508763509 shutdown_connections 2024-09-13T18:43:05.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.407+0000 7f3c34ce4700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3c1c066730 0x7f3c1c068bf0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:05.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.407+0000 7f3c34ce4700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c301537f0 0x7f3c3014aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:05.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.407+0000 7f3c34ce4700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c300c7630 0x7f3c30151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:05.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.407+0000 7f3c34ce4700 1 --2- 172.21.15.146:0/508763509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c300725b0 0x7f3c3014c7c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:05.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.407+0000 7f3c34ce4700 1 -- 172.21.15.146:0/508763509 >> 172.21.15.146:0/508763509 conn(0x7f3c30071a90 msgr2=0x7f3c301570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:05.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.407+0000 7f3c34ce4700 1 -- 172.21.15.146:0/508763509 shutdown_connections 2024-09-13T18:43:05.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:05.407+0000 7f3c34ce4700 1 -- 172.21.15.146:0/508763509 wait complete. 2024-09-13T18:43:05.428 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 63 2024-09-13T18:43:05.439 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:43:07.441 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:08.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad6d3a6700 1 Processor -- start 2024-09-13T18:43:08.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad6d3a6700 1 -- start start 2024-09-13T18:43:08.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad6d3a6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fad600d8c60 0x7fad6004eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:08.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad6d3a6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fad6004f920 0x7fad60005de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:08.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad6d3a6700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad600505d0 con 0x7fad600073f0 2024-09-13T18:43:08.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad6d3a6700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad60049640 con 0x7fad600d8c60 2024-09-13T18:43:08.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad6d3a6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad600508b0 con 0x7fad6004f920 2024-09-13T18:43:08.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad67fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fad600d8c60 0x7fad6004eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:08.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad677fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fad600073f0 0x7fad6000e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57334/0 (socket says 172.21.15.146:57334) 2024-09-13T18:43:08.019 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad677fe700 1 -- 172.21.15.146:0/1670471452 learned_addr learned my addr 172.21.15.146:0/1670471452 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:43:08.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad6cba5700 1 --2- 172.21.15.146:0/1670471452 >> v2:172.21.15.154:3300/0 conn(0x7fad6004f920 0x7fad60005de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:08.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad66ffd700 1 -- 172.21.15.146:0/1670471452 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314223871 0 0) 0x7fad600505d0 con 0x7fad600073f0 2024-09-13T18:43:08.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad66ffd700 1 -- 172.21.15.146:0/1670471452 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad580035d0 con 0x7fad600073f0 2024-09-13T18:43:08.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad67fff700 1 -- 172.21.15.146:0/1670471452 >> v2:172.21.15.154:3300/0 conn(0x7fad6004f920 msgr2=0x7fad60005de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad67fff700 1 --2- 172.21.15.146:0/1670471452 >> v2:172.21.15.154:3300/0 conn(0x7fad6004f920 0x7fad60005de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad67fff700 1 -- 172.21.15.146:0/1670471452 >> v1:172.21.15.146:6789/0 conn(0x7fad600073f0 legacy=0x7fad6000e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad67fff700 1 -- 172.21.15.146:0/1670471452 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6004f120 con 0x7fad600d8c60 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.995+0000 7fad67fff700 1 --2- 172.21.15.146:0/1670471452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad600d8c60 0x7fad6004eab0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fad60051fe0 tx=0x7fad5c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5f46014061260 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.999+0000 7fad66ffd700 1 -- 172.21.15.146:0/1670471452 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c011830 con 0x7fad600d8c60 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.999+0000 7fad66ffd700 1 -- 172.21.15.146:0/1670471452 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5c011990 con 0x7fad600d8c60 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.999+0000 7fad66ffd700 1 -- 172.21.15.146:0/1670471452 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c011d20 con 0x7fad600d8c60 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.999+0000 7fad6d3a6700 1 -- 172.21.15.146:0/1670471452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad600d8c60 msgr2=0x7fad6004eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.999+0000 7fad6d3a6700 1 --2- 172.21.15.146:0/1670471452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad600d8c60 0x7fad6004eab0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fad60051fe0 tx=0x7fad5c009b30 comp rx=0 tx=0).stop 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.999+0000 7fad6d3a6700 1 -- 172.21.15.146:0/1670471452 shutdown_connections 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.999+0000 7fad6d3a6700 1 --2- 172.21.15.146:0/1670471452 >> v2:172.21.15.154:3300/0 conn(0x7fad6004f920 0x7fad60005de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.999+0000 7fad6d3a6700 1 --2- 172.21.15.146:0/1670471452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad600d8c60 0x7fad6004eab0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.999+0000 7fad6d3a6700 1 -- 172.21.15.146:0/1670471452 >> 172.21.15.146:0/1670471452 conn(0x7fad60054520 msgr2=0x7fad60054930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.999+0000 7fad6d3a6700 1 -- 172.21.15.146:0/1670471452 shutdown_connections 2024-09-13T18:43:08.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.999+0000 7fad6d3a6700 1 -- 172.21.15.146:0/1670471452 wait complete. 2024-09-13T18:43:08.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.999+0000 7fad6d3a6700 1 Processor -- start 2024-09-13T18:43:08.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:07.999+0000 7fad6d3a6700 1 -- start start 2024-09-13T18:43:08.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad6d3a6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad600073f0 0x7fad60177bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:08.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad6d3a6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6004f920 0x7fad6004c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:08.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad6d3a6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad600d8c60 0x7fad6004cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:08.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad6d3a6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad600e2a00 con 0x7fad600073f0 2024-09-13T18:43:08.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad6d3a6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad60053550 con 0x7fad6004f920 2024-09-13T18:43:08.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad6d3a6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad600e2440 con 0x7fad600d8c60 2024-09-13T18:43:08.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad67fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad600073f0 0x7fad60177bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:08.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad67fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad600073f0 0x7fad60177bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54742/0 (socket says 172.21.15.146:54742) 2024-09-13T18:43:08.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad67fff700 1 -- 172.21.15.146:0/3042614376 learned_addr learned my addr 172.21.15.146:0/3042614376 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:08.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad677fe700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6004f920 0x7fad6004c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:08.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad6cba5700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad600d8c60 0x7fad6004cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:08.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad67fff700 1 -- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad600d8c60 msgr2=0x7fad6004cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:08.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad67fff700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad600d8c60 0x7fad6004cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:08.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad67fff700 1 -- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6004f920 msgr2=0x7fad6004c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:08.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad67fff700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6004f920 0x7fad6004c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:08.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad67fff700 1 -- 172.21.15.146:0/3042614376 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c0094b0 con 0x7fad600073f0 2024-09-13T18:43:08.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad677fe700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6004f920 0x7fad6004c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:43:08.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad67fff700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad600073f0 0x7fad60177bb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fad5c014040 tx=0x7fad5c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:08.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad64ff9700 1 -- 172.21.15.146:0/3042614376 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0036c0 con 0x7fad600073f0 2024-09-13T18:43:08.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad64ff9700 1 -- 172.21.15.146:0/3042614376 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5c01e070 con 0x7fad600073f0 2024-09-13T18:43:08.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad6d3a6700 1 -- 172.21.15.146:0/3042614376 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad600e2720 con 0x7fad600073f0 2024-09-13T18:43:08.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad6d3a6700 1 -- 172.21.15.146:0/3042614376 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad600c8770 con 0x7fad600073f0 2024-09-13T18:43:08.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad64ff9700 1 -- 172.21.15.146:0/3042614376 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c003a40 con 0x7fad600073f0 2024-09-13T18:43:08.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad64ff9700 1 -- 172.21.15.146:0/3042614376 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad5c003ba0 con 0x7fad600073f0 2024-09-13T18:43:08.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.003+0000 7fad6d3a6700 1 -- 172.21.15.146:0/3042614376 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad48005310 con 0x7fad600073f0 2024-09-13T18:43:08.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.007+0000 7fad64ff9700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fad4c066780 0x7fad4c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:08.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.007+0000 7fad64ff9700 1 -- 172.21.15.146:0/3042614376 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7fad5c085670 con 0x7fad600073f0 2024-09-13T18:43:08.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.007+0000 7fad677fe700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fad4c066780 0x7fad4c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:08.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.007+0000 7fad677fe700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fad4c066780 0x7fad4c068c40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fad60006f30 tx=0x7fad54009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:08.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.011+0000 7fad64ff9700 1 -- 172.21.15.146:0/3042614376 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fad5c052520 con 0x7fad600073f0 2024-09-13T18:43:08.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.227+0000 7fad6d3a6700 1 -- 172.21.15.146:0/3042614376 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad480055c0 con 0x7fad600073f0 2024-09-13T18:43:08.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.231+0000 7fad64ff9700 1 -- 172.21.15.146:0/3042614376 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6290 (secure 0 0 0) 0x7fad5c055b40 con 0x7fad600073f0 2024-09-13T18:43:08.252 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:08.252 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:41:06.615397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:08.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.235+0000 7fad6d3a6700 1 -- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fad4c066780 msgr2=0x7fad4c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:08.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.235+0000 7fad6d3a6700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fad4c066780 0x7fad4c068c40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fad60006f30 tx=0x7fad54009040 comp rx=0 tx=0).stop 2024-09-13T18:43:08.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.235+0000 7fad6d3a6700 1 -- 172.21.15.146:0/3042614376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad600073f0 msgr2=0x7fad60177bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:08.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.235+0000 7fad6d3a6700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad600073f0 0x7fad60177bb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fad5c014040 tx=0x7fad5c004770 comp rx=0 tx=0).stop 2024-09-13T18:43:08.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.235+0000 7fad6d3a6700 1 -- 172.21.15.146:0/3042614376 shutdown_connections 2024-09-13T18:43:08.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.235+0000 7fad6d3a6700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fad4c066780 0x7fad4c068c40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:08.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.235+0000 7fad6d3a6700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad600d8c60 0x7fad6004cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:08.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.235+0000 7fad6d3a6700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6004f920 0x7fad6004c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:08.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.235+0000 7fad6d3a6700 1 --2- 172.21.15.146:0/3042614376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad600073f0 0x7fad60177bb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:08.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.235+0000 7fad6d3a6700 1 -- 172.21.15.146:0/3042614376 >> 172.21.15.146:0/3042614376 conn(0x7fad60054520 msgr2=0x7fad600ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:08.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.235+0000 7fad6d3a6700 1 -- 172.21.15.146:0/3042614376 shutdown_connections 2024-09-13T18:43:08.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:08.235+0000 7fad6d3a6700 1 -- 172.21.15.146:0/3042614376 wait complete. 2024-09-13T18:43:08.256 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 63 2024-09-13T18:43:08.270 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:43:09.909 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:43:09.912 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:43:10.271 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:10.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e49a03700 1 Processor -- start 2024-09-13T18:43:10.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e49a03700 1 -- start start 2024-09-13T18:43:10.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e49a03700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2e440cf550 0x7f2e441595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:10.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e49a03700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4415a540 con 0x7f2e440cb5b0 2024-09-13T18:43:10.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e49a03700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e440c5220 con 0x7f2e440cf170 2024-09-13T18:43:10.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e49a03700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e44159ca0 con 0x7f2e440cf550 2024-09-13T18:43:10.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e43fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2e440cf550 0x7f2e441595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:10.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e43fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2e440cf550 0x7f2e441595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54744/0 (socket says 172.21.15.146:54744) 2024-09-13T18:43:10.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e43fff700 1 -- 172.21.15.146:0/2135372434 learned_addr learned my addr 172.21.15.146:0/2135372434 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:10.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e43fff700 1 -- 172.21.15.146:0/2135372434 >> v1:172.21.15.154:6789/0 conn(0x7f2e440cf170 legacy=0x7f2e440c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:10.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e43fff700 1 -- 172.21.15.146:0/2135372434 >> v1:172.21.15.146:6789/0 conn(0x7f2e440cb5b0 legacy=0x7f2e441534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:10.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e43fff700 1 -- 172.21.15.146:0/2135372434 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e44159f80 con 0x7f2e440cf550 2024-09-13T18:43:10.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e43fff700 1 --2- 172.21.15.146:0/2135372434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440cf550 0x7f2e441595f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009910 tx=0x7f2e30009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8003584eb75e516 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:10.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e437fe700 1 -- 172.21.15.146:0/2135372434 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000ba60 con 0x7f2e440cf550 2024-09-13T18:43:10.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e437fe700 1 -- 172.21.15.146:0/2135372434 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3000bbc0 con 0x7f2e440cf550 2024-09-13T18:43:10.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.367+0000 7f2e437fe700 1 -- 172.21.15.146:0/2135372434 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30009e60 con 0x7f2e440cf550 2024-09-13T18:43:10.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 -- 172.21.15.146:0/2135372434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440cf550 msgr2=0x7f2e441595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:10.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 --2- 172.21.15.146:0/2135372434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440cf550 0x7f2e441595f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009910 tx=0x7f2e30009c20 comp rx=0 tx=0).stop 2024-09-13T18:43:10.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 -- 172.21.15.146:0/2135372434 shutdown_connections 2024-09-13T18:43:10.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 --2- 172.21.15.146:0/2135372434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440cf550 0x7f2e441595f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:10.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 -- 172.21.15.146:0/2135372434 >> 172.21.15.146:0/2135372434 conn(0x7f2e44072220 msgr2=0x7f2e44066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:10.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 -- 172.21.15.146:0/2135372434 shutdown_connections 2024-09-13T18:43:10.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 -- 172.21.15.146:0/2135372434 wait complete. 2024-09-13T18:43:10.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 Processor -- start 2024-09-13T18:43:10.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 -- start start 2024-09-13T18:43:10.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e440cb5b0 0x7f2e440c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:10.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440cf170 0x7f2e440b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:10.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e440cf550 0x7f2e440c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:10.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440b45f0 con 0x7f2e440cf170 2024-09-13T18:43:10.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440c7c80 con 0x7f2e440cb5b0 2024-09-13T18:43:10.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e49a03700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e44153f60 con 0x7f2e440cf550 2024-09-13T18:43:10.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.371+0000 7f2e43fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440cf170 0x7f2e440b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:10.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e43fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440cf170 0x7f2e440b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54752/0 (socket says 172.21.15.146:54752) 2024-09-13T18:43:10.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e48a01700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e440cb5b0 0x7f2e440c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:10.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e43fff700 1 -- 172.21.15.146:0/1156277942 learned_addr learned my addr 172.21.15.146:0/1156277942 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:10.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e49202700 1 --2- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e440cf550 0x7f2e440c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:10.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e48a01700 1 -- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e440cf550 msgr2=0x7f2e440c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:10.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e48a01700 1 --2- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e440cf550 0x7f2e440c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:10.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e48a01700 1 -- 172.21.15.146:0/1156277942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440cf170 msgr2=0x7f2e440b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:10.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e48a01700 1 --2- 172.21.15.146:0/1156277942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440cf170 0x7f2e440b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:10.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e48a01700 1 -- 172.21.15.146:0/1156277942 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e30009570 con 0x7f2e440cb5b0 2024-09-13T18:43:10.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e48a01700 1 --2- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e440cb5b0 0x7f2e440c6200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800c480 tx=0x7f2e3800c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:10.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e417fa700 1 -- 172.21.15.146:0/1156277942 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380052d0 con 0x7f2e440cb5b0 2024-09-13T18:43:10.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e49a03700 1 -- 172.21.15.146:0/1156277942 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e440c7ed0 con 0x7f2e440cb5b0 2024-09-13T18:43:10.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e417fa700 1 -- 172.21.15.146:0/1156277942 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e38005430 con 0x7f2e440cb5b0 2024-09-13T18:43:10.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e417fa700 1 -- 172.21.15.146:0/1156277942 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380055a0 con 0x7f2e440cb5b0 2024-09-13T18:43:10.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e49a03700 1 -- 172.21.15.146:0/1156277942 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e440c7960 con 0x7f2e440cb5b0 2024-09-13T18:43:10.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e49a03700 1 -- 172.21.15.146:0/1156277942 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e440b25a0 con 0x7f2e440cb5b0 2024-09-13T18:43:10.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.375+0000 7f2e417fa700 1 -- 172.21.15.146:0/1156277942 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2e380038e0 con 0x7f2e440cb5b0 2024-09-13T18:43:10.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.379+0000 7f2e417fa700 1 --2- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2e28066760 0x7f2e28068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:10.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.379+0000 7f2e417fa700 1 -- 172.21.15.146:0/1156277942 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f2e38083f70 con 0x7f2e440cb5b0 2024-09-13T18:43:10.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.379+0000 7f2e43fff700 1 --2- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2e28066760 0x7f2e28068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:10.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.379+0000 7f2e43fff700 1 --2- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2e28066760 0x7f2e28068c20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2e441e2c00 tx=0x7f2e30008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:10.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.387+0000 7f2e417fa700 1 -- 172.21.15.146:0/1156277942 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2e38050d00 con 0x7f2e440cb5b0 2024-09-13T18:43:10.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.615+0000 7f2e49a03700 1 -- 172.21.15.146:0/1156277942 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e44066850 con 0x7f2e440cb5b0 2024-09-13T18:43:10.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.615+0000 7f2e417fa700 1 -- 172.21.15.146:0/1156277942 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6290 (secure 0 0 0) 0x7f2e38017090 con 0x7f2e440cb5b0 2024-09-13T18:43:10.636 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:10.636 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:41:06.615397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:10.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.619+0000 7f2e49a03700 1 -- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2e28066760 msgr2=0x7f2e28068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:10.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.619+0000 7f2e49a03700 1 --2- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2e28066760 0x7f2e28068c20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2e441e2c00 tx=0x7f2e30008030 comp rx=0 tx=0).stop 2024-09-13T18:43:10.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.619+0000 7f2e49a03700 1 -- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e440cb5b0 msgr2=0x7f2e440c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:10.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.619+0000 7f2e49a03700 1 --2- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e440cb5b0 0x7f2e440c6200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800c480 tx=0x7f2e3800c790 comp rx=0 tx=0).stop 2024-09-13T18:43:10.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.619+0000 7f2e49a03700 1 -- 172.21.15.146:0/1156277942 shutdown_connections 2024-09-13T18:43:10.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.619+0000 7f2e49a03700 1 --2- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2e28066760 0x7f2e28068c20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:10.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.619+0000 7f2e49a03700 1 --2- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e440cf550 0x7f2e440c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:10.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.619+0000 7f2e49a03700 1 --2- 172.21.15.146:0/1156277942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440cf170 0x7f2e440b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:10.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.619+0000 7f2e49a03700 1 --2- 172.21.15.146:0/1156277942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e440cb5b0 0x7f2e440c6200 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:10.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.619+0000 7f2e49a03700 1 -- 172.21.15.146:0/1156277942 >> 172.21.15.146:0/1156277942 conn(0x7f2e44072220 msgr2=0x7f2e440c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:10.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.619+0000 7f2e49a03700 1 -- 172.21.15.146:0/1156277942 shutdown_connections 2024-09-13T18:43:10.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:10.619+0000 7f2e49a03700 1 -- 172.21.15.146:0/1156277942 wait complete. 2024-09-13T18:43:10.641 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 63 2024-09-13T18:43:10.653 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:43:12.654 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:12.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde71824700 1 Processor -- start 2024-09-13T18:43:12.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde71824700 1 -- start start 2024-09-13T18:43:12.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde71824700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fde6c0c5600 0x7fde6c0c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:12.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde71824700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c0c4890 0x7fde6c160af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:12.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde71824700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6c0c2c90 con 0x7fde6c153750 2024-09-13T18:43:12.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde71824700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6c06e480 con 0x7fde6c0c5600 2024-09-13T18:43:12.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde71824700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c06dee0 con 0x7fde6c0c4890 2024-09-13T18:43:12.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde70822700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fde6c153750 0x7fde6c06ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50934/0 (socket says 172.21.15.146:50934) 2024-09-13T18:43:12.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde70822700 1 -- 172.21.15.146:0/2309424524 learned_addr learned my addr 172.21.15.146:0/2309424524 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:43:12.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde6bfff700 1 --2- 172.21.15.146:0/2309424524 >> v2:172.21.15.146:3300/0 conn(0x7fde6c0c5600 0x7fde6c0c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:12.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde71023700 1 --2- 172.21.15.146:0/2309424524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c0c4890 0x7fde6c160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:12.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde6bfff700 1 -- 172.21.15.146:0/2309424524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c0c4890 msgr2=0x7fde6c160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:12.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde6bfff700 1 --2- 172.21.15.146:0/2309424524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c0c4890 0x7fde6c160af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:12.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde6bfff700 1 -- 172.21.15.146:0/2309424524 >> v1:172.21.15.146:6789/0 conn(0x7fde6c153750 legacy=0x7fde6c06ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:12.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde6bfff700 1 -- 172.21.15.146:0/2309424524 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c0c4cc0 con 0x7fde6c0c5600 2024-09-13T18:43:12.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.751+0000 7fde6b7fe700 1 -- 172.21.15.146:0/2309424524 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528621405 0 0) 0x7fde6c0c2c90 con 0x7fde6c153750 2024-09-13T18:43:12.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde71023700 1 --2- 172.21.15.146:0/2309424524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c0c4890 0x7fde6c160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:43:12.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde6bfff700 1 --2- 172.21.15.146:0/2309424524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c0c5600 0x7fde6c0c4350 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fde58009850 tx=0x7fde58009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35922887c6fe0990 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:12.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde6b7fe700 1 -- 172.21.15.146:0/2309424524 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58011830 con 0x7fde6c0c5600 2024-09-13T18:43:12.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde6b7fe700 1 -- 172.21.15.146:0/2309424524 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58011990 con 0x7fde6c0c5600 2024-09-13T18:43:12.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde6b7fe700 1 -- 172.21.15.146:0/2309424524 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58011d20 con 0x7fde6c0c5600 2024-09-13T18:43:12.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde71824700 1 -- 172.21.15.146:0/2309424524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c0c5600 msgr2=0x7fde6c0c4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:12.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde71824700 1 --2- 172.21.15.146:0/2309424524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c0c5600 0x7fde6c0c4350 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fde58009850 tx=0x7fde58009b60 comp rx=0 tx=0).stop 2024-09-13T18:43:12.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde71824700 1 -- 172.21.15.146:0/2309424524 shutdown_connections 2024-09-13T18:43:12.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde71824700 1 --2- 172.21.15.146:0/2309424524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c0c4890 0x7fde6c160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:12.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde71824700 1 --2- 172.21.15.146:0/2309424524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c0c5600 0x7fde6c0c4350 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:12.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde71824700 1 -- 172.21.15.146:0/2309424524 >> 172.21.15.146:0/2309424524 conn(0x7fde6c072380 msgr2=0x7fde6c066920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:12.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde71824700 1 -- 172.21.15.146:0/2309424524 shutdown_connections 2024-09-13T18:43:12.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde71824700 1 -- 172.21.15.146:0/2309424524 wait complete. 2024-09-13T18:43:12.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde71824700 1 Processor -- start 2024-09-13T18:43:12.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.755+0000 7fde71824700 1 -- start start 2024-09-13T18:43:12.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71824700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c0c4890 0x7fde6c14cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:12.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71824700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde6c0c5600 0x7fde6c14f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:12.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71824700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c153750 0x7fde6c14b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:12.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71824700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c14d080 con 0x7fde6c153750 2024-09-13T18:43:12.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71824700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c14dc90 con 0x7fde6c0c5600 2024-09-13T18:43:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71824700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c14c0a0 con 0x7fde6c0c4890 2024-09-13T18:43:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71023700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c153750 0x7fde6c14b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71023700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c153750 0x7fde6c14b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:41594/0 (socket says 172.21.15.146:41594) 2024-09-13T18:43:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71023700 1 -- 172.21.15.146:0/439488054 learned_addr learned my addr 172.21.15.146:0/439488054 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde6bfff700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde6c0c5600 0x7fde6c14f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde70822700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c0c4890 0x7fde6c14cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71023700 1 -- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c0c4890 msgr2=0x7fde6c14cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71023700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c0c4890 0x7fde6c14cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71023700 1 -- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde6c0c5600 msgr2=0x7fde6c14f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71023700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde6c0c5600 0x7fde6c14f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71023700 1 -- 172.21.15.146:0/439488054 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde580094b0 con 0x7fde6c153750 2024-09-13T18:43:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71023700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c153750 0x7fde6c14b3f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fde6400cef0 tx=0x7fde6400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:12.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde697fa700 1 -- 172.21.15.146:0/439488054 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6400ae50 con 0x7fde6c153750 2024-09-13T18:43:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde697fa700 1 -- 172.21.15.146:0/439488054 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde64015440 con 0x7fde6c153750 2024-09-13T18:43:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71824700 1 -- 172.21.15.146:0/439488054 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c1ed450 con 0x7fde6c153750 2024-09-13T18:43:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde697fa700 1 -- 172.21.15.146:0/439488054 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde640157f0 con 0x7fde6c153750 2024-09-13T18:43:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71824700 1 -- 172.21.15.146:0/439488054 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c14c620 con 0x7fde6c153750 2024-09-13T18:43:12.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde71824700 1 -- 172.21.15.146:0/439488054 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6c153b30 con 0x7fde6c153750 2024-09-13T18:43:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.759+0000 7fde697fa700 1 -- 172.21.15.146:0/439488054 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fde6400d430 con 0x7fde6c153750 2024-09-13T18:43:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.763+0000 7fde697fa700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde50066730 0x7fde50068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.763+0000 7fde697fa700 1 -- 172.21.15.146:0/439488054 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7fde64084e30 con 0x7fde6c153750 2024-09-13T18:43:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.763+0000 7fde70822700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde50066730 0x7fde50068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:12.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.763+0000 7fde70822700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde50066730 0x7fde50068bf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fde60000f60 tx=0x7fde60007cb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:12.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:12.767+0000 7fde697fa700 1 -- 172.21.15.146:0/439488054 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fde64051aa0 con 0x7fde6c153750 2024-09-13T18:43:13.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.003+0000 7fde71824700 1 -- 172.21.15.146:0/439488054 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde6c06dcc0 con 0x7fde6c153750 2024-09-13T18:43:13.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.007+0000 7fde697fa700 1 -- 172.21.15.146:0/439488054 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6290 (secure 0 0 0) 0x7fde6401a070 con 0x7fde6c153750 2024-09-13T18:43:13.027 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:13.027 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:41:06.615397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:13.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.007+0000 7fde71824700 1 -- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde50066730 msgr2=0x7fde50068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:13.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.007+0000 7fde71824700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde50066730 0x7fde50068bf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fde60000f60 tx=0x7fde60007cb0 comp rx=0 tx=0).stop 2024-09-13T18:43:13.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.007+0000 7fde71824700 1 -- 172.21.15.146:0/439488054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c153750 msgr2=0x7fde6c14b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:13.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.007+0000 7fde71824700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c153750 0x7fde6c14b3f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fde6400cef0 tx=0x7fde6400a340 comp rx=0 tx=0).stop 2024-09-13T18:43:13.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.007+0000 7fde71824700 1 -- 172.21.15.146:0/439488054 shutdown_connections 2024-09-13T18:43:13.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.007+0000 7fde71824700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde50066730 0x7fde50068bf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:13.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.007+0000 7fde71824700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c153750 0x7fde6c14b3f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:13.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.007+0000 7fde71824700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde6c0c5600 0x7fde6c14f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:13.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.007+0000 7fde71824700 1 --2- 172.21.15.146:0/439488054 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c0c4890 0x7fde6c14cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:13.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.007+0000 7fde71824700 1 -- 172.21.15.146:0/439488054 >> 172.21.15.146:0/439488054 conn(0x7fde6c072380 msgr2=0x7fde6c14e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:13.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.007+0000 7fde71824700 1 -- 172.21.15.146:0/439488054 shutdown_connections 2024-09-13T18:43:13.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:13.007+0000 7fde71824700 1 -- 172.21.15.146:0/439488054 wait complete. 2024-09-13T18:43:13.032 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 63 2024-09-13T18:43:13.045 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:43:15.046 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:15.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.151+0000 7f545ad7d700 1 Processor -- start 2024-09-13T18:43:15.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.151+0000 7f545ad7d700 1 -- start start 2024-09-13T18:43:15.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545ad7d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f54540c6b30 0x7f545415a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:15.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545ad7d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f545406dec0 0x7f545415a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:15.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545ad7d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54540c56d0 con 0x7f54540c6f10 2024-09-13T18:43:15.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545ad7d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54540712f0 con 0x7f545406dec0 2024-09-13T18:43:15.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545ad7d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545406eca0 con 0x7f54540c6b30 2024-09-13T18:43:15.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545a57c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f545406dec0 0x7f545415a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:15.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545a57c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f545406dec0 0x7f545415a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:41602/0 (socket says 172.21.15.146:41602) 2024-09-13T18:43:15.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545a57c700 1 -- 172.21.15.146:0/3031829335 learned_addr learned my addr 172.21.15.146:0/3031829335 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:15.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f5459d7b700 1 --2- 172.21.15.146:0/3031829335 >> v2:172.21.15.154:3300/0 conn(0x7f54540c6b30 0x7f545415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:15.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545a57c700 1 -- 172.21.15.146:0/3031829335 >> v2:172.21.15.154:3300/0 conn(0x7f54540c6b30 msgr2=0x7f545415a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:15.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545a57c700 1 --2- 172.21.15.146:0/3031829335 >> v2:172.21.15.154:3300/0 conn(0x7f54540c6b30 0x7f545415a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:15.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545a57c700 1 -- 172.21.15.146:0/3031829335 >> v1:172.21.15.154:6789/0 conn(0x7f54540c6f10 legacy=0x7f545406d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:15.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545a57c700 1 -- 172.21.15.146:0/3031829335 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5454071590 con 0x7f545406dec0 2024-09-13T18:43:15.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f5459d7b700 1 --2- 172.21.15.146:0/3031829335 >> v2:172.21.15.154:3300/0 conn(0x7f54540c6b30 0x7f545415a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:43:15.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545a57c700 1 --2- 172.21.15.146:0/3031829335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f545406dec0 0x7f545415a6a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f54540679e0 tx=0x7f544c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f90bee091866d33 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:15.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f5458d79700 1 -- 172.21.15.146:0/3031829335 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c00e070 con 0x7f545406dec0 2024-09-13T18:43:15.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f5458d79700 1 -- 172.21.15.146:0/3031829335 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c00b9c0 con 0x7f545406dec0 2024-09-13T18:43:15.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f5458d79700 1 -- 172.21.15.146:0/3031829335 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c00bd70 con 0x7f545406dec0 2024-09-13T18:43:15.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545ad7d700 1 -- 172.21.15.146:0/3031829335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f545406dec0 msgr2=0x7f545415a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:15.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545ad7d700 1 --2- 172.21.15.146:0/3031829335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f545406dec0 0x7f545415a6a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f54540679e0 tx=0x7f544c00b330 comp rx=0 tx=0).stop 2024-09-13T18:43:15.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545ad7d700 1 -- 172.21.15.146:0/3031829335 shutdown_connections 2024-09-13T18:43:15.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545ad7d700 1 --2- 172.21.15.146:0/3031829335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f545406dec0 0x7f545415a6a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:15.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545ad7d700 1 --2- 172.21.15.146:0/3031829335 >> v2:172.21.15.154:3300/0 conn(0x7f54540c6b30 0x7f545415a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:15.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545ad7d700 1 -- 172.21.15.146:0/3031829335 >> 172.21.15.146:0/3031829335 conn(0x7f54540c8980 msgr2=0x7f54540c8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:15.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545ad7d700 1 -- 172.21.15.146:0/3031829335 shutdown_connections 2024-09-13T18:43:15.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.155+0000 7f545ad7d700 1 -- 172.21.15.146:0/3031829335 wait complete. 2024-09-13T18:43:15.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f545ad7d700 1 Processor -- start 2024-09-13T18:43:15.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f545ad7d700 1 -- start start 2024-09-13T18:43:15.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f545ad7d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f545406dec0 0x7f54540bf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:15.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f545ad7d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54540c6b30 0x7f5454070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:15.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f545ad7d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54540c6f10 0x7f54540708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:15.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f545ad7d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545414a240 con 0x7f545406dec0 2024-09-13T18:43:15.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f545ad7d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54540c0780 con 0x7f54540c6b30 2024-09-13T18:43:15.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f545ad7d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54541f12f0 con 0x7f54540c6f10 2024-09-13T18:43:15.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f5459d7b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f545406dec0 0x7f54540bf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:15.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f5459d7b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f545406dec0 0x7f54540bf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:41614/0 (socket says 172.21.15.146:41614) 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f5459d7b700 1 -- 172.21.15.146:0/2628071325 learned_addr learned my addr 172.21.15.146:0/2628071325 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f545957a700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54540c6b30 0x7f5454070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f545a57c700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54540c6f10 0x7f54540708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f5459d7b700 1 -- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54540c6f10 msgr2=0x7f54540708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f5459d7b700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54540c6f10 0x7f54540708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f5459d7b700 1 -- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54540c6b30 msgr2=0x7f5454070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f5459d7b700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54540c6b30 0x7f5454070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f5459d7b700 1 -- 172.21.15.146:0/2628071325 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544c009b30 con 0x7f545406dec0 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f5459d7b700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f545406dec0 0x7f54540bf190 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f545000e880 tx=0x7f545000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f544a7fc700 1 -- 172.21.15.146:0/2628071325 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450013610 con 0x7f545406dec0 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f544a7fc700 1 -- 172.21.15.146:0/2628071325 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5450013770 con 0x7f545406dec0 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f545ad7d700 1 -- 172.21.15.146:0/2628071325 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5454149f60 con 0x7f545406dec0 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f544a7fc700 1 -- 172.21.15.146:0/2628071325 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450013b00 con 0x7f545406dec0 2024-09-13T18:43:15.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.159+0000 7f545ad7d700 1 -- 172.21.15.146:0/2628071325 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54540bf6d0 con 0x7f545406dec0 2024-09-13T18:43:15.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.163+0000 7f545ad7d700 1 -- 172.21.15.146:0/2628071325 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5454159b90 con 0x7f545406dec0 2024-09-13T18:43:15.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.163+0000 7f544a7fc700 1 -- 172.21.15.146:0/2628071325 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5450013c60 con 0x7f545406dec0 2024-09-13T18:43:15.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.163+0000 7f544a7fc700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5438066730 0x7f5438068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:15.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.163+0000 7f544a7fc700 1 -- 172.21.15.146:0/2628071325 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f54500843b0 con 0x7f545406dec0 2024-09-13T18:43:15.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.163+0000 7f545957a700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5438066730 0x7f5438068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:15.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.163+0000 7f545957a700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5438066730 0x7f5438068bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f545414adb0 tx=0x7f5444009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:15.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.171+0000 7f544a7fc700 1 -- 172.21.15.146:0/2628071325 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f54500510b0 con 0x7f545406dec0 2024-09-13T18:43:15.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f545ad7d700 1 -- 172.21.15.146:0/2628071325 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f545406f0f0 con 0x7f545406dec0 2024-09-13T18:43:15.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f544a7fc700 1 -- 172.21.15.146:0/2628071325 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6290 (secure 0 0 0) 0x7f54500546d0 con 0x7f545406dec0 2024-09-13T18:43:15.416 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:15.416 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:41:06.615397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":5784,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"e","rank":1,"incarnation":46,"state":"up:active","state_seq":29,"addr":"172.21.15.146:6833/3901685195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3901685195},{"type":"v1","addr":"172.21.15.146:6833","nonce":3901685195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:15.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f545ad7d700 1 -- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5438066730 msgr2=0x7f5438068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:15.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f545ad7d700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5438066730 0x7f5438068bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f545414adb0 tx=0x7f5444009040 comp rx=0 tx=0).stop 2024-09-13T18:43:15.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f545ad7d700 1 -- 172.21.15.146:0/2628071325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f545406dec0 msgr2=0x7f54540bf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:15.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f545ad7d700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f545406dec0 0x7f54540bf190 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f545000e880 tx=0x7f545000eb90 comp rx=0 tx=0).stop 2024-09-13T18:43:15.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f545ad7d700 1 -- 172.21.15.146:0/2628071325 shutdown_connections 2024-09-13T18:43:15.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f545ad7d700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5438066730 0x7f5438068bf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:15.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f545ad7d700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54540c6f10 0x7f54540708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:15.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f545ad7d700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54540c6b30 0x7f5454070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:15.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f545ad7d700 1 --2- 172.21.15.146:0/2628071325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f545406dec0 0x7f54540bf190 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:15.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f545ad7d700 1 -- 172.21.15.146:0/2628071325 >> 172.21.15.146:0/2628071325 conn(0x7f54540c8980 msgr2=0x7f54540c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:15.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f545ad7d700 1 -- 172.21.15.146:0/2628071325 shutdown_connections 2024-09-13T18:43:15.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:15.395+0000 7f545ad7d700 1 -- 172.21.15.146:0/2628071325 wait complete. 2024-09-13T18:43:15.420 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 63 2024-09-13T18:43:15.433 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:43:17.433 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:17.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8adf59e700 1 Processor -- start 2024-09-13T18:43:17.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8adf59e700 1 -- start start 2024-09-13T18:43:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8adf59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8ad80d8c60 0x7f8ad804eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8adf59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8ad80073f0 0x7f8ad8005de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8adf59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad80505d0 con 0x7f8ad804f920 2024-09-13T18:43:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8adf59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad8049640 con 0x7f8ad80d8c60 2024-09-13T18:43:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8adf59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad80508b0 con 0x7f8ad80073f0 2024-09-13T18:43:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8ade59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8ad80d8c60 0x7f8ad804eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8aded9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f8ad804f920 0x7f8ad800e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50950/0 (socket says 172.21.15.146:50950) 2024-09-13T18:43:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8aded9d700 1 -- 172.21.15.146:0/1587762437 learned_addr learned my addr 172.21.15.146:0/1587762437 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:43:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8addd9b700 1 --2- 172.21.15.146:0/1587762437 >> v2:172.21.15.154:3300/0 conn(0x7f8ad80073f0 0x7f8ad8005de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8ade59c700 1 -- 172.21.15.146:0/1587762437 >> v2:172.21.15.154:3300/0 conn(0x7f8ad80073f0 msgr2=0x7f8ad8005de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8ade59c700 1 --2- 172.21.15.146:0/1587762437 >> v2:172.21.15.154:3300/0 conn(0x7f8ad80073f0 0x7f8ad8005de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8ade59c700 1 -- 172.21.15.146:0/1587762437 >> v1:172.21.15.146:6789/0 conn(0x7f8ad804f920 legacy=0x7f8ad800e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:17.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8ade59c700 1 -- 172.21.15.146:0/1587762437 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad804f120 con 0x7f8ad80d8c60 2024-09-13T18:43:17.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8ade59c700 1 --2- 172.21.15.146:0/1587762437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ad80d8c60 0x7f8ad804eab0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0000b60 tx=0x7f8ae0061ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957958fbcd9f11b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:17.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8add59a700 1 -- 172.21.15.146:0/1587762437 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00739c0 con 0x7f8ad80d8c60 2024-09-13T18:43:17.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8add59a700 1 -- 172.21.15.146:0/1587762437 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae0075a30 con 0x7f8ad80d8c60 2024-09-13T18:43:17.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.547+0000 7f8add59a700 1 -- 172.21.15.146:0/1587762437 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0075de0 con 0x7f8ad80d8c60 2024-09-13T18:43:17.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 -- 172.21.15.146:0/1587762437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ad80d8c60 msgr2=0x7f8ad804eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:17.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 --2- 172.21.15.146:0/1587762437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ad80d8c60 0x7f8ad804eab0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0000b60 tx=0x7f8ae0061ef0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 -- 172.21.15.146:0/1587762437 shutdown_connections 2024-09-13T18:43:17.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 --2- 172.21.15.146:0/1587762437 >> v2:172.21.15.154:3300/0 conn(0x7f8ad80073f0 0x7f8ad8005de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 --2- 172.21.15.146:0/1587762437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ad80d8c60 0x7f8ad804eab0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 -- 172.21.15.146:0/1587762437 >> 172.21.15.146:0/1587762437 conn(0x7f8ad8054520 msgr2=0x7f8ad8054930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:17.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 -- 172.21.15.146:0/1587762437 shutdown_connections 2024-09-13T18:43:17.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 -- 172.21.15.146:0/1587762437 wait complete. 2024-09-13T18:43:17.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 Processor -- start 2024-09-13T18:43:17.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 -- start start 2024-09-13T18:43:17.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ad80073f0 0x7f8ad80d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ad804f920 0x7f8ad8053420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ad80d8c60 0x7f8ad8179ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad80d18e0 con 0x7f8ad80073f0 2024-09-13T18:43:17.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad80d3990 con 0x7f8ad804f920 2024-09-13T18:43:17.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8adf59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad817ce30 con 0x7f8ad80d8c60 2024-09-13T18:43:17.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8ade59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ad80073f0 0x7f8ad80d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8ade59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ad80073f0 0x7f8ad80d1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:41642/0 (socket says 172.21.15.146:41642) 2024-09-13T18:43:17.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8ade59c700 1 -- 172.21.15.146:0/3357520233 learned_addr learned my addr 172.21.15.146:0/3357520233 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:17.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8addd9b700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ad804f920 0x7f8ad8053420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.551+0000 7f8aded9d700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ad80d8c60 0x7f8ad8179ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8ade59c700 1 -- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ad80d8c60 msgr2=0x7f8ad8179ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:17.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8ade59c700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ad80d8c60 0x7f8ad8179ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8ade59c700 1 -- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ad804f920 msgr2=0x7f8ad8053420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:17.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8ade59c700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ad804f920 0x7f8ad8053420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8ade59c700 1 -- 172.21.15.146:0/3357520233 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae004f5c0 con 0x7f8ad80073f0 2024-09-13T18:43:17.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8ade59c700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ad80073f0 0x7f8ad80d1ba0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8ae002ce60 tx=0x7f8ae0068890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:17.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8aceffd700 1 -- 172.21.15.146:0/3357520233 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae009a070 con 0x7f8ad80073f0 2024-09-13T18:43:17.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8aceffd700 1 -- 172.21.15.146:0/3357520233 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae0067c50 con 0x7f8ad80073f0 2024-09-13T18:43:17.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8adf59e700 1 -- 172.21.15.146:0/3357520233 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad80d21e0 con 0x7f8ad80073f0 2024-09-13T18:43:17.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8aceffd700 1 -- 172.21.15.146:0/3357520233 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0067fc0 con 0x7f8ad80073f0 2024-09-13T18:43:17.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8adf59e700 1 -- 172.21.15.146:0/3357520233 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad817c350 con 0x7f8ad80073f0 2024-09-13T18:43:17.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8adf59e700 1 -- 172.21.15.146:0/3357520233 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad80d2ce0 con 0x7f8ad80073f0 2024-09-13T18:43:17.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8aceffd700 1 -- 172.21.15.146:0/3357520233 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8ae0073c10 con 0x7f8ad80073f0 2024-09-13T18:43:17.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8aceffd700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ac0066730 0x7f8ac0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8aceffd700 1 -- 172.21.15.146:0/3357520233 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f8ae0100150 con 0x7f8ad80073f0 2024-09-13T18:43:17.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.555+0000 7f8addd9b700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ac0066730 0x7f8ac0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.559+0000 7f8addd9b700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ac0066730 0x7f8ac0068bf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009a00 tx=0x7f8ad4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:17.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.563+0000 7f8aceffd700 1 -- 172.21.15.146:0/3357520233 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8ae00cce00 con 0x7f8ad80073f0 2024-09-13T18:43:17.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.783+0000 7f8adf59e700 1 -- 172.21.15.146:0/3357520233 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ad804e560 con 0x7f8ad80073f0 2024-09-13T18:43:17.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.783+0000 7f8aceffd700 1 -- 172.21.15.146:0/3357520233 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5529 (secure 0 0 0) 0x7f8ae00d0420 con 0x7f8ad80073f0 2024-09-13T18:43:17.804 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:17.804 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:15.769420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:17.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.787+0000 7f8adf59e700 1 -- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ac0066730 msgr2=0x7f8ac0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:17.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.787+0000 7f8adf59e700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ac0066730 0x7f8ac0068bf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009a00 tx=0x7f8ad4008040 comp rx=0 tx=0).stop 2024-09-13T18:43:17.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.787+0000 7f8adf59e700 1 -- 172.21.15.146:0/3357520233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ad80073f0 msgr2=0x7f8ad80d1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:17.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.787+0000 7f8adf59e700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ad80073f0 0x7f8ad80d1ba0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8ae002ce60 tx=0x7f8ae0068890 comp rx=0 tx=0).stop 2024-09-13T18:43:17.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.787+0000 7f8adf59e700 1 -- 172.21.15.146:0/3357520233 shutdown_connections 2024-09-13T18:43:17.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.787+0000 7f8adf59e700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ac0066730 0x7f8ac0068bf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.787+0000 7f8adf59e700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ad80d8c60 0x7f8ad8179ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.787+0000 7f8adf59e700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ad804f920 0x7f8ad8053420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.787+0000 7f8adf59e700 1 --2- 172.21.15.146:0/3357520233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ad80073f0 0x7f8ad80d1ba0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.787+0000 7f8adf59e700 1 -- 172.21.15.146:0/3357520233 >> 172.21.15.146:0/3357520233 conn(0x7f8ad8054520 msgr2=0x7f8ad80cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:17.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.787+0000 7f8adf59e700 1 -- 172.21.15.146:0/3357520233 shutdown_connections 2024-09-13T18:43:17.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.787+0000 7f8adf59e700 1 -- 172.21.15.146:0/3357520233 wait complete. 2024-09-13T18:43:17.809 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 64 2024-09-13T18:43:17.823 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2024-09-13T18:43:17.823 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:43:17.823 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:17.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9fa6669700 1 Processor -- start 2024-09-13T18:43:17.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9fa6669700 1 -- start start 2024-09-13T18:43:17.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9fa6669700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9fa0157fd0 0x7f9fa01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9fa6669700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa0158d10 con 0x7f9fa006db40 2024-09-13T18:43:17.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9fa6669700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa0066500 con 0x7f9fa0151590 2024-09-13T18:43:17.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9fa6669700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa00c69b0 con 0x7f9fa0157fd0 2024-09-13T18:43:17.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9fa5e68700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9fa006db40 0x7f9fa0161c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50956/0 (socket says 172.21.15.146:50956) 2024-09-13T18:43:17.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9fa5e68700 1 -- 172.21.15.146:0/1263529509 learned_addr learned my addr 172.21.15.146:0/1263529509 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:43:17.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9fa4e66700 1 --2- 172.21.15.146:0/1263529509 >> v2:172.21.15.154:3300/0 conn(0x7f9fa0157fd0 0x7f9fa01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131058957 0 0) 0x7f9fa0158d10 con 0x7f9fa006db40 2024-09-13T18:43:17.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f8c0035d0 con 0x7f9fa006db40 2024-09-13T18:43:17.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1324766537 0 0) 0x7f9f8c0035d0 con 0x7f9fa006db40 2024-09-13T18:43:17.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 >> v2:172.21.15.154:3300/0 conn(0x7f9fa0157fd0 msgr2=0x7f9fa01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:17.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 --2- 172.21.15.146:0/1263529509 >> v2:172.21.15.154:3300/0 conn(0x7f9fa0157fd0 0x7f9fa01587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 >> v1:172.21.15.154:6789/0 conn(0x7f9fa0151590 legacy=0x7f9fa00c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:17.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa006f740 con 0x7f9fa006db40 2024-09-13T18:43:17.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f9f980019f0 con 0x7f9fa006db40 2024-09-13T18:43:17.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 >> v1:172.21.15.146:6789/0 conn(0x7f9fa006db40 legacy=0x7f9fa0161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:17.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 --2- 172.21.15.146:0/1263529509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f8c003a90 0x7f9f8c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 --2- 172.21.15.146:0/1263529509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f8c004870 0x7f9f8c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 --2- 172.21.15.146:0/1263529509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f8c005230 0x7f9f8c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00c69b0 con 0x7f9f8c003a90 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c00c020 con 0x7f9f8c004870 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c00c1e0 con 0x7f9f8c005230 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9fa5667700 1 --2- 172.21.15.146:0/1263529509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f8c003a90 0x7f9f8c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.915+0000 7f9fa4e66700 1 --2- 172.21.15.146:0/1263529509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f8c004870 0x7f9f8c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa5e68700 1 --2- 172.21.15.146:0/1263529509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f8c005230 0x7f9f8c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa5667700 1 -- 172.21.15.146:0/1263529509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f8c005230 msgr2=0x7f9f8c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa5667700 1 --2- 172.21.15.146:0/1263529509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f8c005230 0x7f9f8c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa5667700 1 -- 172.21.15.146:0/1263529509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f8c004870 msgr2=0x7f9f8c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa5667700 1 --2- 172.21.15.146:0/1263529509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f8c004870 0x7f9f8c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa5667700 1 -- 172.21.15.146:0/1263529509 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c0035d0 con 0x7f9f8c003a90 2024-09-13T18:43:17.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa5e68700 1 --2- 172.21.15.146:0/1263529509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f8c005230 0x7f9f8c00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:43:17.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa5e68700 1 -- 172.21.15.146:0/1263529509 reap_dead start 2024-09-13T18:43:17.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa5667700 1 --2- 172.21.15.146:0/1263529509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f8c003a90 0x7f9f8c003e70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c007fd0 tx=0x7f9f8c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8998dd06ff77a50 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:17.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c001390 con 0x7f9f8c003a90 2024-09-13T18:43:17.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9c0052d0 con 0x7f9f8c003a90 2024-09-13T18:43:17.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9f97fff700 1 -- 172.21.15.146:0/1263529509 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0036a0 con 0x7f9f8c003a90 2024-09-13T18:43:17.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 -- 172.21.15.146:0/1263529509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f8c003a90 msgr2=0x7f9f8c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:17.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 --2- 172.21.15.146:0/1263529509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f8c003a90 0x7f9f8c003e70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c007fd0 tx=0x7f9f8c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 -- 172.21.15.146:0/1263529509 shutdown_connections 2024-09-13T18:43:17.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 --2- 172.21.15.146:0/1263529509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f8c003a90 0x7f9f8c003e70 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 -- 172.21.15.146:0/1263529509 >> 172.21.15.146:0/1263529509 conn(0x7f9fa0066b00 msgr2=0x7f9fa006aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:17.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 -- 172.21.15.146:0/1263529509 shutdown_connections 2024-09-13T18:43:17.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 -- 172.21.15.146:0/1263529509 wait complete. 2024-09-13T18:43:17.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 Processor -- start 2024-09-13T18:43:17.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 -- start start 2024-09-13T18:43:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f8c003a90 0x7f9fa014a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fa0150af0 0x7f9fa01ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fa0145de0 0x7f9fa01ed520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa01f1270 con 0x7f9fa0150af0 2024-09-13T18:43:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa01eda60 con 0x7f9fa0145de0 2024-09-13T18:43:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.919+0000 7f9fa6669700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00c21b0 con 0x7f9f8c003a90 2024-09-13T18:43:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa4e66700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fa0150af0 0x7f9fa01ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa4e66700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fa0150af0 0x7f9fa01ecfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:41660/0 (socket says 172.21.15.146:41660) 2024-09-13T18:43:17.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa4e66700 1 -- 172.21.15.146:0/2089180608 learned_addr learned my addr 172.21.15.146:0/2089180608 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa5667700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f8c003a90 0x7f9fa014a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa5e68700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fa0145de0 0x7f9fa01ed520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa4e66700 1 -- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f8c003a90 msgr2=0x7f9fa014a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa4e66700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f8c003a90 0x7f9fa014a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa4e66700 1 -- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fa0145de0 msgr2=0x7f9fa01ed520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa4e66700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fa0145de0 0x7f9fa01ed520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa4e66700 1 -- 172.21.15.146:0/2089180608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa01f05f0 con 0x7f9fa0150af0 2024-09-13T18:43:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa4e66700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fa0150af0 0x7f9fa01ecfe0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9f90011ad0 tx=0x7f9f90011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9f95ffb700 1 -- 172.21.15.146:0/2089180608 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9000f940 con 0x7f9fa0150af0 2024-09-13T18:43:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9f95ffb700 1 -- 172.21.15.146:0/2089180608 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9000faa0 con 0x7f9fa0150af0 2024-09-13T18:43:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa6669700 1 -- 172.21.15.146:0/2089180608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa01f08d0 con 0x7f9fa0150af0 2024-09-13T18:43:17.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9f95ffb700 1 -- 172.21.15.146:0/2089180608 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9000fe50 con 0x7f9fa0150af0 2024-09-13T18:43:17.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa6669700 1 -- 172.21.15.146:0/2089180608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa01f0e90 con 0x7f9fa0150af0 2024-09-13T18:43:17.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa6669700 1 -- 172.21.15.146:0/2089180608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa00ca910 con 0x7f9fa0150af0 2024-09-13T18:43:17.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9f95ffb700 1 -- 172.21.15.146:0/2089180608 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9f90014440 con 0x7f9fa0150af0 2024-09-13T18:43:17.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9f95ffb700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9f84066760 0x7f9f84068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:17.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9f95ffb700 1 -- 172.21.15.146:0/2089180608 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f9f90084de0 con 0x7f9fa0150af0 2024-09-13T18:43:17.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.923+0000 7f9fa5667700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9f84066760 0x7f9f84068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:17.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.927+0000 7f9fa5667700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9f84066760 0x7f9f84068c20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c001050 tx=0x7f9f9c005000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:17.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:17.931+0000 7f9f95ffb700 1 -- 172.21.15.146:0/2089180608 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9f90051a00 con 0x7f9fa0150af0 2024-09-13T18:43:18.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.159+0000 7f9fa6669700 1 -- 172.21.15.146:0/2089180608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fa006d780 con 0x7f9fa0150af0 2024-09-13T18:43:18.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.159+0000 7f9f95ffb700 1 -- 172.21.15.146:0/2089180608 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5529 (secure 0 0 0) 0x7f9f90022070 con 0x7f9fa0150af0 2024-09-13T18:43:18.181 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:18.181 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:15.769420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:18.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.163+0000 7f9fa6669700 1 -- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9f84066760 msgr2=0x7f9f84068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:18.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.163+0000 7f9fa6669700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9f84066760 0x7f9f84068c20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c001050 tx=0x7f9f9c005000 comp rx=0 tx=0).stop 2024-09-13T18:43:18.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.163+0000 7f9fa6669700 1 -- 172.21.15.146:0/2089180608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fa0150af0 msgr2=0x7f9fa01ecfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:18.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.163+0000 7f9fa6669700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fa0150af0 0x7f9fa01ecfe0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9f90011ad0 tx=0x7f9f90011de0 comp rx=0 tx=0).stop 2024-09-13T18:43:18.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.163+0000 7f9fa6669700 1 -- 172.21.15.146:0/2089180608 shutdown_connections 2024-09-13T18:43:18.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.163+0000 7f9fa6669700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9f84066760 0x7f9f84068c20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:18.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.163+0000 7f9fa6669700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fa0145de0 0x7f9fa01ed520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:18.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.163+0000 7f9fa6669700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fa0150af0 0x7f9fa01ecfe0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:18.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.163+0000 7f9fa6669700 1 --2- 172.21.15.146:0/2089180608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f8c003a90 0x7f9fa014a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:18.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.163+0000 7f9fa6669700 1 -- 172.21.15.146:0/2089180608 >> 172.21.15.146:0/2089180608 conn(0x7f9fa0066b00 msgr2=0x7f9fa00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:18.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.163+0000 7f9fa6669700 1 -- 172.21.15.146:0/2089180608 shutdown_connections 2024-09-13T18:43:18.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:18.163+0000 7f9fa6669700 1 -- 172.21.15.146:0/2089180608 wait complete. 2024-09-13T18:43:18.185 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 64 2024-09-13T18:43:18.197 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 64, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 64, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5800": { "addr": "172.21.15.154:6839/776638146", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 776638146, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 776638146, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5800, "incarnation": 52, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 30 }, "gid_6063": { "addr": "172.21.15.146:6835/832794630", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 832794630, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 832794630, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6063, "incarnation": 58, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 43 }, "gid_6207": { "addr": "172.21.15.146:6837/3234109877", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3234109877, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3234109877, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6207, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 36, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:43:15.769420+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5800, "mds_1": 6207, "mds_2": 6063 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:43:20.199 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:20.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.303+0000 7fabb32d0700 1 Processor -- start 2024-09-13T18:43:20.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.303+0000 7fabb32d0700 1 -- start start 2024-09-13T18:43:20.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fabac150230 0x7fabac06ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:20.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fabac150610 0x7fabac06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:20.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabac071610 0x7fabac06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:20.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac1596b0 con 0x7fabac150610 2024-09-13T18:43:20.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac071410 con 0x7fabac150230 2024-09-13T18:43:20.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac156090 con 0x7fabac071610 2024-09-13T18:43:20.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb1acd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fabac150610 0x7fabac06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:20.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb1acd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fabac150610 0x7fabac06d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:41664/0 (socket says 172.21.15.146:41664) 2024-09-13T18:43:20.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb1acd700 1 -- 172.21.15.146:0/240507392 learned_addr learned my addr 172.21.15.146:0/240507392 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:20.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb22ce700 1 --2- 172.21.15.146:0/240507392 >> v2:172.21.15.154:3300/0 conn(0x7fabac150230 0x7fabac06ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:20.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb2acf700 1 --2- 172.21.15.146:0/240507392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabac071610 0x7fabac06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:20.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb1acd700 1 -- 172.21.15.146:0/240507392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabac071610 msgr2=0x7fabac06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:20.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb1acd700 1 --2- 172.21.15.146:0/240507392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabac071610 0x7fabac06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:20.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb1acd700 1 -- 172.21.15.146:0/240507392 >> v2:172.21.15.154:3300/0 conn(0x7fabac150230 msgr2=0x7fabac06ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:20.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb1acd700 1 --2- 172.21.15.146:0/240507392 >> v2:172.21.15.154:3300/0 conn(0x7fabac150230 0x7fabac06ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:20.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb1acd700 1 -- 172.21.15.146:0/240507392 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac156350 con 0x7fabac150610 2024-09-13T18:43:20.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb1acd700 1 --2- 172.21.15.146:0/240507392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac150610 0x7fabac06d5e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fabac06ca30 tx=0x7fab9c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a6a026e62159db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:20.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb12cc700 1 -- 172.21.15.146:0/240507392 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00b8a0 con 0x7fabac150610 2024-09-13T18:43:20.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb12cc700 1 -- 172.21.15.146:0/240507392 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c00ba00 con 0x7fabac150610 2024-09-13T18:43:20.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb12cc700 1 -- 172.21.15.146:0/240507392 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00bd90 con 0x7fabac150610 2024-09-13T18:43:20.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 -- 172.21.15.146:0/240507392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac150610 msgr2=0x7fabac06d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:20.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 --2- 172.21.15.146:0/240507392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac150610 0x7fabac06d5e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fabac06ca30 tx=0x7fab9c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:43:20.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 -- 172.21.15.146:0/240507392 shutdown_connections 2024-09-13T18:43:20.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 --2- 172.21.15.146:0/240507392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabac071610 0x7fabac06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:20.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 --2- 172.21.15.146:0/240507392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac150610 0x7fabac06d5e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:20.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 --2- 172.21.15.146:0/240507392 >> v2:172.21.15.154:3300/0 conn(0x7fabac150230 0x7fabac06ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:20.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 -- 172.21.15.146:0/240507392 >> 172.21.15.146:0/240507392 conn(0x7fabac071ed0 msgr2=0x7fabac0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:20.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 -- 172.21.15.146:0/240507392 shutdown_connections 2024-09-13T18:43:20.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 -- 172.21.15.146:0/240507392 wait complete. 2024-09-13T18:43:20.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 Processor -- start 2024-09-13T18:43:20.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.307+0000 7fabb32d0700 1 -- start start 2024-09-13T18:43:20.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb32d0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabac071610 0x7fabac1eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:20.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb32d0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac150230 0x7fabac1eb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:20.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb32d0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabac0cb480 0x7fabac158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:20.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb32d0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac14d6a0 con 0x7fabac150230 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb32d0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac1eb560 con 0x7fabac0cb480 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb32d0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0c3320 con 0x7fabac071610 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb1acd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac150230 0x7fabac1eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb1acd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac150230 0x7fabac1eb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:41676/0 (socket says 172.21.15.146:41676) 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb1acd700 1 -- 172.21.15.146:0/3435067544 learned_addr learned my addr 172.21.15.146:0/3435067544 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb22ce700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabac071610 0x7fabac1eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb2acf700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabac0cb480 0x7fabac158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb1acd700 1 -- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabac071610 msgr2=0x7fabac1eaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb1acd700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabac071610 0x7fabac1eaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb1acd700 1 -- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabac0cb480 msgr2=0x7fabac158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb1acd700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabac0cb480 0x7fabac158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb1acd700 1 -- 172.21.15.146:0/3435067544 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c009570 con 0x7fabac150230 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb1acd700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac150230 0x7fabac1eb020 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fab9c000c00 tx=0x7fab9c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7faba2ffd700 1 -- 172.21.15.146:0/3435067544 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c011c50 con 0x7fabac150230 2024-09-13T18:43:20.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7faba2ffd700 1 -- 172.21.15.146:0/3435067544 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c01f070 con 0x7fabac150230 2024-09-13T18:43:20.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7faba2ffd700 1 -- 172.21.15.146:0/3435067544 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00be60 con 0x7fabac150230 2024-09-13T18:43:20.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb32d0700 1 -- 172.21.15.146:0/3435067544 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabac155d10 con 0x7fabac150230 2024-09-13T18:43:20.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb32d0700 1 -- 172.21.15.146:0/3435067544 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac155990 con 0x7fabac150230 2024-09-13T18:43:20.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.311+0000 7fabb32d0700 1 -- 172.21.15.146:0/3435067544 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabac14d7e0 con 0x7fabac150230 2024-09-13T18:43:20.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.315+0000 7faba2ffd700 1 -- 172.21.15.146:0/3435067544 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fab9c01a430 con 0x7fabac150230 2024-09-13T18:43:20.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.315+0000 7faba2ffd700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab90066730 0x7fab90068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:20.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.315+0000 7faba2ffd700 1 -- 172.21.15.146:0/3435067544 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fab9c021070 con 0x7fabac150230 2024-09-13T18:43:20.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.315+0000 7fabb22ce700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab90066730 0x7fab90068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:20.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.315+0000 7fabb22ce700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab90066730 0x7fab90068bf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7faba8006b40 tx=0x7faba8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:20.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.323+0000 7faba2ffd700 1 -- 172.21.15.146:0/3435067544 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fabac14d7e0 con 0x7fabac150230 2024-09-13T18:43:20.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.559+0000 7fabb32d0700 1 -- 172.21.15.146:0/3435067544 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabac14d7e0 con 0x7fabac150230 2024-09-13T18:43:20.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.559+0000 7faba2ffd700 1 -- 172.21.15.146:0/3435067544 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5529 (secure 0 0 0) 0x7fabac14d7e0 con 0x7fabac150230 2024-09-13T18:43:20.581 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:20.581 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:15.769420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:20.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.563+0000 7fabb32d0700 1 -- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab90066730 msgr2=0x7fab90068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:20.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.563+0000 7fabb32d0700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab90066730 0x7fab90068bf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7faba8006b40 tx=0x7faba8008040 comp rx=0 tx=0).stop 2024-09-13T18:43:20.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.563+0000 7fabb32d0700 1 -- 172.21.15.146:0/3435067544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac150230 msgr2=0x7fabac1eb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:20.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.563+0000 7fabb32d0700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac150230 0x7fabac1eb020 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fab9c000c00 tx=0x7fab9c004770 comp rx=0 tx=0).stop 2024-09-13T18:43:20.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.563+0000 7fabb32d0700 1 -- 172.21.15.146:0/3435067544 shutdown_connections 2024-09-13T18:43:20.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.563+0000 7fabb32d0700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab90066730 0x7fab90068bf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:20.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.563+0000 7fabb32d0700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabac0cb480 0x7fabac158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:20.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.563+0000 7fabb32d0700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac150230 0x7fabac1eb020 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:20.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.563+0000 7fabb32d0700 1 --2- 172.21.15.146:0/3435067544 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabac071610 0x7fabac1eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:20.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.563+0000 7fabb32d0700 1 -- 172.21.15.146:0/3435067544 >> 172.21.15.146:0/3435067544 conn(0x7fabac071ed0 msgr2=0x7fabac0c23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:20.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.563+0000 7fabb32d0700 1 -- 172.21.15.146:0/3435067544 shutdown_connections 2024-09-13T18:43:20.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:20.563+0000 7fabb32d0700 1 -- 172.21.15.146:0/3435067544 wait complete. 2024-09-13T18:43:20.586 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 64 2024-09-13T18:43:20.596 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:43:22.597 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:22.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.695+0000 7f84fb59e700 1 Processor -- start 2024-09-13T18:43:22.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.695+0000 7f84fb59e700 1 -- start start 2024-09-13T18:43:22.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fb59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f84fc06e7c0 0x7f84fc0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:22.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fb59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f84fc06eba0 0x7f84fc0c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:22.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fb59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84fc0c1020 con 0x7f84fc1598f0 2024-09-13T18:43:22.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fb59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc1593f0 con 0x7f84fc06eba0 2024-09-13T18:43:22.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fb59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc0c6a70 con 0x7f84fc06e7c0 2024-09-13T18:43:22.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fad9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f84fc1598f0 0x7f84fc159d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:55742/0 (socket says 172.21.15.146:55742) 2024-09-13T18:43:22.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fad9d700 1 -- 172.21.15.146:0/1495388709 learned_addr learned my addr 172.21.15.146:0/1495388709 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:43:22.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84f9d9b700 1 --2- 172.21.15.146:0/1495388709 >> v2:172.21.15.146:3300/0 conn(0x7f84fc06eba0 0x7f84fc0c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:22.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fa59c700 1 --2- 172.21.15.146:0/1495388709 >> v2:172.21.15.154:3300/0 conn(0x7f84fc06e7c0 0x7f84fc0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:22.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84f9d9b700 1 -- 172.21.15.146:0/1495388709 >> v2:172.21.15.154:3300/0 conn(0x7f84fc06e7c0 msgr2=0x7f84fc0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:22.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84f9d9b700 1 --2- 172.21.15.146:0/1495388709 >> v2:172.21.15.154:3300/0 conn(0x7f84fc06e7c0 0x7f84fc0b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:22.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84f9d9b700 1 -- 172.21.15.146:0/1495388709 >> v1:172.21.15.146:6789/0 conn(0x7f84fc1598f0 legacy=0x7f84fc159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:22.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84f959a700 1 -- 172.21.15.146:0/1495388709 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507367842 0 0) 0x7f84fc0c1020 con 0x7f84fc1598f0 2024-09-13T18:43:22.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84f9d9b700 1 -- 172.21.15.146:0/1495388709 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc0c1290 con 0x7f84fc06eba0 2024-09-13T18:43:22.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84f9d9b700 1 --2- 172.21.15.146:0/1495388709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc06eba0 0x7f84fc0c2ce0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0c9ad0 tx=0x7f84e4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74c0312218f09e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:22.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84f959a700 1 -- 172.21.15.146:0/1495388709 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4011830 con 0x7f84fc06eba0 2024-09-13T18:43:22.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84f959a700 1 -- 172.21.15.146:0/1495388709 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e4011990 con 0x7f84fc06eba0 2024-09-13T18:43:22.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84f959a700 1 -- 172.21.15.146:0/1495388709 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4011d20 con 0x7f84fc06eba0 2024-09-13T18:43:22.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fb59e700 1 -- 172.21.15.146:0/1495388709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc06eba0 msgr2=0x7f84fc0c2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:22.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fb59e700 1 --2- 172.21.15.146:0/1495388709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc06eba0 0x7f84fc0c2ce0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0c9ad0 tx=0x7f84e4009b30 comp rx=0 tx=0).stop 2024-09-13T18:43:22.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fb59e700 1 -- 172.21.15.146:0/1495388709 shutdown_connections 2024-09-13T18:43:22.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fb59e700 1 --2- 172.21.15.146:0/1495388709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc06eba0 0x7f84fc0c2ce0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:22.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fb59e700 1 --2- 172.21.15.146:0/1495388709 >> v2:172.21.15.154:3300/0 conn(0x7f84fc06e7c0 0x7f84fc0b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:22.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fb59e700 1 -- 172.21.15.146:0/1495388709 >> 172.21.15.146:0/1495388709 conn(0x7f84fc071ed0 msgr2=0x7f84fc0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:22.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fb59e700 1 -- 172.21.15.146:0/1495388709 shutdown_connections 2024-09-13T18:43:22.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.699+0000 7f84fb59e700 1 -- 172.21.15.146:0/1495388709 wait complete. 2024-09-13T18:43:22.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fb59e700 1 Processor -- start 2024-09-13T18:43:22.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fb59e700 1 -- start start 2024-09-13T18:43:22.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fb59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84fc06e7c0 0x7f84fc1e31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:22.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fb59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84fc06eba0 0x7f84fc1e5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:22.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fb59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc1598f0 0x7f84fc1e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:22.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fb59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc14c4e0 con 0x7f84fc1598f0 2024-09-13T18:43:22.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fb59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc06f7c0 con 0x7f84fc06eba0 2024-09-13T18:43:22.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fb59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc1e3700 con 0x7f84fc06e7c0 2024-09-13T18:43:22.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc1598f0 0x7f84fc1e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:22.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc1598f0 0x7f84fc1e41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60952/0 (socket says 172.21.15.146:60952) 2024-09-13T18:43:22.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fad9d700 1 -- 172.21.15.146:0/3682304308 learned_addr learned my addr 172.21.15.146:0/3682304308 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:22.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fa59c700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84fc06e7c0 0x7f84fc1e31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:22.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84f9d9b700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84fc06eba0 0x7f84fc1e5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:22.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fad9d700 1 -- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84fc06e7c0 msgr2=0x7f84fc1e31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:22.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fad9d700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84fc06e7c0 0x7f84fc1e31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:22.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fad9d700 1 -- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84fc06eba0 msgr2=0x7f84fc1e5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:22.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fad9d700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84fc06eba0 0x7f84fc1e5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:22.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fad9d700 1 -- 172.21.15.146:0/3682304308 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e40094b0 con 0x7f84fc1598f0 2024-09-13T18:43:22.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fad9d700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc1598f0 0x7f84fc1e41f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f84f400cca0 tx=0x7f84f400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:22.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84f2ffd700 1 -- 172.21.15.146:0/3682304308 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f400ab70 con 0x7f84fc1598f0 2024-09-13T18:43:22.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84f2ffd700 1 -- 172.21.15.146:0/3682304308 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f400acd0 con 0x7f84fc1598f0 2024-09-13T18:43:22.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fb59e700 1 -- 172.21.15.146:0/3682304308 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc1f20c0 con 0x7f84fc1598f0 2024-09-13T18:43:22.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84f2ffd700 1 -- 172.21.15.146:0/3682304308 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4007310 con 0x7f84fc1598f0 2024-09-13T18:43:22.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.703+0000 7f84fb59e700 1 -- 172.21.15.146:0/3682304308 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc1f2680 con 0x7f84fc1598f0 2024-09-13T18:43:22.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.707+0000 7f84f2ffd700 1 -- 172.21.15.146:0/3682304308 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f84f4007470 con 0x7f84fc1598f0 2024-09-13T18:43:22.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.707+0000 7f84fb59e700 1 -- 172.21.15.146:0/3682304308 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84fc1e3ed0 con 0x7f84fc1598f0 2024-09-13T18:43:22.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.707+0000 7f84f2ffd700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f84dc066730 0x7f84dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:22.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.707+0000 7f84f2ffd700 1 -- 172.21.15.146:0/3682304308 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f84f4084640 con 0x7f84fc1598f0 2024-09-13T18:43:22.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.707+0000 7f84fa59c700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f84dc066730 0x7f84dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:22.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.707+0000 7f84fa59c700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f84dc066730 0x7f84dc068bf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009b50 tx=0x7f84ec009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:22.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.715+0000 7f84f2ffd700 1 -- 172.21.15.146:0/3682304308 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f84f40512f0 con 0x7f84fc1598f0 2024-09-13T18:43:22.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.951+0000 7f84fb59e700 1 -- 172.21.15.146:0/3682304308 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84fc0b11a0 con 0x7f84fc1598f0 2024-09-13T18:43:22.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.951+0000 7f84f2ffd700 1 -- 172.21.15.146:0/3682304308 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5531 (secure 0 0 0) 0x7f84f4054910 con 0x7f84fc1598f0 2024-09-13T18:43:22.974 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:22.974 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:21.347313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:resolve","state_seq":33,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:22.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.955+0000 7f84fb59e700 1 -- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f84dc066730 msgr2=0x7f84dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:22.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.955+0000 7f84fb59e700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f84dc066730 0x7f84dc068bf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009b50 tx=0x7f84ec009240 comp rx=0 tx=0).stop 2024-09-13T18:43:22.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.955+0000 7f84fb59e700 1 -- 172.21.15.146:0/3682304308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc1598f0 msgr2=0x7f84fc1e41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:22.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.955+0000 7f84fb59e700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc1598f0 0x7f84fc1e41f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f84f400cca0 tx=0x7f84f400a060 comp rx=0 tx=0).stop 2024-09-13T18:43:22.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.955+0000 7f84fb59e700 1 -- 172.21.15.146:0/3682304308 shutdown_connections 2024-09-13T18:43:22.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.955+0000 7f84fb59e700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f84dc066730 0x7f84dc068bf0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:22.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.955+0000 7f84fb59e700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc1598f0 0x7f84fc1e41f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:22.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.955+0000 7f84fb59e700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84fc06eba0 0x7f84fc1e5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:22.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.955+0000 7f84fb59e700 1 --2- 172.21.15.146:0/3682304308 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84fc06e7c0 0x7f84fc1e31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:22.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.955+0000 7f84fb59e700 1 -- 172.21.15.146:0/3682304308 >> 172.21.15.146:0/3682304308 conn(0x7f84fc071ed0 msgr2=0x7f84fc0702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:22.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.955+0000 7f84fb59e700 1 -- 172.21.15.146:0/3682304308 shutdown_connections 2024-09-13T18:43:22.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:22.955+0000 7f84fb59e700 1 -- 172.21.15.146:0/3682304308 wait complete. 2024-09-13T18:43:22.978 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 65 2024-09-13T18:43:22.988 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:43:24.989 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:25.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.095+0000 7fc3ff59e700 1 Processor -- start 2024-09-13T18:43:25.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.095+0000 7fc3ff59e700 1 -- start start 2024-09-13T18:43:25.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3ff59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc4000c2460 0x7fc40006cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:25.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3ff59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4000c6070 0x7fc4000c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:25.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3ff59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc40006e130 con 0x7fc40014fa50 2024-09-13T18:43:25.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3ff59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4000c6c50 con 0x7fc4000c2460 2024-09-13T18:43:25.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3ff59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4000c65d0 con 0x7fc4000c6070 2024-09-13T18:43:25.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3f5d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc4000c2460 0x7fc40006cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:25.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3fe59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc40014fa50 0x7fc4000c7070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:55752/0 (socket says 172.21.15.146:55752) 2024-09-13T18:43:25.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3fe59c700 1 -- 172.21.15.146:0/2926929569 learned_addr learned my addr 172.21.15.146:0/2926929569 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:43:25.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3fed9d700 1 --2- 172.21.15.146:0/2926929569 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4000c6070 0x7fc4000c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:25.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3f5d9b700 1 -- 172.21.15.146:0/2926929569 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4000c6070 msgr2=0x7fc4000c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:25.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3f5d9b700 1 --2- 172.21.15.146:0/2926929569 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4000c6070 0x7fc4000c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:25.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3f5d9b700 1 -- 172.21.15.146:0/2926929569 >> v1:172.21.15.146:6789/0 conn(0x7fc40014fa50 legacy=0x7fc4000c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:25.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3f5d9b700 1 -- 172.21.15.146:0/2926929569 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40006e3f0 con 0x7fc4000c2460 2024-09-13T18:43:25.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3f5d9b700 1 --2- 172.21.15.146:0/2926929569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4000c2460 0x7fc40006cfe0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009880 tx=0x7fc3e8009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f41a9c243049d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:25.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3fdd9b700 1 -- 172.21.15.146:0/2926929569 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8011830 con 0x7fc4000c2460 2024-09-13T18:43:25.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3fdd9b700 1 -- 172.21.15.146:0/2926929569 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e8011990 con 0x7fc4000c2460 2024-09-13T18:43:25.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3fdd9b700 1 -- 172.21.15.146:0/2926929569 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8011d20 con 0x7fc4000c2460 2024-09-13T18:43:25.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3ff59e700 1 -- 172.21.15.146:0/2926929569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4000c2460 msgr2=0x7fc40006cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:25.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3ff59e700 1 --2- 172.21.15.146:0/2926929569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4000c2460 0x7fc40006cfe0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009880 tx=0x7fc3e8009b90 comp rx=0 tx=0).stop 2024-09-13T18:43:25.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3ff59e700 1 -- 172.21.15.146:0/2926929569 shutdown_connections 2024-09-13T18:43:25.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3ff59e700 1 --2- 172.21.15.146:0/2926929569 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4000c6070 0x7fc4000c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:25.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3ff59e700 1 --2- 172.21.15.146:0/2926929569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4000c2460 0x7fc40006cfe0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:25.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3ff59e700 1 -- 172.21.15.146:0/2926929569 >> 172.21.15.146:0/2926929569 conn(0x7fc400071080 msgr2=0x7fc400071490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:25.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3ff59e700 1 -- 172.21.15.146:0/2926929569 shutdown_connections 2024-09-13T18:43:25.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.099+0000 7fc3ff59e700 1 -- 172.21.15.146:0/2926929569 wait complete. 2024-09-13T18:43:25.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3ff59e700 1 Processor -- start 2024-09-13T18:43:25.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3ff59e700 1 -- start start 2024-09-13T18:43:25.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3ff59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4000c2460 0x7fc400152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:25.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3ff59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4000c6070 0x7fc4000c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:25.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3ff59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc40014fa50 0x7fc4001501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:25.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3ff59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc400149860 con 0x7fc4000c2460 2024-09-13T18:43:25.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3ff59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4001522e0 con 0x7fc4000c6070 2024-09-13T18:43:25.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3ff59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4000c9540 con 0x7fc40014fa50 2024-09-13T18:43:25.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3fe59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4000c2460 0x7fc400152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:25.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3fe59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4000c2460 0x7fc400152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60960/0 (socket says 172.21.15.146:60960) 2024-09-13T18:43:25.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3fe59c700 1 -- 172.21.15.146:0/3730560242 learned_addr learned my addr 172.21.15.146:0/3730560242 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:25.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3f5d9b700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4000c6070 0x7fc4000c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:25.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3fed9d700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc40014fa50 0x7fc4001501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:25.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3fe59c700 1 -- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc40014fa50 msgr2=0x7fc4001501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:25.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3fe59c700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc40014fa50 0x7fc4001501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:25.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3fe59c700 1 -- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4000c6070 msgr2=0x7fc4000c4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:25.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3fe59c700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4000c6070 0x7fc4000c4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:25.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3fe59c700 1 -- 172.21.15.146:0/3730560242 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e80094b0 con 0x7fc4000c2460 2024-09-13T18:43:25.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.103+0000 7fc3fe59c700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4000c2460 0x7fc400152770 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000c490 tx=0x7fc3f000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:25.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.107+0000 7fc3f77fe700 1 -- 172.21.15.146:0/3730560242 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0005d90 con 0x7fc4000c2460 2024-09-13T18:43:25.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.107+0000 7fc3ff59e700 1 -- 172.21.15.146:0/3730560242 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc400151a40 con 0x7fc4000c2460 2024-09-13T18:43:25.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.107+0000 7fc3f77fe700 1 -- 172.21.15.146:0/3730560242 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f00070d0 con 0x7fc4000c2460 2024-09-13T18:43:25.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.107+0000 7fc3f77fe700 1 -- 172.21.15.146:0/3730560242 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0007460 con 0x7fc4000c2460 2024-09-13T18:43:25.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.107+0000 7fc3ff59e700 1 -- 172.21.15.146:0/3730560242 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc400151c90 con 0x7fc4000c2460 2024-09-13T18:43:25.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.107+0000 7fc3ff59e700 1 -- 172.21.15.146:0/3730560242 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e4005310 con 0x7fc4000c2460 2024-09-13T18:43:25.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.107+0000 7fc3f77fe700 1 -- 172.21.15.146:0/3730560242 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc3f00075c0 con 0x7fc4000c2460 2024-09-13T18:43:25.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.107+0000 7fc3f77fe700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc3e0066760 0x7fc3e0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:25.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.107+0000 7fc3f77fe700 1 -- 172.21.15.146:0/3730560242 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fc3f00840b0 con 0x7fc4000c2460 2024-09-13T18:43:25.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.107+0000 7fc3f5d9b700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc3e0066760 0x7fc3e0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:25.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.111+0000 7fc3f5d9b700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc3e0066760 0x7fc3e0068c20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009bd0 tx=0x7fc3e8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:25.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.115+0000 7fc3f77fe700 1 -- 172.21.15.146:0/3730560242 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc3f0050d60 con 0x7fc4000c2460 2024-09-13T18:43:25.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.367+0000 7fc3ff59e700 1 -- 172.21.15.146:0/3730560242 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3e4005890 con 0x7fc4000c2460 2024-09-13T18:43:25.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.367+0000 7fc3f77fe700 1 -- 172.21.15.146:0/3730560242 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5531 (secure 0 0 0) 0x7fc3f0017020 con 0x7fc4000c2460 2024-09-13T18:43:25.390 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:25.390 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:21.347313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:resolve","state_seq":33,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:25.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.371+0000 7fc3ff59e700 1 -- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc3e0066760 msgr2=0x7fc3e0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:25.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.371+0000 7fc3ff59e700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc3e0066760 0x7fc3e0068c20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009bd0 tx=0x7fc3e8008030 comp rx=0 tx=0).stop 2024-09-13T18:43:25.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.371+0000 7fc3ff59e700 1 -- 172.21.15.146:0/3730560242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4000c2460 msgr2=0x7fc400152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:25.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.371+0000 7fc3ff59e700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4000c2460 0x7fc400152770 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000c490 tx=0x7fc3f000c850 comp rx=0 tx=0).stop 2024-09-13T18:43:25.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.371+0000 7fc3ff59e700 1 -- 172.21.15.146:0/3730560242 shutdown_connections 2024-09-13T18:43:25.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.371+0000 7fc3ff59e700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc3e0066760 0x7fc3e0068c20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:25.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.371+0000 7fc3ff59e700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc40014fa50 0x7fc4001501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:25.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.371+0000 7fc3ff59e700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4000c6070 0x7fc4000c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:25.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.371+0000 7fc3ff59e700 1 --2- 172.21.15.146:0/3730560242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4000c2460 0x7fc400152770 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:25.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.371+0000 7fc3ff59e700 1 -- 172.21.15.146:0/3730560242 >> 172.21.15.146:0/3730560242 conn(0x7fc400071080 msgr2=0x7fc40006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:25.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.371+0000 7fc3ff59e700 1 -- 172.21.15.146:0/3730560242 shutdown_connections 2024-09-13T18:43:25.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:25.371+0000 7fc3ff59e700 1 -- 172.21.15.146:0/3730560242 wait complete. 2024-09-13T18:43:25.393 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 65 2024-09-13T18:43:25.403 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:43:27.404 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:27.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.499+0000 7f00b359e700 1 Processor -- start 2024-09-13T18:43:27.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.499+0000 7f00b359e700 1 -- start start 2024-09-13T18:43:27.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.499+0000 7f00b359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00b4151590 0x7f00b41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:27.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.499+0000 7f00b359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f00b4157fd0 0x7f00b40c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:27.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.499+0000 7f00b359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f00b406db40 0x7f00b40c5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:27.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.499+0000 7f00b359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b40c69b0 con 0x7f00b406db40 2024-09-13T18:43:27.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b41572e0 con 0x7f00b4157fd0 2024-09-13T18:43:27.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b4158d10 con 0x7f00b4151590 2024-09-13T18:43:27.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b2d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f00b406db40 0x7f00b40c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:27.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b2d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f00b406db40 0x7f00b40c5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60972/0 (socket says 172.21.15.146:60972) 2024-09-13T18:43:27.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b2d9d700 1 -- 172.21.15.146:0/2119989772 learned_addr learned my addr 172.21.15.146:0/2119989772 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:27.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b259c700 1 --2- 172.21.15.146:0/2119989772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00b4151590 0x7f00b41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:27.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00abfff700 1 --2- 172.21.15.146:0/2119989772 >> v2:172.21.15.154:3300/0 conn(0x7f00b4157fd0 0x7f00b40c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:27.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b2d9d700 1 -- 172.21.15.146:0/2119989772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00b4151590 msgr2=0x7f00b41587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:27.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b2d9d700 1 --2- 172.21.15.146:0/2119989772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00b4151590 0x7f00b41587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:27.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b2d9d700 1 -- 172.21.15.146:0/2119989772 >> v2:172.21.15.154:3300/0 conn(0x7f00b4157fd0 msgr2=0x7f00b40c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:27.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b2d9d700 1 --2- 172.21.15.146:0/2119989772 >> v2:172.21.15.154:3300/0 conn(0x7f00b4157fd0 0x7f00b40c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:27.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b2d9d700 1 -- 172.21.15.146:0/2119989772 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b4066500 con 0x7f00b406db40 2024-09-13T18:43:27.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b259c700 1 --2- 172.21.15.146:0/2119989772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00b4151590 0x7f00b41587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:43:27.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b2d9d700 1 --2- 172.21.15.146:0/2119989772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00b406db40 0x7f00b40c5190 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f00b40c5a80 tx=0x7f00ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b345b1ab7c12e327 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:27.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b1d9b700 1 -- 172.21.15.146:0/2119989772 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00e070 con 0x7f00b406db40 2024-09-13T18:43:27.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b1d9b700 1 -- 172.21.15.146:0/2119989772 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac00b910 con 0x7f00b406db40 2024-09-13T18:43:27.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b1d9b700 1 -- 172.21.15.146:0/2119989772 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00bcc0 con 0x7f00b406db40 2024-09-13T18:43:27.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b359e700 1 -- 172.21.15.146:0/2119989772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00b406db40 msgr2=0x7f00b40c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:27.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b359e700 1 --2- 172.21.15.146:0/2119989772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00b406db40 0x7f00b40c5190 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f00b40c5a80 tx=0x7f00ac00b330 comp rx=0 tx=0).stop 2024-09-13T18:43:27.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b359e700 1 -- 172.21.15.146:0/2119989772 shutdown_connections 2024-09-13T18:43:27.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b359e700 1 --2- 172.21.15.146:0/2119989772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00b406db40 0x7f00b40c5190 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:27.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b359e700 1 --2- 172.21.15.146:0/2119989772 >> v2:172.21.15.154:3300/0 conn(0x7f00b4157fd0 0x7f00b40c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:27.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b359e700 1 --2- 172.21.15.146:0/2119989772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00b4151590 0x7f00b41587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:27.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b359e700 1 -- 172.21.15.146:0/2119989772 >> 172.21.15.146:0/2119989772 conn(0x7f00b4066b00 msgr2=0x7f00b406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:27.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b359e700 1 -- 172.21.15.146:0/2119989772 shutdown_connections 2024-09-13T18:43:27.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.503+0000 7f00b359e700 1 -- 172.21.15.146:0/2119989772 wait complete. 2024-09-13T18:43:27.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b359e700 1 Processor -- start 2024-09-13T18:43:27.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b359e700 1 -- start start 2024-09-13T18:43:27.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00b406db40 0x7f00b4154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:27.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00b4151590 0x7f00b40c3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:27.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4157fd0 0x7f00b40c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:27.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b41efb10 con 0x7f00b406db40 2024-09-13T18:43:27.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b40c30e0 con 0x7f00b4157fd0 2024-09-13T18:43:27.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b40c2a40 con 0x7f00b4151590 2024-09-13T18:43:27.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00b406db40 0x7f00b4154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:27.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00b406db40 0x7f00b4154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60988/0 (socket says 172.21.15.146:60988) 2024-09-13T18:43:27.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b259c700 1 -- 172.21.15.146:0/3214153178 learned_addr learned my addr 172.21.15.146:0/3214153178 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:27.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00abfff700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00b4151590 0x7f00b40c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:27.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b2d9d700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4157fd0 0x7f00b40c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:27.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b259c700 1 -- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00b4151590 msgr2=0x7f00b40c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:27.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b259c700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00b4151590 0x7f00b40c3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:27.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b259c700 1 -- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4157fd0 msgr2=0x7f00b40c3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:27.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b259c700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4157fd0 0x7f00b40c3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:27.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b259c700 1 -- 172.21.15.146:0/3214153178 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac009b30 con 0x7f00b406db40 2024-09-13T18:43:27.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b259c700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00b406db40 0x7f00b4154060 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f009c00d800 tx=0x7f009c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:27.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00aaffd700 1 -- 172.21.15.146:0/3214153178 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c004030 con 0x7f00b406db40 2024-09-13T18:43:27.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00aaffd700 1 -- 172.21.15.146:0/3214153178 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c004190 con 0x7f00b406db40 2024-09-13T18:43:27.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b359e700 1 -- 172.21.15.146:0/3214153178 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b40c4670 con 0x7f00b406db40 2024-09-13T18:43:27.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00aaffd700 1 -- 172.21.15.146:0/3214153178 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c004540 con 0x7f00b406db40 2024-09-13T18:43:27.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.507+0000 7f00b359e700 1 -- 172.21.15.146:0/3214153178 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b40c2d00 con 0x7f00b406db40 2024-09-13T18:43:27.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.511+0000 7f00b359e700 1 -- 172.21.15.146:0/3214153178 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b40c3f80 con 0x7f00b406db40 2024-09-13T18:43:27.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.511+0000 7f00aaffd700 1 -- 172.21.15.146:0/3214153178 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f009c0036c0 con 0x7f00b406db40 2024-09-13T18:43:27.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.511+0000 7f00aaffd700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0094066730 0x7f0094068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:27.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.511+0000 7f00aaffd700 1 -- 172.21.15.146:0/3214153178 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f009c083fb0 con 0x7f00b406db40 2024-09-13T18:43:27.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.511+0000 7f00abfff700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0094066730 0x7f0094068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:27.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.511+0000 7f00abfff700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0094066730 0x7f0094068bf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f00b406e770 tx=0x7f00a4009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:27.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.519+0000 7f00aaffd700 1 -- 172.21.15.146:0/3214153178 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f009c0b5db0 con 0x7f00b406db40 2024-09-13T18:43:27.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.775+0000 7f00b359e700 1 -- 172.21.15.146:0/3214153178 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00b40c3350 con 0x7f00b406db40 2024-09-13T18:43:27.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.775+0000 7f00aaffd700 1 -- 172.21.15.146:0/3214153178 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+5533 (secure 0 0 0) 0x7f009c019070 con 0x7f00b406db40 2024-09-13T18:43:27.797 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:27.797 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:26.369535+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:reconnect","state_seq":35,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:27.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.779+0000 7f00b359e700 1 -- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0094066730 msgr2=0x7f0094068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:27.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.779+0000 7f00b359e700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0094066730 0x7f0094068bf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f00b406e770 tx=0x7f00a4009240 comp rx=0 tx=0).stop 2024-09-13T18:43:27.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.779+0000 7f00b359e700 1 -- 172.21.15.146:0/3214153178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00b406db40 msgr2=0x7f00b4154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:27.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.779+0000 7f00b359e700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00b406db40 0x7f00b4154060 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f009c00d800 tx=0x7f009c00db10 comp rx=0 tx=0).stop 2024-09-13T18:43:27.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.779+0000 7f00b359e700 1 -- 172.21.15.146:0/3214153178 shutdown_connections 2024-09-13T18:43:27.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.779+0000 7f00b359e700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0094066730 0x7f0094068bf0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:27.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.779+0000 7f00b359e700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4157fd0 0x7f00b40c3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:27.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.779+0000 7f00b359e700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00b4151590 0x7f00b40c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:27.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.779+0000 7f00b359e700 1 --2- 172.21.15.146:0/3214153178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00b406db40 0x7f00b4154060 secure :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f009c00d800 tx=0x7f009c00db10 comp rx=0 tx=0).stop 2024-09-13T18:43:27.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.779+0000 7f00b359e700 1 -- 172.21.15.146:0/3214153178 >> 172.21.15.146:0/3214153178 conn(0x7f00b4066b00 msgr2=0x7f00b40c4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:27.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.779+0000 7f00b359e700 1 -- 172.21.15.146:0/3214153178 shutdown_connections 2024-09-13T18:43:27.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:27.779+0000 7f00b359e700 1 -- 172.21.15.146:0/3214153178 wait complete. 2024-09-13T18:43:27.803 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 66 2024-09-13T18:43:27.819 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:43:29.821 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:29.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b6f3d700 1 Processor -- start 2024-09-13T18:43:29.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b6f3d700 1 -- start start 2024-09-13T18:43:29.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b6f3d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f18a804a380 0x7f18a8045d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:29.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b6f3d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f18a8047320 0x7f18a80d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:29.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b6f3d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a8007b50 con 0x7f18a804a760 2024-09-13T18:43:29.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b6f3d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a8007fe0 con 0x7f18a804a380 2024-09-13T18:43:29.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b6f3d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a8005e40 con 0x7f18a8047320 2024-09-13T18:43:29.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b5f3b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f18a804a380 0x7f18a8045d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:29.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b5f3b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f18a804a380 0x7f18a8045d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60996/0 (socket says 172.21.15.146:60996) 2024-09-13T18:43:29.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b5f3b700 1 -- 172.21.15.146:0/2537640823 learned_addr learned my addr 172.21.15.146:0/2537640823 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:29.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b673c700 1 --2- 172.21.15.146:0/2537640823 >> v2:172.21.15.154:3300/0 conn(0x7f18a8047320 0x7f18a80d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:29.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b5f3b700 1 -- 172.21.15.146:0/2537640823 >> v2:172.21.15.154:3300/0 conn(0x7f18a8047320 msgr2=0x7f18a80d5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:29.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b5f3b700 1 --2- 172.21.15.146:0/2537640823 >> v2:172.21.15.154:3300/0 conn(0x7f18a8047320 0x7f18a80d5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:29.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b5f3b700 1 -- 172.21.15.146:0/2537640823 >> v1:172.21.15.154:6789/0 conn(0x7f18a804a760 legacy=0x7f18a80deac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:29.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b5f3b700 1 -- 172.21.15.146:0/2537640823 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a80458b0 con 0x7f18a804a380 2024-09-13T18:43:29.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b5f3b700 1 --2- 172.21.15.146:0/2537640823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a804a380 0x7f18a8045d70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009df0 tx=0x7f18ac009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ee3e05809e8c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:29.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b4f39700 1 -- 172.21.15.146:0/2537640823 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac00b8a0 con 0x7f18a804a380 2024-09-13T18:43:29.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b4f39700 1 -- 172.21.15.146:0/2537640823 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ac00ba00 con 0x7f18a804a380 2024-09-13T18:43:29.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.923+0000 7f18b4f39700 1 -- 172.21.15.146:0/2537640823 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac00bd90 con 0x7f18a804a380 2024-09-13T18:43:29.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.927+0000 7f18b6f3d700 1 -- 172.21.15.146:0/2537640823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a804a380 msgr2=0x7f18a8045d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:29.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.927+0000 7f18b6f3d700 1 --2- 172.21.15.146:0/2537640823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a804a380 0x7f18a8045d70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009df0 tx=0x7f18ac009bf0 comp rx=0 tx=0).stop 2024-09-13T18:43:29.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.927+0000 7f18b6f3d700 1 -- 172.21.15.146:0/2537640823 shutdown_connections 2024-09-13T18:43:29.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.927+0000 7f18b6f3d700 1 --2- 172.21.15.146:0/2537640823 >> v2:172.21.15.154:3300/0 conn(0x7f18a8047320 0x7f18a80d5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:29.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.927+0000 7f18b6f3d700 1 --2- 172.21.15.146:0/2537640823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a804a380 0x7f18a8045d70 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:29.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.927+0000 7f18b6f3d700 1 -- 172.21.15.146:0/2537640823 >> 172.21.15.146:0/2537640823 conn(0x7f18a8050610 msgr2=0x7f18a80509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:29.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.927+0000 7f18b6f3d700 1 -- 172.21.15.146:0/2537640823 shutdown_connections 2024-09-13T18:43:29.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.927+0000 7f18b6f3d700 1 -- 172.21.15.146:0/2537640823 wait complete. 2024-09-13T18:43:29.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.927+0000 7f18b6f3d700 1 Processor -- start 2024-09-13T18:43:29.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.927+0000 7f18b6f3d700 1 -- start start 2024-09-13T18:43:29.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b6f3d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a8047320 0x7f18a80cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:29.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b6f3d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a804a380 0x7f18a8171220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:29.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b6f3d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18a804a760 0x7f18a804cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:29.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b6f3d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a8171760 con 0x7f18a804a380 2024-09-13T18:43:29.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b6f3d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a8174e40 con 0x7f18a804a760 2024-09-13T18:43:29.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b6f3d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80cadc0 con 0x7f18a8047320 2024-09-13T18:43:29.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b573a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a804a380 0x7f18a8171220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:29.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b573a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a804a380 0x7f18a8171220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60998/0 (socket says 172.21.15.146:60998) 2024-09-13T18:43:29.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b573a700 1 -- 172.21.15.146:0/787108049 learned_addr learned my addr 172.21.15.146:0/787108049 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:29.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b673c700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18a804a760 0x7f18a804cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:29.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b5f3b700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a8047320 0x7f18a80cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:29.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b573a700 1 -- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a8047320 msgr2=0x7f18a80cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:29.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b573a700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a8047320 0x7f18a80cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:29.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b573a700 1 -- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18a804a760 msgr2=0x7f18a804cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:29.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b573a700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18a804a760 0x7f18a804cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:29.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b573a700 1 -- 172.21.15.146:0/787108049 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac009570 con 0x7f18a804a380 2024-09-13T18:43:29.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b573a700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a804a380 0x7f18a8171220 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f18a000c460 tx=0x7f18a000c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:29.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18a67fc700 1 -- 172.21.15.146:0/787108049 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a00052d0 con 0x7f18a804a380 2024-09-13T18:43:29.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18a67fc700 1 -- 172.21.15.146:0/787108049 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a0005430 con 0x7f18a804a380 2024-09-13T18:43:29.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b6f3d700 1 -- 172.21.15.146:0/787108049 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a8174830 con 0x7f18a804a380 2024-09-13T18:43:29.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18a67fc700 1 -- 172.21.15.146:0/787108049 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a00055a0 con 0x7f18a804a380 2024-09-13T18:43:29.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b6f3d700 1 -- 172.21.15.146:0/787108049 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a8033a10 con 0x7f18a804a380 2024-09-13T18:43:29.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18b6f3d700 1 -- 172.21.15.146:0/787108049 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a8174ff0 con 0x7f18a804a380 2024-09-13T18:43:29.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.931+0000 7f18a67fc700 1 -- 172.21.15.146:0/787108049 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f18a00036a0 con 0x7f18a804a380 2024-09-13T18:43:29.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.935+0000 7f18a67fc700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1894066760 0x7f1894068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:29.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.935+0000 7f18a67fc700 1 -- 172.21.15.146:0/787108049 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f18a00838c0 con 0x7f18a804a380 2024-09-13T18:43:29.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.935+0000 7f18b5f3b700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1894066760 0x7f1894068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:29.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.935+0000 7f18b5f3b700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1894066760 0x7f1894068c20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f18ac000c00 tx=0x7f18ac008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:29.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:29.939+0000 7f18a67fc700 1 -- 172.21.15.146:0/787108049 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f18a8174ff0 con 0x7f18a804a380 2024-09-13T18:43:30.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.151+0000 7f18b6f3d700 1 -- 172.21.15.146:0/787108049 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18a8174ff0 con 0x7f18a804a380 2024-09-13T18:43:30.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.151+0000 7f18a67fc700 1 -- 172.21.15.146:0/787108049 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+5533 (secure 0 0 0) 0x7f18a8174ff0 con 0x7f18a804a380 2024-09-13T18:43:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:30.173 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:26.369535+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:reconnect","state_seq":35,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:30.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.155+0000 7f18b6f3d700 1 -- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1894066760 msgr2=0x7f1894068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:30.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.155+0000 7f18b6f3d700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1894066760 0x7f1894068c20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f18ac000c00 tx=0x7f18ac008030 comp rx=0 tx=0).stop 2024-09-13T18:43:30.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.155+0000 7f18b6f3d700 1 -- 172.21.15.146:0/787108049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a804a380 msgr2=0x7f18a8171220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:30.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.155+0000 7f18b6f3d700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a804a380 0x7f18a8171220 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f18a000c460 tx=0x7f18a000c770 comp rx=0 tx=0).stop 2024-09-13T18:43:30.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.155+0000 7f18b6f3d700 1 -- 172.21.15.146:0/787108049 shutdown_connections 2024-09-13T18:43:30.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.155+0000 7f18b6f3d700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1894066760 0x7f1894068c20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:30.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.155+0000 7f18b6f3d700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18a804a760 0x7f18a804cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:30.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.155+0000 7f18b6f3d700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18a804a380 0x7f18a8171220 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:30.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.155+0000 7f18b6f3d700 1 --2- 172.21.15.146:0/787108049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18a8047320 0x7f18a80cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:30.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.155+0000 7f18b6f3d700 1 -- 172.21.15.146:0/787108049 >> 172.21.15.146:0/787108049 conn(0x7f18a8050610 msgr2=0x7f18a80d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:30.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.155+0000 7f18b6f3d700 1 -- 172.21.15.146:0/787108049 shutdown_connections 2024-09-13T18:43:30.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:30.155+0000 7f18b6f3d700 1 -- 172.21.15.146:0/787108049 wait complete. 2024-09-13T18:43:30.177 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 66 2024-09-13T18:43:30.188 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 66, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 66, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5800": { "addr": "172.21.15.154:6839/776638146", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 776638146, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 776638146, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5800, "incarnation": 52, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 30 }, "gid_6063": { "addr": "172.21.15.146:6835/832794630", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 832794630, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 832794630, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6063, "incarnation": 58, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 43 }, "gid_6207": { "addr": "172.21.15.146:6837/3234109877", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3234109877, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3234109877, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6207, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:reconnect", "state_seq": 35 } }, "last_failure": 0, "last_failure_osd_epoch": 36, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:43:26.369535+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5800, "mds_1": 6207, "mds_2": 6063 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:43:32.189 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:32.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d17fff700 1 Processor -- start 2024-09-13T18:43:32.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d17fff700 1 -- start start 2024-09-13T18:43:32.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d17fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3d18154a40 0x7f3d180721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:32.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d17fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3d180cac60 0x7f3d180cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:32.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d17fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d180c8c70 con 0x7f3d1806f7d0 2024-09-13T18:43:32.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d17fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d18072810 con 0x7f3d18154a40 2024-09-13T18:43:32.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d17fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d18157910 con 0x7f3d180cac60 2024-09-13T18:43:32.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d16ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3d18154a40 0x7f3d180721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:32.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d16ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3d18154a40 0x7f3d180721f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59218/0 (socket says 172.21.15.146:59218) 2024-09-13T18:43:32.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d16ffd700 1 -- 172.21.15.146:0/1426357583 learned_addr learned my addr 172.21.15.146:0/1426357583 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:32.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d177fe700 1 --2- 172.21.15.146:0/1426357583 >> v2:172.21.15.154:3300/0 conn(0x7f3d180cac60 0x7f3d180cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:32.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d16ffd700 1 -- 172.21.15.146:0/1426357583 >> v2:172.21.15.154:3300/0 conn(0x7f3d180cac60 msgr2=0x7f3d180cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:32.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d16ffd700 1 --2- 172.21.15.146:0/1426357583 >> v2:172.21.15.154:3300/0 conn(0x7f3d180cac60 0x7f3d180cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:32.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d16ffd700 1 -- 172.21.15.146:0/1426357583 >> v1:172.21.15.154:6789/0 conn(0x7f3d1806f7d0 legacy=0x7f3d18157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:32.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d16ffd700 1 -- 172.21.15.146:0/1426357583 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d180c8f50 con 0x7f3d18154a40 2024-09-13T18:43:32.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d16ffd700 1 --2- 172.21.15.146:0/1426357583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d18154a40 0x7f3d180721f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3d180cbd50 tx=0x7f3d08009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153dec6c53e2344e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:32.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d15ffb700 1 -- 172.21.15.146:0/1426357583 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08011830 con 0x7f3d18154a40 2024-09-13T18:43:32.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d15ffb700 1 -- 172.21.15.146:0/1426357583 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d08011990 con 0x7f3d18154a40 2024-09-13T18:43:32.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.291+0000 7f3d15ffb700 1 -- 172.21.15.146:0/1426357583 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08011d20 con 0x7f3d18154a40 2024-09-13T18:43:32.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 -- 172.21.15.146:0/1426357583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d18154a40 msgr2=0x7f3d180721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:32.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 --2- 172.21.15.146:0/1426357583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d18154a40 0x7f3d180721f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3d180cbd50 tx=0x7f3d08009bf0 comp rx=0 tx=0).stop 2024-09-13T18:43:32.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 -- 172.21.15.146:0/1426357583 shutdown_connections 2024-09-13T18:43:32.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 --2- 172.21.15.146:0/1426357583 >> v2:172.21.15.154:3300/0 conn(0x7f3d180cac60 0x7f3d180cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:32.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 --2- 172.21.15.146:0/1426357583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d18154a40 0x7f3d180721f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:32.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 -- 172.21.15.146:0/1426357583 >> 172.21.15.146:0/1426357583 conn(0x7f3d18066090 msgr2=0x7f3d18066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:32.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 -- 172.21.15.146:0/1426357583 shutdown_connections 2024-09-13T18:43:32.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 -- 172.21.15.146:0/1426357583 wait complete. 2024-09-13T18:43:32.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 Processor -- start 2024-09-13T18:43:32.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 -- start start 2024-09-13T18:43:32.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d1806f7d0 0x7f3d180c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:32.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d180cac60 0x7f3d181590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:32.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d18154a40 0x7f3d181516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:32.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180c4ff0 con 0x7f3d180cac60 2024-09-13T18:43:32.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1806d6d0 con 0x7f3d1806f7d0 2024-09-13T18:43:32.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d17fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1806d9b0 con 0x7f3d18154a40 2024-09-13T18:43:32.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d167fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d180cac60 0x7f3d181590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:32.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d167fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d180cac60 0x7f3d181590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59232/0 (socket says 172.21.15.146:59232) 2024-09-13T18:43:32.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d167fc700 1 -- 172.21.15.146:0/3868107437 learned_addr learned my addr 172.21.15.146:0/3868107437 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:32.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d16ffd700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d1806f7d0 0x7f3d180c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:32.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d177fe700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d18154a40 0x7f3d181516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:32.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d167fc700 1 -- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d18154a40 msgr2=0x7f3d181516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:32.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d167fc700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d18154a40 0x7f3d181516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:32.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d167fc700 1 -- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d1806f7d0 msgr2=0x7f3d180c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:32.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d167fc700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d1806f7d0 0x7f3d180c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:32.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d167fc700 1 -- 172.21.15.146:0/3868107437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d08009570 con 0x7f3d180cac60 2024-09-13T18:43:32.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.295+0000 7f3d177fe700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d18154a40 0x7f3d181516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:43:32.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.299+0000 7f3d167fc700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d180cac60 0x7f3d181590e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00c500 tx=0x7f3d0c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:32.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.299+0000 7f3cff7fe700 1 -- 172.21.15.146:0/3868107437 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0052d0 con 0x7f3d180cac60 2024-09-13T18:43:32.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.299+0000 7f3cff7fe700 1 -- 172.21.15.146:0/3868107437 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c005430 con 0x7f3d180cac60 2024-09-13T18:43:32.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.299+0000 7f3d17fff700 1 -- 172.21.15.146:0/3868107437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1806dc90 con 0x7f3d180cac60 2024-09-13T18:43:32.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.299+0000 7f3cff7fe700 1 -- 172.21.15.146:0/3868107437 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0055a0 con 0x7f3d180cac60 2024-09-13T18:43:32.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.299+0000 7f3d17fff700 1 -- 172.21.15.146:0/3868107437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d181eebb0 con 0x7f3d180cac60 2024-09-13T18:43:32.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.299+0000 7f3d17fff700 1 -- 172.21.15.146:0/3868107437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d180c9e00 con 0x7f3d180cac60 2024-09-13T18:43:32.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.299+0000 7f3cff7fe700 1 -- 172.21.15.146:0/3868107437 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3d0c0038e0 con 0x7f3d180cac60 2024-09-13T18:43:32.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.299+0000 7f3cff7fe700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d04066730 0x7f3d04068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:32.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.299+0000 7f3cff7fe700 1 -- 172.21.15.146:0/3868107437 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f3d0c083c30 con 0x7f3d180cac60 2024-09-13T18:43:32.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.299+0000 7f3d16ffd700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d04066730 0x7f3d04068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:32.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.303+0000 7f3d16ffd700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d04066730 0x7f3d04068bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3d08000c00 tx=0x7f3d08008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:32.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.307+0000 7f3cff7fe700 1 -- 172.21.15.146:0/3868107437 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3d0c050850 con 0x7f3d180cac60 2024-09-13T18:43:32.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.551+0000 7f3d17fff700 1 -- 172.21.15.146:0/3868107437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d18154e20 con 0x7f3d180cac60 2024-09-13T18:43:32.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.551+0000 7f3cff7fe700 1 -- 172.21.15.146:0/3868107437 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5530 (secure 0 0 0) 0x7f3d0c017090 con 0x7f3d180cac60 2024-09-13T18:43:32.573 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:32.573 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:31.401217+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:rejoin","state_seq":37,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:32.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.555+0000 7f3d17fff700 1 -- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d04066730 msgr2=0x7f3d04068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:32.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.555+0000 7f3d17fff700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d04066730 0x7f3d04068bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3d08000c00 tx=0x7f3d08008030 comp rx=0 tx=0).stop 2024-09-13T18:43:32.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.555+0000 7f3d17fff700 1 -- 172.21.15.146:0/3868107437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d180cac60 msgr2=0x7f3d181590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:32.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.555+0000 7f3d17fff700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d180cac60 0x7f3d181590e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00c500 tx=0x7f3d0c00c810 comp rx=0 tx=0).stop 2024-09-13T18:43:32.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.555+0000 7f3d17fff700 1 -- 172.21.15.146:0/3868107437 shutdown_connections 2024-09-13T18:43:32.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.555+0000 7f3d17fff700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d04066730 0x7f3d04068bf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:32.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.555+0000 7f3d17fff700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d18154a40 0x7f3d181516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:32.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.555+0000 7f3d17fff700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d180cac60 0x7f3d181590e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:32.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.555+0000 7f3d17fff700 1 --2- 172.21.15.146:0/3868107437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d1806f7d0 0x7f3d180c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:32.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.555+0000 7f3d17fff700 1 -- 172.21.15.146:0/3868107437 >> 172.21.15.146:0/3868107437 conn(0x7f3d18066090 msgr2=0x7f3d18157c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:32.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.555+0000 7f3d17fff700 1 -- 172.21.15.146:0/3868107437 shutdown_connections 2024-09-13T18:43:32.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:32.555+0000 7f3d17fff700 1 -- 172.21.15.146:0/3868107437 wait complete. 2024-09-13T18:43:32.577 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 67 2024-09-13T18:43:32.590 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:43:34.591 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:34.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.707+0000 7fbed8893700 1 Processor -- start 2024-09-13T18:43:34.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.707+0000 7fbed8893700 1 -- start start 2024-09-13T18:43:34.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.707+0000 7fbed8893700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbed40cf170 0x7fbed41595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:34.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed8893700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbed40cf550 0x7fbed40c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:34.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed8893700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed415a540 con 0x7fbed40cb5b0 2024-09-13T18:43:34.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed8893700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed40c5220 con 0x7fbed40cf550 2024-09-13T18:43:34.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed8893700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed4159ca0 con 0x7fbed40cf170 2024-09-13T18:43:34.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbecbfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbed40cf550 0x7fbed40c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:34.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbecbfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbed40cf550 0x7fbed40c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59242/0 (socket says 172.21.15.146:59242) 2024-09-13T18:43:34.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbecbfff700 1 -- 172.21.15.146:0/1689527335 learned_addr learned my addr 172.21.15.146:0/1689527335 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:34.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed37fe700 1 --2- 172.21.15.146:0/1689527335 >> v2:172.21.15.154:3300/0 conn(0x7fbed40cf170 0x7fbed41595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:34.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbecbfff700 1 -- 172.21.15.146:0/1689527335 >> v2:172.21.15.154:3300/0 conn(0x7fbed40cf170 msgr2=0x7fbed41595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:34.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbecbfff700 1 --2- 172.21.15.146:0/1689527335 >> v2:172.21.15.154:3300/0 conn(0x7fbed40cf170 0x7fbed41595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:34.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbecbfff700 1 -- 172.21.15.146:0/1689527335 >> v1:172.21.15.154:6789/0 conn(0x7fbed40cb5b0 legacy=0x7fbed41534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:34.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbecbfff700 1 -- 172.21.15.146:0/1689527335 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed4159f80 con 0x7fbed40cf550 2024-09-13T18:43:34.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbecbfff700 1 --2- 172.21.15.146:0/1689527335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed40cf550 0x7fbed40c56b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbed408c1e0 tx=0x7fbebc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd33fa67ea2cfd0a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:34.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed2ffd700 1 -- 172.21.15.146:0/1689527335 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00b8a0 con 0x7fbed40cf550 2024-09-13T18:43:34.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed2ffd700 1 -- 172.21.15.146:0/1689527335 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc00ba00 con 0x7fbed40cf550 2024-09-13T18:43:34.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed2ffd700 1 -- 172.21.15.146:0/1689527335 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00bd90 con 0x7fbed40cf550 2024-09-13T18:43:34.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed8893700 1 -- 172.21.15.146:0/1689527335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed40cf550 msgr2=0x7fbed40c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:34.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed8893700 1 --2- 172.21.15.146:0/1689527335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed40cf550 0x7fbed40c56b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbed408c1e0 tx=0x7fbebc009bf0 comp rx=0 tx=0).stop 2024-09-13T18:43:34.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed8893700 1 -- 172.21.15.146:0/1689527335 shutdown_connections 2024-09-13T18:43:34.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed8893700 1 --2- 172.21.15.146:0/1689527335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed40cf550 0x7fbed40c56b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:34.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed8893700 1 --2- 172.21.15.146:0/1689527335 >> v2:172.21.15.154:3300/0 conn(0x7fbed40cf170 0x7fbed41595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:34.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed8893700 1 -- 172.21.15.146:0/1689527335 >> 172.21.15.146:0/1689527335 conn(0x7fbed4072220 msgr2=0x7fbed4066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:34.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed8893700 1 -- 172.21.15.146:0/1689527335 shutdown_connections 2024-09-13T18:43:34.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.711+0000 7fbed8893700 1 -- 172.21.15.146:0/1689527335 wait complete. 2024-09-13T18:43:34.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed8893700 1 Processor -- start 2024-09-13T18:43:34.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed8893700 1 -- start start 2024-09-13T18:43:34.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed8893700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbed40cb5b0 0x7fbed414a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:34.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed8893700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbed40cf170 0x7fbed414e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:34.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed8893700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed41e23d0 0x7fbed41f2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:34.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed8893700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed414ee10 con 0x7fbed41e23d0 2024-09-13T18:43:34.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed8893700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed41e0100 con 0x7fbed40cb5b0 2024-09-13T18:43:34.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed8893700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed4150a50 con 0x7fbed40cf170 2024-09-13T18:43:34.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed41e23d0 0x7fbed41f2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:34.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed41e23d0 0x7fbed41f2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59248/0 (socket says 172.21.15.146:59248) 2024-09-13T18:43:34.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed3fff700 1 -- 172.21.15.146:0/2165377123 learned_addr learned my addr 172.21.15.146:0/2165377123 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:34.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbecbfff700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbed40cf170 0x7fbed414e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:34.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed37fe700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbed40cb5b0 0x7fbed414a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:34.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed3fff700 1 -- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbed40cf170 msgr2=0x7fbed414e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:34.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed3fff700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbed40cf170 0x7fbed414e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:34.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed3fff700 1 -- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbed40cb5b0 msgr2=0x7fbed414a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:34.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed3fff700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbed40cb5b0 0x7fbed414a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:34.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed3fff700 1 -- 172.21.15.146:0/2165377123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc009570 con 0x7fbed41e23d0 2024-09-13T18:43:34.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed3fff700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed41e23d0 0x7fbed41f2d00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00cbc0 tx=0x7fbecc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:34.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed0ff9700 1 -- 172.21.15.146:0/2165377123 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc003c20 con 0x7fbed41e23d0 2024-09-13T18:43:34.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed0ff9700 1 -- 172.21.15.146:0/2165377123 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc003d80 con 0x7fbed41e23d0 2024-09-13T18:43:34.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed8893700 1 -- 172.21.15.146:0/2165377123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed41df810 con 0x7fbed41e23d0 2024-09-13T18:43:34.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed0ff9700 1 -- 172.21.15.146:0/2165377123 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc00aa80 con 0x7fbed41e23d0 2024-09-13T18:43:34.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.715+0000 7fbed8893700 1 -- 172.21.15.146:0/2165377123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed41dfab0 con 0x7fbed41e23d0 2024-09-13T18:43:34.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.719+0000 7fbed8893700 1 -- 172.21.15.146:0/2165377123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed41e02b0 con 0x7fbed41e23d0 2024-09-13T18:43:34.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.719+0000 7fbed0ff9700 1 -- 172.21.15.146:0/2165377123 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbecc00abe0 con 0x7fbed41e23d0 2024-09-13T18:43:34.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.719+0000 7fbed0ff9700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbeb8066730 0x7fbeb8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:34.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.719+0000 7fbed0ff9700 1 -- 172.21.15.146:0/2165377123 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fbecc084830 con 0x7fbed41e23d0 2024-09-13T18:43:34.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.719+0000 7fbed37fe700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbeb8066730 0x7fbeb8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:34.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.719+0000 7fbed37fe700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbeb8066730 0x7fbeb8068bf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbec4006b40 tx=0x7fbec4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:34.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.727+0000 7fbed0ff9700 1 -- 172.21.15.146:0/2165377123 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbecc051450 con 0x7fbed41e23d0 2024-09-13T18:43:34.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.947+0000 7fbed8893700 1 -- 172.21.15.146:0/2165377123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbed4066850 con 0x7fbed41e23d0 2024-09-13T18:43:34.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.947+0000 7fbed0ff9700 1 -- 172.21.15.146:0/2165377123 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5530 (secure 0 0 0) 0x7fbecc054a70 con 0x7fbed41e23d0 2024-09-13T18:43:34.971 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:34.971 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:31.401217+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:rejoin","state_seq":37,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:34.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.951+0000 7fbed8893700 1 -- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbeb8066730 msgr2=0x7fbeb8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:34.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.951+0000 7fbed8893700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbeb8066730 0x7fbeb8068bf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbec4006b40 tx=0x7fbec4008040 comp rx=0 tx=0).stop 2024-09-13T18:43:34.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.951+0000 7fbed8893700 1 -- 172.21.15.146:0/2165377123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed41e23d0 msgr2=0x7fbed41f2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:34.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.951+0000 7fbed8893700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed41e23d0 0x7fbed41f2d00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00cbc0 tx=0x7fbecc00ced0 comp rx=0 tx=0).stop 2024-09-13T18:43:34.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.951+0000 7fbed8893700 1 -- 172.21.15.146:0/2165377123 shutdown_connections 2024-09-13T18:43:34.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.951+0000 7fbed8893700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbeb8066730 0x7fbeb8068bf0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:34.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.951+0000 7fbed8893700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed41e23d0 0x7fbed41f2d00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:34.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.951+0000 7fbed8893700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbed40cf170 0x7fbed414e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:34.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.951+0000 7fbed8893700 1 --2- 172.21.15.146:0/2165377123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbed40cb5b0 0x7fbed414a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:34.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.951+0000 7fbed8893700 1 -- 172.21.15.146:0/2165377123 >> 172.21.15.146:0/2165377123 conn(0x7fbed4072220 msgr2=0x7fbed41528c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:34.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.951+0000 7fbed8893700 1 -- 172.21.15.146:0/2165377123 shutdown_connections 2024-09-13T18:43:34.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:34.951+0000 7fbed8893700 1 -- 172.21.15.146:0/2165377123 wait complete. 2024-09-13T18:43:34.976 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 67 2024-09-13T18:43:34.989 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:43:36.990 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:37.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2639f24700 1 Processor -- start 2024-09-13T18:43:37.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2639f24700 1 -- start start 2024-09-13T18:43:37.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2639f24700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f26340c7b00 0x7f26340c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:37.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2639f24700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26340c6a30 con 0x7f263414fb50 2024-09-13T18:43:37.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2639f24700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2634156810 con 0x7f26340c65a0 2024-09-13T18:43:37.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2639f24700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26341559b0 con 0x7f26340c7b00 2024-09-13T18:43:37.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2633fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f26340c7b00 0x7f26340c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:37.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2638f22700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f263414fb50 0x7f26340c7fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43636/0 (socket says 172.21.15.146:43636) 2024-09-13T18:43:37.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2633fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f26340c7b00 0x7f26340c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59252/0 (socket says 172.21.15.146:59252) 2024-09-13T18:43:37.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2638f22700 1 -- 172.21.15.146:0/1635984226 learned_addr learned my addr 172.21.15.146:0/1635984226 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:43:37.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2633fff700 1 -- 172.21.15.146:0/1635984226 >> v1:172.21.15.154:6789/0 conn(0x7f26340c65a0 legacy=0x7f2634159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:37.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2633fff700 1 -- 172.21.15.146:0/1635984226 >> v1:172.21.15.146:6789/0 conn(0x7f263414fb50 legacy=0x7f26340c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:37.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2633fff700 1 -- 172.21.15.146:0/1635984226 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2634155c90 con 0x7f26340c7b00 2024-09-13T18:43:37.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2633fff700 1 --2- 172.21.15.146:0/1635984226 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26340c7b00 0x7f26340c6060 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2620009880 tx=0x7f2620009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6d80f01ea3bf81 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:37.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f26337fe700 1 -- 172.21.15.146:0/1635984226 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620004230 con 0x7f26340c7b00 2024-09-13T18:43:37.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f26337fe700 1 -- 172.21.15.146:0/1635984226 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2620004390 con 0x7f26340c7b00 2024-09-13T18:43:37.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f26337fe700 1 -- 172.21.15.146:0/1635984226 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262000f740 con 0x7f26340c7b00 2024-09-13T18:43:37.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2639f24700 1 -- 172.21.15.146:0/1635984226 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26340c7b00 msgr2=0x7f26340c6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:37.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2639f24700 1 --2- 172.21.15.146:0/1635984226 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26340c7b00 0x7f26340c6060 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2620009880 tx=0x7f2620009b90 comp rx=0 tx=0).stop 2024-09-13T18:43:37.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2639f24700 1 -- 172.21.15.146:0/1635984226 shutdown_connections 2024-09-13T18:43:37.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2639f24700 1 --2- 172.21.15.146:0/1635984226 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26340c7b00 0x7f26340c6060 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:37.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2639f24700 1 -- 172.21.15.146:0/1635984226 >> 172.21.15.146:0/1635984226 conn(0x7f2634066b00 msgr2=0x7f263406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:37.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2639f24700 1 -- 172.21.15.146:0/1635984226 shutdown_connections 2024-09-13T18:43:37.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.091+0000 7f2639f24700 1 -- 172.21.15.146:0/1635984226 wait complete. 2024-09-13T18:43:37.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2639f24700 1 Processor -- start 2024-09-13T18:43:37.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2639f24700 1 -- start start 2024-09-13T18:43:37.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2639f24700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26340c65a0 0x7f26340c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:37.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2639f24700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26340c7b00 0x7f263413f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:37.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2639f24700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f263414fb50 0x7f2634143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:37.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2639f24700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2634155630 con 0x7f26340c7b00 2024-09-13T18:43:37.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2639f24700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2634155350 con 0x7f26340c65a0 2024-09-13T18:43:37.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2639f24700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26341548b0 con 0x7f263414fb50 2024-09-13T18:43:37.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2633fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26340c7b00 0x7f263413f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:37.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2633fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26340c7b00 0x7f263413f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59264/0 (socket says 172.21.15.146:59264) 2024-09-13T18:43:37.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2633fff700 1 -- 172.21.15.146:0/1988368191 learned_addr learned my addr 172.21.15.146:0/1988368191 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:37.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2638f22700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26340c65a0 0x7f26340c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:37.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2639723700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f263414fb50 0x7f2634143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:37.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2633fff700 1 -- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f263414fb50 msgr2=0x7f2634143c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2633fff700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f263414fb50 0x7f2634143c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2633fff700 1 -- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26340c65a0 msgr2=0x7f26340c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2633fff700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26340c65a0 0x7f26340c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2633fff700 1 -- 172.21.15.146:0/1988368191 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26200094b0 con 0x7f26340c7b00 2024-09-13T18:43:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f2633fff700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26340c7b00 0x7f263413f150 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f26200097d0 tx=0x7f2620004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.095+0000 7f26317fa700 1 -- 172.21.15.146:0/1988368191 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620006ea0 con 0x7f26340c7b00 2024-09-13T18:43:37.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.099+0000 7f26317fa700 1 -- 172.21.15.146:0/1988368191 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26200038d0 con 0x7f26340c7b00 2024-09-13T18:43:37.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.099+0000 7f2639f24700 1 -- 172.21.15.146:0/1988368191 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2634153800 con 0x7f26340c7b00 2024-09-13T18:43:37.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.099+0000 7f26317fa700 1 -- 172.21.15.146:0/1988368191 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620003c60 con 0x7f26340c7b00 2024-09-13T18:43:37.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.099+0000 7f2639f24700 1 -- 172.21.15.146:0/1988368191 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2634154b00 con 0x7f26340c7b00 2024-09-13T18:43:37.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.099+0000 7f2639f24700 1 -- 172.21.15.146:0/1988368191 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2634153990 con 0x7f26340c7b00 2024-09-13T18:43:37.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.099+0000 7f26317fa700 1 -- 172.21.15.146:0/1988368191 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2620021460 con 0x7f26340c7b00 2024-09-13T18:43:37.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.099+0000 7f26317fa700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2618066760 0x7f2618068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:37.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.099+0000 7f26317fa700 1 -- 172.21.15.146:0/1988368191 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f2620013070 con 0x7f26340c7b00 2024-09-13T18:43:37.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.099+0000 7f2638f22700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2618066760 0x7f2618068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:37.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.099+0000 7f2638f22700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2618066760 0x7f2618068c20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2628000f60 tx=0x7f2628007f30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:37.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.107+0000 7f26317fa700 1 -- 172.21.15.146:0/1988368191 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2620054d20 con 0x7f26340c7b00 2024-09-13T18:43:37.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.323+0000 7f2639f24700 1 -- 172.21.15.146:0/1988368191 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2634154cf0 con 0x7f26340c7b00 2024-09-13T18:43:37.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.323+0000 7f26317fa700 1 -- 172.21.15.146:0/1988368191 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+5530 (secure 0 0 0) 0x7f2620025090 con 0x7f26340c7b00 2024-09-13T18:43:37.347 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:37.347 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:36.435329+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:37.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.327+0000 7f2639f24700 1 -- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2618066760 msgr2=0x7f2618068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:37.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.327+0000 7f2639f24700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2618066760 0x7f2618068c20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2628000f60 tx=0x7f2628007f30 comp rx=0 tx=0).stop 2024-09-13T18:43:37.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.327+0000 7f2639f24700 1 -- 172.21.15.146:0/1988368191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26340c7b00 msgr2=0x7f263413f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:37.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.327+0000 7f2639f24700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26340c7b00 0x7f263413f150 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f26200097d0 tx=0x7f2620004770 comp rx=0 tx=0).stop 2024-09-13T18:43:37.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.327+0000 7f2639f24700 1 -- 172.21.15.146:0/1988368191 shutdown_connections 2024-09-13T18:43:37.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.327+0000 7f2639f24700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2618066760 0x7f2618068c20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:37.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.327+0000 7f2639f24700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f263414fb50 0x7f2634143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:37.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.327+0000 7f2639f24700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26340c7b00 0x7f263413f150 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:37.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.327+0000 7f2639f24700 1 --2- 172.21.15.146:0/1988368191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26340c65a0 0x7f26340c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:37.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.327+0000 7f2639f24700 1 -- 172.21.15.146:0/1988368191 >> 172.21.15.146:0/1988368191 conn(0x7f2634066b00 msgr2=0x7f263406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:37.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.327+0000 7f2639f24700 1 -- 172.21.15.146:0/1988368191 shutdown_connections 2024-09-13T18:43:37.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:37.327+0000 7f2639f24700 1 -- 172.21.15.146:0/1988368191 wait complete. 2024-09-13T18:43:37.353 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 68 2024-09-13T18:43:37.363 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=1, replacing gid=5784 2024-09-13T18:43:37.363 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 3 secs before reviving mds.e 2024-09-13T18:43:40.271 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:43:40.273 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:43:40.364 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2024-09-13T18:43:40.364 INFO:tasks.ceph.mds.e:Restarting daemon 2024-09-13T18:43:40.364 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-09-13T18:43:40.367 INFO:tasks.ceph.mds.e:Started 2024-09-13T18:43:40.367 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:40.450 INFO:tasks.ceph.mds.e.smithi146.stdout:starting mds.e at 2024-09-13T18:43:40.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff999b5d700 1 Processor -- start 2024-09-13T18:43:40.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff999b5d700 1 -- start start 2024-09-13T18:43:40.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff999b5d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff994157fd0 0x7ff9941587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:40.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff999b5d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff99406db40 0x7ff994161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:40.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff999b5d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff994158d10 con 0x7ff994151590 2024-09-13T18:43:40.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff999b5d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9940c69b0 con 0x7ff994157fd0 2024-09-13T18:43:40.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff999b5d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9941572e0 con 0x7ff99406db40 2024-09-13T18:43:40.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff998b5b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff994151590 0x7ff9940c4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43654/0 (socket says 172.21.15.146:43654) 2024-09-13T18:43:40.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff998b5b700 1 -- 172.21.15.146:0/2916765709 learned_addr learned my addr 172.21.15.146:0/2916765709 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:43:40.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff993fff700 1 --2- 172.21.15.146:0/2916765709 >> v2:172.21.15.146:3300/0 conn(0x7ff994157fd0 0x7ff9941587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:40.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff99935c700 1 --2- 172.21.15.146:0/2916765709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff99406db40 0x7ff994161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:40.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff9937fe700 1 -- 172.21.15.146:0/2916765709 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608937868 0 0) 0x7ff994158d10 con 0x7ff994151590 2024-09-13T18:43:40.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff9937fe700 1 -- 172.21.15.146:0/2916765709 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff97c0035d0 con 0x7ff994151590 2024-09-13T18:43:40.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.455+0000 7ff993fff700 1 -- 172.21.15.146:0/2916765709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff99406db40 msgr2=0x7ff994161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:40.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff993fff700 1 --2- 172.21.15.146:0/2916765709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff99406db40 0x7ff994161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:40.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff993fff700 1 -- 172.21.15.146:0/2916765709 >> v1:172.21.15.146:6789/0 conn(0x7ff994151590 legacy=0x7ff9940c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:40.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff993fff700 1 -- 172.21.15.146:0/2916765709 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff994066500 con 0x7ff994157fd0 2024-09-13T18:43:40.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff99935c700 1 --2- 172.21.15.146:0/2916765709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff99406db40 0x7ff994161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:43:40.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff993fff700 1 --2- 172.21.15.146:0/2916765709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff994157fd0 0x7ff9941587d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff988009850 tx=0x7ff988009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e281b22657aaab75 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:40.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff9937fe700 1 -- 172.21.15.146:0/2916765709 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988011830 con 0x7ff994157fd0 2024-09-13T18:43:40.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff9937fe700 1 -- 172.21.15.146:0/2916765709 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff988011990 con 0x7ff994157fd0 2024-09-13T18:43:40.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff9937fe700 1 -- 172.21.15.146:0/2916765709 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988011d40 con 0x7ff994157fd0 2024-09-13T18:43:40.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff999b5d700 1 -- 172.21.15.146:0/2916765709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff994157fd0 msgr2=0x7ff9941587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:40.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff999b5d700 1 --2- 172.21.15.146:0/2916765709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff994157fd0 0x7ff9941587d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff988009850 tx=0x7ff988009b60 comp rx=0 tx=0).stop 2024-09-13T18:43:40.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff999b5d700 1 -- 172.21.15.146:0/2916765709 shutdown_connections 2024-09-13T18:43:40.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff999b5d700 1 --2- 172.21.15.146:0/2916765709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff99406db40 0x7ff994161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:40.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff999b5d700 1 --2- 172.21.15.146:0/2916765709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff994157fd0 0x7ff9941587d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:40.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff999b5d700 1 -- 172.21.15.146:0/2916765709 >> 172.21.15.146:0/2916765709 conn(0x7ff994066b00 msgr2=0x7ff99406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:40.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff999b5d700 1 -- 172.21.15.146:0/2916765709 shutdown_connections 2024-09-13T18:43:40.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff999b5d700 1 -- 172.21.15.146:0/2916765709 wait complete. 2024-09-13T18:43:40.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff999b5d700 1 Processor -- start 2024-09-13T18:43:40.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.459+0000 7ff999b5d700 1 -- start start 2024-09-13T18:43:40.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff999b5d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff99406db40 0x7ff994144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:40.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff999b5d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff994151590 0x7ff994153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:40.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff999b5d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff994157fd0 0x7ff9941f0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:40.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff999b5d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940c6d90 con 0x7ff994157fd0 2024-09-13T18:43:40.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff999b5d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9941546c0 con 0x7ff99406db40 2024-09-13T18:43:40.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff999b5d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff994152310 con 0x7ff994151590 2024-09-13T18:43:40.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff99935c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff994157fd0 0x7ff9941f0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:40.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff998b5b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff99406db40 0x7ff994144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:40.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff99935c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff994157fd0 0x7ff9941f0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59296/0 (socket says 172.21.15.146:59296) 2024-09-13T18:43:40.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff99935c700 1 -- 172.21.15.146:0/534950340 learned_addr learned my addr 172.21.15.146:0/534950340 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:40.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff993fff700 1 --2- 172.21.15.146:0/534950340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff994151590 0x7ff994153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:40.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff99935c700 1 -- 172.21.15.146:0/534950340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff994151590 msgr2=0x7ff994153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:40.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff99935c700 1 --2- 172.21.15.146:0/534950340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff994151590 0x7ff994153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:40.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff99935c700 1 -- 172.21.15.146:0/534950340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff99406db40 msgr2=0x7ff994144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:40.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff99935c700 1 --2- 172.21.15.146:0/534950340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff99406db40 0x7ff994144130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:40.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff99935c700 1 -- 172.21.15.146:0/534950340 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c009a70 con 0x7ff994157fd0 2024-09-13T18:43:40.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff99935c700 1 --2- 172.21.15.146:0/534950340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff994157fd0 0x7ff9941f0080 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00efc0 tx=0x7ff98c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:40.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff9917fa700 1 -- 172.21.15.146:0/534950340 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00cb40 con 0x7ff994157fd0 2024-09-13T18:43:40.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff9917fa700 1 -- 172.21.15.146:0/534950340 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c00f040 con 0x7ff994157fd0 2024-09-13T18:43:40.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff9917fa700 1 -- 172.21.15.146:0/534950340 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c003a40 con 0x7ff994157fd0 2024-09-13T18:43:40.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff999b5d700 1 -- 172.21.15.146:0/534950340 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9880094b0 con 0x7ff994157fd0 2024-09-13T18:43:40.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff999b5d700 1 -- 172.21.15.146:0/534950340 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff994154930 con 0x7ff994157fd0 2024-09-13T18:43:40.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff999b5d700 1 -- 172.21.15.146:0/534950340 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9940c27b0 con 0x7ff994157fd0 2024-09-13T18:43:40.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.463+0000 7ff9917fa700 1 -- 172.21.15.146:0/534950340 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff98c014480 con 0x7ff994157fd0 2024-09-13T18:43:40.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.467+0000 7ff9917fa700 1 --2- 172.21.15.146:0/534950340 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff980066730 0x7ff980068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:40.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.467+0000 7ff9917fa700 1 -- 172.21.15.146:0/534950340 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7ff98c085130 con 0x7ff994157fd0 2024-09-13T18:43:40.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.467+0000 7ff998b5b700 1 --2- 172.21.15.146:0/534950340 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff980066730 0x7ff980068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:40.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.467+0000 7ff998b5b700 1 --2- 172.21.15.146:0/534950340 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff980066730 0x7ff980068bf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff984007fd0 tx=0x7ff984009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:40.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.471+0000 7ff9917fa700 1 -- 172.21.15.146:0/534950340 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff98c051e70 con 0x7ff994157fd0 2024-09-13T18:43:40.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.691+0000 7ff999b5d700 1 -- 172.21.15.146:0/534950340 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff994154b50 con 0x7ff994157fd0 2024-09-13T18:43:40.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.695+0000 7ff9917fa700 1 -- 172.21.15.146:0/534950340 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+5530 (secure 0 0 0) 0x7ff98c009d70 con 0x7ff994157fd0 2024-09-13T18:43:40.716 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:40.716 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:36.435329+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:40.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.695+0000 7ff999b5d700 1 -- 172.21.15.146:0/534950340 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff980066730 msgr2=0x7ff980068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:40.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.695+0000 7ff999b5d700 1 --2- 172.21.15.146:0/534950340 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff980066730 0x7ff980068bf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff984007fd0 tx=0x7ff984009040 comp rx=0 tx=0).stop 2024-09-13T18:43:40.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.695+0000 7ff999b5d700 1 -- 172.21.15.146:0/534950340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff994157fd0 msgr2=0x7ff9941f0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:40.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.695+0000 7ff999b5d700 1 --2- 172.21.15.146:0/534950340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff994157fd0 0x7ff9941f0080 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00efc0 tx=0x7ff98c00c3a0 comp rx=0 tx=0).stop 2024-09-13T18:43:40.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.695+0000 7ff999b5d700 1 -- 172.21.15.146:0/534950340 shutdown_connections 2024-09-13T18:43:40.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.695+0000 7ff999b5d700 1 --2- 172.21.15.146:0/534950340 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff980066730 0x7ff980068bf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:40.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.695+0000 7ff999b5d700 1 --2- 172.21.15.146:0/534950340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff994157fd0 0x7ff9941f0080 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:40.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.695+0000 7ff999b5d700 1 --2- 172.21.15.146:0/534950340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff994151590 0x7ff994153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:40.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.695+0000 7ff999b5d700 1 --2- 172.21.15.146:0/534950340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff99406db40 0x7ff994144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:40.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.695+0000 7ff999b5d700 1 -- 172.21.15.146:0/534950340 >> 172.21.15.146:0/534950340 conn(0x7ff994066b00 msgr2=0x7ff9940c7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:40.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.695+0000 7ff999b5d700 1 -- 172.21.15.146:0/534950340 shutdown_connections 2024-09-13T18:43:40.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:40.695+0000 7ff999b5d700 1 -- 172.21.15.146:0/534950340 wait complete. 2024-09-13T18:43:40.719 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 68 2024-09-13T18:43:40.730 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2024-09-13T18:43:42.730 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:42.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 Processor -- start 2024-09-13T18:43:42.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 -- start start 2024-09-13T18:43:42.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8a04153860 0x7f8a040c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:42.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8a040c7970 0x7f8a0406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:42.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a040c98d0 con 0x7f8a0406c5b0 2024-09-13T18:43:42.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a040c4200 con 0x7f8a04153860 2024-09-13T18:43:42.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a040c8b20 con 0x7f8a040c7970 2024-09-13T18:43:42.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a03fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8a04153860 0x7f8a040c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:42.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a03fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8a04153860 0x7f8a040c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36686/0 (socket says 172.21.15.146:36686) 2024-09-13T18:43:42.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a03fff700 1 -- 172.21.15.146:0/2894250171 learned_addr learned my addr 172.21.15.146:0/2894250171 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:42.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a037fe700 1 --2- 172.21.15.146:0/2894250171 >> v2:172.21.15.154:3300/0 conn(0x7f8a040c7970 0x7f8a0406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:42.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a03fff700 1 -- 172.21.15.146:0/2894250171 >> v2:172.21.15.154:3300/0 conn(0x7f8a040c7970 msgr2=0x7f8a0406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a03fff700 1 --2- 172.21.15.146:0/2894250171 >> v2:172.21.15.154:3300/0 conn(0x7f8a040c7970 0x7f8a0406c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a03fff700 1 -- 172.21.15.146:0/2894250171 >> v1:172.21.15.154:6789/0 conn(0x7f8a0406c5b0 legacy=0x7f8a04070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a03fff700 1 -- 172.21.15.146:0/2894250171 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a040706d0 con 0x7f8a04153860 2024-09-13T18:43:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a037fe700 1 --2- 172.21.15.146:0/2894250171 >> v2:172.21.15.154:3300/0 conn(0x7f8a040c7970 0x7f8a0406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:43:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a03fff700 1 --2- 172.21.15.146:0/2894250171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a04153860 0x7f8a040c85e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8a0406d190 tx=0x7f89fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d09ead998475f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a02ffd700 1 -- 172.21.15.146:0/2894250171 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00e070 con 0x7f8a04153860 2024-09-13T18:43:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a02ffd700 1 -- 172.21.15.146:0/2894250171 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc003a40 con 0x7f8a04153860 2024-09-13T18:43:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a02ffd700 1 -- 172.21.15.146:0/2894250171 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc003df0 con 0x7f8a04153860 2024-09-13T18:43:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 -- 172.21.15.146:0/2894250171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a04153860 msgr2=0x7f8a040c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 --2- 172.21.15.146:0/2894250171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a04153860 0x7f8a040c85e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8a0406d190 tx=0x7f89fc00b330 comp rx=0 tx=0).stop 2024-09-13T18:43:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 -- 172.21.15.146:0/2894250171 shutdown_connections 2024-09-13T18:43:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 --2- 172.21.15.146:0/2894250171 >> v2:172.21.15.154:3300/0 conn(0x7f8a040c7970 0x7f8a0406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 --2- 172.21.15.146:0/2894250171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a04153860 0x7f8a040c85e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:42.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 -- 172.21.15.146:0/2894250171 >> 172.21.15.146:0/2894250171 conn(0x7f8a040ccf70 msgr2=0x7f8a040cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:42.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 -- 172.21.15.146:0/2894250171 shutdown_connections 2024-09-13T18:43:42.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.827+0000 7f8a0975c700 1 -- 172.21.15.146:0/2894250171 wait complete. 2024-09-13T18:43:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a0975c700 1 Processor -- start 2024-09-13T18:43:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a0975c700 1 -- start start 2024-09-13T18:43:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a0975c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a0406c5b0 0x7f8a041f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a0975c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a040c7970 0x7f8a040c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a0975c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a04153860 0x7f8a0414b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a0975c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a040c6d50 con 0x7f8a04153860 2024-09-13T18:43:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a0975c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a040c71e0 con 0x7f8a040c7970 2024-09-13T18:43:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a0975c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a041f2f10 con 0x7f8a0406c5b0 2024-09-13T18:43:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a08f5b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a04153860 0x7f8a0414b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a08f5b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a04153860 0x7f8a0414b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36694/0 (socket says 172.21.15.146:36694) 2024-09-13T18:43:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a08f5b700 1 -- 172.21.15.146:0/3087765991 learned_addr learned my addr 172.21.15.146:0/3087765991 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a037fe700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a040c7970 0x7f8a040c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a03fff700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a0406c5b0 0x7f8a041f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a08f5b700 1 -- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a0406c5b0 msgr2=0x7f8a041f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a08f5b700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a0406c5b0 0x7f8a041f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a08f5b700 1 -- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a040c7970 msgr2=0x7f8a040c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a08f5b700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a040c7970 0x7f8a040c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a08f5b700 1 -- 172.21.15.146:0/3087765991 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc009b30 con 0x7f8a04153860 2024-09-13T18:43:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a08f5b700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a04153860 0x7f8a0414b670 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f89f800c470 tx=0x7f89f800c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a00ff9700 1 -- 172.21.15.146:0/3087765991 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8011790 con 0x7f8a04153860 2024-09-13T18:43:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a00ff9700 1 -- 172.21.15.146:0/3087765991 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f80118f0 con 0x7f8a04153860 2024-09-13T18:43:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a0975c700 1 -- 172.21.15.146:0/3087765991 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a04153d60 con 0x7f8a04153860 2024-09-13T18:43:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a00ff9700 1 -- 172.21.15.146:0/3087765991 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8011ca0 con 0x7f8a04153860 2024-09-13T18:43:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.831+0000 7f8a0975c700 1 -- 172.21.15.146:0/3087765991 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a04154020 con 0x7f8a04153860 2024-09-13T18:43:42.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.835+0000 7f8a0975c700 1 -- 172.21.15.146:0/3087765991 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a04067a60 con 0x7f8a04153860 2024-09-13T18:43:42.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.835+0000 7f8a00ff9700 1 -- 172.21.15.146:0/3087765991 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f89f801c460 con 0x7f8a04153860 2024-09-13T18:43:42.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.835+0000 7f8a00ff9700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f89f0066730 0x7f89f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:42.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.835+0000 7f8a00ff9700 1 -- 172.21.15.146:0/3087765991 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f89f80844a0 con 0x7f8a04153860 2024-09-13T18:43:42.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.835+0000 7f8a03fff700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f89f0066730 0x7f89f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:42.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.835+0000 7f8a03fff700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f89f0066730 0x7f89f0068bf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f89fc015040 tx=0x7f89fc009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:42.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:42.843+0000 7f8a00ff9700 1 -- 172.21.15.146:0/3087765991 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f89f8051160 con 0x7f8a04153860 2024-09-13T18:43:43.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.067+0000 7f8a0975c700 1 -- 172.21.15.146:0/3087765991 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a040cb610 con 0x7f8a04153860 2024-09-13T18:43:43.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.071+0000 7f8a00ff9700 1 -- 172.21.15.146:0/3087765991 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+6290 (secure 0 0 0) 0x7f89f8054780 con 0x7f8a04153860 2024-09-13T18:43:43.091 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:43.092 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:36.435329+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:43.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.071+0000 7f8a0975c700 1 -- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f89f0066730 msgr2=0x7f89f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:43.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.071+0000 7f8a0975c700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f89f0066730 0x7f89f0068bf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f89fc015040 tx=0x7f89fc009000 comp rx=0 tx=0).stop 2024-09-13T18:43:43.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.071+0000 7f8a0975c700 1 -- 172.21.15.146:0/3087765991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a04153860 msgr2=0x7f8a0414b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:43.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.071+0000 7f8a0975c700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a04153860 0x7f8a0414b670 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f89f800c470 tx=0x7f89f800c780 comp rx=0 tx=0).stop 2024-09-13T18:43:43.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.071+0000 7f8a0975c700 1 -- 172.21.15.146:0/3087765991 shutdown_connections 2024-09-13T18:43:43.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.071+0000 7f8a0975c700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f89f0066730 0x7f89f0068bf0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:43.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.071+0000 7f8a0975c700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a04153860 0x7f8a0414b670 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:43.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.071+0000 7f8a0975c700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a040c7970 0x7f8a040c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:43.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.071+0000 7f8a0975c700 1 --2- 172.21.15.146:0/3087765991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a0406c5b0 0x7f8a041f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:43.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.071+0000 7f8a0975c700 1 -- 172.21.15.146:0/3087765991 >> 172.21.15.146:0/3087765991 conn(0x7f8a040ccf70 msgr2=0x7f8a040cd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:43.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.071+0000 7f8a0975c700 1 -- 172.21.15.146:0/3087765991 shutdown_connections 2024-09-13T18:43:43.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:43.071+0000 7f8a0975c700 1 -- 172.21.15.146:0/3087765991 wait complete. 2024-09-13T18:43:43.096 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 69 2024-09-13T18:43:43.107 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2024-09-13T18:43:43.108 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 13 secs before thrashing 2024-09-13T18:43:56.109 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:56.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.211+0000 7fda7973b700 1 Processor -- start 2024-09-13T18:43:56.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.211+0000 7fda7973b700 1 -- start start 2024-09-13T18:43:56.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.211+0000 7fda7973b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda74151590 0x7fda741587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:56.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.211+0000 7fda7973b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fda7406db40 0x7fda74161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:56.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.211+0000 7fda7973b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda74158d10 con 0x7fda74157fd0 2024-09-13T18:43:56.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.211+0000 7fda7973b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda740c69b0 con 0x7fda7406db40 2024-09-13T18:43:56.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.211+0000 7fda7973b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda741572e0 con 0x7fda74151590 2024-09-13T18:43:56.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda737fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fda74157fd0 0x7fda740c4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56812/0 (socket says 172.21.15.146:56812) 2024-09-13T18:43:56.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda737fe700 1 -- 172.21.15.146:0/4220789845 learned_addr learned my addr 172.21.15.146:0/4220789845 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:43:56.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda78f3a700 1 --2- 172.21.15.146:0/4220789845 >> v2:172.21.15.146:3300/0 conn(0x7fda7406db40 0x7fda74161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:56.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda73fff700 1 --2- 172.21.15.146:0/4220789845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda74151590 0x7fda741587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:56.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda72ffd700 1 -- 172.21.15.146:0/4220789845 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642915226 0 0) 0x7fda74158d10 con 0x7fda74157fd0 2024-09-13T18:43:56.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda72ffd700 1 -- 172.21.15.146:0/4220789845 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda5c0035d0 con 0x7fda74157fd0 2024-09-13T18:43:56.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda78f3a700 1 -- 172.21.15.146:0/4220789845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda74151590 msgr2=0x7fda741587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:56.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda78f3a700 1 --2- 172.21.15.146:0/4220789845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda74151590 0x7fda741587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda78f3a700 1 -- 172.21.15.146:0/4220789845 >> v1:172.21.15.146:6789/0 conn(0x7fda74157fd0 legacy=0x7fda740c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:56.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda78f3a700 1 -- 172.21.15.146:0/4220789845 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda74066500 con 0x7fda7406db40 2024-09-13T18:43:56.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda73fff700 1 --2- 172.21.15.146:0/4220789845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda74151590 0x7fda741587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:43:56.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda78f3a700 1 --2- 172.21.15.146:0/4220789845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda7406db40 0x7fda74161c60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009d90 tx=0x7fda6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3a3534b00e968a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:56.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda72ffd700 1 -- 172.21.15.146:0/4220789845 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c00e070 con 0x7fda7406db40 2024-09-13T18:43:56.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda72ffd700 1 -- 172.21.15.146:0/4220789845 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c00bb20 con 0x7fda7406db40 2024-09-13T18:43:56.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda72ffd700 1 -- 172.21.15.146:0/4220789845 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c012430 con 0x7fda7406db40 2024-09-13T18:43:56.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda7973b700 1 -- 172.21.15.146:0/4220789845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda7406db40 msgr2=0x7fda74161c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:56.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda7973b700 1 --2- 172.21.15.146:0/4220789845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda7406db40 0x7fda74161c60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009d90 tx=0x7fda6c00b330 comp rx=0 tx=0).stop 2024-09-13T18:43:56.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda7973b700 1 -- 172.21.15.146:0/4220789845 shutdown_connections 2024-09-13T18:43:56.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda7973b700 1 --2- 172.21.15.146:0/4220789845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda7406db40 0x7fda74161c60 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda7973b700 1 --2- 172.21.15.146:0/4220789845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda74151590 0x7fda741587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda7973b700 1 -- 172.21.15.146:0/4220789845 >> 172.21.15.146:0/4220789845 conn(0x7fda74066b00 msgr2=0x7fda7406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:56.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda7973b700 1 -- 172.21.15.146:0/4220789845 shutdown_connections 2024-09-13T18:43:56.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda7973b700 1 -- 172.21.15.146:0/4220789845 wait complete. 2024-09-13T18:43:56.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.215+0000 7fda7973b700 1 Processor -- start 2024-09-13T18:43:56.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda7973b700 1 -- start start 2024-09-13T18:43:56.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda7973b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda7406db40 0x7fda74149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:56.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda7973b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda74151590 0x7fda7414b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:56.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda7973b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda74157fd0 0x7fda7414bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:56.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda7973b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda741edd40 con 0x7fda74151590 2024-09-13T18:43:56.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda7973b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda74154510 con 0x7fda7406db40 2024-09-13T18:43:56.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda7973b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7415b600 con 0x7fda74157fd0 2024-09-13T18:43:56.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda737fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda74151590 0x7fda7414b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:56.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda737fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda74151590 0x7fda7414b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43744/0 (socket says 172.21.15.146:43744) 2024-09-13T18:43:56.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda737fe700 1 -- 172.21.15.146:0/890496019 learned_addr learned my addr 172.21.15.146:0/890496019 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:56.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda73fff700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda7406db40 0x7fda74149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:56.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda78f3a700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda74157fd0 0x7fda7414bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:56.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda737fe700 1 -- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda74157fd0 msgr2=0x7fda7414bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:56.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda737fe700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda74157fd0 0x7fda7414bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda737fe700 1 -- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda7406db40 msgr2=0x7fda74149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:56.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda737fe700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda7406db40 0x7fda74149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda737fe700 1 -- 172.21.15.146:0/890496019 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c009a70 con 0x7fda74151590 2024-09-13T18:43:56.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda737fe700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda74151590 0x7fda7414b5f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fda68005930 tx=0x7fda6800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:56.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda70ff9700 1 -- 172.21.15.146:0/890496019 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68005d90 con 0x7fda74151590 2024-09-13T18:43:56.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda70ff9700 1 -- 172.21.15.146:0/890496019 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda680070d0 con 0x7fda74151590 2024-09-13T18:43:56.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda70ff9700 1 -- 172.21.15.146:0/890496019 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68007480 con 0x7fda74151590 2024-09-13T18:43:56.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda7973b700 1 -- 172.21.15.146:0/890496019 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda741ed1c0 con 0x7fda74151590 2024-09-13T18:43:56.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda7973b700 1 -- 172.21.15.146:0/890496019 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda741ed780 con 0x7fda74151590 2024-09-13T18:43:56.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.219+0000 7fda7973b700 1 -- 172.21.15.146:0/890496019 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda740c6250 con 0x7fda74151590 2024-09-13T18:43:56.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.223+0000 7fda70ff9700 1 -- 172.21.15.146:0/890496019 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fda680075e0 con 0x7fda74151590 2024-09-13T18:43:56.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.223+0000 7fda70ff9700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fda60066730 0x7fda60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:56.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.223+0000 7fda70ff9700 1 -- 172.21.15.146:0/890496019 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fda68083e00 con 0x7fda74151590 2024-09-13T18:43:56.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.223+0000 7fda73fff700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fda60066730 0x7fda60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:56.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.223+0000 7fda73fff700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fda60066730 0x7fda60068bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fda64009950 tx=0x7fda64008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:56.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.231+0000 7fda70ff9700 1 -- 172.21.15.146:0/890496019 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fda68050a30 con 0x7fda74151590 2024-09-13T18:43:56.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.463+0000 7fda7973b700 1 -- 172.21.15.146:0/890496019 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda74157ab0 con 0x7fda74151590 2024-09-13T18:43:56.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.467+0000 7fda70ff9700 1 -- 172.21.15.146:0/890496019 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+6290 (secure 0 0 0) 0x7fda68054050 con 0x7fda74151590 2024-09-13T18:43:56.488 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:56.488 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:36.435329+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:active","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:56.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.467+0000 7fda7973b700 1 -- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fda60066730 msgr2=0x7fda60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:56.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.467+0000 7fda7973b700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fda60066730 0x7fda60068bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fda64009950 tx=0x7fda64008040 comp rx=0 tx=0).stop 2024-09-13T18:43:56.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.467+0000 7fda7973b700 1 -- 172.21.15.146:0/890496019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda74151590 msgr2=0x7fda7414b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:56.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.467+0000 7fda7973b700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda74151590 0x7fda7414b5f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fda68005930 tx=0x7fda6800c860 comp rx=0 tx=0).stop 2024-09-13T18:43:56.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.467+0000 7fda7973b700 1 -- 172.21.15.146:0/890496019 shutdown_connections 2024-09-13T18:43:56.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.467+0000 7fda7973b700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fda60066730 0x7fda60068bf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.467+0000 7fda7973b700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda74157fd0 0x7fda7414bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.467+0000 7fda7973b700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda74151590 0x7fda7414b5f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.467+0000 7fda7973b700 1 --2- 172.21.15.146:0/890496019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda7406db40 0x7fda74149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.467+0000 7fda7973b700 1 -- 172.21.15.146:0/890496019 >> 172.21.15.146:0/890496019 conn(0x7fda74066b00 msgr2=0x7fda74155e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:56.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.467+0000 7fda7973b700 1 -- 172.21.15.146:0/890496019 shutdown_connections 2024-09-13T18:43:56.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.467+0000 7fda7973b700 1 -- 172.21.15.146:0/890496019 wait complete. 2024-09-13T18:43:56.492 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 69 2024-09-13T18:43:56.503 INFO:tasks.mds_thrash.fs.[cephfs]:thrashing max_mds: 3 -> 2 2024-09-13T18:43:56.503 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-09-13T18:43:56.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.595+0000 7ff7bebc1700 1 Processor -- start 2024-09-13T18:43:56.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.595+0000 7ff7bebc1700 1 -- start start 2024-09-13T18:43:56.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.595+0000 7ff7bebc1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff7b80c15d0 0x7ff7b8159a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:56.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.595+0000 7ff7bebc1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7b80c2e40 0x7ff7b8159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:56.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.595+0000 7ff7bebc1700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b80c7070 con 0x7ff7b80c11f0 2024-09-13T18:43:56.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.595+0000 7ff7bebc1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b8159630 con 0x7ff7b80c15d0 2024-09-13T18:43:56.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.595+0000 7ff7bebc1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b806e480 con 0x7ff7b80c2e40 2024-09-13T18:43:56.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.595+0000 7ff7bdbbf700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff7b80c11f0 0x7ff7b80c6960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56826/0 (socket says 172.21.15.146:56826) 2024-09-13T18:43:56.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.595+0000 7ff7bdbbf700 1 -- 172.21.15.146:0/2569840277 learned_addr learned my addr 172.21.15.146:0/2569840277 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:43:56.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.595+0000 7ff7bd3be700 1 --2- 172.21.15.146:0/2569840277 >> v2:172.21.15.154:3300/0 conn(0x7ff7b80c15d0 0x7ff7b8159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:56.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.595+0000 7ff7be3c0700 1 --2- 172.21.15.146:0/2569840277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7b80c2e40 0x7ff7b8159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:56.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.595+0000 7ff7bcbbd700 1 -- 172.21.15.146:0/2569840277 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227463645 0 0) 0x7ff7b80c7070 con 0x7ff7b80c11f0 2024-09-13T18:43:56.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.595+0000 7ff7bcbbd700 1 -- 172.21.15.146:0/2569840277 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a40035d0 con 0x7ff7b80c11f0 2024-09-13T18:43:56.621 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bd3be700 1 -- 172.21.15.146:0/2569840277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7b80c2e40 msgr2=0x7ff7b8159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:56.621 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bd3be700 1 --2- 172.21.15.146:0/2569840277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7b80c2e40 0x7ff7b8159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.621 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bd3be700 1 -- 172.21.15.146:0/2569840277 >> v1:172.21.15.146:6789/0 conn(0x7ff7b80c11f0 legacy=0x7ff7b80c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:56.621 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bcbbd700 1 -- 172.21.15.146:0/2569840277 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1981915409 0 0) 0x7ff7a40035d0 con 0x7ff7b80c11f0 2024-09-13T18:43:56.621 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bd3be700 1 -- 172.21.15.146:0/2569840277 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b806e720 con 0x7ff7b80c15d0 2024-09-13T18:43:56.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bd3be700 1 --2- 172.21.15.146:0/2569840277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b80c15d0 0x7ff7b8159a80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009850 tx=0x7ff7a8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab519a144781ec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:56.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bcbbd700 1 -- 172.21.15.146:0/2569840277 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4001670 con 0x7ff7b80c15d0 2024-09-13T18:43:56.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bcbbd700 1 -- 172.21.15.146:0/2569840277 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a8004030 con 0x7ff7b80c15d0 2024-09-13T18:43:56.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bcbbd700 1 -- 172.21.15.146:0/2569840277 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80043a0 con 0x7ff7b80c15d0 2024-09-13T18:43:56.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bebc1700 1 -- 172.21.15.146:0/2569840277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b80c15d0 msgr2=0x7ff7b8159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:56.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bebc1700 1 --2- 172.21.15.146:0/2569840277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b80c15d0 0x7ff7b8159a80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009850 tx=0x7ff7a8009b60 comp rx=0 tx=0).stop 2024-09-13T18:43:56.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bebc1700 1 -- 172.21.15.146:0/2569840277 shutdown_connections 2024-09-13T18:43:56.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bebc1700 1 --2- 172.21.15.146:0/2569840277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7b80c2e40 0x7ff7b8159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bebc1700 1 --2- 172.21.15.146:0/2569840277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b80c15d0 0x7ff7b8159a80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bebc1700 1 -- 172.21.15.146:0/2569840277 >> 172.21.15.146:0/2569840277 conn(0x7ff7b8071ed0 msgr2=0x7ff7b80722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:56.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bebc1700 1 -- 172.21.15.146:0/2569840277 shutdown_connections 2024-09-13T18:43:56.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bebc1700 1 -- 172.21.15.146:0/2569840277 wait complete. 2024-09-13T18:43:56.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bebc1700 1 Processor -- start 2024-09-13T18:43:56.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.599+0000 7ff7bebc1700 1 -- start start 2024-09-13T18:43:56.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7bebc1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7b80c11f0 0x7ff7b80c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:56.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7bebc1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b80c15d0 0x7ff7b80c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:56.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7bebc1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b80c2e40 0x7ff7b81ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:56.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7bebc1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81eeb00 con 0x7ff7b80c2e40 2024-09-13T18:43:56.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7bebc1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81512d0 con 0x7ff7b80c15d0 2024-09-13T18:43:56.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7bebc1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81eecf0 con 0x7ff7b80c11f0 2024-09-13T18:43:56.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7be3c0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b80c2e40 0x7ff7b81ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:56.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7be3c0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b80c2e40 0x7ff7b81ef1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43748/0 (socket says 172.21.15.146:43748) 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7be3c0700 1 -- 172.21.15.146:0/1093224641 learned_addr learned my addr 172.21.15.146:0/1093224641 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7bdbbf700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7b80c11f0 0x7ff7b80c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7bd3be700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b80c15d0 0x7ff7b80c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7be3c0700 1 -- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7b80c11f0 msgr2=0x7ff7b80c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7be3c0700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7b80c11f0 0x7ff7b80c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7be3c0700 1 -- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b80c15d0 msgr2=0x7ff7b80c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7be3c0700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b80c15d0 0x7ff7b80c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7be3c0700 1 -- 172.21.15.146:0/1093224641 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a80094b0 con 0x7ff7b80c2e40 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7bdbbf700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7b80c11f0 0x7ff7b80c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7bd3be700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b80c15d0 0x7ff7b80c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7be3c0700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b80c2e40 0x7ff7b81ef1f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000ec10 tx=0x7ff7b000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7ae7fc700 1 -- 172.21.15.146:0/1093224641 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000cab0 con 0x7ff7b80c2e40 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7ae7fc700 1 -- 172.21.15.146:0/1093224641 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b000f040 con 0x7ff7b80c2e40 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7ae7fc700 1 -- 172.21.15.146:0/1093224641 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000ce50 con 0x7ff7b80c2e40 2024-09-13T18:43:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7bebc1700 1 -- 172.21.15.146:0/1093224641 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b8152920 con 0x7ff7b80c2e40 2024-09-13T18:43:56.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7bebc1700 1 -- 172.21.15.146:0/1093224641 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b81f2a80 con 0x7ff7b80c2e40 2024-09-13T18:43:56.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7bebc1700 1 -- 172.21.15.146:0/1093224641 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b81eeec0 con 0x7ff7b80c2e40 2024-09-13T18:43:56.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.603+0000 7ff7ae7fc700 1 -- 172.21.15.146:0/1093224641 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff7b0014480 con 0x7ff7b80c2e40 2024-09-13T18:43:56.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.607+0000 7ff7ae7fc700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff79c066730 0x7ff79c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:56.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.607+0000 7ff7ae7fc700 1 -- 172.21.15.146:0/1093224641 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7ff7b0084f20 con 0x7ff7b80c2e40 2024-09-13T18:43:56.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.607+0000 7ff7bdbbf700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff79c066730 0x7ff79c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:56.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.607+0000 7ff7bdbbf700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff79c066730 0x7ff79c068bf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4001eb0 tx=0x7ff7b4001d00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:56.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.611+0000 7ff7ae7fc700 1 -- 172.21.15.146:0/1093224641 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff7b0051c90 con 0x7ff7b80c2e40 2024-09-13T18:43:56.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:56.847+0000 7ff7bebc1700 1 -- 172.21.15.146:0/1093224641 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7ff7b80b0e00 con 0x7ff7b80c2e40 2024-09-13T18:43:57.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.511+0000 7ff7ae7fc700 1 -- 172.21.15.146:0/1093224641 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v70) v1 ==== 105+0+0 (secure 0 0 0) 0x7ff7b0018020 con 0x7ff7b80c2e40 2024-09-13T18:43:57.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.515+0000 7ff7bebc1700 1 -- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff79c066730 msgr2=0x7ff79c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:57.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.515+0000 7ff7bebc1700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff79c066730 0x7ff79c068bf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4001eb0 tx=0x7ff7b4001d00 comp rx=0 tx=0).stop 2024-09-13T18:43:57.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.515+0000 7ff7bebc1700 1 -- 172.21.15.146:0/1093224641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b80c2e40 msgr2=0x7ff7b81ef1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:57.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.515+0000 7ff7bebc1700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b80c2e40 0x7ff7b81ef1f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000ec10 tx=0x7ff7b000ef20 comp rx=0 tx=0).stop 2024-09-13T18:43:57.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.515+0000 7ff7bebc1700 1 -- 172.21.15.146:0/1093224641 shutdown_connections 2024-09-13T18:43:57.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.515+0000 7ff7bebc1700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff79c066730 0x7ff79c068bf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.515+0000 7ff7bebc1700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b80c2e40 0x7ff7b81ef1f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.515+0000 7ff7bebc1700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b80c15d0 0x7ff7b80c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.515+0000 7ff7bebc1700 1 --2- 172.21.15.146:0/1093224641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7b80c11f0 0x7ff7b80c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.515+0000 7ff7bebc1700 1 -- 172.21.15.146:0/1093224641 >> 172.21.15.146:0/1093224641 conn(0x7ff7b8071ed0 msgr2=0x7ff7b815a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:57.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.515+0000 7ff7bebc1700 1 -- 172.21.15.146:0/1093224641 shutdown_connections 2024-09-13T18:43:57.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.515+0000 7ff7bebc1700 1 -- 172.21.15.146:0/1093224641 wait complete. 2024-09-13T18:43:57.559 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:43:57.559 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:43:57.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb1359e700 1 Processor -- start 2024-09-13T18:43:57.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb1359e700 1 -- start start 2024-09-13T18:43:57.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb1359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdb0c0d6650 0x7fdb0c044be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:57.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb1359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdb0c0509b0 0x7fdb0c045120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:57.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb1359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb0c04b9f0 0x7fdb0c0dc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:57.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb1359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c04c610 con 0x7fdb0c0509b0 2024-09-13T18:43:57.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb1359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c04b840 con 0x7fdb0c0d6650 2024-09-13T18:43:57.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb1359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c04c370 con 0x7fdb0c04b9f0 2024-09-13T18:43:57.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb11d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdb0c0509b0 0x7fdb0c045120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:57.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb11d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdb0c0509b0 0x7fdb0c045120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43750/0 (socket says 172.21.15.146:43750) 2024-09-13T18:43:57.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb11d9b700 1 -- 172.21.15.146:0/4149434816 learned_addr learned my addr 172.21.15.146:0/4149434816 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:57.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb1259c700 1 --2- 172.21.15.146:0/4149434816 >> v2:172.21.15.154:3300/0 conn(0x7fdb0c0d6650 0x7fdb0c044be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:57.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb12d9d700 1 --2- 172.21.15.146:0/4149434816 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb0c04b9f0 0x7fdb0c0dc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:57.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb11d9b700 1 -- 172.21.15.146:0/4149434816 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb0c04b9f0 msgr2=0x7fdb0c0dc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:57.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb11d9b700 1 --2- 172.21.15.146:0/4149434816 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb0c04b9f0 0x7fdb0c0dc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb11d9b700 1 -- 172.21.15.146:0/4149434816 >> v2:172.21.15.154:3300/0 conn(0x7fdb0c0d6650 msgr2=0x7fdb0c044be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:43:57.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb11d9b700 1 --2- 172.21.15.146:0/4149434816 >> v2:172.21.15.154:3300/0 conn(0x7fdb0c0d6650 0x7fdb0c044be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb11d9b700 1 -- 172.21.15.146:0/4149434816 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c04af10 con 0x7fdb0c0509b0 2024-09-13T18:43:57.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.639+0000 7fdb12d9d700 1 --2- 172.21.15.146:0/4149434816 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb0c04b9f0 0x7fdb0c0dc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:43:57.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb11d9b700 1 --2- 172.21.15.146:0/4149434816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb0c0509b0 0x7fdb0c045120 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00d410 tx=0x7fdafc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad933163fed53224 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:57.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1159a700 1 -- 172.21.15.146:0/4149434816 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc00b8a0 con 0x7fdb0c0509b0 2024-09-13T18:43:57.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1159a700 1 -- 172.21.15.146:0/4149434816 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc00ba00 con 0x7fdb0c0509b0 2024-09-13T18:43:57.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1159a700 1 -- 172.21.15.146:0/4149434816 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc00bd90 con 0x7fdb0c0509b0 2024-09-13T18:43:57.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1359e700 1 -- 172.21.15.146:0/4149434816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb0c0509b0 msgr2=0x7fdb0c045120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:57.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1359e700 1 --2- 172.21.15.146:0/4149434816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb0c0509b0 0x7fdb0c045120 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00d410 tx=0x7fdafc009bf0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1359e700 1 -- 172.21.15.146:0/4149434816 shutdown_connections 2024-09-13T18:43:57.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1359e700 1 --2- 172.21.15.146:0/4149434816 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb0c04b9f0 0x7fdb0c0dc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1359e700 1 --2- 172.21.15.146:0/4149434816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb0c0509b0 0x7fdb0c045120 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1359e700 1 --2- 172.21.15.146:0/4149434816 >> v2:172.21.15.154:3300/0 conn(0x7fdb0c0d6650 0x7fdb0c044be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1359e700 1 -- 172.21.15.146:0/4149434816 >> 172.21.15.146:0/4149434816 conn(0x7fdb0c04fd00 msgr2=0x7fdb0c050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:57.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1359e700 1 -- 172.21.15.146:0/4149434816 shutdown_connections 2024-09-13T18:43:57.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1359e700 1 -- 172.21.15.146:0/4149434816 wait complete. 2024-09-13T18:43:57.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1359e700 1 Processor -- start 2024-09-13T18:43:57.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.643+0000 7fdb1359e700 1 -- start start 2024-09-13T18:43:57.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb1359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c04b9f0 0x7fdb0c0d9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:57.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb1359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb0c0509b0 0x7fdb0c175400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:57.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb1359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb0c0d6650 0x7fdb0c048990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:57.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb1359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c172760 con 0x7fdb0c0d6650 2024-09-13T18:43:57.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb1359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0d28b0 con 0x7fdb0c04b9f0 2024-09-13T18:43:57.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb1359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0d2b50 con 0x7fdb0c0509b0 2024-09-13T18:43:57.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb12d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb0c0d6650 0x7fdb0c048990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb12d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb0c0d6650 0x7fdb0c048990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43764/0 (socket says 172.21.15.146:43764) 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb12d9d700 1 -- 172.21.15.146:0/2456487205 learned_addr learned my addr 172.21.15.146:0/2456487205 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb11d9b700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb0c0509b0 0x7fdb0c175400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb1259c700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c04b9f0 0x7fdb0c0d9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb12d9d700 1 -- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb0c0509b0 msgr2=0x7fdb0c175400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb12d9d700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb0c0509b0 0x7fdb0c175400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb12d9d700 1 -- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c04b9f0 msgr2=0x7fdb0c0d9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb12d9d700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c04b9f0 0x7fdb0c0d9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb12d9d700 1 -- 172.21.15.146:0/2456487205 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009570 con 0x7fdb0c0d6650 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb1259c700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c04b9f0 0x7fdb0c0d9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb11d9b700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb0c0509b0 0x7fdb0c175400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb12d9d700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb0c0d6650 0x7fdb0c048990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdb14075280 tx=0x7fdb140739e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb0affd700 1 -- 172.21.15.146:0/2456487205 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb14067950 con 0x7fdb0c0d6650 2024-09-13T18:43:57.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb0affd700 1 -- 172.21.15.146:0/2456487205 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb14067c50 con 0x7fdb0c0d6650 2024-09-13T18:43:57.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb1359e700 1 -- 172.21.15.146:0/2456487205 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c171e50 con 0x7fdb0c0d6650 2024-09-13T18:43:57.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb0affd700 1 -- 172.21.15.146:0/2456487205 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb14067fc0 con 0x7fdb0c0d6650 2024-09-13T18:43:57.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb1359e700 1 -- 172.21.15.146:0/2456487205 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c1723f0 con 0x7fdb0c0d6650 2024-09-13T18:43:57.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb0affd700 1 -- 172.21.15.146:0/2456487205 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdb14070740 con 0x7fdb0c0d6650 2024-09-13T18:43:57.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb1359e700 1 -- 172.21.15.146:0/2456487205 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c009be0 con 0x7fdb0c0d6650 2024-09-13T18:43:57.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb0affd700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdaf4066730 0x7fdaf4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:43:57.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb0affd700 1 -- 172.21.15.146:0/2456487205 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fdb140feea0 con 0x7fdb0c0d6650 2024-09-13T18:43:57.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.647+0000 7fdb1259c700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdaf4066730 0x7fdaf4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:43:57.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.651+0000 7fdb1259c700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdaf4066730 0x7fdaf4068bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdb04007a70 tx=0x7fdb04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:43:57.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.655+0000 7fdb0affd700 1 -- 172.21.15.146:0/2456487205 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdb140cbac0 con 0x7fdb0c0d6650 2024-09-13T18:43:57.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.887+0000 7fdb1359e700 1 -- 172.21.15.146:0/2456487205 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb0c035bd0 con 0x7fdb0c0d6650 2024-09-13T18:43:57.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.887+0000 7fdb0affd700 1 -- 172.21.15.146:0/2456487205 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6292 (secure 0 0 0) 0x7fdb140cf0e0 con 0x7fdb0c0d6650 2024-09-13T18:43:57.909 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:43:57.909 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:57.516990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:stopping","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:43:57.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.891+0000 7fdb1359e700 1 -- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdaf4066730 msgr2=0x7fdaf4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:57.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.891+0000 7fdb1359e700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdaf4066730 0x7fdaf4068bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdb04007a70 tx=0x7fdb04008040 comp rx=0 tx=0).stop 2024-09-13T18:43:57.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.891+0000 7fdb1359e700 1 -- 172.21.15.146:0/2456487205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb0c0d6650 msgr2=0x7fdb0c048990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:43:57.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.891+0000 7fdb1359e700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb0c0d6650 0x7fdb0c048990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdb14075280 tx=0x7fdb140739e0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.891+0000 7fdb1359e700 1 -- 172.21.15.146:0/2456487205 shutdown_connections 2024-09-13T18:43:57.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.891+0000 7fdb1359e700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdaf4066730 0x7fdaf4068bf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.891+0000 7fdb1359e700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb0c0d6650 0x7fdb0c048990 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.891+0000 7fdb1359e700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb0c0509b0 0x7fdb0c175400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.891+0000 7fdb1359e700 1 --2- 172.21.15.146:0/2456487205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c04b9f0 0x7fdb0c0d9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:43:57.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.891+0000 7fdb1359e700 1 -- 172.21.15.146:0/2456487205 >> 172.21.15.146:0/2456487205 conn(0x7fdb0c04fd00 msgr2=0x7fdb0c044be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:43:57.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.891+0000 7fdb1359e700 1 -- 172.21.15.146:0/2456487205 shutdown_connections 2024-09-13T18:43:57.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:43:57.891+0000 7fdb1359e700 1 -- 172.21.15.146:0/2456487205 wait complete. 2024-09-13T18:43:57.913 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 71 2024-09-13T18:43:57.924 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-09-13T18:43:57.925 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 71, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 71, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5800": { "addr": "172.21.15.154:6839/776638146", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 776638146, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 776638146, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5800, "incarnation": 52, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 30 }, "gid_6063": { "addr": "172.21.15.146:6835/832794630", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 832794630, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 832794630, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6063, "incarnation": 58, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:stopping", "state_seq": 43 }, "gid_6207": { "addr": "172.21.15.146:6837/3234109877", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3234109877, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3234109877, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6207, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 39 } }, "last_failure": 0, "last_failure_osd_epoch": 36, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:43:57.516990+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5800, "mds_1": 6207, "mds_2": 6063 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.146:6833/3023779168", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3023779168, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3023779168, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 69, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6187, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:43:59.926 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:00.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169a53700 1 Processor -- start 2024-09-13T18:44:00.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169a53700 1 -- start start 2024-09-13T18:44:00.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169a53700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1640c6590 0x7fa16415d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:00.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169a53700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa1640c5440 0x7fa164153340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:00.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169a53700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa164066780 con 0x7fa1640c61b0 2024-09-13T18:44:00.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169a53700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1641594a0 con 0x7fa1640c5440 2024-09-13T18:44:00.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169a53700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16415d960 con 0x7fa1640c6590 2024-09-13T18:44:00.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169252700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa1640c5440 0x7fa164153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:00.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169252700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa1640c5440 0x7fa164153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43766/0 (socket says 172.21.15.146:43766) 2024-09-13T18:44:00.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169252700 1 -- 172.21.15.146:0/3757662484 learned_addr learned my addr 172.21.15.146:0/3757662484 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:00.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa163fff700 1 --2- 172.21.15.146:0/3757662484 >> v2:172.21.15.154:3300/0 conn(0x7fa1640c6590 0x7fa16415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:00.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169252700 1 -- 172.21.15.146:0/3757662484 >> v2:172.21.15.154:3300/0 conn(0x7fa1640c6590 msgr2=0x7fa16415d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:00.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169252700 1 --2- 172.21.15.146:0/3757662484 >> v2:172.21.15.154:3300/0 conn(0x7fa1640c6590 0x7fa16415d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:00.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169252700 1 -- 172.21.15.146:0/3757662484 >> v1:172.21.15.154:6789/0 conn(0x7fa1640c61b0 legacy=0x7fa16406cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:00.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169252700 1 -- 172.21.15.146:0/3757662484 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16406eec0 con 0x7fa1640c5440 2024-09-13T18:44:00.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa163fff700 1 --2- 172.21.15.146:0/3757662484 >> v2:172.21.15.154:3300/0 conn(0x7fa1640c6590 0x7fa16415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:00.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa169252700 1 --2- 172.21.15.146:0/3757662484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1640c5440 0x7fa164153340 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa16406ebb0 tx=0x7fa15c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d28069e490be59 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:00.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa1637fe700 1 -- 172.21.15.146:0/3757662484 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00e070 con 0x7fa1640c5440 2024-09-13T18:44:00.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa1637fe700 1 -- 172.21.15.146:0/3757662484 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c00b9c0 con 0x7fa1640c5440 2024-09-13T18:44:00.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.027+0000 7fa1637fe700 1 -- 172.21.15.146:0/3757662484 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00bd70 con 0x7fa1640c5440 2024-09-13T18:44:00.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 -- 172.21.15.146:0/3757662484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1640c5440 msgr2=0x7fa164153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:00.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 --2- 172.21.15.146:0/3757662484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1640c5440 0x7fa164153340 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa16406ebb0 tx=0x7fa15c00b330 comp rx=0 tx=0).stop 2024-09-13T18:44:00.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 -- 172.21.15.146:0/3757662484 shutdown_connections 2024-09-13T18:44:00.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 --2- 172.21.15.146:0/3757662484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1640c5440 0x7fa164153340 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:00.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 --2- 172.21.15.146:0/3757662484 >> v2:172.21.15.154:3300/0 conn(0x7fa1640c6590 0x7fa16415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:00.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 -- 172.21.15.146:0/3757662484 >> 172.21.15.146:0/3757662484 conn(0x7fa164066b00 msgr2=0x7fa16406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:00.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 -- 172.21.15.146:0/3757662484 shutdown_connections 2024-09-13T18:44:00.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 -- 172.21.15.146:0/3757662484 wait complete. 2024-09-13T18:44:00.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 Processor -- start 2024-09-13T18:44:00.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 -- start start 2024-09-13T18:44:00.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1640c5440 0x7fa1640cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:00.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1640c61b0 0x7fa164157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:00.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640c6590 0x7fa1641f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:00.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16415c140 con 0x7fa1640c61b0 2024-09-13T18:44:00.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641f2aa0 con 0x7fa1640c6590 2024-09-13T18:44:00.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169a53700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa164151430 con 0x7fa1640c5440 2024-09-13T18:44:00.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa163fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1640c61b0 0x7fa164157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:00.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa163fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1640c61b0 0x7fa164157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43782/0 (socket says 172.21.15.146:43782) 2024-09-13T18:44:00.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa163fff700 1 -- 172.21.15.146:0/1933997147 learned_addr learned my addr 172.21.15.146:0/1933997147 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:00.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa169252700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640c6590 0x7fa1641f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:00.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.031+0000 7fa168a51700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1640c5440 0x7fa1640cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:00.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa163fff700 1 -- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1640c5440 msgr2=0x7fa1640cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:00.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa163fff700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1640c5440 0x7fa1640cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:00.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa163fff700 1 -- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640c6590 msgr2=0x7fa1641f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:00.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa163fff700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640c6590 0x7fa1641f2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:00.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa163fff700 1 -- 172.21.15.146:0/1933997147 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c009b30 con 0x7fa1640c61b0 2024-09-13T18:44:00.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa168a51700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1640c5440 0x7fa1640cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:00.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa163fff700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1640c61b0 0x7fa164157b10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa150014e80 tx=0x7fa150016330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:00.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa1617fa700 1 -- 172.21.15.146:0/1933997147 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150019070 con 0x7fa1640c61b0 2024-09-13T18:44:00.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa1617fa700 1 -- 172.21.15.146:0/1933997147 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa150016b50 con 0x7fa1640c61b0 2024-09-13T18:44:00.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa169a53700 1 -- 172.21.15.146:0/1933997147 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa164150e70 con 0x7fa1640c61b0 2024-09-13T18:44:00.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa1617fa700 1 -- 172.21.15.146:0/1933997147 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150004030 con 0x7fa1640c61b0 2024-09-13T18:44:00.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa169a53700 1 -- 172.21.15.146:0/1933997147 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1641f2730 con 0x7fa1640c61b0 2024-09-13T18:44:00.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa1617fa700 1 -- 172.21.15.146:0/1933997147 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa150004190 con 0x7fa1640c61b0 2024-09-13T18:44:00.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa169a53700 1 -- 172.21.15.146:0/1933997147 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa144005310 con 0x7fa1640c61b0 2024-09-13T18:44:00.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa1617fa700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa148066730 0x7fa148068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:00.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa1617fa700 1 -- 172.21.15.146:0/1933997147 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fa15008c010 con 0x7fa1640c61b0 2024-09-13T18:44:00.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.035+0000 7fa168a51700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa148066730 0x7fa148068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:00.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.039+0000 7fa168a51700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa148066730 0x7fa148068bf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa158000f60 tx=0x7fa158009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:00.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.043+0000 7fa1617fa700 1 -- 172.21.15.146:0/1933997147 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa150058e00 con 0x7fa1640c61b0 2024-09-13T18:44:00.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.311+0000 7fa169a53700 1 -- 172.21.15.146:0/1933997147 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1440055c0 con 0x7fa1640c61b0 2024-09-13T18:44:00.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.311+0000 7fa1617fa700 1 -- 172.21.15.146:0/1933997147 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6292 (secure 0 0 0) 0x7fa150014c60 con 0x7fa1640c61b0 2024-09-13T18:44:00.335 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:00.335 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:57.516990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:stopping","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:00.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.315+0000 7fa169a53700 1 -- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa148066730 msgr2=0x7fa148068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:00.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.315+0000 7fa169a53700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa148066730 0x7fa148068bf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa158000f60 tx=0x7fa158009040 comp rx=0 tx=0).stop 2024-09-13T18:44:00.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.315+0000 7fa169a53700 1 -- 172.21.15.146:0/1933997147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1640c61b0 msgr2=0x7fa164157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:00.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.315+0000 7fa169a53700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1640c61b0 0x7fa164157b10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa150014e80 tx=0x7fa150016330 comp rx=0 tx=0).stop 2024-09-13T18:44:00.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.315+0000 7fa169a53700 1 -- 172.21.15.146:0/1933997147 shutdown_connections 2024-09-13T18:44:00.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.315+0000 7fa169a53700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa148066730 0x7fa148068bf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:00.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.315+0000 7fa169a53700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640c6590 0x7fa1641f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:00.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.315+0000 7fa169a53700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1640c61b0 0x7fa164157b10 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:00.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.315+0000 7fa169a53700 1 --2- 172.21.15.146:0/1933997147 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1640c5440 0x7fa1640cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:00.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.315+0000 7fa169a53700 1 -- 172.21.15.146:0/1933997147 >> 172.21.15.146:0/1933997147 conn(0x7fa164066b00 msgr2=0x7fa1640c80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:00.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.315+0000 7fa169a53700 1 -- 172.21.15.146:0/1933997147 shutdown_connections 2024-09-13T18:44:00.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:00.315+0000 7fa169a53700 1 -- 172.21.15.146:0/1933997147 wait complete. 2024-09-13T18:44:00.339 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 71 2024-09-13T18:44:00.349 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:02.350 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:02.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.447+0000 7f658a656700 1 Processor -- start 2024-09-13T18:44:02.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.447+0000 7f658a656700 1 -- start start 2024-09-13T18:44:02.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f658a656700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65840cf170 0x7f65841595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:02.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f658a656700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f65840cf550 0x7f65840c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:02.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f658a656700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658415a540 con 0x7f65840cb5b0 2024-09-13T18:44:02.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f658a656700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65840c5220 con 0x7f65840cf550 2024-09-13T18:44:02.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f658a656700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6584159ca0 con 0x7f65840cf170 2024-09-13T18:44:02.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f6588e53700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f65840cf550 0x7f65840c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:02.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f6589e55700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f65840cb5b0 0x7f65841534c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60034/0 (socket says 172.21.15.146:60034) 2024-09-13T18:44:02.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f6588e53700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f65840cf550 0x7f65840c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:42992/0 (socket says 172.21.15.146:42992) 2024-09-13T18:44:02.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f6588e53700 1 -- 172.21.15.146:0/3654392901 learned_addr learned my addr 172.21.15.146:0/3654392901 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:02.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f6589654700 1 --2- 172.21.15.146:0/3654392901 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65840cf170 0x7f65841595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:02.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f6588e53700 1 -- 172.21.15.146:0/3654392901 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65840cf170 msgr2=0x7f65841595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:02.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f6588e53700 1 --2- 172.21.15.146:0/3654392901 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65840cf170 0x7f65841595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:02.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f6588e53700 1 -- 172.21.15.146:0/3654392901 >> v1:172.21.15.146:6789/0 conn(0x7f65840cb5b0 legacy=0x7f65841534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:02.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f657bfff700 1 -- 172.21.15.146:0/3654392901 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049640734 0 0) 0x7f658415a540 con 0x7f65840cb5b0 2024-09-13T18:44:02.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f6588e53700 1 -- 172.21.15.146:0/3654392901 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6584159f80 con 0x7f65840cf550 2024-09-13T18:44:02.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f6589654700 1 --2- 172.21.15.146:0/3654392901 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65840cf170 0x7f65841595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:02.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f6588e53700 1 --2- 172.21.15.146:0/3654392901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65840cf550 0x7f65840c56b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6574009910 tx=0x7f6574009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f91c54b381b5b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:02.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f657bfff700 1 -- 172.21.15.146:0/3654392901 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657400ba60 con 0x7f65840cf550 2024-09-13T18:44:02.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f657bfff700 1 -- 172.21.15.146:0/3654392901 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657400bbc0 con 0x7f65840cf550 2024-09-13T18:44:02.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f657bfff700 1 -- 172.21.15.146:0/3654392901 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574009e60 con 0x7f65840cf550 2024-09-13T18:44:02.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f658a656700 1 -- 172.21.15.146:0/3654392901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65840cf550 msgr2=0x7f65840c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:02.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f658a656700 1 --2- 172.21.15.146:0/3654392901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65840cf550 0x7f65840c56b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6574009910 tx=0x7f6574009c20 comp rx=0 tx=0).stop 2024-09-13T18:44:02.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f658a656700 1 -- 172.21.15.146:0/3654392901 shutdown_connections 2024-09-13T18:44:02.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f658a656700 1 --2- 172.21.15.146:0/3654392901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65840cf550 0x7f65840c56b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:02.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f658a656700 1 --2- 172.21.15.146:0/3654392901 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65840cf170 0x7f65841595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:02.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f658a656700 1 -- 172.21.15.146:0/3654392901 >> 172.21.15.146:0/3654392901 conn(0x7f6584072220 msgr2=0x7f6584066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:02.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f658a656700 1 -- 172.21.15.146:0/3654392901 shutdown_connections 2024-09-13T18:44:02.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.451+0000 7f658a656700 1 -- 172.21.15.146:0/3654392901 wait complete. 2024-09-13T18:44:02.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f658a656700 1 Processor -- start 2024-09-13T18:44:02.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f658a656700 1 -- start start 2024-09-13T18:44:02.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f658a656700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65840cb5b0 0x7f6584154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:02.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f658a656700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65840cf170 0x7f65841e3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:02.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f658a656700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65840cf550 0x7f65841458c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:02.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f658a656700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6584156880 con 0x7f65840cf170 2024-09-13T18:44:02.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f658a656700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6584153f60 con 0x7f65840cb5b0 2024-09-13T18:44:02.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f658a656700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65841e3390 con 0x7f65840cf550 2024-09-13T18:44:02.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6588e53700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65840cf170 0x7f65841e3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6588e53700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65840cf170 0x7f65841e3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43004/0 (socket says 172.21.15.146:43004) 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6588e53700 1 -- 172.21.15.146:0/258838774 learned_addr learned my addr 172.21.15.146:0/258838774 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6589654700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65840cb5b0 0x7f6584154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6589e55700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65840cf550 0x7f65841458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6588e53700 1 -- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65840cf550 msgr2=0x7f65841458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6588e53700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65840cf550 0x7f65841458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6588e53700 1 -- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65840cb5b0 msgr2=0x7f6584154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6588e53700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65840cb5b0 0x7f6584154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6588e53700 1 -- 172.21.15.146:0/258838774 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6574009570 con 0x7f65840cf170 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6589e55700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65840cf550 0x7f65841458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6588e53700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65840cf170 0x7f65841e3650 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6574000c00 tx=0x7f6574004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6579ffb700 1 -- 172.21.15.146:0/258838774 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657401a840 con 0x7f65840cf170 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6579ffb700 1 -- 172.21.15.146:0/258838774 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657401f070 con 0x7f65840cf170 2024-09-13T18:44:02.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f658a656700 1 -- 172.21.15.146:0/258838774 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65841e1c70 con 0x7f65840cf170 2024-09-13T18:44:02.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f658a656700 1 -- 172.21.15.146:0/258838774 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65841565a0 con 0x7f65840cf170 2024-09-13T18:44:02.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f6579ffb700 1 -- 172.21.15.146:0/258838774 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657401abc0 con 0x7f65840cf170 2024-09-13T18:44:02.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.455+0000 7f658a656700 1 -- 172.21.15.146:0/258838774 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6564005310 con 0x7f65840cf170 2024-09-13T18:44:02.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.459+0000 7f6579ffb700 1 -- 172.21.15.146:0/258838774 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f657401ad20 con 0x7f65840cf170 2024-09-13T18:44:02.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.459+0000 7f6579ffb700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6568066760 0x7f6568068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:02.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.459+0000 7f6579ffb700 1 -- 172.21.15.146:0/258838774 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f6574021070 con 0x7f65840cf170 2024-09-13T18:44:02.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.459+0000 7f6589654700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6568066760 0x7f6568068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:02.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.459+0000 7f6589654700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6568066760 0x7f6568068c20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f65841ecdf0 tx=0x7f6580008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:02.487 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.463+0000 7f6579ffb700 1 -- 172.21.15.146:0/258838774 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6574052cb0 con 0x7f65840cf170 2024-09-13T18:44:02.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.707+0000 7f658a656700 1 -- 172.21.15.146:0/258838774 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65640055c0 con 0x7f65840cf170 2024-09-13T18:44:02.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.711+0000 7f6579ffb700 1 -- 172.21.15.146:0/258838774 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6292 (secure 0 0 0) 0x7f6574015090 con 0x7f65840cf170 2024-09-13T18:44:02.732 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:02.732 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:57.516990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:stopping","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:02.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.711+0000 7f658a656700 1 -- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6568066760 msgr2=0x7f6568068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:02.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.711+0000 7f658a656700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6568066760 0x7f6568068c20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f65841ecdf0 tx=0x7f6580008040 comp rx=0 tx=0).stop 2024-09-13T18:44:02.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.711+0000 7f658a656700 1 -- 172.21.15.146:0/258838774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65840cf170 msgr2=0x7f65841e3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:02.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.711+0000 7f658a656700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65840cf170 0x7f65841e3650 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6574000c00 tx=0x7f6574004770 comp rx=0 tx=0).stop 2024-09-13T18:44:02.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.711+0000 7f658a656700 1 -- 172.21.15.146:0/258838774 shutdown_connections 2024-09-13T18:44:02.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.711+0000 7f658a656700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6568066760 0x7f6568068c20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:02.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.711+0000 7f658a656700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65840cf550 0x7f65841458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:02.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.711+0000 7f658a656700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65840cf170 0x7f65841e3650 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:02.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.711+0000 7f658a656700 1 --2- 172.21.15.146:0/258838774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65840cb5b0 0x7f6584154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:02.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.711+0000 7f658a656700 1 -- 172.21.15.146:0/258838774 >> 172.21.15.146:0/258838774 conn(0x7f6584072220 msgr2=0x7f65840b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:02.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.711+0000 7f658a656700 1 -- 172.21.15.146:0/258838774 shutdown_connections 2024-09-13T18:44:02.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:02.711+0000 7f658a656700 1 -- 172.21.15.146:0/258838774 wait complete. 2024-09-13T18:44:02.736 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 71 2024-09-13T18:44:02.746 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:04.747 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:04.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37d111c700 1 Processor -- start 2024-09-13T18:44:04.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37d111c700 1 -- start start 2024-09-13T18:44:04.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37d111c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37cc0ca210 0x7f37cc0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:04.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37d111c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37cc15da20 con 0x7f37cc0cbd50 2024-09-13T18:44:04.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37d111c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37cc0ca750 con 0x7f37cc071390 2024-09-13T18:44:04.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37d111c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0caa30 con 0x7f37cc0ca210 2024-09-13T18:44:04.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37cbfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f37cc0cbd50 0x7f37cc0701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60040/0 (socket says 172.21.15.146:60040) 2024-09-13T18:44:04.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37cbfff700 1 -- 172.21.15.146:0/423190848 learned_addr learned my addr 172.21.15.146:0/423190848 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:04.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37c3fff700 1 --2- 172.21.15.146:0/423190848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37cc0ca210 0x7f37cc0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:04.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37cb7fe700 1 -- 172.21.15.146:0/423190848 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189520569 0 0) 0x7f37cc15da20 con 0x7f37cc0cbd50 2024-09-13T18:44:04.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37cb7fe700 1 -- 172.21.15.146:0/423190848 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37b80035d0 con 0x7f37cc0cbd50 2024-09-13T18:44:04.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37cb7fe700 1 -- 172.21.15.146:0/423190848 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1004195011 0 0) 0x7f37b80035d0 con 0x7f37cc0cbd50 2024-09-13T18:44:04.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37c3fff700 1 -- 172.21.15.146:0/423190848 >> v1:172.21.15.154:6789/0 conn(0x7f37cc071390 legacy=0x7f37cc163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:04.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37c3fff700 1 -- 172.21.15.146:0/423190848 >> v1:172.21.15.146:6789/0 conn(0x7f37cc0cbd50 legacy=0x7f37cc0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:04.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37c3fff700 1 -- 172.21.15.146:0/423190848 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc15cc60 con 0x7f37cc0ca210 2024-09-13T18:44:04.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.838+0000 7f37c3fff700 1 --2- 172.21.15.146:0/423190848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37cc0ca210 0x7f37cc0b1c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009880 tx=0x7f37b4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83b20d682e566ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:04.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37cb7fe700 1 -- 172.21.15.146:0/423190848 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc001670 con 0x7f37cc0ca210 2024-09-13T18:44:04.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37cb7fe700 1 -- 172.21.15.146:0/423190848 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b4004030 con 0x7f37cc0ca210 2024-09-13T18:44:04.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37cb7fe700 1 -- 172.21.15.146:0/423190848 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b40043a0 con 0x7f37cc0ca210 2024-09-13T18:44:04.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 -- 172.21.15.146:0/423190848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37cc0ca210 msgr2=0x7f37cc0b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:04.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 --2- 172.21.15.146:0/423190848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37cc0ca210 0x7f37cc0b1c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009880 tx=0x7f37b4009b90 comp rx=0 tx=0).stop 2024-09-13T18:44:04.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 -- 172.21.15.146:0/423190848 shutdown_connections 2024-09-13T18:44:04.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 --2- 172.21.15.146:0/423190848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37cc0ca210 0x7f37cc0b1c40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 -- 172.21.15.146:0/423190848 >> 172.21.15.146:0/423190848 conn(0x7f37cc066270 msgr2=0x7f37cc066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 -- 172.21.15.146:0/423190848 shutdown_connections 2024-09-13T18:44:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 -- 172.21.15.146:0/423190848 wait complete. 2024-09-13T18:44:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 Processor -- start 2024-09-13T18:44:04.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 -- start start 2024-09-13T18:44:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37cc071390 0x7f37cc15a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37cc0ca210 0x7f37cc1f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37cc0cbd50 0x7f37cc0ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0cea10 con 0x7f37cc071390 2024-09-13T18:44:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0cf9f0 con 0x7f37cc0cbd50 2024-09-13T18:44:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d111c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0c8fd0 con 0x7f37cc0ca210 2024-09-13T18:44:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37cbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37cc071390 0x7f37cc15a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37cbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37cc071390 0x7f37cc15a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43018/0 (socket says 172.21.15.146:43018) 2024-09-13T18:44:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37cbfff700 1 -- 172.21.15.146:0/3214731729 learned_addr learned my addr 172.21.15.146:0/3214731729 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37d091b700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37cc0cbd50 0x7f37cc0ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.842+0000 7f37c3fff700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37cc0ca210 0x7f37cc1f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:04.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37cbfff700 1 -- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37cc0ca210 msgr2=0x7f37cc1f32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:04.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37cbfff700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37cc0ca210 0x7f37cc1f32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:04.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37cbfff700 1 -- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37cc0cbd50 msgr2=0x7f37cc0ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:04.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37cbfff700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37cc0cbd50 0x7f37cc0ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:04.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37cbfff700 1 -- 172.21.15.146:0/3214731729 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b40094b0 con 0x7f37cc071390 2024-09-13T18:44:04.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37cbfff700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37cc071390 0x7f37cc15a690 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c520 tx=0x7f37bc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37c97fa700 1 -- 172.21.15.146:0/3214731729 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc004ca0 con 0x7f37cc071390 2024-09-13T18:44:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37c97fa700 1 -- 172.21.15.146:0/3214731729 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc004e00 con 0x7f37cc071390 2024-09-13T18:44:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37d111c700 1 -- 172.21.15.146:0/3214731729 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37cc1f2800 con 0x7f37cc071390 2024-09-13T18:44:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37c97fa700 1 -- 172.21.15.146:0/3214731729 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc007310 con 0x7f37cc071390 2024-09-13T18:44:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37d111c700 1 -- 172.21.15.146:0/3214731729 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37cc1f2dc0 con 0x7f37cc071390 2024-09-13T18:44:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37d111c700 1 -- 172.21.15.146:0/3214731729 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37cc153b50 con 0x7f37cc071390 2024-09-13T18:44:04.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37c97fa700 1 -- 172.21.15.146:0/3214731729 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f37bc007470 con 0x7f37cc071390 2024-09-13T18:44:04.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37c97fa700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37ac066730 0x7f37ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:04.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37c97fa700 1 -- 172.21.15.146:0/3214731729 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f37bc084460 con 0x7f37cc071390 2024-09-13T18:44:04.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.846+0000 7f37c3fff700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37ac066730 0x7f37ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:04.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.850+0000 7f37c3fff700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37ac066730 0x7f37ac068bf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f37b40097d0 tx=0x7f37b4010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:04.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:04.854+0000 7f37c97fa700 1 -- 172.21.15.146:0/3214731729 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f37bc051120 con 0x7f37cc071390 2024-09-13T18:44:05.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.078+0000 7f37d111c700 1 -- 172.21.15.146:0/3214731729 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37cc0c9ff0 con 0x7f37cc071390 2024-09-13T18:44:05.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.078+0000 7f37c97fa700 1 -- 172.21.15.146:0/3214731729 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6292 (secure 0 0 0) 0x7f37bc007720 con 0x7f37cc071390 2024-09-13T18:44:05.102 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:05.102 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:57.516990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:stopping","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:05.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.082+0000 7f37d111c700 1 -- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37ac066730 msgr2=0x7f37ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:05.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.082+0000 7f37d111c700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37ac066730 0x7f37ac068bf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f37b40097d0 tx=0x7f37b4010000 comp rx=0 tx=0).stop 2024-09-13T18:44:05.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.082+0000 7f37d111c700 1 -- 172.21.15.146:0/3214731729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37cc071390 msgr2=0x7f37cc15a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:05.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.082+0000 7f37d111c700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37cc071390 0x7f37cc15a690 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c520 tx=0x7f37bc00c830 comp rx=0 tx=0).stop 2024-09-13T18:44:05.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.082+0000 7f37d111c700 1 -- 172.21.15.146:0/3214731729 shutdown_connections 2024-09-13T18:44:05.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.082+0000 7f37d111c700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f37ac066730 0x7f37ac068bf0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:05.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.082+0000 7f37d111c700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37cc0cbd50 0x7f37cc0ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:05.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.082+0000 7f37d111c700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37cc0ca210 0x7f37cc1f32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:05.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.082+0000 7f37d111c700 1 --2- 172.21.15.146:0/3214731729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37cc071390 0x7f37cc15a690 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:05.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.082+0000 7f37d111c700 1 -- 172.21.15.146:0/3214731729 >> 172.21.15.146:0/3214731729 conn(0x7f37cc066270 msgr2=0x7f37cc151e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:05.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.082+0000 7f37d111c700 1 -- 172.21.15.146:0/3214731729 shutdown_connections 2024-09-13T18:44:05.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:05.082+0000 7f37d111c700 1 -- 172.21.15.146:0/3214731729 wait complete. 2024-09-13T18:44:05.106 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 71 2024-09-13T18:44:05.117 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:07.118 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:07.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.218+0000 7f53065db700 1 Processor -- start 2024-09-13T18:44:07.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.218+0000 7f53065db700 1 -- start start 2024-09-13T18:44:07.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53065db700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53000c61b0 0x7f530015d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:07.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53065db700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f53000c6590 0x7f530006cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:07.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53065db700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5300066780 con 0x7f530006d2c0 2024-09-13T18:44:07.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53065db700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53001594a0 con 0x7f53000c6590 2024-09-13T18:44:07.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53065db700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530015d960 con 0x7f53000c61b0 2024-09-13T18:44:07.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f5305dda700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f530006d2c0 0x7f5300153340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60056/0 (socket says 172.21.15.146:60056) 2024-09-13T18:44:07.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f5305dda700 1 -- 172.21.15.146:0/1543554602 learned_addr learned my addr 172.21.15.146:0/1543554602 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:07.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f5304dd8700 1 --2- 172.21.15.146:0/1543554602 >> v2:172.21.15.146:3300/0 conn(0x7f53000c6590 0x7f530006cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:07.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53055d9700 1 --2- 172.21.15.146:0/1543554602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53000c61b0 0x7f530015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:07.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f52f7fff700 1 -- 172.21.15.146:0/1543554602 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548066709 0 0) 0x7f52f8001490 con 0x7f530006d2c0 2024-09-13T18:44:07.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f52f7fff700 1 -- 172.21.15.146:0/1543554602 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52ec0035d0 con 0x7f530006d2c0 2024-09-13T18:44:07.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f5304dd8700 1 -- 172.21.15.146:0/1543554602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53000c61b0 msgr2=0x7f530015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:07.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f5304dd8700 1 --2- 172.21.15.146:0/1543554602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53000c61b0 0x7f530015d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:07.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f5304dd8700 1 -- 172.21.15.146:0/1543554602 >> v1:172.21.15.146:6789/0 conn(0x7f530006d2c0 legacy=0x7f5300153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:07.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f5304dd8700 1 -- 172.21.15.146:0/1543554602 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530006eec0 con 0x7f53000c6590 2024-09-13T18:44:07.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f5304dd8700 1 --2- 172.21.15.146:0/1543554602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53000c6590 0x7f530006cd80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009850 tx=0x7f52f0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632b1bf6b9c6e6b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:07.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f52f7fff700 1 -- 172.21.15.146:0/1543554602 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0011830 con 0x7f53000c6590 2024-09-13T18:44:07.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f52f7fff700 1 -- 172.21.15.146:0/1543554602 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f0011990 con 0x7f53000c6590 2024-09-13T18:44:07.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f52f7fff700 1 -- 172.21.15.146:0/1543554602 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0011d40 con 0x7f53000c6590 2024-09-13T18:44:07.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53065db700 1 -- 172.21.15.146:0/1543554602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53000c6590 msgr2=0x7f530006cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:07.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53065db700 1 --2- 172.21.15.146:0/1543554602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53000c6590 0x7f530006cd80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009850 tx=0x7f52f0009b60 comp rx=0 tx=0).stop 2024-09-13T18:44:07.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53065db700 1 -- 172.21.15.146:0/1543554602 shutdown_connections 2024-09-13T18:44:07.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53065db700 1 --2- 172.21.15.146:0/1543554602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53000c6590 0x7f530006cd80 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:07.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53065db700 1 --2- 172.21.15.146:0/1543554602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53000c61b0 0x7f530015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:07.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53065db700 1 -- 172.21.15.146:0/1543554602 >> 172.21.15.146:0/1543554602 conn(0x7f5300066b00 msgr2=0x7f530006aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:07.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53065db700 1 -- 172.21.15.146:0/1543554602 shutdown_connections 2024-09-13T18:44:07.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.222+0000 7f53065db700 1 -- 172.21.15.146:0/1543554602 wait complete. 2024-09-13T18:44:07.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f53065db700 1 Processor -- start 2024-09-13T18:44:07.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f53065db700 1 -- start start 2024-09-13T18:44:07.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f53065db700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f530006d2c0 0x7f5300152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:07.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f53065db700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53000c61b0 0x7f53001f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:07.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f53065db700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53000c6590 0x7f5300147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:07.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f53065db700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5300151f80 con 0x7f53000c6590 2024-09-13T18:44:07.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f53065db700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53001511e0 con 0x7f53000c61b0 2024-09-13T18:44:07.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f53065db700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5300158ec0 con 0x7f530006d2c0 2024-09-13T18:44:07.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f5305dda700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53000c6590 0x7f5300147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:07.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f5305dda700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53000c6590 0x7f5300147cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43034/0 (socket says 172.21.15.146:43034) 2024-09-13T18:44:07.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f5305dda700 1 -- 172.21.15.146:0/3418385239 learned_addr learned my addr 172.21.15.146:0/3418385239 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:07.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f53055d9700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f530006d2c0 0x7f5300152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:07.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f5304dd8700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53000c61b0 0x7f53001f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:07.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f5305dda700 1 -- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f530006d2c0 msgr2=0x7f5300152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:07.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f5305dda700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f530006d2c0 0x7f5300152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:07.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f5305dda700 1 -- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53000c61b0 msgr2=0x7f53001f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:07.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f5305dda700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53000c61b0 0x7f53001f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:07.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f5305dda700 1 -- 172.21.15.146:0/3418385239 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f00094b0 con 0x7f53000c6590 2024-09-13T18:44:07.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f5305dda700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53000c6590 0x7f5300147cb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f52f800cb10 tx=0x7f52f800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:07.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f52f5ffb700 1 -- 172.21.15.146:0/3418385239 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8007480 con 0x7f53000c6590 2024-09-13T18:44:07.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f52f5ffb700 1 -- 172.21.15.146:0/3418385239 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f80075e0 con 0x7f53000c6590 2024-09-13T18:44:07.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f53065db700 1 -- 172.21.15.146:0/3418385239 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53001591a0 con 0x7f53000c6590 2024-09-13T18:44:07.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f52f5ffb700 1 -- 172.21.15.146:0/3418385239 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8007970 con 0x7f53000c6590 2024-09-13T18:44:07.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.226+0000 7f53065db700 1 -- 172.21.15.146:0/3418385239 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5300151780 con 0x7f53000c6590 2024-09-13T18:44:07.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.230+0000 7f53065db700 1 -- 172.21.15.146:0/3418385239 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53000c27b0 con 0x7f53000c6590 2024-09-13T18:44:07.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.230+0000 7f52f5ffb700 1 -- 172.21.15.146:0/3418385239 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f52f801b3b0 con 0x7f53000c6590 2024-09-13T18:44:07.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.230+0000 7f52f5ffb700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f52e4066760 0x7f52e4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:07.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.230+0000 7f52f5ffb700 1 -- 172.21.15.146:0/3418385239 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f52f8084100 con 0x7f53000c6590 2024-09-13T18:44:07.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.230+0000 7f53055d9700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f52e4066760 0x7f52e4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:07.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.230+0000 7f53055d9700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f52e4066760 0x7f52e4068c20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009bf0 tx=0x7f52fc009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:07.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.234+0000 7f52f5ffb700 1 -- 172.21.15.146:0/3418385239 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f52f8050ca0 con 0x7f53000c6590 2024-09-13T18:44:07.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.470+0000 7f53065db700 1 -- 172.21.15.146:0/3418385239 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5300066780 con 0x7f53000c6590 2024-09-13T18:44:07.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.470+0000 7f52f5ffb700 1 -- 172.21.15.146:0/3418385239 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6292 (secure 0 0 0) 0x7f5300066780 con 0x7f53000c6590 2024-09-13T18:44:07.494 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:07.494 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:57.516990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:stopping","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:07.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.474+0000 7f53065db700 1 -- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f52e4066760 msgr2=0x7f52e4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:07.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.474+0000 7f53065db700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f52e4066760 0x7f52e4068c20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009bf0 tx=0x7f52fc009240 comp rx=0 tx=0).stop 2024-09-13T18:44:07.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.474+0000 7f53065db700 1 -- 172.21.15.146:0/3418385239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53000c6590 msgr2=0x7f5300147cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:07.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.474+0000 7f53065db700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53000c6590 0x7f5300147cb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f52f800cb10 tx=0x7f52f800cb40 comp rx=0 tx=0).stop 2024-09-13T18:44:07.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.474+0000 7f53065db700 1 -- 172.21.15.146:0/3418385239 shutdown_connections 2024-09-13T18:44:07.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.474+0000 7f53065db700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f52e4066760 0x7f52e4068c20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:07.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.474+0000 7f53065db700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53000c6590 0x7f5300147cb0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:07.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.474+0000 7f53065db700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53000c61b0 0x7f53001f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:07.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.474+0000 7f53065db700 1 --2- 172.21.15.146:0/3418385239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f530006d2c0 0x7f5300152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:07.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.474+0000 7f53065db700 1 -- 172.21.15.146:0/3418385239 >> 172.21.15.146:0/3418385239 conn(0x7f5300066b00 msgr2=0x7f530014f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:07.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.474+0000 7f53065db700 1 -- 172.21.15.146:0/3418385239 shutdown_connections 2024-09-13T18:44:07.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:07.474+0000 7f53065db700 1 -- 172.21.15.146:0/3418385239 wait complete. 2024-09-13T18:44:07.499 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 71 2024-09-13T18:44:07.511 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:09.512 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:09.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420f59e700 1 Processor -- start 2024-09-13T18:44:09.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420f59e700 1 -- start start 2024-09-13T18:44:09.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420f59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f42080d8c60 0x7f420804eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:09.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42080073f0 0x7f4208005de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:09.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420f59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42080505d0 con 0x7f420804f920 2024-09-13T18:44:09.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420f59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4208049640 con 0x7f42080d8c60 2024-09-13T18:44:09.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42080508b0 con 0x7f42080073f0 2024-09-13T18:44:09.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420e59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f42080d8c60 0x7f420804eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:09.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420ed9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f420804f920 0x7f420800e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60070/0 (socket says 172.21.15.146:60070) 2024-09-13T18:44:09.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420ed9d700 1 -- 172.21.15.146:0/2111411385 learned_addr learned my addr 172.21.15.146:0/2111411385 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:09.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420dd9b700 1 --2- 172.21.15.146:0/2111411385 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42080073f0 0x7f4208005de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:09.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420e59c700 1 -- 172.21.15.146:0/2111411385 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42080073f0 msgr2=0x7f4208005de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:09.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420e59c700 1 --2- 172.21.15.146:0/2111411385 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42080073f0 0x7f4208005de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:09.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420e59c700 1 -- 172.21.15.146:0/2111411385 >> v1:172.21.15.146:6789/0 conn(0x7f420804f920 legacy=0x7f420800e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:09.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420e59c700 1 -- 172.21.15.146:0/2111411385 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420804f120 con 0x7f42080d8c60 2024-09-13T18:44:09.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420d59a700 1 -- 172.21.15.146:0/2111411385 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091780131 0 0) 0x7f42080505d0 con 0x7f420804f920 2024-09-13T18:44:09.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420e59c700 1 --2- 172.21.15.146:0/2111411385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42080d8c60 0x7f420804eab0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4200009850 tx=0x7f4200009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5bfe5ba5c2b477 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:09.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420d59a700 1 -- 172.21.15.146:0/2111411385 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200011830 con 0x7f42080d8c60 2024-09-13T18:44:09.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420d59a700 1 -- 172.21.15.146:0/2111411385 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4200011990 con 0x7f42080d8c60 2024-09-13T18:44:09.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.614+0000 7f420d59a700 1 -- 172.21.15.146:0/2111411385 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200011d20 con 0x7f42080d8c60 2024-09-13T18:44:09.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 -- 172.21.15.146:0/2111411385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42080d8c60 msgr2=0x7f420804eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:09.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 --2- 172.21.15.146:0/2111411385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42080d8c60 0x7f420804eab0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4200009850 tx=0x7f4200009b60 comp rx=0 tx=0).stop 2024-09-13T18:44:09.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 -- 172.21.15.146:0/2111411385 shutdown_connections 2024-09-13T18:44:09.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 --2- 172.21.15.146:0/2111411385 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42080073f0 0x7f4208005de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:09.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 --2- 172.21.15.146:0/2111411385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42080d8c60 0x7f420804eab0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:09.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 -- 172.21.15.146:0/2111411385 >> 172.21.15.146:0/2111411385 conn(0x7f4208054520 msgr2=0x7f4208054930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:09.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 -- 172.21.15.146:0/2111411385 shutdown_connections 2024-09-13T18:44:09.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 -- 172.21.15.146:0/2111411385 wait complete. 2024-09-13T18:44:09.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 Processor -- start 2024-09-13T18:44:09.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 -- start start 2024-09-13T18:44:09.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42080073f0 0x7f42080cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:09.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f420804f920 0x7f42080d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:09.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42080d8c60 0x7f42080cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:09.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42080d1a20 con 0x7f42080073f0 2024-09-13T18:44:09.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4208038840 con 0x7f42080d8c60 2024-09-13T18:44:09.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4208038ae0 con 0x7f420804f920 2024-09-13T18:44:09.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420e59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42080073f0 0x7f42080cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:09.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420e59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42080073f0 0x7f42080cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43056/0 (socket says 172.21.15.146:43056) 2024-09-13T18:44:09.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420e59c700 1 -- 172.21.15.146:0/3882168222 learned_addr learned my addr 172.21.15.146:0/3882168222 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:09.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420dd9b700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f420804f920 0x7f42080d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:09.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420ed9d700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42080d8c60 0x7f42080cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:09.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420e59c700 1 -- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f420804f920 msgr2=0x7f42080d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:09.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420e59c700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f420804f920 0x7f42080d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:09.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420e59c700 1 -- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42080d8c60 msgr2=0x7f42080cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:09.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420e59c700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42080d8c60 0x7f42080cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:09.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420e59c700 1 -- 172.21.15.146:0/3882168222 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42000094b0 con 0x7f42080073f0 2024-09-13T18:44:09.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420ed9d700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42080d8c60 0x7f42080cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:09.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420e59c700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42080073f0 0x7f42080cfbf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4200000c00 tx=0x7f4200004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:09.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f41feffd700 1 -- 172.21.15.146:0/3882168222 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000038c0 con 0x7f42080073f0 2024-09-13T18:44:09.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f41feffd700 1 -- 172.21.15.146:0/3882168222 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420001e070 con 0x7f42080073f0 2024-09-13T18:44:09.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 -- 172.21.15.146:0/3882168222 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4208038dc0 con 0x7f42080073f0 2024-09-13T18:44:09.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f41feffd700 1 -- 172.21.15.146:0/3882168222 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200003c80 con 0x7f42080073f0 2024-09-13T18:44:09.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.618+0000 7f420f59e700 1 -- 172.21.15.146:0/3882168222 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42080d5ce0 con 0x7f42080073f0 2024-09-13T18:44:09.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.622+0000 7f420f59e700 1 -- 172.21.15.146:0/3882168222 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42080d3760 con 0x7f42080073f0 2024-09-13T18:44:09.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.622+0000 7f41feffd700 1 -- 172.21.15.146:0/3882168222 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f420002d450 con 0x7f42080073f0 2024-09-13T18:44:09.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.622+0000 7f41feffd700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f41f0066730 0x7f41f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:09.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.622+0000 7f41feffd700 1 -- 172.21.15.146:0/3882168222 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f4200085710 con 0x7f42080073f0 2024-09-13T18:44:09.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.622+0000 7f420dd9b700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f41f0066730 0x7f41f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:09.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.622+0000 7f420dd9b700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f41f0066730 0x7f41f0068bf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f42080063a0 tx=0x7f4204008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:09.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.626+0000 7f41feffd700 1 -- 172.21.15.146:0/3882168222 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4200052450 con 0x7f42080073f0 2024-09-13T18:44:09.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.846+0000 7f420f59e700 1 -- 172.21.15.146:0/3882168222 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42080d66d0 con 0x7f42080073f0 2024-09-13T18:44:09.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.846+0000 7f41feffd700 1 -- 172.21.15.146:0/3882168222 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6292 (secure 0 0 0) 0x7f420005ae60 con 0x7f42080073f0 2024-09-13T18:44:09.871 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:09.871 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:57.516990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:stopping","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:09.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.850+0000 7f420f59e700 1 -- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f41f0066730 msgr2=0x7f41f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:09.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.850+0000 7f420f59e700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f41f0066730 0x7f41f0068bf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f42080063a0 tx=0x7f4204008040 comp rx=0 tx=0).stop 2024-09-13T18:44:09.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.850+0000 7f420f59e700 1 -- 172.21.15.146:0/3882168222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42080073f0 msgr2=0x7f42080cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:09.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.850+0000 7f420f59e700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42080073f0 0x7f42080cfbf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4200000c00 tx=0x7f4200004770 comp rx=0 tx=0).stop 2024-09-13T18:44:09.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.850+0000 7f420f59e700 1 -- 172.21.15.146:0/3882168222 shutdown_connections 2024-09-13T18:44:09.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.850+0000 7f420f59e700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f41f0066730 0x7f41f0068bf0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:09.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.850+0000 7f420f59e700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42080d8c60 0x7f42080cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:09.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.850+0000 7f420f59e700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f420804f920 0x7f42080d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:09.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.850+0000 7f420f59e700 1 --2- 172.21.15.146:0/3882168222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42080073f0 0x7f42080cfbf0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:09.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.850+0000 7f420f59e700 1 -- 172.21.15.146:0/3882168222 >> 172.21.15.146:0/3882168222 conn(0x7f4208054520 msgr2=0x7f4208006950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:09.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.850+0000 7f420f59e700 1 -- 172.21.15.146:0/3882168222 shutdown_connections 2024-09-13T18:44:09.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:09.850+0000 7f420f59e700 1 -- 172.21.15.146:0/3882168222 wait complete. 2024-09-13T18:44:09.875 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 71 2024-09-13T18:44:09.885 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-09-13T18:44:09.886 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 71, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 71, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5800": { "addr": "172.21.15.154:6839/776638146", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 776638146, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 776638146, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5800, "incarnation": 52, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 30 }, "gid_6063": { "addr": "172.21.15.146:6835/832794630", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 832794630, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 832794630, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6063, "incarnation": 58, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:stopping", "state_seq": 43 }, "gid_6207": { "addr": "172.21.15.146:6837/3234109877", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3234109877, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3234109877, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6207, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 39 } }, "last_failure": 0, "last_failure_osd_epoch": 36, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:43:57.516990+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5800, "mds_1": 6207, "mds_2": 6063 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.146:6833/3023779168", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3023779168, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3023779168, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 69, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6187, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:44:10.327 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:44:10.329 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:44:11.888 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:12.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 Processor -- start 2024-09-13T18:44:12.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 -- start start 2024-09-13T18:44:12.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faa88150230 0x7faa8806ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:12.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa88150610 0x7faa8806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:12.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faa88071610 0x7faa8806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:12.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa881596b0 con 0x7faa88150230 2024-09-13T18:44:12.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa88071410 con 0x7faa88071610 2024-09-13T18:44:12.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa88156090 con 0x7faa88150610 2024-09-13T18:44:12.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8c8ae700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faa88150230 0x7faa8806ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:12.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8c8ae700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faa88150230 0x7faa8806ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36374/0 (socket says 172.21.15.146:36374) 2024-09-13T18:44:12.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8c8ae700 1 -- 172.21.15.146:0/1332800125 learned_addr learned my addr 172.21.15.146:0/1332800125 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:12.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa87fff700 1 --2- 172.21.15.146:0/1332800125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa88150610 0x7faa8806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:12.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d0af700 1 --2- 172.21.15.146:0/1332800125 >> v2:172.21.15.154:3300/0 conn(0x7faa88071610 0x7faa8806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:12.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8c8ae700 1 -- 172.21.15.146:0/1332800125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa88150610 msgr2=0x7faa8806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:12.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8c8ae700 1 --2- 172.21.15.146:0/1332800125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa88150610 0x7faa8806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:12.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8c8ae700 1 -- 172.21.15.146:0/1332800125 >> v2:172.21.15.154:3300/0 conn(0x7faa88071610 msgr2=0x7faa8806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:12.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8c8ae700 1 --2- 172.21.15.146:0/1332800125 >> v2:172.21.15.154:3300/0 conn(0x7faa88071610 0x7faa8806db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:12.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8c8ae700 1 -- 172.21.15.146:0/1332800125 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa88156350 con 0x7faa88150230 2024-09-13T18:44:12.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d0af700 1 --2- 172.21.15.146:0/1332800125 >> v2:172.21.15.154:3300/0 conn(0x7faa88071610 0x7faa8806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:12.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa87fff700 1 --2- 172.21.15.146:0/1332800125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa88150610 0x7faa8806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:12.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8c8ae700 1 --2- 172.21.15.146:0/1332800125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa88150230 0x7faa8806ebd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7faa88067260 tx=0x7faa7c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f0e719bbe8ed01 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:12.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa877fe700 1 -- 172.21.15.146:0/1332800125 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c00b8a0 con 0x7faa88150230 2024-09-13T18:44:12.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa877fe700 1 -- 172.21.15.146:0/1332800125 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c00ba00 con 0x7faa88150230 2024-09-13T18:44:12.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa877fe700 1 -- 172.21.15.146:0/1332800125 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c00bd90 con 0x7faa88150230 2024-09-13T18:44:12.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 -- 172.21.15.146:0/1332800125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa88150230 msgr2=0x7faa8806ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:12.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 --2- 172.21.15.146:0/1332800125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa88150230 0x7faa8806ebd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7faa88067260 tx=0x7faa7c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:44:12.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 -- 172.21.15.146:0/1332800125 shutdown_connections 2024-09-13T18:44:12.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 --2- 172.21.15.146:0/1332800125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa88150610 0x7faa8806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:12.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 --2- 172.21.15.146:0/1332800125 >> v2:172.21.15.154:3300/0 conn(0x7faa88071610 0x7faa8806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:12.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 --2- 172.21.15.146:0/1332800125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa88150230 0x7faa8806ebd0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:12.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 -- 172.21.15.146:0/1332800125 >> 172.21.15.146:0/1332800125 conn(0x7faa88071ed0 msgr2=0x7faa880722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:12.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 -- 172.21.15.146:0/1332800125 shutdown_connections 2024-09-13T18:44:12.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.990+0000 7faa8d8b0700 1 -- 172.21.15.146:0/1332800125 wait complete. 2024-09-13T18:44:12.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d8b0700 1 Processor -- start 2024-09-13T18:44:12.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d8b0700 1 -- start start 2024-09-13T18:44:12.019 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d8b0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa88071610 0x7faa880c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:12.019 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d8b0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa88150230 0x7faa881571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:12.019 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d8b0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa881f2090 0x7faa8806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:12.019 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d8b0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa88151450 con 0x7faa881f2090 2024-09-13T18:44:12.019 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d8b0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa880ca630 con 0x7faa88071610 2024-09-13T18:44:12.019 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d8b0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa880ca910 con 0x7faa88150230 2024-09-13T18:44:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d0af700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa881f2090 0x7faa8806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d0af700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa881f2090 0x7faa8806f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36380/0 (socket says 172.21.15.146:36380) 2024-09-13T18:44:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d0af700 1 -- 172.21.15.146:0/775230691 learned_addr learned my addr 172.21.15.146:0/775230691 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8c8ae700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa88071610 0x7faa880c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa87fff700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa88150230 0x7faa881571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d0af700 1 -- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa88150230 msgr2=0x7faa881571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d0af700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa88150230 0x7faa881571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:12.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d0af700 1 -- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa88071610 msgr2=0x7faa880c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:12.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d0af700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa88071610 0x7faa880c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:12.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.994+0000 7faa8d0af700 1 -- 172.21.15.146:0/775230691 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c009570 con 0x7faa881f2090 2024-09-13T18:44:12.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.998+0000 7faa8d0af700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa881f2090 0x7faa8806f5c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7faa80007e10 tx=0x7faa8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:12.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.998+0000 7faa857fa700 1 -- 172.21.15.146:0/775230691 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8000dd20 con 0x7faa881f2090 2024-09-13T18:44:12.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.998+0000 7faa857fa700 1 -- 172.21.15.146:0/775230691 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8000f040 con 0x7faa881f2090 2024-09-13T18:44:12.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.998+0000 7faa8d8b0700 1 -- 172.21.15.146:0/775230691 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa880cabf0 con 0x7faa881f2090 2024-09-13T18:44:12.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.998+0000 7faa8d8b0700 1 -- 172.21.15.146:0/775230691 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa880cb1b0 con 0x7faa881f2090 2024-09-13T18:44:12.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.998+0000 7faa857fa700 1 -- 172.21.15.146:0/775230691 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80003c80 con 0x7faa881f2090 2024-09-13T18:44:12.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.998+0000 7faa8d8b0700 1 -- 172.21.15.146:0/775230691 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa68005310 con 0x7faa881f2090 2024-09-13T18:44:12.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.998+0000 7faa857fa700 1 -- 172.21.15.146:0/775230691 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa80004030 con 0x7faa881f2090 2024-09-13T18:44:12.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.998+0000 7faa857fa700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faa6c06ef70 0x7faa6c071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:12.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.998+0000 7faa857fa700 1 -- 172.21.15.146:0/775230691 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7faa800847f0 con 0x7faa881f2090 2024-09-13T18:44:12.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:11.998+0000 7faa8c8ae700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faa6c06ef70 0x7faa6c071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:12.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.002+0000 7faa8c8ae700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faa6c06ef70 0x7faa6c071430 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faa7c000c00 tx=0x7faa7c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:12.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.006+0000 7faa857fa700 1 -- 172.21.15.146:0/775230691 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faa800514a0 con 0x7faa881f2090 2024-09-13T18:44:12.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.238+0000 7faa8d8b0700 1 -- 172.21.15.146:0/775230691 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa680055c0 con 0x7faa881f2090 2024-09-13T18:44:12.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.238+0000 7faa857fa700 1 -- 172.21.15.146:0/775230691 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6292 (secure 0 0 0) 0x7faa80054ac0 con 0x7faa881f2090 2024-09-13T18:44:12.263 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:12.263 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:57.516990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:stopping","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:12.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.242+0000 7faa8d8b0700 1 -- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faa6c06ef70 msgr2=0x7faa6c071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:12.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.242+0000 7faa8d8b0700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faa6c06ef70 0x7faa6c071430 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faa7c000c00 tx=0x7faa7c008030 comp rx=0 tx=0).stop 2024-09-13T18:44:12.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.242+0000 7faa8d8b0700 1 -- 172.21.15.146:0/775230691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa881f2090 msgr2=0x7faa8806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:12.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.242+0000 7faa8d8b0700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa881f2090 0x7faa8806f5c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7faa80007e10 tx=0x7faa8000d3b0 comp rx=0 tx=0).stop 2024-09-13T18:44:12.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.242+0000 7faa8d8b0700 1 -- 172.21.15.146:0/775230691 shutdown_connections 2024-09-13T18:44:12.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.242+0000 7faa8d8b0700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faa6c06ef70 0x7faa6c071430 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:12.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.242+0000 7faa8d8b0700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa881f2090 0x7faa8806f5c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:12.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.242+0000 7faa8d8b0700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa88150230 0x7faa881571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:12.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.242+0000 7faa8d8b0700 1 --2- 172.21.15.146:0/775230691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa88071610 0x7faa880c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:12.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.242+0000 7faa8d8b0700 1 -- 172.21.15.146:0/775230691 >> 172.21.15.146:0/775230691 conn(0x7faa88071ed0 msgr2=0x7faa8806b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:12.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.242+0000 7faa8d8b0700 1 -- 172.21.15.146:0/775230691 shutdown_connections 2024-09-13T18:44:12.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:12.242+0000 7faa8d8b0700 1 -- 172.21.15.146:0/775230691 wait complete. 2024-09-13T18:44:12.267 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 71 2024-09-13T18:44:12.278 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:14.279 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:14.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.390+0000 7f5b4352f700 1 Processor -- start 2024-09-13T18:44:14.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.390+0000 7f5b4352f700 1 -- start start 2024-09-13T18:44:14.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4352f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b3c0721f0 0x7f5b3c157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:14.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4352f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5b3c0cac60 0x7f5b3c0cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:14.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4352f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3c0c8c70 con 0x7f5b3c154a40 2024-09-13T18:44:14.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4352f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3c072810 con 0x7f5b3c0cac60 2024-09-13T18:44:14.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4352f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c157910 con 0x7f5b3c0721f0 2024-09-13T18:44:14.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4252d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5b3c154a40 0x7f5b3c06f7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34264/0 (socket says 172.21.15.146:34264) 2024-09-13T18:44:14.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b42d2e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5b3c0cac60 0x7f5b3c0cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:14.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4252d700 1 -- 172.21.15.146:0/3789427550 learned_addr learned my addr 172.21.15.146:0/3789427550 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:14.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b41d2c700 1 --2- 172.21.15.146:0/3789427550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b3c0721f0 0x7f5b3c157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:14.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b42d2e700 1 -- 172.21.15.146:0/3789427550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b3c0721f0 msgr2=0x7f5b3c157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:14.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b42d2e700 1 --2- 172.21.15.146:0/3789427550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b3c0721f0 0x7f5b3c157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:14.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b42d2e700 1 -- 172.21.15.146:0/3789427550 >> v1:172.21.15.146:6789/0 conn(0x7f5b3c154a40 legacy=0x7f5b3c06f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:14.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b42d2e700 1 -- 172.21.15.146:0/3789427550 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c0c8f50 con 0x7f5b3c0cac60 2024-09-13T18:44:14.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b42d2e700 1 --2- 172.21.15.146:0/3789427550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c0cac60 0x7f5b3c0cb0f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009d90 tx=0x7f5b3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede60b22fc35e747 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:14.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4152b700 1 -- 172.21.15.146:0/3789427550 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400e070 con 0x7f5b3c0cac60 2024-09-13T18:44:14.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4152b700 1 -- 172.21.15.146:0/3789427550 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3400bad0 con 0x7f5b3c0cac60 2024-09-13T18:44:14.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4152b700 1 -- 172.21.15.146:0/3789427550 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400be80 con 0x7f5b3c0cac60 2024-09-13T18:44:14.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4352f700 1 -- 172.21.15.146:0/3789427550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c0cac60 msgr2=0x7f5b3c0cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:14.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4352f700 1 --2- 172.21.15.146:0/3789427550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c0cac60 0x7f5b3c0cb0f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009d90 tx=0x7f5b3400b330 comp rx=0 tx=0).stop 2024-09-13T18:44:14.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4352f700 1 -- 172.21.15.146:0/3789427550 shutdown_connections 2024-09-13T18:44:14.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4352f700 1 --2- 172.21.15.146:0/3789427550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c0cac60 0x7f5b3c0cb0f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:14.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4352f700 1 --2- 172.21.15.146:0/3789427550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b3c0721f0 0x7f5b3c157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:14.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4352f700 1 -- 172.21.15.146:0/3789427550 >> 172.21.15.146:0/3789427550 conn(0x7f5b3c066090 msgr2=0x7f5b3c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:14.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4352f700 1 -- 172.21.15.146:0/3789427550 shutdown_connections 2024-09-13T18:44:14.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.394+0000 7f5b4352f700 1 -- 172.21.15.146:0/3789427550 wait complete. 2024-09-13T18:44:14.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b4352f700 1 Processor -- start 2024-09-13T18:44:14.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b4352f700 1 -- start start 2024-09-13T18:44:14.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b4352f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b3c0721f0 0x7f5b3c1505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:14.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b4352f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b3c0cac60 0x7f5b3c15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:14.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b4352f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c154a40 0x7f5b3c1584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:14.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b4352f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c1f0d60 con 0x7f5b3c154a40 2024-09-13T18:44:14.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b4352f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c1f22d0 con 0x7f5b3c0721f0 2024-09-13T18:44:14.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b4352f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c150af0 con 0x7f5b3c0cac60 2024-09-13T18:44:14.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b42d2e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c154a40 0x7f5b3c1584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:14.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b42d2e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c154a40 0x7f5b3c1584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36394/0 (socket says 172.21.15.146:36394) 2024-09-13T18:44:14.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b42d2e700 1 -- 172.21.15.146:0/3785584680 learned_addr learned my addr 172.21.15.146:0/3785584680 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:14.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b4252d700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b3c0721f0 0x7f5b3c1505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:14.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b41d2c700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b3c0cac60 0x7f5b3c15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:14.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b42d2e700 1 -- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b3c0cac60 msgr2=0x7f5b3c15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:14.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b42d2e700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b3c0cac60 0x7f5b3c15ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:14.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b42d2e700 1 -- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b3c0721f0 msgr2=0x7f5b3c1505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:14.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b42d2e700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b3c0721f0 0x7f5b3c1505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:14.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b42d2e700 1 -- 172.21.15.146:0/3785584680 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b34009a70 con 0x7f5b3c154a40 2024-09-13T18:44:14.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b42d2e700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c154a40 0x7f5b3c1584f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c072b40 tx=0x7f5b34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:14.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b32ffd700 1 -- 172.21.15.146:0/3785584680 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34012630 con 0x7f5b3c154a40 2024-09-13T18:44:14.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b32ffd700 1 -- 172.21.15.146:0/3785584680 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b34012790 con 0x7f5b3c154a40 2024-09-13T18:44:14.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b32ffd700 1 -- 172.21.15.146:0/3785584680 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34012b40 con 0x7f5b3c154a40 2024-09-13T18:44:14.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b4352f700 1 -- 172.21.15.146:0/3785584680 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3c15b380 con 0x7f5b3c154a40 2024-09-13T18:44:14.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.398+0000 7f5b4352f700 1 -- 172.21.15.146:0/3785584680 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3c1f0f50 con 0x7f5b3c154a40 2024-09-13T18:44:14.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.402+0000 7f5b30ff9700 1 -- 172.21.15.146:0/3785584680 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b18005310 con 0x7f5b3c154a40 2024-09-13T18:44:14.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.402+0000 7f5b32ffd700 1 -- 172.21.15.146:0/3785584680 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5b34012ca0 con 0x7f5b3c154a40 2024-09-13T18:44:14.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.402+0000 7f5b32ffd700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5b20066730 0x7f5b20068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:14.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.402+0000 7f5b32ffd700 1 -- 172.21.15.146:0/3785584680 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f5b34086f60 con 0x7f5b3c154a40 2024-09-13T18:44:14.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.402+0000 7f5b4252d700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5b20066730 0x7f5b20068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:14.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.406+0000 7f5b4252d700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5b20066730 0x7f5b20068bf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5b380013e0 tx=0x7f5b38001370 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:14.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.410+0000 7f5b32ffd700 1 -- 172.21.15.146:0/3785584680 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5b340577b0 con 0x7f5b3c154a40 2024-09-13T18:44:14.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.626+0000 7f5b30ff9700 1 -- 172.21.15.146:0/3785584680 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b18005b40 con 0x7f5b3c154a40 2024-09-13T18:44:14.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.626+0000 7f5b32ffd700 1 -- 172.21.15.146:0/3785584680 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6292 (secure 0 0 0) 0x7f5b34057340 con 0x7f5b3c154a40 2024-09-13T18:44:14.649 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:14.649 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:57.516990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:stopping","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:14.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.626+0000 7f5b30ff9700 1 -- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5b20066730 msgr2=0x7f5b20068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:14.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.626+0000 7f5b30ff9700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5b20066730 0x7f5b20068bf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5b380013e0 tx=0x7f5b38001370 comp rx=0 tx=0).stop 2024-09-13T18:44:14.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.626+0000 7f5b30ff9700 1 -- 172.21.15.146:0/3785584680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c154a40 msgr2=0x7f5b3c1584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:14.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.626+0000 7f5b30ff9700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c154a40 0x7f5b3c1584f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c072b40 tx=0x7f5b34004af0 comp rx=0 tx=0).stop 2024-09-13T18:44:14.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.626+0000 7f5b30ff9700 1 -- 172.21.15.146:0/3785584680 shutdown_connections 2024-09-13T18:44:14.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.626+0000 7f5b30ff9700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5b20066730 0x7f5b20068bf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:14.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.626+0000 7f5b30ff9700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c154a40 0x7f5b3c1584f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:14.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.626+0000 7f5b30ff9700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b3c0cac60 0x7f5b3c15ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:14.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.626+0000 7f5b30ff9700 1 --2- 172.21.15.146:0/3785584680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b3c0721f0 0x7f5b3c1505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:14.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.626+0000 7f5b30ff9700 1 -- 172.21.15.146:0/3785584680 >> 172.21.15.146:0/3785584680 conn(0x7f5b3c066090 msgr2=0x7f5b3c153cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:14.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.630+0000 7f5b30ff9700 1 -- 172.21.15.146:0/3785584680 shutdown_connections 2024-09-13T18:44:14.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:14.630+0000 7f5b30ff9700 1 -- 172.21.15.146:0/3785584680 wait complete. 2024-09-13T18:44:14.654 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 71 2024-09-13T18:44:14.664 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:16.665 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:16.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.770+0000 7fcd1a613700 1 Processor -- start 2024-09-13T18:44:16.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.770+0000 7fcd1a613700 1 -- start start 2024-09-13T18:44:16.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.770+0000 7fcd1a613700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd0c04a380 0x7fcd0c045d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:16.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.770+0000 7fcd1a613700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd0c04a760 0x7fcd0c0deac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:16.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.770+0000 7fcd1a613700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0c007b50 con 0x7fcd0c0df000 2024-09-13T18:44:16.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.770+0000 7fcd1a613700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd0c007fe0 con 0x7fcd0c04a760 2024-09-13T18:44:16.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.770+0000 7fcd1a613700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c005e40 con 0x7fcd0c04a380 2024-09-13T18:44:16.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.770+0000 7fcd19e12700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fcd0c0df000 0x7fcd0c0d5440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34276/0 (socket says 172.21.15.146:34276) 2024-09-13T18:44:16.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.770+0000 7fcd19e12700 1 -- 172.21.15.146:0/1536993382 learned_addr learned my addr 172.21.15.146:0/1536993382 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:16.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd18e10700 1 --2- 172.21.15.146:0/1536993382 >> v2:172.21.15.154:3300/0 conn(0x7fcd0c04a760 0x7fcd0c0deac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:16.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd19611700 1 --2- 172.21.15.146:0/1536993382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd0c04a380 0x7fcd0c045d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:16.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd0bfff700 1 -- 172.21.15.146:0/1536993382 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208395059 0 0) 0x7fcd0c007b50 con 0x7fcd0c0df000 2024-09-13T18:44:16.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd0bfff700 1 -- 172.21.15.146:0/1536993382 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd000035d0 con 0x7fcd0c0df000 2024-09-13T18:44:16.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd19611700 1 -- 172.21.15.146:0/1536993382 >> v2:172.21.15.154:3300/0 conn(0x7fcd0c04a760 msgr2=0x7fcd0c0deac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:16.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd19611700 1 --2- 172.21.15.146:0/1536993382 >> v2:172.21.15.154:3300/0 conn(0x7fcd0c04a760 0x7fcd0c0deac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:16.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd19611700 1 -- 172.21.15.146:0/1536993382 >> v1:172.21.15.146:6789/0 conn(0x7fcd0c0df000 legacy=0x7fcd0c0d5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:16.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd0bfff700 1 -- 172.21.15.146:0/1536993382 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1182562190 0 0) 0x7fcd000035d0 con 0x7fcd0c0df000 2024-09-13T18:44:16.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd19611700 1 -- 172.21.15.146:0/1536993382 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0458b0 con 0x7fcd0c04a380 2024-09-13T18:44:16.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd19611700 1 --2- 172.21.15.146:0/1536993382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd0c04a380 0x7fcd0c045d70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009850 tx=0x7fcd10009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94224b785e8addd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:16.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd0bfff700 1 -- 172.21.15.146:0/1536993382 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14074a30 con 0x7fcd0c04a380 2024-09-13T18:44:16.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd0bfff700 1 -- 172.21.15.146:0/1536993382 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd10004030 con 0x7fcd0c04a380 2024-09-13T18:44:16.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd0bfff700 1 -- 172.21.15.146:0/1536993382 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd100043a0 con 0x7fcd0c04a380 2024-09-13T18:44:16.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd1a613700 1 -- 172.21.15.146:0/1536993382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd0c04a380 msgr2=0x7fcd0c045d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:16.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd1a613700 1 --2- 172.21.15.146:0/1536993382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd0c04a380 0x7fcd0c045d70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009850 tx=0x7fcd10009b60 comp rx=0 tx=0).stop 2024-09-13T18:44:16.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd1a613700 1 -- 172.21.15.146:0/1536993382 shutdown_connections 2024-09-13T18:44:16.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd1a613700 1 --2- 172.21.15.146:0/1536993382 >> v2:172.21.15.154:3300/0 conn(0x7fcd0c04a760 0x7fcd0c0deac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:16.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd1a613700 1 --2- 172.21.15.146:0/1536993382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd0c04a380 0x7fcd0c045d70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:16.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd1a613700 1 -- 172.21.15.146:0/1536993382 >> 172.21.15.146:0/1536993382 conn(0x7fcd0c050610 msgr2=0x7fcd0c0509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:16.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd1a613700 1 -- 172.21.15.146:0/1536993382 shutdown_connections 2024-09-13T18:44:16.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd1a613700 1 -- 172.21.15.146:0/1536993382 wait complete. 2024-09-13T18:44:16.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd1a613700 1 Processor -- start 2024-09-13T18:44:16.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.774+0000 7fcd1a613700 1 -- start start 2024-09-13T18:44:16.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd1a613700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd0c04a380 0x7fcd0c173630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:16.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd1a613700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd0c04a760 0x7fcd0c032dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:16.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd1a613700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd0c0df000 0x7fcd0c033c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:16.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd1a613700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c0333e0 con 0x7fcd0c04a380 2024-09-13T18:44:16.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd1a613700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c0336a0 con 0x7fcd0c0df000 2024-09-13T18:44:16.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd1a613700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c172b50 con 0x7fcd0c04a760 2024-09-13T18:44:16.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd19611700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd0c04a380 0x7fcd0c173630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:16.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd19611700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd0c04a380 0x7fcd0c173630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36396/0 (socket says 172.21.15.146:36396) 2024-09-13T18:44:16.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd19611700 1 -- 172.21.15.146:0/2111819162 learned_addr learned my addr 172.21.15.146:0/2111819162 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:16.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd18e10700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd0c04a760 0x7fcd0c032dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:16.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd19e12700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd0c0df000 0x7fcd0c033c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:16.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd19611700 1 -- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd0c04a760 msgr2=0x7fcd0c032dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:16.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd19611700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd0c04a760 0x7fcd0c032dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:16.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd19611700 1 -- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd0c0df000 msgr2=0x7fcd0c033c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:16.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd19611700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd0c0df000 0x7fcd0c033c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:16.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd19611700 1 -- 172.21.15.146:0/2111819162 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100094b0 con 0x7fcd0c04a380 2024-09-13T18:44:16.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd19611700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd0c04a380 0x7fcd0c173630 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcd100090b0 tx=0x7fcd10004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:16.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd09ffb700 1 -- 172.21.15.146:0/2111819162 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000f8a0 con 0x7fcd0c04a380 2024-09-13T18:44:16.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd09ffb700 1 -- 172.21.15.146:0/2111819162 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1001f040 con 0x7fcd0c04a380 2024-09-13T18:44:16.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd1a613700 1 -- 172.21.15.146:0/2111819162 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0c172e30 con 0x7fcd0c04a380 2024-09-13T18:44:16.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd09ffb700 1 -- 172.21.15.146:0/2111819162 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000fc40 con 0x7fcd0c04a380 2024-09-13T18:44:16.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd1a613700 1 -- 172.21.15.146:0/2111819162 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0c04b0e0 con 0x7fcd0c04a380 2024-09-13T18:44:16.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd1a613700 1 -- 172.21.15.146:0/2111819162 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0c0deac0 con 0x7fcd0c04a380 2024-09-13T18:44:16.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.778+0000 7fcd09ffb700 1 -- 172.21.15.146:0/2111819162 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcd1002c410 con 0x7fcd0c04a380 2024-09-13T18:44:16.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.782+0000 7fcd09ffb700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fccf8066730 0x7fccf8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:16.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.782+0000 7fcd09ffb700 1 -- 172.21.15.146:0/2111819162 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fcd10087af0 con 0x7fcd0c04a380 2024-09-13T18:44:16.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.782+0000 7fcd18e10700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fccf8066730 0x7fccf8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:16.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.782+0000 7fcd18e10700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fccf8066730 0x7fccf8068bf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0367d0 tx=0x7fcd04009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:16.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:16.786+0000 7fcd09ffb700 1 -- 172.21.15.146:0/2111819162 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcd100548d0 con 0x7fcd0c04a380 2024-09-13T18:44:17.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.010+0000 7fcd1a613700 1 -- 172.21.15.146:0/2111819162 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd0c047c00 con 0x7fcd0c04a380 2024-09-13T18:44:17.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.014+0000 7fcd09ffb700 1 -- 172.21.15.146:0/2111819162 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6292 (secure 0 0 0) 0x7fcd10057ef0 con 0x7fcd0c04a380 2024-09-13T18:44:17.036 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:17.036 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:43:57.516990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":5800,"mds_1":6207,"mds_2":6063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6063":{"gid":6063,"name":"a","rank":2,"incarnation":58,"state":"up:stopping","state_seq":43,"addr":"172.21.15.146:6835/832794630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":832794630},{"type":"v1","addr":"172.21.15.146:6835","nonce":832794630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:17.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.014+0000 7fcd1a613700 1 -- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fccf8066730 msgr2=0x7fccf8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:17.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.014+0000 7fcd1a613700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fccf8066730 0x7fccf8068bf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0367d0 tx=0x7fcd04009240 comp rx=0 tx=0).stop 2024-09-13T18:44:17.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.014+0000 7fcd1a613700 1 -- 172.21.15.146:0/2111819162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd0c04a380 msgr2=0x7fcd0c173630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:17.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.014+0000 7fcd1a613700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd0c04a380 0x7fcd0c173630 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcd100090b0 tx=0x7fcd10004770 comp rx=0 tx=0).stop 2024-09-13T18:44:17.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.014+0000 7fcd1a613700 1 -- 172.21.15.146:0/2111819162 shutdown_connections 2024-09-13T18:44:17.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.014+0000 7fcd1a613700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fccf8066730 0x7fccf8068bf0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:17.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.014+0000 7fcd1a613700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd0c0df000 0x7fcd0c033c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:17.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.014+0000 7fcd1a613700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd0c04a760 0x7fcd0c032dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:17.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.014+0000 7fcd1a613700 1 --2- 172.21.15.146:0/2111819162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd0c04a380 0x7fcd0c173630 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:17.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.014+0000 7fcd1a613700 1 -- 172.21.15.146:0/2111819162 >> 172.21.15.146:0/2111819162 conn(0x7fcd0c050610 msgr2=0x7fcd0c045d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:17.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.014+0000 7fcd1a613700 1 -- 172.21.15.146:0/2111819162 shutdown_connections 2024-09-13T18:44:17.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:17.014+0000 7fcd1a613700 1 -- 172.21.15.146:0/2111819162 wait complete. 2024-09-13T18:44:17.040 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 71 2024-09-13T18:44:17.052 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:17.661 INFO:tasks.ceph.mds.a.smithi146.stdout:starting mds.a at 2024-09-13T18:44:19.052 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:19.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 Processor -- start 2024-09-13T18:44:19.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 -- start start 2024-09-13T18:44:19.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f85740cb2a0 0x7f857415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:19.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85740cf3c0 0x7f85740cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:19.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85740cfee0 con 0x7f85740caec0 2024-09-13T18:44:19.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85740cc0a0 con 0x7f85740cb2a0 2024-09-13T18:44:19.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85740cb790 con 0x7f85740cf3c0 2024-09-13T18:44:19.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f8571d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f85740cb2a0 0x7f857415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:19.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f8571d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f85740cb2a0 0x7f857415ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36418/0 (socket says 172.21.15.146:36418) 2024-09-13T18:44:19.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f8571d9b700 1 -- 172.21.15.146:0/2813575372 learned_addr learned my addr 172.21.15.146:0/2813575372 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:19.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f8572d9d700 1 --2- 172.21.15.146:0/2813575372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85740cf3c0 0x7f85740cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:19.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f8571d9b700 1 -- 172.21.15.146:0/2813575372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85740cf3c0 msgr2=0x7f85740cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:19.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f8571d9b700 1 --2- 172.21.15.146:0/2813575372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85740cf3c0 0x7f85740cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f8571d9b700 1 -- 172.21.15.146:0/2813575372 >> v1:172.21.15.154:6789/0 conn(0x7f85740caec0 legacy=0x7f85741596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:19.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f8571d9b700 1 -- 172.21.15.146:0/2813575372 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857415a180 con 0x7f85740cb2a0 2024-09-13T18:44:19.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f8571d9b700 1 --2- 172.21.15.146:0/2813575372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85740cb2a0 0x7f857415ab40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8564009910 tx=0x7f8564009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c930e7902ec8620 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:19.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857159a700 1 -- 172.21.15.146:0/2813575372 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564011830 con 0x7f85740cb2a0 2024-09-13T18:44:19.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857159a700 1 -- 172.21.15.146:0/2813575372 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8564011990 con 0x7f85740cb2a0 2024-09-13T18:44:19.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857159a700 1 -- 172.21.15.146:0/2813575372 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564011d40 con 0x7f85740cb2a0 2024-09-13T18:44:19.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 -- 172.21.15.146:0/2813575372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85740cb2a0 msgr2=0x7f857415ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:19.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 --2- 172.21.15.146:0/2813575372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85740cb2a0 0x7f857415ab40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8564009910 tx=0x7f8564009c20 comp rx=0 tx=0).stop 2024-09-13T18:44:19.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 -- 172.21.15.146:0/2813575372 shutdown_connections 2024-09-13T18:44:19.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 --2- 172.21.15.146:0/2813575372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85740cf3c0 0x7f85740cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 --2- 172.21.15.146:0/2813575372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85740cb2a0 0x7f857415ab40 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 -- 172.21.15.146:0/2813575372 >> 172.21.15.146:0/2813575372 conn(0x7f8574071d20 msgr2=0x7f8574072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:19.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 -- 172.21.15.146:0/2813575372 shutdown_connections 2024-09-13T18:44:19.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.154+0000 7f857359e700 1 -- 172.21.15.146:0/2813575372 wait complete. 2024-09-13T18:44:19.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f857359e700 1 Processor -- start 2024-09-13T18:44:19.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f857359e700 1 -- start start 2024-09-13T18:44:19.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f857359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85740caec0 0x7f85741f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:19.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f857359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85740cb2a0 0x7f85741eec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:19.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f857359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85740cf3c0 0x7f8574155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:19.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f857359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85741f2860 con 0x7f85740cf3c0 2024-09-13T18:44:19.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f857359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85740c65d0 con 0x7f85740caec0 2024-09-13T18:44:19.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f857359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85740c6800 con 0x7f85740cb2a0 2024-09-13T18:44:19.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f8572d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85740cf3c0 0x7f8574155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:19.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f857259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85740caec0 0x7f85741f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:19.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f8571d9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85740cb2a0 0x7f85741eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:19.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f857259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85740caec0 0x7f85741f2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:45164/0 (socket says 172.21.15.146:45164) 2024-09-13T18:44:19.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f857259c700 1 -- 172.21.15.146:0/2557294862 learned_addr learned my addr 172.21.15.146:0/2557294862 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:19.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f8572d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85740cf3c0 0x7f8574155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36424/0 (socket says 172.21.15.146:36424) 2024-09-13T18:44:19.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f8572d9d700 1 -- 172.21.15.146:0/2557294862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85740cb2a0 msgr2=0x7f85741eec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:19.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f8572d9d700 1 --2- 172.21.15.146:0/2557294862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85740cb2a0 0x7f85741eec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f8572d9d700 1 -- 172.21.15.146:0/2557294862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85740caec0 msgr2=0x7f85741f2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:19.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f8572d9d700 1 --2- 172.21.15.146:0/2557294862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85740caec0 0x7f85741f2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f8572d9d700 1 -- 172.21.15.146:0/2557294862 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8564009570 con 0x7f85740cf3c0 2024-09-13T18:44:19.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f8572d9d700 1 --2- 172.21.15.146:0/2557294862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85740cf3c0 0x7f8574155870 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f856c00efc0 tx=0x7f856c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:19.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f8562ffd700 1 -- 172.21.15.146:0/2557294862 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c00cc90 con 0x7f85740cf3c0 2024-09-13T18:44:19.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f8562ffd700 1 -- 172.21.15.146:0/2557294862 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c00f040 con 0x7f85740cf3c0 2024-09-13T18:44:19.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f857359e700 1 -- 172.21.15.146:0/2557294862 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85740c56d0 con 0x7f85740cf3c0 2024-09-13T18:44:19.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f8562ffd700 1 -- 172.21.15.146:0/2557294862 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c0146c0 con 0x7f85740cf3c0 2024-09-13T18:44:19.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.158+0000 7f857359e700 1 -- 172.21.15.146:0/2557294862 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85741f21a0 con 0x7f85740cf3c0 2024-09-13T18:44:19.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.162+0000 7f857359e700 1 -- 172.21.15.146:0/2557294862 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85740c9450 con 0x7f85740cf3c0 2024-09-13T18:44:19.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.162+0000 7f8562ffd700 1 -- 172.21.15.146:0/2557294862 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f856c014820 con 0x7f85740cf3c0 2024-09-13T18:44:19.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.162+0000 7f8562ffd700 1 --2- 172.21.15.146:0/2557294862 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8554066760 0x7f8554068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:19.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.162+0000 7f8562ffd700 1 -- 172.21.15.146:0/2557294862 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f856c0852b0 con 0x7f85740cf3c0 2024-09-13T18:44:19.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.162+0000 7f857259c700 1 --2- 172.21.15.146:0/2557294862 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8554066760 0x7f8554068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:19.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.162+0000 7f857259c700 1 --2- 172.21.15.146:0/2557294862 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8554066760 0x7f8554068c20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8568000f60 tx=0x7f8568009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:19.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.166+0000 7f8562ffd700 1 -- 172.21.15.146:0/2557294862 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f856c051f60 con 0x7f85740cf3c0 2024-09-13T18:44:19.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f857359e700 1 -- 172.21.15.146:0/2557294862 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85741f29a0 con 0x7f85740cf3c0 2024-09-13T18:44:19.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f8562ffd700 1 -- 172.21.15.146:0/2557294862 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f85741f29a0 con 0x7f85740cf3c0 2024-09-13T18:44:19.409 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:19.409 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:17.579418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5800,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:19.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f857359e700 1 -- 172.21.15.146:0/2557294862 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8554066760 msgr2=0x7f8554068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:19.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f857359e700 1 --2- 172.21.15.146:0/2557294862 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8554066760 0x7f8554068c20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8568000f60 tx=0x7f8568009040 comp rx=0 tx=0).stop 2024-09-13T18:44:19.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f857359e700 1 -- 172.21.15.146:0/2557294862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85740cf3c0 msgr2=0x7f8574155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:19.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f857359e700 1 --2- 172.21.15.146:0/2557294862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85740cf3c0 0x7f8574155870 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f856c00efc0 tx=0x7f856c00c3a0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f857359e700 1 -- 172.21.15.146:0/2557294862 shutdown_connections 2024-09-13T18:44:19.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f857359e700 1 --2- 172.21.15.146:0/2557294862 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8554066760 0x7f8554068c20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f857359e700 1 --2- 172.21.15.146:0/2557294862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85740cf3c0 0x7f8574155870 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f857359e700 1 --2- 172.21.15.146:0/2557294862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85740cb2a0 0x7f85741eec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f857359e700 1 --2- 172.21.15.146:0/2557294862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85740caec0 0x7f85741f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f857359e700 1 -- 172.21.15.146:0/2557294862 >> 172.21.15.146:0/2557294862 conn(0x7f8574071d20 msgr2=0x7f85740c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:19.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f857359e700 1 -- 172.21.15.146:0/2557294862 shutdown_connections 2024-09-13T18:44:19.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.386+0000 7f857359e700 1 -- 172.21.15.146:0/2557294862 wait complete. 2024-09-13T18:44:19.413 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 73 2024-09-13T18:44:19.424 INFO:tasks.mds_thrash.fs.[cephfs]:mds cluster has 2 alive and active, now stable! 2024-09-13T18:44:19.424 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=0) 2024-09-13T18:44:19.424 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-09-13T18:44:19.424 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:44:19.441 INFO:tasks.ceph.mds.f:Stopped 2024-09-13T18:44:19.441 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:19.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 Processor -- start 2024-09-13T18:44:19.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 -- start start 2024-09-13T18:44:19.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff0dc0cbd50 0x7ff0dc0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:19.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0dc071390 0x7ff0dc0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:19.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0dc0cc670 con 0x7ff0dc0ca210 2024-09-13T18:44:19.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0dc0caa30 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0cb810 con 0x7ff0dc071390 2024-09-13T18:44:19.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e0aec700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff0dc0cbd50 0x7ff0dc0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:19.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e0aec700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff0dc0cbd50 0x7ff0dc0b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36434/0 (socket says 172.21.15.146:36434) 2024-09-13T18:44:19.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e0aec700 1 -- 172.21.15.146:0/418788958 learned_addr learned my addr 172.21.15.146:0/418788958 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:19.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e12ed700 1 --2- 172.21.15.146:0/418788958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0dc071390 0x7ff0dc0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:19.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e0aec700 1 -- 172.21.15.146:0/418788958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0dc071390 msgr2=0x7ff0dc0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:19.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e0aec700 1 --2- 172.21.15.146:0/418788958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0dc071390 0x7ff0dc0cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e0aec700 1 -- 172.21.15.146:0/418788958 >> v1:172.21.15.154:6789/0 conn(0x7ff0dc0ca210 legacy=0x7ff0dc0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:19.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e0aec700 1 -- 172.21.15.146:0/418788958 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc15da20 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e12ed700 1 --2- 172.21.15.146:0/418788958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0dc071390 0x7ff0dc0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:19.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e0aec700 1 --2- 172.21.15.146:0/418788958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0dc0cbd50 0x7ff0dc0b1c40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009910 tx=0x7ff0d8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e180ac8d878202d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:19.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0d37fe700 1 -- 172.21.15.146:0/418788958 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d800ba60 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0d37fe700 1 -- 172.21.15.146:0/418788958 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d800bbc0 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0d37fe700 1 -- 172.21.15.146:0/418788958 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8009e60 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 -- 172.21.15.146:0/418788958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0dc0cbd50 msgr2=0x7ff0dc0b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:19.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 --2- 172.21.15.146:0/418788958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0dc0cbd50 0x7ff0dc0b1c40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009910 tx=0x7ff0d8009c20 comp rx=0 tx=0).stop 2024-09-13T18:44:19.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 -- 172.21.15.146:0/418788958 shutdown_connections 2024-09-13T18:44:19.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 --2- 172.21.15.146:0/418788958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0dc071390 0x7ff0dc0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 --2- 172.21.15.146:0/418788958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0dc0cbd50 0x7ff0dc0b1c40 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 -- 172.21.15.146:0/418788958 >> 172.21.15.146:0/418788958 conn(0x7ff0dc066270 msgr2=0x7ff0dc066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:19.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 -- 172.21.15.146:0/418788958 shutdown_connections 2024-09-13T18:44:19.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.522+0000 7ff0e1aee700 1 -- 172.21.15.146:0/418788958 wait complete. 2024-09-13T18:44:19.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e1aee700 1 Processor -- start 2024-09-13T18:44:19.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e1aee700 1 -- start start 2024-09-13T18:44:19.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e1aee700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0dc071390 0x7ff0dc0b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:19.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e1aee700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0dc0ca210 0x7ff0dc1efce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:19.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e1aee700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0dc0cbd50 0x7ff0dc1467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:19.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e1aee700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc1ef8c0 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e1aee700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc15c4c0 con 0x7ff0dc071390 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e1aee700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc1ef550 con 0x7ff0dc0ca210 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e12ed700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0dc0cbd50 0x7ff0dc1467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e12ed700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0dc0cbd50 0x7ff0dc1467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36438/0 (socket says 172.21.15.146:36438) 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e12ed700 1 -- 172.21.15.146:0/4215161937 learned_addr learned my addr 172.21.15.146:0/4215161937 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e0aec700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0dc071390 0x7ff0dc0b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0d3fff700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0dc0ca210 0x7ff0dc1efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e12ed700 1 -- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0dc0ca210 msgr2=0x7ff0dc1efce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e12ed700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0dc0ca210 0x7ff0dc1efce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e12ed700 1 -- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0dc071390 msgr2=0x7ff0dc0b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e12ed700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0dc071390 0x7ff0dc0b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e12ed700 1 -- 172.21.15.146:0/4215161937 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d8009570 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0d3fff700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0dc0ca210 0x7ff0dc1efce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:19.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e12ed700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0dc0cbd50 0x7ff0dc1467a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4007f20 tx=0x7ff0d400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:19.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0d17fa700 1 -- 172.21.15.146:0/4215161937 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400dde0 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0d17fa700 1 -- 172.21.15.146:0/4215161937 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d400f040 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e1aee700 1 -- 172.21.15.146:0/4215161937 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0dc15bcc0 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0d17fa700 1 -- 172.21.15.146:0/4215161937 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4003c80 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.526+0000 7ff0e1aee700 1 -- 172.21.15.146:0/4215161937 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0dc15c9b0 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.530+0000 7ff0e1aee700 1 -- 172.21.15.146:0/4215161937 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0dc15cc30 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.530+0000 7ff0d17fa700 1 -- 172.21.15.146:0/4215161937 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff0d4004030 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.530+0000 7ff0d17fa700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff0c8066760 0x7ff0c8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:19.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.530+0000 7ff0d17fa700 1 -- 172.21.15.146:0/4215161937 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7ff0d40848b0 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.530+0000 7ff0e0aec700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff0c8066760 0x7ff0c8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:19.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.530+0000 7ff0e0aec700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff0c8066760 0x7ff0c8068c20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8000c00 tx=0x7ff0d8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:19.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.538+0000 7ff0d17fa700 1 -- 172.21.15.146:0/4215161937 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff0d4051560 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0e1aee700 1 -- 172.21.15.146:0/4215161937 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0dc1efa00 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0d17fa700 1 -- 172.21.15.146:0/4215161937 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+6276 (secure 0 0 0) 0x7ff0d4054b80 con 0x7ff0dc0cbd50 2024-09-13T18:44:19.801 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:19.801 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:17.579418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5800,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:19.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0e1aee700 1 -- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff0c8066760 msgr2=0x7ff0c8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:19.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0e1aee700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff0c8066760 0x7ff0c8068c20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8000c00 tx=0x7ff0d8008030 comp rx=0 tx=0).stop 2024-09-13T18:44:19.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0e1aee700 1 -- 172.21.15.146:0/4215161937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0dc0cbd50 msgr2=0x7ff0dc1467a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:19.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0e1aee700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0dc0cbd50 0x7ff0dc1467a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4007f20 tx=0x7ff0d400d3b0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0e1aee700 1 -- 172.21.15.146:0/4215161937 shutdown_connections 2024-09-13T18:44:19.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0e1aee700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff0c8066760 0x7ff0c8068c20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0e1aee700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0dc0cbd50 0x7ff0dc1467a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0e1aee700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0dc0ca210 0x7ff0dc1efce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0e1aee700 1 --2- 172.21.15.146:0/4215161937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0dc071390 0x7ff0dc0b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:19.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0e1aee700 1 -- 172.21.15.146:0/4215161937 >> 172.21.15.146:0/4215161937 conn(0x7ff0dc066270 msgr2=0x7ff0dc151370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:19.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0e1aee700 1 -- 172.21.15.146:0/4215161937 shutdown_connections 2024-09-13T18:44:19.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:19.778+0000 7ff0e1aee700 1 -- 172.21.15.146:0/4215161937 wait complete. 2024-09-13T18:44:19.804 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 73 2024-09-13T18:44:19.816 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:44:21.816 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:21.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39bd7c0700 1 Processor -- start 2024-09-13T18:44:21.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39bd7c0700 1 -- start start 2024-09-13T18:44:21.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39bd7c0700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f39b80c5fc0 0x7f39b80c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:21.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39bd7c0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b8159d70 0x7f39b80c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:21.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39bd7c0700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b80c8c70 con 0x7f39b80c63a0 2024-09-13T18:44:21.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39bd7c0700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b80c9bd0 con 0x7f39b80c5fc0 2024-09-13T18:44:21.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39bd7c0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b80c8f50 con 0x7f39b8159d70 2024-09-13T18:44:21.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39af7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f39b80c63a0 0x7f39b8159660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56940/0 (socket says 172.21.15.146:56940) 2024-09-13T18:44:21.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39af7fe700 1 -- 172.21.15.146:0/4142521777 learned_addr learned my addr 172.21.15.146:0/4142521777 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:21.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39b7fff700 1 --2- 172.21.15.146:0/4142521777 >> v2:172.21.15.146:3300/0 conn(0x7f39b80c5fc0 0x7f39b80c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:21.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39bcfbf700 1 --2- 172.21.15.146:0/4142521777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b8159d70 0x7f39b80c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:21.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39b77fe700 1 -- 172.21.15.146:0/4142521777 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256140053 0 0) 0x7f39b80c8c70 con 0x7f39b80c63a0 2024-09-13T18:44:21.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39b7fff700 1 -- 172.21.15.146:0/4142521777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b8159d70 msgr2=0x7f39b80c7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:21.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39b7fff700 1 --2- 172.21.15.146:0/4142521777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b8159d70 0x7f39b80c7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:21.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39b7fff700 1 -- 172.21.15.146:0/4142521777 >> v1:172.21.15.146:6789/0 conn(0x7f39b80c63a0 legacy=0x7f39b8159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:21.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39b7fff700 1 -- 172.21.15.146:0/4142521777 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b80c9230 con 0x7f39b80c5fc0 2024-09-13T18:44:21.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39bcfbf700 1 --2- 172.21.15.146:0/4142521777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b8159d70 0x7f39b80c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:21.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.918+0000 7f39b7fff700 1 --2- 172.21.15.146:0/4142521777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39b80c5fc0 0x7f39b80c6c00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009850 tx=0x7f39a8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba540fd8e07e661 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:21.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39b77fe700 1 -- 172.21.15.146:0/4142521777 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8011830 con 0x7f39b80c5fc0 2024-09-13T18:44:21.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39b77fe700 1 -- 172.21.15.146:0/4142521777 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a8011990 con 0x7f39b80c5fc0 2024-09-13T18:44:21.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39b77fe700 1 -- 172.21.15.146:0/4142521777 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8011d20 con 0x7f39b80c5fc0 2024-09-13T18:44:21.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4142521777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39b80c5fc0 msgr2=0x7f39b80c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:21.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 --2- 172.21.15.146:0/4142521777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39b80c5fc0 0x7f39b80c6c00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009850 tx=0x7f39a8009b60 comp rx=0 tx=0).stop 2024-09-13T18:44:21.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4142521777 shutdown_connections 2024-09-13T18:44:21.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 --2- 172.21.15.146:0/4142521777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b8159d70 0x7f39b80c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:21.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 --2- 172.21.15.146:0/4142521777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39b80c5fc0 0x7f39b80c6c00 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:21.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4142521777 >> 172.21.15.146:0/4142521777 conn(0x7f39b8066090 msgr2=0x7f39b8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:21.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4142521777 shutdown_connections 2024-09-13T18:44:21.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4142521777 wait complete. 2024-09-13T18:44:21.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 Processor -- start 2024-09-13T18:44:21.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 -- start start 2024-09-13T18:44:21.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39b80c5fc0 0x7f39b80ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:21.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b80c63a0 0x7f39b8155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:21.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39b8159d70 0x7f39b81f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:21.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b80c1460 con 0x7f39b8159d70 2024-09-13T18:44:21.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b80af290 con 0x7f39b80c5fc0 2024-09-13T18:44:21.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.922+0000 7f39bd7c0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b81e1a40 con 0x7f39b80c63a0 2024-09-13T18:44:21.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39bcfbf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39b8159d70 0x7f39b81f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:21.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39bcfbf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39b8159d70 0x7f39b81f23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60930/0 (socket says 172.21.15.146:60930) 2024-09-13T18:44:21.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39bcfbf700 1 -- 172.21.15.146:0/4220200531 learned_addr learned my addr 172.21.15.146:0/4220200531 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:21.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39af7fe700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b80c63a0 0x7f39b8155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:21.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39b7fff700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39b80c5fc0 0x7f39b80ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:21.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39bcfbf700 1 -- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b80c63a0 msgr2=0x7f39b8155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:21.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39bcfbf700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b80c63a0 0x7f39b8155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:21.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39bcfbf700 1 -- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39b80c5fc0 msgr2=0x7f39b80ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:21.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39bcfbf700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39b80c5fc0 0x7f39b80ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:21.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39bcfbf700 1 -- 172.21.15.146:0/4220200531 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80094b0 con 0x7f39b8159d70 2024-09-13T18:44:21.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39bcfbf700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39b8159d70 0x7f39b81f23c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f39b0007df0 tx=0x7f39b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:21.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39b57fa700 1 -- 172.21.15.146:0/4220200531 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b000ddd0 con 0x7f39b8159d70 2024-09-13T18:44:21.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39b57fa700 1 -- 172.21.15.146:0/4220200531 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b000f040 con 0x7f39b8159d70 2024-09-13T18:44:21.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4220200531 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b80aeef0 con 0x7f39b8159d70 2024-09-13T18:44:21.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39b57fa700 1 -- 172.21.15.146:0/4220200531 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0003c80 con 0x7f39b8159d70 2024-09-13T18:44:21.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4220200531 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b80adcd0 con 0x7f39b8159d70 2024-09-13T18:44:21.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4220200531 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b80cb790 con 0x7f39b8159d70 2024-09-13T18:44:21.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.926+0000 7f39b57fa700 1 -- 172.21.15.146:0/4220200531 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f39b0004030 con 0x7f39b8159d70 2024-09-13T18:44:21.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.930+0000 7f39b57fa700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3998066760 0x7f3998068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:21.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.930+0000 7f39b57fa700 1 -- 172.21.15.146:0/4220200531 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f39b0084980 con 0x7f39b8159d70 2024-09-13T18:44:21.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.930+0000 7f39b7fff700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3998066760 0x7f3998068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:21.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.930+0000 7f39b7fff700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3998066760 0x7f3998068c20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f39a8014040 tx=0x7f39a8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:21.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:21.934+0000 7f39b57fa700 1 -- 172.21.15.146:0/4220200531 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f39b00516c0 con 0x7f39b8159d70 2024-09-13T18:44:22.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.142+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4220200531 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39b806f5c0 con 0x7f39b8159d70 2024-09-13T18:44:22.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.142+0000 7f39b57fa700 1 -- 172.21.15.146:0/4220200531 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f39b0054ce0 con 0x7f39b8159d70 2024-09-13T18:44:22.168 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:22.168 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:17.579418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5800,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:22.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.146+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3998066760 msgr2=0x7f3998068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:22.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.146+0000 7f39bd7c0700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3998066760 0x7f3998068c20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f39a8014040 tx=0x7f39a8008030 comp rx=0 tx=0).stop 2024-09-13T18:44:22.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.146+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4220200531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39b8159d70 msgr2=0x7f39b81f23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:22.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.146+0000 7f39bd7c0700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39b8159d70 0x7f39b81f23c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f39b0007df0 tx=0x7f39b000d3b0 comp rx=0 tx=0).stop 2024-09-13T18:44:22.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.146+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4220200531 shutdown_connections 2024-09-13T18:44:22.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.146+0000 7f39bd7c0700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3998066760 0x7f3998068c20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:22.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.146+0000 7f39bd7c0700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39b8159d70 0x7f39b81f23c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:22.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.146+0000 7f39bd7c0700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b80c63a0 0x7f39b8155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:22.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.146+0000 7f39bd7c0700 1 --2- 172.21.15.146:0/4220200531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39b80c5fc0 0x7f39b80ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:22.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.146+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4220200531 >> 172.21.15.146:0/4220200531 conn(0x7f39b8066090 msgr2=0x7f39b8159430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:22.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.146+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4220200531 shutdown_connections 2024-09-13T18:44:22.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:22.146+0000 7f39bd7c0700 1 -- 172.21.15.146:0/4220200531 wait complete. 2024-09-13T18:44:22.172 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 73 2024-09-13T18:44:22.182 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:44:24.183 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:24.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa37bfff700 1 Processor -- start 2024-09-13T18:44:24.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa37bfff700 1 -- start start 2024-09-13T18:44:24.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa37bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa37c0c8f70 0x7fa37c0a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:24.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa37bfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37c159f80 con 0x7fa37c0c7530 2024-09-13T18:44:24.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa37bfff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37c159670 con 0x7fa37c0c9350 2024-09-13T18:44:24.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa37bfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37c0ca1d0 con 0x7fa37c0c8f70 2024-09-13T18:44:24.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa37affd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa37c0c8f70 0x7fa37c0a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:24.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa37b7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa37c0c7530 0x7fa37c0cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56946/0 (socket says 172.21.15.146:56946) 2024-09-13T18:44:24.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa37b7fe700 1 -- 172.21.15.146:0/1574164117 learned_addr learned my addr 172.21.15.146:0/1574164117 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:24.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa37affd700 1 -- 172.21.15.146:0/1574164117 >> v1:172.21.15.154:6789/0 conn(0x7fa37c0c9350 legacy=0x7fa37c0723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:24.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa37affd700 1 -- 172.21.15.146:0/1574164117 >> v1:172.21.15.146:6789/0 conn(0x7fa37c0c7530 legacy=0x7fa37c0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:24.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa379ffb700 1 -- 172.21.15.146:0/1574164117 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142388167 0 0) 0x7fa37c159f80 con 0x7fa37c0c7530 2024-09-13T18:44:24.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa37affd700 1 -- 172.21.15.146:0/1574164117 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c0aa3a0 con 0x7fa37c0c8f70 2024-09-13T18:44:24.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa37affd700 1 --2- 172.21.15.146:0/1574164117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa37c0c8f70 0x7fa37c0a9cf0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009850 tx=0x7fa36c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916027f2ba3f0442 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:24.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.282+0000 7fa379ffb700 1 -- 172.21.15.146:0/1574164117 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c011830 con 0x7fa37c0c8f70 2024-09-13T18:44:24.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.286+0000 7fa379ffb700 1 -- 172.21.15.146:0/1574164117 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c011990 con 0x7fa37c0c8f70 2024-09-13T18:44:24.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.286+0000 7fa379ffb700 1 -- 172.21.15.146:0/1574164117 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c011d20 con 0x7fa37c0c8f70 2024-09-13T18:44:24.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.286+0000 7fa37bfff700 1 -- 172.21.15.146:0/1574164117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa37c0c8f70 msgr2=0x7fa37c0a9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:24.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.286+0000 7fa37bfff700 1 --2- 172.21.15.146:0/1574164117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa37c0c8f70 0x7fa37c0a9cf0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009850 tx=0x7fa36c009b60 comp rx=0 tx=0).stop 2024-09-13T18:44:24.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.286+0000 7fa37bfff700 1 -- 172.21.15.146:0/1574164117 shutdown_connections 2024-09-13T18:44:24.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.286+0000 7fa37bfff700 1 --2- 172.21.15.146:0/1574164117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa37c0c8f70 0x7fa37c0a9cf0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:24.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.286+0000 7fa37bfff700 1 -- 172.21.15.146:0/1574164117 >> 172.21.15.146:0/1574164117 conn(0x7fa37c066090 msgr2=0x7fa37c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:24.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.286+0000 7fa37bfff700 1 -- 172.21.15.146:0/1574164117 shutdown_connections 2024-09-13T18:44:24.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.286+0000 7fa37bfff700 1 -- 172.21.15.146:0/1574164117 wait complete. 2024-09-13T18:44:24.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37bfff700 1 Processor -- start 2024-09-13T18:44:24.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37bfff700 1 -- start start 2024-09-13T18:44:24.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa37c0c7530 0x7fa37c0aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:24.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa37c0c8f70 0x7fa37c1ef580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:24.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa37c0c9350 0x7fa37c1f0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:24.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37bfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0acf60 con 0x7fa37c0c8f70 2024-09-13T18:44:24.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37bfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0af8e0 con 0x7fa37c0c9350 2024-09-13T18:44:24.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c1efac0 con 0x7fa37c0c7530 2024-09-13T18:44:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37a7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa37c0c8f70 0x7fa37c1ef580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37a7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa37c0c8f70 0x7fa37c1ef580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60948/0 (socket says 172.21.15.146:60948) 2024-09-13T18:44:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37a7fc700 1 -- 172.21.15.146:0/601323452 learned_addr learned my addr 172.21.15.146:0/601323452 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37affd700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa37c0c7530 0x7fa37c0aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37b7fe700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa37c0c9350 0x7fa37c1f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37a7fc700 1 -- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa37c0c7530 msgr2=0x7fa37c0aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37a7fc700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa37c0c7530 0x7fa37c0aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37a7fc700 1 -- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa37c0c9350 msgr2=0x7fa37c1f0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37a7fc700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa37c0c9350 0x7fa37c1f0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37a7fc700 1 -- 172.21.15.146:0/601323452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c0094b0 con 0x7fa37c0c8f70 2024-09-13T18:44:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37b7fe700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa37c0c9350 0x7fa37c1f0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37a7fc700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa37c0c8f70 0x7fa37c1ef580 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa36400cbd0 tx=0x7fa36400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa3737fe700 1 -- 172.21.15.146:0/601323452 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364003c20 con 0x7fa37c0c8f70 2024-09-13T18:44:24.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37bfff700 1 -- 172.21.15.146:0/601323452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37c0af600 con 0x7fa37c0c8f70 2024-09-13T18:44:24.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa3737fe700 1 -- 172.21.15.146:0/601323452 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa364003d80 con 0x7fa37c0c8f70 2024-09-13T18:44:24.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa3737fe700 1 -- 172.21.15.146:0/601323452 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36400ab40 con 0x7fa37c0c8f70 2024-09-13T18:44:24.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.290+0000 7fa37bfff700 1 -- 172.21.15.146:0/601323452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37c0aded0 con 0x7fa37c0c8f70 2024-09-13T18:44:24.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.294+0000 7fa37bfff700 1 -- 172.21.15.146:0/601323452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37c15a3f0 con 0x7fa37c0c8f70 2024-09-13T18:44:24.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.294+0000 7fa3737fe700 1 -- 172.21.15.146:0/601323452 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa36400aca0 con 0x7fa37c0c8f70 2024-09-13T18:44:24.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.294+0000 7fa3737fe700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa35c066760 0x7fa35c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:24.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.294+0000 7fa3737fe700 1 -- 172.21.15.146:0/601323452 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fa3640853d0 con 0x7fa37c0c8f70 2024-09-13T18:44:24.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.294+0000 7fa37affd700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa35c066760 0x7fa35c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:24.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.298+0000 7fa37affd700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa35c066760 0x7fa35c068c20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa36c000c00 tx=0x7fa36c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:24.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.302+0000 7fa3737fe700 1 -- 172.21.15.146:0/601323452 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa364051fe0 con 0x7fa37c0c8f70 2024-09-13T18:44:24.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.510+0000 7fa37bfff700 1 -- 172.21.15.146:0/601323452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa37c1f0580 con 0x7fa37c0c8f70 2024-09-13T18:44:24.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.514+0000 7fa3737fe700 1 -- 172.21.15.146:0/601323452 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+6276 (secure 0 0 0) 0x7fa364055600 con 0x7fa37c0c8f70 2024-09-13T18:44:24.537 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:24.537 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:17.579418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5800,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:24.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.514+0000 7fa37bfff700 1 -- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa35c066760 msgr2=0x7fa35c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:24.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.514+0000 7fa37bfff700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa35c066760 0x7fa35c068c20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa36c000c00 tx=0x7fa36c008030 comp rx=0 tx=0).stop 2024-09-13T18:44:24.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.514+0000 7fa37bfff700 1 -- 172.21.15.146:0/601323452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa37c0c8f70 msgr2=0x7fa37c1ef580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:24.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.514+0000 7fa37bfff700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa37c0c8f70 0x7fa37c1ef580 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa36400cbd0 tx=0x7fa36400cee0 comp rx=0 tx=0).stop 2024-09-13T18:44:24.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.514+0000 7fa37bfff700 1 -- 172.21.15.146:0/601323452 shutdown_connections 2024-09-13T18:44:24.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.514+0000 7fa37bfff700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa35c066760 0x7fa35c068c20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:24.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.514+0000 7fa37bfff700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa37c0c9350 0x7fa37c1f0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:24.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.514+0000 7fa37bfff700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa37c0c8f70 0x7fa37c1ef580 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:24.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.514+0000 7fa37bfff700 1 --2- 172.21.15.146:0/601323452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa37c0c7530 0x7fa37c0aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:24.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.514+0000 7fa37bfff700 1 -- 172.21.15.146:0/601323452 >> 172.21.15.146:0/601323452 conn(0x7fa37c066090 msgr2=0x7fa37c0cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:24.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.514+0000 7fa37bfff700 1 -- 172.21.15.146:0/601323452 shutdown_connections 2024-09-13T18:44:24.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:24.514+0000 7fa37bfff700 1 -- 172.21.15.146:0/601323452 wait complete. 2024-09-13T18:44:24.541 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 73 2024-09-13T18:44:24.553 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:44:26.554 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:26.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.650+0000 7fb3b58af700 1 Processor -- start 2024-09-13T18:44:26.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.650+0000 7fb3b58af700 1 -- start start 2024-09-13T18:44:26.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.650+0000 7fb3b58af700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb3b0153a60 0x7fb3b0159700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:26.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.650+0000 7fb3b58af700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb3b00c9ed0 0x7fb3b00b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:26.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.650+0000 7fb3b58af700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b00cc2d0 con 0x7fb3b00cf540 2024-09-13T18:44:26.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.650+0000 7fb3b58af700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b0153e40 con 0x7fb3b0153a60 2024-09-13T18:44:26.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.650+0000 7fb3b58af700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b00cb3e0 con 0x7fb3b00c9ed0 2024-09-13T18:44:26.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.650+0000 7fb3b48ad700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb3b0153a60 0x7fb3b0159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:26.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.650+0000 7fb3b48ad700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb3b0153a60 0x7fb3b0159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60954/0 (socket says 172.21.15.146:60954) 2024-09-13T18:44:26.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.650+0000 7fb3b48ad700 1 -- 172.21.15.146:0/3141192889 learned_addr learned my addr 172.21.15.146:0/3141192889 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:26.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.650+0000 7fb3affff700 1 --2- 172.21.15.146:0/3141192889 >> v2:172.21.15.154:3300/0 conn(0x7fb3b00c9ed0 0x7fb3b00b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:26.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b48ad700 1 -- 172.21.15.146:0/3141192889 >> v2:172.21.15.154:3300/0 conn(0x7fb3b00c9ed0 msgr2=0x7fb3b00b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:26.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b48ad700 1 --2- 172.21.15.146:0/3141192889 >> v2:172.21.15.154:3300/0 conn(0x7fb3b00c9ed0 0x7fb3b00b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:26.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b48ad700 1 -- 172.21.15.146:0/3141192889 >> v1:172.21.15.154:6789/0 conn(0x7fb3b00cf540 legacy=0x7fb3b015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:26.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b48ad700 1 -- 172.21.15.146:0/3141192889 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b00cb6c0 con 0x7fb3b0153a60 2024-09-13T18:44:26.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b48ad700 1 --2- 172.21.15.146:0/3141192889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b0153a60 0x7fb3b0159700 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00c60b0 tx=0x7fb3a0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422cf31b21666148 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:26.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3af7fe700 1 -- 172.21.15.146:0/3141192889 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0011830 con 0x7fb3b0153a60 2024-09-13T18:44:26.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3af7fe700 1 -- 172.21.15.146:0/3141192889 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a0011990 con 0x7fb3b0153a60 2024-09-13T18:44:26.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3af7fe700 1 -- 172.21.15.146:0/3141192889 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0011d20 con 0x7fb3b0153a60 2024-09-13T18:44:26.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b58af700 1 -- 172.21.15.146:0/3141192889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b0153a60 msgr2=0x7fb3b0159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:26.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b58af700 1 --2- 172.21.15.146:0/3141192889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b0153a60 0x7fb3b0159700 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00c60b0 tx=0x7fb3a0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:44:26.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b58af700 1 -- 172.21.15.146:0/3141192889 shutdown_connections 2024-09-13T18:44:26.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b58af700 1 --2- 172.21.15.146:0/3141192889 >> v2:172.21.15.154:3300/0 conn(0x7fb3b00c9ed0 0x7fb3b00b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:26.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b58af700 1 --2- 172.21.15.146:0/3141192889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b0153a60 0x7fb3b0159700 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:26.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b58af700 1 -- 172.21.15.146:0/3141192889 >> 172.21.15.146:0/3141192889 conn(0x7fb3b0071ec0 msgr2=0x7fb3b00722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:26.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b58af700 1 -- 172.21.15.146:0/3141192889 shutdown_connections 2024-09-13T18:44:26.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b58af700 1 -- 172.21.15.146:0/3141192889 wait complete. 2024-09-13T18:44:26.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b58af700 1 Processor -- start 2024-09-13T18:44:26.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.654+0000 7fb3b58af700 1 -- start start 2024-09-13T18:44:26.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b58af700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b00cf540 0x7fb3b014f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:26.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b58af700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b0153a60 0x7fb3b00b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:26.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b58af700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b014ead0 0x7fb3b014c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:26.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b58af700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00ad080 con 0x7fb3b014ead0 2024-09-13T18:44:26.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b58af700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b01efad0 con 0x7fb3b00cf540 2024-09-13T18:44:26.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b58af700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b014fc10 con 0x7fb3b0153a60 2024-09-13T18:44:26.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b50ae700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b014ead0 0x7fb3b014c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:26.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b48ad700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b00cf540 0x7fb3b014f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b50ae700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b014ead0 0x7fb3b014c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60960/0 (socket says 172.21.15.146:60960) 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b50ae700 1 -- 172.21.15.146:0/1364752522 learned_addr learned my addr 172.21.15.146:0/1364752522 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3affff700 1 --2- 172.21.15.146:0/1364752522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b0153a60 0x7fb3b00b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b50ae700 1 -- 172.21.15.146:0/1364752522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b0153a60 msgr2=0x7fb3b00b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b50ae700 1 --2- 172.21.15.146:0/1364752522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b0153a60 0x7fb3b00b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b50ae700 1 -- 172.21.15.146:0/1364752522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b00cf540 msgr2=0x7fb3b014f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b50ae700 1 --2- 172.21.15.146:0/1364752522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b00cf540 0x7fb3b014f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b50ae700 1 -- 172.21.15.146:0/1364752522 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a0009570 con 0x7fb3b014ead0 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b50ae700 1 --2- 172.21.15.146:0/1364752522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b014ead0 0x7fb3b014c930 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800cb50 tx=0x7fb3a800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3ad7fa700 1 -- 172.21.15.146:0/1364752522 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8003c20 con 0x7fb3b014ead0 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3ad7fa700 1 -- 172.21.15.146:0/1364752522 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a8003d80 con 0x7fb3b014ead0 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3ad7fa700 1 -- 172.21.15.146:0/1364752522 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a800aa20 con 0x7fb3b014ead0 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b58af700 1 -- 172.21.15.146:0/1364752522 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b01eeec0 con 0x7fb3b014ead0 2024-09-13T18:44:26.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b58af700 1 -- 172.21.15.146:0/1364752522 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b01ef480 con 0x7fb3b014ead0 2024-09-13T18:44:26.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.658+0000 7fb3b58af700 1 -- 172.21.15.146:0/1364752522 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b01efc80 con 0x7fb3b014ead0 2024-09-13T18:44:26.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.662+0000 7fb3ad7fa700 1 -- 172.21.15.146:0/1364752522 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb3a800ab80 con 0x7fb3b014ead0 2024-09-13T18:44:26.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.662+0000 7fb3ad7fa700 1 --2- 172.21.15.146:0/1364752522 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb390066730 0x7fb390068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:26.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.662+0000 7fb3ad7fa700 1 -- 172.21.15.146:0/1364752522 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fb3a80848a0 con 0x7fb3b014ead0 2024-09-13T18:44:26.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.662+0000 7fb3b48ad700 1 --2- 172.21.15.146:0/1364752522 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb390066730 0x7fb390068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:26.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.662+0000 7fb3b48ad700 1 --2- 172.21.15.146:0/1364752522 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb390066730 0x7fb390068bf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0000c00 tx=0x7fb3a0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:26.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.666+0000 7fb3ad7fa700 1 -- 172.21.15.146:0/1364752522 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb3b01efc80 con 0x7fb3b014ead0 2024-09-13T18:44:26.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.878+0000 7fb3b58af700 1 -- 172.21.15.146:0/1364752522 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3b01efc80 con 0x7fb3b014ead0 2024-09-13T18:44:26.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.878+0000 7fb3ad7fa700 1 -- 172.21.15.146:0/1364752522 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+6276 (secure 0 0 0) 0x7fb3b01efc80 con 0x7fb3b014ead0 2024-09-13T18:44:26.901 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:26.902 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:17.579418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5800,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:26.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.878+0000 7fb3b58af700 1 -- 172.21.15.146:0/1364752522 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb390066730 msgr2=0x7fb390068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:26.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.878+0000 7fb3b58af700 1 --2- 172.21.15.146:0/1364752522 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb390066730 0x7fb390068bf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0000c00 tx=0x7fb3a0008030 comp rx=0 tx=0).stop 2024-09-13T18:44:26.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.878+0000 7fb3b58af700 1 -- 172.21.15.146:0/1364752522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b014ead0 msgr2=0x7fb3b014c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:26.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.878+0000 7fb3b58af700 1 --2- 172.21.15.146:0/1364752522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b014ead0 0x7fb3b014c930 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800cb50 tx=0x7fb3a800ce60 comp rx=0 tx=0).stop 2024-09-13T18:44:26.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.878+0000 7fb3b58af700 1 -- 172.21.15.146:0/1364752522 shutdown_connections 2024-09-13T18:44:26.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.878+0000 7fb3b58af700 1 --2- 172.21.15.146:0/1364752522 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb390066730 0x7fb390068bf0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:26.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.878+0000 7fb3b58af700 1 --2- 172.21.15.146:0/1364752522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b014ead0 0x7fb3b014c930 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:26.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.878+0000 7fb3b58af700 1 --2- 172.21.15.146:0/1364752522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b0153a60 0x7fb3b00b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:26.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.878+0000 7fb3b58af700 1 --2- 172.21.15.146:0/1364752522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b00cf540 0x7fb3b014f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:26.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.878+0000 7fb3b58af700 1 -- 172.21.15.146:0/1364752522 >> 172.21.15.146:0/1364752522 conn(0x7fb3b0071ec0 msgr2=0x7fb3b00cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:26.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.882+0000 7fb3b58af700 1 -- 172.21.15.146:0/1364752522 shutdown_connections 2024-09-13T18:44:26.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:26.882+0000 7fb3b58af700 1 -- 172.21.15.146:0/1364752522 wait complete. 2024-09-13T18:44:26.905 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 73 2024-09-13T18:44:26.917 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:44:28.918 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:29.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.018+0000 7f7db9d81700 1 Processor -- start 2024-09-13T18:44:29.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.018+0000 7f7db9d81700 1 -- start start 2024-09-13T18:44:29.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9d81700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7db4153bc0 0x7f7db40c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:29.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9d81700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7db40c6610 0x7f7db40c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:29.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9d81700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db4153fa0 con 0x7f7db40cac60 2024-09-13T18:44:29.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9d81700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db40aa500 con 0x7f7db40c6610 2024-09-13T18:44:29.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9d81700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db415a7e0 con 0x7f7db4153bc0 2024-09-13T18:44:29.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9580700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f7db40cac60 0x7f7db40c7740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56950/0 (socket says 172.21.15.146:56950) 2024-09-13T18:44:29.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9580700 1 -- 172.21.15.146:0/2386541109 learned_addr learned my addr 172.21.15.146:0/2386541109 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:29.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db8d7f700 1 --2- 172.21.15.146:0/2386541109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7db4153bc0 0x7f7db40c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:29.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db3fff700 1 --2- 172.21.15.146:0/2386541109 >> v2:172.21.15.154:3300/0 conn(0x7f7db40c6610 0x7f7db40c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:29.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db37fe700 1 -- 172.21.15.146:0/2386541109 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629603379 0 0) 0x7f7db4153fa0 con 0x7f7db40cac60 2024-09-13T18:44:29.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db37fe700 1 -- 172.21.15.146:0/2386541109 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7da00035d0 con 0x7f7db40cac60 2024-09-13T18:44:29.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db3fff700 1 -- 172.21.15.146:0/2386541109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7db4153bc0 msgr2=0x7f7db40c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:29.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db3fff700 1 --2- 172.21.15.146:0/2386541109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7db4153bc0 0x7f7db40c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:29.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db3fff700 1 -- 172.21.15.146:0/2386541109 >> v1:172.21.15.146:6789/0 conn(0x7f7db40cac60 legacy=0x7f7db40c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:29.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db3fff700 1 -- 172.21.15.146:0/2386541109 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db40c9320 con 0x7f7db40c6610 2024-09-13T18:44:29.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db8d7f700 1 --2- 172.21.15.146:0/2386541109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7db4153bc0 0x7f7db40c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:44:29.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db3fff700 1 --2- 172.21.15.146:0/2386541109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40c6610 0x7f7db40c6a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009d90 tx=0x7f7d9c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7c903acfcff60b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:29.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db37fe700 1 -- 172.21.15.146:0/2386541109 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00e070 con 0x7f7db40c6610 2024-09-13T18:44:29.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db37fe700 1 -- 172.21.15.146:0/2386541109 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c00bb20 con 0x7f7db40c6610 2024-09-13T18:44:29.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db37fe700 1 -- 172.21.15.146:0/2386541109 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c012430 con 0x7f7db40c6610 2024-09-13T18:44:29.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9d81700 1 -- 172.21.15.146:0/2386541109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40c6610 msgr2=0x7f7db40c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:29.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9d81700 1 --2- 172.21.15.146:0/2386541109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40c6610 0x7f7db40c6a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009d90 tx=0x7f7d9c00b330 comp rx=0 tx=0).stop 2024-09-13T18:44:29.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9d81700 1 -- 172.21.15.146:0/2386541109 shutdown_connections 2024-09-13T18:44:29.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9d81700 1 --2- 172.21.15.146:0/2386541109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40c6610 0x7f7db40c6a50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:29.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9d81700 1 --2- 172.21.15.146:0/2386541109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7db4153bc0 0x7f7db40c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:29.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9d81700 1 -- 172.21.15.146:0/2386541109 >> 172.21.15.146:0/2386541109 conn(0x7f7db4066090 msgr2=0x7f7db4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:29.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9d81700 1 -- 172.21.15.146:0/2386541109 shutdown_connections 2024-09-13T18:44:29.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.022+0000 7f7db9d81700 1 -- 172.21.15.146:0/2386541109 wait complete. 2024-09-13T18:44:29.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db9d81700 1 Processor -- start 2024-09-13T18:44:29.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db9d81700 1 -- start start 2024-09-13T18:44:29.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db9d81700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db40c6610 0x7f7db41eec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:29.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db9d81700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40cac60 0x7f7db41ef1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:29.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db9d81700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7db4153bc0 0x7f7db40ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:29.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db9d81700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db40afe50 con 0x7f7db40c6610 2024-09-13T18:44:29.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db9d81700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db40c4510 con 0x7f7db40cac60 2024-09-13T18:44:29.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db9d81700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db40ad2b0 con 0x7f7db4153bc0 2024-09-13T18:44:29.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db8d7f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db40c6610 0x7f7db41eec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:29.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db8d7f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db40c6610 0x7f7db41eec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60974/0 (socket says 172.21.15.146:60974) 2024-09-13T18:44:29.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db8d7f700 1 -- 172.21.15.146:0/1615047318 learned_addr learned my addr 172.21.15.146:0/1615047318 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:29.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db3fff700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40cac60 0x7f7db41ef1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:29.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db9580700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7db4153bc0 0x7f7db40ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:29.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db8d7f700 1 -- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7db4153bc0 msgr2=0x7f7db40ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:29.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db8d7f700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7db4153bc0 0x7f7db40ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:29.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db8d7f700 1 -- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40cac60 msgr2=0x7f7db41ef1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:29.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db8d7f700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40cac60 0x7f7db41ef1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:29.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db8d7f700 1 -- 172.21.15.146:0/1615047318 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da40094b0 con 0x7f7db40c6610 2024-09-13T18:44:29.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db8d7f700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db40c6610 0x7f7db41eec90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7da400ec10 tx=0x7f7da400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:29.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.026+0000 7f7db17fa700 1 -- 172.21.15.146:0/1615047318 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4009770 con 0x7f7db40c6610 2024-09-13T18:44:29.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.030+0000 7f7db17fa700 1 -- 172.21.15.146:0/1615047318 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da400c850 con 0x7f7db40c6610 2024-09-13T18:44:29.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.030+0000 7f7db9d81700 1 -- 172.21.15.146:0/1615047318 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c009a70 con 0x7f7db40c6610 2024-09-13T18:44:29.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.030+0000 7f7db17fa700 1 -- 172.21.15.146:0/1615047318 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da400cbc0 con 0x7f7db40c6610 2024-09-13T18:44:29.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.030+0000 7f7db9d81700 1 -- 172.21.15.146:0/1615047318 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db40c36c0 con 0x7f7db40c6610 2024-09-13T18:44:29.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.030+0000 7f7db9d81700 1 -- 172.21.15.146:0/1615047318 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db4159a00 con 0x7f7db40c6610 2024-09-13T18:44:29.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.030+0000 7f7db17fa700 1 -- 172.21.15.146:0/1615047318 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7da400cd20 con 0x7f7db40c6610 2024-09-13T18:44:29.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.030+0000 7f7db17fa700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7d94066730 0x7f7d94068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:29.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.030+0000 7f7db17fa700 1 -- 172.21.15.146:0/1615047318 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f7da40848e0 con 0x7f7db40c6610 2024-09-13T18:44:29.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.034+0000 7f7db3fff700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7d94066730 0x7f7d94068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:29.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.034+0000 7f7db3fff700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7d94066730 0x7f7d94068bf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009d90 tx=0x7f7d9c011000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:29.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.038+0000 7f7db17fa700 1 -- 172.21.15.146:0/1615047318 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7db4159a00 con 0x7f7db40c6610 2024-09-13T18:44:29.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.246+0000 7f7db9d81700 1 -- 172.21.15.146:0/1615047318 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7db4159a00 con 0x7f7db40c6610 2024-09-13T18:44:29.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.250+0000 7f7db17fa700 1 -- 172.21.15.146:0/1615047318 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f7db4159a00 con 0x7f7db40c6610 2024-09-13T18:44:29.273 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:29.273 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:17.579418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5800,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:29.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.254+0000 7f7db9d81700 1 -- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7d94066730 msgr2=0x7f7d94068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:29.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.254+0000 7f7db9d81700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7d94066730 0x7f7d94068bf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009d90 tx=0x7f7d9c011000 comp rx=0 tx=0).stop 2024-09-13T18:44:29.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.254+0000 7f7db9d81700 1 -- 172.21.15.146:0/1615047318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db40c6610 msgr2=0x7f7db41eec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:29.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.254+0000 7f7db9d81700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db40c6610 0x7f7db41eec90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7da400ec10 tx=0x7f7da400ef20 comp rx=0 tx=0).stop 2024-09-13T18:44:29.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.254+0000 7f7db9d81700 1 -- 172.21.15.146:0/1615047318 shutdown_connections 2024-09-13T18:44:29.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.254+0000 7f7db9d81700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7d94066730 0x7f7d94068bf0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:29.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.254+0000 7f7db9d81700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7db4153bc0 0x7f7db40ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:29.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.254+0000 7f7db9d81700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40cac60 0x7f7db41ef1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:29.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.254+0000 7f7db9d81700 1 --2- 172.21.15.146:0/1615047318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db40c6610 0x7f7db41eec90 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:29.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.254+0000 7f7db9d81700 1 -- 172.21.15.146:0/1615047318 >> 172.21.15.146:0/1615047318 conn(0x7f7db4066090 msgr2=0x7f7db406fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:29.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.254+0000 7f7db9d81700 1 -- 172.21.15.146:0/1615047318 shutdown_connections 2024-09-13T18:44:29.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:29.254+0000 7f7db9d81700 1 -- 172.21.15.146:0/1615047318 wait complete. 2024-09-13T18:44:29.280 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 73 2024-09-13T18:44:29.292 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:44:31.294 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:31.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc83587a700 1 Processor -- start 2024-09-13T18:44:31.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc83587a700 1 -- start start 2024-09-13T18:44:31.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc83587a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8301506f0 0x7fc8300cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:31.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc83587a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc83006ec90 con 0x7fc8300adca0 2024-09-13T18:44:31.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc83587a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8300c1da0 con 0x7fc830150ad0 2024-09-13T18:44:31.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc83587a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83006d040 con 0x7fc8301506f0 2024-09-13T18:44:31.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc82f7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc8300adca0 0x7fc8300c6ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:47982/0 (socket says 172.21.15.146:47982) 2024-09-13T18:44:31.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc82f7fe700 1 -- 172.21.15.146:0/1338448851 learned_addr learned my addr 172.21.15.146:0/1338448851 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:31.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc82effd700 1 --2- 172.21.15.146:0/1338448851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8301506f0 0x7fc8300cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:31.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc82dffb700 1 -- 172.21.15.146:0/1338448851 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775101956 0 0) 0x7fc828001110 con 0x7fc8300adca0 2024-09-13T18:44:31.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc82dffb700 1 -- 172.21.15.146:0/1338448851 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8180057e0 con 0x7fc8300adca0 2024-09-13T18:44:31.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc82dffb700 1 -- 172.21.15.146:0/1338448851 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152085965 0 0) 0x7fc8300c1da0 con 0x7fc830150ad0 2024-09-13T18:44:31.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc82dffb700 1 -- 172.21.15.146:0/1338448851 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc828001110 con 0x7fc830150ad0 2024-09-13T18:44:31.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.390+0000 7fc82effd700 1 -- 172.21.15.146:0/1338448851 >> v1:172.21.15.154:6789/0 conn(0x7fc830150ad0 legacy=0x7fc83006e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:31.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc82effd700 1 -- 172.21.15.146:0/1338448851 >> v1:172.21.15.146:6789/0 conn(0x7fc8300adca0 legacy=0x7fc8300c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:31.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc82dffb700 1 -- 172.21.15.146:0/1338448851 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1216159578 0 0) 0x7fc8180057e0 con 0x7fc8300adca0 2024-09-13T18:44:31.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc82effd700 1 -- 172.21.15.146:0/1338448851 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83006c7e0 con 0x7fc8301506f0 2024-09-13T18:44:31.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc82effd700 1 --2- 172.21.15.146:0/1338448851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8301506f0 0x7fc8300cbcf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc824009880 tx=0x7fc824009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eabc84b2529699 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:31.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc82dffb700 1 -- 172.21.15.146:0/1338448851 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828001800 con 0x7fc8301506f0 2024-09-13T18:44:31.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc82dffb700 1 -- 172.21.15.146:0/1338448851 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc824004030 con 0x7fc8301506f0 2024-09-13T18:44:31.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc82dffb700 1 -- 172.21.15.146:0/1338448851 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8240043c0 con 0x7fc8301506f0 2024-09-13T18:44:31.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc83587a700 1 -- 172.21.15.146:0/1338448851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8301506f0 msgr2=0x7fc8300cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:31.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc83587a700 1 --2- 172.21.15.146:0/1338448851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8301506f0 0x7fc8300cbcf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc824009880 tx=0x7fc824009b90 comp rx=0 tx=0).stop 2024-09-13T18:44:31.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc83587a700 1 -- 172.21.15.146:0/1338448851 shutdown_connections 2024-09-13T18:44:31.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc83587a700 1 --2- 172.21.15.146:0/1338448851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8301506f0 0x7fc8300cbcf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:31.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc83587a700 1 -- 172.21.15.146:0/1338448851 >> 172.21.15.146:0/1338448851 conn(0x7fc8300cb1d0 msgr2=0x7fc8300cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:31.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc83587a700 1 -- 172.21.15.146:0/1338448851 shutdown_connections 2024-09-13T18:44:31.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.394+0000 7fc83587a700 1 -- 172.21.15.146:0/1338448851 wait complete. 2024-09-13T18:44:31.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc83587a700 1 Processor -- start 2024-09-13T18:44:31.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc83587a700 1 -- start start 2024-09-13T18:44:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc83587a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8300adca0 0x7fc8300c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc83587a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8301506f0 0x7fc8301f0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc83587a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc830150ad0 0x7fc8300c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc83587a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc830151780 con 0x7fc8300adca0 2024-09-13T18:44:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc83587a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300c66c0 con 0x7fc8301506f0 2024-09-13T18:44:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc83587a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300b10f0 con 0x7fc830150ad0 2024-09-13T18:44:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc82effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8300adca0 0x7fc8300c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc82effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8300adca0 0x7fc8300c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52472/0 (socket says 172.21.15.146:52472) 2024-09-13T18:44:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc82effd700 1 -- 172.21.15.146:0/3844460404 learned_addr learned my addr 172.21.15.146:0/3844460404 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc82f7fe700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc830150ad0 0x7fc8300c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc82e7fc700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8301506f0 0x7fc8301f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc82effd700 1 -- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc830150ad0 msgr2=0x7fc8300c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc82effd700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc830150ad0 0x7fc8300c6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc82effd700 1 -- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8301506f0 msgr2=0x7fc8301f0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc82effd700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8301506f0 0x7fc8301f0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc82effd700 1 -- 172.21.15.146:0/3844460404 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240094b0 con 0x7fc8300adca0 2024-09-13T18:44:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc82f7fe700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc830150ad0 0x7fc8300c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc82e7fc700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8301506f0 0x7fc8301f0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:31.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc82effd700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8300adca0 0x7fc8300c4780 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc824014040 tx=0x7fc824004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:31.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc817fff700 1 -- 172.21.15.146:0/3844460404 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824003d40 con 0x7fc8300adca0 2024-09-13T18:44:31.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc817fff700 1 -- 172.21.15.146:0/3844460404 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc824011630 con 0x7fc8300adca0 2024-09-13T18:44:31.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc83587a700 1 -- 172.21.15.146:0/3844460404 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8300b1390 con 0x7fc8300adca0 2024-09-13T18:44:31.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc817fff700 1 -- 172.21.15.146:0/3844460404 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824003ea0 con 0x7fc8300adca0 2024-09-13T18:44:31.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.398+0000 7fc83587a700 1 -- 172.21.15.146:0/3844460404 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8301f3130 con 0x7fc8300adca0 2024-09-13T18:44:31.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.402+0000 7fc83587a700 1 -- 172.21.15.146:0/3844460404 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8301f3350 con 0x7fc8300adca0 2024-09-13T18:44:31.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.402+0000 7fc817fff700 1 -- 172.21.15.146:0/3844460404 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc824018050 con 0x7fc8300adca0 2024-09-13T18:44:31.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.402+0000 7fc817fff700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc81c066730 0x7fc81c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:31.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.402+0000 7fc817fff700 1 -- 172.21.15.146:0/3844460404 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fc824085280 con 0x7fc8300adca0 2024-09-13T18:44:31.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.402+0000 7fc82e7fc700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc81c066730 0x7fc81c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:31.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.402+0000 7fc82e7fc700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc81c066730 0x7fc81c068bf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc830074020 tx=0x7fc820009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:31.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.410+0000 7fc817fff700 1 -- 172.21.15.146:0/3844460404 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc824051fc0 con 0x7fc8300adca0 2024-09-13T18:44:31.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.610+0000 7fc83587a700 1 -- 172.21.15.146:0/3844460404 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8300cbcf0 con 0x7fc8300adca0 2024-09-13T18:44:31.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.614+0000 7fc817fff700 1 -- 172.21.15.146:0/3844460404 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+6276 (secure 0 0 0) 0x7fc8240555e0 con 0x7fc8300adca0 2024-09-13T18:44:31.637 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:31.637 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:17.579418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5800,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:31.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.614+0000 7fc83587a700 1 -- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc81c066730 msgr2=0x7fc81c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:31.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.614+0000 7fc83587a700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc81c066730 0x7fc81c068bf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc830074020 tx=0x7fc820009040 comp rx=0 tx=0).stop 2024-09-13T18:44:31.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.614+0000 7fc83587a700 1 -- 172.21.15.146:0/3844460404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8300adca0 msgr2=0x7fc8300c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:31.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.614+0000 7fc83587a700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8300adca0 0x7fc8300c4780 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc824014040 tx=0x7fc824004770 comp rx=0 tx=0).stop 2024-09-13T18:44:31.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.614+0000 7fc83587a700 1 -- 172.21.15.146:0/3844460404 shutdown_connections 2024-09-13T18:44:31.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.614+0000 7fc83587a700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc81c066730 0x7fc81c068bf0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:31.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.614+0000 7fc83587a700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc830150ad0 0x7fc8300c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:31.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.614+0000 7fc83587a700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8301506f0 0x7fc8301f0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:31.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.614+0000 7fc83587a700 1 --2- 172.21.15.146:0/3844460404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8300adca0 0x7fc8300c4780 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:31.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.614+0000 7fc83587a700 1 -- 172.21.15.146:0/3844460404 >> 172.21.15.146:0/3844460404 conn(0x7fc8300cb1d0 msgr2=0x7fc8300c7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:31.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.614+0000 7fc83587a700 1 -- 172.21.15.146:0/3844460404 shutdown_connections 2024-09-13T18:44:31.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:31.614+0000 7fc83587a700 1 -- 172.21.15.146:0/3844460404 wait complete. 2024-09-13T18:44:31.640 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 73 2024-09-13T18:44:31.651 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:44:33.652 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:33.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 Processor -- start 2024-09-13T18:44:33.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 -- start start 2024-09-13T18:44:33.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c080caed0 0x7f0c080723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:33.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c080cb2b0 0x7f0c081619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:33.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c08161550 con 0x7f0c080a9840 2024-09-13T18:44:33.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c08072a30 con 0x7f0c080cb2b0 2024-09-13T18:44:33.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c080aa800 con 0x7f0c080caed0 2024-09-13T18:44:33.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c067fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c080cb2b0 0x7f0c081619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c067fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c080cb2b0 0x7f0c081619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52488/0 (socket says 172.21.15.146:52488) 2024-09-13T18:44:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c067fc700 1 -- 172.21.15.146:0/4190407168 learned_addr learned my addr 172.21.15.146:0/4190407168 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c077fe700 1 --1- 172.21.15.146:0/4190407168 >> v1:172.21.15.146:6789/0 conn(0x7f0c080a9840 0x7f0c0815e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:47988/0 (socket says 172.21.15.146:47988) 2024-09-13T18:44:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c06ffd700 1 --2- 172.21.15.146:0/4190407168 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c080caed0 0x7f0c080723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c067fc700 1 -- 172.21.15.146:0/4190407168 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c080caed0 msgr2=0x7f0c080723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c067fc700 1 --2- 172.21.15.146:0/4190407168 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c080caed0 0x7f0c080723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c067fc700 1 -- 172.21.15.146:0/4190407168 >> v1:172.21.15.146:6789/0 conn(0x7f0c080a9840 legacy=0x7f0c0815e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c067fc700 1 -- 172.21.15.146:0/4190407168 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c08161220 con 0x7f0c080cb2b0 2024-09-13T18:44:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c067fc700 1 --2- 172.21.15.146:0/4190407168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c080cb2b0 0x7f0c081619e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009910 tx=0x7f0bf8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5e95da9809e854 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c05ffb700 1 -- 172.21.15.146:0/4190407168 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800ba60 con 0x7f0c080cb2b0 2024-09-13T18:44:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c05ffb700 1 -- 172.21.15.146:0/4190407168 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf800bbc0 con 0x7f0c080cb2b0 2024-09-13T18:44:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c05ffb700 1 -- 172.21.15.146:0/4190407168 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8009e60 con 0x7f0c080cb2b0 2024-09-13T18:44:33.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 -- 172.21.15.146:0/4190407168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c080cb2b0 msgr2=0x7f0c081619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:33.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 --2- 172.21.15.146:0/4190407168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c080cb2b0 0x7f0c081619e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009910 tx=0x7f0bf8009c20 comp rx=0 tx=0).stop 2024-09-13T18:44:33.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 -- 172.21.15.146:0/4190407168 shutdown_connections 2024-09-13T18:44:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 --2- 172.21.15.146:0/4190407168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c080cb2b0 0x7f0c081619e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 --2- 172.21.15.146:0/4190407168 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c080caed0 0x7f0c080723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 -- 172.21.15.146:0/4190407168 >> 172.21.15.146:0/4190407168 conn(0x7f0c08066090 msgr2=0x7f0c08066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 -- 172.21.15.146:0/4190407168 shutdown_connections 2024-09-13T18:44:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.746+0000 7f0c07fff700 1 -- 172.21.15.146:0/4190407168 wait complete. 2024-09-13T18:44:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c07fff700 1 Processor -- start 2024-09-13T18:44:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c07fff700 1 -- start start 2024-09-13T18:44:33.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c07fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c080a9840 0x7f0c08124630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:33.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c07fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c080caed0 0x7f0c081f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:33.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c07fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c080cb2b0 0x7f0c081eaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:33.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c07fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c08123e50 con 0x7f0c080caed0 2024-09-13T18:44:33.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c07fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c081f3e30 con 0x7f0c080a9840 2024-09-13T18:44:33.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c07fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c08124080 con 0x7f0c080cb2b0 2024-09-13T18:44:33.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c067fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c080caed0 0x7f0c081f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:33.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c067fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c080caed0 0x7f0c081f38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52502/0 (socket says 172.21.15.146:52502) 2024-09-13T18:44:33.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c067fc700 1 -- 172.21.15.146:0/2609904683 learned_addr learned my addr 172.21.15.146:0/2609904683 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:33.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c06ffd700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c080a9840 0x7f0c08124630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:33.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c077fe700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c080cb2b0 0x7f0c081eaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:33.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c067fc700 1 -- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c080cb2b0 msgr2=0x7f0c081eaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c067fc700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c080cb2b0 0x7f0c081eaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c067fc700 1 -- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c080a9840 msgr2=0x7f0c08124630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c067fc700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c080a9840 0x7f0c08124630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c067fc700 1 -- 172.21.15.146:0/2609904683 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf8009570 con 0x7f0c080caed0 2024-09-13T18:44:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.750+0000 7f0c067fc700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c080caed0 0x7f0c081f38f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8000c00 tx=0x7f0bf8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.754+0000 7f0beffff700 1 -- 172.21.15.146:0/2609904683 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8004450 con 0x7f0c080caed0 2024-09-13T18:44:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.754+0000 7f0beffff700 1 -- 172.21.15.146:0/2609904683 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf801f070 con 0x7f0c080caed0 2024-09-13T18:44:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.754+0000 7f0c07fff700 1 -- 172.21.15.146:0/2609904683 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c08124360 con 0x7f0c080caed0 2024-09-13T18:44:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.754+0000 7f0beffff700 1 -- 172.21.15.146:0/2609904683 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80038e0 con 0x7f0c080caed0 2024-09-13T18:44:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.754+0000 7f0c07fff700 1 -- 172.21.15.146:0/2609904683 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c08126910 con 0x7f0c080caed0 2024-09-13T18:44:33.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.754+0000 7f0c07fff700 1 -- 172.21.15.146:0/2609904683 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c080ca4f0 con 0x7f0c080caed0 2024-09-13T18:44:33.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.754+0000 7f0beffff700 1 -- 172.21.15.146:0/2609904683 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0bf8003a40 con 0x7f0c080caed0 2024-09-13T18:44:33.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.754+0000 7f0beffff700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0bf4066730 0x7f0bf4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:33.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.754+0000 7f0beffff700 1 -- 172.21.15.146:0/2609904683 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f0bf8021070 con 0x7f0c080caed0 2024-09-13T18:44:33.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.754+0000 7f0c06ffd700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0bf4066730 0x7f0bf4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:33.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.758+0000 7f0c06ffd700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0bf4066730 0x7f0bf4068bf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc007a60 tx=0x7f0bfc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:33.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.762+0000 7f0beffff700 1 -- 172.21.15.146:0/2609904683 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0bf80529e0 con 0x7f0c080caed0 2024-09-13T18:44:33.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.974+0000 7f0c07fff700 1 -- 172.21.15.146:0/2609904683 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c0806ec10 con 0x7f0c080caed0 2024-09-13T18:44:33.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.974+0000 7f0beffff700 1 -- 172.21.15.146:0/2609904683 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f0bf8015090 con 0x7f0c080caed0 2024-09-13T18:44:33.998 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:33.999 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:17.579418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5800,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5800":{"gid":5800,"name":"f","rank":0,"incarnation":52,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/776638146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":776638146},{"type":"v1","addr":"172.21.15.154:6839","nonce":776638146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:34.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.978+0000 7f0c07fff700 1 -- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0bf4066730 msgr2=0x7f0bf4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:34.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.978+0000 7f0c07fff700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0bf4066730 0x7f0bf4068bf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc007a60 tx=0x7f0bfc008040 comp rx=0 tx=0).stop 2024-09-13T18:44:34.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.978+0000 7f0c07fff700 1 -- 172.21.15.146:0/2609904683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c080caed0 msgr2=0x7f0c081f38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:34.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.978+0000 7f0c07fff700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c080caed0 0x7f0c081f38f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8000c00 tx=0x7f0bf8004770 comp rx=0 tx=0).stop 2024-09-13T18:44:34.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.978+0000 7f0c07fff700 1 -- 172.21.15.146:0/2609904683 shutdown_connections 2024-09-13T18:44:34.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.978+0000 7f0c07fff700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0bf4066730 0x7f0bf4068bf0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:34.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.978+0000 7f0c07fff700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c080cb2b0 0x7f0c081eaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:34.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.978+0000 7f0c07fff700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c080caed0 0x7f0c081f38f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:34.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.978+0000 7f0c07fff700 1 --2- 172.21.15.146:0/2609904683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c080a9840 0x7f0c08124630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:34.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.978+0000 7f0c07fff700 1 -- 172.21.15.146:0/2609904683 >> 172.21.15.146:0/2609904683 conn(0x7f0c08066090 msgr2=0x7f0c0815faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:34.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.978+0000 7f0c07fff700 1 -- 172.21.15.146:0/2609904683 shutdown_connections 2024-09-13T18:44:34.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:33.978+0000 7f0c07fff700 1 -- 172.21.15.146:0/2609904683 wait complete. 2024-09-13T18:44:34.003 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 73 2024-09-13T18:44:34.013 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:44:36.013 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:36.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b48951700 1 Processor -- start 2024-09-13T18:44:36.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b48951700 1 -- start start 2024-09-13T18:44:36.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b48951700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6b440c2f20 0x7f6b4406ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b48951700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4406b2e0 con 0x7f6b4415caf0 2024-09-13T18:44:36.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b48951700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b44071700 con 0x7f6b4414ff70 2024-09-13T18:44:36.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b48951700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b44070850 con 0x7f6b440c2f20 2024-09-13T18:44:36.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b42ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6b440c2f20 0x7f6b4406ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b43fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6b4415caf0 0x7f6b4415cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:47994/0 (socket says 172.21.15.146:47994) 2024-09-13T18:44:36.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b43fff700 1 -- 172.21.15.146:0/1948293356 learned_addr learned my addr 172.21.15.146:0/1948293356 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:36.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b42ffd700 1 -- 172.21.15.146:0/1948293356 >> v1:172.21.15.154:6789/0 conn(0x7f6b4414ff70 legacy=0x7f6b440c2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:36.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b42ffd700 1 -- 172.21.15.146:0/1948293356 >> v1:172.21.15.146:6789/0 conn(0x7f6b4415caf0 legacy=0x7f6b4415cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:36.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b42ffd700 1 -- 172.21.15.146:0/1948293356 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b44070af0 con 0x7f6b440c2f20 2024-09-13T18:44:36.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b42ffd700 1 --2- 172.21.15.146:0/1948293356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b440c2f20 0x7f6b4406ada0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c009850 tx=0x7f6b2c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19313da77b00488 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:36.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b427fc700 1 -- 172.21.15.146:0/1948293356 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c011830 con 0x7f6b440c2f20 2024-09-13T18:44:36.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b427fc700 1 -- 172.21.15.146:0/1948293356 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2c011990 con 0x7f6b440c2f20 2024-09-13T18:44:36.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b427fc700 1 -- 172.21.15.146:0/1948293356 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c011d20 con 0x7f6b440c2f20 2024-09-13T18:44:36.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b48951700 1 -- 172.21.15.146:0/1948293356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b440c2f20 msgr2=0x7f6b4406ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:36.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b48951700 1 --2- 172.21.15.146:0/1948293356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b440c2f20 0x7f6b4406ada0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c009850 tx=0x7f6b2c009b60 comp rx=0 tx=0).stop 2024-09-13T18:44:36.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b48951700 1 -- 172.21.15.146:0/1948293356 shutdown_connections 2024-09-13T18:44:36.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b48951700 1 --2- 172.21.15.146:0/1948293356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b440c2f20 0x7f6b4406ada0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b48951700 1 -- 172.21.15.146:0/1948293356 >> 172.21.15.146:0/1948293356 conn(0x7f6b4406f5c0 msgr2=0x7f6b4406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:36.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.106+0000 7f6b48951700 1 -- 172.21.15.146:0/1948293356 shutdown_connections 2024-09-13T18:44:36.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b48951700 1 -- 172.21.15.146:0/1948293356 wait complete. 2024-09-13T18:44:36.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b48951700 1 Processor -- start 2024-09-13T18:44:36.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b48951700 1 -- start start 2024-09-13T18:44:36.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b48951700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b440c2f20 0x7f6b441e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b48951700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b4414ff70 0x7f6b441f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b48951700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b4415caf0 0x7f6b44146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b48951700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b440c7510 con 0x7f6b440c2f20 2024-09-13T18:44:36.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b48951700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b440c6fc0 con 0x7f6b4415caf0 2024-09-13T18:44:36.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b48951700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b441e5330 con 0x7f6b4414ff70 2024-09-13T18:44:36.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b437fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b440c2f20 0x7f6b441e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b437fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b440c2f20 0x7f6b441e4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52520/0 (socket says 172.21.15.146:52520) 2024-09-13T18:44:36.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b437fe700 1 -- 172.21.15.146:0/3129293084 learned_addr learned my addr 172.21.15.146:0/3129293084 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:36.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b42ffd700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b4414ff70 0x7f6b441f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b43fff700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b4415caf0 0x7f6b44146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b437fe700 1 -- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b4414ff70 msgr2=0x7f6b441f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:36.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b437fe700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b4414ff70 0x7f6b441f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b437fe700 1 -- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b4415caf0 msgr2=0x7f6b44146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:36.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b437fe700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b4415caf0 0x7f6b44146900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.110+0000 7f6b437fe700 1 -- 172.21.15.146:0/3129293084 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c0094b0 con 0x7f6b440c2f20 2024-09-13T18:44:36.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.114+0000 7f6b437fe700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b440c2f20 0x7f6b441e4df0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400c530 tx=0x7f6b3400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:36.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.114+0000 7f6b3bfff700 1 -- 172.21.15.146:0/3129293084 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b340052d0 con 0x7f6b440c2f20 2024-09-13T18:44:36.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.114+0000 7f6b3bfff700 1 -- 172.21.15.146:0/3129293084 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b34005430 con 0x7f6b440c2f20 2024-09-13T18:44:36.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.114+0000 7f6b48951700 1 -- 172.21.15.146:0/3129293084 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b441f2e20 con 0x7f6b440c2f20 2024-09-13T18:44:36.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.114+0000 7f6b3bfff700 1 -- 172.21.15.146:0/3129293084 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b340055a0 con 0x7f6b440c2f20 2024-09-13T18:44:36.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.114+0000 7f6b48951700 1 -- 172.21.15.146:0/3129293084 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b440c7230 con 0x7f6b440c2f20 2024-09-13T18:44:36.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.114+0000 7f6b48951700 1 -- 172.21.15.146:0/3129293084 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b440c7d10 con 0x7f6b440c2f20 2024-09-13T18:44:36.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.114+0000 7f6b3bfff700 1 -- 172.21.15.146:0/3129293084 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b340036a0 con 0x7f6b440c2f20 2024-09-13T18:44:36.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.114+0000 7f6b3bfff700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6b24066760 0x7f6b24068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.114+0000 7f6b3bfff700 1 -- 172.21.15.146:0/3129293084 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f6b34083a40 con 0x7f6b440c2f20 2024-09-13T18:44:36.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.114+0000 7f6b42ffd700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6b24066760 0x7f6b24068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.118+0000 7f6b42ffd700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6b24066760 0x7f6b24068c20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c009fd0 tx=0x7f6b2c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:36.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.122+0000 7f6b3bfff700 1 -- 172.21.15.146:0/3129293084 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6b34050680 con 0x7f6b440c2f20 2024-09-13T18:44:36.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.330+0000 7f6b48951700 1 -- 172.21.15.146:0/3129293084 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b440caa90 con 0x7f6b440c2f20 2024-09-13T18:44:36.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.330+0000 7f6b3bfff700 1 -- 172.21.15.146:0/3129293084 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5518 (secure 0 0 0) 0x7f6b34053ca0 con 0x7f6b440c2f20 2024-09-13T18:44:36.353 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:36.354 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:35.791682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:36.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.330+0000 7f6b48951700 1 -- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6b24066760 msgr2=0x7f6b24068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:36.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.330+0000 7f6b48951700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6b24066760 0x7f6b24068c20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c009fd0 tx=0x7f6b2c008030 comp rx=0 tx=0).stop 2024-09-13T18:44:36.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.330+0000 7f6b48951700 1 -- 172.21.15.146:0/3129293084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b440c2f20 msgr2=0x7f6b441e4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:36.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.330+0000 7f6b48951700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b440c2f20 0x7f6b441e4df0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400c530 tx=0x7f6b3400c840 comp rx=0 tx=0).stop 2024-09-13T18:44:36.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.334+0000 7f6b48951700 1 -- 172.21.15.146:0/3129293084 shutdown_connections 2024-09-13T18:44:36.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.334+0000 7f6b48951700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6b24066760 0x7f6b24068c20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.334+0000 7f6b48951700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b4415caf0 0x7f6b44146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.334+0000 7f6b48951700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b4414ff70 0x7f6b441f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.334+0000 7f6b48951700 1 --2- 172.21.15.146:0/3129293084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b440c2f20 0x7f6b441e4df0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.334+0000 7f6b48951700 1 -- 172.21.15.146:0/3129293084 >> 172.21.15.146:0/3129293084 conn(0x7f6b4406f5c0 msgr2=0x7f6b4414e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:36.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.334+0000 7f6b48951700 1 -- 172.21.15.146:0/3129293084 shutdown_connections 2024-09-13T18:44:36.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.334+0000 7f6b48951700 1 -- 172.21.15.146:0/3129293084 wait complete. 2024-09-13T18:44:36.357 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 74 2024-09-13T18:44:36.370 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2024-09-13T18:44:36.370 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:44:36.370 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:36.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8398ee700 1 Processor -- start 2024-09-13T18:44:36.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8398ee700 1 -- start start 2024-09-13T18:44:36.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8398ee700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa8340cbd50 0x7fa8340b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8398ee700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8340ca210 0x7fa8340cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8398ee700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8340cc670 con 0x7fa834071390 2024-09-13T18:44:36.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8398ee700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8340caa30 con 0x7fa8340cbd50 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8398ee700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340cb810 con 0x7fa8340ca210 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8390ed700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa834071390 0x7fa8340701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48010/0 (socket says 172.21.15.146:48010) 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8390ed700 1 -- 172.21.15.146:0/3402170095 learned_addr learned my addr 172.21.15.146:0/3402170095 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189759247 0 0) 0x7fa82c001490 con 0x7fa834071390 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8240035d0 con 0x7fa834071390 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8388ec700 1 --2- 172.21.15.146:0/3402170095 >> v2:172.21.15.154:3300/0 conn(0x7fa8340cbd50 0x7fa8340b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa833fff700 1 --2- 172.21.15.146:0/3402170095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8340ca210 0x7fa8340cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 311169182 0 0) 0x7fa8240035d0 con 0x7fa834071390 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8340ca210 msgr2=0x7fa8340cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 --2- 172.21.15.146:0/3402170095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8340ca210 0x7fa8340cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 >> v2:172.21.15.154:3300/0 conn(0x7fa8340cbd50 msgr2=0x7fa8340b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 --2- 172.21.15.146:0/3402170095 >> v2:172.21.15.154:3300/0 conn(0x7fa8340cbd50 0x7fa8340b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83415da20 con 0x7fa834071390 2024-09-13T18:44:36.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7fa82c001b80 con 0x7fa834071390 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 >> v1:172.21.15.146:6789/0 conn(0x7fa834071390 legacy=0x7fa8340701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 --2- 172.21.15.146:0/3402170095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8340ca210 0x7fa824003ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 --2- 172.21.15.146:0/3402170095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa824004040 0x7fa824004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 --2- 172.21.15.146:0/3402170095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa824004db0 0x7fa824005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340caa30 con 0x7fa824004040 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340cb810 con 0x7fa824004db0 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82400ba80 con 0x7fa8340ca210 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa833fff700 1 --2- 172.21.15.146:0/3402170095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa824004040 0x7fa824004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8388ec700 1 --2- 172.21.15.146:0/3402170095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8340ca210 0x7fa824003ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.450+0000 7fa8390ed700 1 --2- 172.21.15.146:0/3402170095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa824004db0 0x7fa824005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa833fff700 1 -- 172.21.15.146:0/3402170095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8340ca210 msgr2=0x7fa824003ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa833fff700 1 --2- 172.21.15.146:0/3402170095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8340ca210 0x7fa824003ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa833fff700 1 -- 172.21.15.146:0/3402170095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa824004db0 msgr2=0x7fa824005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa833fff700 1 --2- 172.21.15.146:0/3402170095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa824004db0 0x7fa824005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa833fff700 1 -- 172.21.15.146:0/3402170095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8240035d0 con 0x7fa824004040 2024-09-13T18:44:36.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa833fff700 1 --2- 172.21.15.146:0/3402170095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa824004040 0x7fa824004870 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa820009ba0 tx=0x7fa8240058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50a6c5cea7a54e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:36.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82000e6c0 con 0x7fa824004040 2024-09-13T18:44:36.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82000e820 con 0x7fa824004040 2024-09-13T18:44:36.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa8337fe700 1 -- 172.21.15.146:0/3402170095 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82000eb90 con 0x7fa824004040 2024-09-13T18:44:36.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa8398ee700 1 -- 172.21.15.146:0/3402170095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa824004040 msgr2=0x7fa824004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:36.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa8398ee700 1 --2- 172.21.15.146:0/3402170095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa824004040 0x7fa824004870 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa820009ba0 tx=0x7fa8240058a0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa8390ed700 1 -- 172.21.15.146:0/3402170095 reap_dead start 2024-09-13T18:44:36.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa8398ee700 1 -- 172.21.15.146:0/3402170095 shutdown_connections 2024-09-13T18:44:36.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa8398ee700 1 -- 172.21.15.146:0/3402170095 >> 172.21.15.146:0/3402170095 conn(0x7fa834066270 msgr2=0x7fa834066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:36.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa8398ee700 1 -- 172.21.15.146:0/3402170095 shutdown_connections 2024-09-13T18:44:36.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa8398ee700 1 -- 172.21.15.146:0/3402170095 wait complete. 2024-09-13T18:44:36.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa8398ee700 1 Processor -- start 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.454+0000 7fa8398ee700 1 -- start start 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8398ee700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa824004040 0x7fa8340b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8398ee700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa83415ae50 0x7fa83415b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8398ee700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8341f2630 0x7fa8341f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8398ee700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83415c3e0 con 0x7fa824004040 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8398ee700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340cd530 con 0x7fa8341f2630 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8398ee700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340c76e0 con 0x7fa83415ae50 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8388ec700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa824004040 0x7fa8340b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8388ec700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa824004040 0x7fa8340b2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52532/0 (socket says 172.21.15.146:52532) 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8388ec700 1 -- 172.21.15.146:0/1658631580 learned_addr learned my addr 172.21.15.146:0/1658631580 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8390ed700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8341f2630 0x7fa8341f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa833fff700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa83415ae50 0x7fa83415b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8388ec700 1 -- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa83415ae50 msgr2=0x7fa83415b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:36.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8388ec700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa83415ae50 0x7fa83415b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8388ec700 1 -- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8341f2630 msgr2=0x7fa8341f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:36.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8388ec700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8341f2630 0x7fa8341f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8388ec700 1 -- 172.21.15.146:0/1658631580 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83415a690 con 0x7fa824004040 2024-09-13T18:44:36.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa833fff700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa83415ae50 0x7fa83415b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:36.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8388ec700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa824004040 0x7fa8340b2c70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa828011aa0 tx=0x7fa828011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:36.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8317fa700 1 -- 172.21.15.146:0/1658631580 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82800f920 con 0x7fa824004040 2024-09-13T18:44:36.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8317fa700 1 -- 172.21.15.146:0/1658631580 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82800fa80 con 0x7fa824004040 2024-09-13T18:44:36.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8398ee700 1 -- 172.21.15.146:0/1658631580 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83415c100 con 0x7fa824004040 2024-09-13T18:44:36.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8317fa700 1 -- 172.21.15.146:0/1658631580 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82800fe30 con 0x7fa824004040 2024-09-13T18:44:36.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8398ee700 1 -- 172.21.15.146:0/1658631580 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83415bc40 con 0x7fa824004040 2024-09-13T18:44:36.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8398ee700 1 -- 172.21.15.146:0/1658631580 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83415d500 con 0x7fa824004040 2024-09-13T18:44:36.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.458+0000 7fa8317fa700 1 -- 172.21.15.146:0/1658631580 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa82801d400 con 0x7fa824004040 2024-09-13T18:44:36.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.462+0000 7fa8317fa700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa818066730 0x7fa818068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:36.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.462+0000 7fa8317fa700 1 -- 172.21.15.146:0/1658631580 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fa828085060 con 0x7fa824004040 2024-09-13T18:44:36.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.462+0000 7fa833fff700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa818066730 0x7fa818068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:36.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.462+0000 7fa833fff700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa818066730 0x7fa818068bf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa8200090b0 tx=0x7fa820008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:36.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.466+0000 7fa8317fa700 1 -- 172.21.15.146:0/1658631580 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa828051cb0 con 0x7fa824004040 2024-09-13T18:44:36.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.670+0000 7fa8398ee700 1 -- 172.21.15.146:0/1658631580 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8340c9ff0 con 0x7fa824004040 2024-09-13T18:44:36.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.670+0000 7fa8317fa700 1 -- 172.21.15.146:0/1658631580 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5518 (secure 0 0 0) 0x7fa8280552d0 con 0x7fa824004040 2024-09-13T18:44:36.695 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:36.695 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:35.791682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.674+0000 7fa8398ee700 1 -- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa818066730 msgr2=0x7fa818068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:36.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.674+0000 7fa8398ee700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa818066730 0x7fa818068bf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa8200090b0 tx=0x7fa820008030 comp rx=0 tx=0).stop 2024-09-13T18:44:36.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.674+0000 7fa8398ee700 1 -- 172.21.15.146:0/1658631580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa824004040 msgr2=0x7fa8340b2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:36.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.674+0000 7fa8398ee700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa824004040 0x7fa8340b2c70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa828011aa0 tx=0x7fa828011e60 comp rx=0 tx=0).stop 2024-09-13T18:44:36.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.674+0000 7fa8398ee700 1 -- 172.21.15.146:0/1658631580 shutdown_connections 2024-09-13T18:44:36.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.674+0000 7fa8398ee700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa818066730 0x7fa818068bf0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.674+0000 7fa8398ee700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8341f2630 0x7fa8341f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.674+0000 7fa8398ee700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa83415ae50 0x7fa83415b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.674+0000 7fa8398ee700 1 --2- 172.21.15.146:0/1658631580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa824004040 0x7fa8340b2c70 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:36.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.674+0000 7fa8398ee700 1 -- 172.21.15.146:0/1658631580 >> 172.21.15.146:0/1658631580 conn(0x7fa834066270 msgr2=0x7fa83414ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:36.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.674+0000 7fa8398ee700 1 -- 172.21.15.146:0/1658631580 shutdown_connections 2024-09-13T18:44:36.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:36.674+0000 7fa8398ee700 1 -- 172.21.15.146:0/1658631580 wait complete. 2024-09-13T18:44:36.699 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 74 2024-09-13T18:44:36.709 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 74, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 74, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6207": { "addr": "172.21.15.146:6837/3234109877", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3234109877, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3234109877, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6207, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 39 }, "gid_6459": { "addr": "172.21.15.146:6835/294014284", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 294014284, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 294014284, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6459, "incarnation": 74, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 37, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:44:35.791682+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6459, "mds_1": 6207 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.146:6833/3023779168", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3023779168, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3023779168, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 69, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6187, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:44:38.710 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:38.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.807+0000 7f3d36a34700 1 Processor -- start 2024-09-13T18:44:38.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.807+0000 7f3d36a34700 1 -- start start 2024-09-13T18:44:38.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.807+0000 7f3d36a34700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d3006de80 0x7f3d3006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:38.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.807+0000 7f3d36a34700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3d30070420 0x7f3d300c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:38.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.807+0000 7f3d36a34700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d300c7910 con 0x7f3d301575e0 2024-09-13T18:44:38.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.807+0000 7f3d36a34700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d3006edb0 con 0x7f3d30070420 2024-09-13T18:44:38.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.807+0000 7f3d36a34700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d300c7630 con 0x7f3d3006de80 2024-09-13T18:44:38.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.807+0000 7f3d34fd1700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f3d301575e0 0x7f3d30157a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48016/0 (socket says 172.21.15.146:48016) 2024-09-13T18:44:38.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.807+0000 7f3d34fd1700 1 -- 172.21.15.146:0/2099210071 learned_addr learned my addr 172.21.15.146:0/2099210071 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:38.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d2ffff700 1 --2- 172.21.15.146:0/2099210071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d3006de80 0x7f3d3006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:38.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d2f7fe700 1 --2- 172.21.15.146:0/2099210071 >> v2:172.21.15.154:3300/0 conn(0x7f3d30070420 0x7f3d300c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:38.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d2effd700 1 -- 172.21.15.146:0/2099210071 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878364324 0 0) 0x7f3d300c7910 con 0x7f3d301575e0 2024-09-13T18:44:38.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d2effd700 1 -- 172.21.15.146:0/2099210071 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d180035d0 con 0x7f3d301575e0 2024-09-13T18:44:38.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d2f7fe700 1 -- 172.21.15.146:0/2099210071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d3006de80 msgr2=0x7f3d3006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:38.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d2f7fe700 1 --2- 172.21.15.146:0/2099210071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d3006de80 0x7f3d3006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:38.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d2f7fe700 1 -- 172.21.15.146:0/2099210071 >> v1:172.21.15.146:6789/0 conn(0x7f3d301575e0 legacy=0x7f3d30157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:38.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d2f7fe700 1 -- 172.21.15.146:0/2099210071 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3006f660 con 0x7f3d30070420 2024-09-13T18:44:38.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d2f7fe700 1 --2- 172.21.15.146:0/2099210071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d30070420 0x7f3d300c65e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009d90 tx=0x7f3d2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dde1362061d8316 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:38.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d2effd700 1 -- 172.21.15.146:0/2099210071 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2000e070 con 0x7f3d30070420 2024-09-13T18:44:38.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d2effd700 1 -- 172.21.15.146:0/2099210071 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2000bb20 con 0x7f3d30070420 2024-09-13T18:44:38.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d2effd700 1 -- 172.21.15.146:0/2099210071 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d20012430 con 0x7f3d30070420 2024-09-13T18:44:38.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d36a34700 1 -- 172.21.15.146:0/2099210071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d30070420 msgr2=0x7f3d300c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:38.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d36a34700 1 --2- 172.21.15.146:0/2099210071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d30070420 0x7f3d300c65e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009d90 tx=0x7f3d2000b330 comp rx=0 tx=0).stop 2024-09-13T18:44:38.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d36a34700 1 -- 172.21.15.146:0/2099210071 shutdown_connections 2024-09-13T18:44:38.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d36a34700 1 --2- 172.21.15.146:0/2099210071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d30070420 0x7f3d300c65e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:38.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d36a34700 1 --2- 172.21.15.146:0/2099210071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d3006de80 0x7f3d3006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:38.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d36a34700 1 -- 172.21.15.146:0/2099210071 >> 172.21.15.146:0/2099210071 conn(0x7f3d300cb1d0 msgr2=0x7f3d300cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:38.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d36a34700 1 -- 172.21.15.146:0/2099210071 shutdown_connections 2024-09-13T18:44:38.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d36a34700 1 -- 172.21.15.146:0/2099210071 wait complete. 2024-09-13T18:44:38.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d36a34700 1 Processor -- start 2024-09-13T18:44:38.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.811+0000 7f3d36a34700 1 -- start start 2024-09-13T18:44:38.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d36a34700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d3006de80 0x7f3d300b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:38.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d36a34700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d30070420 0x7f3d3014a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:38.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d36a34700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d301575e0 0x7f3d301eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:38.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d36a34700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d300c50d0 con 0x7f3d30070420 2024-09-13T18:44:38.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d36a34700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d30152050 con 0x7f3d301575e0 2024-09-13T18:44:38.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d36a34700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d300af5f0 con 0x7f3d3006de80 2024-09-13T18:44:38.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d30070420 0x7f3d3014a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:38.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d30070420 0x7f3d3014a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52540/0 (socket says 172.21.15.146:52540) 2024-09-13T18:44:38.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2f7fe700 1 -- 172.21.15.146:0/3214697898 learned_addr learned my addr 172.21.15.146:0/3214697898 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:38.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d34fd1700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d301575e0 0x7f3d301eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:38.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2ffff700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d3006de80 0x7f3d300b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:38.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2f7fe700 1 -- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d3006de80 msgr2=0x7f3d300b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:38.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2f7fe700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d3006de80 0x7f3d300b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:38.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2f7fe700 1 -- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d301575e0 msgr2=0x7f3d301eecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:38.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2f7fe700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d301575e0 0x7f3d301eecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:38.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2f7fe700 1 -- 172.21.15.146:0/3214697898 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d20009a70 con 0x7f3d30070420 2024-09-13T18:44:38.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d34fd1700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d301575e0 0x7f3d301eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:38.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2ffff700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d3006de80 0x7f3d300b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:38.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2f7fe700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d30070420 0x7f3d3014a600 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3d20014040 tx=0x7f3d20004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:38.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2cff9700 1 -- 172.21.15.146:0/3214697898 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d20012430 con 0x7f3d30070420 2024-09-13T18:44:38.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2cff9700 1 -- 172.21.15.146:0/3214697898 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d20020070 con 0x7f3d30070420 2024-09-13T18:44:38.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d36a34700 1 -- 172.21.15.146:0/3214697898 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d300b1550 con 0x7f3d30070420 2024-09-13T18:44:38.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d2cff9700 1 -- 172.21.15.146:0/3214697898 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d20004280 con 0x7f3d30070420 2024-09-13T18:44:38.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d36a34700 1 -- 172.21.15.146:0/3214697898 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d300ae7b0 con 0x7f3d30070420 2024-09-13T18:44:38.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.815+0000 7f3d36a34700 1 -- 172.21.15.146:0/3214697898 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d300af760 con 0x7f3d30070420 2024-09-13T18:44:38.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.819+0000 7f3d2cff9700 1 -- 172.21.15.146:0/3214697898 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3d200043e0 con 0x7f3d30070420 2024-09-13T18:44:38.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.819+0000 7f3d2cff9700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d1c066730 0x7f3d1c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:38.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.819+0000 7f3d2cff9700 1 -- 172.21.15.146:0/3214697898 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f3d20022070 con 0x7f3d30070420 2024-09-13T18:44:38.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.819+0000 7f3d2ffff700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d1c066730 0x7f3d1c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:38.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.819+0000 7f3d2ffff700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d1c066730 0x7f3d1c068bf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009ba0 tx=0x7f3d24008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:38.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:38.823+0000 7f3d2cff9700 1 -- 172.21.15.146:0/3214697898 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3d20053640 con 0x7f3d30070420 2024-09-13T18:44:39.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.031+0000 7f3d36a34700 1 -- 172.21.15.146:0/3214697898 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d3006e8c0 con 0x7f3d30070420 2024-09-13T18:44:39.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.035+0000 7f3d2cff9700 1 -- 172.21.15.146:0/3214697898 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5518 (secure 0 0 0) 0x7f3d20056c60 con 0x7f3d30070420 2024-09-13T18:44:39.056 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:39.056 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:35.791682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:39.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.035+0000 7f3d36a34700 1 -- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d1c066730 msgr2=0x7f3d1c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:39.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.035+0000 7f3d36a34700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d1c066730 0x7f3d1c068bf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009ba0 tx=0x7f3d24008040 comp rx=0 tx=0).stop 2024-09-13T18:44:39.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.035+0000 7f3d36a34700 1 -- 172.21.15.146:0/3214697898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d30070420 msgr2=0x7f3d3014a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:39.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.035+0000 7f3d36a34700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d30070420 0x7f3d3014a600 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3d20014040 tx=0x7f3d20004af0 comp rx=0 tx=0).stop 2024-09-13T18:44:39.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.035+0000 7f3d36a34700 1 -- 172.21.15.146:0/3214697898 shutdown_connections 2024-09-13T18:44:39.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.035+0000 7f3d36a34700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3d1c066730 0x7f3d1c068bf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:39.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.035+0000 7f3d36a34700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d301575e0 0x7f3d301eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:39.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.035+0000 7f3d36a34700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d30070420 0x7f3d3014a600 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:39.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.035+0000 7f3d36a34700 1 --2- 172.21.15.146:0/3214697898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d3006de80 0x7f3d300b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:39.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.035+0000 7f3d36a34700 1 -- 172.21.15.146:0/3214697898 >> 172.21.15.146:0/3214697898 conn(0x7f3d300cb1d0 msgr2=0x7f3d300cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:39.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.035+0000 7f3d36a34700 1 -- 172.21.15.146:0/3214697898 shutdown_connections 2024-09-13T18:44:39.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:39.035+0000 7f3d36a34700 1 -- 172.21.15.146:0/3214697898 wait complete. 2024-09-13T18:44:39.060 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 74 2024-09-13T18:44:39.071 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:40.390 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:44:40.393 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:44:41.072 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:41.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 Processor -- start 2024-09-13T18:44:41.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 -- start start 2024-09-13T18:44:41.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82080cf170 0x7f82081595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:41.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f82080cb5b0 0x7f82081534c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:41.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820815a540 con 0x7f82080cf550 2024-09-13T18:44:41.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82080c5220 con 0x7f82080cb5b0 2024-09-13T18:44:41.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8208159ca0 con 0x7f82080cf170 2024-09-13T18:44:41.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e0b6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f82080cb5b0 0x7f82081534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:41.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820d0b4700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f82080cf550 0x7f82080c56b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34938/0 (socket says 172.21.15.146:34938) 2024-09-13T18:44:41.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820d0b4700 1 -- 172.21.15.146:0/4062488741 learned_addr learned my addr 172.21.15.146:0/4062488741 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:41.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820d8b5700 1 --2- 172.21.15.146:0/4062488741 >> v2:172.21.15.154:3300/0 conn(0x7f82080cf170 0x7f82081595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:41.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e0b6700 1 -- 172.21.15.146:0/4062488741 >> v2:172.21.15.154:3300/0 conn(0x7f82080cf170 msgr2=0x7f82081595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:41.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e0b6700 1 --2- 172.21.15.146:0/4062488741 >> v2:172.21.15.154:3300/0 conn(0x7f82080cf170 0x7f82081595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:41.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e0b6700 1 -- 172.21.15.146:0/4062488741 >> v1:172.21.15.146:6789/0 conn(0x7f82080cf550 legacy=0x7f82080c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:41.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e0b6700 1 -- 172.21.15.146:0/4062488741 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8208159f80 con 0x7f82080cb5b0 2024-09-13T18:44:41.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820c8b3700 1 -- 172.21.15.146:0/4062488741 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276413522 0 0) 0x7f820815a540 con 0x7f82080cf550 2024-09-13T18:44:41.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820d8b5700 1 --2- 172.21.15.146:0/4062488741 >> v2:172.21.15.154:3300/0 conn(0x7f82080cf170 0x7f82081595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:44:41.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e0b6700 1 --2- 172.21.15.146:0/4062488741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82080cb5b0 0x7f82081534c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f820808c1e0 tx=0x7f81f8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621215191a079111 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:41.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820c8b3700 1 -- 172.21.15.146:0/4062488741 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8011830 con 0x7f82080cb5b0 2024-09-13T18:44:41.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820c8b3700 1 -- 172.21.15.146:0/4062488741 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f8011990 con 0x7f82080cb5b0 2024-09-13T18:44:41.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820c8b3700 1 -- 172.21.15.146:0/4062488741 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8011d40 con 0x7f82080cb5b0 2024-09-13T18:44:41.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 -- 172.21.15.146:0/4062488741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82080cb5b0 msgr2=0x7f82081534c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:41.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 --2- 172.21.15.146:0/4062488741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82080cb5b0 0x7f82081534c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f820808c1e0 tx=0x7f81f8009b30 comp rx=0 tx=0).stop 2024-09-13T18:44:41.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 -- 172.21.15.146:0/4062488741 shutdown_connections 2024-09-13T18:44:41.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 --2- 172.21.15.146:0/4062488741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82080cb5b0 0x7f82081534c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:41.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 --2- 172.21.15.146:0/4062488741 >> v2:172.21.15.154:3300/0 conn(0x7f82080cf170 0x7f82081595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:41.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 -- 172.21.15.146:0/4062488741 >> 172.21.15.146:0/4062488741 conn(0x7f8208072220 msgr2=0x7f8208066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:41.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 -- 172.21.15.146:0/4062488741 shutdown_connections 2024-09-13T18:44:41.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.171+0000 7f820e8b7700 1 -- 172.21.15.146:0/4062488741 wait complete. 2024-09-13T18:44:41.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820e8b7700 1 Processor -- start 2024-09-13T18:44:41.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820e8b7700 1 -- start start 2024-09-13T18:44:41.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820e8b7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82080cb5b0 0x7f8208149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:41.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820e8b7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82080cf170 0x7f82080b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:41.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820e8b7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82080cf550 0x7f82081efad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:41.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820e8b7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8208159950 con 0x7f82080cb5b0 2024-09-13T18:44:41.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820e8b7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82081ef6b0 con 0x7f82080cf170 2024-09-13T18:44:41.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820e8b7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82081ef0c0 con 0x7f82080cf550 2024-09-13T18:44:41.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820d8b5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82080cb5b0 0x7f8208149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:41.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820d8b5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82080cb5b0 0x7f8208149b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48960/0 (socket says 172.21.15.146:48960) 2024-09-13T18:44:41.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820d8b5700 1 -- 172.21.15.146:0/1648757721 learned_addr learned my addr 172.21.15.146:0/1648757721 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:41.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820d0b4700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82080cf170 0x7f82080b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:41.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820e0b6700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82080cf550 0x7f82081efad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:41.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820d8b5700 1 -- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82080cf550 msgr2=0x7f82081efad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:41.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820d8b5700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82080cf550 0x7f82081efad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:41.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820d8b5700 1 -- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82080cf170 msgr2=0x7f82080b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:41.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820d8b5700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82080cf170 0x7f82080b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:41.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820d8b5700 1 -- 172.21.15.146:0/1648757721 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f80094b0 con 0x7f82080cb5b0 2024-09-13T18:44:41.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820d0b4700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82080cf170 0x7f82080b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:44:41.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820d8b5700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82080cb5b0 0x7f8208149b70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f820000cf60 tx=0x7f820000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:41.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f81fe7fc700 1 -- 172.21.15.146:0/1648757721 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8200015440 con 0x7f82080cb5b0 2024-09-13T18:44:41.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f81fe7fc700 1 -- 172.21.15.146:0/1648757721 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82000155a0 con 0x7f82080cb5b0 2024-09-13T18:44:41.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820e8b7700 1 -- 172.21.15.146:0/1648757721 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82081f0010 con 0x7f82080cb5b0 2024-09-13T18:44:41.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f81fe7fc700 1 -- 172.21.15.146:0/1648757721 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8200015930 con 0x7f82080cb5b0 2024-09-13T18:44:41.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.175+0000 7f820e8b7700 1 -- 172.21.15.146:0/1648757721 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8208159670 con 0x7f82080cb5b0 2024-09-13T18:44:41.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.179+0000 7f820e8b7700 1 -- 172.21.15.146:0/1648757721 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8208066850 con 0x7f82080cb5b0 2024-09-13T18:44:41.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.179+0000 7f81fe7fc700 1 -- 172.21.15.146:0/1648757721 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8200015a90 con 0x7f82080cb5b0 2024-09-13T18:44:41.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.179+0000 7f81fe7fc700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f81ec066730 0x7f81ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:41.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.179+0000 7f81fe7fc700 1 -- 172.21.15.146:0/1648757721 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f8200084d40 con 0x7f82080cb5b0 2024-09-13T18:44:41.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.179+0000 7f820d0b4700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f81ec066730 0x7f81ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:41.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.179+0000 7f820d0b4700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f81ec066730 0x7f81ec068bf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f82080cae60 tx=0x7f8204009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:41.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.183+0000 7f81fe7fc700 1 -- 172.21.15.146:0/1648757721 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8200051a40 con 0x7f82080cb5b0 2024-09-13T18:44:41.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f820e8b7700 1 -- 172.21.15.146:0/1648757721 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8208150cf0 con 0x7f82080cb5b0 2024-09-13T18:44:41.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f81fe7fc700 1 -- 172.21.15.146:0/1648757721 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5518 (secure 0 0 0) 0x7f8200055060 con 0x7f82080cb5b0 2024-09-13T18:44:41.413 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:41.413 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:35.791682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:41.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f820e8b7700 1 -- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f81ec066730 msgr2=0x7f81ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:41.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f820e8b7700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f81ec066730 0x7f81ec068bf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f82080cae60 tx=0x7f8204009040 comp rx=0 tx=0).stop 2024-09-13T18:44:41.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f820e8b7700 1 -- 172.21.15.146:0/1648757721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82080cb5b0 msgr2=0x7f8208149b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:41.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f820e8b7700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82080cb5b0 0x7f8208149b70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f820000cf60 tx=0x7f820000cf90 comp rx=0 tx=0).stop 2024-09-13T18:44:41.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f820e8b7700 1 -- 172.21.15.146:0/1648757721 shutdown_connections 2024-09-13T18:44:41.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f820e8b7700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f81ec066730 0x7f81ec068bf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:41.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f820e8b7700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82080cf550 0x7f82081efad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:41.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f820e8b7700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82080cf170 0x7f82080b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:41.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f820e8b7700 1 --2- 172.21.15.146:0/1648757721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82080cb5b0 0x7f8208149b70 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:41.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f820e8b7700 1 -- 172.21.15.146:0/1648757721 >> 172.21.15.146:0/1648757721 conn(0x7f8208072220 msgr2=0x7f82080b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:41.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f820e8b7700 1 -- 172.21.15.146:0/1648757721 shutdown_connections 2024-09-13T18:44:41.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:41.391+0000 7f820e8b7700 1 -- 172.21.15.146:0/1648757721 wait complete. 2024-09-13T18:44:41.417 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 74 2024-09-13T18:44:41.428 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:43.429 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:43.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.527+0000 7fa85eb24700 1 Processor -- start 2024-09-13T18:44:43.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.527+0000 7fa85eb24700 1 -- start start 2024-09-13T18:44:43.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85eb24700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa8580c63a0 0x7fa858159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:43.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85eb24700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa8580c7310 0x7fa858159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:43.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85eb24700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8580c8c70 con 0x7fa8580c5fc0 2024-09-13T18:44:43.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85eb24700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8580c9bd0 con 0x7fa8580c7310 2024-09-13T18:44:43.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85eb24700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa85815a0e0 con 0x7fa8580c63a0 2024-09-13T18:44:43.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85e323700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa8580c7310 0x7fa858159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:43.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85e323700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa8580c7310 0x7fa858159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48976/0 (socket says 172.21.15.146:48976) 2024-09-13T18:44:43.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85e323700 1 -- 172.21.15.146:0/3539466531 learned_addr learned my addr 172.21.15.146:0/3539466531 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:43.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85d321700 1 --2- 172.21.15.146:0/3539466531 >> v2:172.21.15.154:3300/0 conn(0x7fa8580c63a0 0x7fa858159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:43.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85e323700 1 -- 172.21.15.146:0/3539466531 >> v2:172.21.15.154:3300/0 conn(0x7fa8580c63a0 msgr2=0x7fa858159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:43.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85e323700 1 --2- 172.21.15.146:0/3539466531 >> v2:172.21.15.154:3300/0 conn(0x7fa8580c63a0 0x7fa858159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:43.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85e323700 1 -- 172.21.15.146:0/3539466531 >> v1:172.21.15.154:6789/0 conn(0x7fa8580c5fc0 legacy=0x7fa8580c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:43.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85e323700 1 -- 172.21.15.146:0/3539466531 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8580c8f50 con 0x7fa8580c7310 2024-09-13T18:44:43.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85e323700 1 --2- 172.21.15.146:0/3539466531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8580c7310 0x7fa858159ba0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa8580673c0 tx=0x7fa85000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df835fd45606e6af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:43.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85cb20700 1 -- 172.21.15.146:0/3539466531 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850010070 con 0x7fa8580c7310 2024-09-13T18:44:43.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85cb20700 1 -- 172.21.15.146:0/3539466531 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85000dd90 con 0x7fa8580c7310 2024-09-13T18:44:43.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85cb20700 1 -- 172.21.15.146:0/3539466531 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85000b310 con 0x7fa8580c7310 2024-09-13T18:44:43.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85eb24700 1 -- 172.21.15.146:0/3539466531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8580c7310 msgr2=0x7fa858159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:43.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85eb24700 1 --2- 172.21.15.146:0/3539466531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8580c7310 0x7fa858159ba0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa8580673c0 tx=0x7fa85000d430 comp rx=0 tx=0).stop 2024-09-13T18:44:43.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85eb24700 1 -- 172.21.15.146:0/3539466531 shutdown_connections 2024-09-13T18:44:43.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85eb24700 1 --2- 172.21.15.146:0/3539466531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8580c7310 0x7fa858159ba0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:43.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85eb24700 1 --2- 172.21.15.146:0/3539466531 >> v2:172.21.15.154:3300/0 conn(0x7fa8580c63a0 0x7fa858159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:43.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85eb24700 1 -- 172.21.15.146:0/3539466531 >> 172.21.15.146:0/3539466531 conn(0x7fa858066090 msgr2=0x7fa858066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:43.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85eb24700 1 -- 172.21.15.146:0/3539466531 shutdown_connections 2024-09-13T18:44:43.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.531+0000 7fa85eb24700 1 -- 172.21.15.146:0/3539466531 wait complete. 2024-09-13T18:44:43.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85eb24700 1 Processor -- start 2024-09-13T18:44:43.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85eb24700 1 -- start start 2024-09-13T18:44:43.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85eb24700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8580c5fc0 0x7fa8581551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:43.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85eb24700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8580c63a0 0x7fa8580ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:43.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85eb24700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8580c7310 0x7fa8580c14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:43.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85eb24700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8581ee3e0 con 0x7fa8580c63a0 2024-09-13T18:44:43.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85eb24700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8581eb350 con 0x7fa8580c5fc0 2024-09-13T18:44:43.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85eb24700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580ad190 con 0x7fa8580c7310 2024-09-13T18:44:43.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85d321700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8580c63a0 0x7fa8580ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:43.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85d321700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8580c63a0 0x7fa8580ac4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48984/0 (socket says 172.21.15.146:48984) 2024-09-13T18:44:43.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85d321700 1 -- 172.21.15.146:0/618422888 learned_addr learned my addr 172.21.15.146:0/618422888 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:43.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85e323700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8580c7310 0x7fa8580c14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:43.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85db22700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8580c5fc0 0x7fa8581551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:43.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85d321700 1 -- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8580c7310 msgr2=0x7fa8580c14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:43.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85d321700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8580c7310 0x7fa8580c14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:43.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85d321700 1 -- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8580c5fc0 msgr2=0x7fa8581551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:43.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85d321700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8580c5fc0 0x7fa8581551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:43.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85d321700 1 -- 172.21.15.146:0/618422888 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85000bd40 con 0x7fa8580c63a0 2024-09-13T18:44:43.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85db22700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8580c5fc0 0x7fa8581551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:43.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85d321700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8580c63a0 0x7fa8580ac4a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00e760 tx=0x7fa84c00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:43.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa84a7fc700 1 -- 172.21.15.146:0/618422888 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c004220 con 0x7fa8580c63a0 2024-09-13T18:44:43.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa84a7fc700 1 -- 172.21.15.146:0/618422888 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84c004380 con 0x7fa8580c63a0 2024-09-13T18:44:43.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85eb24700 1 -- 172.21.15.146:0/618422888 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa858155710 con 0x7fa8580c63a0 2024-09-13T18:44:43.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa84a7fc700 1 -- 172.21.15.146:0/618422888 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c0036c0 con 0x7fa8580c63a0 2024-09-13T18:44:43.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.535+0000 7fa85eb24700 1 -- 172.21.15.146:0/618422888 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8581ee100 con 0x7fa8580c63a0 2024-09-13T18:44:43.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.539+0000 7fa85eb24700 1 -- 172.21.15.146:0/618422888 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8580cb790 con 0x7fa8580c63a0 2024-09-13T18:44:43.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.539+0000 7fa84a7fc700 1 -- 172.21.15.146:0/618422888 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa84c003820 con 0x7fa8580c63a0 2024-09-13T18:44:43.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.539+0000 7fa84a7fc700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa83c066730 0x7fa83c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:43.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.539+0000 7fa84a7fc700 1 -- 172.21.15.146:0/618422888 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fa84c0840d0 con 0x7fa8580c63a0 2024-09-13T18:44:43.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.539+0000 7fa85db22700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa83c066730 0x7fa83c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:43.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.543+0000 7fa85db22700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa83c066730 0x7fa83c068bf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa8540017a0 tx=0x7fa854009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:43.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.551+0000 7fa84a7fc700 1 -- 172.21.15.146:0/618422888 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa84c050d30 con 0x7fa8580c63a0 2024-09-13T18:44:43.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa85eb24700 1 -- 172.21.15.146:0/618422888 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8580c1f40 con 0x7fa8580c63a0 2024-09-13T18:44:43.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa84a7fc700 1 -- 172.21.15.146:0/618422888 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5519 (secure 0 0 0) 0x7fa84c054350 con 0x7fa8580c63a0 2024-09-13T18:44:43.785 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:43.785 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:41.684542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:resolve","state_seq":7,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:43.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa85eb24700 1 -- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa83c066730 msgr2=0x7fa83c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:43.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa85eb24700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa83c066730 0x7fa83c068bf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa8540017a0 tx=0x7fa854009040 comp rx=0 tx=0).stop 2024-09-13T18:44:43.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa85eb24700 1 -- 172.21.15.146:0/618422888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8580c63a0 msgr2=0x7fa8580ac4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:43.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa85eb24700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8580c63a0 0x7fa8580ac4a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00e760 tx=0x7fa84c00ea70 comp rx=0 tx=0).stop 2024-09-13T18:44:43.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa85eb24700 1 -- 172.21.15.146:0/618422888 shutdown_connections 2024-09-13T18:44:43.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa85eb24700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa83c066730 0x7fa83c068bf0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:43.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa85eb24700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8580c7310 0x7fa8580c14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:43.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa85eb24700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8580c63a0 0x7fa8580ac4a0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:43.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa85eb24700 1 --2- 172.21.15.146:0/618422888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8580c5fc0 0x7fa8581551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:43.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa85eb24700 1 -- 172.21.15.146:0/618422888 >> 172.21.15.146:0/618422888 conn(0x7fa858066090 msgr2=0x7fa858157c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:43.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa85eb24700 1 -- 172.21.15.146:0/618422888 shutdown_connections 2024-09-13T18:44:43.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:43.763+0000 7fa85eb24700 1 -- 172.21.15.146:0/618422888 wait complete. 2024-09-13T18:44:43.789 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 75 2024-09-13T18:44:43.800 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:45.801 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:45.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.899+0000 7f2800f02700 1 Processor -- start 2024-09-13T18:44:45.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f2800f02700 1 -- start start 2024-09-13T18:44:45.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f2800f02700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27fc0c83e0 0x7f27fc0c7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:45.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f2800f02700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f27fc0c87c0 0x7f27fc0c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:45.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f2800f02700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0c6b40 0x7f27fc0c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:45.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f2800f02700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27fc0c1880 con 0x7f27fc0c87c0 2024-09-13T18:44:45.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f2800f02700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27fc0716e0 con 0x7f27fc0c83e0 2024-09-13T18:44:45.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f2800f02700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc072260 con 0x7f27fc0c6b40 2024-09-13T18:44:45.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f27f9d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f27fc0c87c0 0x7f27fc0c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:45.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f27f9d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f27fc0c87c0 0x7f27fc0c26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49000/0 (socket says 172.21.15.146:49000) 2024-09-13T18:44:45.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f27f9d9b700 1 -- 172.21.15.146:0/3522346956 learned_addr learned my addr 172.21.15.146:0/3522346956 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:45.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f27fa59c700 1 --2- 172.21.15.146:0/3522346956 >> v2:172.21.15.154:3300/0 conn(0x7f27fc0c83e0 0x7f27fc0c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:45.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f27fad9d700 1 --2- 172.21.15.146:0/3522346956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0c6b40 0x7f27fc0c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:45.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f27f9d9b700 1 -- 172.21.15.146:0/3522346956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0c6b40 msgr2=0x7f27fc0c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:45.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f27f9d9b700 1 --2- 172.21.15.146:0/3522346956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0c6b40 0x7f27fc0c2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:45.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f27f9d9b700 1 -- 172.21.15.146:0/3522346956 >> v2:172.21.15.154:3300/0 conn(0x7f27fc0c83e0 msgr2=0x7f27fc0c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:45.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f27f9d9b700 1 --2- 172.21.15.146:0/3522346956 >> v2:172.21.15.154:3300/0 conn(0x7f27fc0c83e0 0x7f27fc0c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:45.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f27f9d9b700 1 -- 172.21.15.146:0/3522346956 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc0c7a90 con 0x7f27fc0c87c0 2024-09-13T18:44:45.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f27f9d9b700 1 --2- 172.21.15.146:0/3522346956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27fc0c87c0 0x7f27fc0c26f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f27fc06aef0 tx=0x7f27e4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851f9a77a7302123 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:45.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.903+0000 7f27f959a700 1 -- 172.21.15.146:0/3522346956 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e4011830 con 0x7f27fc0c87c0 2024-09-13T18:44:45.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.907+0000 7f27f959a700 1 -- 172.21.15.146:0/3522346956 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e4011990 con 0x7f27fc0c87c0 2024-09-13T18:44:45.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.907+0000 7f27f959a700 1 -- 172.21.15.146:0/3522346956 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e4011d20 con 0x7f27fc0c87c0 2024-09-13T18:44:45.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.907+0000 7f2800f02700 1 -- 172.21.15.146:0/3522346956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27fc0c87c0 msgr2=0x7f27fc0c26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:45.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.907+0000 7f2800f02700 1 --2- 172.21.15.146:0/3522346956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27fc0c87c0 0x7f27fc0c26f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f27fc06aef0 tx=0x7f27e4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:44:45.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.907+0000 7f2800f02700 1 -- 172.21.15.146:0/3522346956 shutdown_connections 2024-09-13T18:44:45.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.907+0000 7f2800f02700 1 --2- 172.21.15.146:0/3522346956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0c6b40 0x7f27fc0c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:45.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.907+0000 7f2800f02700 1 --2- 172.21.15.146:0/3522346956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27fc0c87c0 0x7f27fc0c26f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:45.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.907+0000 7f2800f02700 1 --2- 172.21.15.146:0/3522346956 >> v2:172.21.15.154:3300/0 conn(0x7f27fc0c83e0 0x7f27fc0c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:45.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.907+0000 7f2800f02700 1 -- 172.21.15.146:0/3522346956 >> 172.21.15.146:0/3522346956 conn(0x7f27fc0cc140 msgr2=0x7f27fc0cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:45.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.907+0000 7f2800f02700 1 -- 172.21.15.146:0/3522346956 shutdown_connections 2024-09-13T18:44:45.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.907+0000 7f2800f02700 1 -- 172.21.15.146:0/3522346956 wait complete. 2024-09-13T18:44:45.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.907+0000 7f2800f02700 1 Processor -- start 2024-09-13T18:44:45.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.907+0000 7f2800f02700 1 -- start start 2024-09-13T18:44:45.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f2800f02700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27fc0c6b40 0x7f27fc0adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:45.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f2800f02700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0c83e0 0x7f27fc1ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:45.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f2800f02700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27fc0c87c0 0x7f27fc1ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:45.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f2800f02700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc14ad20 con 0x7f27fc0c87c0 2024-09-13T18:44:45.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f2800f02700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc1f34c0 con 0x7f27fc0c6b40 2024-09-13T18:44:45.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f2800f02700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc0727d0 con 0x7f27fc0c83e0 2024-09-13T18:44:45.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27fa59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27fc0c6b40 0x7f27fc0adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:45.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27fad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27fc0c87c0 0x7f27fc1ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:45.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27fa59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27fc0c6b40 0x7f27fc0adb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:40324/0 (socket says 172.21.15.146:40324) 2024-09-13T18:44:45.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27fa59c700 1 -- 172.21.15.146:0/2704575233 learned_addr learned my addr 172.21.15.146:0/2704575233 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:45.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27f9d9b700 1 --2- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0c83e0 0x7f27fc1ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:45.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27fa59c700 1 -- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0c83e0 msgr2=0x7f27fc1ef380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:45.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27fa59c700 1 --2- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0c83e0 0x7f27fc1ef380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:45.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27fa59c700 1 -- 172.21.15.146:0/2704575233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27fc0c87c0 msgr2=0x7f27fc1ef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:45.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27fa59c700 1 --2- 172.21.15.146:0/2704575233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27fc0c87c0 0x7f27fc1ef8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:45.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27fa59c700 1 -- 172.21.15.146:0/2704575233 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e4009570 con 0x7f27fc0c6b40 2024-09-13T18:44:45.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27fa59c700 1 --2- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27fc0c6b40 0x7f27fc0adb10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f27f400efc0 tx=0x7f27f400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:45.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27f2ffd700 1 -- 172.21.15.146:0/2704575233 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f400cd10 con 0x7f27fc0c6b40 2024-09-13T18:44:45.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27f2ffd700 1 -- 172.21.15.146:0/2704575233 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f400f040 con 0x7f27fc0c6b40 2024-09-13T18:44:45.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f2800f02700 1 -- 172.21.15.146:0/2704575233 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27fc1f28b0 con 0x7f27fc0c6b40 2024-09-13T18:44:45.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f27f2ffd700 1 -- 172.21.15.146:0/2704575233 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f40146a0 con 0x7f27fc0c6b40 2024-09-13T18:44:45.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.911+0000 7f2800f02700 1 -- 172.21.15.146:0/2704575233 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27fc1f2e70 con 0x7f27fc0c6b40 2024-09-13T18:44:45.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.915+0000 7f2800f02700 1 -- 172.21.15.146:0/2704575233 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27fc156dd0 con 0x7f27fc0c6b40 2024-09-13T18:44:45.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.915+0000 7f27f2ffd700 1 -- 172.21.15.146:0/2704575233 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f27f4014800 con 0x7f27fc0c6b40 2024-09-13T18:44:45.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.915+0000 7f27f2ffd700 1 --2- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f27e0066730 0x7f27e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:45.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.915+0000 7f27f2ffd700 1 -- 172.21.15.146:0/2704575233 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f27f4085360 con 0x7f27fc0c6b40 2024-09-13T18:44:45.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.915+0000 7f27f9d9b700 1 --2- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f27e0066730 0x7f27e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:45.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.915+0000 7f27f9d9b700 1 --2- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f27e0066730 0x7f27e0068bf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f27e4000c00 tx=0x7f27e4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:45.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:45.923+0000 7f27f2ffd700 1 -- 172.21.15.146:0/2704575233 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f27f4051f30 con 0x7f27fc0c6b40 2024-09-13T18:44:46.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.131+0000 7f2800f02700 1 -- 172.21.15.146:0/2704575233 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27fc0b1cf0 con 0x7f27fc0c6b40 2024-09-13T18:44:46.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.131+0000 7f27f2ffd700 1 -- 172.21.15.146:0/2704575233 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5519 (secure 0 0 0) 0x7f27f4055550 con 0x7f27fc0c6b40 2024-09-13T18:44:46.155 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:46.155 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:41.684542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:resolve","state_seq":7,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:46.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.135+0000 7f2800f02700 1 -- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f27e0066730 msgr2=0x7f27e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:46.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.135+0000 7f2800f02700 1 --2- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f27e0066730 0x7f27e0068bf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f27e4000c00 tx=0x7f27e4008030 comp rx=0 tx=0).stop 2024-09-13T18:44:46.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.135+0000 7f2800f02700 1 -- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27fc0c6b40 msgr2=0x7f27fc0adb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:46.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.135+0000 7f2800f02700 1 --2- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27fc0c6b40 0x7f27fc0adb10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f27f400efc0 tx=0x7f27f400c3a0 comp rx=0 tx=0).stop 2024-09-13T18:44:46.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.135+0000 7f2800f02700 1 -- 172.21.15.146:0/2704575233 shutdown_connections 2024-09-13T18:44:46.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.135+0000 7f2800f02700 1 --2- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f27e0066730 0x7f27e0068bf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:46.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.135+0000 7f2800f02700 1 --2- 172.21.15.146:0/2704575233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27fc0c87c0 0x7f27fc1ef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:46.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.135+0000 7f2800f02700 1 --2- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0c83e0 0x7f27fc1ef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:46.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.135+0000 7f2800f02700 1 --2- 172.21.15.146:0/2704575233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27fc0c6b40 0x7f27fc0adb10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:46.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.135+0000 7f2800f02700 1 -- 172.21.15.146:0/2704575233 >> 172.21.15.146:0/2704575233 conn(0x7f27fc0cc140 msgr2=0x7f27fc06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:46.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.135+0000 7f2800f02700 1 -- 172.21.15.146:0/2704575233 shutdown_connections 2024-09-13T18:44:46.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:46.135+0000 7f2800f02700 1 -- 172.21.15.146:0/2704575233 wait complete. 2024-09-13T18:44:46.159 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 75 2024-09-13T18:44:46.170 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:48.172 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:48.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.271+0000 7fd59b3b1700 1 Processor -- start 2024-09-13T18:44:48.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.271+0000 7fd59b3b1700 1 -- start start 2024-09-13T18:44:48.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59b3b1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd58c0d5640 0x7fd58c005ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:48.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59b3b1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd58c045080 0x7fd58c0462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:48.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59b3b1700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd58c035870 con 0x7fd58c0467e0 2024-09-13T18:44:48.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59b3b1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58c033710 con 0x7fd58c0d5640 2024-09-13T18:44:48.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59b3b1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c0db2b0 con 0x7fd58c045080 2024-09-13T18:44:48.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59a3af700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd58c0d5640 0x7fd58c005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:48.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59abb0700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd58c0467e0 0x7fd58c0db700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:53598/0 (socket says 172.21.15.146:53598) 2024-09-13T18:44:48.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd599bae700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd58c045080 0x7fd58c0462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:48.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59abb0700 1 -- 172.21.15.146:0/1546722074 learned_addr learned my addr 172.21.15.146:0/1546722074 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:48.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd5993ad700 1 -- 172.21.15.146:0/1546722074 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023608777 0 0) 0x7fd58c035870 con 0x7fd58c0467e0 2024-09-13T18:44:48.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd5993ad700 1 -- 172.21.15.146:0/1546722074 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5800035d0 con 0x7fd58c0467e0 2024-09-13T18:44:48.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59a3af700 1 -- 172.21.15.146:0/1546722074 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd58c045080 msgr2=0x7fd58c0462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:48.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59a3af700 1 --2- 172.21.15.146:0/1546722074 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd58c045080 0x7fd58c0462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:48.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59a3af700 1 -- 172.21.15.146:0/1546722074 >> v1:172.21.15.154:6789/0 conn(0x7fd58c0467e0 legacy=0x7fd58c0db700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:48.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59a3af700 1 -- 172.21.15.146:0/1546722074 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c006160 con 0x7fd58c0d5640 2024-09-13T18:44:48.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd599bae700 1 --2- 172.21.15.146:0/1546722074 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd58c045080 0x7fd58c0462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:44:48.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59a3af700 1 --2- 172.21.15.146:0/1546722074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd58c0d5640 0x7fd58c005ab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd590009850 tx=0x7fd590009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0057936ae9fb597 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:48.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd5993ad700 1 -- 172.21.15.146:0/1546722074 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590011830 con 0x7fd58c0d5640 2024-09-13T18:44:48.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd5993ad700 1 -- 172.21.15.146:0/1546722074 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd590011990 con 0x7fd58c0d5640 2024-09-13T18:44:48.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd5993ad700 1 -- 172.21.15.146:0/1546722074 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590011d40 con 0x7fd58c0d5640 2024-09-13T18:44:48.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59b3b1700 1 -- 172.21.15.146:0/1546722074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd58c0d5640 msgr2=0x7fd58c005ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:48.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59b3b1700 1 --2- 172.21.15.146:0/1546722074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd58c0d5640 0x7fd58c005ab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd590009850 tx=0x7fd590009b60 comp rx=0 tx=0).stop 2024-09-13T18:44:48.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59b3b1700 1 -- 172.21.15.146:0/1546722074 shutdown_connections 2024-09-13T18:44:48.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59b3b1700 1 --2- 172.21.15.146:0/1546722074 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd58c045080 0x7fd58c0462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:48.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59b3b1700 1 --2- 172.21.15.146:0/1546722074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd58c0d5640 0x7fd58c005ab0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:48.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59b3b1700 1 -- 172.21.15.146:0/1546722074 >> 172.21.15.146:0/1546722074 conn(0x7fd58c04fd00 msgr2=0x7fd58c050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:48.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59b3b1700 1 -- 172.21.15.146:0/1546722074 shutdown_connections 2024-09-13T18:44:48.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.275+0000 7fd59b3b1700 1 -- 172.21.15.146:0/1546722074 wait complete. 2024-09-13T18:44:48.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd59b3b1700 1 Processor -- start 2024-09-13T18:44:48.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd59b3b1700 1 -- start start 2024-09-13T18:44:48.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd59b3b1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd58c045080 0x7fd58c04f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:48.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd59b3b1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd58c0467e0 0x7fd58c0d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:48.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd59b3b1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd58c0d5640 0x7fd58c0d1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:48.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd59b3b1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c0d0670 con 0x7fd58c0467e0 2024-09-13T18:44:48.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd59b3b1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c0d0c30 con 0x7fd58c045080 2024-09-13T18:44:48.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd59b3b1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c049110 con 0x7fd58c0d5640 2024-09-13T18:44:48.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd599bae700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd58c0467e0 0x7fd58c0d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:48.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd599bae700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd58c0467e0 0x7fd58c0d10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49016/0 (socket says 172.21.15.146:49016) 2024-09-13T18:44:48.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd599bae700 1 -- 172.21.15.146:0/2885930903 learned_addr learned my addr 172.21.15.146:0/2885930903 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:48.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd59a3af700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd58c045080 0x7fd58c04f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:48.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd59abb0700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd58c0d5640 0x7fd58c0d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:48.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd599bae700 1 -- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd58c0d5640 msgr2=0x7fd58c0d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:48.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd599bae700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd58c0d5640 0x7fd58c0d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:48.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd599bae700 1 -- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd58c045080 msgr2=0x7fd58c04f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:48.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd599bae700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd58c045080 0x7fd58c04f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:48.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd599bae700 1 -- 172.21.15.146:0/2885930903 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5880094b0 con 0x7fd58c0467e0 2024-09-13T18:44:48.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd599bae700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd58c0467e0 0x7fd58c0d10c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd58800ec10 tx=0x7fd58800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:48.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd586ffd700 1 -- 172.21.15.146:0/2885930903 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58800c890 con 0x7fd58c0467e0 2024-09-13T18:44:48.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd59b3b1700 1 -- 172.21.15.146:0/2885930903 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5900094b0 con 0x7fd58c0467e0 2024-09-13T18:44:48.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd586ffd700 1 -- 172.21.15.146:0/2885930903 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58800c9f0 con 0x7fd58c0467e0 2024-09-13T18:44:48.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.279+0000 7fd59b3b1700 1 -- 172.21.15.146:0/2885930903 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c049bf0 con 0x7fd58c0467e0 2024-09-13T18:44:48.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.283+0000 7fd586ffd700 1 -- 172.21.15.146:0/2885930903 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58800cd60 con 0x7fd58c0467e0 2024-09-13T18:44:48.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.283+0000 7fd59b3b1700 1 -- 172.21.15.146:0/2885930903 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58c032a50 con 0x7fd58c0467e0 2024-09-13T18:44:48.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.283+0000 7fd586ffd700 1 -- 172.21.15.146:0/2885930903 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd588004020 con 0x7fd58c0467e0 2024-09-13T18:44:48.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.283+0000 7fd586ffd700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd578066760 0x7fd578068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:48.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.283+0000 7fd586ffd700 1 -- 172.21.15.146:0/2885930903 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fd5880846d0 con 0x7fd58c0467e0 2024-09-13T18:44:48.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.283+0000 7fd59a3af700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd578066760 0x7fd578068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:48.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.287+0000 7fd59a3af700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd578066760 0x7fd578068c20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd5900090b0 tx=0x7fd59001c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:48.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.291+0000 7fd586ffd700 1 -- 172.21.15.146:0/2885930903 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd588051200 con 0x7fd58c0467e0 2024-09-13T18:44:48.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.503+0000 7fd59b3b1700 1 -- 172.21.15.146:0/2885930903 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd58c050820 con 0x7fd58c0467e0 2024-09-13T18:44:48.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.503+0000 7fd586ffd700 1 -- 172.21.15.146:0/2885930903 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+5521 (secure 0 0 0) 0x7fd588054820 con 0x7fd58c0467e0 2024-09-13T18:44:48.527 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:48.527 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:46.752966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:reconnect","state_seq":9,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:48.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.507+0000 7fd59b3b1700 1 -- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd578066760 msgr2=0x7fd578068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:48.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.507+0000 7fd59b3b1700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd578066760 0x7fd578068c20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd5900090b0 tx=0x7fd59001c040 comp rx=0 tx=0).stop 2024-09-13T18:44:48.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.507+0000 7fd59b3b1700 1 -- 172.21.15.146:0/2885930903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd58c0467e0 msgr2=0x7fd58c0d10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:48.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.507+0000 7fd59b3b1700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd58c0467e0 0x7fd58c0d10c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd58800ec10 tx=0x7fd58800ef20 comp rx=0 tx=0).stop 2024-09-13T18:44:48.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.507+0000 7fd59b3b1700 1 -- 172.21.15.146:0/2885930903 shutdown_connections 2024-09-13T18:44:48.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.507+0000 7fd59b3b1700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd578066760 0x7fd578068c20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:48.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.507+0000 7fd59b3b1700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd58c0d5640 0x7fd58c0d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:48.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.507+0000 7fd59b3b1700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd58c0467e0 0x7fd58c0d10c0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:48.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.507+0000 7fd59b3b1700 1 --2- 172.21.15.146:0/2885930903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd58c045080 0x7fd58c04f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:48.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.507+0000 7fd59b3b1700 1 -- 172.21.15.146:0/2885930903 >> 172.21.15.146:0/2885930903 conn(0x7fd58c04fd00 msgr2=0x7fd58c04a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:48.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.507+0000 7fd59b3b1700 1 -- 172.21.15.146:0/2885930903 shutdown_connections 2024-09-13T18:44:48.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:48.507+0000 7fd59b3b1700 1 -- 172.21.15.146:0/2885930903 wait complete. 2024-09-13T18:44:48.531 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 76 2024-09-13T18:44:48.543 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 76, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 76, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6207": { "addr": "172.21.15.146:6837/3234109877", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3234109877, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3234109877, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6207, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 39 }, "gid_6459": { "addr": "172.21.15.146:6835/294014284", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 294014284, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 294014284, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6459, "incarnation": 74, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 37, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:44:46.752966+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6459, "mds_1": 6207 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.146:6833/3023779168", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3023779168, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3023779168, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 69, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6187, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:44:50.544 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:50.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.643+0000 7fc565569700 1 Processor -- start 2024-09-13T18:44:50.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.643+0000 7fc565569700 1 -- start start 2024-09-13T18:44:50.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.643+0000 7fc565569700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5600cb2b0 0x7fc5600723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:50.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.643+0000 7fc565569700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc5600a9840 0x7fc56015e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:50.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.643+0000 7fc565569700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc560161550 con 0x7fc5600caed0 2024-09-13T18:44:50.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.643+0000 7fc565569700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc560072a30 con 0x7fc5600a9840 2024-09-13T18:44:50.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.643+0000 7fc565569700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5600aa800 con 0x7fc5600cb2b0 2024-09-13T18:44:50.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.643+0000 7fc564d68700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc5600a9840 0x7fc56015e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:50.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.643+0000 7fc55ffff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc5600caed0 0x7fc5601619e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:53610/0 (socket says 172.21.15.146:53610) 2024-09-13T18:44:50.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.643+0000 7fc55ffff700 1 -- 172.21.15.146:0/162427675 learned_addr learned my addr 172.21.15.146:0/162427675 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:50.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.643+0000 7fc55f7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5600cb2b0 0x7fc5600723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:50.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc55f7fe700 1 -- 172.21.15.146:0/162427675 >> v2:172.21.15.154:3300/0 conn(0x7fc5600a9840 msgr2=0x7fc56015e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:50.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc55f7fe700 1 --2- 172.21.15.146:0/162427675 >> v2:172.21.15.154:3300/0 conn(0x7fc5600a9840 0x7fc56015e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:50.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc55f7fe700 1 -- 172.21.15.146:0/162427675 >> v1:172.21.15.154:6789/0 conn(0x7fc5600caed0 legacy=0x7fc5601619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:50.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc55f7fe700 1 -- 172.21.15.146:0/162427675 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc560161220 con 0x7fc5600cb2b0 2024-09-13T18:44:50.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc564d68700 1 --2- 172.21.15.146:0/162427675 >> v2:172.21.15.154:3300/0 conn(0x7fc5600a9840 0x7fc56015e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:44:50.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc55f7fe700 1 --2- 172.21.15.146:0/162427675 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5600cb2b0 0x7fc5600723f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc548009d90 tx=0x7fc54800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3976d009e8cdff22 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:50.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc55effd700 1 -- 172.21.15.146:0/162427675 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54800e070 con 0x7fc5600cb2b0 2024-09-13T18:44:50.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc55effd700 1 -- 172.21.15.146:0/162427675 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54800bb20 con 0x7fc5600cb2b0 2024-09-13T18:44:50.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc55effd700 1 -- 172.21.15.146:0/162427675 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548012430 con 0x7fc5600cb2b0 2024-09-13T18:44:50.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc565569700 1 -- 172.21.15.146:0/162427675 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5600cb2b0 msgr2=0x7fc5600723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:50.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc565569700 1 --2- 172.21.15.146:0/162427675 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5600cb2b0 0x7fc5600723f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc548009d90 tx=0x7fc54800b330 comp rx=0 tx=0).stop 2024-09-13T18:44:50.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc565569700 1 -- 172.21.15.146:0/162427675 shutdown_connections 2024-09-13T18:44:50.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc565569700 1 --2- 172.21.15.146:0/162427675 >> v2:172.21.15.154:3300/0 conn(0x7fc5600a9840 0x7fc56015e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:50.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc565569700 1 --2- 172.21.15.146:0/162427675 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5600cb2b0 0x7fc5600723f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:50.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc565569700 1 -- 172.21.15.146:0/162427675 >> 172.21.15.146:0/162427675 conn(0x7fc560066090 msgr2=0x7fc560066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:50.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc565569700 1 -- 172.21.15.146:0/162427675 shutdown_connections 2024-09-13T18:44:50.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc565569700 1 -- 172.21.15.146:0/162427675 wait complete. 2024-09-13T18:44:50.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc565569700 1 Processor -- start 2024-09-13T18:44:50.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.647+0000 7fc565569700 1 -- start start 2024-09-13T18:44:50.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc565569700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5600a9840 0x7fc560124150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:50.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc565569700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5600caed0 0x7fc560124690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:50.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc565569700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5600cb2b0 0x7fc560127420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:50.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc565569700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5600ab550 con 0x7fc5600cb2b0 2024-09-13T18:44:50.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc565569700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5600ae2d0 con 0x7fc5600a9840 2024-09-13T18:44:50.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc565569700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5600aba40 con 0x7fc5600caed0 2024-09-13T18:44:50.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc564d68700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5600cb2b0 0x7fc560127420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:50.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc55ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5600a9840 0x7fc560124150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:50.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc55f7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5600caed0 0x7fc560124690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:50.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc564d68700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5600cb2b0 0x7fc560127420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49026/0 (socket says 172.21.15.146:49026) 2024-09-13T18:44:50.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc564d68700 1 -- 172.21.15.146:0/4131197051 learned_addr learned my addr 172.21.15.146:0/4131197051 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:50.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc55f7fe700 1 -- 172.21.15.146:0/4131197051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5600a9840 msgr2=0x7fc560124150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:50.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc55f7fe700 1 --2- 172.21.15.146:0/4131197051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5600a9840 0x7fc560124150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:50.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc55f7fe700 1 -- 172.21.15.146:0/4131197051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5600cb2b0 msgr2=0x7fc560127420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:50.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc55f7fe700 1 --2- 172.21.15.146:0/4131197051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5600cb2b0 0x7fc560127420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:50.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc55f7fe700 1 -- 172.21.15.146:0/4131197051 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5580094b0 con 0x7fc5600caed0 2024-09-13T18:44:50.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc564d68700 1 --2- 172.21.15.146:0/4131197051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5600cb2b0 0x7fc560127420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:44:50.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc55f7fe700 1 --2- 172.21.15.146:0/4131197051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5600caed0 0x7fc560124690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc5600c87e0 tx=0x7fc548004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:50.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc55cff9700 1 -- 172.21.15.146:0/4131197051 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548012430 con 0x7fc5600caed0 2024-09-13T18:44:50.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc55cff9700 1 -- 172.21.15.146:0/4131197051 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc548020070 con 0x7fc5600caed0 2024-09-13T18:44:50.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc565569700 1 -- 172.21.15.146:0/4131197051 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc548009a70 con 0x7fc5600caed0 2024-09-13T18:44:50.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc55cff9700 1 -- 172.21.15.146:0/4131197051 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548004280 con 0x7fc5600caed0 2024-09-13T18:44:50.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.651+0000 7fc565569700 1 -- 172.21.15.146:0/4131197051 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5600adc80 con 0x7fc5600caed0 2024-09-13T18:44:50.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.655+0000 7fc565569700 1 -- 172.21.15.146:0/4131197051 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5600727b0 con 0x7fc5600caed0 2024-09-13T18:44:50.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.655+0000 7fc55cff9700 1 -- 172.21.15.146:0/4131197051 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc5480043e0 con 0x7fc5600caed0 2024-09-13T18:44:50.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.655+0000 7fc55cff9700 1 --2- 172.21.15.146:0/4131197051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc544066730 0x7fc544068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:50.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.655+0000 7fc55cff9700 1 -- 172.21.15.146:0/4131197051 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fc548022070 con 0x7fc5600caed0 2024-09-13T18:44:50.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.655+0000 7fc55ffff700 1 --2- 172.21.15.146:0/4131197051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc544066730 0x7fc544068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:50.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.655+0000 7fc55ffff700 1 --2- 172.21.15.146:0/4131197051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc544066730 0x7fc544068bf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc550005930 tx=0x7fc550009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:50.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.663+0000 7fc55cff9700 1 -- 172.21.15.146:0/4131197051 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc548053a10 con 0x7fc5600caed0 2024-09-13T18:44:50.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.879+0000 7fc565569700 1 -- 172.21.15.146:0/4131197051 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc56006f5c0 con 0x7fc5600caed0 2024-09-13T18:44:50.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.879+0000 7fc55cff9700 1 -- 172.21.15.146:0/4131197051 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+5521 (secure 0 0 0) 0x7fc548057030 con 0x7fc5600caed0 2024-09-13T18:44:50.902 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:50.902 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:46.752966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:reconnect","state_seq":9,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:50.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.883+0000 7fc565569700 1 -- 172.21.15.146:0/4131197051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc544066730 msgr2=0x7fc544068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:50.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.883+0000 7fc565569700 1 --2- 172.21.15.146:0/4131197051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc544066730 0x7fc544068bf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc550005930 tx=0x7fc550009040 comp rx=0 tx=0).stop 2024-09-13T18:44:50.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.883+0000 7fc565569700 1 -- 172.21.15.146:0/4131197051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5600caed0 msgr2=0x7fc560124690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:50.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.883+0000 7fc565569700 1 --2- 172.21.15.146:0/4131197051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5600caed0 0x7fc560124690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc5600c87e0 tx=0x7fc548004af0 comp rx=0 tx=0).stop 2024-09-13T18:44:50.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.883+0000 7fc565569700 1 -- 172.21.15.146:0/4131197051 shutdown_connections 2024-09-13T18:44:50.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.883+0000 7fc565569700 1 --2- 172.21.15.146:0/4131197051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc544066730 0x7fc544068bf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:50.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.883+0000 7fc565569700 1 --2- 172.21.15.146:0/4131197051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5600cb2b0 0x7fc560127420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:50.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.883+0000 7fc565569700 1 --2- 172.21.15.146:0/4131197051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5600caed0 0x7fc560124690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:50.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.883+0000 7fc565569700 1 --2- 172.21.15.146:0/4131197051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5600a9840 0x7fc560124150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:50.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.883+0000 7fc565569700 1 -- 172.21.15.146:0/4131197051 >> 172.21.15.146:0/4131197051 conn(0x7fc560066090 msgr2=0x7fc56006fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:50.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.883+0000 7fc565569700 1 -- 172.21.15.146:0/4131197051 shutdown_connections 2024-09-13T18:44:50.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:50.883+0000 7fc565569700 1 -- 172.21.15.146:0/4131197051 wait complete. 2024-09-13T18:44:50.906 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 76 2024-09-13T18:44:50.917 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:52.918 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:53.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.019+0000 7f30b359e700 1 Processor -- start 2024-09-13T18:44:53.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.019+0000 7f30b359e700 1 -- start start 2024-09-13T18:44:53.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.019+0000 7f30b359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30b4153bc0 0x7f30b40c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:53.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.019+0000 7f30b359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f30b40c6610 0x7f30b40c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:53.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.019+0000 7f30b359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f30b40cac60 0x7f30b40cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:53.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.019+0000 7f30b359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b4153fa0 con 0x7f30b40cac60 2024-09-13T18:44:53.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.019+0000 7f30b359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b40aa500 con 0x7f30b40c6610 2024-09-13T18:44:53.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.019+0000 7f30b359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b415a7e0 con 0x7f30b4153bc0 2024-09-13T18:44:53.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.019+0000 7f30b2d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f30b40cac60 0x7f30b40cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:53.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.019+0000 7f30b2d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f30b40cac60 0x7f30b40cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45830/0 (socket says 172.21.15.146:45830) 2024-09-13T18:44:53.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.019+0000 7f30b2d9d700 1 -- 172.21.15.146:0/1956505152 learned_addr learned my addr 172.21.15.146:0/1956505152 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:53.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.019+0000 7f30a9d9b700 1 --2- 172.21.15.146:0/1956505152 >> v2:172.21.15.154:3300/0 conn(0x7f30b40c6610 0x7f30b40c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:53.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.019+0000 7f30b259c700 1 --2- 172.21.15.146:0/1956505152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30b4153bc0 0x7f30b40c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:53.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b2d9d700 1 -- 172.21.15.146:0/1956505152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30b4153bc0 msgr2=0x7f30b40c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:53.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b2d9d700 1 --2- 172.21.15.146:0/1956505152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30b4153bc0 0x7f30b40c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:53.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b2d9d700 1 -- 172.21.15.146:0/1956505152 >> v2:172.21.15.154:3300/0 conn(0x7f30b40c6610 msgr2=0x7f30b40c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:53.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b2d9d700 1 --2- 172.21.15.146:0/1956505152 >> v2:172.21.15.154:3300/0 conn(0x7f30b40c6610 0x7f30b40c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:53.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b2d9d700 1 -- 172.21.15.146:0/1956505152 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b40c9320 con 0x7f30b40cac60 2024-09-13T18:44:53.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30a9d9b700 1 --2- 172.21.15.146:0/1956505152 >> v2:172.21.15.154:3300/0 conn(0x7f30b40c6610 0x7f30b40c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:53.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b2d9d700 1 --2- 172.21.15.146:0/1956505152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30b40cac60 0x7f30b40cb0d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f30b40cb9e0 tx=0x7f30ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2531dc214605cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:53.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b1d9b700 1 -- 172.21.15.146:0/1956505152 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac00e070 con 0x7f30b40cac60 2024-09-13T18:44:53.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b1d9b700 1 -- 172.21.15.146:0/1956505152 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac00b910 con 0x7f30b40cac60 2024-09-13T18:44:53.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b1d9b700 1 -- 172.21.15.146:0/1956505152 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac00bcc0 con 0x7f30b40cac60 2024-09-13T18:44:53.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b359e700 1 -- 172.21.15.146:0/1956505152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30b40cac60 msgr2=0x7f30b40cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:53.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b359e700 1 --2- 172.21.15.146:0/1956505152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30b40cac60 0x7f30b40cb0d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f30b40cb9e0 tx=0x7f30ac00b330 comp rx=0 tx=0).stop 2024-09-13T18:44:53.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b359e700 1 -- 172.21.15.146:0/1956505152 shutdown_connections 2024-09-13T18:44:53.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b359e700 1 --2- 172.21.15.146:0/1956505152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30b40cac60 0x7f30b40cb0d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:53.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b359e700 1 --2- 172.21.15.146:0/1956505152 >> v2:172.21.15.154:3300/0 conn(0x7f30b40c6610 0x7f30b40c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:53.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b359e700 1 --2- 172.21.15.146:0/1956505152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30b4153bc0 0x7f30b40c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:53.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b359e700 1 -- 172.21.15.146:0/1956505152 >> 172.21.15.146:0/1956505152 conn(0x7f30b4066090 msgr2=0x7f30b4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:53.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b359e700 1 -- 172.21.15.146:0/1956505152 shutdown_connections 2024-09-13T18:44:53.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b359e700 1 -- 172.21.15.146:0/1956505152 wait complete. 2024-09-13T18:44:53.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b359e700 1 Processor -- start 2024-09-13T18:44:53.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.023+0000 7f30b359e700 1 -- start start 2024-09-13T18:44:53.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30b40c6610 0x7f30b41f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:53.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30b40cac60 0x7f30b41eee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:53.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30b4153bc0 0x7f30b4158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:53.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b4159480 con 0x7f30b40c6610 2024-09-13T18:44:53.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b40ab8a0 con 0x7f30b40cac60 2024-09-13T18:44:53.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b40ad1d0 con 0x7f30b4153bc0 2024-09-13T18:44:53.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30b40c6610 0x7f30b41f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:53.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30b40c6610 0x7f30b41f2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45836/0 (socket says 172.21.15.146:45836) 2024-09-13T18:44:53.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b259c700 1 -- 172.21.15.146:0/948208344 learned_addr learned my addr 172.21.15.146:0/948208344 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:53.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30a9d9b700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30b40cac60 0x7f30b41eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:53.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b2d9d700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30b4153bc0 0x7f30b4158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:53.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b259c700 1 -- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30b4153bc0 msgr2=0x7f30b4158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:53.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b259c700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30b4153bc0 0x7f30b4158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:53.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b259c700 1 -- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30b40cac60 msgr2=0x7f30b41eee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:53.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b259c700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30b40cac60 0x7f30b41eee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:53.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b259c700 1 -- 172.21.15.146:0/948208344 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac009b30 con 0x7f30b40c6610 2024-09-13T18:44:53.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b2d9d700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30b4153bc0 0x7f30b4158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:44:53.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b259c700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30b40c6610 0x7f30b41f2e80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f30a400e840 tx=0x7f30a400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:53.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30ab7fe700 1 -- 172.21.15.146:0/948208344 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a40095a0 con 0x7f30b40c6610 2024-09-13T18:44:53.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30ab7fe700 1 -- 172.21.15.146:0/948208344 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a4013680 con 0x7f30b40c6610 2024-09-13T18:44:53.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b359e700 1 -- 172.21.15.146:0/948208344 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b40ad470 con 0x7f30b40c6610 2024-09-13T18:44:53.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30ab7fe700 1 -- 172.21.15.146:0/948208344 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a40139f0 con 0x7f30b40c6610 2024-09-13T18:44:53.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b359e700 1 -- 172.21.15.146:0/948208344 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b41f23a0 con 0x7f30b40c6610 2024-09-13T18:44:53.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.027+0000 7f30b359e700 1 -- 172.21.15.146:0/948208344 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b415a2c0 con 0x7f30b40c6610 2024-09-13T18:44:53.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.031+0000 7f30ab7fe700 1 -- 172.21.15.146:0/948208344 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f30a4013b50 con 0x7f30b40c6610 2024-09-13T18:44:53.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.031+0000 7f30ab7fe700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3094066730 0x7f3094068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:53.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.031+0000 7f30ab7fe700 1 -- 172.21.15.146:0/948208344 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f30a4084700 con 0x7f30b40c6610 2024-09-13T18:44:53.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.031+0000 7f30a9d9b700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3094066730 0x7f3094068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:53.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.031+0000 7f30a9d9b700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3094066730 0x7f3094068bf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f30b41597c0 tx=0x7f309c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:53.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.035+0000 7f30ab7fe700 1 -- 172.21.15.146:0/948208344 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f30a40512b0 con 0x7f30b40c6610 2024-09-13T18:44:53.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.243+0000 7f30b359e700 1 -- 172.21.15.146:0/948208344 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30b406f690 con 0x7f30b40c6610 2024-09-13T18:44:53.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.247+0000 7f30ab7fe700 1 -- 172.21.15.146:0/948208344 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+5519 (secure 0 0 0) 0x7f30a40548d0 con 0x7f30b40c6610 2024-09-13T18:44:53.268 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:53.268 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:51.773630+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:rejoin","state_seq":11,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:53.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.247+0000 7f30b359e700 1 -- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3094066730 msgr2=0x7f3094068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:53.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.247+0000 7f30b359e700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3094066730 0x7f3094068bf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f30b41597c0 tx=0x7f309c008040 comp rx=0 tx=0).stop 2024-09-13T18:44:53.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.247+0000 7f30b359e700 1 -- 172.21.15.146:0/948208344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30b40c6610 msgr2=0x7f30b41f2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:53.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.247+0000 7f30b359e700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30b40c6610 0x7f30b41f2e80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f30a400e840 tx=0x7f30a400eb50 comp rx=0 tx=0).stop 2024-09-13T18:44:53.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.247+0000 7f30b359e700 1 -- 172.21.15.146:0/948208344 shutdown_connections 2024-09-13T18:44:53.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.247+0000 7f30b359e700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3094066730 0x7f3094068bf0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:53.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.247+0000 7f30b359e700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30b4153bc0 0x7f30b4158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:53.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.247+0000 7f30b359e700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30b40cac60 0x7f30b41eee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:53.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.247+0000 7f30b359e700 1 --2- 172.21.15.146:0/948208344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30b40c6610 0x7f30b41f2e80 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:53.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.247+0000 7f30b359e700 1 -- 172.21.15.146:0/948208344 >> 172.21.15.146:0/948208344 conn(0x7f30b4066090 msgr2=0x7f30b40c0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:53.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.247+0000 7f30b359e700 1 -- 172.21.15.146:0/948208344 shutdown_connections 2024-09-13T18:44:53.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:53.247+0000 7f30b359e700 1 -- 172.21.15.146:0/948208344 wait complete. 2024-09-13T18:44:53.272 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 77 2024-09-13T18:44:53.284 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:55.285 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:55.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.387+0000 7fcc8ca1f700 1 Processor -- start 2024-09-13T18:44:55.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.387+0000 7fcc8ca1f700 1 -- start start 2024-09-13T18:44:55.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.387+0000 7fcc8ca1f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcc880c6610 0x7fcc880cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:55.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.387+0000 7fcc8ca1f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc880c6a80 0x7fcc880cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:55.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.387+0000 7fcc8ca1f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc88153fa0 con 0x7fcc88153bc0 2024-09-13T18:44:55.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.387+0000 7fcc8ca1f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc880aa500 con 0x7fcc880c6610 2024-09-13T18:44:55.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.387+0000 7fcc8ca1f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8815a7e0 con 0x7fcc880c6a80 2024-09-13T18:44:55.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc86d9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fcc88153bc0 0x7fcc880c8de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48064/0 (socket says 172.21.15.146:48064) 2024-09-13T18:44:55.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc86d9d700 1 -- 172.21.15.146:0/1954215626 learned_addr learned my addr 172.21.15.146:0/1954215626 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:55.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8759e700 1 --2- 172.21.15.146:0/1954215626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc880c6a80 0x7fcc880cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:55.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8659c700 1 --2- 172.21.15.146:0/1954215626 >> v2:172.21.15.154:3300/0 conn(0x7fcc880c6610 0x7fcc880cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:55.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc85d9b700 1 -- 172.21.15.146:0/1954215626 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 173505646 0 0) 0x7fcc88153fa0 con 0x7fcc88153bc0 2024-09-13T18:44:55.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc85d9b700 1 -- 172.21.15.146:0/1954215626 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc700035d0 con 0x7fcc88153bc0 2024-09-13T18:44:55.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8759e700 1 -- 172.21.15.146:0/1954215626 >> v2:172.21.15.154:3300/0 conn(0x7fcc880c6610 msgr2=0x7fcc880cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:55.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8759e700 1 --2- 172.21.15.146:0/1954215626 >> v2:172.21.15.154:3300/0 conn(0x7fcc880c6610 0x7fcc880cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:55.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8759e700 1 -- 172.21.15.146:0/1954215626 >> v1:172.21.15.146:6789/0 conn(0x7fcc88153bc0 legacy=0x7fcc880c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:55.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc85d9b700 1 -- 172.21.15.146:0/1954215626 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3286507436 0 0) 0x7fcc700035d0 con 0x7fcc88153bc0 2024-09-13T18:44:55.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8759e700 1 -- 172.21.15.146:0/1954215626 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc880aa220 con 0x7fcc880c6a80 2024-09-13T18:44:55.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8659c700 1 --2- 172.21.15.146:0/1954215626 >> v2:172.21.15.154:3300/0 conn(0x7fcc880c6610 0x7fcc880cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:44:55.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8759e700 1 --2- 172.21.15.146:0/1954215626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc880c6a80 0x7fcc880cb1a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009850 tx=0x7fcc80009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5973c6ec1a7c114 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:55.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc85d9b700 1 -- 172.21.15.146:0/1954215626 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c001670 con 0x7fcc880c6a80 2024-09-13T18:44:55.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc85d9b700 1 -- 172.21.15.146:0/1954215626 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc80004030 con 0x7fcc880c6a80 2024-09-13T18:44:55.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc85d9b700 1 -- 172.21.15.146:0/1954215626 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc800043a0 con 0x7fcc880c6a80 2024-09-13T18:44:55.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/1954215626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc880c6a80 msgr2=0x7fcc880cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:55.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8ca1f700 1 --2- 172.21.15.146:0/1954215626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc880c6a80 0x7fcc880cb1a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009850 tx=0x7fcc80009b60 comp rx=0 tx=0).stop 2024-09-13T18:44:55.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/1954215626 shutdown_connections 2024-09-13T18:44:55.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8ca1f700 1 --2- 172.21.15.146:0/1954215626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc880c6a80 0x7fcc880cb1a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:55.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8ca1f700 1 --2- 172.21.15.146:0/1954215626 >> v2:172.21.15.154:3300/0 conn(0x7fcc880c6610 0x7fcc880cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:55.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/1954215626 >> 172.21.15.146:0/1954215626 conn(0x7fcc88066090 msgr2=0x7fcc88066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:55.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/1954215626 shutdown_connections 2024-09-13T18:44:55.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/1954215626 wait complete. 2024-09-13T18:44:55.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8ca1f700 1 Processor -- start 2024-09-13T18:44:55.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.391+0000 7fcc8ca1f700 1 -- start start 2024-09-13T18:44:55.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8ca1f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc880c6610 0x7fcc880af450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:55.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8ca1f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc880c6a80 0x7fcc881f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:55.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8ca1f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc88153bc0 0x7fcc8814c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:55.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8ca1f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880c3560 con 0x7fcc88153bc0 2024-09-13T18:44:55.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8ca1f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880ab7c0 con 0x7fcc880c6610 2024-09-13T18:44:55.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8ca1f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880c4e30 con 0x7fcc880c6a80 2024-09-13T18:44:55.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc88153bc0 0x7fcc8814c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:55.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc88153bc0 0x7fcc8814c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45846/0 (socket says 172.21.15.146:45846) 2024-09-13T18:44:55.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc86d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc880c6610 0x7fcc880af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:55.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8759e700 1 -- 172.21.15.146:0/816482592 learned_addr learned my addr 172.21.15.146:0/816482592 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:55.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8659c700 1 --2- 172.21.15.146:0/816482592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc880c6a80 0x7fcc881f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:55.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8759e700 1 -- 172.21.15.146:0/816482592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc880c6a80 msgr2=0x7fcc881f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:55.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8759e700 1 --2- 172.21.15.146:0/816482592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc880c6a80 0x7fcc881f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:55.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8759e700 1 -- 172.21.15.146:0/816482592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc880c6610 msgr2=0x7fcc880af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:55.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8759e700 1 --2- 172.21.15.146:0/816482592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc880c6610 0x7fcc880af450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:55.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8759e700 1 -- 172.21.15.146:0/816482592 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc78009a70 con 0x7fcc88153bc0 2024-09-13T18:44:55.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8759e700 1 --2- 172.21.15.146:0/816482592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc88153bc0 0x7fcc8814c490 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcc8815a920 tx=0x7fcc80004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:55.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc6f7fe700 1 -- 172.21.15.146:0/816482592 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80003d90 con 0x7fcc88153bc0 2024-09-13T18:44:55.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc6f7fe700 1 -- 172.21.15.146:0/816482592 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8001f070 con 0x7fcc88153bc0 2024-09-13T18:44:55.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/816482592 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc800094b0 con 0x7fcc88153bc0 2024-09-13T18:44:55.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc6f7fe700 1 -- 172.21.15.146:0/816482592 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80011870 con 0x7fcc88153bc0 2024-09-13T18:44:55.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/816482592 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc881ef370 con 0x7fcc88153bc0 2024-09-13T18:44:55.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.395+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/816482592 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc880caaa0 con 0x7fcc88153bc0 2024-09-13T18:44:55.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.399+0000 7fcc6f7fe700 1 -- 172.21.15.146:0/816482592 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcc800119d0 con 0x7fcc88153bc0 2024-09-13T18:44:55.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.399+0000 7fcc6f7fe700 1 --2- 172.21.15.146:0/816482592 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcc74066730 0x7fcc74068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:55.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.399+0000 7fcc6f7fe700 1 -- 172.21.15.146:0/816482592 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fcc80021070 con 0x7fcc88153bc0 2024-09-13T18:44:55.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.399+0000 7fcc86d9d700 1 --2- 172.21.15.146:0/816482592 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcc74066730 0x7fcc74068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:55.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.399+0000 7fcc86d9d700 1 --2- 172.21.15.146:0/816482592 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcc74066730 0x7fcc74068bf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c0011e0 tx=0x7fcc7c001da0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:55.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.407+0000 7fcc6f7fe700 1 -- 172.21.15.146:0/816482592 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcc80052d20 con 0x7fcc88153bc0 2024-09-13T18:44:55.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.619+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/816482592 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc880a9de0 con 0x7fcc88153bc0 2024-09-13T18:44:55.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.623+0000 7fcc6f7fe700 1 -- 172.21.15.146:0/816482592 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+5519 (secure 0 0 0) 0x7fcc8002b090 con 0x7fcc88153bc0 2024-09-13T18:44:55.643 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:55.644 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:51.773630+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:rejoin","state_seq":11,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:55.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.623+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/816482592 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcc74066730 msgr2=0x7fcc74068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:55.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.623+0000 7fcc8ca1f700 1 --2- 172.21.15.146:0/816482592 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcc74066730 0x7fcc74068bf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c0011e0 tx=0x7fcc7c001da0 comp rx=0 tx=0).stop 2024-09-13T18:44:55.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.623+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/816482592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc88153bc0 msgr2=0x7fcc8814c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:55.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.623+0000 7fcc8ca1f700 1 --2- 172.21.15.146:0/816482592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc88153bc0 0x7fcc8814c490 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcc8815a920 tx=0x7fcc80004770 comp rx=0 tx=0).stop 2024-09-13T18:44:55.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.623+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/816482592 shutdown_connections 2024-09-13T18:44:55.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.623+0000 7fcc8ca1f700 1 --2- 172.21.15.146:0/816482592 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fcc74066730 0x7fcc74068bf0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:55.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.623+0000 7fcc8ca1f700 1 --2- 172.21.15.146:0/816482592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc88153bc0 0x7fcc8814c490 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:55.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.623+0000 7fcc8ca1f700 1 --2- 172.21.15.146:0/816482592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc880c6a80 0x7fcc881f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:55.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.623+0000 7fcc8ca1f700 1 --2- 172.21.15.146:0/816482592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc880c6610 0x7fcc880af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:55.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.623+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/816482592 >> 172.21.15.146:0/816482592 conn(0x7fcc88066090 msgr2=0x7fcc88066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:55.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.623+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/816482592 shutdown_connections 2024-09-13T18:44:55.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:55.623+0000 7fcc8ca1f700 1 -- 172.21.15.146:0/816482592 wait complete. 2024-09-13T18:44:55.647 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 77 2024-09-13T18:44:55.659 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:44:57.660 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:57.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.763+0000 7f4c47475700 1 Processor -- start 2024-09-13T18:44:57.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.763+0000 7f4c47475700 1 -- start start 2024-09-13T18:44:57.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c47475700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c400c6610 0x7f4c400c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:57.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c47475700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4c400cac60 0x7f4c400cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:57.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c47475700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c40153fa0 con 0x7f4c40153bc0 2024-09-13T18:44:57.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c47475700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c400aa500 con 0x7f4c400cac60 2024-09-13T18:44:57.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c47475700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4015a7e0 con 0x7f4c400c6610 2024-09-13T18:44:57.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c46473700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4c40153bc0 0x7f4c400c8de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48080/0 (socket says 172.21.15.146:48080) 2024-09-13T18:44:57.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c46473700 1 -- 172.21.15.146:0/1124081063 learned_addr learned my addr 172.21.15.146:0/1124081063 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:57.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c46c74700 1 --2- 172.21.15.146:0/1124081063 >> v2:172.21.15.154:3300/0 conn(0x7f4c400cac60 0x7f4c400cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:57.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c45c72700 1 --2- 172.21.15.146:0/1124081063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c400c6610 0x7f4c400c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:57.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c45471700 1 -- 172.21.15.146:0/1124081063 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444496693 0 0) 0x7f4c40153fa0 con 0x7f4c40153bc0 2024-09-13T18:44:57.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c45471700 1 -- 172.21.15.146:0/1124081063 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c2c0035d0 con 0x7f4c40153bc0 2024-09-13T18:44:57.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c46c74700 1 -- 172.21.15.146:0/1124081063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c400c6610 msgr2=0x7f4c400c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:57.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c46c74700 1 --2- 172.21.15.146:0/1124081063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c400c6610 0x7f4c400c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:57.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c46c74700 1 -- 172.21.15.146:0/1124081063 >> v1:172.21.15.146:6789/0 conn(0x7f4c40153bc0 legacy=0x7f4c400c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:57.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c45471700 1 -- 172.21.15.146:0/1124081063 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2505938021 0 0) 0x7f4c2c0035d0 con 0x7f4c40153bc0 2024-09-13T18:44:57.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c46c74700 1 -- 172.21.15.146:0/1124081063 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c400aa220 con 0x7f4c400cac60 2024-09-13T18:44:57.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c46c74700 1 --2- 172.21.15.146:0/1124081063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c400cac60 0x7f4c400cb0d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009850 tx=0x7f4c38009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f3ca327af0513ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:57.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c45471700 1 -- 172.21.15.146:0/1124081063 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30001670 con 0x7f4c400cac60 2024-09-13T18:44:57.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c45471700 1 -- 172.21.15.146:0/1124081063 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38004030 con 0x7f4c400cac60 2024-09-13T18:44:57.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c45471700 1 -- 172.21.15.146:0/1124081063 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c380043e0 con 0x7f4c400cac60 2024-09-13T18:44:57.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c47475700 1 -- 172.21.15.146:0/1124081063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c400cac60 msgr2=0x7f4c400cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:57.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c47475700 1 --2- 172.21.15.146:0/1124081063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c400cac60 0x7f4c400cb0d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009850 tx=0x7f4c38009b60 comp rx=0 tx=0).stop 2024-09-13T18:44:57.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c47475700 1 -- 172.21.15.146:0/1124081063 shutdown_connections 2024-09-13T18:44:57.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c47475700 1 --2- 172.21.15.146:0/1124081063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c400cac60 0x7f4c400cb0d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:57.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c47475700 1 --2- 172.21.15.146:0/1124081063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c400c6610 0x7f4c400c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:57.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c47475700 1 -- 172.21.15.146:0/1124081063 >> 172.21.15.146:0/1124081063 conn(0x7f4c40066090 msgr2=0x7f4c40066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:57.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c47475700 1 -- 172.21.15.146:0/1124081063 shutdown_connections 2024-09-13T18:44:57.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.767+0000 7f4c47475700 1 -- 172.21.15.146:0/1124081063 wait complete. 2024-09-13T18:44:57.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c47475700 1 Processor -- start 2024-09-13T18:44:57.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c47475700 1 -- start start 2024-09-13T18:44:57.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c47475700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c400c6610 0x7f4c400aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:57.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c47475700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c400cac60 0x7f4c400ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:57.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c47475700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c40153bc0 0x7f4c400ac620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:57.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c47475700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c401ef9b0 con 0x7f4c400cac60 2024-09-13T18:44:57.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c47475700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c40152fd0 con 0x7f4c400c6610 2024-09-13T18:44:57.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c47475700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c401efc70 con 0x7f4c40153bc0 2024-09-13T18:44:57.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c45c72700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c400cac60 0x7f4c400ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:57.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c46473700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c400c6610 0x7f4c400aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:57.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c45c72700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c400cac60 0x7f4c400ac0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45848/0 (socket says 172.21.15.146:45848) 2024-09-13T18:44:57.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c45c72700 1 -- 172.21.15.146:0/2679669856 learned_addr learned my addr 172.21.15.146:0/2679669856 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:57.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c46c74700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c40153bc0 0x7f4c400ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:57.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c45c72700 1 -- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c40153bc0 msgr2=0x7f4c400ac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:57.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c45c72700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c40153bc0 0x7f4c400ac620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:57.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c45c72700 1 -- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c400c6610 msgr2=0x7f4c400aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:57.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c45c72700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c400c6610 0x7f4c400aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:57.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c45c72700 1 -- 172.21.15.146:0/2679669856 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c380094b0 con 0x7f4c400cac60 2024-09-13T18:44:57.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c46473700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c400c6610 0x7f4c400aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:44:57.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c45c72700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c400cac60 0x7f4c400ac0e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00efc0 tx=0x7f4c3c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:57.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c36ffd700 1 -- 172.21.15.146:0/2679669856 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00cd10 con 0x7f4c400cac60 2024-09-13T18:44:57.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c36ffd700 1 -- 172.21.15.146:0/2679669856 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c00f040 con 0x7f4c400cac60 2024-09-13T18:44:57.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c47475700 1 -- 172.21.15.146:0/2679669856 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c400ad3d0 con 0x7f4c400cac60 2024-09-13T18:44:57.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c36ffd700 1 -- 172.21.15.146:0/2679669856 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c0146a0 con 0x7f4c400cac60 2024-09-13T18:44:57.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.771+0000 7f4c47475700 1 -- 172.21.15.146:0/2679669856 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c400ad0d0 con 0x7f4c400cac60 2024-09-13T18:44:57.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.775+0000 7f4c47475700 1 -- 172.21.15.146:0/2679669856 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c400a9de0 con 0x7f4c400cac60 2024-09-13T18:44:57.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.775+0000 7f4c36ffd700 1 -- 172.21.15.146:0/2679669856 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4c3c014800 con 0x7f4c400cac60 2024-09-13T18:44:57.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.775+0000 7f4c36ffd700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c24066760 0x7f4c24068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:57.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.775+0000 7f4c36ffd700 1 -- 172.21.15.146:0/2679669856 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f4c3c0853e0 con 0x7f4c400cac60 2024-09-13T18:44:57.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.775+0000 7f4c46473700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c24066760 0x7f4c24068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:57.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.779+0000 7f4c46473700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c24066760 0x7f4c24068c20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4c30001820 tx=0x7f4c30001e00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:57.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.783+0000 7f4c36ffd700 1 -- 172.21.15.146:0/2679669856 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4c3c051fb0 con 0x7f4c400cac60 2024-09-13T18:44:58.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.995+0000 7f4c47475700 1 -- 172.21.15.146:0/2679669856 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c4006f690 con 0x7f4c400cac60 2024-09-13T18:44:58.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.995+0000 7f4c36ffd700 1 -- 172.21.15.146:0/2679669856 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+5519 (secure 0 0 0) 0x7f4c3c0555d0 con 0x7f4c400cac60 2024-09-13T18:44:58.018 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:58.018 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:56.807765+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:58.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.999+0000 7f4c47475700 1 -- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c24066760 msgr2=0x7f4c24068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:58.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.999+0000 7f4c47475700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c24066760 0x7f4c24068c20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4c30001820 tx=0x7f4c30001e00 comp rx=0 tx=0).stop 2024-09-13T18:44:58.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.999+0000 7f4c47475700 1 -- 172.21.15.146:0/2679669856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c400cac60 msgr2=0x7f4c400ac0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:58.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.999+0000 7f4c47475700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c400cac60 0x7f4c400ac0e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00efc0 tx=0x7f4c3c00c3a0 comp rx=0 tx=0).stop 2024-09-13T18:44:58.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.999+0000 7f4c47475700 1 -- 172.21.15.146:0/2679669856 shutdown_connections 2024-09-13T18:44:58.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.999+0000 7f4c47475700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4c24066760 0x7f4c24068c20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:58.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.999+0000 7f4c47475700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c40153bc0 0x7f4c400ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:58.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.999+0000 7f4c47475700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c400cac60 0x7f4c400ac0e0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:58.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.999+0000 7f4c47475700 1 --2- 172.21.15.146:0/2679669856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c400c6610 0x7f4c400aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:58.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.999+0000 7f4c47475700 1 -- 172.21.15.146:0/2679669856 >> 172.21.15.146:0/2679669856 conn(0x7f4c40066090 msgr2=0x7f4c40071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:58.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.999+0000 7f4c47475700 1 -- 172.21.15.146:0/2679669856 shutdown_connections 2024-09-13T18:44:58.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:57.999+0000 7f4c47475700 1 -- 172.21.15.146:0/2679669856 wait complete. 2024-09-13T18:44:58.022 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 78 2024-09-13T18:44:58.037 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=5800 2024-09-13T18:44:58.037 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.f 2024-09-13T18:44:59.038 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2024-09-13T18:44:59.039 INFO:tasks.ceph.mds.f:Restarting daemon 2024-09-13T18:44:59.039 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-09-13T18:44:59.041 INFO:tasks.ceph.mds.f:Started 2024-09-13T18:44:59.041 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:44:59.129 INFO:tasks.ceph.mds.f.smithi154.stdout:starting mds.f at 2024-09-13T18:44:59.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f475203b700 1 Processor -- start 2024-09-13T18:44:59.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f475203b700 1 -- start start 2024-09-13T18:44:59.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f475203b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f474c070570 0x7f474c0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:59.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f475203b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f474c0c5bd0 0x7f474c0c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:59.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f475203b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f474c0adfe0 con 0x7f474c157e10 2024-09-13T18:44:59.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f475203b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474c15e690 con 0x7f474c0c5bd0 2024-09-13T18:44:59.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f475203b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474c0c7570 con 0x7f474c070570 2024-09-13T18:44:59.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f474bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f474c0c5bd0 0x7f474c0c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:59.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f474affd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f474c070570 0x7f474c0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:59.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f474b7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f474c157e10 0x7f474c06fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:55396/0 (socket says 172.21.15.146:55396) 2024-09-13T18:44:59.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f474b7fe700 1 -- 172.21.15.146:0/2812473390 learned_addr learned my addr 172.21.15.146:0/2812473390 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:44:59.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f474bfff700 1 -- 172.21.15.146:0/2812473390 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f474c070570 msgr2=0x7f474c0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:44:59.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f474bfff700 1 --2- 172.21.15.146:0/2812473390 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f474c070570 0x7f474c0c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:59.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f474bfff700 1 -- 172.21.15.146:0/2812473390 >> v1:172.21.15.154:6789/0 conn(0x7f474c157e10 legacy=0x7f474c06fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:59.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f474bfff700 1 -- 172.21.15.146:0/2812473390 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c15eb80 con 0x7f474c0c5bd0 2024-09-13T18:44:59.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f474bfff700 1 --2- 172.21.15.146:0/2812473390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f474c0c5bd0 0x7f474c0c6080 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4744009d90 tx=0x7f474400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6134565be18ad90 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:59.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f474a7fc700 1 -- 172.21.15.146:0/2812473390 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474400e070 con 0x7f474c0c5bd0 2024-09-13T18:44:59.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f474a7fc700 1 -- 172.21.15.146:0/2812473390 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474400bb20 con 0x7f474c0c5bd0 2024-09-13T18:44:59.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f474a7fc700 1 -- 172.21.15.146:0/2812473390 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744012430 con 0x7f474c0c5bd0 2024-09-13T18:44:59.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f475203b700 1 -- 172.21.15.146:0/2812473390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f474c0c5bd0 msgr2=0x7f474c0c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:59.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.147+0000 7f475203b700 1 --2- 172.21.15.146:0/2812473390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f474c0c5bd0 0x7f474c0c6080 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4744009d90 tx=0x7f474400b330 comp rx=0 tx=0).stop 2024-09-13T18:44:59.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.151+0000 7f475203b700 1 -- 172.21.15.146:0/2812473390 shutdown_connections 2024-09-13T18:44:59.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.151+0000 7f475203b700 1 --2- 172.21.15.146:0/2812473390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f474c0c5bd0 0x7f474c0c6080 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:59.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.151+0000 7f475203b700 1 --2- 172.21.15.146:0/2812473390 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f474c070570 0x7f474c0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:59.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.151+0000 7f475203b700 1 -- 172.21.15.146:0/2812473390 >> 172.21.15.146:0/2812473390 conn(0x7f474c0cb1d0 msgr2=0x7f474c0cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:59.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.151+0000 7f475203b700 1 -- 172.21.15.146:0/2812473390 shutdown_connections 2024-09-13T18:44:59.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.151+0000 7f475203b700 1 -- 172.21.15.146:0/2812473390 wait complete. 2024-09-13T18:44:59.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.151+0000 7f475203b700 1 Processor -- start 2024-09-13T18:44:59.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.151+0000 7f475203b700 1 -- start start 2024-09-13T18:44:59.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f475203b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f474c070570 0x7f474c15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:59.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f475203b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f474c0c5bd0 0x7f474c150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:59.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f475203b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f474c157e10 0x7f474c1508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:59.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f475203b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474c1540e0 con 0x7f474c157e10 2024-09-13T18:44:59.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f475203b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474c1f39f0 con 0x7f474c070570 2024-09-13T18:44:59.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f475203b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474c1f7a80 con 0x7f474c0c5bd0 2024-09-13T18:44:59.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f474bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f474c157e10 0x7f474c1508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:59.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f474bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f474c157e10 0x7f474c1508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45870/0 (socket says 172.21.15.146:45870) 2024-09-13T18:44:59.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f474bfff700 1 -- 172.21.15.146:0/4232829363 learned_addr learned my addr 172.21.15.146:0/4232829363 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:44:59.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f474b7fe700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f474c070570 0x7f474c15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:59.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f474affd700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f474c0c5bd0 0x7f474c150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:59.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f474bfff700 1 -- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f474c0c5bd0 msgr2=0x7f474c150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:59.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f474bfff700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f474c0c5bd0 0x7f474c150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:59.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f474bfff700 1 -- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f474c070570 msgr2=0x7f474c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:59.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f474bfff700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f474c070570 0x7f474c15d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:59.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f474bfff700 1 -- 172.21.15.146:0/4232829363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4744009a70 con 0x7f474c157e10 2024-09-13T18:44:59.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f474b7fe700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f474c070570 0x7f474c15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:44:59.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f474bfff700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f474c157e10 0x7f474c1508c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4744014040 tx=0x7f4744004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:59.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f4733fff700 1 -- 172.21.15.146:0/4232829363 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744012430 con 0x7f474c157e10 2024-09-13T18:44:59.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f4733fff700 1 -- 172.21.15.146:0/4232829363 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4744020070 con 0x7f474c157e10 2024-09-13T18:44:59.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f475203b700 1 -- 172.21.15.146:0/4232829363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f474c1f7d60 con 0x7f474c157e10 2024-09-13T18:44:59.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f4733fff700 1 -- 172.21.15.146:0/4232829363 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744004280 con 0x7f474c157e10 2024-09-13T18:44:59.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f475203b700 1 -- 172.21.15.146:0/4232829363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f474c15b100 con 0x7f474c157e10 2024-09-13T18:44:59.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f475203b700 1 -- 172.21.15.146:0/4232829363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f474c1581f0 con 0x7f474c157e10 2024-09-13T18:44:59.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.155+0000 7f4733fff700 1 -- 172.21.15.146:0/4232829363 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f47440043e0 con 0x7f474c157e10 2024-09-13T18:44:59.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.159+0000 7f4733fff700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4738066730 0x7f4738068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:44:59.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.159+0000 7f4733fff700 1 -- 172.21.15.146:0/4232829363 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f4744022070 con 0x7f474c157e10 2024-09-13T18:44:59.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.159+0000 7f474b7fe700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4738066730 0x7f4738068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:44:59.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.159+0000 7f474b7fe700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4738066730 0x7f4738068bf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4740007fd0 tx=0x7f4740009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:44:59.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.163+0000 7f4733fff700 1 -- 172.21.15.146:0/4232829363 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f47440536a0 con 0x7f474c157e10 2024-09-13T18:44:59.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.371+0000 7f475203b700 1 -- 172.21.15.146:0/4232829363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f474c0c1f70 con 0x7f474c157e10 2024-09-13T18:44:59.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.371+0000 7f4733fff700 1 -- 172.21.15.146:0/4232829363 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+5519 (secure 0 0 0) 0x7f474401ae60 con 0x7f474c157e10 2024-09-13T18:44:59.395 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:44:59.395 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:56.807765+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:44:59.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.375+0000 7f475203b700 1 -- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4738066730 msgr2=0x7f4738068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:59.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.375+0000 7f475203b700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4738066730 0x7f4738068bf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4740007fd0 tx=0x7f4740009040 comp rx=0 tx=0).stop 2024-09-13T18:44:59.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.375+0000 7f475203b700 1 -- 172.21.15.146:0/4232829363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f474c157e10 msgr2=0x7f474c1508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:44:59.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.375+0000 7f475203b700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f474c157e10 0x7f474c1508c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4744014040 tx=0x7f4744004af0 comp rx=0 tx=0).stop 2024-09-13T18:44:59.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.375+0000 7f475203b700 1 -- 172.21.15.146:0/4232829363 shutdown_connections 2024-09-13T18:44:59.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.375+0000 7f475203b700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4738066730 0x7f4738068bf0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:59.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.375+0000 7f475203b700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f474c157e10 0x7f474c1508c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:59.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.375+0000 7f475203b700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f474c0c5bd0 0x7f474c150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:59.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.375+0000 7f475203b700 1 --2- 172.21.15.146:0/4232829363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f474c070570 0x7f474c15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:44:59.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.375+0000 7f475203b700 1 -- 172.21.15.146:0/4232829363 >> 172.21.15.146:0/4232829363 conn(0x7f474c0cb1d0 msgr2=0x7f474c0c0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:44:59.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.375+0000 7f475203b700 1 -- 172.21.15.146:0/4232829363 shutdown_connections 2024-09-13T18:44:59.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:44:59.375+0000 7f475203b700 1 -- 172.21.15.146:0/4232829363 wait complete. 2024-09-13T18:44:59.399 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 78 2024-09-13T18:44:59.410 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2024-09-13T18:45:01.411 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:45:01.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab63156700 1 Processor -- start 2024-09-13T18:45:01.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab63156700 1 -- start start 2024-09-13T18:45:01.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab63156700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fab5c06de80 0x7fab5c06ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:01.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab63156700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab5c1575e0 0x7fab5c157a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:01.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab63156700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5c0c6cf0 con 0x7fab5c070420 2024-09-13T18:45:01.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab63156700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5c06edb0 con 0x7fab5c06de80 2024-09-13T18:45:01.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab63156700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0c7910 con 0x7fab5c1575e0 2024-09-13T18:45:01.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab616f3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab5c1575e0 0x7fab5c157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:01.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab616f3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab5c1575e0 0x7fab5c157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:55822/0 (socket says 172.21.15.146:55822) 2024-09-13T18:45:01.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab616f3700 1 -- 172.21.15.146:0/2829453507 learned_addr learned my addr 172.21.15.146:0/2829453507 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:45:01.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab60ef2700 1 --2- 172.21.15.146:0/2829453507 >> v2:172.21.15.154:3300/0 conn(0x7fab5c06de80 0x7fab5c06ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:01.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab5b7fe700 1 -- 172.21.15.146:0/2829453507 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777448139 0 0) 0x7fab5c0c6cf0 con 0x7fab5c070420 2024-09-13T18:45:01.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab5b7fe700 1 -- 172.21.15.146:0/2829453507 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab440035d0 con 0x7fab5c070420 2024-09-13T18:45:01.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab616f3700 1 -- 172.21.15.146:0/2829453507 >> v2:172.21.15.154:3300/0 conn(0x7fab5c06de80 msgr2=0x7fab5c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:45:01.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab616f3700 1 --2- 172.21.15.146:0/2829453507 >> v2:172.21.15.154:3300/0 conn(0x7fab5c06de80 0x7fab5c06ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:01.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab616f3700 1 -- 172.21.15.146:0/2829453507 >> v1:172.21.15.154:6789/0 conn(0x7fab5c070420 legacy=0x7fab5c0c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:01.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab616f3700 1 -- 172.21.15.146:0/2829453507 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0c7630 con 0x7fab5c1575e0 2024-09-13T18:45:01.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.511+0000 7fab616f3700 1 --2- 172.21.15.146:0/2829453507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab5c1575e0 0x7fab5c157a70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab54009910 tx=0x7fab54009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36c1c5707c0c5f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:01.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.515+0000 7fab5b7fe700 1 -- 172.21.15.146:0/2829453507 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54011830 con 0x7fab5c1575e0 2024-09-13T18:45:01.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.515+0000 7fab5b7fe700 1 -- 172.21.15.146:0/2829453507 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab54011990 con 0x7fab5c1575e0 2024-09-13T18:45:01.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.515+0000 7fab5b7fe700 1 -- 172.21.15.146:0/2829453507 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54011d00 con 0x7fab5c1575e0 2024-09-13T18:45:01.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.515+0000 7fab63156700 1 -- 172.21.15.146:0/2829453507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab5c1575e0 msgr2=0x7fab5c157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:01.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.515+0000 7fab63156700 1 --2- 172.21.15.146:0/2829453507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab5c1575e0 0x7fab5c157a70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab54009910 tx=0x7fab54009c20 comp rx=0 tx=0).stop 2024-09-13T18:45:01.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.515+0000 7fab63156700 1 -- 172.21.15.146:0/2829453507 shutdown_connections 2024-09-13T18:45:01.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.515+0000 7fab63156700 1 --2- 172.21.15.146:0/2829453507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab5c1575e0 0x7fab5c157a70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:01.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.515+0000 7fab63156700 1 --2- 172.21.15.146:0/2829453507 >> v2:172.21.15.154:3300/0 conn(0x7fab5c06de80 0x7fab5c06ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:01.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.515+0000 7fab63156700 1 -- 172.21.15.146:0/2829453507 >> 172.21.15.146:0/2829453507 conn(0x7fab5c0cb1d0 msgr2=0x7fab5c0cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:01.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.515+0000 7fab63156700 1 -- 172.21.15.146:0/2829453507 shutdown_connections 2024-09-13T18:45:01.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.515+0000 7fab63156700 1 -- 172.21.15.146:0/2829453507 wait complete. 2024-09-13T18:45:01.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.515+0000 7fab63156700 1 Processor -- start 2024-09-13T18:45:01.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.515+0000 7fab63156700 1 -- start start 2024-09-13T18:45:01.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab63156700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab5c06de80 0x7fab5c146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:01.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab63156700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab5c070420 0x7fab5c1e2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:01.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab63156700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab5c1575e0 0x7fab5c13dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:01.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab63156700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c1e0fa0 con 0x7fab5c070420 2024-09-13T18:45:01.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab63156700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c149160 con 0x7fab5c06de80 2024-09-13T18:45:01.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab63156700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c14a9c0 con 0x7fab5c1575e0 2024-09-13T18:45:01.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab5bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab5c070420 0x7fab5c1e2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:01.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab5bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab5c070420 0x7fab5c1e2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37054/0 (socket says 172.21.15.146:37054) 2024-09-13T18:45:01.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab5bfff700 1 -- 172.21.15.146:0/3180301223 learned_addr learned my addr 172.21.15.146:0/3180301223 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:45:01.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab616f3700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab5c1575e0 0x7fab5c13dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:01.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab60ef2700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab5c06de80 0x7fab5c146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:01.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab5bfff700 1 -- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab5c1575e0 msgr2=0x7fab5c13dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:01.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab5bfff700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab5c1575e0 0x7fab5c13dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:01.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab5bfff700 1 -- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab5c06de80 msgr2=0x7fab5c146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:01.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab5bfff700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab5c06de80 0x7fab5c146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:01.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab5bfff700 1 -- 172.21.15.146:0/3180301223 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab54009570 con 0x7fab5c070420 2024-09-13T18:45:01.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab5bfff700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab5c070420 0x7fab5c1e2270 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00cc50 tx=0x7fab4c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:01.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab597fa700 1 -- 172.21.15.146:0/3180301223 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c003c20 con 0x7fab5c070420 2024-09-13T18:45:01.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab597fa700 1 -- 172.21.15.146:0/3180301223 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c003da0 con 0x7fab5c070420 2024-09-13T18:45:01.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab63156700 1 -- 172.21.15.146:0/3180301223 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c147ae0 con 0x7fab5c070420 2024-09-13T18:45:01.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab597fa700 1 -- 172.21.15.146:0/3180301223 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c00aad0 con 0x7fab5c070420 2024-09-13T18:45:01.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.519+0000 7fab63156700 1 -- 172.21.15.146:0/3180301223 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c1480a0 con 0x7fab5c070420 2024-09-13T18:45:01.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.523+0000 7fab63156700 1 -- 172.21.15.146:0/3180301223 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c1e10e0 con 0x7fab5c070420 2024-09-13T18:45:01.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.523+0000 7fab597fa700 1 -- 172.21.15.146:0/3180301223 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fab4c00ac30 con 0x7fab5c070420 2024-09-13T18:45:01.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.523+0000 7fab597fa700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab48066760 0x7fab48068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:01.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.523+0000 7fab597fa700 1 -- 172.21.15.146:0/3180301223 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fab4c0848a0 con 0x7fab5c070420 2024-09-13T18:45:01.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.523+0000 7fab60ef2700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab48066760 0x7fab48068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:01.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.523+0000 7fab60ef2700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab48066760 0x7fab48068c20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fab50009b50 tx=0x7fab50009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:01.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.527+0000 7fab597fa700 1 -- 172.21.15.146:0/3180301223 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fab5c1e10e0 con 0x7fab5c070420 2024-09-13T18:45:01.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.731+0000 7fab63156700 1 -- 172.21.15.146:0/3180301223 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab5c1e10e0 con 0x7fab5c070420 2024-09-13T18:45:01.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.735+0000 7fab597fa700 1 -- 172.21.15.146:0/3180301223 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+6276 (secure 0 0 0) 0x7fab5c1e10e0 con 0x7fab5c070420 2024-09-13T18:45:01.756 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:45:01.756 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6370,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:56.807765+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:45:01.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.735+0000 7fab63156700 1 -- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab48066760 msgr2=0x7fab48068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:01.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.735+0000 7fab63156700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab48066760 0x7fab48068c20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fab50009b50 tx=0x7fab50009240 comp rx=0 tx=0).stop 2024-09-13T18:45:01.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.735+0000 7fab63156700 1 -- 172.21.15.146:0/3180301223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab5c070420 msgr2=0x7fab5c1e2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:01.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.735+0000 7fab63156700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab5c070420 0x7fab5c1e2270 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00cc50 tx=0x7fab4c00cf60 comp rx=0 tx=0).stop 2024-09-13T18:45:01.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.735+0000 7fab63156700 1 -- 172.21.15.146:0/3180301223 shutdown_connections 2024-09-13T18:45:01.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.735+0000 7fab63156700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fab48066760 0x7fab48068c20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:01.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.735+0000 7fab63156700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab5c1575e0 0x7fab5c13dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:01.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.735+0000 7fab63156700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab5c070420 0x7fab5c1e2270 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:01.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.735+0000 7fab63156700 1 --2- 172.21.15.146:0/3180301223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab5c06de80 0x7fab5c146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:01.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.735+0000 7fab63156700 1 -- 172.21.15.146:0/3180301223 >> 172.21.15.146:0/3180301223 conn(0x7fab5c0cb1d0 msgr2=0x7fab5c14da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:01.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.735+0000 7fab63156700 1 -- 172.21.15.146:0/3180301223 shutdown_connections 2024-09-13T18:45:01.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:01.735+0000 7fab63156700 1 -- 172.21.15.146:0/3180301223 wait complete. 2024-09-13T18:45:01.760 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 79 2024-09-13T18:45:01.772 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2024-09-13T18:45:01.772 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 41 secs before thrashing 2024-09-13T18:45:04.621 INFO:tasks.ceph.mds.a.smithi146.stderr:2024-09-13T18:45:04.599+0000 7f2d72109700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:45:04.621 INFO:tasks.ceph.mds.a.smithi146.stderr:2024-09-13T18:45:04.599+0000 7f2d72109700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-13T18:45:10.439 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:45:10.441 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:45:40.534 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:45:40.537 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:45:42.773 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:45:42.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.881+0000 7f491e1e0700 1 Processor -- start 2024-09-13T18:45:42.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.881+0000 7f491e1e0700 1 -- start start 2024-09-13T18:45:42.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491e1e0700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4918070940 0x7f4918071210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:42.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491e1e0700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4918071830 con 0x7f4918070d20 2024-09-13T18:45:42.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491e1e0700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49180c24b0 con 0x7f4918150910 2024-09-13T18:45:42.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491e1e0700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49180c2790 con 0x7f4918070940 2024-09-13T18:45:42.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491c9dd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4918070d20 0x7f4918150200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34566/0 (socket says 172.21.15.146:34566) 2024-09-13T18:45:42.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491d1de700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4918070940 0x7f4918071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:42.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491c9dd700 1 -- 172.21.15.146:0/2311058 learned_addr learned my addr 172.21.15.146:0/2311058 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:45:42.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f490ffff700 1 -- 172.21.15.146:0/2311058 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597563870 0 0) 0x7f4918071830 con 0x7f4918070d20 2024-09-13T18:45:42.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491d1de700 1 -- 172.21.15.146:0/2311058 >> v1:172.21.15.154:6789/0 conn(0x7f4918150910 legacy=0x7f4918156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:42.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491d1de700 1 -- 172.21.15.146:0/2311058 >> v1:172.21.15.146:6789/0 conn(0x7f4918070d20 legacy=0x7f4918150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:42.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491d1de700 1 -- 172.21.15.146:0/2311058 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49180c2a70 con 0x7f4918070940 2024-09-13T18:45:42.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491d1de700 1 --2- 172.21.15.146:0/2311058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4918070940 0x7f4918071210 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4914009850 tx=0x7f4914009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd519cca3cd6658b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:42.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f490ffff700 1 -- 172.21.15.146:0/2311058 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491400f8a0 con 0x7f4918070940 2024-09-13T18:45:42.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f490ffff700 1 -- 172.21.15.146:0/2311058 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491400fa00 con 0x7f4918070940 2024-09-13T18:45:42.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f490ffff700 1 -- 172.21.15.146:0/2311058 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491400fd90 con 0x7f4918070940 2024-09-13T18:45:42.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491e1e0700 1 -- 172.21.15.146:0/2311058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4918070940 msgr2=0x7f4918071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:42.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491e1e0700 1 --2- 172.21.15.146:0/2311058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4918070940 0x7f4918071210 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4914009850 tx=0x7f4914009b60 comp rx=0 tx=0).stop 2024-09-13T18:45:42.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491e1e0700 1 -- 172.21.15.146:0/2311058 shutdown_connections 2024-09-13T18:45:42.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491e1e0700 1 --2- 172.21.15.146:0/2311058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4918070940 0x7f4918071210 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:42.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491e1e0700 1 -- 172.21.15.146:0/2311058 >> 172.21.15.146:0/2311058 conn(0x7f4918071a90 msgr2=0x7f4918071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:42.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491e1e0700 1 -- 172.21.15.146:0/2311058 shutdown_connections 2024-09-13T18:45:42.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.885+0000 7f491e1e0700 1 -- 172.21.15.146:0/2311058 wait complete. 2024-09-13T18:45:42.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491e1e0700 1 Processor -- start 2024-09-13T18:45:42.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491e1e0700 1 -- start start 2024-09-13T18:45:42.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491e1e0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4918070940 0x7f4918141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:42.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491e1e0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4918070d20 0x7f4918147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:42.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491e1e0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4918150910 0x7f4918144440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:42.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491e1e0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491814b350 con 0x7f4918150910 2024-09-13T18:45:42.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491e1e0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49180cab80 con 0x7f4918070d20 2024-09-13T18:45:42.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491e1e0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49180c66c0 con 0x7f4918070940 2024-09-13T18:45:42.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491d9df700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4918150910 0x7f4918144440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:42.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491d9df700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4918150910 0x7f4918144440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44734/0 (socket says 172.21.15.146:44734) 2024-09-13T18:45:42.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491d9df700 1 -- 172.21.15.146:0/3561573548 learned_addr learned my addr 172.21.15.146:0/3561573548 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:45:42.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491d1de700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4918070940 0x7f4918141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:42.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491c9dd700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4918070d20 0x7f4918147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:42.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491d9df700 1 -- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4918070940 msgr2=0x7f4918141f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:42.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491d9df700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4918070940 0x7f4918141f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:42.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491d9df700 1 -- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4918070d20 msgr2=0x7f4918147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:42.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491d9df700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4918070d20 0x7f4918147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:42.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491d9df700 1 -- 172.21.15.146:0/3561573548 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49140094b0 con 0x7f4918150910 2024-09-13T18:45:42.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491d9df700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4918150910 0x7f4918144440 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f491000cb80 tx=0x7f491000ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:42.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f490dffb700 1 -- 172.21.15.146:0/3561573548 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910003c20 con 0x7f4918150910 2024-09-13T18:45:42.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f490dffb700 1 -- 172.21.15.146:0/3561573548 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4910003d80 con 0x7f4918150910 2024-09-13T18:45:42.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491e1e0700 1 -- 172.21.15.146:0/3561573548 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49180c6960 con 0x7f4918150910 2024-09-13T18:45:42.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f490dffb700 1 -- 172.21.15.146:0/3561573548 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491000aae0 con 0x7f4918150910 2024-09-13T18:45:42.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.889+0000 7f491e1e0700 1 -- 172.21.15.146:0/3561573548 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49180cadd0 con 0x7f4918150910 2024-09-13T18:45:42.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.893+0000 7f491e1e0700 1 -- 172.21.15.146:0/3561573548 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49180b1be0 con 0x7f4918150910 2024-09-13T18:45:42.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.893+0000 7f490dffb700 1 -- 172.21.15.146:0/3561573548 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f491000ac40 con 0x7f4918150910 2024-09-13T18:45:42.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.893+0000 7f490dffb700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48fc066730 0x7f48fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:42.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.893+0000 7f490dffb700 1 -- 172.21.15.146:0/3561573548 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f4910084910 con 0x7f4918150910 2024-09-13T18:45:42.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.893+0000 7f491d1de700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48fc066730 0x7f48fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:42.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.893+0000 7f491d1de700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48fc066730 0x7f48fc068bf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4914009ba0 tx=0x7f4914008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:42.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:42.901+0000 7f490dffb700 1 -- 172.21.15.146:0/3561573548 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f49100514d0 con 0x7f4918150910 2024-09-13T18:45:43.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.121+0000 7f491e1e0700 1 -- 172.21.15.146:0/3561573548 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49181424a0 con 0x7f4918150910 2024-09-13T18:45:43.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.121+0000 7f490dffb700 1 -- 172.21.15.146:0/3561573548 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f4910054af0 con 0x7f4918150910 2024-09-13T18:45:43.140 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:45:43.140 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6370,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:56.807765+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:45:43.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.125+0000 7f491e1e0700 1 -- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48fc066730 msgr2=0x7f48fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:43.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.125+0000 7f491e1e0700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48fc066730 0x7f48fc068bf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4914009ba0 tx=0x7f4914008030 comp rx=0 tx=0).stop 2024-09-13T18:45:43.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.125+0000 7f491e1e0700 1 -- 172.21.15.146:0/3561573548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4918150910 msgr2=0x7f4918144440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:43.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.125+0000 7f491e1e0700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4918150910 0x7f4918144440 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f491000cb80 tx=0x7f491000ce90 comp rx=0 tx=0).stop 2024-09-13T18:45:43.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.125+0000 7f491e1e0700 1 -- 172.21.15.146:0/3561573548 shutdown_connections 2024-09-13T18:45:43.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.125+0000 7f491e1e0700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48fc066730 0x7f48fc068bf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.125+0000 7f491e1e0700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4918150910 0x7f4918144440 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.125+0000 7f491e1e0700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4918070d20 0x7f4918147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.125+0000 7f491e1e0700 1 --2- 172.21.15.146:0/3561573548 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4918070940 0x7f4918141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.125+0000 7f491e1e0700 1 -- 172.21.15.146:0/3561573548 >> 172.21.15.146:0/3561573548 conn(0x7f4918071a90 msgr2=0x7f49180c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:43.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.125+0000 7f491e1e0700 1 -- 172.21.15.146:0/3561573548 shutdown_connections 2024-09-13T18:45:43.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.125+0000 7f491e1e0700 1 -- 172.21.15.146:0/3561573548 wait complete. 2024-09-13T18:45:43.144 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 79 2024-09-13T18:45:43.157 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=1) 2024-09-13T18:45:43.157 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T18:45:43.158 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:45:43.169 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T18:45:43.170 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:45:43.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94dcbfc700 1 Processor -- start 2024-09-13T18:45:43.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94dcbfc700 1 -- start start 2024-09-13T18:45:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94dcbfc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94d80cf790 0x7f94d8157980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94dcbfc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f94d80cfb70 0x7f94d8157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94dcbfc700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d81541b0 con 0x7f94d80c8ba0 2024-09-13T18:45:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94dcbfc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d815cb00 con 0x7f94d80cfb70 2024-09-13T18:45:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94dcbfc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d815bba0 con 0x7f94d80cf790 2024-09-13T18:45:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94d5d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f94d80cfb70 0x7f94d8157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94d659c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94d80cf790 0x7f94d8157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:43.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94d6d9d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f94d80c8ba0 0x7f94d80c9030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:40732/0 (socket says 172.21.15.146:40732) 2024-09-13T18:45:43.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94d6d9d700 1 -- 172.21.15.146:0/1748450892 learned_addr learned my addr 172.21.15.146:0/1748450892 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:45:43.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94d5d9b700 1 -- 172.21.15.146:0/1748450892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94d80cf790 msgr2=0x7f94d8157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:45:43.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94d5d9b700 1 --2- 172.21.15.146:0/1748450892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94d80cf790 0x7f94d8157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94d559a700 1 -- 172.21.15.146:0/1748450892 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217875503 0 0) 0x7f94d81541b0 con 0x7f94d80c8ba0 2024-09-13T18:45:43.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94d5d9b700 1 -- 172.21.15.146:0/1748450892 >> v1:172.21.15.154:6789/0 conn(0x7f94d80c8ba0 legacy=0x7f94d80c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:43.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94d5d9b700 1 -- 172.21.15.146:0/1748450892 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d815be80 con 0x7f94d80cfb70 2024-09-13T18:45:43.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94d5d9b700 1 --2- 172.21.15.146:0/1748450892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d80cfb70 0x7f94d8157ec0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009850 tx=0x7f94cc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53e46a8e228ed3c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:43.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94d559a700 1 -- 172.21.15.146:0/1748450892 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc011830 con 0x7f94d80cfb70 2024-09-13T18:45:43.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94d559a700 1 -- 172.21.15.146:0/1748450892 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94cc011990 con 0x7f94d80cfb70 2024-09-13T18:45:43.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.313+0000 7f94d559a700 1 -- 172.21.15.146:0/1748450892 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc011d40 con 0x7f94d80cfb70 2024-09-13T18:45:43.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 -- 172.21.15.146:0/1748450892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d80cfb70 msgr2=0x7f94d8157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:43.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 --2- 172.21.15.146:0/1748450892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d80cfb70 0x7f94d8157ec0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009850 tx=0x7f94cc009b60 comp rx=0 tx=0).stop 2024-09-13T18:45:43.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 -- 172.21.15.146:0/1748450892 shutdown_connections 2024-09-13T18:45:43.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 --2- 172.21.15.146:0/1748450892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d80cfb70 0x7f94d8157ec0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 --2- 172.21.15.146:0/1748450892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94d80cf790 0x7f94d8157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 -- 172.21.15.146:0/1748450892 >> 172.21.15.146:0/1748450892 conn(0x7f94d8071d90 msgr2=0x7f94d80721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:43.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 -- 172.21.15.146:0/1748450892 shutdown_connections 2024-09-13T18:45:43.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 -- 172.21.15.146:0/1748450892 wait complete. 2024-09-13T18:45:43.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 Processor -- start 2024-09-13T18:45:43.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 -- start start 2024-09-13T18:45:43.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94d80c8ba0 0x7f94d8156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:43.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94d80cf790 0x7f94d80c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:43.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d80cfb70 0x7f94d81ee900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:43.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d80c5930 con 0x7f94d80c8ba0 2024-09-13T18:45:43.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d81e67f0 con 0x7f94d80cfb70 2024-09-13T18:45:43.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94dcbfc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8156530 con 0x7f94d80cf790 2024-09-13T18:45:43.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94d659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94d80c8ba0 0x7f94d8156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:43.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94d659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94d80c8ba0 0x7f94d8156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44748/0 (socket says 172.21.15.146:44748) 2024-09-13T18:45:43.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94d659c700 1 -- 172.21.15.146:0/601713541 learned_addr learned my addr 172.21.15.146:0/601713541 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:45:43.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94d5d9b700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94d80cf790 0x7f94d80c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:43.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94d6d9d700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d80cfb70 0x7f94d81ee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:43.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94d659c700 1 -- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94d80cf790 msgr2=0x7f94d80c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:43.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94d659c700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94d80cf790 0x7f94d80c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94d659c700 1 -- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d80cfb70 msgr2=0x7f94d81ee900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:43.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94d659c700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d80cfb70 0x7f94d81ee900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94d659c700 1 -- 172.21.15.146:0/601713541 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc0094b0 con 0x7f94d80c8ba0 2024-09-13T18:45:43.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.317+0000 7f94d659c700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94d80c8ba0 0x7f94d8156ff0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f94c800e9d0 tx=0x7f94c800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:43.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.321+0000 7f94c6ffd700 1 -- 172.21.15.146:0/601713541 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8009720 con 0x7f94d80c8ba0 2024-09-13T18:45:43.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.321+0000 7f94c6ffd700 1 -- 172.21.15.146:0/601713541 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c800ee70 con 0x7f94d80c8ba0 2024-09-13T18:45:43.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.321+0000 7f94dcbfc700 1 -- 172.21.15.146:0/601713541 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d8156810 con 0x7f94d80c8ba0 2024-09-13T18:45:43.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.321+0000 7f94c6ffd700 1 -- 172.21.15.146:0/601713541 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c800ca50 con 0x7f94d80c8ba0 2024-09-13T18:45:43.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.321+0000 7f94dcbfc700 1 -- 172.21.15.146:0/601713541 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d81f2ba0 con 0x7f94d80c8ba0 2024-09-13T18:45:43.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.321+0000 7f94dcbfc700 1 -- 172.21.15.146:0/601713541 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d81e62e0 con 0x7f94d80c8ba0 2024-09-13T18:45:43.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.321+0000 7f94c6ffd700 1 -- 172.21.15.146:0/601713541 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f94c800cbb0 con 0x7f94d80c8ba0 2024-09-13T18:45:43.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.321+0000 7f94c6ffd700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f94b8066730 0x7f94b8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:43.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.321+0000 7f94c6ffd700 1 -- 172.21.15.146:0/601713541 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f94c8084790 con 0x7f94d80c8ba0 2024-09-13T18:45:43.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.321+0000 7f94d5d9b700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f94b8066730 0x7f94b8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:43.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.321+0000 7f94d5d9b700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f94b8066730 0x7f94b8068bf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0090b0 tx=0x7f94cc01c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:43.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.329+0000 7f94c6ffd700 1 -- 172.21.15.146:0/601713541 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f94d81e62e0 con 0x7f94d80c8ba0 2024-09-13T18:45:43.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.569+0000 7f94dcbfc700 1 -- 172.21.15.146:0/601713541 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94d81e62e0 con 0x7f94d80c8ba0 2024-09-13T18:45:43.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.569+0000 7f94c6ffd700 1 -- 172.21.15.146:0/601713541 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f94d81e62e0 con 0x7f94d80c8ba0 2024-09-13T18:45:43.587 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:45:43.587 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6370,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:56.807765+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:45:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.573+0000 7f94dcbfc700 1 -- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f94b8066730 msgr2=0x7f94b8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.573+0000 7f94dcbfc700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f94b8066730 0x7f94b8068bf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0090b0 tx=0x7f94cc01c040 comp rx=0 tx=0).stop 2024-09-13T18:45:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.573+0000 7f94dcbfc700 1 -- 172.21.15.146:0/601713541 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94d80c8ba0 msgr2=0x7f94d8156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.573+0000 7f94dcbfc700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94d80c8ba0 0x7f94d8156ff0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f94c800e9d0 tx=0x7f94c800ece0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.573+0000 7f94dcbfc700 1 -- 172.21.15.146:0/601713541 shutdown_connections 2024-09-13T18:45:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.573+0000 7f94dcbfc700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f94b8066730 0x7f94b8068bf0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.573+0000 7f94dcbfc700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d80cfb70 0x7f94d81ee900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.573+0000 7f94dcbfc700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94d80cf790 0x7f94d80c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.573+0000 7f94dcbfc700 1 --2- 172.21.15.146:0/601713541 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94d80c8ba0 0x7f94d8156ff0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.573+0000 7f94dcbfc700 1 -- 172.21.15.146:0/601713541 >> 172.21.15.146:0/601713541 conn(0x7f94d8071d90 msgr2=0x7f94d8072170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.573+0000 7f94dcbfc700 1 -- 172.21.15.146:0/601713541 shutdown_connections 2024-09-13T18:45:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:43.573+0000 7f94dcbfc700 1 -- 172.21.15.146:0/601713541 wait complete. 2024-09-13T18:45:43.593 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 79 2024-09-13T18:45:43.605 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:45:45.606 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:45:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.709+0000 7f1d0cc3f700 1 Processor -- start 2024-09-13T18:45:45.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.709+0000 7f1d0cc3f700 1 -- start start 2024-09-13T18:45:45.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d0cc3f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d080cbd50 0x7f1d080b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:45.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d0cc3f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1d08071390 0x7f1d080cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:45.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d0cc3f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d080cc670 con 0x7f1d080ca210 2024-09-13T18:45:45.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d0cc3f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d080caa30 con 0x7f1d08071390 2024-09-13T18:45:45.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d0cc3f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080cb810 con 0x7f1d080cbd50 2024-09-13T18:45:45.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1cfffff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1d080ca210 0x7f1d080701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34576/0 (socket says 172.21.15.146:34576) 2024-09-13T18:45:45.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1cfffff700 1 -- 172.21.15.146:0/3549657582 learned_addr learned my addr 172.21.15.146:0/3549657582 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:45:45.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d077fe700 1 --2- 172.21.15.146:0/3549657582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d080cbd50 0x7f1d080b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:45.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d07fff700 1 --2- 172.21.15.146:0/3549657582 >> v2:172.21.15.154:3300/0 conn(0x7f1d08071390 0x7f1d080cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:45.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d06ffd700 1 -- 172.21.15.146:0/3549657582 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090005974 0 0) 0x7f1cf0001110 con 0x7f1d080ca210 2024-09-13T18:45:45.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d06ffd700 1 -- 172.21.15.146:0/3549657582 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf40035d0 con 0x7f1d080ca210 2024-09-13T18:45:45.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d07fff700 1 -- 172.21.15.146:0/3549657582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d080cbd50 msgr2=0x7f1d080b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:45:45.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d07fff700 1 --2- 172.21.15.146:0/3549657582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d080cbd50 0x7f1d080b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:45.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d07fff700 1 -- 172.21.15.146:0/3549657582 >> v1:172.21.15.146:6789/0 conn(0x7f1d080ca210 legacy=0x7f1d080701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:45.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d07fff700 1 -- 172.21.15.146:0/3549657582 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0815da20 con 0x7f1d08071390 2024-09-13T18:45:45.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d07fff700 1 --2- 172.21.15.146:0/3549657582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d08071390 0x7f1d080cc130 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009dc0 tx=0x7f1d0000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2a28466acbe73bc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:45.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d06ffd700 1 -- 172.21.15.146:0/3549657582 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0000e070 con 0x7f1d08071390 2024-09-13T18:45:45.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d06ffd700 1 -- 172.21.15.146:0/3549657582 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0000bae0 con 0x7f1d08071390 2024-09-13T18:45:45.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d06ffd700 1 -- 172.21.15.146:0/3549657582 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0000be90 con 0x7f1d08071390 2024-09-13T18:45:45.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/3549657582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d08071390 msgr2=0x7f1d080cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:45.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d0cc3f700 1 --2- 172.21.15.146:0/3549657582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d08071390 0x7f1d080cc130 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009dc0 tx=0x7f1d0000b330 comp rx=0 tx=0).stop 2024-09-13T18:45:45.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/3549657582 shutdown_connections 2024-09-13T18:45:45.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d0cc3f700 1 --2- 172.21.15.146:0/3549657582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d08071390 0x7f1d080cc130 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:45.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d0cc3f700 1 --2- 172.21.15.146:0/3549657582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d080cbd50 0x7f1d080b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:45.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/3549657582 >> 172.21.15.146:0/3549657582 conn(0x7f1d08066270 msgr2=0x7f1d08066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:45.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/3549657582 shutdown_connections 2024-09-13T18:45:45.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.713+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/3549657582 wait complete. 2024-09-13T18:45:45.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d0cc3f700 1 Processor -- start 2024-09-13T18:45:45.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d0cc3f700 1 -- start start 2024-09-13T18:45:45.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d0cc3f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d08071390 0x7f1d081e4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:45.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d0cc3f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d080ca210 0x7f1d081f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:45.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d0cc3f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d080cbd50 0x7f1d081f0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:45.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d0cc3f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0815b0a0 con 0x7f1d080ca210 2024-09-13T18:45:45.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d0cc3f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d08151140 con 0x7f1d080cbd50 2024-09-13T18:45:45.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d0cc3f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d081e3c20 con 0x7f1d08071390 2024-09-13T18:45:45.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1cfffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d080ca210 0x7f1d081f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:45.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1cfffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d080ca210 0x7f1d081f03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44750/0 (socket says 172.21.15.146:44750) 2024-09-13T18:45:45.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d080cbd50 0x7f1d081f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:45.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d077fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d08071390 0x7f1d081e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:45.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1cfffff700 1 -- 172.21.15.146:0/1848684765 learned_addr learned my addr 172.21.15.146:0/1848684765 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:45:45.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d07fff700 1 -- 172.21.15.146:0/1848684765 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d08071390 msgr2=0x7f1d081e4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:45.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d07fff700 1 --2- 172.21.15.146:0/1848684765 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d08071390 0x7f1d081e4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:45.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d07fff700 1 -- 172.21.15.146:0/1848684765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d080ca210 msgr2=0x7f1d081f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:45.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d07fff700 1 --2- 172.21.15.146:0/1848684765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d080ca210 0x7f1d081f03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:45.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d07fff700 1 -- 172.21.15.146:0/1848684765 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d00009a70 con 0x7f1d080cbd50 2024-09-13T18:45:45.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1cfffff700 1 --2- 172.21.15.146:0/1848684765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d080ca210 0x7f1d081f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:45:45.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d07fff700 1 --2- 172.21.15.146:0/1848684765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d080cbd50 0x7f1d081f0930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1d00016000 tx=0x7f1d00004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:45.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d04ff9700 1 -- 172.21.15.146:0/1848684765 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00004550 con 0x7f1d080cbd50 2024-09-13T18:45:45.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d04ff9700 1 -- 172.21.15.146:0/1848684765 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d00020070 con 0x7f1d080cbd50 2024-09-13T18:45:45.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/1848684765 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d081e3f00 con 0x7f1d080cbd50 2024-09-13T18:45:45.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d04ff9700 1 -- 172.21.15.146:0/1848684765 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00004910 con 0x7f1d080cbd50 2024-09-13T18:45:45.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.717+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/1848684765 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0815b5d0 con 0x7f1d080cbd50 2024-09-13T18:45:45.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.721+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/1848684765 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0815d500 con 0x7f1d080cbd50 2024-09-13T18:45:45.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.721+0000 7f1d04ff9700 1 -- 172.21.15.146:0/1848684765 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d00012430 con 0x7f1d080cbd50 2024-09-13T18:45:45.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.721+0000 7f1d04ff9700 1 --2- 172.21.15.146:0/1848684765 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1ce8066730 0x7f1ce8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:45.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.721+0000 7f1d04ff9700 1 -- 172.21.15.146:0/1848684765 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f1d00022070 con 0x7f1d080cbd50 2024-09-13T18:45:45.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.721+0000 7f1d077fe700 1 --2- 172.21.15.146:0/1848684765 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1ce8066730 0x7f1ce8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:45.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.721+0000 7f1d077fe700 1 --2- 172.21.15.146:0/1848684765 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1ce8066730 0x7f1ce8068bf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009b50 tx=0x7f1cf8009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:45.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.725+0000 7f1d04ff9700 1 -- 172.21.15.146:0/1848684765 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1d00057220 con 0x7f1d080cbd50 2024-09-13T18:45:45.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.953+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/1848684765 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d080c9ff0 con 0x7f1d080cbd50 2024-09-13T18:45:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.957+0000 7f1d04ff9700 1 -- 172.21.15.146:0/1848684765 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f1d00056db0 con 0x7f1d080cbd50 2024-09-13T18:45:45.974 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:45:45.974 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6370,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:56.807765+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:45:45.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.957+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/1848684765 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1ce8066730 msgr2=0x7f1ce8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:45.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.957+0000 7f1d0cc3f700 1 --2- 172.21.15.146:0/1848684765 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1ce8066730 0x7f1ce8068bf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009b50 tx=0x7f1cf8009240 comp rx=0 tx=0).stop 2024-09-13T18:45:45.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.957+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/1848684765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d080cbd50 msgr2=0x7f1d081f0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:45.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.957+0000 7f1d0cc3f700 1 --2- 172.21.15.146:0/1848684765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d080cbd50 0x7f1d081f0930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1d00016000 tx=0x7f1d00004af0 comp rx=0 tx=0).stop 2024-09-13T18:45:45.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.957+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/1848684765 shutdown_connections 2024-09-13T18:45:45.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.957+0000 7f1d0cc3f700 1 --2- 172.21.15.146:0/1848684765 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1ce8066730 0x7f1ce8068bf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:45.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.957+0000 7f1d0cc3f700 1 --2- 172.21.15.146:0/1848684765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d080cbd50 0x7f1d081f0930 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:45.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.957+0000 7f1d0cc3f700 1 --2- 172.21.15.146:0/1848684765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d080ca210 0x7f1d081f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:45.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.957+0000 7f1d0cc3f700 1 --2- 172.21.15.146:0/1848684765 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d08071390 0x7f1d081e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:45.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.957+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/1848684765 >> 172.21.15.146:0/1848684765 conn(0x7f1d08066270 msgr2=0x7f1d08152b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:45.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.957+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/1848684765 shutdown_connections 2024-09-13T18:45:45.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:45.957+0000 7f1d0cc3f700 1 -- 172.21.15.146:0/1848684765 wait complete. 2024-09-13T18:45:45.978 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 79 2024-09-13T18:45:45.991 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:45:47.992 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:45:48.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.081+0000 7f453f59e700 1 Processor -- start 2024-09-13T18:45:48.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.081+0000 7f453f59e700 1 -- start start 2024-09-13T18:45:48.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453f59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f45400cb5b0 0x7f45401595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:48.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453f59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454015a540 con 0x7f45400cf550 2024-09-13T18:45:48.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453f59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45400c5220 con 0x7f45400cf170 2024-09-13T18:45:48.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453f59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4540159ca0 con 0x7f45400cb5b0 2024-09-13T18:45:48.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453dd9b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f45400cf550 0x7f45401534c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34586/0 (socket says 172.21.15.146:34586) 2024-09-13T18:45:48.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453ed9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f45400cb5b0 0x7f45401595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:48.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453dd9b700 1 -- 172.21.15.146:0/4140945882 learned_addr learned my addr 172.21.15.146:0/4140945882 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:45:48.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453ed9d700 1 -- 172.21.15.146:0/4140945882 >> v1:172.21.15.154:6789/0 conn(0x7f45400cf170 legacy=0x7f45400c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:48.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453ed9d700 1 -- 172.21.15.146:0/4140945882 >> v1:172.21.15.146:6789/0 conn(0x7f45400cf550 legacy=0x7f45401534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:48.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453ed9d700 1 -- 172.21.15.146:0/4140945882 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4540159f80 con 0x7f45400cb5b0 2024-09-13T18:45:48.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453d59a700 1 -- 172.21.15.146:0/4140945882 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153143452 0 0) 0x7f454015a540 con 0x7f45400cf550 2024-09-13T18:45:48.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453ed9d700 1 --2- 172.21.15.146:0/4140945882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45400cb5b0 0x7f45401595f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4538009d90 tx=0x7f453800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75f4d79263408d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453d59a700 1 -- 172.21.15.146:0/4140945882 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453800e070 con 0x7f45400cb5b0 2024-09-13T18:45:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453d59a700 1 -- 172.21.15.146:0/4140945882 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453800bbd0 con 0x7f45400cb5b0 2024-09-13T18:45:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453d59a700 1 -- 172.21.15.146:0/4140945882 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538012430 con 0x7f45400cb5b0 2024-09-13T18:45:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453f59e700 1 -- 172.21.15.146:0/4140945882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45400cb5b0 msgr2=0x7f45401595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453f59e700 1 --2- 172.21.15.146:0/4140945882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45400cb5b0 0x7f45401595f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4538009d90 tx=0x7f453800b330 comp rx=0 tx=0).stop 2024-09-13T18:45:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453f59e700 1 -- 172.21.15.146:0/4140945882 shutdown_connections 2024-09-13T18:45:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453f59e700 1 --2- 172.21.15.146:0/4140945882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45400cb5b0 0x7f45401595f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453f59e700 1 -- 172.21.15.146:0/4140945882 >> 172.21.15.146:0/4140945882 conn(0x7f4540072220 msgr2=0x7f4540066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:48.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453f59e700 1 -- 172.21.15.146:0/4140945882 shutdown_connections 2024-09-13T18:45:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.085+0000 7f453f59e700 1 -- 172.21.15.146:0/4140945882 wait complete. 2024-09-13T18:45:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453f59e700 1 Processor -- start 2024-09-13T18:45:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453f59e700 1 -- start start 2024-09-13T18:45:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453f59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45400cb5b0 0x7f45401ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45400cf170 0x7f4540155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45400cf550 0x7f4540152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453f59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45401eff00 con 0x7f45400cf550 2024-09-13T18:45:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453f59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4540157250 con 0x7f45400cb5b0 2024-09-13T18:45:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4540157af0 con 0x7f45400cf170 2024-09-13T18:45:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453ed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45400cf550 0x7f4540152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:48.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453ed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45400cf550 0x7f4540152600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44758/0 (socket says 172.21.15.146:44758) 2024-09-13T18:45:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453ed9d700 1 -- 172.21.15.146:0/489919968 learned_addr learned my addr 172.21.15.146:0/489919968 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:45:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453e59c700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45400cb5b0 0x7f45401ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453dd9b700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45400cf170 0x7f4540155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453ed9d700 1 -- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45400cf170 msgr2=0x7f4540155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453ed9d700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45400cf170 0x7f4540155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453ed9d700 1 -- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45400cb5b0 msgr2=0x7f45401ef9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453ed9d700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45400cb5b0 0x7f45401ef9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453ed9d700 1 -- 172.21.15.146:0/489919968 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4538009a70 con 0x7f45400cf550 2024-09-13T18:45:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453ed9d700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45400cf550 0x7f4540152600 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4538014040 tx=0x7f4538004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f452effd700 1 -- 172.21.15.146:0/489919968 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453801b820 con 0x7f45400cf550 2024-09-13T18:45:48.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f452effd700 1 -- 172.21.15.146:0/489919968 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4538020070 con 0x7f45400cf550 2024-09-13T18:45:48.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453f59e700 1 -- 172.21.15.146:0/489919968 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4540155850 con 0x7f45400cf550 2024-09-13T18:45:48.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f452effd700 1 -- 172.21.15.146:0/489919968 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453801bc00 con 0x7f45400cf550 2024-09-13T18:45:48.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.089+0000 7f453f59e700 1 -- 172.21.15.146:0/489919968 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45401596b0 con 0x7f45400cf550 2024-09-13T18:45:48.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.093+0000 7f453f59e700 1 -- 172.21.15.146:0/489919968 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4540066850 con 0x7f45400cf550 2024-09-13T18:45:48.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.093+0000 7f452effd700 1 -- 172.21.15.146:0/489919968 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4538026400 con 0x7f45400cf550 2024-09-13T18:45:48.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.093+0000 7f452effd700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4520066730 0x7f4520068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:48.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.093+0000 7f452effd700 1 -- 172.21.15.146:0/489919968 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f4538086720 con 0x7f45400cf550 2024-09-13T18:45:48.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.093+0000 7f453e59c700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4520066730 0x7f4520068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:48.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.093+0000 7f453e59c700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4520066730 0x7f4520068bf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4530000f60 tx=0x7f4530009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:48.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.101+0000 7f452effd700 1 -- 172.21.15.146:0/489919968 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4538056e00 con 0x7f45400cf550 2024-09-13T18:45:48.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.321+0000 7f453f59e700 1 -- 172.21.15.146:0/489919968 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4540157c30 con 0x7f45400cf550 2024-09-13T18:45:48.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.321+0000 7f452effd700 1 -- 172.21.15.146:0/489919968 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f4538056990 con 0x7f45400cf550 2024-09-13T18:45:48.344 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:45:48.344 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6370,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:56.807765+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:45:48.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.329+0000 7f453f59e700 1 -- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4520066730 msgr2=0x7f4520068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:48.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.329+0000 7f453f59e700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4520066730 0x7f4520068bf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4530000f60 tx=0x7f4530009040 comp rx=0 tx=0).stop 2024-09-13T18:45:48.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.329+0000 7f453f59e700 1 -- 172.21.15.146:0/489919968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45400cf550 msgr2=0x7f4540152600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:48.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.329+0000 7f453f59e700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45400cf550 0x7f4540152600 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4538014040 tx=0x7f4538004af0 comp rx=0 tx=0).stop 2024-09-13T18:45:48.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.329+0000 7f453f59e700 1 -- 172.21.15.146:0/489919968 shutdown_connections 2024-09-13T18:45:48.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.329+0000 7f453f59e700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4520066730 0x7f4520068bf0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:48.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.329+0000 7f453f59e700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45400cf550 0x7f4540152600 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:48.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.329+0000 7f453f59e700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45400cf170 0x7f4540155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:48.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.329+0000 7f453f59e700 1 --2- 172.21.15.146:0/489919968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45400cb5b0 0x7f45401ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:48.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.329+0000 7f453f59e700 1 -- 172.21.15.146:0/489919968 >> 172.21.15.146:0/489919968 conn(0x7f4540072220 msgr2=0x7f45400c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:48.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.329+0000 7f453f59e700 1 -- 172.21.15.146:0/489919968 shutdown_connections 2024-09-13T18:45:48.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:48.329+0000 7f453f59e700 1 -- 172.21.15.146:0/489919968 wait complete. 2024-09-13T18:45:48.350 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 79 2024-09-13T18:45:48.360 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:45:50.360 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:45:50.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.473+0000 7f0681e5c700 1 Processor -- start 2024-09-13T18:45:50.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.473+0000 7f0681e5c700 1 -- start start 2024-09-13T18:45:50.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.473+0000 7f0681e5c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06740d6900 0x7f0674008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:50.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.473+0000 7f0681e5c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0674046930 0x7f06740e6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:50.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.473+0000 7f0681e5c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06740e0a00 con 0x7f06740464e0 2024-09-13T18:45:50.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.473+0000 7f0681e5c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0674006d70 con 0x7f0674046930 2024-09-13T18:45:50.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.473+0000 7f0681e5c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06740d6ce0 con 0x7f06740d6900 2024-09-13T18:45:50.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.473+0000 7f068165b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0674046930 0x7f06740e6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:50.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.473+0000 7f068165b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0674046930 0x7f06740e6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44760/0 (socket says 172.21.15.146:44760) 2024-09-13T18:45:50.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.473+0000 7f068165b700 1 -- 172.21.15.146:0/1123855732 learned_addr learned my addr 172.21.15.146:0/1123855732 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:45:50.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.473+0000 7f0680e5a700 1 --2- 172.21.15.146:0/1123855732 >> v2:172.21.15.154:3300/0 conn(0x7f06740d6900 0x7f0674008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:50.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f068165b700 1 -- 172.21.15.146:0/1123855732 >> v2:172.21.15.154:3300/0 conn(0x7f06740d6900 msgr2=0x7f0674008d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:45:50.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f068165b700 1 --2- 172.21.15.146:0/1123855732 >> v2:172.21.15.154:3300/0 conn(0x7f06740d6900 0x7f0674008d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:50.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f068165b700 1 -- 172.21.15.146:0/1123855732 >> v1:172.21.15.154:6789/0 conn(0x7f06740464e0 legacy=0x7f0674036090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:50.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f068165b700 1 -- 172.21.15.146:0/1123855732 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0674006a90 con 0x7f0674046930 2024-09-13T18:45:50.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f068165b700 1 --2- 172.21.15.146:0/1123855732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0674046930 0x7f06740e6e30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f067c0731e0 tx=0x7f067c061ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67493cf7555fb696 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:50.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f067b7fe700 1 -- 172.21.15.146:0/1123855732 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c073900 con 0x7f0674046930 2024-09-13T18:45:50.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f067b7fe700 1 -- 172.21.15.146:0/1123855732 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c073a60 con 0x7f0674046930 2024-09-13T18:45:50.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f067b7fe700 1 -- 172.21.15.146:0/1123855732 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c075c50 con 0x7f0674046930 2024-09-13T18:45:50.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 -- 172.21.15.146:0/1123855732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0674046930 msgr2=0x7f06740e6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:50.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 --2- 172.21.15.146:0/1123855732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0674046930 0x7f06740e6e30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f067c0731e0 tx=0x7f067c061ef0 comp rx=0 tx=0).stop 2024-09-13T18:45:50.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 -- 172.21.15.146:0/1123855732 shutdown_connections 2024-09-13T18:45:50.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 --2- 172.21.15.146:0/1123855732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0674046930 0x7f06740e6e30 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:50.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 --2- 172.21.15.146:0/1123855732 >> v2:172.21.15.154:3300/0 conn(0x7f06740d6900 0x7f0674008d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:50.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 -- 172.21.15.146:0/1123855732 >> 172.21.15.146:0/1123855732 conn(0x7f0674050e10 msgr2=0x7f0674051220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:50.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 -- 172.21.15.146:0/1123855732 shutdown_connections 2024-09-13T18:45:50.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 -- 172.21.15.146:0/1123855732 wait complete. 2024-09-13T18:45:50.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 Processor -- start 2024-09-13T18:45:50.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 -- start start 2024-09-13T18:45:50.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06740464e0 0x7f06740d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:50.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0674046930 0x7f067404a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:50.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06740d6900 0x7f067404ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:50.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06740d1140 con 0x7f06740d6900 2024-09-13T18:45:50.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0674176360 con 0x7f06740464e0 2024-09-13T18:45:50.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0681e5c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06740d2360 con 0x7f0674046930 2024-09-13T18:45:50.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f068165b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06740d6900 0x7f067404ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:50.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f068165b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06740d6900 0x7f067404ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:44772/0 (socket says 172.21.15.146:44772) 2024-09-13T18:45:50.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f068165b700 1 -- 172.21.15.146:0/2077441907 learned_addr learned my addr 172.21.15.146:0/2077441907 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:45:50.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f0680e5a700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06740464e0 0x7f06740d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:50.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.477+0000 7f067bfff700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0674046930 0x7f067404a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:50.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f068165b700 1 -- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0674046930 msgr2=0x7f067404a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:50.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f068165b700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0674046930 0x7f067404a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:50.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f068165b700 1 -- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06740464e0 msgr2=0x7f06740d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:50.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f068165b700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06740464e0 0x7f06740d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:50.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f068165b700 1 -- 172.21.15.146:0/2077441907 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c04f5c0 con 0x7f06740d6900 2024-09-13T18:45:50.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f068165b700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06740d6900 0x7f067404ab40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f067c000b60 tx=0x7f067c04f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:50.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f06797fa700 1 -- 172.21.15.146:0/2077441907 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c099070 con 0x7f06740d6900 2024-09-13T18:45:50.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f06797fa700 1 -- 172.21.15.146:0/2077441907 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c067c50 con 0x7f06740d6900 2024-09-13T18:45:50.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f06797fa700 1 -- 172.21.15.146:0/2077441907 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c067980 con 0x7f06740d6900 2024-09-13T18:45:50.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f0681e5c700 1 -- 172.21.15.146:0/2077441907 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06741765e0 con 0x7f06740d6900 2024-09-13T18:45:50.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f0681e5c700 1 -- 172.21.15.146:0/2077441907 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06740d36d0 con 0x7f06740d6900 2024-09-13T18:45:50.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f0681e5c700 1 -- 172.21.15.146:0/2077441907 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0674033a10 con 0x7f06740d6900 2024-09-13T18:45:50.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f06797fa700 1 -- 172.21.15.146:0/2077441907 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f067c067fc0 con 0x7f06740d6900 2024-09-13T18:45:50.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.481+0000 7f06797fa700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0660066730 0x7f0660068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:50.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.485+0000 7f06797fa700 1 -- 172.21.15.146:0/2077441907 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f067c100000 con 0x7f06740d6900 2024-09-13T18:45:50.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.485+0000 7f0680e5a700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0660066730 0x7f0660068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:50.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.485+0000 7f0680e5a700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0660066730 0x7f0660068bf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0670009b50 tx=0x7f0670009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.489+0000 7f06797fa700 1 -- 172.21.15.146:0/2077441907 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f067c0ccce0 con 0x7f06740d6900 2024-09-13T18:45:50.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.733+0000 7f0681e5c700 1 -- 172.21.15.146:0/2077441907 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f067404b1d0 con 0x7f06740d6900 2024-09-13T18:45:50.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.733+0000 7f06797fa700 1 -- 172.21.15.146:0/2077441907 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f067c0d0300 con 0x7f06740d6900 2024-09-13T18:45:50.753 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:45:50.753 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6370,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:56.807765+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:45:50.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.737+0000 7f0681e5c700 1 -- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0660066730 msgr2=0x7f0660068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:50.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.737+0000 7f0681e5c700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0660066730 0x7f0660068bf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0670009b50 tx=0x7f0670009240 comp rx=0 tx=0).stop 2024-09-13T18:45:50.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.737+0000 7f0681e5c700 1 -- 172.21.15.146:0/2077441907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06740d6900 msgr2=0x7f067404ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:50.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.737+0000 7f0681e5c700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06740d6900 0x7f067404ab40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f067c000b60 tx=0x7f067c04f750 comp rx=0 tx=0).stop 2024-09-13T18:45:50.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.737+0000 7f0681e5c700 1 -- 172.21.15.146:0/2077441907 shutdown_connections 2024-09-13T18:45:50.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.737+0000 7f0681e5c700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0660066730 0x7f0660068bf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:50.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.737+0000 7f0681e5c700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06740d6900 0x7f067404ab40 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:50.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.737+0000 7f0681e5c700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0674046930 0x7f067404a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:50.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.737+0000 7f0681e5c700 1 --2- 172.21.15.146:0/2077441907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06740464e0 0x7f06740d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:50.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.737+0000 7f0681e5c700 1 -- 172.21.15.146:0/2077441907 >> 172.21.15.146:0/2077441907 conn(0x7f0674050e10 msgr2=0x7f067404fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:50.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.737+0000 7f0681e5c700 1 -- 172.21.15.146:0/2077441907 shutdown_connections 2024-09-13T18:45:50.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:50.737+0000 7f0681e5c700 1 -- 172.21.15.146:0/2077441907 wait complete. 2024-09-13T18:45:50.757 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 79 2024-09-13T18:45:50.772 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:45:52.772 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:45:52.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.877+0000 7f69e310a700 1 Processor -- start 2024-09-13T18:45:52.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.877+0000 7f69e310a700 1 -- start start 2024-09-13T18:45:52.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e310a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f69dc0cbd50 0x7f69dc0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:52.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e310a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69dc15da20 con 0x7f69dc071390 2024-09-13T18:45:52.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e310a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69dc0ca750 con 0x7f69dc0ca210 2024-09-13T18:45:52.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e310a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69dc0caa30 con 0x7f69dc0cbd50 2024-09-13T18:45:52.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e2909700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f69dc071390 0x7f69dc163e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50920/0 (socket says 172.21.15.146:50920) 2024-09-13T18:45:52.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e2909700 1 -- 172.21.15.146:0/1575753214 learned_addr learned my addr 172.21.15.146:0/1575753214 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:45:52.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e2108700 1 --2- 172.21.15.146:0/1575753214 >> v2:172.21.15.154:3300/0 conn(0x7f69dc0cbd50 0x7f69dc0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:52.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340497751 0 0) 0x7f69dc15da20 con 0x7f69dc071390 2024-09-13T18:45:52.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69c80035d0 con 0x7f69dc071390 2024-09-13T18:45:52.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 865962383 0 0) 0x7f69c80035d0 con 0x7f69dc071390 2024-09-13T18:45:52.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 >> v2:172.21.15.154:3300/0 conn(0x7f69dc0cbd50 msgr2=0x7f69dc0b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:45:52.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 --2- 172.21.15.146:0/1575753214 >> v2:172.21.15.154:3300/0 conn(0x7f69dc0cbd50 0x7f69dc0b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 >> v1:172.21.15.154:6789/0 conn(0x7f69dc0ca210 legacy=0x7f69dc0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc15cc60 con 0x7f69dc071390 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f69d40019f0 con 0x7f69dc071390 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 >> v1:172.21.15.146:6789/0 conn(0x7f69dc071390 legacy=0x7f69dc163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 --2- 172.21.15.146:0/1575753214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69c8003a90 0x7f69c8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 --2- 172.21.15.146:0/1575753214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69c8004870 0x7f69c8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 --2- 172.21.15.146:0/1575753214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c8005230 0x7f69c800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc0caa30 con 0x7f69c8003a90 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c800c0b0 con 0x7f69c8005230 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c800c270 con 0x7f69c8004870 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e2108700 1 --2- 172.21.15.146:0/1575753214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69c8003a90 0x7f69c8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1907700 1 --2- 172.21.15.146:0/1575753214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69c8004870 0x7f69c8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e2909700 1 --2- 172.21.15.146:0/1575753214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c8005230 0x7f69c800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:52.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e2108700 1 -- 172.21.15.146:0/1575753214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69c8004870 msgr2=0x7f69c8004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:45:52.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e2108700 1 --2- 172.21.15.146:0/1575753214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69c8004870 0x7f69c8004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:52.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e2108700 1 -- 172.21.15.146:0/1575753214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c8005230 msgr2=0x7f69c800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:45:52.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e2108700 1 --2- 172.21.15.146:0/1575753214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c8005230 0x7f69c800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:52.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e2909700 1 -- 172.21.15.146:0/1575753214 reap_dead start 2024-09-13T18:45:52.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e2108700 1 -- 172.21.15.146:0/1575753214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c80035d0 con 0x7f69c8003a90 2024-09-13T18:45:52.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e2108700 1 --2- 172.21.15.146:0/1575753214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69c8003a90 0x7f69c8003e70 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009b70 tx=0x7f69c80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b5cb9339bcbcca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:52.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d800e6c0 con 0x7f69c8003a90 2024-09-13T18:45:52.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d800e820 con 0x7f69c8003a90 2024-09-13T18:45:52.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e1106700 1 -- 172.21.15.146:0/1575753214 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d800eb90 con 0x7f69c8003a90 2024-09-13T18:45:52.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e310a700 1 -- 172.21.15.146:0/1575753214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69c8003a90 msgr2=0x7f69c8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:52.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.881+0000 7f69e310a700 1 --2- 172.21.15.146:0/1575753214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69c8003a90 0x7f69c8003e70 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009b70 tx=0x7f69c80058a0 comp rx=0 tx=0).stop 2024-09-13T18:45:52.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e310a700 1 -- 172.21.15.146:0/1575753214 shutdown_connections 2024-09-13T18:45:52.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e310a700 1 --2- 172.21.15.146:0/1575753214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69c8003a90 0x7f69c8003e70 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:52.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e310a700 1 -- 172.21.15.146:0/1575753214 >> 172.21.15.146:0/1575753214 conn(0x7f69dc066270 msgr2=0x7f69dc066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:52.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e310a700 1 -- 172.21.15.146:0/1575753214 shutdown_connections 2024-09-13T18:45:52.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e310a700 1 -- 172.21.15.146:0/1575753214 wait complete. 2024-09-13T18:45:52.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e310a700 1 Processor -- start 2024-09-13T18:45:52.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e310a700 1 -- start start 2024-09-13T18:45:52.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e310a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c8003a90 0x7f69dc1e3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:52.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e310a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69dc156c60 0x7f69dc148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:52.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e310a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69dc1564f0 0x7f69dc1f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:52.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e310a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc1f3be0 con 0x7f69dc1564f0 2024-09-13T18:45:52.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e310a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc066ad0 con 0x7f69c8003a90 2024-09-13T18:45:52.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e310a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc1e4360 con 0x7f69dc156c60 2024-09-13T18:45:52.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e2909700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69dc1564f0 0x7f69dc1f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:52.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e2909700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69dc1564f0 0x7f69dc1f2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53006/0 (socket says 172.21.15.146:53006) 2024-09-13T18:45:52.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.885+0000 7f69e2909700 1 -- 172.21.15.146:0/2976100173 learned_addr learned my addr 172.21.15.146:0/2976100173 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:45:52.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69e1907700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69dc156c60 0x7f69dc148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:52.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69e2108700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c8003a90 0x7f69dc1e3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:52.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69e2909700 1 -- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69dc156c60 msgr2=0x7f69dc148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:52.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69e2909700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69dc156c60 0x7f69dc148c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:52.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69e2909700 1 -- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c8003a90 msgr2=0x7f69dc1e3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:52.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69e2909700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c8003a90 0x7f69dc1e3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:52.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69e2909700 1 -- 172.21.15.146:0/2976100173 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc1526b0 con 0x7f69dc1564f0 2024-09-13T18:45:52.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69e2909700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69dc1564f0 0x7f69dc1f2c60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0c9c00 tx=0x7f69d4001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:52.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69d2ffd700 1 -- 172.21.15.146:0/2976100173 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d40076f0 con 0x7f69dc1564f0 2024-09-13T18:45:52.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69e310a700 1 -- 172.21.15.146:0/2976100173 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69dc0667f0 con 0x7f69dc1564f0 2024-09-13T18:45:52.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69d2ffd700 1 -- 172.21.15.146:0/2976100173 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d4007870 con 0x7f69dc1564f0 2024-09-13T18:45:52.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69d2ffd700 1 -- 172.21.15.146:0/2976100173 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d40133f0 con 0x7f69dc1564f0 2024-09-13T18:45:52.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69e310a700 1 -- 172.21.15.146:0/2976100173 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69dc155cf0 con 0x7f69dc1564f0 2024-09-13T18:45:52.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69e310a700 1 -- 172.21.15.146:0/2976100173 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69dc1f3970 con 0x7f69dc1564f0 2024-09-13T18:45:52.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.889+0000 7f69d2ffd700 1 -- 172.21.15.146:0/2976100173 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69d4013550 con 0x7f69dc1564f0 2024-09-13T18:45:52.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.893+0000 7f69d2ffd700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69c0066730 0x7f69c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:52.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.893+0000 7f69d2ffd700 1 -- 172.21.15.146:0/2976100173 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f69d4084e50 con 0x7f69dc1564f0 2024-09-13T18:45:52.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.893+0000 7f69e2108700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69c0066730 0x7f69c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:52.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.893+0000 7f69e2108700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69c0066730 0x7f69c0068bf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f69d8000c00 tx=0x7f69d8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:52.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:52.897+0000 7f69d2ffd700 1 -- 172.21.15.146:0/2976100173 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f69d4051a20 con 0x7f69dc1564f0 2024-09-13T18:45:53.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.133+0000 7f69e310a700 1 -- 172.21.15.146:0/2976100173 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69dc153b50 con 0x7f69dc1564f0 2024-09-13T18:45:53.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.133+0000 7f69d2ffd700 1 -- 172.21.15.146:0/2976100173 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f69d4055040 con 0x7f69dc1564f0 2024-09-13T18:45:53.151 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:45:53.151 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6370,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:56.807765+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:45:53.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.133+0000 7f69e310a700 1 -- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69c0066730 msgr2=0x7f69c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:53.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.133+0000 7f69e310a700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69c0066730 0x7f69c0068bf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f69d8000c00 tx=0x7f69d8008030 comp rx=0 tx=0).stop 2024-09-13T18:45:53.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.133+0000 7f69e310a700 1 -- 172.21.15.146:0/2976100173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69dc1564f0 msgr2=0x7f69dc1f2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:53.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.133+0000 7f69e310a700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69dc1564f0 0x7f69dc1f2c60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0c9c00 tx=0x7f69d4001a20 comp rx=0 tx=0).stop 2024-09-13T18:45:53.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.137+0000 7f69e310a700 1 -- 172.21.15.146:0/2976100173 shutdown_connections 2024-09-13T18:45:53.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.137+0000 7f69e310a700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f69c0066730 0x7f69c0068bf0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:53.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.137+0000 7f69e310a700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69dc1564f0 0x7f69dc1f2c60 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:53.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.137+0000 7f69e310a700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69dc156c60 0x7f69dc148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:53.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.137+0000 7f69e310a700 1 --2- 172.21.15.146:0/2976100173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c8003a90 0x7f69dc1e3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:53.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.137+0000 7f69e310a700 1 -- 172.21.15.146:0/2976100173 >> 172.21.15.146:0/2976100173 conn(0x7f69dc066270 msgr2=0x7f69dc0cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:53.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.137+0000 7f69e310a700 1 -- 172.21.15.146:0/2976100173 shutdown_connections 2024-09-13T18:45:53.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:53.137+0000 7f69e310a700 1 -- 172.21.15.146:0/2976100173 wait complete. 2024-09-13T18:45:53.155 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 79 2024-09-13T18:45:53.167 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:45:55.168 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:45:55.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f8bfff700 1 Processor -- start 2024-09-13T18:45:55.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f8bfff700 1 -- start start 2024-09-13T18:45:55.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f8bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9f840d5040 0x7f9f84047070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:55.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f8bfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f84046be0 con 0x7f9f840066d0 2024-09-13T18:45:55.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f8bfff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f840d5420 con 0x7f9f8404ca00 2024-09-13T18:45:55.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f8bfff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f84006b60 con 0x7f9f840d5040 2024-09-13T18:45:55.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f8b7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9f840066d0 0x7f9f840dac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50924/0 (socket says 172.21.15.146:50924) 2024-09-13T18:45:55.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f8b7fe700 1 -- 172.21.15.146:0/574092456 learned_addr learned my addr 172.21.15.146:0/574092456 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:45:55.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f8affd700 1 --2- 172.21.15.146:0/574092456 >> v2:172.21.15.154:3300/0 conn(0x7f9f840d5040 0x7f9f84047070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:55.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f89ffb700 1 -- 172.21.15.146:0/574092456 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336716983 0 0) 0x7f9f84046be0 con 0x7f9f840066d0 2024-09-13T18:45:55.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f89ffb700 1 -- 172.21.15.146:0/574092456 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f740035d0 con 0x7f9f840066d0 2024-09-13T18:45:55.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f8affd700 1 -- 172.21.15.146:0/574092456 >> v1:172.21.15.154:6789/0 conn(0x7f9f8404ca00 legacy=0x7f9f8404b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:55.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f8affd700 1 -- 172.21.15.146:0/574092456 >> v1:172.21.15.146:6789/0 conn(0x7f9f840066d0 legacy=0x7f9f840dac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:55.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f89ffb700 1 -- 172.21.15.146:0/574092456 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2817268832 0 0) 0x7f9f740035d0 con 0x7f9f840066d0 2024-09-13T18:45:55.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f8affd700 1 -- 172.21.15.146:0/574092456 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f840dbc00 con 0x7f9f840d5040 2024-09-13T18:45:55.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f8affd700 1 --2- 172.21.15.146:0/574092456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f840d5040 0x7f9f84047070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009850 tx=0x7f9f7c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3795877f95cc691 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:55.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f89ffb700 1 -- 172.21.15.146:0/574092456 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c074a30 con 0x7f9f840d5040 2024-09-13T18:45:55.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f89ffb700 1 -- 172.21.15.146:0/574092456 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c004030 con 0x7f9f840d5040 2024-09-13T18:45:55.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.269+0000 7f9f89ffb700 1 -- 172.21.15.146:0/574092456 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0043e0 con 0x7f9f840d5040 2024-09-13T18:45:55.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.273+0000 7f9f8bfff700 1 -- 172.21.15.146:0/574092456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f840d5040 msgr2=0x7f9f84047070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:55.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.273+0000 7f9f8bfff700 1 --2- 172.21.15.146:0/574092456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f840d5040 0x7f9f84047070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009850 tx=0x7f9f7c009b60 comp rx=0 tx=0).stop 2024-09-13T18:45:55.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.273+0000 7f9f8bfff700 1 -- 172.21.15.146:0/574092456 shutdown_connections 2024-09-13T18:45:55.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.273+0000 7f9f8bfff700 1 --2- 172.21.15.146:0/574092456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f840d5040 0x7f9f84047070 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:55.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.273+0000 7f9f8bfff700 1 -- 172.21.15.146:0/574092456 >> 172.21.15.146:0/574092456 conn(0x7f9f84050610 msgr2=0x7f9f840509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:55.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.273+0000 7f9f8bfff700 1 -- 172.21.15.146:0/574092456 shutdown_connections 2024-09-13T18:45:55.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.273+0000 7f9f8bfff700 1 -- 172.21.15.146:0/574092456 wait complete. 2024-09-13T18:45:55.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.273+0000 7f9f8bfff700 1 Processor -- start 2024-09-13T18:45:55.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.273+0000 7f9f8bfff700 1 -- start start 2024-09-13T18:45:55.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f840066d0 0x7f9f840d6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:55.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f8404ca00 0x7f9f840ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:55.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f840d5040 0x7f9f840d7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:55.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8bfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840d0850 con 0x7f9f8404ca00 2024-09-13T18:45:55.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8bfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840d55e0 con 0x7f9f840066d0 2024-09-13T18:45:55.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840d58c0 con 0x7f9f840d5040 2024-09-13T18:45:55.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8a7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f8404ca00 0x7f9f840ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:55.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8a7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f8404ca00 0x7f9f840ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53008/0 (socket says 172.21.15.146:53008) 2024-09-13T18:45:55.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8a7fc700 1 -- 172.21.15.146:0/923445068 learned_addr learned my addr 172.21.15.146:0/923445068 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:45:55.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8affd700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f840066d0 0x7f9f840d6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:55.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8b7fe700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f840d5040 0x7f9f840d7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:55.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8a7fc700 1 -- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f840d5040 msgr2=0x7f9f840d7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:55.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8a7fc700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f840d5040 0x7f9f840d7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:55.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8a7fc700 1 -- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f840066d0 msgr2=0x7f9f840d6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:55.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8a7fc700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f840066d0 0x7f9f840d6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:55.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8a7fc700 1 -- 172.21.15.146:0/923445068 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c0094b0 con 0x7f9f8404ca00 2024-09-13T18:45:55.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8b7fe700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f840d5040 0x7f9f840d7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:45:55.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8a7fc700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f8404ca00 0x7f9f840ce890 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000c500 tx=0x7f9f8000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:55.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f737fe700 1 -- 172.21.15.146:0/923445068 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f800052d0 con 0x7f9f8404ca00 2024-09-13T18:45:55.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f737fe700 1 -- 172.21.15.146:0/923445068 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f80005430 con 0x7f9f8404ca00 2024-09-13T18:45:55.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8bfff700 1 -- 172.21.15.146:0/923445068 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f840d5ba0 con 0x7f9f8404ca00 2024-09-13T18:45:55.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f737fe700 1 -- 172.21.15.146:0/923445068 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f800055a0 con 0x7f9f8404ca00 2024-09-13T18:45:55.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8bfff700 1 -- 172.21.15.146:0/923445068 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f840d0df0 con 0x7f9f8404ca00 2024-09-13T18:45:55.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.277+0000 7f9f8bfff700 1 -- 172.21.15.146:0/923445068 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f84047620 con 0x7f9f8404ca00 2024-09-13T18:45:55.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.281+0000 7f9f737fe700 1 -- 172.21.15.146:0/923445068 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9f800038e0 con 0x7f9f8404ca00 2024-09-13T18:45:55.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.281+0000 7f9f737fe700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9f78066730 0x7f9f78068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:55.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.281+0000 7f9f737fe700 1 -- 172.21.15.146:0/923445068 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f9f80083cd0 con 0x7f9f8404ca00 2024-09-13T18:45:55.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.281+0000 7f9f8affd700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9f78066730 0x7f9f78068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:55.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.281+0000 7f9f8affd700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9f78066730 0x7f9f78068bf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0090b0 tx=0x7f9f7c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:55.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.289+0000 7f9f737fe700 1 -- 172.21.15.146:0/923445068 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9f80050890 con 0x7f9f8404ca00 2024-09-13T18:45:55.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.533+0000 7f9f8bfff700 1 -- 172.21.15.146:0/923445068 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f84047050 con 0x7f9f8404ca00 2024-09-13T18:45:55.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.533+0000 7f9f737fe700 1 -- 172.21.15.146:0/923445068 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f9f80081e60 con 0x7f9f8404ca00 2024-09-13T18:45:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:45:55.553 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6370,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:44:56.807765+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6207},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6207":{"gid":6207,"name":"c","rank":1,"incarnation":64,"state":"up:active","state_seq":39,"addr":"172.21.15.146:6837/3234109877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3234109877},{"type":"v1","addr":"172.21.15.146:6837","nonce":3234109877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:45:55.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.537+0000 7f9f8bfff700 1 -- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9f78066730 msgr2=0x7f9f78068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:55.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.537+0000 7f9f8bfff700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9f78066730 0x7f9f78068bf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0090b0 tx=0x7f9f7c008030 comp rx=0 tx=0).stop 2024-09-13T18:45:55.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.537+0000 7f9f8bfff700 1 -- 172.21.15.146:0/923445068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f8404ca00 msgr2=0x7f9f840ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:55.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.537+0000 7f9f8bfff700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f8404ca00 0x7f9f840ce890 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000c500 tx=0x7f9f8000c810 comp rx=0 tx=0).stop 2024-09-13T18:45:55.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.537+0000 7f9f8bfff700 1 -- 172.21.15.146:0/923445068 shutdown_connections 2024-09-13T18:45:55.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.537+0000 7f9f8bfff700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9f78066730 0x7f9f78068bf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:55.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.537+0000 7f9f8bfff700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f840d5040 0x7f9f840d7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:55.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.537+0000 7f9f8bfff700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f8404ca00 0x7f9f840ce890 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:55.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.537+0000 7f9f8bfff700 1 --2- 172.21.15.146:0/923445068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f840066d0 0x7f9f840d6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:55.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.537+0000 7f9f8bfff700 1 -- 172.21.15.146:0/923445068 >> 172.21.15.146:0/923445068 conn(0x7f9f84050610 msgr2=0x7f9f840da8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:55.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.537+0000 7f9f8bfff700 1 -- 172.21.15.146:0/923445068 shutdown_connections 2024-09-13T18:45:55.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:55.537+0000 7f9f8bfff700 1 -- 172.21.15.146:0/923445068 wait complete. 2024-09-13T18:45:55.558 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 79 2024-09-13T18:45:55.571 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:45:57.572 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:45:57.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.681+0000 7f7904e83700 1 Processor -- start 2024-09-13T18:45:57.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 -- start start 2024-09-13T18:45:57.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f78f80d6730 0x7f78f8047030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:57.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f80dcc90 0x7f78f8047570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:57.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f804c8a0 con 0x7f78f80d6b10 2024-09-13T18:45:57.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f80352b0 con 0x7f78f80d6730 2024-09-13T18:45:57.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f804cad0 con 0x7f78f80dcc90 2024-09-13T18:45:57.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f78ff7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f78f80d6730 0x7f78f8047030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:57.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f78ff7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f78f80d6730 0x7f78f8047030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53014/0 (socket says 172.21.15.146:53014) 2024-09-13T18:45:57.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f78ff7fe700 1 -- 172.21.15.146:0/1245370572 learned_addr learned my addr 172.21.15.146:0/1245370572 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:45:57.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f78fffff700 1 --2- 172.21.15.146:0/1245370572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f80dcc90 0x7f78f8047570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:57.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f78ff7fe700 1 -- 172.21.15.146:0/1245370572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f80dcc90 msgr2=0x7f78f8047570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:45:57.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f78ff7fe700 1 --2- 172.21.15.146:0/1245370572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f80dcc90 0x7f78f8047570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:57.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f78ff7fe700 1 -- 172.21.15.146:0/1245370572 >> v1:172.21.15.154:6789/0 conn(0x7f78f80d6b10 legacy=0x7f78f80dc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:57.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f78ff7fe700 1 -- 172.21.15.146:0/1245370572 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f804cd90 con 0x7f78f80d6730 2024-09-13T18:45:57.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f78ff7fe700 1 --2- 172.21.15.146:0/1245370572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f80d6730 0x7f78f8047030 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009910 tx=0x7f78f4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1803035c7f0d67 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:57.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f78fe7fc700 1 -- 172.21.15.146:0/1245370572 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f400ba60 con 0x7f78f80d6730 2024-09-13T18:45:57.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f78fe7fc700 1 -- 172.21.15.146:0/1245370572 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f400bbc0 con 0x7f78f80d6730 2024-09-13T18:45:57.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f78fe7fc700 1 -- 172.21.15.146:0/1245370572 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4009e60 con 0x7f78f80d6730 2024-09-13T18:45:57.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 -- 172.21.15.146:0/1245370572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f80d6730 msgr2=0x7f78f8047030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:57.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 --2- 172.21.15.146:0/1245370572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f80d6730 0x7f78f8047030 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009910 tx=0x7f78f4009c20 comp rx=0 tx=0).stop 2024-09-13T18:45:57.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 -- 172.21.15.146:0/1245370572 shutdown_connections 2024-09-13T18:45:57.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 --2- 172.21.15.146:0/1245370572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f80dcc90 0x7f78f8047570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:57.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 --2- 172.21.15.146:0/1245370572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f80d6730 0x7f78f8047030 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:57.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 -- 172.21.15.146:0/1245370572 >> 172.21.15.146:0/1245370572 conn(0x7f78f804e9a0 msgr2=0x7f78f804ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:57.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 -- 172.21.15.146:0/1245370572 shutdown_connections 2024-09-13T18:45:57.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.685+0000 7f7904e83700 1 -- 172.21.15.146:0/1245370572 wait complete. 2024-09-13T18:45:57.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f7904e83700 1 Processor -- start 2024-09-13T18:45:57.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f7904e83700 1 -- start start 2024-09-13T18:45:57.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f7904e83700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f80d6730 0x7f78f8170a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:57.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f7904e83700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f80d6b10 0x7f78f804b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:57.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f7904e83700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f8171410 0x7f78f80510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:57.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f7904e83700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f8170f60 con 0x7f78f80d6b10 2024-09-13T18:45:57.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f7904e83700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f804acb0 con 0x7f78f80d6730 2024-09-13T18:45:57.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f7904e83700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f804b680 con 0x7f78f8171410 2024-09-13T18:45:57.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78feffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f80d6b10 0x7f78f804b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:57.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78feffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f80d6b10 0x7f78f804b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53016/0 (socket says 172.21.15.146:53016) 2024-09-13T18:45:57.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78feffd700 1 -- 172.21.15.146:0/3714713608 learned_addr learned my addr 172.21.15.146:0/3714713608 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:45:57.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78fffff700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f8171410 0x7f78f80510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:57.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78ff7fe700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f80d6730 0x7f78f8170a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:57.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78feffd700 1 -- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f8171410 msgr2=0x7f78f80510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:57.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78feffd700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f8171410 0x7f78f80510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:57.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78feffd700 1 -- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f80d6730 msgr2=0x7f78f8170a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:57.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78feffd700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f80d6730 0x7f78f8170a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:57.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78feffd700 1 -- 172.21.15.146:0/3714713608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f4009570 con 0x7f78f80d6b10 2024-09-13T18:45:57.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78feffd700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f80d6b10 0x7f78f804b140 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00c500 tx=0x7f78ec00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:57.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78ebfff700 1 -- 172.21.15.146:0/3714713608 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec0052d0 con 0x7f78f80d6b10 2024-09-13T18:45:57.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78ebfff700 1 -- 172.21.15.146:0/3714713608 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec005430 con 0x7f78f80d6b10 2024-09-13T18:45:57.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f7904e83700 1 -- 172.21.15.146:0/3714713608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f8049890 con 0x7f78f80d6b10 2024-09-13T18:45:57.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f78ebfff700 1 -- 172.21.15.146:0/3714713608 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec0055a0 con 0x7f78f80d6b10 2024-09-13T18:45:57.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.689+0000 7f7904e83700 1 -- 172.21.15.146:0/3714713608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f8049b30 con 0x7f78f80d6b10 2024-09-13T18:45:57.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.693+0000 7f7904e83700 1 -- 172.21.15.146:0/3714713608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f80dc580 con 0x7f78f80d6b10 2024-09-13T18:45:57.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.693+0000 7f78ebfff700 1 -- 172.21.15.146:0/3714713608 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f78ec0038e0 con 0x7f78f80d6b10 2024-09-13T18:45:57.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.693+0000 7f78ebfff700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f78e4066730 0x7f78e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:57.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.693+0000 7f78ebfff700 1 -- 172.21.15.146:0/3714713608 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7f78ec083be0 con 0x7f78f80d6b10 2024-09-13T18:45:57.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.693+0000 7f78ff7fe700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f78e4066730 0x7f78e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:57.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.693+0000 7f78ff7fe700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f78e4066730 0x7f78e4068bf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f78f4000c00 tx=0x7f78f4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:57.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.701+0000 7f78ebfff700 1 -- 172.21.15.146:0/3714713608 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f78ec050870 con 0x7f78f80d6b10 2024-09-13T18:45:57.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f7904e83700 1 -- 172.21.15.146:0/3714713608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78f80d3b00 con 0x7f78f80d6b10 2024-09-13T18:45:57.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f78ebfff700 1 -- 172.21.15.146:0/3714713608 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5515 (secure 0 0 0) 0x7f78ec053e90 con 0x7f78f80d6b10 2024-09-13T18:45:57.983 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:45:57.983 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:45:55.808966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:45:57.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f7904e83700 1 -- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f78e4066730 msgr2=0x7f78e4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:57.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f7904e83700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f78e4066730 0x7f78e4068bf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f78f4000c00 tx=0x7f78f4008030 comp rx=0 tx=0).stop 2024-09-13T18:45:57.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f7904e83700 1 -- 172.21.15.146:0/3714713608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f80d6b10 msgr2=0x7f78f804b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:57.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f7904e83700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f80d6b10 0x7f78f804b140 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00c500 tx=0x7f78ec00c810 comp rx=0 tx=0).stop 2024-09-13T18:45:57.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f7904e83700 1 -- 172.21.15.146:0/3714713608 shutdown_connections 2024-09-13T18:45:57.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f7904e83700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f78e4066730 0x7f78e4068bf0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:57.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f7904e83700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f8171410 0x7f78f80510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:57.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f7904e83700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f80d6b10 0x7f78f804b140 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:57.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f7904e83700 1 --2- 172.21.15.146:0/3714713608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f80d6730 0x7f78f8170a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:57.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f7904e83700 1 -- 172.21.15.146:0/3714713608 >> 172.21.15.146:0/3714713608 conn(0x7f78f804e9a0 msgr2=0x7f78f8036120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:57.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f7904e83700 1 -- 172.21.15.146:0/3714713608 shutdown_connections 2024-09-13T18:45:57.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:57.965+0000 7f7904e83700 1 -- 172.21.15.146:0/3714713608 wait complete. 2024-09-13T18:45:57.987 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 80 2024-09-13T18:45:57.997 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-09-13T18:45:57.997 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:45:57.998 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:45:58.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.093+0000 7efc8ffff700 1 Processor -- start 2024-09-13T18:45:58.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.093+0000 7efc8ffff700 1 -- start start 2024-09-13T18:45:58.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc90150230 0x7efc9006ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:58.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8ffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc90071410 con 0x7efc90071610 2024-09-13T18:45:58.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8ffff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc90156090 con 0x7efc90150610 2024-09-13T18:45:58.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc901596b0 con 0x7efc90150230 2024-09-13T18:45:58.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8f7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7efc90071610 0x7efc90159b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50926/0 (socket says 172.21.15.146:50926) 2024-09-13T18:45:58.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8f7fe700 1 -- 172.21.15.146:0/1338994587 learned_addr learned my addr 172.21.15.146:0/1338994587 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:45:58.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8effd700 1 --2- 172.21.15.146:0/1338994587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc90150230 0x7efc9006ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:58.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8dffb700 1 -- 172.21.15.146:0/1338994587 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347601926 0 0) 0x7efc90071410 con 0x7efc90071610 2024-09-13T18:45:58.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8dffb700 1 -- 172.21.15.146:0/1338994587 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc780035d0 con 0x7efc90071610 2024-09-13T18:45:58.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8dffb700 1 -- 172.21.15.146:0/1338994587 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381920736 0 0) 0x7efc90156090 con 0x7efc90150610 2024-09-13T18:45:58.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8dffb700 1 -- 172.21.15.146:0/1338994587 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc90071410 con 0x7efc90150610 2024-09-13T18:45:58.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8effd700 1 -- 172.21.15.146:0/1338994587 >> v1:172.21.15.154:6789/0 conn(0x7efc90150610 legacy=0x7efc9006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:58.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8effd700 1 -- 172.21.15.146:0/1338994587 >> v1:172.21.15.146:6789/0 conn(0x7efc90071610 legacy=0x7efc90159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:58.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8effd700 1 -- 172.21.15.146:0/1338994587 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc90156350 con 0x7efc90150230 2024-09-13T18:45:58.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8effd700 1 --2- 172.21.15.146:0/1338994587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc90150230 0x7efc9006ebd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc80009880 tx=0x7efc80009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d4a189382004d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:58.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8dffb700 1 -- 172.21.15.146:0/1338994587 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc880019f0 con 0x7efc90150230 2024-09-13T18:45:58.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8dffb700 1 -- 172.21.15.146:0/1338994587 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc80010590 con 0x7efc90150230 2024-09-13T18:45:58.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8dffb700 1 -- 172.21.15.146:0/1338994587 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80010900 con 0x7efc90150230 2024-09-13T18:45:58.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8ffff700 1 -- 172.21.15.146:0/1338994587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc90150230 msgr2=0x7efc9006ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:58.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8ffff700 1 --2- 172.21.15.146:0/1338994587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc90150230 0x7efc9006ebd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc80009880 tx=0x7efc80009b90 comp rx=0 tx=0).stop 2024-09-13T18:45:58.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.097+0000 7efc8ffff700 1 -- 172.21.15.146:0/1338994587 shutdown_connections 2024-09-13T18:45:58.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8ffff700 1 --2- 172.21.15.146:0/1338994587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc90150230 0x7efc9006ebd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:58.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8ffff700 1 -- 172.21.15.146:0/1338994587 >> 172.21.15.146:0/1338994587 conn(0x7efc90071ed0 msgr2=0x7efc900722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:58.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8ffff700 1 -- 172.21.15.146:0/1338994587 shutdown_connections 2024-09-13T18:45:58.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8ffff700 1 -- 172.21.15.146:0/1338994587 wait complete. 2024-09-13T18:45:58.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8ffff700 1 Processor -- start 2024-09-13T18:45:58.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8ffff700 1 -- start start 2024-09-13T18:45:58.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc90071610 0x7efc9014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:58.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc90150230 0x7efc90157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:58.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc90155de0 0x7efc9013e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:58.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc90151940 con 0x7efc90150230 2024-09-13T18:45:58.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8ffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc90152250 con 0x7efc90071610 2024-09-13T18:45:58.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc90152b20 con 0x7efc90155de0 2024-09-13T18:45:58.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc90150230 0x7efc90157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:58.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc90150230 0x7efc90157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53028/0 (socket says 172.21.15.146:53028) 2024-09-13T18:45:58.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8e7fc700 1 -- 172.21.15.146:0/1180071985 learned_addr learned my addr 172.21.15.146:0/1180071985 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:45:58.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8effd700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc90071610 0x7efc9014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:58.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8f7fe700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc90155de0 0x7efc9013e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:58.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8e7fc700 1 -- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc90155de0 msgr2=0x7efc9013e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:58.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8e7fc700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc90155de0 0x7efc9013e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:58.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8e7fc700 1 -- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc90071610 msgr2=0x7efc9014f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:58.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8e7fc700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc90071610 0x7efc9014f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:58.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8e7fc700 1 -- 172.21.15.146:0/1180071985 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc800094b0 con 0x7efc90150230 2024-09-13T18:45:58.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.101+0000 7efc8e7fc700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc90150230 0x7efc90157110 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7efc8400c520 tx=0x7efc8400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:58.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.105+0000 7efc777fe700 1 -- 172.21.15.146:0/1180071985 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc840052d0 con 0x7efc90150230 2024-09-13T18:45:58.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.105+0000 7efc777fe700 1 -- 172.21.15.146:0/1180071985 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc84005430 con 0x7efc90150230 2024-09-13T18:45:58.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.105+0000 7efc8ffff700 1 -- 172.21.15.146:0/1180071985 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc90150a60 con 0x7efc90150230 2024-09-13T18:45:58.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.105+0000 7efc777fe700 1 -- 172.21.15.146:0/1180071985 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc840055a0 con 0x7efc90150230 2024-09-13T18:45:58.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.105+0000 7efc8ffff700 1 -- 172.21.15.146:0/1180071985 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc90150fa0 con 0x7efc90150230 2024-09-13T18:45:58.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.105+0000 7efc8ffff700 1 -- 172.21.15.146:0/1180071985 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc70005310 con 0x7efc90150230 2024-09-13T18:45:58.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.105+0000 7efc777fe700 1 -- 172.21.15.146:0/1180071985 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efc840038e0 con 0x7efc90150230 2024-09-13T18:45:58.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.105+0000 7efc777fe700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc7c066760 0x7efc7c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:45:58.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.105+0000 7efc777fe700 1 -- 172.21.15.146:0/1180071985 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7efc84083eb0 con 0x7efc90150230 2024-09-13T18:45:58.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.105+0000 7efc8effd700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc7c066760 0x7efc7c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:45:58.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.109+0000 7efc8effd700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc7c066760 0x7efc7c068c20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efc80013040 tx=0x7efc80010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:45:58.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.113+0000 7efc777fe700 1 -- 172.21.15.146:0/1180071985 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efc840509a0 con 0x7efc90150230 2024-09-13T18:45:58.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc8ffff700 1 -- 172.21.15.146:0/1180071985 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc700055c0 con 0x7efc90150230 2024-09-13T18:45:58.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc777fe700 1 -- 172.21.15.146:0/1180071985 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5515 (secure 0 0 0) 0x7efc84017090 con 0x7efc90150230 2024-09-13T18:45:58.367 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:45:58.367 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:45:55.808966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:45:58.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc8ffff700 1 -- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc7c066760 msgr2=0x7efc7c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:58.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc8ffff700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc7c066760 0x7efc7c068c20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efc80013040 tx=0x7efc80010000 comp rx=0 tx=0).stop 2024-09-13T18:45:58.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc8ffff700 1 -- 172.21.15.146:0/1180071985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc90150230 msgr2=0x7efc90157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:45:58.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc8ffff700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc90150230 0x7efc90157110 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7efc8400c520 tx=0x7efc8400c8e0 comp rx=0 tx=0).stop 2024-09-13T18:45:58.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc8ffff700 1 -- 172.21.15.146:0/1180071985 shutdown_connections 2024-09-13T18:45:58.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc8ffff700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc7c066760 0x7efc7c068c20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:58.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc8ffff700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc90155de0 0x7efc9013e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:58.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc8ffff700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc90150230 0x7efc90157110 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:58.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc8ffff700 1 --2- 172.21.15.146:0/1180071985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc90071610 0x7efc9014f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:45:58.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc8ffff700 1 -- 172.21.15.146:0/1180071985 >> 172.21.15.146:0/1180071985 conn(0x7efc90071ed0 msgr2=0x7efc9006d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:45:58.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc8ffff700 1 -- 172.21.15.146:0/1180071985 shutdown_connections 2024-09-13T18:45:58.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:45:58.349+0000 7efc8ffff700 1 -- 172.21.15.146:0/1180071985 wait complete. 2024-09-13T18:45:58.370 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 80 2024-09-13T18:45:58.381 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 80, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 80, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6370": { "addr": "172.21.15.154:6839/925899756", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 925899756, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 925899756, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6370, "incarnation": 80, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:replay", "state_seq": 1 }, "gid_6459": { "addr": "172.21.15.146:6835/294014284", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 294014284, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 294014284, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6459, "incarnation": 74, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 13 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:45:55.808966+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6459, "mds_1": 6370 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.146:6833/3023779168", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3023779168, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3023779168, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 69, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6187, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:46:00.383 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:46:00.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.473+0000 7f809359e700 1 Processor -- start 2024-09-13T18:46:00.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.473+0000 7f809359e700 1 -- start start 2024-09-13T18:46:00.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80940cf170 0x7f80941595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:00.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f80940cf550 0x7f80940c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:00.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809359e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809415a540 con 0x7f80940cb5b0 2024-09-13T18:46:00.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80940c5220 con 0x7f80940cf550 2024-09-13T18:46:00.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8094159ca0 con 0x7f80940cf170 2024-09-13T18:46:00.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80940cf170 0x7f80941595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:00.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80940cf170 0x7f80941595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:40656/0 (socket says 172.21.15.146:40656) 2024-09-13T18:46:00.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f8092d9d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f80940cb5b0 0x7f80941534c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:43486/0 (socket says 172.21.15.146:43486) 2024-09-13T18:46:00.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f8091d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f80940cf550 0x7f80940c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:00.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f8092d9d700 1 -- 172.21.15.146:0/933629621 learned_addr learned my addr 172.21.15.146:0/933629621 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:46:00.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809259c700 1 -- 172.21.15.146:0/933629621 >> v2:172.21.15.154:3300/0 conn(0x7f80940cf550 msgr2=0x7f80940c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:46:00.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809259c700 1 --2- 172.21.15.146:0/933629621 >> v2:172.21.15.154:3300/0 conn(0x7f80940cf550 0x7f80940c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:00.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809259c700 1 -- 172.21.15.146:0/933629621 >> v1:172.21.15.154:6789/0 conn(0x7f80940cb5b0 legacy=0x7f80941534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:00.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809259c700 1 -- 172.21.15.146:0/933629621 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094159f80 con 0x7f80940cf170 2024-09-13T18:46:00.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f8091d9b700 1 --2- 172.21.15.146:0/933629621 >> v2:172.21.15.154:3300/0 conn(0x7f80940cf550 0x7f80940c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:46:00.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809259c700 1 --2- 172.21.15.146:0/933629621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80940cf170 0x7f80941595f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8084009850 tx=0x7f8084009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40fec5372891702a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:00.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809159a700 1 -- 172.21.15.146:0/933629621 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084011830 con 0x7f80940cf170 2024-09-13T18:46:00.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809159a700 1 -- 172.21.15.146:0/933629621 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8084011990 con 0x7f80940cf170 2024-09-13T18:46:00.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809159a700 1 -- 172.21.15.146:0/933629621 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084011d40 con 0x7f80940cf170 2024-09-13T18:46:00.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809359e700 1 -- 172.21.15.146:0/933629621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80940cf170 msgr2=0x7f80941595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:00.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809359e700 1 --2- 172.21.15.146:0/933629621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80940cf170 0x7f80941595f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8084009850 tx=0x7f8084009b60 comp rx=0 tx=0).stop 2024-09-13T18:46:00.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809359e700 1 -- 172.21.15.146:0/933629621 shutdown_connections 2024-09-13T18:46:00.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809359e700 1 --2- 172.21.15.146:0/933629621 >> v2:172.21.15.154:3300/0 conn(0x7f80940cf550 0x7f80940c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:00.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809359e700 1 --2- 172.21.15.146:0/933629621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80940cf170 0x7f80941595f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:00.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809359e700 1 -- 172.21.15.146:0/933629621 >> 172.21.15.146:0/933629621 conn(0x7f8094072220 msgr2=0x7f8094066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:00.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809359e700 1 -- 172.21.15.146:0/933629621 shutdown_connections 2024-09-13T18:46:00.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.477+0000 7f809359e700 1 -- 172.21.15.146:0/933629621 wait complete. 2024-09-13T18:46:00.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f809359e700 1 Processor -- start 2024-09-13T18:46:00.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f809359e700 1 -- start start 2024-09-13T18:46:00.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f809359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80940cb5b0 0x7f8094158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:00.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f809359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80940cf170 0x7f80940ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:00.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f809359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80940cf550 0x7f80941eed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:00.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f809359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80941e4a00 con 0x7f80940cf550 2024-09-13T18:46:00.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f809359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80941e5730 con 0x7f80940cf170 2024-09-13T18:46:00.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f809359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8094159180 con 0x7f80940cb5b0 2024-09-13T18:46:00.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f8092d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80940cf550 0x7f80941eed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:00.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f8092d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80940cf550 0x7f80941eed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53040/0 (socket says 172.21.15.146:53040) 2024-09-13T18:46:00.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f8092d9d700 1 -- 172.21.15.146:0/888141971 learned_addr learned my addr 172.21.15.146:0/888141971 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:00.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f8091d9b700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80940cf170 0x7f80940ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:00.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f809259c700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80940cb5b0 0x7f8094158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:00.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f8092d9d700 1 -- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80940cb5b0 msgr2=0x7f8094158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:00.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f8092d9d700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80940cb5b0 0x7f8094158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:00.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f8092d9d700 1 -- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80940cf170 msgr2=0x7f80940ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:00.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f8092d9d700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80940cf170 0x7f80940ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:00.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f8092d9d700 1 -- 172.21.15.146:0/888141971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80880094b0 con 0x7f80940cf550 2024-09-13T18:46:00.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f8092d9d700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80940cf550 0x7f80941eed40 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f808c00caa0 tx=0x7f808c00cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:00.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f8082ffd700 1 -- 172.21.15.146:0/888141971 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c007480 con 0x7f80940cf550 2024-09-13T18:46:00.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f809359e700 1 -- 172.21.15.146:0/888141971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80840094b0 con 0x7f80940cf550 2024-09-13T18:46:00.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f809359e700 1 -- 172.21.15.146:0/888141971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80941e4c50 con 0x7f80940cf550 2024-09-13T18:46:00.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f8082ffd700 1 -- 172.21.15.146:0/888141971 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808c0075e0 con 0x7f80940cf550 2024-09-13T18:46:00.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.481+0000 7f8082ffd700 1 -- 172.21.15.146:0/888141971 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c007950 con 0x7f80940cf550 2024-09-13T18:46:00.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.485+0000 7f809359e700 1 -- 172.21.15.146:0/888141971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8078005310 con 0x7f80940cf550 2024-09-13T18:46:00.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.485+0000 7f8082ffd700 1 -- 172.21.15.146:0/888141971 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f808c012400 con 0x7f80940cf550 2024-09-13T18:46:00.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.485+0000 7f8082ffd700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8074066760 0x7f8074068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:00.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.485+0000 7f8082ffd700 1 -- 172.21.15.146:0/888141971 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7f808c083f00 con 0x7f80940cf550 2024-09-13T18:46:00.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.485+0000 7f809259c700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8074066760 0x7f8074068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:00.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.489+0000 7f809259c700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8074066760 0x7f8074068c20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f80840090b0 tx=0x7f8084008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:00.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.493+0000 7f8082ffd700 1 -- 172.21.15.146:0/888141971 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f808c054650 con 0x7f80940cf550 2024-09-13T18:46:00.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.733+0000 7f809359e700 1 -- 172.21.15.146:0/888141971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8078005180 con 0x7f80940cf550 2024-09-13T18:46:00.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.733+0000 7f8082ffd700 1 -- 172.21.15.146:0/888141971 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5515 (secure 0 0 0) 0x7f808c001b10 con 0x7f80940cf550 2024-09-13T18:46:00.751 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:46:00.751 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:45:55.808966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:46:00.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.737+0000 7f809359e700 1 -- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8074066760 msgr2=0x7f8074068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:00.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.737+0000 7f809359e700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8074066760 0x7f8074068c20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f80840090b0 tx=0x7f8084008030 comp rx=0 tx=0).stop 2024-09-13T18:46:00.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.737+0000 7f809359e700 1 -- 172.21.15.146:0/888141971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80940cf550 msgr2=0x7f80941eed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:00.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.737+0000 7f809359e700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80940cf550 0x7f80941eed40 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f808c00caa0 tx=0x7f808c00cad0 comp rx=0 tx=0).stop 2024-09-13T18:46:00.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.737+0000 7f809359e700 1 -- 172.21.15.146:0/888141971 shutdown_connections 2024-09-13T18:46:00.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.737+0000 7f809359e700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8074066760 0x7f8074068c20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:00.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.737+0000 7f809359e700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80940cf550 0x7f80941eed40 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:00.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.737+0000 7f809359e700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80940cf170 0x7f80940ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:00.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.737+0000 7f809359e700 1 --2- 172.21.15.146:0/888141971 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80940cb5b0 0x7f8094158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:00.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.737+0000 7f809359e700 1 -- 172.21.15.146:0/888141971 >> 172.21.15.146:0/888141971 conn(0x7f8094072220 msgr2=0x7f8094153290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:00.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.737+0000 7f809359e700 1 -- 172.21.15.146:0/888141971 shutdown_connections 2024-09-13T18:46:00.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:00.737+0000 7f809359e700 1 -- 172.21.15.146:0/888141971 wait complete. 2024-09-13T18:46:00.755 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 80 2024-09-13T18:46:00.769 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:46:02.769 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:46:02.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd573208700 1 Processor -- start 2024-09-13T18:46:02.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd573208700 1 -- start start 2024-09-13T18:46:02.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd573208700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd56c0c82b0 0x7fd56c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:02.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd573208700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd56c0c8690 0x7fd56c155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:02.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd573208700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd56c06cef0 0x7fd56c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:02.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd573208700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56c15a0b0 con 0x7fd56c06cef0 2024-09-13T18:46:02.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd573208700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56c159120 con 0x7fd56c0c82b0 2024-09-13T18:46:02.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd573208700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c1596e0 con 0x7fd56c0c8690 2024-09-13T18:46:02.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd572a07700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd56c06cef0 0x7fd56c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:02.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd572a07700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd56c06cef0 0x7fd56c0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60226/0 (socket says 172.21.15.146:60226) 2024-09-13T18:46:02.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd572a07700 1 -- 172.21.15.146:0/973921409 learned_addr learned my addr 172.21.15.146:0/973921409 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:02.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd572206700 1 --2- 172.21.15.146:0/973921409 >> v2:172.21.15.154:3300/0 conn(0x7fd56c0c82b0 0x7fd56c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:02.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd571a05700 1 --2- 172.21.15.146:0/973921409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd56c0c8690 0x7fd56c155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:02.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd572a07700 1 -- 172.21.15.146:0/973921409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd56c0c8690 msgr2=0x7fd56c155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:46:02.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd572a07700 1 --2- 172.21.15.146:0/973921409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd56c0c8690 0x7fd56c155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:02.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd572a07700 1 -- 172.21.15.146:0/973921409 >> v2:172.21.15.154:3300/0 conn(0x7fd56c0c82b0 msgr2=0x7fd56c159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:46:02.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd572a07700 1 --2- 172.21.15.146:0/973921409 >> v2:172.21.15.154:3300/0 conn(0x7fd56c0c82b0 0x7fd56c159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:02.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd572a07700 1 -- 172.21.15.146:0/973921409 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c1563d0 con 0x7fd56c06cef0 2024-09-13T18:46:02.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd572a07700 1 --2- 172.21.15.146:0/973921409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd56c06cef0 0x7fd56c0c7760 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0c8f80 tx=0x7fd56400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d892566c7c89ef80 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:02.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd571204700 1 -- 172.21.15.146:0/973921409 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56400e070 con 0x7fd56c06cef0 2024-09-13T18:46:02.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd571204700 1 -- 172.21.15.146:0/973921409 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56400b910 con 0x7fd56c06cef0 2024-09-13T18:46:02.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.873+0000 7fd571204700 1 -- 172.21.15.146:0/973921409 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56400bcc0 con 0x7fd56c06cef0 2024-09-13T18:46:02.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.877+0000 7fd573208700 1 -- 172.21.15.146:0/973921409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd56c06cef0 msgr2=0x7fd56c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:02.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.877+0000 7fd573208700 1 --2- 172.21.15.146:0/973921409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd56c06cef0 0x7fd56c0c7760 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0c8f80 tx=0x7fd56400b330 comp rx=0 tx=0).stop 2024-09-13T18:46:02.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.877+0000 7fd573208700 1 -- 172.21.15.146:0/973921409 shutdown_connections 2024-09-13T18:46:02.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.877+0000 7fd573208700 1 --2- 172.21.15.146:0/973921409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd56c0c8690 0x7fd56c155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:02.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.877+0000 7fd573208700 1 --2- 172.21.15.146:0/973921409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd56c06cef0 0x7fd56c0c7760 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:02.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.877+0000 7fd573208700 1 --2- 172.21.15.146:0/973921409 >> v2:172.21.15.154:3300/0 conn(0x7fd56c0c82b0 0x7fd56c159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:02.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.877+0000 7fd573208700 1 -- 172.21.15.146:0/973921409 >> 172.21.15.146:0/973921409 conn(0x7fd56c072270 msgr2=0x7fd56c06ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:02.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.877+0000 7fd573208700 1 -- 172.21.15.146:0/973921409 shutdown_connections 2024-09-13T18:46:02.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.877+0000 7fd573208700 1 -- 172.21.15.146:0/973921409 wait complete. 2024-09-13T18:46:02.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.877+0000 7fd573208700 1 Processor -- start 2024-09-13T18:46:02.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.877+0000 7fd573208700 1 -- start start 2024-09-13T18:46:02.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd573208700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd56c06cef0 0x7fd56c158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:02.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd573208700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd56c0c82b0 0x7fd56c14fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:02.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd573208700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd56c0c8690 0x7fd56c151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:02.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd573208700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c0c44c0 con 0x7fd56c0c8690 2024-09-13T18:46:02.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd573208700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c150be0 con 0x7fd56c0c82b0 2024-09-13T18:46:02.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd573208700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c1508b0 con 0x7fd56c06cef0 2024-09-13T18:46:02.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd572a07700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd56c0c8690 0x7fd56c151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:02.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd572a07700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd56c0c8690 0x7fd56c151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60228/0 (socket says 172.21.15.146:60228) 2024-09-13T18:46:02.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd572a07700 1 -- 172.21.15.146:0/2662101987 learned_addr learned my addr 172.21.15.146:0/2662101987 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:02.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd572206700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd56c06cef0 0x7fd56c158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:02.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd571a05700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd56c0c82b0 0x7fd56c14fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:02.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd572a07700 1 -- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd56c06cef0 msgr2=0x7fd56c158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:02.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd572a07700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd56c06cef0 0x7fd56c158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:02.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd572a07700 1 -- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd56c0c82b0 msgr2=0x7fd56c14fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:02.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd572a07700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd56c0c82b0 0x7fd56c14fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:02.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd572a07700 1 -- 172.21.15.146:0/2662101987 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd564009b30 con 0x7fd56c0c8690 2024-09-13T18:46:02.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd572a07700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd56c0c8690 0x7fd56c151070 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0707d0 tx=0x7fd564004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:02.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd562ffd700 1 -- 172.21.15.146:0/2662101987 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5640099b0 con 0x7fd56c0c8690 2024-09-13T18:46:02.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd562ffd700 1 -- 172.21.15.146:0/2662101987 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56400ce80 con 0x7fd56c0c8690 2024-09-13T18:46:02.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd573208700 1 -- 172.21.15.146:0/2662101987 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c0c5790 con 0x7fd56c0c8690 2024-09-13T18:46:02.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd562ffd700 1 -- 172.21.15.146:0/2662101987 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564004490 con 0x7fd56c0c8690 2024-09-13T18:46:02.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.881+0000 7fd573208700 1 -- 172.21.15.146:0/2662101987 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56c0c49b0 con 0x7fd56c0c8690 2024-09-13T18:46:02.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.885+0000 7fd562ffd700 1 -- 172.21.15.146:0/2662101987 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd5640045f0 con 0x7fd56c0c8690 2024-09-13T18:46:02.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.885+0000 7fd573208700 1 -- 172.21.15.146:0/2662101987 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56c150d90 con 0x7fd56c0c8690 2024-09-13T18:46:02.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.885+0000 7fd562ffd700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd550066790 0x7fd550068c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:02.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.885+0000 7fd562ffd700 1 -- 172.21.15.146:0/2662101987 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7fd564085ab0 con 0x7fd56c0c8690 2024-09-13T18:46:02.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.885+0000 7fd572206700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd550066790 0x7fd550068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:02.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.885+0000 7fd572206700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd550066790 0x7fd550068c50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd568009bf0 tx=0x7fd568009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:02.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:02.893+0000 7fd562ffd700 1 -- 172.21.15.146:0/2662101987 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd56c150d90 con 0x7fd56c0c8690 2024-09-13T18:46:03.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.129+0000 7fd573208700 1 -- 172.21.15.146:0/2662101987 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd56c0c3210 con 0x7fd56c0c8690 2024-09-13T18:46:03.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.129+0000 7fd562ffd700 1 -- 172.21.15.146:0/2662101987 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5517 (secure 0 0 0) 0x7fd564020090 con 0x7fd56c0c8690 2024-09-13T18:46:03.149 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:46:03.149 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:01.457026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:resolve","state_seq":17,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:46:03.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.133+0000 7fd573208700 1 -- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd550066790 msgr2=0x7fd550068c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:03.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.133+0000 7fd573208700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd550066790 0x7fd550068c50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd568009bf0 tx=0x7fd568009240 comp rx=0 tx=0).stop 2024-09-13T18:46:03.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.133+0000 7fd573208700 1 -- 172.21.15.146:0/2662101987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd56c0c8690 msgr2=0x7fd56c151070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:03.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.133+0000 7fd573208700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd56c0c8690 0x7fd56c151070 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0707d0 tx=0x7fd564004af0 comp rx=0 tx=0).stop 2024-09-13T18:46:03.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.133+0000 7fd573208700 1 -- 172.21.15.146:0/2662101987 shutdown_connections 2024-09-13T18:46:03.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.133+0000 7fd573208700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd550066790 0x7fd550068c50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:03.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.133+0000 7fd573208700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd56c0c8690 0x7fd56c151070 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:03.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.133+0000 7fd573208700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd56c0c82b0 0x7fd56c14fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:03.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.133+0000 7fd573208700 1 --2- 172.21.15.146:0/2662101987 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd56c06cef0 0x7fd56c158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:03.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.133+0000 7fd573208700 1 -- 172.21.15.146:0/2662101987 >> 172.21.15.146:0/2662101987 conn(0x7fd56c072270 msgr2=0x7fd56c14d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:03.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.133+0000 7fd573208700 1 -- 172.21.15.146:0/2662101987 shutdown_connections 2024-09-13T18:46:03.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:03.133+0000 7fd573208700 1 -- 172.21.15.146:0/2662101987 wait complete. 2024-09-13T18:46:03.153 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 81 2024-09-13T18:46:03.169 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:46:05.169 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:46:05.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 Processor -- start 2024-09-13T18:46:05.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 -- start start 2024-09-13T18:46:05.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fde8c070940 0x7fde8c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:05.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fde8c06bb10 0x7fde8c070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:05.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8c15da60 con 0x7fde8c071260 2024-09-13T18:46:05.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8c15a530 con 0x7fde8c070940 2024-09-13T18:46:05.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8c15cc00 con 0x7fde8c06bb10 2024-09-13T18:46:05.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde90c6c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fde8c070940 0x7fde8c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:05.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde9146d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fde8c071260 0x7fde8c1530d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34956/0 (socket says 172.21.15.146:34956) 2024-09-13T18:46:05.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde9146d700 1 -- 172.21.15.146:0/485970834 learned_addr learned my addr 172.21.15.146:0/485970834 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:46:05.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde8bfff700 1 --2- 172.21.15.146:0/485970834 >> v2:172.21.15.154:3300/0 conn(0x7fde8c06bb10 0x7fde8c070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:05.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde90c6c700 1 -- 172.21.15.146:0/485970834 >> v2:172.21.15.154:3300/0 conn(0x7fde8c06bb10 msgr2=0x7fde8c070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:46:05.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde90c6c700 1 --2- 172.21.15.146:0/485970834 >> v2:172.21.15.154:3300/0 conn(0x7fde8c06bb10 0x7fde8c070d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:05.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde90c6c700 1 -- 172.21.15.146:0/485970834 >> v1:172.21.15.146:6789/0 conn(0x7fde8c071260 legacy=0x7fde8c1530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:05.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde90c6c700 1 -- 172.21.15.146:0/485970834 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c15cee0 con 0x7fde8c070940 2024-09-13T18:46:05.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde8b7fe700 1 -- 172.21.15.146:0/485970834 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414286925 0 0) 0x7fde8c15da60 con 0x7fde8c071260 2024-09-13T18:46:05.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde8bfff700 1 --2- 172.21.15.146:0/485970834 >> v2:172.21.15.154:3300/0 conn(0x7fde8c06bb10 0x7fde8c070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:46:05.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde90c6c700 1 --2- 172.21.15.146:0/485970834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8c070940 0x7fde8c159ff0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0c7780 tx=0x7fde8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f49f9e06666308 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:05.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde8b7fe700 1 -- 172.21.15.146:0/485970834 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8400e070 con 0x7fde8c070940 2024-09-13T18:46:05.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde8b7fe700 1 -- 172.21.15.146:0/485970834 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8400bbd0 con 0x7fde8c070940 2024-09-13T18:46:05.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde8b7fe700 1 -- 172.21.15.146:0/485970834 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84012430 con 0x7fde8c070940 2024-09-13T18:46:05.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 -- 172.21.15.146:0/485970834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8c070940 msgr2=0x7fde8c159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:05.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 --2- 172.21.15.146:0/485970834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8c070940 0x7fde8c159ff0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0c7780 tx=0x7fde8400b330 comp rx=0 tx=0).stop 2024-09-13T18:46:05.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 -- 172.21.15.146:0/485970834 shutdown_connections 2024-09-13T18:46:05.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 --2- 172.21.15.146:0/485970834 >> v2:172.21.15.154:3300/0 conn(0x7fde8c06bb10 0x7fde8c070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:05.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 --2- 172.21.15.146:0/485970834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8c070940 0x7fde8c159ff0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:05.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 -- 172.21.15.146:0/485970834 >> 172.21.15.146:0/485970834 conn(0x7fde8c06c2d0 msgr2=0x7fde8c06c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:05.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 -- 172.21.15.146:0/485970834 shutdown_connections 2024-09-13T18:46:05.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.285+0000 7fde91c6e700 1 -- 172.21.15.146:0/485970834 wait complete. 2024-09-13T18:46:05.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde91c6e700 1 Processor -- start 2024-09-13T18:46:05.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde91c6e700 1 -- start start 2024-09-13T18:46:05.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde91c6e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde8c06bb10 0x7fde8c1e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:05.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde91c6e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8c070940 0x7fde8c0af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:05.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde91c6e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde8c071260 0x7fde8c1f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:05.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde91c6e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0c8660 con 0x7fde8c070940 2024-09-13T18:46:05.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde91c6e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c1f2970 con 0x7fde8c071260 2024-09-13T18:46:05.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde91c6e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0c87f0 con 0x7fde8c06bb10 2024-09-13T18:46:05.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde8bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8c070940 0x7fde8c0af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:05.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde8bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8c070940 0x7fde8c0af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60242/0 (socket says 172.21.15.146:60242) 2024-09-13T18:46:05.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde8bfff700 1 -- 172.21.15.146:0/3820566296 learned_addr learned my addr 172.21.15.146:0/3820566296 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:05.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde9146d700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde8c071260 0x7fde8c1f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:05.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde90c6c700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde8c06bb10 0x7fde8c1e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:05.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde8bfff700 1 -- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde8c06bb10 msgr2=0x7fde8c1e47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:05.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde8bfff700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde8c06bb10 0x7fde8c1e47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:05.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde8bfff700 1 -- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde8c071260 msgr2=0x7fde8c1f2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:05.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde8bfff700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde8c071260 0x7fde8c1f2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:05.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde8bfff700 1 -- 172.21.15.146:0/3820566296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde84009a70 con 0x7fde8c070940 2024-09-13T18:46:05.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde90c6c700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde8c06bb10 0x7fde8c1e47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:46:05.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde8bfff700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8c070940 0x7fde8c0af030 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fde7400e760 tx=0x7fde7400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:05.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde897fa700 1 -- 172.21.15.146:0/3820566296 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74004230 con 0x7fde8c070940 2024-09-13T18:46:05.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde897fa700 1 -- 172.21.15.146:0/3820566296 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde740043b0 con 0x7fde8c070940 2024-09-13T18:46:05.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde91c6e700 1 -- 172.21.15.146:0/3820566296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8c0c02d0 con 0x7fde8c070940 2024-09-13T18:46:05.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde897fa700 1 -- 172.21.15.146:0/3820566296 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde740036c0 con 0x7fde8c070940 2024-09-13T18:46:05.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.289+0000 7fde91c6e700 1 -- 172.21.15.146:0/3820566296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8c0c8a00 con 0x7fde8c070940 2024-09-13T18:46:05.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.293+0000 7fde91c6e700 1 -- 172.21.15.146:0/3820566296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8c0c8130 con 0x7fde8c070940 2024-09-13T18:46:05.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.293+0000 7fde897fa700 1 -- 172.21.15.146:0/3820566296 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fde74003820 con 0x7fde8c070940 2024-09-13T18:46:05.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.293+0000 7fde897fa700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde6c066730 0x7fde6c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:05.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.293+0000 7fde897fa700 1 -- 172.21.15.146:0/3820566296 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7fde740843a0 con 0x7fde8c070940 2024-09-13T18:46:05.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.293+0000 7fde90c6c700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde6c066730 0x7fde6c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:05.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.293+0000 7fde90c6c700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde6c066730 0x7fde6c068bf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fde8400b370 tx=0x7fde84011000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:05.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.297+0000 7fde897fa700 1 -- 172.21.15.146:0/3820566296 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fde74051030 con 0x7fde8c070940 2024-09-13T18:46:05.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.537+0000 7fde91c6e700 1 -- 172.21.15.146:0/3820566296 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde8c070d20 con 0x7fde8c070940 2024-09-13T18:46:05.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.537+0000 7fde897fa700 1 -- 172.21.15.146:0/3820566296 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5517 (secure 0 0 0) 0x7fde74054650 con 0x7fde8c070940 2024-09-13T18:46:05.556 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:46:05.557 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:01.457026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:resolve","state_seq":17,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:46:05.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.541+0000 7fde91c6e700 1 -- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde6c066730 msgr2=0x7fde6c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:05.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.541+0000 7fde91c6e700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde6c066730 0x7fde6c068bf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fde8400b370 tx=0x7fde84011000 comp rx=0 tx=0).stop 2024-09-13T18:46:05.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.541+0000 7fde91c6e700 1 -- 172.21.15.146:0/3820566296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8c070940 msgr2=0x7fde8c0af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:05.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.541+0000 7fde91c6e700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8c070940 0x7fde8c0af030 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fde7400e760 tx=0x7fde7400eb20 comp rx=0 tx=0).stop 2024-09-13T18:46:05.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.541+0000 7fde91c6e700 1 -- 172.21.15.146:0/3820566296 shutdown_connections 2024-09-13T18:46:05.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.541+0000 7fde91c6e700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde6c066730 0x7fde6c068bf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:05.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.541+0000 7fde91c6e700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde8c071260 0x7fde8c1f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:05.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.541+0000 7fde91c6e700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8c070940 0x7fde8c0af030 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:05.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.541+0000 7fde91c6e700 1 --2- 172.21.15.146:0/3820566296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde8c06bb10 0x7fde8c1e47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:05.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.541+0000 7fde91c6e700 1 -- 172.21.15.146:0/3820566296 >> 172.21.15.146:0/3820566296 conn(0x7fde8c06c2d0 msgr2=0x7fde8c15b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:05.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.541+0000 7fde91c6e700 1 -- 172.21.15.146:0/3820566296 shutdown_connections 2024-09-13T18:46:05.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:05.541+0000 7fde91c6e700 1 -- 172.21.15.146:0/3820566296 wait complete. 2024-09-13T18:46:05.561 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 81 2024-09-13T18:46:05.573 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:46:07.574 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:46:07.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.657+0000 7fa93e43e700 1 Processor -- start 2024-09-13T18:46:07.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.657+0000 7fa93e43e700 1 -- start start 2024-09-13T18:46:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.657+0000 7fa93e43e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9380ca210 0x7fa9380b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.657+0000 7fa93e43e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa938071390 0x7fa9380cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.657+0000 7fa93e43e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9380cc670 con 0x7fa9380cbd50 2024-09-13T18:46:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.657+0000 7fa93e43e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9380caa30 con 0x7fa938071390 2024-09-13T18:46:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.657+0000 7fa93e43e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380cb810 con 0x7fa9380ca210 2024-09-13T18:46:07.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93cc3b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9380ca210 0x7fa9380b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93dc3d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa938071390 0x7fa9380cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93cc3b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9380ca210 0x7fa9380b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:45448/0 (socket says 172.21.15.146:45448) 2024-09-13T18:46:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93cc3b700 1 -- 172.21.15.146:0/493932168 learned_addr learned my addr 172.21.15.146:0/493932168 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93cc3b700 1 -- 172.21.15.146:0/493932168 >> v2:172.21.15.154:3300/0 conn(0x7fa938071390 msgr2=0x7fa9380cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:46:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93cc3b700 1 --2- 172.21.15.146:0/493932168 >> v2:172.21.15.154:3300/0 conn(0x7fa938071390 0x7fa9380cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93cc3b700 1 -- 172.21.15.146:0/493932168 >> v1:172.21.15.154:6789/0 conn(0x7fa9380cbd50 legacy=0x7fa9380701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93cc3b700 1 -- 172.21.15.146:0/493932168 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93815da20 con 0x7fa9380ca210 2024-09-13T18:46:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93dc3d700 1 --2- 172.21.15.146:0/493932168 >> v2:172.21.15.154:3300/0 conn(0x7fa938071390 0x7fa9380cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:46:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93cc3b700 1 --2- 172.21.15.146:0/493932168 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9380ca210 0x7fa9380b1c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa928009910 tx=0x7fa928009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ada438423a44c861 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa92ffff700 1 -- 172.21.15.146:0/493932168 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928011830 con 0x7fa9380ca210 2024-09-13T18:46:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa92ffff700 1 -- 172.21.15.146:0/493932168 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa928011990 con 0x7fa9380ca210 2024-09-13T18:46:07.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa92ffff700 1 -- 172.21.15.146:0/493932168 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928011d40 con 0x7fa9380ca210 2024-09-13T18:46:07.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93e43e700 1 -- 172.21.15.146:0/493932168 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9380ca210 msgr2=0x7fa9380b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:07.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93e43e700 1 --2- 172.21.15.146:0/493932168 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9380ca210 0x7fa9380b1c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa928009910 tx=0x7fa928009c20 comp rx=0 tx=0).stop 2024-09-13T18:46:07.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93e43e700 1 -- 172.21.15.146:0/493932168 shutdown_connections 2024-09-13T18:46:07.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93e43e700 1 --2- 172.21.15.146:0/493932168 >> v2:172.21.15.154:3300/0 conn(0x7fa938071390 0x7fa9380cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:07.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93e43e700 1 --2- 172.21.15.146:0/493932168 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9380ca210 0x7fa9380b1c40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:07.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93e43e700 1 -- 172.21.15.146:0/493932168 >> 172.21.15.146:0/493932168 conn(0x7fa938066270 msgr2=0x7fa938066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:07.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93e43e700 1 -- 172.21.15.146:0/493932168 shutdown_connections 2024-09-13T18:46:07.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.661+0000 7fa93e43e700 1 -- 172.21.15.146:0/493932168 wait complete. 2024-09-13T18:46:07.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93e43e700 1 Processor -- start 2024-09-13T18:46:07.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93e43e700 1 -- start start 2024-09-13T18:46:07.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93e43e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa938071390 0x7fa9381e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:07.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93e43e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9380ca210 0x7fa9380c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:07.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93e43e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9380cbd50 0x7fa93815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:07.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93e43e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381f3500 con 0x7fa9380ca210 2024-09-13T18:46:07.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93e43e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381e4640 con 0x7fa9380cbd50 2024-09-13T18:46:07.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93e43e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380c7a10 con 0x7fa938071390 2024-09-13T18:46:07.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93cc3b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9380ca210 0x7fa9380c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:07.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93cc3b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9380ca210 0x7fa9380c8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60244/0 (socket says 172.21.15.146:60244) 2024-09-13T18:46:07.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93cc3b700 1 -- 172.21.15.146:0/308557716 learned_addr learned my addr 172.21.15.146:0/308557716 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:07.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93dc3d700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9380cbd50 0x7fa93815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:07.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93d43c700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa938071390 0x7fa9381e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:07.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93cc3b700 1 -- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa938071390 msgr2=0x7fa9381e47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:07.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93cc3b700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa938071390 0x7fa9381e47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:07.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93cc3b700 1 -- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9380cbd50 msgr2=0x7fa93815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:07.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93cc3b700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9380cbd50 0x7fa93815b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:07.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93cc3b700 1 -- 172.21.15.146:0/308557716 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa930009a70 con 0x7fa9380ca210 2024-09-13T18:46:07.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93cc3b700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9380ca210 0x7fa9380c8420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa928011de0 tx=0x7fa928004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:07.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa92dffb700 1 -- 172.21.15.146:0/308557716 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92800b930 con 0x7fa9380ca210 2024-09-13T18:46:07.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa92dffb700 1 -- 172.21.15.146:0/308557716 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92801f070 con 0x7fa9380ca210 2024-09-13T18:46:07.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93e43e700 1 -- 172.21.15.146:0/308557716 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa928009570 con 0x7fa9380ca210 2024-09-13T18:46:07.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa92dffb700 1 -- 172.21.15.146:0/308557716 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92800bcf0 con 0x7fa9380ca210 2024-09-13T18:46:07.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.665+0000 7fa93e43e700 1 -- 172.21.15.146:0/308557716 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9381e4080 con 0x7fa9380ca210 2024-09-13T18:46:07.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.669+0000 7fa93e43e700 1 -- 172.21.15.146:0/308557716 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93815d500 con 0x7fa9380ca210 2024-09-13T18:46:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.669+0000 7fa92dffb700 1 -- 172.21.15.146:0/308557716 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa92801a430 con 0x7fa9380ca210 2024-09-13T18:46:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.669+0000 7fa92dffb700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa91c066730 0x7fa91c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.669+0000 7fa92dffb700 1 -- 172.21.15.146:0/308557716 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7fa928021070 con 0x7fa9380ca210 2024-09-13T18:46:07.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.669+0000 7fa93d43c700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa91c066730 0x7fa91c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:07.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.669+0000 7fa93d43c700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa91c066730 0x7fa91c068bf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa934000f60 tx=0x7fa934009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:07.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.677+0000 7fa92dffb700 1 -- 172.21.15.146:0/308557716 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa928052dd0 con 0x7fa9380ca210 2024-09-13T18:46:07.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.913+0000 7fa93e43e700 1 -- 172.21.15.146:0/308557716 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9380c9ff0 con 0x7fa9380ca210 2024-09-13T18:46:07.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.917+0000 7fa92dffb700 1 -- 172.21.15.146:0/308557716 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+5519 (secure 0 0 0) 0x7fa928015090 con 0x7fa9380ca210 2024-09-13T18:46:07.937 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:46:07.937 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:06.481808+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:reconnect","state_seq":19,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:46:07.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.917+0000 7fa93e43e700 1 -- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa91c066730 msgr2=0x7fa91c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:07.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.917+0000 7fa93e43e700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa91c066730 0x7fa91c068bf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa934000f60 tx=0x7fa934009040 comp rx=0 tx=0).stop 2024-09-13T18:46:07.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.917+0000 7fa93e43e700 1 -- 172.21.15.146:0/308557716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9380ca210 msgr2=0x7fa9380c8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:07.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.917+0000 7fa93e43e700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9380ca210 0x7fa9380c8420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa928011de0 tx=0x7fa928004770 comp rx=0 tx=0).stop 2024-09-13T18:46:07.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.917+0000 7fa93e43e700 1 -- 172.21.15.146:0/308557716 shutdown_connections 2024-09-13T18:46:07.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.917+0000 7fa93e43e700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa91c066730 0x7fa91c068bf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:07.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.917+0000 7fa93e43e700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9380cbd50 0x7fa93815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:07.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.917+0000 7fa93e43e700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9380ca210 0x7fa9380c8420 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:07.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.917+0000 7fa93e43e700 1 --2- 172.21.15.146:0/308557716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa938071390 0x7fa9381e47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:07.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.917+0000 7fa93e43e700 1 -- 172.21.15.146:0/308557716 >> 172.21.15.146:0/308557716 conn(0x7fa938066270 msgr2=0x7fa9380c45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:07.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.917+0000 7fa93e43e700 1 -- 172.21.15.146:0/308557716 shutdown_connections 2024-09-13T18:46:07.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:07.917+0000 7fa93e43e700 1 -- 172.21.15.146:0/308557716 wait complete. 2024-09-13T18:46:07.945 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 82 2024-09-13T18:46:07.952 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:46:09.953 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:46:10.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.046+0000 7faffffff700 1 Processor -- start 2024-09-13T18:46:10.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.046+0000 7faffffff700 1 -- start start 2024-09-13T18:46:10.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faff80d6720 0x7faff8046df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:10.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faff804d0f0 0x7faff80e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:10.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff80dd3a0 con 0x7faff80dc470 2024-09-13T18:46:10.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffffff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff80dcde0 con 0x7faff80d6720 2024-09-13T18:46:10.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff80dd0c0 con 0x7faff804d0f0 2024-09-13T18:46:10.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffeffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faff80d6720 0x7faff8046df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:10.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffeffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faff80d6720 0x7faff8046df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60260/0 (socket says 172.21.15.146:60260) 2024-09-13T18:46:10.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffeffd700 1 -- 172.21.15.146:0/3450262726 learned_addr learned my addr 172.21.15.146:0/3450262726 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:10.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7fafff7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7faff80dc470 0x7faff804de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34962/0 (socket says 172.21.15.146:34962) 2024-09-13T18:46:10.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffe7fc700 1 --2- 172.21.15.146:0/3450262726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faff804d0f0 0x7faff80e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:10.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffeffd700 1 -- 172.21.15.146:0/3450262726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faff804d0f0 msgr2=0x7faff80e0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:46:10.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffeffd700 1 --2- 172.21.15.146:0/3450262726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faff804d0f0 0x7faff80e0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:10.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffeffd700 1 -- 172.21.15.146:0/3450262726 >> v1:172.21.15.146:6789/0 conn(0x7faff80dc470 legacy=0x7faff804de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:10.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffeffd700 1 -- 172.21.15.146:0/3450262726 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff80dfaa0 con 0x7faff80d6720 2024-09-13T18:46:10.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffe7fc700 1 --2- 172.21.15.146:0/3450262726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faff804d0f0 0x7faff80e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:46:10.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffeffd700 1 --2- 172.21.15.146:0/3450262726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff80d6720 0x7faff8046df0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7faff0009910 tx=0x7faff0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3753af0bd15fe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:10.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffdffb700 1 -- 172.21.15.146:0/3450262726 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff000ba60 con 0x7faff80d6720 2024-09-13T18:46:10.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffdffb700 1 -- 172.21.15.146:0/3450262726 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff000bbc0 con 0x7faff80d6720 2024-09-13T18:46:10.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffdffb700 1 -- 172.21.15.146:0/3450262726 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff0009e60 con 0x7faff80d6720 2024-09-13T18:46:10.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffffff700 1 -- 172.21.15.146:0/3450262726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff80d6720 msgr2=0x7faff8046df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:10.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffffff700 1 --2- 172.21.15.146:0/3450262726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff80d6720 0x7faff8046df0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7faff0009910 tx=0x7faff0009c20 comp rx=0 tx=0).stop 2024-09-13T18:46:10.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffffff700 1 -- 172.21.15.146:0/3450262726 shutdown_connections 2024-09-13T18:46:10.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffffff700 1 --2- 172.21.15.146:0/3450262726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faff804d0f0 0x7faff80e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:10.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffffff700 1 --2- 172.21.15.146:0/3450262726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff80d6720 0x7faff8046df0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:10.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffffff700 1 -- 172.21.15.146:0/3450262726 >> 172.21.15.146:0/3450262726 conn(0x7faff800dc40 msgr2=0x7faff800e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:10.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffffff700 1 -- 172.21.15.146:0/3450262726 shutdown_connections 2024-09-13T18:46:10.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.050+0000 7faffffff700 1 -- 172.21.15.146:0/3450262726 wait complete. 2024-09-13T18:46:10.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7faffffff700 1 Processor -- start 2024-09-13T18:46:10.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7faffffff700 1 -- start start 2024-09-13T18:46:10.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7faffffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faff804d0f0 0x7faff80d5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:10.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7faffffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faff80d6720 0x7faff8167040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:10.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7faffffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff80dc470 0x7faff8167580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:10.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7faffffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff81669f0 con 0x7faff80dc470 2024-09-13T18:46:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7faffffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff8172070 con 0x7faff804d0f0 2024-09-13T18:46:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7faffffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff8172e90 con 0x7faff80d6720 2024-09-13T18:46:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7fafff7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff80dc470 0x7faff8167580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7fafff7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff80dc470 0x7faff8167580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60276/0 (socket says 172.21.15.146:60276) 2024-09-13T18:46:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7fafff7fe700 1 -- 172.21.15.146:0/3012197612 learned_addr learned my addr 172.21.15.146:0/3012197612 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7faffe7fc700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faff80d6720 0x7faff8167040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7faffeffd700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faff804d0f0 0x7faff80d5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7fafff7fe700 1 -- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faff80d6720 msgr2=0x7faff8167040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7fafff7fe700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faff80d6720 0x7faff8167040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7fafff7fe700 1 -- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faff804d0f0 msgr2=0x7faff80d5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7fafff7fe700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faff804d0f0 0x7faff80d5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7fafff7fe700 1 -- 172.21.15.146:0/3012197612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff0009570 con 0x7faff80dc470 2024-09-13T18:46:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7fafff7fe700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff80dc470 0x7faff8167580 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb000068650 tx=0x7fb000075d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:10.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7fafe7fff700 1 -- 172.21.15.146:0/3012197612 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00006f270 con 0x7faff80dc470 2024-09-13T18:46:10.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7fafe7fff700 1 -- 172.21.15.146:0/3012197612 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb000067c50 con 0x7faff80dc470 2024-09-13T18:46:10.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7faffffff700 1 -- 172.21.15.146:0/3012197612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff80cf310 con 0x7faff80dc470 2024-09-13T18:46:10.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7fafe7fff700 1 -- 172.21.15.146:0/3012197612 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000068020 con 0x7faff80dc470 2024-09-13T18:46:10.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.054+0000 7faffffff700 1 -- 172.21.15.146:0/3012197612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff8172560 con 0x7faff80dc470 2024-09-13T18:46:10.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.058+0000 7faffffff700 1 -- 172.21.15.146:0/3012197612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff8172fd0 con 0x7faff80dc470 2024-09-13T18:46:10.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.058+0000 7fafe7fff700 1 -- 172.21.15.146:0/3012197612 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb00006f7d0 con 0x7faff80dc470 2024-09-13T18:46:10.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.058+0000 7fafe7fff700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fafec066760 0x7fafec068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:10.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.058+0000 7fafe7fff700 1 -- 172.21.15.146:0/3012197612 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7fb0000febe0 con 0x7faff80dc470 2024-09-13T18:46:10.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.058+0000 7faffeffd700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fafec066760 0x7fafec068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:10.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.058+0000 7faffeffd700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fafec066760 0x7fafec068c20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7faff0009c60 tx=0x7faff0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:10.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.062+0000 7fafe7fff700 1 -- 172.21.15.146:0/3012197612 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faff8172fd0 con 0x7faff80dc470 2024-09-13T18:46:10.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7faffffff700 1 -- 172.21.15.146:0/3012197612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faff8172fd0 con 0x7faff80dc470 2024-09-13T18:46:10.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7fafe7fff700 1 -- 172.21.15.146:0/3012197612 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+5519 (secure 0 0 0) 0x7faff8172fd0 con 0x7faff80dc470 2024-09-13T18:46:10.306 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:46:10.306 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:06.481808+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:reconnect","state_seq":19,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:46:10.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7faffffff700 1 -- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fafec066760 msgr2=0x7fafec068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:10.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7faffffff700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fafec066760 0x7fafec068c20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7faff0009c60 tx=0x7faff0008030 comp rx=0 tx=0).stop 2024-09-13T18:46:10.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7faffffff700 1 -- 172.21.15.146:0/3012197612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff80dc470 msgr2=0x7faff8167580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:10.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7faffffff700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff80dc470 0x7faff8167580 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb000068650 tx=0x7fb000075d10 comp rx=0 tx=0).stop 2024-09-13T18:46:10.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7faffffff700 1 -- 172.21.15.146:0/3012197612 shutdown_connections 2024-09-13T18:46:10.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7faffffff700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fafec066760 0x7fafec068c20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:10.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7faffffff700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff80dc470 0x7faff8167580 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:10.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7faffffff700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faff80d6720 0x7faff8167040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:10.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7faffffff700 1 --2- 172.21.15.146:0/3012197612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faff804d0f0 0x7faff80d5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:10.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7faffffff700 1 -- 172.21.15.146:0/3012197612 >> 172.21.15.146:0/3012197612 conn(0x7faff800dc40 msgr2=0x7faff80db620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:10.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7faffffff700 1 -- 172.21.15.146:0/3012197612 shutdown_connections 2024-09-13T18:46:10.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:10.286+0000 7faffffff700 1 -- 172.21.15.146:0/3012197612 wait complete. 2024-09-13T18:46:10.311 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 82 2024-09-13T18:46:10.324 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 82, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 82, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6370": { "addr": "172.21.15.154:6839/925899756", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 925899756, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 925899756, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6370, "incarnation": 80, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:reconnect", "state_seq": 19 }, "gid_6459": { "addr": "172.21.15.146:6835/294014284", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 294014284, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 294014284, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6459, "incarnation": 74, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 13 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:46:06.481808+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6459, "mds_1": 6370 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.146:6833/3023779168", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3023779168, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3023779168, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 69, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6187, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:46:10.580 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:46:10.582 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:46:12.326 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:46:12.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.426+0000 7f4dc2786700 1 Processor -- start 2024-09-13T18:46:12.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.426+0000 7f4dc2786700 1 -- start start 2024-09-13T18:46:12.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc2786700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4dbc150230 0x7f4dbc06ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:12.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc2786700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dbc150610 0x7f4dbc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:12.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc2786700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dbc071410 con 0x7f4dbc071610 2024-09-13T18:46:12.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc2786700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dbc1596b0 con 0x7f4dbc150230 2024-09-13T18:46:12.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc2786700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc156090 con 0x7f4dbc150610 2024-09-13T18:46:12.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc1784700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4dbc150230 0x7f4dbc06ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:12.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc1784700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4dbc150230 0x7f4dbc06ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32842/0 (socket says 172.21.15.146:32842) 2024-09-13T18:46:12.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc1784700 1 -- 172.21.15.146:0/3301890758 learned_addr learned my addr 172.21.15.146:0/3301890758 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:12.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc0f83700 1 --2- 172.21.15.146:0/3301890758 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dbc150610 0x7f4dbc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:12.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc1784700 1 -- 172.21.15.146:0/3301890758 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dbc150610 msgr2=0x7f4dbc06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:46:12.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc1784700 1 --2- 172.21.15.146:0/3301890758 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dbc150610 0x7f4dbc06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:12.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc1784700 1 -- 172.21.15.146:0/3301890758 >> v1:172.21.15.154:6789/0 conn(0x7f4dbc071610 legacy=0x7f4dbc159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:12.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc1784700 1 -- 172.21.15.146:0/3301890758 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dbc156350 con 0x7f4dbc150230 2024-09-13T18:46:12.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc1784700 1 --2- 172.21.15.146:0/3301890758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dbc150230 0x7f4dbc06ebd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009910 tx=0x7f4db8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11c6ac43cdd33c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:12.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4db3fff700 1 -- 172.21.15.146:0/3301890758 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db800ba60 con 0x7f4dbc150230 2024-09-13T18:46:12.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4db3fff700 1 -- 172.21.15.146:0/3301890758 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db800bbc0 con 0x7f4dbc150230 2024-09-13T18:46:12.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4db3fff700 1 -- 172.21.15.146:0/3301890758 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8009e60 con 0x7f4dbc150230 2024-09-13T18:46:12.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc2786700 1 -- 172.21.15.146:0/3301890758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dbc150230 msgr2=0x7f4dbc06ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:12.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc2786700 1 --2- 172.21.15.146:0/3301890758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dbc150230 0x7f4dbc06ebd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009910 tx=0x7f4db8009c20 comp rx=0 tx=0).stop 2024-09-13T18:46:12.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc2786700 1 -- 172.21.15.146:0/3301890758 shutdown_connections 2024-09-13T18:46:12.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc2786700 1 --2- 172.21.15.146:0/3301890758 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dbc150610 0x7f4dbc06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:12.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc2786700 1 --2- 172.21.15.146:0/3301890758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dbc150230 0x7f4dbc06ebd0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:12.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc2786700 1 -- 172.21.15.146:0/3301890758 >> 172.21.15.146:0/3301890758 conn(0x7f4dbc071ed0 msgr2=0x7f4dbc0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:12.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc2786700 1 -- 172.21.15.146:0/3301890758 shutdown_connections 2024-09-13T18:46:12.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.430+0000 7f4dc2786700 1 -- 172.21.15.146:0/3301890758 wait complete. 2024-09-13T18:46:12.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc2786700 1 Processor -- start 2024-09-13T18:46:12.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc2786700 1 -- start start 2024-09-13T18:46:12.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc2786700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dbc071610 0x7f4dbc145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:12.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc2786700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dbc150230 0x7f4dbc0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:12.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc2786700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dbc153630 0x7f4dbc0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:12.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc2786700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc1ef060 con 0x7f4dbc150230 2024-09-13T18:46:12.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc2786700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc155e30 con 0x7f4dbc153630 2024-09-13T18:46:12.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc2786700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc1463a0 con 0x7f4dbc071610 2024-09-13T18:46:12.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc0f83700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dbc150230 0x7f4dbc0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:12.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc1f85700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dbc153630 0x7f4dbc0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:12.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc1784700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dbc071610 0x7f4dbc145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:12.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc0f83700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dbc150230 0x7f4dbc0cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32852/0 (socket says 172.21.15.146:32852) 2024-09-13T18:46:12.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc1784700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dbc071610 0x7f4dbc145e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:34080/0 (socket says 172.21.15.146:34080) 2024-09-13T18:46:12.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc0f83700 1 -- 172.21.15.146:0/3784199545 learned_addr learned my addr 172.21.15.146:0/3784199545 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:12.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc0f83700 1 -- 172.21.15.146:0/3784199545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dbc071610 msgr2=0x7f4dbc145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:12.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc0f83700 1 --2- 172.21.15.146:0/3784199545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dbc071610 0x7f4dbc145e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:12.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc0f83700 1 -- 172.21.15.146:0/3784199545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dbc153630 msgr2=0x7f4dbc0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:12.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc0f83700 1 --2- 172.21.15.146:0/3784199545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dbc153630 0x7f4dbc0cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:12.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc0f83700 1 -- 172.21.15.146:0/3784199545 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8009570 con 0x7f4dbc150230 2024-09-13T18:46:12.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc0f83700 1 --2- 172.21.15.146:0/3784199545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dbc150230 0x7f4dbc0cadb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00d800 tx=0x7f4dac00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:12.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4db1ffb700 1 -- 172.21.15.146:0/3784199545 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac0126b0 con 0x7f4dbc150230 2024-09-13T18:46:12.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4db1ffb700 1 -- 172.21.15.146:0/3784199545 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac012830 con 0x7f4dbc150230 2024-09-13T18:46:12.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc2786700 1 -- 172.21.15.146:0/3784199545 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dbc1586a0 con 0x7f4dbc150230 2024-09-13T18:46:12.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4db1ffb700 1 -- 172.21.15.146:0/3784199545 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac012be0 con 0x7f4dbc150230 2024-09-13T18:46:12.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.434+0000 7f4dc2786700 1 -- 172.21.15.146:0/3784199545 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dbc158f90 con 0x7f4dbc150230 2024-09-13T18:46:12.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.438+0000 7f4dc2786700 1 -- 172.21.15.146:0/3784199545 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9c005310 con 0x7f4dbc150230 2024-09-13T18:46:12.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.438+0000 7f4db1ffb700 1 -- 172.21.15.146:0/3784199545 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4dac012d40 con 0x7f4dbc150230 2024-09-13T18:46:12.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.438+0000 7f4db1ffb700 1 --2- 172.21.15.146:0/3784199545 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4da0066730 0x7f4da0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:12.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.438+0000 7f4db1ffb700 1 -- 172.21.15.146:0/3784199545 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7f4dac0848e0 con 0x7f4dbc150230 2024-09-13T18:46:12.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.438+0000 7f4dc1784700 1 --2- 172.21.15.146:0/3784199545 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4da0066730 0x7f4da0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:12.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.438+0000 7f4dc1784700 1 --2- 172.21.15.146:0/3784199545 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4da0066730 0x7f4da0068bf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009c60 tx=0x7f4db8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:12.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.446+0000 7f4db1ffb700 1 -- 172.21.15.146:0/3784199545 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4dac051570 con 0x7f4dbc150230 2024-09-13T18:46:12.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.658+0000 7f4dc2786700 1 -- 172.21.15.146:0/3784199545 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d9c0055c0 con 0x7f4dbc150230 2024-09-13T18:46:12.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.658+0000 7f4db1ffb700 1 -- 172.21.15.146:0/3784199545 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5516 (secure 0 0 0) 0x7f4dac054b90 con 0x7f4dbc150230 2024-09-13T18:46:12.681 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:46:12.681 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:11.821509+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:46:12.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.662+0000 7f4dc2786700 1 -- 172.21.15.146:0/3784199545 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4da0066730 msgr2=0x7f4da0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:12.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.662+0000 7f4dc2786700 1 --2- 172.21.15.146:0/3784199545 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4da0066730 0x7f4da0068bf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009c60 tx=0x7f4db8008030 comp rx=0 tx=0).stop 2024-09-13T18:46:12.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.662+0000 7f4dc2786700 1 -- 172.21.15.146:0/3784199545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dbc150230 msgr2=0x7f4dbc0cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:12.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.662+0000 7f4dc2786700 1 --2- 172.21.15.146:0/3784199545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dbc150230 0x7f4dbc0cadb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00d800 tx=0x7f4dac00db10 comp rx=0 tx=0).stop 2024-09-13T18:46:12.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.662+0000 7f4dc2786700 1 -- 172.21.15.146:0/3784199545 shutdown_connections 2024-09-13T18:46:12.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.662+0000 7f4dc2786700 1 --2- 172.21.15.146:0/3784199545 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4da0066730 0x7f4da0068bf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:12.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.662+0000 7f4dc2786700 1 --2- 172.21.15.146:0/3784199545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dbc153630 0x7f4dbc0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:12.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.662+0000 7f4dc2786700 1 --2- 172.21.15.146:0/3784199545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dbc150230 0x7f4dbc0cadb0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:12.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.662+0000 7f4dc2786700 1 --2- 172.21.15.146:0/3784199545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dbc071610 0x7f4dbc145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:12.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.662+0000 7f4dc2786700 1 -- 172.21.15.146:0/3784199545 >> 172.21.15.146:0/3784199545 conn(0x7f4dbc071ed0 msgr2=0x7f4dbc14c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:12.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.662+0000 7f4dc2786700 1 -- 172.21.15.146:0/3784199545 shutdown_connections 2024-09-13T18:46:12.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:12.662+0000 7f4dc2786700 1 -- 172.21.15.146:0/3784199545 wait complete. 2024-09-13T18:46:12.685 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 83 2024-09-13T18:46:12.700 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:46:14.701 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:46:14.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.798+0000 7f95dedc7700 1 Processor -- start 2024-09-13T18:46:14.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.798+0000 7f95dedc7700 1 -- start start 2024-09-13T18:46:14.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dedc7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95d8153bc0 0x7f95d80c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:14.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dedc7700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d8153fa0 con 0x7f95d80c6610 2024-09-13T18:46:14.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dedc7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d80aa500 con 0x7f95d80c6a50 2024-09-13T18:46:14.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dedc7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d815a7e0 con 0x7f95d8153bc0 2024-09-13T18:46:14.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dd5c4700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f95d80c6610 0x7f95d80cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:55444/0 (socket says 172.21.15.146:55444) 2024-09-13T18:46:14.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dd5c4700 1 -- 172.21.15.146:0/2710104558 learned_addr learned my addr 172.21.15.146:0/2710104558 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:46:14.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dddc5700 1 --2- 172.21.15.146:0/2710104558 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95d8153bc0 0x7f95d80c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:14.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dcdc3700 1 -- 172.21.15.146:0/2710104558 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922032042 0 0) 0x7f95d8153fa0 con 0x7f95d80c6610 2024-09-13T18:46:14.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dcdc3700 1 -- 172.21.15.146:0/2710104558 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c40035d0 con 0x7f95d80c6610 2024-09-13T18:46:14.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dcdc3700 1 -- 172.21.15.146:0/2710104558 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515328732 0 0) 0x7f95d80aa500 con 0x7f95d80c6a50 2024-09-13T18:46:14.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dcdc3700 1 -- 172.21.15.146:0/2710104558 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d8153fa0 con 0x7f95d80c6a50 2024-09-13T18:46:14.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dcdc3700 1 -- 172.21.15.146:0/2710104558 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 817253268 0 0) 0x7f95c40035d0 con 0x7f95d80c6610 2024-09-13T18:46:14.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dddc5700 1 -- 172.21.15.146:0/2710104558 >> v1:172.21.15.154:6789/0 conn(0x7f95d80c6a50 legacy=0x7f95d8160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:14.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dddc5700 1 -- 172.21.15.146:0/2710104558 >> v1:172.21.15.146:6789/0 conn(0x7f95d80c6610 legacy=0x7f95d80cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:14.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dddc5700 1 -- 172.21.15.146:0/2710104558 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d80c9320 con 0x7f95d8153bc0 2024-09-13T18:46:14.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dddc5700 1 --2- 172.21.15.146:0/2710104558 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95d8153bc0 0x7f95d80c8de0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009880 tx=0x7f95d4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11c26cab1fcf3e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:14.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dcdc3700 1 -- 172.21.15.146:0/2710104558 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c80015f0 con 0x7f95d8153bc0 2024-09-13T18:46:14.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dcdc3700 1 -- 172.21.15.146:0/2710104558 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d4004030 con 0x7f95d8153bc0 2024-09-13T18:46:14.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.802+0000 7f95dcdc3700 1 -- 172.21.15.146:0/2710104558 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d40043c0 con 0x7f95d8153bc0 2024-09-13T18:46:14.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.806+0000 7f95dedc7700 1 -- 172.21.15.146:0/2710104558 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95d8153bc0 msgr2=0x7f95d80c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:14.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.806+0000 7f95dedc7700 1 --2- 172.21.15.146:0/2710104558 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95d8153bc0 0x7f95d80c8de0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009880 tx=0x7f95d4009b90 comp rx=0 tx=0).stop 2024-09-13T18:46:14.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.806+0000 7f95dedc7700 1 -- 172.21.15.146:0/2710104558 shutdown_connections 2024-09-13T18:46:14.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.806+0000 7f95dedc7700 1 --2- 172.21.15.146:0/2710104558 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95d8153bc0 0x7f95d80c8de0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:14.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.806+0000 7f95dedc7700 1 -- 172.21.15.146:0/2710104558 >> 172.21.15.146:0/2710104558 conn(0x7f95d8066090 msgr2=0x7f95d8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:14.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.806+0000 7f95dedc7700 1 -- 172.21.15.146:0/2710104558 shutdown_connections 2024-09-13T18:46:14.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.806+0000 7f95dedc7700 1 -- 172.21.15.146:0/2710104558 wait complete. 2024-09-13T18:46:14.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dedc7700 1 Processor -- start 2024-09-13T18:46:14.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dedc7700 1 -- start start 2024-09-13T18:46:14.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dedc7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95d80c6610 0x7f95d80acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:14.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dedc7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95d80c6a50 0x7f95d80c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:14.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dedc7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95d8153bc0 0x7f95d80c2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:14.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dedc7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d81efcd0 con 0x7f95d80c6a50 2024-09-13T18:46:14.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dedc7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d81eede0 con 0x7f95d80c6610 2024-09-13T18:46:14.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dedc7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d81ef0c0 con 0x7f95d8153bc0 2024-09-13T18:46:14.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dd5c4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95d80c6a50 0x7f95d80c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:14.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dd5c4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95d80c6a50 0x7f95d80c4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32858/0 (socket says 172.21.15.146:32858) 2024-09-13T18:46:14.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dd5c4700 1 -- 172.21.15.146:0/2997325142 learned_addr learned my addr 172.21.15.146:0/2997325142 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:14.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dddc5700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95d80c6610 0x7f95d80acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:14.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95de5c6700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95d8153bc0 0x7f95d80c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:14.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dd5c4700 1 -- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95d8153bc0 msgr2=0x7f95d80c2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:14.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dd5c4700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95d8153bc0 0x7f95d80c2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:14.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dd5c4700 1 -- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95d80c6610 msgr2=0x7f95d80acb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:14.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dd5c4700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95d80c6610 0x7f95d80acb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:14.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dd5c4700 1 -- 172.21.15.146:0/2997325142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40094b0 con 0x7f95d80c6a50 2024-09-13T18:46:14.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dddc5700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95d80c6610 0x7f95d80acb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:46:14.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dd5c4700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95d80c6a50 0x7f95d80c4930 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c550 tx=0x7f95c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:14.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95ce7fc700 1 -- 172.21.15.146:0/2997325142 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8004ca0 con 0x7f95d80c6a50 2024-09-13T18:46:14.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95ce7fc700 1 -- 172.21.15.146:0/2997325142 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c8004e00 con 0x7f95d80c6a50 2024-09-13T18:46:14.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dedc7700 1 -- 172.21.15.146:0/2997325142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d81ef3a0 con 0x7f95d80c6a50 2024-09-13T18:46:14.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95ce7fc700 1 -- 172.21.15.146:0/2997325142 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8007310 con 0x7f95d80c6a50 2024-09-13T18:46:14.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.810+0000 7f95dedc7700 1 -- 172.21.15.146:0/2997325142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d81ef960 con 0x7f95d80c6a50 2024-09-13T18:46:14.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.814+0000 7f95dedc7700 1 -- 172.21.15.146:0/2997325142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d80c22a0 con 0x7f95d80c6a50 2024-09-13T18:46:14.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.814+0000 7f95ce7fc700 1 -- 172.21.15.146:0/2997325142 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f95c8007470 con 0x7f95d80c6a50 2024-09-13T18:46:14.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.814+0000 7f95ce7fc700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95bc066760 0x7f95bc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:14.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.814+0000 7f95ce7fc700 1 -- 172.21.15.146:0/2997325142 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7f95c8084640 con 0x7f95d80c6a50 2024-09-13T18:46:14.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.814+0000 7f95dddc5700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95bc066760 0x7f95bc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:14.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.814+0000 7f95dddc5700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95bc066760 0x7f95bc068c20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f95d40090b0 tx=0x7f95d4010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:14.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:14.818+0000 7f95ce7fc700 1 -- 172.21.15.146:0/2997325142 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f95d80c22a0 con 0x7f95d80c6a50 2024-09-13T18:46:15.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.054+0000 7f95dedc7700 1 -- 172.21.15.146:0/2997325142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95d80c22a0 con 0x7f95d80c6a50 2024-09-13T18:46:15.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.054+0000 7f95ce7fc700 1 -- 172.21.15.146:0/2997325142 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5516 (secure 0 0 0) 0x7f95d80c22a0 con 0x7f95d80c6a50 2024-09-13T18:46:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:46:15.076 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:11.821509+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:46:15.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.058+0000 7f95dedc7700 1 -- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95bc066760 msgr2=0x7f95bc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:15.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.058+0000 7f95dedc7700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95bc066760 0x7f95bc068c20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f95d40090b0 tx=0x7f95d4010000 comp rx=0 tx=0).stop 2024-09-13T18:46:15.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.058+0000 7f95dedc7700 1 -- 172.21.15.146:0/2997325142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95d80c6a50 msgr2=0x7f95d80c4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:15.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.058+0000 7f95dedc7700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95d80c6a50 0x7f95d80c4930 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c550 tx=0x7f95c800c910 comp rx=0 tx=0).stop 2024-09-13T18:46:15.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.058+0000 7f95dedc7700 1 -- 172.21.15.146:0/2997325142 shutdown_connections 2024-09-13T18:46:15.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.058+0000 7f95dedc7700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95bc066760 0x7f95bc068c20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:15.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.058+0000 7f95dedc7700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95d8153bc0 0x7f95d80c2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:15.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.058+0000 7f95dedc7700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95d80c6a50 0x7f95d80c4930 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:15.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.058+0000 7f95dedc7700 1 --2- 172.21.15.146:0/2997325142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95d80c6610 0x7f95d80acb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:15.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.058+0000 7f95dedc7700 1 -- 172.21.15.146:0/2997325142 >> 172.21.15.146:0/2997325142 conn(0x7f95d8066090 msgr2=0x7f95d8073c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:15.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.058+0000 7f95dedc7700 1 -- 172.21.15.146:0/2997325142 shutdown_connections 2024-09-13T18:46:15.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:15.058+0000 7f95dedc7700 1 -- 172.21.15.146:0/2997325142 wait complete. 2024-09-13T18:46:15.080 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 83 2024-09-13T18:46:15.090 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:46:17.091 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:46:17.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 Processor -- start 2024-09-13T18:46:17.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 -- start start 2024-09-13T18:46:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb8406f830 0x7ffb84156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffb84070130 0x7ffb84156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb84156790 con 0x7ffb8406fc10 2024-09-13T18:46:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8406d2a0 con 0x7ffb84070130 2024-09-13T18:46:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8406d580 con 0x7ffb8406f830 2024-09-13T18:46:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb88b75700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffb84070130 0x7ffb84156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb89b77700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ffb8406fc10 0x7ffb8414f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:55458/0 (socket says 172.21.15.146:55458) 2024-09-13T18:46:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb89b77700 1 -- 172.21.15.146:0/3195442580 learned_addr learned my addr 172.21.15.146:0/3195442580 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:46:17.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb89376700 1 --2- 172.21.15.146:0/3195442580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb8406f830 0x7ffb84156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:17.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb88b75700 1 -- 172.21.15.146:0/3195442580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb8406f830 msgr2=0x7ffb84156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:46:17.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb88b75700 1 --2- 172.21.15.146:0/3195442580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb8406f830 0x7ffb84156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:17.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb88b75700 1 -- 172.21.15.146:0/3195442580 >> v1:172.21.15.146:6789/0 conn(0x7ffb8406fc10 legacy=0x7ffb8414f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:17.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb88b75700 1 -- 172.21.15.146:0/3195442580 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8406d860 con 0x7ffb84070130 2024-09-13T18:46:17.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb88b75700 1 --2- 172.21.15.146:0/3195442580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb84070130 0x7ffb84156cc0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400a9f0 tx=0x7ffb7400ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cace89a95028a65e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:17.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb7bfff700 1 -- 172.21.15.146:0/3195442580 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7400cb70 con 0x7ffb84070130 2024-09-13T18:46:17.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb7bfff700 1 -- 172.21.15.146:0/3195442580 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7400ccd0 con 0x7ffb84070130 2024-09-13T18:46:17.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb7bfff700 1 -- 172.21.15.146:0/3195442580 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74018620 con 0x7ffb84070130 2024-09-13T18:46:17.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 -- 172.21.15.146:0/3195442580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb84070130 msgr2=0x7ffb84156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:17.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 --2- 172.21.15.146:0/3195442580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb84070130 0x7ffb84156cc0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400a9f0 tx=0x7ffb7400ad00 comp rx=0 tx=0).stop 2024-09-13T18:46:17.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 -- 172.21.15.146:0/3195442580 shutdown_connections 2024-09-13T18:46:17.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 --2- 172.21.15.146:0/3195442580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb84070130 0x7ffb84156cc0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:17.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 --2- 172.21.15.146:0/3195442580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb8406f830 0x7ffb84156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:17.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 -- 172.21.15.146:0/3195442580 >> 172.21.15.146:0/3195442580 conn(0x7ffb8406aaa0 msgr2=0x7ffb8406ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:17.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.194+0000 7ffb8a378700 1 -- 172.21.15.146:0/3195442580 shutdown_connections 2024-09-13T18:46:17.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb8a378700 1 -- 172.21.15.146:0/3195442580 wait complete. 2024-09-13T18:46:17.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb8a378700 1 Processor -- start 2024-09-13T18:46:17.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb8a378700 1 -- start start 2024-09-13T18:46:17.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb8a378700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb8406f830 0x7ffb8413d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:17.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb8a378700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8406fc10 0x7ffb84153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:17.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb8a378700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb84070130 0x7ffb84142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:17.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb8a378700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb841ee330 con 0x7ffb84070130 2024-09-13T18:46:17.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb8a378700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8413d6e0 con 0x7ffb8406fc10 2024-09-13T18:46:17.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb8a378700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb84153f60 con 0x7ffb8406f830 2024-09-13T18:46:17.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb89b77700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb84070130 0x7ffb84142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:17.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb89b77700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb84070130 0x7ffb84142150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32882/0 (socket says 172.21.15.146:32882) 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb89b77700 1 -- 172.21.15.146:0/2468241200 learned_addr learned my addr 172.21.15.146:0/2468241200 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb89376700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb8406f830 0x7ffb8413d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb88b75700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8406fc10 0x7ffb84153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb89b77700 1 -- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb8406f830 msgr2=0x7ffb8413d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb89b77700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb8406f830 0x7ffb8413d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb89b77700 1 -- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8406fc10 msgr2=0x7ffb84153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb89b77700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8406fc10 0x7ffb84153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb89b77700 1 -- 172.21.15.146:0/2468241200 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7400a650 con 0x7ffb84070130 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb89b77700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb84070130 0x7ffb84142150 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00ed60 tx=0x7ffb7c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb79ffb700 1 -- 172.21.15.146:0/2468241200 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00cbf0 con 0x7ffb84070130 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb79ffb700 1 -- 172.21.15.146:0/2468241200 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c00cd50 con 0x7ffb84070130 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb8a378700 1 -- 172.21.15.146:0/2468241200 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb841543b0 con 0x7ffb84070130 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb79ffb700 1 -- 172.21.15.146:0/2468241200 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c011670 con 0x7ffb84070130 2024-09-13T18:46:17.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.198+0000 7ffb8a378700 1 -- 172.21.15.146:0/2468241200 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb84154970 con 0x7ffb84070130 2024-09-13T18:46:17.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.202+0000 7ffb79ffb700 1 -- 172.21.15.146:0/2468241200 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffb7c0117d0 con 0x7ffb84070130 2024-09-13T18:46:17.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.202+0000 7ffb8a378700 1 -- 172.21.15.146:0/2468241200 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb840c7bf0 con 0x7ffb84070130 2024-09-13T18:46:17.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.202+0000 7ffb79ffb700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb68066730 0x7ffb68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:17.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.202+0000 7ffb79ffb700 1 -- 172.21.15.146:0/2468241200 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7ffb7c087000 con 0x7ffb84070130 2024-09-13T18:46:17.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.202+0000 7ffb89376700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb68066730 0x7ffb68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:17.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.202+0000 7ffb89376700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb68066730 0x7ffb68068bf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009a00 tx=0x7ffb80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:17.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.210+0000 7ffb79ffb700 1 -- 172.21.15.146:0/2468241200 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ffb7c053c00 con 0x7ffb84070130 2024-09-13T18:46:17.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.438+0000 7ffb8a378700 1 -- 172.21.15.146:0/2468241200 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb841540d0 con 0x7ffb84070130 2024-09-13T18:46:17.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.438+0000 7ffb79ffb700 1 -- 172.21.15.146:0/2468241200 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+5516 (secure 0 0 0) 0x7ffb7c057220 con 0x7ffb84070130 2024-09-13T18:46:17.460 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:46:17.460 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:16.846531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:46:17.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.442+0000 7ffb8a378700 1 -- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb68066730 msgr2=0x7ffb68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:17.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.442+0000 7ffb8a378700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb68066730 0x7ffb68068bf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009a00 tx=0x7ffb80008040 comp rx=0 tx=0).stop 2024-09-13T18:46:17.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.442+0000 7ffb8a378700 1 -- 172.21.15.146:0/2468241200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb84070130 msgr2=0x7ffb84142150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:17.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.442+0000 7ffb8a378700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb84070130 0x7ffb84142150 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00ed60 tx=0x7ffb7c00ed90 comp rx=0 tx=0).stop 2024-09-13T18:46:17.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.442+0000 7ffb8a378700 1 -- 172.21.15.146:0/2468241200 shutdown_connections 2024-09-13T18:46:17.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.442+0000 7ffb8a378700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb68066730 0x7ffb68068bf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:17.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.442+0000 7ffb8a378700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb84070130 0x7ffb84142150 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:17.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.442+0000 7ffb8a378700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8406fc10 0x7ffb84153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:17.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.442+0000 7ffb8a378700 1 --2- 172.21.15.146:0/2468241200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb8406f830 0x7ffb8413d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:17.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.442+0000 7ffb8a378700 1 -- 172.21.15.146:0/2468241200 >> 172.21.15.146:0/2468241200 conn(0x7ffb8406aaa0 msgr2=0x7ffb840c6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:17.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.442+0000 7ffb8a378700 1 -- 172.21.15.146:0/2468241200 shutdown_connections 2024-09-13T18:46:17.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:17.442+0000 7ffb8a378700 1 -- 172.21.15.146:0/2468241200 wait complete. 2024-09-13T18:46:17.463 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 84 2024-09-13T18:46:17.478 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=1, replacing gid=6207 2024-09-13T18:46:17.478 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.c 2024-09-13T18:46:19.624 INFO:tasks.ceph.mds.a.smithi146.stderr:2024-09-13T18:46:19.602+0000 7f2d72109700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-13T18:46:23.479 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-09-13T18:46:23.479 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T18:46:23.479 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T18:46:23.481 INFO:tasks.ceph.mds.c:Started 2024-09-13T18:46:23.482 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:46:23.568 INFO:tasks.ceph.mds.c.smithi146.stdout:starting mds.c at 2024-09-13T18:46:23.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.570+0000 7f6395b2d700 1 Processor -- start 2024-09-13T18:46:23.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.570+0000 7f6395b2d700 1 -- start start 2024-09-13T18:46:23.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6395b2d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f63880d6900 0x7f6388008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:23.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6395b2d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6388046930 0x7f63880e6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:23.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6395b2d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63880e0a00 con 0x7f63880464e0 2024-09-13T18:46:23.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6395b2d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6388006d70 con 0x7f63880d6900 2024-09-13T18:46:23.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6395b2d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63880d6ce0 con 0x7f6388046930 2024-09-13T18:46:23.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6394b2b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f63880d6900 0x7f6388008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:23.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f638ffff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f63880464e0 0x7f6388036090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:54744/0 (socket says 172.21.15.146:54744) 2024-09-13T18:46:23.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f638ffff700 1 -- 172.21.15.146:0/2965737172 learned_addr learned my addr 172.21.15.146:0/2965737172 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:46:23.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f639532c700 1 --2- 172.21.15.146:0/2965737172 >> v2:172.21.15.154:3300/0 conn(0x7f6388046930 0x7f63880e6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:23.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6394b2b700 1 -- 172.21.15.146:0/2965737172 >> v2:172.21.15.154:3300/0 conn(0x7f6388046930 msgr2=0x7f63880e6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:46:23.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6394b2b700 1 --2- 172.21.15.146:0/2965737172 >> v2:172.21.15.154:3300/0 conn(0x7f6388046930 0x7f63880e6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:23.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6394b2b700 1 -- 172.21.15.146:0/2965737172 >> v1:172.21.15.146:6789/0 conn(0x7f63880464e0 legacy=0x7f6388036090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:23.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f638f7fe700 1 -- 172.21.15.146:0/2965737172 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247887057 0 0) 0x7f63880e0a00 con 0x7f63880464e0 2024-09-13T18:46:23.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6394b2b700 1 -- 172.21.15.146:0/2965737172 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388006a90 con 0x7f63880d6900 2024-09-13T18:46:23.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f639532c700 1 --2- 172.21.15.146:0/2965737172 >> v2:172.21.15.154:3300/0 conn(0x7f6388046930 0x7f63880e6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:46:23.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6394b2b700 1 --2- 172.21.15.146:0/2965737172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d6900 0x7f6388008d40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f638800a6c0 tx=0x7f6384009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f467479b7f831248 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:23.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f638f7fe700 1 -- 172.21.15.146:0/2965737172 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6384011830 con 0x7f63880d6900 2024-09-13T18:46:23.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f638f7fe700 1 -- 172.21.15.146:0/2965737172 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6384011990 con 0x7f63880d6900 2024-09-13T18:46:23.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f638f7fe700 1 -- 172.21.15.146:0/2965737172 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6384011d00 con 0x7f63880d6900 2024-09-13T18:46:23.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6395b2d700 1 -- 172.21.15.146:0/2965737172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d6900 msgr2=0x7f6388008d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:23.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6395b2d700 1 --2- 172.21.15.146:0/2965737172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d6900 0x7f6388008d40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f638800a6c0 tx=0x7f6384009b60 comp rx=0 tx=0).stop 2024-09-13T18:46:23.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6395b2d700 1 -- 172.21.15.146:0/2965737172 shutdown_connections 2024-09-13T18:46:23.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6395b2d700 1 --2- 172.21.15.146:0/2965737172 >> v2:172.21.15.154:3300/0 conn(0x7f6388046930 0x7f63880e6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:23.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6395b2d700 1 --2- 172.21.15.146:0/2965737172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d6900 0x7f6388008d40 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:23.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6395b2d700 1 -- 172.21.15.146:0/2965737172 >> 172.21.15.146:0/2965737172 conn(0x7f6388050e10 msgr2=0x7f6388051220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:23.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6395b2d700 1 -- 172.21.15.146:0/2965737172 shutdown_connections 2024-09-13T18:46:23.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.574+0000 7f6395b2d700 1 -- 172.21.15.146:0/2965737172 wait complete. 2024-09-13T18:46:23.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f6395b2d700 1 Processor -- start 2024-09-13T18:46:23.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f6395b2d700 1 -- start start 2024-09-13T18:46:23.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f6395b2d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63880464e0 0x7f63880dd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:23.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f6395b2d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6388046930 0x7f63880353e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:23.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f6395b2d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d6900 0x7f6388174660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:23.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f6395b2d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6388174240 con 0x7f63880d6900 2024-09-13T18:46:23.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f6395b2d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63880ddcd0 con 0x7f6388046930 2024-09-13T18:46:23.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f6395b2d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63881783f0 con 0x7f63880464e0 2024-09-13T18:46:23.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f639532c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d6900 0x7f6388174660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:23.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f639532c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d6900 0x7f6388174660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33178/0 (socket says 172.21.15.146:33178) 2024-09-13T18:46:23.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f639532c700 1 -- 172.21.15.146:0/2043298209 learned_addr learned my addr 172.21.15.146:0/2043298209 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:23.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f638ffff700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6388046930 0x7f63880353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:23.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f6394b2b700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63880464e0 0x7f63880dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:23.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f639532c700 1 -- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63880464e0 msgr2=0x7f63880dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:23.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f639532c700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63880464e0 0x7f63880dd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:23.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f639532c700 1 -- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6388046930 msgr2=0x7f63880353e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:23.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f639532c700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6388046930 0x7f63880353e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:23.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f639532c700 1 -- 172.21.15.146:0/2043298209 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63840094b0 con 0x7f63880d6900 2024-09-13T18:46:23.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f638ffff700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6388046930 0x7f63880353e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:46:23.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f639532c700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d6900 0x7f6388174660 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6390075280 tx=0x7f63900739e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:23.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f638d7fa700 1 -- 172.21.15.146:0/2043298209 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390067c50 con 0x7f63880d6900 2024-09-13T18:46:23.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f638d7fa700 1 -- 172.21.15.146:0/2043298209 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6390067db0 con 0x7f63880d6900 2024-09-13T18:46:23.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f6395b2d700 1 -- 172.21.15.146:0/2043298209 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6388177870 con 0x7f63880d6900 2024-09-13T18:46:23.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f638d7fa700 1 -- 172.21.15.146:0/2043298209 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390070740 con 0x7f63880d6900 2024-09-13T18:46:23.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.578+0000 7f6395b2d700 1 -- 172.21.15.146:0/2043298209 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6388177e30 con 0x7f63880d6900 2024-09-13T18:46:23.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.582+0000 7f6395b2d700 1 -- 172.21.15.146:0/2043298209 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6370005310 con 0x7f63880d6900 2024-09-13T18:46:23.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.582+0000 7f638d7fa700 1 -- 172.21.15.146:0/2043298209 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f63900708a0 con 0x7f63880d6900 2024-09-13T18:46:23.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.582+0000 7f638d7fa700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6374066730 0x7f6374068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:23.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.582+0000 7f638d7fa700 1 -- 172.21.15.146:0/2043298209 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7f63900ffad0 con 0x7f63880d6900 2024-09-13T18:46:23.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.582+0000 7f6394b2b700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6374066730 0x7f6374068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:23.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.582+0000 7f6394b2b700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6374066730 0x7f6374068bf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f63840097d0 tx=0x7f6384008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:23.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.586+0000 7f638d7fa700 1 -- 172.21.15.146:0/2043298209 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f63900cc640 con 0x7f63880d6900 2024-09-13T18:46:23.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.814+0000 7f6395b2d700 1 -- 172.21.15.146:0/2043298209 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63700055c0 con 0x7f63880d6900 2024-09-13T18:46:23.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.814+0000 7f638d7fa700 1 -- 172.21.15.146:0/2043298209 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+5516 (secure 0 0 0) 0x7f639009a020 con 0x7f63880d6900 2024-09-13T18:46:23.836 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:46:23.837 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:16.846531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:46:23.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.818+0000 7f6395b2d700 1 -- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6374066730 msgr2=0x7f6374068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:23.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.818+0000 7f6395b2d700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6374066730 0x7f6374068bf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f63840097d0 tx=0x7f6384008030 comp rx=0 tx=0).stop 2024-09-13T18:46:23.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.818+0000 7f6395b2d700 1 -- 172.21.15.146:0/2043298209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d6900 msgr2=0x7f6388174660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:23.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.818+0000 7f6395b2d700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d6900 0x7f6388174660 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6390075280 tx=0x7f63900739e0 comp rx=0 tx=0).stop 2024-09-13T18:46:23.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.818+0000 7f6395b2d700 1 -- 172.21.15.146:0/2043298209 shutdown_connections 2024-09-13T18:46:23.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.818+0000 7f6395b2d700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6374066730 0x7f6374068bf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:23.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.818+0000 7f6395b2d700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d6900 0x7f6388174660 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:23.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.818+0000 7f6395b2d700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6388046930 0x7f63880353e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:23.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.818+0000 7f6395b2d700 1 --2- 172.21.15.146:0/2043298209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63880464e0 0x7f63880dd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:23.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.818+0000 7f6395b2d700 1 -- 172.21.15.146:0/2043298209 >> 172.21.15.146:0/2043298209 conn(0x7f6388050e10 msgr2=0x7f638804e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:23.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.818+0000 7f6395b2d700 1 -- 172.21.15.146:0/2043298209 shutdown_connections 2024-09-13T18:46:23.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:23.818+0000 7f6395b2d700 1 -- 172.21.15.146:0/2043298209 wait complete. 2024-09-13T18:46:23.840 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 84 2024-09-13T18:46:23.856 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-09-13T18:46:25.857 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:46:25.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e311b700 1 Processor -- start 2024-09-13T18:46:25.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e311b700 1 -- start start 2024-09-13T18:46:25.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e311b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f59d404c670 0x7f59d40dd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:25.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e311b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f59d4048a00 0x7f59d40d63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:25.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e311b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d40514c0 con 0x7f59d4048620 2024-09-13T18:46:25.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e311b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d40dd680 con 0x7f59d404c670 2024-09-13T18:46:25.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e311b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d404c1e0 con 0x7f59d4048a00 2024-09-13T18:46:25.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e1918700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f59d404c670 0x7f59d40dd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:25.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e1918700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f59d404c670 0x7f59d40dd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33182/0 (socket says 172.21.15.146:33182) 2024-09-13T18:46:25.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e2119700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f59d4048620 0x7f59d40ddb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:54746/0 (socket says 172.21.15.146:54746) 2024-09-13T18:46:25.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e1918700 1 -- 172.21.15.146:0/555817626 learned_addr learned my addr 172.21.15.146:0/555817626 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:25.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e291a700 1 --2- 172.21.15.146:0/555817626 >> v2:172.21.15.154:3300/0 conn(0x7f59d4048a00 0x7f59d40d63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:25.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e1117700 1 -- 172.21.15.146:0/555817626 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160313358 0 0) 0x7f59d40514c0 con 0x7f59d4048620 2024-09-13T18:46:25.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e1117700 1 -- 172.21.15.146:0/555817626 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59c80035d0 con 0x7f59d4048620 2024-09-13T18:46:25.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e1918700 1 -- 172.21.15.146:0/555817626 >> v2:172.21.15.154:3300/0 conn(0x7f59d4048a00 msgr2=0x7f59d40d63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:46:25.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e1918700 1 --2- 172.21.15.146:0/555817626 >> v2:172.21.15.154:3300/0 conn(0x7f59d4048a00 0x7f59d40d63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:25.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e1918700 1 -- 172.21.15.146:0/555817626 >> v1:172.21.15.146:6789/0 conn(0x7f59d4048620 legacy=0x7f59d40ddb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:25.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e1918700 1 -- 172.21.15.146:0/555817626 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d404b2f0 con 0x7f59d404c670 2024-09-13T18:46:25.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e1918700 1 --2- 172.21.15.146:0/555817626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59d404c670 0x7f59d40dd140 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009e10 tx=0x7f59cc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a82da0f9f6a9c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:25.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.958+0000 7f59e1117700 1 -- 172.21.15.146:0/555817626 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc011830 con 0x7f59d404c670 2024-09-13T18:46:25.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e1117700 1 -- 172.21.15.146:0/555817626 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc011990 con 0x7f59d404c670 2024-09-13T18:46:25.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e1117700 1 -- 172.21.15.146:0/555817626 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc011d20 con 0x7f59d404c670 2024-09-13T18:46:25.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 -- 172.21.15.146:0/555817626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59d404c670 msgr2=0x7f59d40dd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:25.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 --2- 172.21.15.146:0/555817626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59d404c670 0x7f59d40dd140 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009e10 tx=0x7f59cc009bf0 comp rx=0 tx=0).stop 2024-09-13T18:46:25.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 -- 172.21.15.146:0/555817626 shutdown_connections 2024-09-13T18:46:25.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 --2- 172.21.15.146:0/555817626 >> v2:172.21.15.154:3300/0 conn(0x7f59d4048a00 0x7f59d40d63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:25.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 --2- 172.21.15.146:0/555817626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59d404c670 0x7f59d40dd140 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:25.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 -- 172.21.15.146:0/555817626 >> 172.21.15.146:0/555817626 conn(0x7f59d4050610 msgr2=0x7f59d40509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:25.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 -- 172.21.15.146:0/555817626 shutdown_connections 2024-09-13T18:46:25.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 -- 172.21.15.146:0/555817626 wait complete. 2024-09-13T18:46:25.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 Processor -- start 2024-09-13T18:46:25.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 -- start start 2024-09-13T18:46:25.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59d4048620 0x7f59d402ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:25.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59d4048a00 0x7f59d4032a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:25.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59d404c670 0x7f59d4046b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:25.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40304f0 con 0x7f59d4048620 2024-09-13T18:46:25.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4032fd0 con 0x7f59d4048a00 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e311b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4177cf0 con 0x7f59d404c670 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e2119700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59d4048620 0x7f59d402ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e2119700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59d4048620 0x7f59d402ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33196/0 (socket says 172.21.15.146:33196) 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e2119700 1 -- 172.21.15.146:0/3955185021 learned_addr learned my addr 172.21.15.146:0/3955185021 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e1918700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59d4048a00 0x7f59d4032a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e291a700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59d404c670 0x7f59d4046b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e2119700 1 -- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59d404c670 msgr2=0x7f59d4046b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e2119700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59d404c670 0x7f59d4046b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e2119700 1 -- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59d4048a00 msgr2=0x7f59d4032a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e2119700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59d4048a00 0x7f59d4032a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.962+0000 7f59e2119700 1 -- 172.21.15.146:0/3955185021 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc009570 con 0x7f59d4048620 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.966+0000 7f59e2119700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59d4048620 0x7f59d402ffb0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f59d800c500 tx=0x7f59d800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.966+0000 7f59d2ffd700 1 -- 172.21.15.146:0/3955185021 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8005cd0 con 0x7f59d4048620 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.966+0000 7f59d2ffd700 1 -- 172.21.15.146:0/3955185021 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d8005e30 con 0x7f59d4048620 2024-09-13T18:46:25.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.966+0000 7f59e311b700 1 -- 172.21.15.146:0/3955185021 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d40306f0 con 0x7f59d4048620 2024-09-13T18:46:25.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.966+0000 7f59d2ffd700 1 -- 172.21.15.146:0/3955185021 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8007310 con 0x7f59d4048620 2024-09-13T18:46:25.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.966+0000 7f59e311b700 1 -- 172.21.15.146:0/3955185021 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d4178240 con 0x7f59d4048620 2024-09-13T18:46:25.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.966+0000 7f59e311b700 1 -- 172.21.15.146:0/3955185021 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d404be30 con 0x7f59d4048620 2024-09-13T18:46:25.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.966+0000 7f59d2ffd700 1 -- 172.21.15.146:0/3955185021 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59d8007470 con 0x7f59d4048620 2024-09-13T18:46:25.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.966+0000 7f59d2ffd700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f59c0066730 0x7f59c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:46:25.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.966+0000 7f59d2ffd700 1 -- 172.21.15.146:0/3955185021 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7f59d8083c80 con 0x7f59d4048620 2024-09-13T18:46:25.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.966+0000 7f59e1918700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f59c0066730 0x7f59c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:46:25.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.970+0000 7f59e1918700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f59c0066730 0x7f59c0068bf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f59d404ca70 tx=0x7f59cc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:46:25.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:25.974+0000 7f59d2ffd700 1 -- 172.21.15.146:0/3955185021 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f59d8050910 con 0x7f59d4048620 2024-09-13T18:46:26.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.214+0000 7f59e311b700 1 -- 172.21.15.146:0/3955185021 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59d404af90 con 0x7f59d4048620 2024-09-13T18:46:26.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.218+0000 7f59d2ffd700 1 -- 172.21.15.146:0/3955185021 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f59d8017090 con 0x7f59d4048620 2024-09-13T18:46:26.237 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:46:26.237 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6714,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:16.846531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:46:26.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.218+0000 7f59e311b700 1 -- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f59c0066730 msgr2=0x7f59c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:26.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.218+0000 7f59e311b700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f59c0066730 0x7f59c0068bf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f59d404ca70 tx=0x7f59cc008030 comp rx=0 tx=0).stop 2024-09-13T18:46:26.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.218+0000 7f59e311b700 1 -- 172.21.15.146:0/3955185021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59d4048620 msgr2=0x7f59d402ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:46:26.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.218+0000 7f59e311b700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59d4048620 0x7f59d402ffb0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f59d800c500 tx=0x7f59d800c810 comp rx=0 tx=0).stop 2024-09-13T18:46:26.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.222+0000 7f59e311b700 1 -- 172.21.15.146:0/3955185021 shutdown_connections 2024-09-13T18:46:26.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.222+0000 7f59e311b700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f59c0066730 0x7f59c0068bf0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:26.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.222+0000 7f59e311b700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59d404c670 0x7f59d4046b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:26.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.222+0000 7f59e311b700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59d4048a00 0x7f59d4032a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:26.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.222+0000 7f59e311b700 1 --2- 172.21.15.146:0/3955185021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59d4048620 0x7f59d402ffb0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:46:26.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.222+0000 7f59e311b700 1 -- 172.21.15.146:0/3955185021 >> 172.21.15.146:0/3955185021 conn(0x7f59d4050610 msgr2=0x7f59d40da320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:46:26.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.222+0000 7f59e311b700 1 -- 172.21.15.146:0/3955185021 shutdown_connections 2024-09-13T18:46:26.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:46:26.222+0000 7f59e311b700 1 -- 172.21.15.146:0/3955185021 wait complete. 2024-09-13T18:46:26.243 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 85 2024-09-13T18:46:26.254 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-09-13T18:46:26.254 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 99 secs before thrashing 2024-09-13T18:46:40.711 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:46:40.713 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:47:10.758 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:47:10.761 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:47:40.820 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:47:40.823 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:48:05.256 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:05.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.356+0000 7f497efd1700 1 Processor -- start 2024-09-13T18:48:05.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.356+0000 7f497efd1700 1 -- start start 2024-09-13T18:48:05.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f49700d8b80 0x7f4970007950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:05.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f497004b7e0 0x7f4970045f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:05.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49700e1cc0 0x7f49700e2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:05.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4970008110 con 0x7f49700d8b80 2024-09-13T18:48:05.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49700478d0 con 0x7f497004b7e0 2024-09-13T18:48:05.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497004bc10 con 0x7f49700e1cc0 2024-09-13T18:48:05.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497dfcf700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f49700d8b80 0x7f4970007950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:05.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497dfcf700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f49700d8b80 0x7f4970007950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50326/0 (socket says 172.21.15.146:50326) 2024-09-13T18:48:05.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497dfcf700 1 -- 172.21.15.146:0/230924852 learned_addr learned my addr 172.21.15.146:0/230924852 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:05.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497d7ce700 1 --2- 172.21.15.146:0/230924852 >> v2:172.21.15.154:3300/0 conn(0x7f497004b7e0 0x7f4970045f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:05.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497e7d0700 1 --2- 172.21.15.146:0/230924852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49700e1cc0 0x7f49700e2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:05.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497dfcf700 1 -- 172.21.15.146:0/230924852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49700e1cc0 msgr2=0x7f49700e2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:05.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497dfcf700 1 --2- 172.21.15.146:0/230924852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49700e1cc0 0x7f49700e2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497dfcf700 1 -- 172.21.15.146:0/230924852 >> v2:172.21.15.154:3300/0 conn(0x7f497004b7e0 msgr2=0x7f4970045f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:05.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497dfcf700 1 --2- 172.21.15.146:0/230924852 >> v2:172.21.15.154:3300/0 conn(0x7f497004b7e0 0x7f4970045f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497dfcf700 1 -- 172.21.15.146:0/230924852 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970047b90 con 0x7f49700d8b80 2024-09-13T18:48:05.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497dfcf700 1 --2- 172.21.15.146:0/230924852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49700d8b80 0x7f4970007950 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4970033ba0 tx=0x7f4974009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d94ddf2dacfbeb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:05.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497cfcd700 1 -- 172.21.15.146:0/230924852 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400b8a0 con 0x7f49700d8b80 2024-09-13T18:48:05.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497cfcd700 1 -- 172.21.15.146:0/230924852 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400ba00 con 0x7f49700d8b80 2024-09-13T18:48:05.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497cfcd700 1 -- 172.21.15.146:0/230924852 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400bd90 con 0x7f49700d8b80 2024-09-13T18:48:05.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 -- 172.21.15.146:0/230924852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49700d8b80 msgr2=0x7f4970007950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:05.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 --2- 172.21.15.146:0/230924852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49700d8b80 0x7f4970007950 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4970033ba0 tx=0x7f4974009bf0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 -- 172.21.15.146:0/230924852 shutdown_connections 2024-09-13T18:48:05.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 --2- 172.21.15.146:0/230924852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49700e1cc0 0x7f49700e2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 --2- 172.21.15.146:0/230924852 >> v2:172.21.15.154:3300/0 conn(0x7f497004b7e0 0x7f4970045f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 --2- 172.21.15.146:0/230924852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49700d8b80 0x7f4970007950 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 -- 172.21.15.146:0/230924852 >> 172.21.15.146:0/230924852 conn(0x7f497004d8e0 msgr2=0x7f497004dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:05.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 -- 172.21.15.146:0/230924852 shutdown_connections 2024-09-13T18:48:05.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.360+0000 7f497efd1700 1 -- 172.21.15.146:0/230924852 wait complete. 2024-09-13T18:48:05.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497efd1700 1 Processor -- start 2024-09-13T18:48:05.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497efd1700 1 -- start start 2024-09-13T18:48:05.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497efd1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f497004b7e0 0x7f4970048950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:05.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497efd1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49700d8b80 0x7f49700cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:05.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497efd1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49700e1cc0 0x7f49700cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497efd1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49700cba10 con 0x7f49700e1cc0 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497efd1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970178f40 con 0x7f49700d8b80 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497efd1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497004ade0 con 0x7f497004b7e0 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497e7d0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49700e1cc0 0x7f49700cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497e7d0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49700e1cc0 0x7f49700cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50340/0 (socket says 172.21.15.146:50340) 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497e7d0700 1 -- 172.21.15.146:0/1331065897 learned_addr learned my addr 172.21.15.146:0/1331065897 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497dfcf700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f497004b7e0 0x7f4970048950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497d7ce700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49700d8b80 0x7f49700cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497e7d0700 1 -- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f497004b7e0 msgr2=0x7f4970048950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497e7d0700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f497004b7e0 0x7f4970048950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497e7d0700 1 -- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49700d8b80 msgr2=0x7f49700cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497e7d0700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49700d8b80 0x7f49700cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497e7d0700 1 -- 172.21.15.146:0/1331065897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4974009570 con 0x7f49700e1cc0 2024-09-13T18:48:05.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497e7d0700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49700e1cc0 0x7f49700cc1f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f49780751e0 tx=0x7f49780739e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f496e7fc700 1 -- 172.21.15.146:0/1331065897 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978067950 con 0x7f49700e1cc0 2024-09-13T18:48:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f496e7fc700 1 -- 172.21.15.146:0/1331065897 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4978067c50 con 0x7f49700e1cc0 2024-09-13T18:48:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497efd1700 1 -- 172.21.15.146:0/1331065897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49700cb470 con 0x7f49700e1cc0 2024-09-13T18:48:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f496e7fc700 1 -- 172.21.15.146:0/1331065897 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978068000 con 0x7f49700e1cc0 2024-09-13T18:48:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.364+0000 7f497efd1700 1 -- 172.21.15.146:0/1331065897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4970178c60 con 0x7f49700e1cc0 2024-09-13T18:48:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.368+0000 7f497efd1700 1 -- 172.21.15.146:0/1331065897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49700cb620 con 0x7f49700e1cc0 2024-09-13T18:48:05.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.368+0000 7f496e7fc700 1 -- 172.21.15.146:0/1331065897 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4978070740 con 0x7f49700e1cc0 2024-09-13T18:48:05.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.368+0000 7f496e7fc700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f495c066730 0x7f495c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:05.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.368+0000 7f496e7fc700 1 -- 172.21.15.146:0/1331065897 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7f49780fea50 con 0x7f49700e1cc0 2024-09-13T18:48:05.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.368+0000 7f497dfcf700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f495c066730 0x7f495c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:05.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.368+0000 7f497dfcf700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f495c066730 0x7f495c068bf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4974000c00 tx=0x7f4974008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:05.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.376+0000 7f496e7fc700 1 -- 172.21.15.146:0/1331065897 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f49700cb620 con 0x7f49700e1cc0 2024-09-13T18:48:05.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.588+0000 7f497efd1700 1 -- 172.21.15.146:0/1331065897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49700cb620 con 0x7f49700e1cc0 2024-09-13T18:48:05.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.588+0000 7f496e7fc700 1 -- 172.21.15.146:0/1331065897 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f49700cb620 con 0x7f49700e1cc0 2024-09-13T18:48:05.605 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:05.605 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6714,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:16.846531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:05.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.592+0000 7f497efd1700 1 -- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f495c066730 msgr2=0x7f495c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:05.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.592+0000 7f497efd1700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f495c066730 0x7f495c068bf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4974000c00 tx=0x7f4974008030 comp rx=0 tx=0).stop 2024-09-13T18:48:05.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.592+0000 7f497efd1700 1 -- 172.21.15.146:0/1331065897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49700e1cc0 msgr2=0x7f49700cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:05.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.592+0000 7f497efd1700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49700e1cc0 0x7f49700cc1f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f49780751e0 tx=0x7f49780739e0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.592+0000 7f497efd1700 1 -- 172.21.15.146:0/1331065897 shutdown_connections 2024-09-13T18:48:05.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.592+0000 7f497efd1700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f495c066730 0x7f495c068bf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.592+0000 7f497efd1700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49700e1cc0 0x7f49700cc1f0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.592+0000 7f497efd1700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49700d8b80 0x7f49700cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.592+0000 7f497efd1700 1 --2- 172.21.15.146:0/1331065897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f497004b7e0 0x7f4970048950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.592+0000 7f497efd1700 1 -- 172.21.15.146:0/1331065897 >> 172.21.15.146:0/1331065897 conn(0x7f497004d8e0 msgr2=0x7f49700e0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:05.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.592+0000 7f497efd1700 1 -- 172.21.15.146:0/1331065897 shutdown_connections 2024-09-13T18:48:05.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.592+0000 7f497efd1700 1 -- 172.21.15.146:0/1331065897 wait complete. 2024-09-13T18:48:05.609 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 85 2024-09-13T18:48:05.621 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=1) 2024-09-13T18:48:05.621 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-09-13T18:48:05.621 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:48:05.638 INFO:tasks.ceph.mds.f:Stopped 2024-09-13T18:48:05.638 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:05.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aac8e6700 1 Processor -- start 2024-09-13T18:48:05.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aac8e6700 1 -- start start 2024-09-13T18:48:05.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aac8e6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9aa80cbd50 0x7f9aa80b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:05.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aac8e6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9aa8071390 0x7f9aa80cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:05.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aac8e6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa80cc670 con 0x7f9aa80ca210 2024-09-13T18:48:05.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aac8e6700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aa80caa30 con 0x7f9aa8071390 2024-09-13T18:48:05.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aac8e6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aa80cb810 con 0x7f9aa80cbd50 2024-09-13T18:48:05.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aa7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9aa8071390 0x7f9aa80cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:05.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aa7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9aa8071390 0x7f9aa80cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50342/0 (socket says 172.21.15.146:50342) 2024-09-13T18:48:05.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aa7fff700 1 -- 172.21.15.146:0/1572668021 learned_addr learned my addr 172.21.15.146:0/1572668021 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:05.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aa77fe700 1 --2- 172.21.15.146:0/1572668021 >> v2:172.21.15.154:3300/0 conn(0x7f9aa80cbd50 0x7f9aa80b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:05.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aa7fff700 1 -- 172.21.15.146:0/1572668021 >> v2:172.21.15.154:3300/0 conn(0x7f9aa80cbd50 msgr2=0x7f9aa80b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:05.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aa7fff700 1 --2- 172.21.15.146:0/1572668021 >> v2:172.21.15.154:3300/0 conn(0x7f9aa80cbd50 0x7f9aa80b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aa7fff700 1 -- 172.21.15.146:0/1572668021 >> v1:172.21.15.154:6789/0 conn(0x7f9aa80ca210 legacy=0x7f9aa80701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:05.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aa7fff700 1 -- 172.21.15.146:0/1572668021 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa815da20 con 0x7f9aa8071390 2024-09-13T18:48:05.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aa7fff700 1 --2- 172.21.15.146:0/1572668021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa8071390 0x7f9aa80cc130 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9aa808c2d0 tx=0x7f9aa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f12e2a4a582895 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:05.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.732+0000 7f9aa6ffd700 1 -- 172.21.15.146:0/1572668021 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa000e070 con 0x7f9aa8071390 2024-09-13T18:48:05.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.736+0000 7f9aa6ffd700 1 -- 172.21.15.146:0/1572668021 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa000b9c0 con 0x7f9aa8071390 2024-09-13T18:48:05.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.736+0000 7f9aa6ffd700 1 -- 172.21.15.146:0/1572668021 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa000bd70 con 0x7f9aa8071390 2024-09-13T18:48:05.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.736+0000 7f9aac8e6700 1 -- 172.21.15.146:0/1572668021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa8071390 msgr2=0x7f9aa80cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:05.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.736+0000 7f9aac8e6700 1 --2- 172.21.15.146:0/1572668021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa8071390 0x7f9aa80cc130 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9aa808c2d0 tx=0x7f9aa000b330 comp rx=0 tx=0).stop 2024-09-13T18:48:05.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.736+0000 7f9aac8e6700 1 -- 172.21.15.146:0/1572668021 shutdown_connections 2024-09-13T18:48:05.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.736+0000 7f9aac8e6700 1 --2- 172.21.15.146:0/1572668021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa8071390 0x7f9aa80cc130 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.736+0000 7f9aac8e6700 1 --2- 172.21.15.146:0/1572668021 >> v2:172.21.15.154:3300/0 conn(0x7f9aa80cbd50 0x7f9aa80b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.736+0000 7f9aac8e6700 1 -- 172.21.15.146:0/1572668021 >> 172.21.15.146:0/1572668021 conn(0x7f9aa8066270 msgr2=0x7f9aa8066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:05.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.736+0000 7f9aac8e6700 1 -- 172.21.15.146:0/1572668021 shutdown_connections 2024-09-13T18:48:05.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.736+0000 7f9aac8e6700 1 -- 172.21.15.146:0/1572668021 wait complete. 2024-09-13T18:48:05.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.736+0000 7f9aac8e6700 1 Processor -- start 2024-09-13T18:48:05.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.736+0000 7f9aac8e6700 1 -- start start 2024-09-13T18:48:05.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aac8e6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9aa8071390 0x7f9aa8147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:05.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aac8e6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9aa80ca210 0x7f9aa81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:05.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aac8e6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa80cbd50 0x7f9aa8149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:05.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aac8e6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa814f8f0 con 0x7f9aa80cbd50 2024-09-13T18:48:05.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aac8e6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa81efc70 con 0x7f9aa8071390 2024-09-13T18:48:05.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aac8e6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa81eff50 con 0x7f9aa80ca210 2024-09-13T18:48:05.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aa7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa80cbd50 0x7f9aa8149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:05.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aa7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa80cbd50 0x7f9aa8149160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50346/0 (socket says 172.21.15.146:50346) 2024-09-13T18:48:05.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aa7fff700 1 -- 172.21.15.146:0/386328775 learned_addr learned my addr 172.21.15.146:0/386328775 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:05.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aa77fe700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9aa8071390 0x7f9aa8147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:05.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9a9ffff700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9aa80ca210 0x7f9aa81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:05.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aa7fff700 1 -- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9aa80ca210 msgr2=0x7f9aa81478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:05.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aa7fff700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9aa80ca210 0x7f9aa81478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aa7fff700 1 -- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9aa8071390 msgr2=0x7f9aa8147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:05.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aa7fff700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9aa8071390 0x7f9aa8147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:05.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aa7fff700 1 -- 172.21.15.146:0/386328775 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa0009b30 con 0x7f9aa80cbd50 2024-09-13T18:48:05.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aa7fff700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa80cbd50 0x7f9aa8149160 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80cb990 tx=0x7f9aa0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:05.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aa4ff9700 1 -- 172.21.15.146:0/386328775 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0004930 con 0x7f9aa80cbd50 2024-09-13T18:48:05.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aa4ff9700 1 -- 172.21.15.146:0/386328775 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa00092e0 con 0x7f9aa80cbd50 2024-09-13T18:48:05.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aac8e6700 1 -- 172.21.15.146:0/386328775 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa81f0230 con 0x7f9aa80cbd50 2024-09-13T18:48:05.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aa4ff9700 1 -- 172.21.15.146:0/386328775 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa001c660 con 0x7f9aa80cbd50 2024-09-13T18:48:05.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.740+0000 7f9aac8e6700 1 -- 172.21.15.146:0/386328775 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa815a830 con 0x7f9aa80cbd50 2024-09-13T18:48:05.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.744+0000 7f9aac8e6700 1 -- 172.21.15.146:0/386328775 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa815a670 con 0x7f9aa80cbd50 2024-09-13T18:48:05.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.744+0000 7f9aa4ff9700 1 -- 172.21.15.146:0/386328775 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9aa001c7c0 con 0x7f9aa80cbd50 2024-09-13T18:48:05.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.744+0000 7f9aa4ff9700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9a88066760 0x7f9a88068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:05.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.744+0000 7f9aa4ff9700 1 -- 172.21.15.146:0/386328775 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7f9aa0086130 con 0x7f9aa80cbd50 2024-09-13T18:48:05.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.744+0000 7f9aa77fe700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9a88066760 0x7f9a88068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:05.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.744+0000 7f9aa77fe700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9a88066760 0x7f9a88068c20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9a98006b40 tx=0x7f9a98008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:05.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:05.748+0000 7f9aa4ff9700 1 -- 172.21.15.146:0/386328775 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9aa0052ee0 con 0x7f9aa80cbd50 2024-09-13T18:48:06.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.000+0000 7f9aac8e6700 1 -- 172.21.15.146:0/386328775 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9aa815d740 con 0x7f9aa80cbd50 2024-09-13T18:48:06.019 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.004+0000 7f9aa4ff9700 1 -- 172.21.15.146:0/386328775 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f9aa0056500 con 0x7f9aa80cbd50 2024-09-13T18:48:06.019 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:06.019 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6714,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:16.846531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:06.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.004+0000 7f9aac8e6700 1 -- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9a88066760 msgr2=0x7f9a88068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:06.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.004+0000 7f9aac8e6700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9a88066760 0x7f9a88068c20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9a98006b40 tx=0x7f9a98008040 comp rx=0 tx=0).stop 2024-09-13T18:48:06.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.004+0000 7f9aac8e6700 1 -- 172.21.15.146:0/386328775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa80cbd50 msgr2=0x7f9aa8149160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:06.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.004+0000 7f9aac8e6700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa80cbd50 0x7f9aa8149160 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80cb990 tx=0x7f9aa0004af0 comp rx=0 tx=0).stop 2024-09-13T18:48:06.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.004+0000 7f9aac8e6700 1 -- 172.21.15.146:0/386328775 shutdown_connections 2024-09-13T18:48:06.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.004+0000 7f9aac8e6700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9a88066760 0x7f9a88068c20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:06.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.004+0000 7f9aac8e6700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa80cbd50 0x7f9aa8149160 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:06.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.004+0000 7f9aac8e6700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9aa80ca210 0x7f9aa81478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:06.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.004+0000 7f9aac8e6700 1 --2- 172.21.15.146:0/386328775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9aa8071390 0x7f9aa8147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:06.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.004+0000 7f9aac8e6700 1 -- 172.21.15.146:0/386328775 >> 172.21.15.146:0/386328775 conn(0x7f9aa8066270 msgr2=0x7f9aa80b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:06.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.004+0000 7f9aac8e6700 1 -- 172.21.15.146:0/386328775 shutdown_connections 2024-09-13T18:48:06.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:06.004+0000 7f9aac8e6700 1 -- 172.21.15.146:0/386328775 wait complete. 2024-09-13T18:48:06.024 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 85 2024-09-13T18:48:06.042 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:48:08.042 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:08.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f077a781700 1 Processor -- start 2024-09-13T18:48:08.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f077a781700 1 -- start start 2024-09-13T18:48:08.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f077a781700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f07740c57f0 0x7f07740cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:08.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f077a781700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07740cf450 0x7f07740cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:08.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f077a781700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07740c5bd0 con 0x7f07740ca020 2024-09-13T18:48:08.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f077a781700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077415a160 con 0x7f07740c57f0 2024-09-13T18:48:08.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f077a781700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0774159830 con 0x7f07740cf450 2024-09-13T18:48:08.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f0778d1e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07740cf450 0x7f07740cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:08.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f0773fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f07740c57f0 0x7f07740cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:08.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f07737fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f07740ca020 0x7f07740ca850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:33550/0 (socket says 172.21.15.146:33550) 2024-09-13T18:48:08.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f07737fe700 1 -- 172.21.15.146:0/761630637 learned_addr learned my addr 172.21.15.146:0/761630637 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:48:08.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f0773fff700 1 -- 172.21.15.146:0/761630637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07740cf450 msgr2=0x7f07740cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:08.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f0773fff700 1 --2- 172.21.15.146:0/761630637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07740cf450 0x7f07740cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:08.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f0773fff700 1 -- 172.21.15.146:0/761630637 >> v1:172.21.15.154:6789/0 conn(0x7f07740ca020 legacy=0x7f07740ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:08.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f0773fff700 1 -- 172.21.15.146:0/761630637 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07740c9bb0 con 0x7f07740c57f0 2024-09-13T18:48:08.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.144+0000 7f0773fff700 1 --2- 172.21.15.146:0/761630637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07740c57f0 0x7f07740cb930 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0768009850 tx=0x7f0768009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5684d68d7c31ccbf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:08.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f0772ffd700 1 -- 172.21.15.146:0/761630637 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768011830 con 0x7f07740c57f0 2024-09-13T18:48:08.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f0772ffd700 1 -- 172.21.15.146:0/761630637 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0768011990 con 0x7f07740c57f0 2024-09-13T18:48:08.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f0772ffd700 1 -- 172.21.15.146:0/761630637 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768011d40 con 0x7f07740c57f0 2024-09-13T18:48:08.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 -- 172.21.15.146:0/761630637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07740c57f0 msgr2=0x7f07740cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:08.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 --2- 172.21.15.146:0/761630637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07740c57f0 0x7f07740cb930 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0768009850 tx=0x7f0768009b60 comp rx=0 tx=0).stop 2024-09-13T18:48:08.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 -- 172.21.15.146:0/761630637 shutdown_connections 2024-09-13T18:48:08.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 --2- 172.21.15.146:0/761630637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07740cf450 0x7f07740cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:08.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 --2- 172.21.15.146:0/761630637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07740c57f0 0x7f07740cb930 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:08.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 -- 172.21.15.146:0/761630637 >> 172.21.15.146:0/761630637 conn(0x7f0774071d30 msgr2=0x7f0774072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:08.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 -- 172.21.15.146:0/761630637 shutdown_connections 2024-09-13T18:48:08.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 -- 172.21.15.146:0/761630637 wait complete. 2024-09-13T18:48:08.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 Processor -- start 2024-09-13T18:48:08.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 -- start start 2024-09-13T18:48:08.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07740c57f0 0x7f07741e3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:08.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07740ca020 0x7f07741e3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:08.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07740cf450 0x7f077414ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:08.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07741e15d0 con 0x7f07740ca020 2024-09-13T18:48:08.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07741e2f10 con 0x7f07740c57f0 2024-09-13T18:48:08.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f077a781700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07741e28e0 con 0x7f07740cf450 2024-09-13T18:48:08.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f07737fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07740ca020 0x7f07741e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:08.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f07737fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07740ca020 0x7f07741e3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50356/0 (socket says 172.21.15.146:50356) 2024-09-13T18:48:08.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f07737fe700 1 -- 172.21.15.146:0/2856929487 learned_addr learned my addr 172.21.15.146:0/2856929487 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:08.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f0773fff700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07740c57f0 0x7f07741e3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:08.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f0778d1e700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07740cf450 0x7f077414ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:08.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f07737fe700 1 -- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07740cf450 msgr2=0x7f077414ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:08.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f07737fe700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07740cf450 0x7f077414ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:08.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f07737fe700 1 -- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07740c57f0 msgr2=0x7f07741e3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:08.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f07737fe700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07740c57f0 0x7f07741e3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:08.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f07737fe700 1 -- 172.21.15.146:0/2856929487 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07680094b0 con 0x7f07740ca020 2024-09-13T18:48:08.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.148+0000 7f0773fff700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07740c57f0 0x7f07741e3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:48:08.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.152+0000 7f07737fe700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07740ca020 0x7f07741e3840 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f076400cba0 tx=0x7f076400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:08.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.152+0000 7f0770ff9700 1 -- 172.21.15.146:0/2856929487 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764007480 con 0x7f07740ca020 2024-09-13T18:48:08.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.152+0000 7f0770ff9700 1 -- 172.21.15.146:0/2856929487 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07640075e0 con 0x7f07740ca020 2024-09-13T18:48:08.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.152+0000 7f077a781700 1 -- 172.21.15.146:0/2856929487 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07741e2c50 con 0x7f07740ca020 2024-09-13T18:48:08.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.152+0000 7f0770ff9700 1 -- 172.21.15.146:0/2856929487 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764007950 con 0x7f07740ca020 2024-09-13T18:48:08.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.152+0000 7f077a781700 1 -- 172.21.15.146:0/2856929487 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077414e510 con 0x7f07740ca020 2024-09-13T18:48:08.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.152+0000 7f077a781700 1 -- 172.21.15.146:0/2856929487 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077415a560 con 0x7f07740ca020 2024-09-13T18:48:08.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.152+0000 7f0770ff9700 1 -- 172.21.15.146:0/2856929487 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f076401b3b0 con 0x7f07740ca020 2024-09-13T18:48:08.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.152+0000 7f0770ff9700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0760066730 0x7f0760068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:08.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.152+0000 7f0770ff9700 1 -- 172.21.15.146:0/2856929487 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7f07640840c0 con 0x7f07740ca020 2024-09-13T18:48:08.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.152+0000 7f0773fff700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0760066730 0x7f0760068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:08.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.156+0000 7f0773fff700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0760066730 0x7f0760068bf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f07680090b0 tx=0x7f0768008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:08.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.160+0000 7f0770ff9700 1 -- 172.21.15.146:0/2856929487 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0764050d70 con 0x7f07740ca020 2024-09-13T18:48:08.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.392+0000 7f077a781700 1 -- 172.21.15.146:0/2856929487 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07741595f0 con 0x7f07740ca020 2024-09-13T18:48:08.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.392+0000 7f0770ff9700 1 -- 172.21.15.146:0/2856929487 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f0764017090 con 0x7f07740ca020 2024-09-13T18:48:08.409 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:08.409 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6714,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:16.846531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:08.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.396+0000 7f077a781700 1 -- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0760066730 msgr2=0x7f0760068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:08.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.396+0000 7f077a781700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0760066730 0x7f0760068bf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f07680090b0 tx=0x7f0768008030 comp rx=0 tx=0).stop 2024-09-13T18:48:08.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.396+0000 7f077a781700 1 -- 172.21.15.146:0/2856929487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07740ca020 msgr2=0x7f07741e3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:08.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.396+0000 7f077a781700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07740ca020 0x7f07741e3840 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f076400cba0 tx=0x7f076400cbd0 comp rx=0 tx=0).stop 2024-09-13T18:48:08.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.396+0000 7f077a781700 1 -- 172.21.15.146:0/2856929487 shutdown_connections 2024-09-13T18:48:08.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.396+0000 7f077a781700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0760066730 0x7f0760068bf0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:08.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.396+0000 7f077a781700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07740cf450 0x7f077414ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:08.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.396+0000 7f077a781700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07740ca020 0x7f07741e3840 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:08.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.396+0000 7f077a781700 1 --2- 172.21.15.146:0/2856929487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07740c57f0 0x7f07741e3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:08.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.396+0000 7f077a781700 1 -- 172.21.15.146:0/2856929487 >> 172.21.15.146:0/2856929487 conn(0x7f0774071d30 msgr2=0x7f07740c8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:08.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.396+0000 7f077a781700 1 -- 172.21.15.146:0/2856929487 shutdown_connections 2024-09-13T18:48:08.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:08.396+0000 7f077a781700 1 -- 172.21.15.146:0/2856929487 wait complete. 2024-09-13T18:48:08.413 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 85 2024-09-13T18:48:08.422 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:48:10.424 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:10.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.516+0000 7f9bf0c0e700 1 Processor -- start 2024-09-13T18:48:10.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.516+0000 7f9bf0c0e700 1 -- start start 2024-09-13T18:48:10.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bf0c0e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9bec1572e0 0x7f9bec0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:10.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bf0c0e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9bec0b2d20 0x7f9bec06b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:10.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bf0c0e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bec06ae30 con 0x7f9bec1514b0 2024-09-13T18:48:10.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bf0c0e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bec1576c0 con 0x7f9bec1572e0 2024-09-13T18:48:10.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bf0c0e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bec0caa70 con 0x7f9bec0b2d20 2024-09-13T18:48:10.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9beaffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9bec1572e0 0x7f9bec0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:10.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bebfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9bec0b2d20 0x7f9bec06b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:10.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9beb7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9bec1514b0 0x7f9bec0b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:33564/0 (socket says 172.21.15.146:33564) 2024-09-13T18:48:10.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9beaffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9bec1572e0 0x7f9bec0b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50364/0 (socket says 172.21.15.146:50364) 2024-09-13T18:48:10.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9beb7fe700 1 -- 172.21.15.146:0/192657019 learned_addr learned my addr 172.21.15.146:0/192657019 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:48:10.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9beaffd700 1 -- 172.21.15.146:0/192657019 >> v2:172.21.15.154:3300/0 conn(0x7f9bec0b2d20 msgr2=0x7f9bec06b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:10.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9beaffd700 1 --2- 172.21.15.146:0/192657019 >> v2:172.21.15.154:3300/0 conn(0x7f9bec0b2d20 0x7f9bec06b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:10.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9beaffd700 1 -- 172.21.15.146:0/192657019 >> v1:172.21.15.154:6789/0 conn(0x7f9bec1514b0 legacy=0x7f9bec0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:10.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9beaffd700 1 -- 172.21.15.146:0/192657019 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec0b0b70 con 0x7f9bec1572e0 2024-09-13T18:48:10.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9beaffd700 1 --2- 172.21.15.146:0/192657019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bec1572e0 0x7f9bec0b27e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0701c0 tx=0x7f9be0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4589290b4b48c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:10.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bea7fc700 1 -- 172.21.15.146:0/192657019 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0011830 con 0x7f9bec1572e0 2024-09-13T18:48:10.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bea7fc700 1 -- 172.21.15.146:0/192657019 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be0011990 con 0x7f9bec1572e0 2024-09-13T18:48:10.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bea7fc700 1 -- 172.21.15.146:0/192657019 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0011d20 con 0x7f9bec1572e0 2024-09-13T18:48:10.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/192657019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bec1572e0 msgr2=0x7f9bec0b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:10.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bf0c0e700 1 --2- 172.21.15.146:0/192657019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bec1572e0 0x7f9bec0b27e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0701c0 tx=0x7f9be0009b30 comp rx=0 tx=0).stop 2024-09-13T18:48:10.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/192657019 shutdown_connections 2024-09-13T18:48:10.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bf0c0e700 1 --2- 172.21.15.146:0/192657019 >> v2:172.21.15.154:3300/0 conn(0x7f9bec0b2d20 0x7f9bec06b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:10.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bf0c0e700 1 --2- 172.21.15.146:0/192657019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bec1572e0 0x7f9bec0b27e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:10.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/192657019 >> 172.21.15.146:0/192657019 conn(0x7f9bec06c2d0 msgr2=0x7f9bec06c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:10.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/192657019 shutdown_connections 2024-09-13T18:48:10.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.520+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/192657019 wait complete. 2024-09-13T18:48:10.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bf0c0e700 1 Processor -- start 2024-09-13T18:48:10.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bf0c0e700 1 -- start start 2024-09-13T18:48:10.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bf0c0e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec0b2d20 0x7f9bec153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:10.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bf0c0e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bec1514b0 0x7f9bec154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:10.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bf0c0e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bec1572e0 0x7f9bec1523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:10.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bf0c0e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec153d70 con 0x7f9bec1572e0 2024-09-13T18:48:10.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bf0c0e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec151f60 con 0x7f9bec0b2d20 2024-09-13T18:48:10.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bf0c0e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec0c74e0 con 0x7f9bec1514b0 2024-09-13T18:48:10.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bebfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bec1572e0 0x7f9bec1523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:10.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bebfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bec1572e0 0x7f9bec1523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50376/0 (socket says 172.21.15.146:50376) 2024-09-13T18:48:10.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bebfff700 1 -- 172.21.15.146:0/3013819353 learned_addr learned my addr 172.21.15.146:0/3013819353 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:10.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9beb7fe700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec0b2d20 0x7f9bec153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:10.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9beaffd700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bec1514b0 0x7f9bec154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:10.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bebfff700 1 -- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bec1514b0 msgr2=0x7f9bec154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:10.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bebfff700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bec1514b0 0x7f9bec154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:10.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bebfff700 1 -- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec0b2d20 msgr2=0x7f9bec153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:10.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bebfff700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec0b2d20 0x7f9bec153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:10.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bebfff700 1 -- 172.21.15.146:0/3013819353 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be00094b0 con 0x7f9bec1572e0 2024-09-13T18:48:10.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9beb7fe700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec0b2d20 0x7f9bec153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:48:10.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bebfff700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bec1572e0 0x7f9bec1523f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009dd0 tx=0x7f9be400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:10.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bd3fff700 1 -- 172.21.15.146:0/3013819353 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be4009c40 con 0x7f9bec1572e0 2024-09-13T18:48:10.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bd3fff700 1 -- 172.21.15.146:0/3013819353 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be400f040 con 0x7f9bec1572e0 2024-09-13T18:48:10.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/3013819353 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bec0c7240 con 0x7f9bec1572e0 2024-09-13T18:48:10.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bd3fff700 1 -- 172.21.15.146:0/3013819353 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be4014480 con 0x7f9bec1572e0 2024-09-13T18:48:10.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.524+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/3013819353 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bec151bf0 con 0x7f9bec1572e0 2024-09-13T18:48:10.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.528+0000 7f9bd3fff700 1 -- 172.21.15.146:0/3013819353 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9be40145e0 con 0x7f9bec1572e0 2024-09-13T18:48:10.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.528+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/3013819353 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bec1efb50 con 0x7f9bec1572e0 2024-09-13T18:48:10.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.528+0000 7f9bd3fff700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9bd8066760 0x7f9bd8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:10.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.528+0000 7f9bd3fff700 1 -- 172.21.15.146:0/3013819353 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7f9be4085260 con 0x7f9bec1572e0 2024-09-13T18:48:10.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.528+0000 7f9beb7fe700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9bd8066760 0x7f9bd8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:10.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.528+0000 7f9beb7fe700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9bd8066760 0x7f9bd8068c20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc007fd0 tx=0x7f9bdc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:10.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.532+0000 7f9bd3fff700 1 -- 172.21.15.146:0/3013819353 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9be4051e60 con 0x7f9bec1572e0 2024-09-13T18:48:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.768+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/3013819353 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bec1520d0 con 0x7f9bec1572e0 2024-09-13T18:48:10.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.772+0000 7f9bd3fff700 1 -- 172.21.15.146:0/3013819353 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f9be4055480 con 0x7f9bec1572e0 2024-09-13T18:48:10.787 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:10.787 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6714,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:16.846531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.772+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9bd8066760 msgr2=0x7f9bd8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.772+0000 7f9bf0c0e700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9bd8066760 0x7f9bd8068c20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc007fd0 tx=0x7f9bdc009040 comp rx=0 tx=0).stop 2024-09-13T18:48:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.772+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/3013819353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bec1572e0 msgr2=0x7f9bec1523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.772+0000 7f9bf0c0e700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bec1572e0 0x7f9bec1523f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009dd0 tx=0x7f9be400c3a0 comp rx=0 tx=0).stop 2024-09-13T18:48:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.772+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/3013819353 shutdown_connections 2024-09-13T18:48:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.772+0000 7f9bf0c0e700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9bd8066760 0x7f9bd8068c20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.772+0000 7f9bf0c0e700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bec1572e0 0x7f9bec1523f0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.772+0000 7f9bf0c0e700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bec1514b0 0x7f9bec154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.772+0000 7f9bf0c0e700 1 --2- 172.21.15.146:0/3013819353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec0b2d20 0x7f9bec153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.772+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/3013819353 >> 172.21.15.146:0/3013819353 conn(0x7f9bec06c2d0 msgr2=0x7f9bec0c33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.772+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/3013819353 shutdown_connections 2024-09-13T18:48:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:10.772+0000 7f9bf0c0e700 1 -- 172.21.15.146:0/3013819353 wait complete. 2024-09-13T18:48:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 85 2024-09-13T18:48:10.803 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:48:10.917 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:48:10.919 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:48:12.804 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:12.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.912+0000 7f5c5f56c700 1 Processor -- start 2024-09-13T18:48:12.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.912+0000 7f5c5f56c700 1 -- start start 2024-09-13T18:48:12.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5f56c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c58153a60 0x7f5c58159700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:12.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5f56c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c580c9ed0 0x7f5c580b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:12.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5f56c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c580cc2d0 con 0x7f5c580cf540 2024-09-13T18:48:12.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5f56c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c58153e40 con 0x7f5c58153a60 2024-09-13T18:48:12.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5f56c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c580cb3e0 con 0x7f5c580c9ed0 2024-09-13T18:48:12.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5ed6b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5c580cf540 0x7f5c5815cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48536/0 (socket says 172.21.15.146:48536) 2024-09-13T18:48:12.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5e56a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c58153a60 0x7f5c58159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:12.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5ed6b700 1 -- 172.21.15.146:0/2593960312 learned_addr learned my addr 172.21.15.146:0/2593960312 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:48:12.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5dd69700 1 --2- 172.21.15.146:0/2593960312 >> v2:172.21.15.154:3300/0 conn(0x7f5c580c9ed0 0x7f5c580b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:12.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5e56a700 1 -- 172.21.15.146:0/2593960312 >> v2:172.21.15.154:3300/0 conn(0x7f5c580c9ed0 msgr2=0x7f5c580b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:12.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5e56a700 1 --2- 172.21.15.146:0/2593960312 >> v2:172.21.15.154:3300/0 conn(0x7f5c580c9ed0 0x7f5c580b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:12.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5e56a700 1 -- 172.21.15.146:0/2593960312 >> v1:172.21.15.146:6789/0 conn(0x7f5c580cf540 legacy=0x7f5c5815cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:12.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5e56a700 1 -- 172.21.15.146:0/2593960312 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c580cb6c0 con 0x7f5c58153a60 2024-09-13T18:48:12.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5e56a700 1 --2- 172.21.15.146:0/2593960312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c58153a60 0x7f5c58159700 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5c580c60b0 tx=0x7f5c54009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0280dd16aa9fb89 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:12.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5d568700 1 -- 172.21.15.146:0/2593960312 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54011830 con 0x7f5c58153a60 2024-09-13T18:48:12.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5d568700 1 -- 172.21.15.146:0/2593960312 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c54011990 con 0x7f5c58153a60 2024-09-13T18:48:12.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5d568700 1 -- 172.21.15.146:0/2593960312 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54011d40 con 0x7f5c58153a60 2024-09-13T18:48:12.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5f56c700 1 -- 172.21.15.146:0/2593960312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c58153a60 msgr2=0x7f5c58159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:12.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5f56c700 1 --2- 172.21.15.146:0/2593960312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c58153a60 0x7f5c58159700 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5c580c60b0 tx=0x7f5c54009b30 comp rx=0 tx=0).stop 2024-09-13T18:48:12.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5f56c700 1 -- 172.21.15.146:0/2593960312 shutdown_connections 2024-09-13T18:48:12.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5f56c700 1 --2- 172.21.15.146:0/2593960312 >> v2:172.21.15.154:3300/0 conn(0x7f5c580c9ed0 0x7f5c580b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:12.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5f56c700 1 --2- 172.21.15.146:0/2593960312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c58153a60 0x7f5c58159700 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:12.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5f56c700 1 -- 172.21.15.146:0/2593960312 >> 172.21.15.146:0/2593960312 conn(0x7f5c58071ec0 msgr2=0x7f5c580722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:12.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5f56c700 1 -- 172.21.15.146:0/2593960312 shutdown_connections 2024-09-13T18:48:12.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.916+0000 7f5c5f56c700 1 -- 172.21.15.146:0/2593960312 wait complete. 2024-09-13T18:48:12.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5f56c700 1 Processor -- start 2024-09-13T18:48:12.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5f56c700 1 -- start start 2024-09-13T18:48:12.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5f56c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c580cf540 0x7f5c5814f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:12.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5f56c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c58153a60 0x7f5c580b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:12.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5f56c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c5814ead0 0x7f5c5814c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:12.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5f56c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580ad080 con 0x7f5c5814ead0 2024-09-13T18:48:12.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5f56c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c581efad0 con 0x7f5c58153a60 2024-09-13T18:48:12.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5f56c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5814fc10 con 0x7f5c580cf540 2024-09-13T18:48:12.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5ed6b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c5814ead0 0x7f5c5814c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:12.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5ed6b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c5814ead0 0x7f5c5814c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49100/0 (socket says 172.21.15.146:49100) 2024-09-13T18:48:12.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5ed6b700 1 -- 172.21.15.146:0/3814482837 learned_addr learned my addr 172.21.15.146:0/3814482837 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:12.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5dd69700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c58153a60 0x7f5c580b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:12.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5e56a700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c580cf540 0x7f5c5814f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:12.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.920+0000 7f5c5ed6b700 1 -- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c580cf540 msgr2=0x7f5c5814f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:12.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.928+0000 7f5c5ed6b700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c580cf540 0x7f5c5814f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:12.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.928+0000 7f5c5ed6b700 1 -- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c58153a60 msgr2=0x7f5c580b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:12.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.928+0000 7f5c5ed6b700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c58153a60 0x7f5c580b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:12.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.928+0000 7f5c5ed6b700 1 -- 172.21.15.146:0/3814482837 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c540094b0 con 0x7f5c5814ead0 2024-09-13T18:48:12.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.928+0000 7f5c5dd69700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c58153a60 0x7f5c580b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:48:12.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.928+0000 7f5c5e56a700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c580cf540 0x7f5c5814f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:48:12.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.932+0000 7f5c5ed6b700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c5814ead0 0x7f5c5814c930 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000ca80 tx=0x7f5c5000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:12.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.932+0000 7f5c4effd700 1 -- 172.21.15.146:0/3814482837 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50007480 con 0x7f5c5814ead0 2024-09-13T18:48:12.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.932+0000 7f5c4effd700 1 -- 172.21.15.146:0/3814482837 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c500075e0 con 0x7f5c5814ead0 2024-09-13T18:48:12.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.932+0000 7f5c4effd700 1 -- 172.21.15.146:0/3814482837 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50007950 con 0x7f5c5814ead0 2024-09-13T18:48:12.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.932+0000 7f5c5f56c700 1 -- 172.21.15.146:0/3814482837 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c581eeec0 con 0x7f5c5814ead0 2024-09-13T18:48:12.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.932+0000 7f5c5f56c700 1 -- 172.21.15.146:0/3814482837 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c581ef480 con 0x7f5c5814ead0 2024-09-13T18:48:12.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.932+0000 7f5c5f56c700 1 -- 172.21.15.146:0/3814482837 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c581efc80 con 0x7f5c5814ead0 2024-09-13T18:48:12.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.932+0000 7f5c4effd700 1 -- 172.21.15.146:0/3814482837 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5c5001b3b0 con 0x7f5c5814ead0 2024-09-13T18:48:12.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.932+0000 7f5c4effd700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c3c066730 0x7f5c3c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:12.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.932+0000 7f5c4effd700 1 -- 172.21.15.146:0/3814482837 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7f5c50084070 con 0x7f5c5814ead0 2024-09-13T18:48:12.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.932+0000 7f5c5e56a700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c3c066730 0x7f5c3c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:12.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.936+0000 7f5c5e56a700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c3c066730 0x7f5c3c068bf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009b70 tx=0x7f5c54008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:12.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:12.940+0000 7f5c4effd700 1 -- 172.21.15.146:0/3814482837 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5c581efc80 con 0x7f5c5814ead0 2024-09-13T18:48:13.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.168+0000 7f5c5f56c700 1 -- 172.21.15.146:0/3814482837 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c581efc80 con 0x7f5c5814ead0 2024-09-13T18:48:13.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.168+0000 7f5c4effd700 1 -- 172.21.15.146:0/3814482837 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f5c581efc80 con 0x7f5c5814ead0 2024-09-13T18:48:13.186 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:13.186 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6714,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:16.846531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:13.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.172+0000 7f5c5f56c700 1 -- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c3c066730 msgr2=0x7f5c3c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:13.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.172+0000 7f5c5f56c700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c3c066730 0x7f5c3c068bf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009b70 tx=0x7f5c54008030 comp rx=0 tx=0).stop 2024-09-13T18:48:13.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.172+0000 7f5c5f56c700 1 -- 172.21.15.146:0/3814482837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c5814ead0 msgr2=0x7f5c5814c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:13.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.172+0000 7f5c5f56c700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c5814ead0 0x7f5c5814c930 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000ca80 tx=0x7f5c5000cab0 comp rx=0 tx=0).stop 2024-09-13T18:48:13.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.172+0000 7f5c5f56c700 1 -- 172.21.15.146:0/3814482837 shutdown_connections 2024-09-13T18:48:13.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.172+0000 7f5c5f56c700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c3c066730 0x7f5c3c068bf0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:13.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.172+0000 7f5c5f56c700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c5814ead0 0x7f5c5814c930 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:13.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.172+0000 7f5c5f56c700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c58153a60 0x7f5c580b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:13.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.172+0000 7f5c5f56c700 1 --2- 172.21.15.146:0/3814482837 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c580cf540 0x7f5c5814f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:13.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.172+0000 7f5c5f56c700 1 -- 172.21.15.146:0/3814482837 >> 172.21.15.146:0/3814482837 conn(0x7f5c58071ec0 msgr2=0x7f5c580cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:13.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.172+0000 7f5c5f56c700 1 -- 172.21.15.146:0/3814482837 shutdown_connections 2024-09-13T18:48:13.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:13.172+0000 7f5c5f56c700 1 -- 172.21.15.146:0/3814482837 wait complete. 2024-09-13T18:48:13.189 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 85 2024-09-13T18:48:13.202 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:48:15.204 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:15.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.328+0000 7fc622497700 1 Processor -- start 2024-09-13T18:48:15.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.328+0000 7fc622497700 1 -- start start 2024-09-13T18:48:15.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc622497700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc6140d5740 0x7fc6140dbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:15.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc622497700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc614008c40 0x7fc61404bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:15.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc622497700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc614045cf0 con 0x7fc614008860 2024-09-13T18:48:15.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc622497700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc614036e30 con 0x7fc6140d5740 2024-09-13T18:48:15.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc622497700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc614008440 con 0x7fc614008c40 2024-09-13T18:48:15.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc621c96700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc614008c40 0x7fc61404bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:15.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc621c96700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc614008c40 0x7fc61404bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:48918/0 (socket says 172.21.15.146:48918) 2024-09-13T18:48:15.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc621c96700 1 -- 172.21.15.146:0/3497121868 learned_addr learned my addr 172.21.15.146:0/3497121868 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:15.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc621495700 1 --2- 172.21.15.146:0/3497121868 >> v2:172.21.15.154:3300/0 conn(0x7fc6140d5740 0x7fc6140dbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:15.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc621c96700 1 -- 172.21.15.146:0/3497121868 >> v2:172.21.15.154:3300/0 conn(0x7fc6140d5740 msgr2=0x7fc6140dbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:15.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc621c96700 1 --2- 172.21.15.146:0/3497121868 >> v2:172.21.15.154:3300/0 conn(0x7fc6140d5740 0x7fc6140dbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:15.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc621c96700 1 -- 172.21.15.146:0/3497121868 >> v1:172.21.15.154:6789/0 conn(0x7fc614008860 legacy=0x7fc61404cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:15.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc621c96700 1 -- 172.21.15.146:0/3497121868 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc614007800 con 0x7fc614008c40 2024-09-13T18:48:15.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc621c96700 1 --2- 172.21.15.146:0/3497121868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc614008c40 0x7fc61404bb90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0731e0 tx=0x7fc61c061ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d12552c08e3eb41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:15.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc613fff700 1 -- 172.21.15.146:0/3497121868 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c073900 con 0x7fc614008c40 2024-09-13T18:48:15.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc613fff700 1 -- 172.21.15.146:0/3497121868 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c073a60 con 0x7fc614008c40 2024-09-13T18:48:15.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc613fff700 1 -- 172.21.15.146:0/3497121868 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c075c50 con 0x7fc614008c40 2024-09-13T18:48:15.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc622497700 1 -- 172.21.15.146:0/3497121868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc614008c40 msgr2=0x7fc61404bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:15.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc622497700 1 --2- 172.21.15.146:0/3497121868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc614008c40 0x7fc61404bb90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0731e0 tx=0x7fc61c061ef0 comp rx=0 tx=0).stop 2024-09-13T18:48:15.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc622497700 1 -- 172.21.15.146:0/3497121868 shutdown_connections 2024-09-13T18:48:15.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc622497700 1 --2- 172.21.15.146:0/3497121868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc614008c40 0x7fc61404bb90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:15.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc622497700 1 --2- 172.21.15.146:0/3497121868 >> v2:172.21.15.154:3300/0 conn(0x7fc6140d5740 0x7fc6140dbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:15.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc622497700 1 -- 172.21.15.146:0/3497121868 >> 172.21.15.146:0/3497121868 conn(0x7fc614050e10 msgr2=0x7fc614051220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:15.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc622497700 1 -- 172.21.15.146:0/3497121868 shutdown_connections 2024-09-13T18:48:15.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.332+0000 7fc622497700 1 -- 172.21.15.146:0/3497121868 wait complete. 2024-09-13T18:48:15.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc622497700 1 Processor -- start 2024-09-13T18:48:15.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc622497700 1 -- start start 2024-09-13T18:48:15.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc622497700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc614008860 0x7fc6140d7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:15.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc622497700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc614008c40 0x7fc6140cf840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:15.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc622497700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6140d5740 0x7fc6140cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:15.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc622497700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6140d8a40 con 0x7fc614008860 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc622497700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6140d0aa0 con 0x7fc614008c40 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc622497700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6140cf3d0 con 0x7fc6140d5740 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc621495700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc614008860 0x7fc6140d7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc621495700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc614008860 0x7fc6140d7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49112/0 (socket says 172.21.15.146:49112) 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc621495700 1 -- 172.21.15.146:0/3538205010 learned_addr learned my addr 172.21.15.146:0/3538205010 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc620c94700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc614008c40 0x7fc6140cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc621c96700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6140d5740 0x7fc6140cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc621495700 1 -- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6140d5740 msgr2=0x7fc6140cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc621495700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6140d5740 0x7fc6140cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc621495700 1 -- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc614008c40 msgr2=0x7fc6140cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc621495700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc614008c40 0x7fc6140cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc621495700 1 -- 172.21.15.146:0/3538205010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c04f5c0 con 0x7fc614008860 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc621495700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc614008860 0x7fc6140d7e10 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc61800e840 tx=0x7fc61800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:15.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc611ffb700 1 -- 172.21.15.146:0/3538205010 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6180095a0 con 0x7fc614008860 2024-09-13T18:48:15.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc611ffb700 1 -- 172.21.15.146:0/3538205010 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc618013680 con 0x7fc614008860 2024-09-13T18:48:15.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc622497700 1 -- 172.21.15.146:0/3538205010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6140cf0f0 con 0x7fc614008860 2024-09-13T18:48:15.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc611ffb700 1 -- 172.21.15.146:0/3538205010 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6180139f0 con 0x7fc614008860 2024-09-13T18:48:15.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.336+0000 7fc622497700 1 -- 172.21.15.146:0/3538205010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc614171930 con 0x7fc614008860 2024-09-13T18:48:15.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.340+0000 7fc622497700 1 -- 172.21.15.146:0/3538205010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc614171790 con 0x7fc614008860 2024-09-13T18:48:15.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.340+0000 7fc611ffb700 1 -- 172.21.15.146:0/3538205010 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc618013b50 con 0x7fc614008860 2024-09-13T18:48:15.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.340+0000 7fc611ffb700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc600066730 0x7fc600068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:15.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.340+0000 7fc611ffb700 1 -- 172.21.15.146:0/3538205010 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7fc618084b80 con 0x7fc614008860 2024-09-13T18:48:15.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.340+0000 7fc620c94700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc600066730 0x7fc600068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:15.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.340+0000 7fc620c94700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc600066730 0x7fc600068bf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc614050130 tx=0x7fc60c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:15.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.348+0000 7fc611ffb700 1 -- 172.21.15.146:0/3538205010 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc614171790 con 0x7fc614008860 2024-09-13T18:48:15.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.576+0000 7fc622497700 1 -- 172.21.15.146:0/3538205010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc614171790 con 0x7fc614008860 2024-09-13T18:48:15.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.576+0000 7fc611ffb700 1 -- 172.21.15.146:0/3538205010 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6276 (secure 0 0 0) 0x7fc614171790 con 0x7fc614008860 2024-09-13T18:48:15.593 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:15.593 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6714,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:16.846531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:15.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.580+0000 7fc622497700 1 -- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc600066730 msgr2=0x7fc600068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:15.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.580+0000 7fc622497700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc600066730 0x7fc600068bf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc614050130 tx=0x7fc60c009040 comp rx=0 tx=0).stop 2024-09-13T18:48:15.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.580+0000 7fc622497700 1 -- 172.21.15.146:0/3538205010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc614008860 msgr2=0x7fc6140d7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:15.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.580+0000 7fc622497700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc614008860 0x7fc6140d7e10 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc61800e840 tx=0x7fc61800eb50 comp rx=0 tx=0).stop 2024-09-13T18:48:15.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.580+0000 7fc622497700 1 -- 172.21.15.146:0/3538205010 shutdown_connections 2024-09-13T18:48:15.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.580+0000 7fc622497700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc600066730 0x7fc600068bf0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:15.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.580+0000 7fc622497700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6140d5740 0x7fc6140cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:15.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.580+0000 7fc622497700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc614008c40 0x7fc6140cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:15.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.580+0000 7fc622497700 1 --2- 172.21.15.146:0/3538205010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc614008860 0x7fc6140d7e10 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:15.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.580+0000 7fc622497700 1 -- 172.21.15.146:0/3538205010 >> 172.21.15.146:0/3538205010 conn(0x7fc614050e10 msgr2=0x7fc61404f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:15.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.580+0000 7fc622497700 1 -- 172.21.15.146:0/3538205010 shutdown_connections 2024-09-13T18:48:15.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:15.580+0000 7fc622497700 1 -- 172.21.15.146:0/3538205010 wait complete. 2024-09-13T18:48:15.598 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 85 2024-09-13T18:48:15.610 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:48:17.610 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:17.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.712+0000 7f3742b91700 1 Processor -- start 2024-09-13T18:48:17.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.712+0000 7f3742b91700 1 -- start start 2024-09-13T18:48:17.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742b91700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f37340d5730 0x7f37340df330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:17.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742b91700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3734045470 0x7f37340dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:17.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742b91700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3734033ab0 con 0x7f37340dbfb0 2024-09-13T18:48:17.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742b91700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37340db650 con 0x7f3734045470 2024-09-13T18:48:17.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742b91700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37340d5b10 con 0x7f37340d5730 2024-09-13T18:48:17.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f374138e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3734045470 0x7f37340dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:17.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742390700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f37340dbfb0 0x7f37340459e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48550/0 (socket says 172.21.15.146:48550) 2024-09-13T18:48:17.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742390700 1 -- 172.21.15.146:0/223882232 learned_addr learned my addr 172.21.15.146:0/223882232 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:48:17.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3741b8f700 1 --2- 172.21.15.146:0/223882232 >> v2:172.21.15.154:3300/0 conn(0x7f37340d5730 0x7f37340df330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:17.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f374138e700 1 -- 172.21.15.146:0/223882232 >> v2:172.21.15.154:3300/0 conn(0x7f37340d5730 msgr2=0x7f37340df330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:17.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f374138e700 1 --2- 172.21.15.146:0/223882232 >> v2:172.21.15.154:3300/0 conn(0x7f37340d5730 0x7f37340df330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:17.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f374138e700 1 -- 172.21.15.146:0/223882232 >> v1:172.21.15.146:6789/0 conn(0x7f37340dbfb0 legacy=0x7f37340459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:17.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f374138e700 1 -- 172.21.15.146:0/223882232 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3734005ea0 con 0x7f3734045470 2024-09-13T18:48:17.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3740b8d700 1 -- 172.21.15.146:0/223882232 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712456554 0 0) 0x7f3734033ab0 con 0x7f37340dbfb0 2024-09-13T18:48:17.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f374138e700 1 --2- 172.21.15.146:0/223882232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3734045470 0x7f37340dba70 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f373400a010 tx=0x7f372c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1ab8cc0c4b9683 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:17.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3740b8d700 1 -- 172.21.15.146:0/223882232 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c011830 con 0x7f3734045470 2024-09-13T18:48:17.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3740b8d700 1 -- 172.21.15.146:0/223882232 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c011990 con 0x7f3734045470 2024-09-13T18:48:17.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3740b8d700 1 -- 172.21.15.146:0/223882232 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c011d20 con 0x7f3734045470 2024-09-13T18:48:17.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742b91700 1 -- 172.21.15.146:0/223882232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3734045470 msgr2=0x7f37340dba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:17.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742b91700 1 --2- 172.21.15.146:0/223882232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3734045470 0x7f37340dba70 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f373400a010 tx=0x7f372c009b30 comp rx=0 tx=0).stop 2024-09-13T18:48:17.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742b91700 1 -- 172.21.15.146:0/223882232 shutdown_connections 2024-09-13T18:48:17.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742b91700 1 --2- 172.21.15.146:0/223882232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3734045470 0x7f37340dba70 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:17.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742b91700 1 --2- 172.21.15.146:0/223882232 >> v2:172.21.15.154:3300/0 conn(0x7f37340d5730 0x7f37340df330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:17.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742b91700 1 -- 172.21.15.146:0/223882232 >> 172.21.15.146:0/223882232 conn(0x7f3734050590 msgr2=0x7f37340509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:17.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742b91700 1 -- 172.21.15.146:0/223882232 shutdown_connections 2024-09-13T18:48:17.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.716+0000 7f3742b91700 1 -- 172.21.15.146:0/223882232 wait complete. 2024-09-13T18:48:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f3742b91700 1 Processor -- start 2024-09-13T18:48:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f3742b91700 1 -- start start 2024-09-13T18:48:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f3742b91700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3734045470 0x7f3734035ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f3742b91700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37340d5730 0x7f37340c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f3742b91700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37340dbfb0 0x7f37340c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f3742b91700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3734048a70 con 0x7f37340d5730 2024-09-13T18:48:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f3742b91700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37340c8020 con 0x7f37340dbfb0 2024-09-13T18:48:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f3742b91700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37340c92b0 con 0x7f3734045470 2024-09-13T18:48:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f374138e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37340d5730 0x7f37340c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f374138e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37340d5730 0x7f37340c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49120/0 (socket says 172.21.15.146:49120) 2024-09-13T18:48:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f374138e700 1 -- 172.21.15.146:0/1048079309 learned_addr learned my addr 172.21.15.146:0/1048079309 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f3741b8f700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3734045470 0x7f3734035ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f3742390700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37340dbfb0 0x7f37340c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f374138e700 1 -- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3734045470 msgr2=0x7f3734035ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f374138e700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3734045470 0x7f3734035ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f374138e700 1 -- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37340dbfb0 msgr2=0x7f37340c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f374138e700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37340dbfb0 0x7f37340c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f374138e700 1 -- 172.21.15.146:0/1048079309 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372c0094b0 con 0x7f37340d5730 2024-09-13T18:48:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f3742390700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37340dbfb0 0x7f37340c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:48:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f374138e700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37340d5730 0x7f37340c8830 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f372c009fd0 tx=0x7f372c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f37327fc700 1 -- 172.21.15.146:0/1048079309 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0045b0 con 0x7f37340d5730 2024-09-13T18:48:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f37327fc700 1 -- 172.21.15.146:0/1048079309 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c01f070 con 0x7f37340d5730 2024-09-13T18:48:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f3742b91700 1 -- 172.21.15.146:0/1048079309 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37340c8270 con 0x7f37340d5730 2024-09-13T18:48:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f37327fc700 1 -- 172.21.15.146:0/1048079309 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0038e0 con 0x7f37340d5730 2024-09-13T18:48:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.720+0000 7f3742b91700 1 -- 172.21.15.146:0/1048079309 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3734048c80 con 0x7f37340d5730 2024-09-13T18:48:17.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.724+0000 7f3742b91700 1 -- 172.21.15.146:0/1048079309 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37340078a0 con 0x7f37340d5730 2024-09-13T18:48:17.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.724+0000 7f37327fc700 1 -- 172.21.15.146:0/1048079309 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f372c021030 con 0x7f37340d5730 2024-09-13T18:48:17.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.724+0000 7f37327fc700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3720066730 0x7f3720068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:17.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.724+0000 7f37327fc700 1 -- 172.21.15.146:0/1048079309 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7f372c085840 con 0x7f37340d5730 2024-09-13T18:48:17.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.724+0000 7f3741b8f700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3720066730 0x7f3720068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:17.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.724+0000 7f3741b8f700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3720066730 0x7f3720068bf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f373800bd80 tx=0x7f373800b450 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:17.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.732+0000 7f37327fc700 1 -- 172.21.15.146:0/1048079309 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f372c052440 con 0x7f37340d5730 2024-09-13T18:48:17.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.964+0000 7f3742b91700 1 -- 172.21.15.146:0/1048079309 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3734007c20 con 0x7f37340d5730 2024-09-13T18:48:17.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.968+0000 7f37327fc700 1 -- 172.21.15.146:0/1048079309 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f372c003a40 con 0x7f37340d5730 2024-09-13T18:48:17.983 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:17.983 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6714,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:16.846531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:17.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.968+0000 7f3742b91700 1 -- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3720066730 msgr2=0x7f3720068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:17.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.968+0000 7f3742b91700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3720066730 0x7f3720068bf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f373800bd80 tx=0x7f373800b450 comp rx=0 tx=0).stop 2024-09-13T18:48:17.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.968+0000 7f3742b91700 1 -- 172.21.15.146:0/1048079309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37340d5730 msgr2=0x7f37340c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:17.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.968+0000 7f3742b91700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37340d5730 0x7f37340c8830 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f372c009fd0 tx=0x7f372c004770 comp rx=0 tx=0).stop 2024-09-13T18:48:17.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.968+0000 7f3742b91700 1 -- 172.21.15.146:0/1048079309 shutdown_connections 2024-09-13T18:48:17.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.968+0000 7f3742b91700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3720066730 0x7f3720068bf0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:17.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.968+0000 7f3742b91700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37340dbfb0 0x7f37340c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:17.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.968+0000 7f3742b91700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37340d5730 0x7f37340c8830 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:17.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.968+0000 7f3742b91700 1 --2- 172.21.15.146:0/1048079309 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3734045470 0x7f3734035ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:17.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.968+0000 7f3742b91700 1 -- 172.21.15.146:0/1048079309 >> 172.21.15.146:0/1048079309 conn(0x7f3734050590 msgr2=0x7f373404b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:17.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.968+0000 7f3742b91700 1 -- 172.21.15.146:0/1048079309 shutdown_connections 2024-09-13T18:48:17.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:17.968+0000 7f3742b91700 1 -- 172.21.15.146:0/1048079309 wait complete. 2024-09-13T18:48:17.986 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 85 2024-09-13T18:48:17.996 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:48:19.997 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:20.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.101+0000 7f0c50803700 1 Processor -- start 2024-09-13T18:48:20.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.101+0000 7f0c50803700 1 -- start start 2024-09-13T18:48:20.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c50803700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c4c0cbd50 0x7f0c4c0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:20.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c50803700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0c4c0ca210 0x7f0c4c0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:20.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c50803700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4c0cc670 con 0x7f0c4c071390 2024-09-13T18:48:20.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c50803700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4c0caa30 con 0x7f0c4c0cbd50 2024-09-13T18:48:20.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c50803700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4c0cb810 con 0x7f0c4c0ca210 2024-09-13T18:48:20.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c4ad9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c4c0cbd50 0x7f0c4c0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:20.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c4ad9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c4c0cbd50 0x7f0c4c0b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49132/0 (socket says 172.21.15.146:49132) 2024-09-13T18:48:20.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c4ad9d700 1 -- 172.21.15.146:0/1607499789 learned_addr learned my addr 172.21.15.146:0/1607499789 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:20.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c4a59c700 1 --2- 172.21.15.146:0/1607499789 >> v2:172.21.15.154:3300/0 conn(0x7f0c4c0ca210 0x7f0c4c0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:20.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c49d9b700 1 -- 172.21.15.146:0/1607499789 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413281389 0 0) 0x7f0c44001490 con 0x7f0c4c071390 2024-09-13T18:48:20.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c49d9b700 1 -- 172.21.15.146:0/1607499789 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c340035d0 con 0x7f0c4c071390 2024-09-13T18:48:20.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c4ad9d700 1 -- 172.21.15.146:0/1607499789 >> v2:172.21.15.154:3300/0 conn(0x7f0c4c0ca210 msgr2=0x7f0c4c0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:20.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c4ad9d700 1 --2- 172.21.15.146:0/1607499789 >> v2:172.21.15.154:3300/0 conn(0x7f0c4c0ca210 0x7f0c4c0cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:20.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c4ad9d700 1 -- 172.21.15.146:0/1607499789 >> v1:172.21.15.154:6789/0 conn(0x7f0c4c071390 legacy=0x7f0c4c0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:20.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c4ad9d700 1 -- 172.21.15.146:0/1607499789 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c15da20 con 0x7f0c4c0cbd50 2024-09-13T18:48:20.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c4ad9d700 1 --2- 172.21.15.146:0/1607499789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c4c0cbd50 0x7f0c4c0b1c40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0b18a0 tx=0x7f0c3c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b79fa4a7c911d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:20.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c49d9b700 1 -- 172.21.15.146:0/1607499789 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c011830 con 0x7f0c4c0cbd50 2024-09-13T18:48:20.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c49d9b700 1 -- 172.21.15.146:0/1607499789 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c011990 con 0x7f0c4c0cbd50 2024-09-13T18:48:20.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c49d9b700 1 -- 172.21.15.146:0/1607499789 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c011d20 con 0x7f0c4c0cbd50 2024-09-13T18:48:20.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c50803700 1 -- 172.21.15.146:0/1607499789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c4c0cbd50 msgr2=0x7f0c4c0b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:20.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c50803700 1 --2- 172.21.15.146:0/1607499789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c4c0cbd50 0x7f0c4c0b1c40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0b18a0 tx=0x7f0c3c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:48:20.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c50803700 1 -- 172.21.15.146:0/1607499789 shutdown_connections 2024-09-13T18:48:20.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c50803700 1 --2- 172.21.15.146:0/1607499789 >> v2:172.21.15.154:3300/0 conn(0x7f0c4c0ca210 0x7f0c4c0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:20.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c50803700 1 --2- 172.21.15.146:0/1607499789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c4c0cbd50 0x7f0c4c0b1c40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:20.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c50803700 1 -- 172.21.15.146:0/1607499789 >> 172.21.15.146:0/1607499789 conn(0x7f0c4c066270 msgr2=0x7f0c4c066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:20.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c50803700 1 -- 172.21.15.146:0/1607499789 shutdown_connections 2024-09-13T18:48:20.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.105+0000 7f0c50803700 1 -- 172.21.15.146:0/1607499789 wait complete. 2024-09-13T18:48:20.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c50803700 1 Processor -- start 2024-09-13T18:48:20.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c50803700 1 -- start start 2024-09-13T18:48:20.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c50803700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c4c071390 0x7f0c4c1529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:20.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c50803700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c4c0ca210 0x7f0c4c152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:20.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c50803700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c4c0cbd50 0x7f0c4c1e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:20.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c50803700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c0b37c0 con 0x7f0c4c071390 2024-09-13T18:48:20.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c50803700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c1e67c0 con 0x7f0c4c0cbd50 2024-09-13T18:48:20.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c50803700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c1efa70 con 0x7f0c4c0ca210 2024-09-13T18:48:20.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c4ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c4c071390 0x7f0c4c1529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:20.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c4ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c4c071390 0x7f0c4c1529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49148/0 (socket says 172.21.15.146:49148) 2024-09-13T18:48:20.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c4ad9d700 1 -- 172.21.15.146:0/1137246154 learned_addr learned my addr 172.21.15.146:0/1137246154 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:20.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c4a59c700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c4c0ca210 0x7f0c4c152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c4b59e700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c4c0cbd50 0x7f0c4c1e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c4ad9d700 1 -- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c4c0ca210 msgr2=0x7f0c4c152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c4ad9d700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c4c0ca210 0x7f0c4c152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c4ad9d700 1 -- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c4c0cbd50 msgr2=0x7f0c4c1e5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c4ad9d700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c4c0cbd50 0x7f0c4c1e5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c4ad9d700 1 -- 172.21.15.146:0/1137246154 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c009570 con 0x7f0c4c071390 2024-09-13T18:48:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.109+0000 7f0c4ad9d700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c4c071390 0x7f0c4c1529f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c000c00 tx=0x7f0c3c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.113+0000 7f0c337fe700 1 -- 172.21.15.146:0/1137246154 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c004510 con 0x7f0c4c071390 2024-09-13T18:48:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.113+0000 7f0c337fe700 1 -- 172.21.15.146:0/1137246154 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c01e070 con 0x7f0c4c071390 2024-09-13T18:48:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.113+0000 7f0c50803700 1 -- 172.21.15.146:0/1137246154 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4c1efd50 con 0x7f0c4c071390 2024-09-13T18:48:20.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.113+0000 7f0c337fe700 1 -- 172.21.15.146:0/1137246154 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c003900 con 0x7f0c4c071390 2024-09-13T18:48:20.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.113+0000 7f0c50803700 1 -- 172.21.15.146:0/1137246154 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4c1f0310 con 0x7f0c4c071390 2024-09-13T18:48:20.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.113+0000 7f0c50803700 1 -- 172.21.15.146:0/1137246154 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4c15d500 con 0x7f0c4c071390 2024-09-13T18:48:20.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.113+0000 7f0c337fe700 1 -- 172.21.15.146:0/1137246154 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0c3c003a60 con 0x7f0c4c071390 2024-09-13T18:48:20.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.113+0000 7f0c337fe700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c38066730 0x7f0c38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:20.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.113+0000 7f0c337fe700 1 -- 172.21.15.146:0/1137246154 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7609+0+0 (secure 0 0 0) 0x7f0c3c020070 con 0x7f0c4c071390 2024-09-13T18:48:20.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.113+0000 7f0c4a59c700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c38066730 0x7f0c38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:20.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.113+0000 7f0c4a59c700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c38066730 0x7f0c38068bf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c14da60 tx=0x7f0c40009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:20.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.121+0000 7f0c337fe700 1 -- 172.21.15.146:0/1137246154 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0c3c052830 con 0x7f0c4c071390 2024-09-13T18:48:20.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c50803700 1 -- 172.21.15.146:0/1137246154 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c4c151370 con 0x7f0c4c071390 2024-09-13T18:48:20.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c337fe700 1 -- 172.21.15.146:0/1137246154 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6276 (secure 0 0 0) 0x7f0c3c055e50 con 0x7f0c4c071390 2024-09-13T18:48:20.376 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:20.376 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6714,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:46:16.846531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6370},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6370":{"gid":6370,"name":"f","rank":1,"incarnation":80,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/925899756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":925899756},{"type":"v1","addr":"172.21.15.154:6839","nonce":925899756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:20.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c50803700 1 -- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c38066730 msgr2=0x7f0c38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:20.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c50803700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c38066730 0x7f0c38068bf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c14da60 tx=0x7f0c40009240 comp rx=0 tx=0).stop 2024-09-13T18:48:20.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c50803700 1 -- 172.21.15.146:0/1137246154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c4c071390 msgr2=0x7f0c4c1529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:20.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c50803700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c4c071390 0x7f0c4c1529f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c000c00 tx=0x7f0c3c004770 comp rx=0 tx=0).stop 2024-09-13T18:48:20.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c50803700 1 -- 172.21.15.146:0/1137246154 shutdown_connections 2024-09-13T18:48:20.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c50803700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c38066730 0x7f0c38068bf0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:20.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c50803700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c4c0cbd50 0x7f0c4c1e5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:20.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c50803700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c4c0ca210 0x7f0c4c152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:20.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c50803700 1 --2- 172.21.15.146:0/1137246154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c4c071390 0x7f0c4c1529f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:20.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c50803700 1 -- 172.21.15.146:0/1137246154 >> 172.21.15.146:0/1137246154 conn(0x7f0c4c066270 msgr2=0x7f0c4c0cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:20.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c50803700 1 -- 172.21.15.146:0/1137246154 shutdown_connections 2024-09-13T18:48:20.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:20.361+0000 7f0c50803700 1 -- 172.21.15.146:0/1137246154 wait complete. 2024-09-13T18:48:20.379 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 85 2024-09-13T18:48:20.391 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:48:22.391 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:22.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.505+0000 7f78f4a2d700 1 Processor -- start 2024-09-13T18:48:22.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 -- start start 2024-09-13T18:48:22.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f78f00c5fc0 0x7f78f00c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:22.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f0159d70 0x7f78f00c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:22.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f00c8c70 con 0x7f78f00c63a0 2024-09-13T18:48:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f00c9bd0 con 0x7f78f00c5fc0 2024-09-13T18:48:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f00c8f50 con 0x7f78f0159d70 2024-09-13T18:48:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78eed9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f78f00c5fc0 0x7f78f00c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78eed9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f78f00c5fc0 0x7f78f00c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:52320/0 (socket says 172.21.15.146:52320) 2024-09-13T18:48:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78eed9d700 1 -- 172.21.15.146:0/4127462839 learned_addr learned my addr 172.21.15.146:0/4127462839 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78ef59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f0159d70 0x7f78f00c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78e7fff700 1 --1- 172.21.15.146:0/4127462839 >> v1:172.21.15.154:6789/0 conn(0x7f78f00c63a0 0x7f78f0159660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:46446/0 (socket says 172.21.15.146:46446) 2024-09-13T18:48:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78eed9d700 1 -- 172.21.15.146:0/4127462839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f0159d70 msgr2=0x7f78f00c7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78eed9d700 1 --2- 172.21.15.146:0/4127462839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f0159d70 0x7f78f00c7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78eed9d700 1 -- 172.21.15.146:0/4127462839 >> v1:172.21.15.154:6789/0 conn(0x7f78f00c63a0 legacy=0x7f78f0159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78eed9d700 1 -- 172.21.15.146:0/4127462839 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f00c9230 con 0x7f78f00c5fc0 2024-09-13T18:48:22.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78eed9d700 1 --2- 172.21.15.146:0/4127462839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f00c5fc0 0x7f78f00c6c00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f78e0006fd0 tx=0x7f78e000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4bd51813f71ec95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78ee59c700 1 -- 172.21.15.146:0/4127462839 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e000dc20 con 0x7f78f00c5fc0 2024-09-13T18:48:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78ee59c700 1 -- 172.21.15.146:0/4127462839 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e000dd80 con 0x7f78f00c5fc0 2024-09-13T18:48:22.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78ee59c700 1 -- 172.21.15.146:0/4127462839 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e0004250 con 0x7f78f00c5fc0 2024-09-13T18:48:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 -- 172.21.15.146:0/4127462839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f00c5fc0 msgr2=0x7f78f00c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 --2- 172.21.15.146:0/4127462839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f00c5fc0 0x7f78f00c6c00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f78e0006fd0 tx=0x7f78e000bea0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 -- 172.21.15.146:0/4127462839 shutdown_connections 2024-09-13T18:48:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 --2- 172.21.15.146:0/4127462839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f0159d70 0x7f78f00c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 --2- 172.21.15.146:0/4127462839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f00c5fc0 0x7f78f00c6c00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 -- 172.21.15.146:0/4127462839 >> 172.21.15.146:0/4127462839 conn(0x7f78f0066090 msgr2=0x7f78f0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 -- 172.21.15.146:0/4127462839 shutdown_connections 2024-09-13T18:48:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.509+0000 7f78f4a2d700 1 -- 172.21.15.146:0/4127462839 wait complete. 2024-09-13T18:48:22.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78f4a2d700 1 Processor -- start 2024-09-13T18:48:22.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78f4a2d700 1 -- start start 2024-09-13T18:48:22.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78f4a2d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f00c5fc0 0x7f78f0152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78f4a2d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f00c63a0 0x7f78f00c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78f4a2d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f0159d70 0x7f78f01edc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78f4a2d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f00c2950 con 0x7f78f00c63a0 2024-09-13T18:48:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78f4a2d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f0152dc0 con 0x7f78f00c5fc0 2024-09-13T18:48:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78f4a2d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f00a9490 con 0x7f78f0159d70 2024-09-13T18:48:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f00c63a0 0x7f78f00c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f00c63a0 0x7f78f00c10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39616/0 (socket says 172.21.15.146:39616) 2024-09-13T18:48:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78e7fff700 1 -- 172.21.15.146:0/570756014 learned_addr learned my addr 172.21.15.146:0/570756014 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:22.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78eed9d700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f00c5fc0 0x7f78f0152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78ef59e700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f0159d70 0x7f78f01edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78e7fff700 1 -- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f0159d70 msgr2=0x7f78f01edc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78e7fff700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f0159d70 0x7f78f01edc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78e7fff700 1 -- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f00c5fc0 msgr2=0x7f78f0152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78e7fff700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f00c5fc0 0x7f78f0152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.513+0000 7f78e7fff700 1 -- 172.21.15.146:0/570756014 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e000b820 con 0x7f78f00c63a0 2024-09-13T18:48:22.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.517+0000 7f78e7fff700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f00c63a0 0x7f78f00c10a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f78d800c6b0 tx=0x7f78d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:22.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.517+0000 7f78e77fe700 1 -- 172.21.15.146:0/570756014 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8005cd0 con 0x7f78f00c63a0 2024-09-13T18:48:22.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.517+0000 7f78e77fe700 1 -- 172.21.15.146:0/570756014 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d8005e30 con 0x7f78f00c63a0 2024-09-13T18:48:22.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.517+0000 7f78e77fe700 1 -- 172.21.15.146:0/570756014 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d80072d0 con 0x7f78f00c63a0 2024-09-13T18:48:22.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.517+0000 7f78f4a2d700 1 -- 172.21.15.146:0/570756014 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f00c20d0 con 0x7f78f00c63a0 2024-09-13T18:48:22.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.517+0000 7f78f4a2d700 1 -- 172.21.15.146:0/570756014 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f00c2670 con 0x7f78f00c63a0 2024-09-13T18:48:22.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.517+0000 7f78f4a2d700 1 -- 172.21.15.146:0/570756014 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f00cb790 con 0x7f78f00c63a0 2024-09-13T18:48:22.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.517+0000 7f78e77fe700 1 -- 172.21.15.146:0/570756014 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f78d8007430 con 0x7f78f00c63a0 2024-09-13T18:48:22.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.517+0000 7f78e77fe700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f78d0066780 0x7f78d0068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:22.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.517+0000 7f78e77fe700 1 -- 172.21.15.146:0/570756014 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f78d8083ad0 con 0x7f78f00c63a0 2024-09-13T18:48:22.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.517+0000 7f78eed9d700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f78d0066780 0x7f78d0068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:22.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.521+0000 7f78eed9d700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f78d0066780 0x7f78d0068c40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f78e0019040 tx=0x7f78e0014000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:22.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.525+0000 7f78e77fe700 1 -- 172.21.15.146:0/570756014 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f78d8050810 con 0x7f78f00c63a0 2024-09-13T18:48:22.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.757+0000 7f78f4a2d700 1 -- 172.21.15.146:0/570756014 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78f006f5c0 con 0x7f78f00c63a0 2024-09-13T18:48:22.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.757+0000 7f78e77fe700 1 -- 172.21.15.146:0/570756014 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+5518 (secure 0 0 0) 0x7f78d805a030 con 0x7f78f00c63a0 2024-09-13T18:48:22.772 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:22.772 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:20.845594+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:22.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.757+0000 7f78f4a2d700 1 -- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f78d0066780 msgr2=0x7f78d0068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:22.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.757+0000 7f78f4a2d700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f78d0066780 0x7f78d0068c40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f78e0019040 tx=0x7f78e0014000 comp rx=0 tx=0).stop 2024-09-13T18:48:22.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.757+0000 7f78f4a2d700 1 -- 172.21.15.146:0/570756014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f00c63a0 msgr2=0x7f78f00c10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:22.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.757+0000 7f78f4a2d700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f00c63a0 0x7f78f00c10a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f78d800c6b0 tx=0x7f78d800c9c0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.757+0000 7f78f4a2d700 1 -- 172.21.15.146:0/570756014 shutdown_connections 2024-09-13T18:48:22.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.757+0000 7f78f4a2d700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f78d0066780 0x7f78d0068c40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.761+0000 7f78f4a2d700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78f0159d70 0x7f78f01edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.761+0000 7f78f4a2d700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78f00c63a0 0x7f78f00c10a0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.761+0000 7f78f4a2d700 1 --2- 172.21.15.146:0/570756014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78f00c5fc0 0x7f78f0152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.761+0000 7f78f4a2d700 1 -- 172.21.15.146:0/570756014 >> 172.21.15.146:0/570756014 conn(0x7f78f0066090 msgr2=0x7f78f0155b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:22.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.761+0000 7f78f4a2d700 1 -- 172.21.15.146:0/570756014 shutdown_connections 2024-09-13T18:48:22.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.761+0000 7f78f4a2d700 1 -- 172.21.15.146:0/570756014 wait complete. 2024-09-13T18:48:22.776 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 86 2024-09-13T18:48:22.787 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2024-09-13T18:48:22.787 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:48:22.788 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:22.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.877+0000 7f946d892700 1 Processor -- start 2024-09-13T18:48:22.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 -- start start 2024-09-13T18:48:22.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9468151590 0x7f94681587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:22.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9468157fd0 0x7f94680c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:22.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9468158d10 con 0x7f946806db40 2024-09-13T18:48:22.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94680c69b0 con 0x7f9468151590 2024-09-13T18:48:22.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94681572e0 con 0x7f9468157fd0 2024-09-13T18:48:22.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946c890700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9468151590 0x7f94681587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:22.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946c890700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9468151590 0x7f94681587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39624/0 (socket says 172.21.15.146:39624) 2024-09-13T18:48:22.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946c890700 1 -- 172.21.15.146:0/2766714222 learned_addr learned my addr 172.21.15.146:0/2766714222 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:22.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f9467fff700 1 --2- 172.21.15.146:0/2766714222 >> v2:172.21.15.154:3300/0 conn(0x7f9468157fd0 0x7f94680c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:22.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946c890700 1 -- 172.21.15.146:0/2766714222 >> v2:172.21.15.154:3300/0 conn(0x7f9468157fd0 msgr2=0x7f94680c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:22.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946c890700 1 --2- 172.21.15.146:0/2766714222 >> v2:172.21.15.154:3300/0 conn(0x7f9468157fd0 0x7f94680c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946c890700 1 -- 172.21.15.146:0/2766714222 >> v1:172.21.15.146:6789/0 conn(0x7f946806db40 legacy=0x7f946815a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:22.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946c890700 1 -- 172.21.15.146:0/2766714222 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9468066500 con 0x7f9468151590 2024-09-13T18:48:22.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f94677fe700 1 -- 172.21.15.146:0/2766714222 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978320098 0 0) 0x7f9468158d10 con 0x7f946806db40 2024-09-13T18:48:22.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f9467fff700 1 --2- 172.21.15.146:0/2766714222 >> v2:172.21.15.154:3300/0 conn(0x7f9468157fd0 0x7f94680c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:48:22.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946c890700 1 --2- 172.21.15.146:0/2766714222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9468151590 0x7f94681587d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f94680c2890 tx=0x7f945c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d06f31e726b41be server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:22.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f94677fe700 1 -- 172.21.15.146:0/2766714222 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00b8a0 con 0x7f9468151590 2024-09-13T18:48:22.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f94677fe700 1 -- 172.21.15.146:0/2766714222 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c00ba00 con 0x7f9468151590 2024-09-13T18:48:22.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f94677fe700 1 -- 172.21.15.146:0/2766714222 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00bd90 con 0x7f9468151590 2024-09-13T18:48:22.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 -- 172.21.15.146:0/2766714222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9468151590 msgr2=0x7f94681587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:22.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 --2- 172.21.15.146:0/2766714222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9468151590 0x7f94681587d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f94680c2890 tx=0x7f945c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 -- 172.21.15.146:0/2766714222 shutdown_connections 2024-09-13T18:48:22.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 --2- 172.21.15.146:0/2766714222 >> v2:172.21.15.154:3300/0 conn(0x7f9468157fd0 0x7f94680c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 --2- 172.21.15.146:0/2766714222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9468151590 0x7f94681587d0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 -- 172.21.15.146:0/2766714222 >> 172.21.15.146:0/2766714222 conn(0x7f9468066b00 msgr2=0x7f946806aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:22.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 -- 172.21.15.146:0/2766714222 shutdown_connections 2024-09-13T18:48:22.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.881+0000 7f946d892700 1 -- 172.21.15.146:0/2766714222 wait complete. 2024-09-13T18:48:22.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f946d892700 1 Processor -- start 2024-09-13T18:48:22.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f946d892700 1 -- start start 2024-09-13T18:48:22.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f946d892700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f946806db40 0x7f94680c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:22.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f946d892700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9468151590 0x7f94681ea7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:22.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f946d892700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9468157fd0 0x7f94680c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:22.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f946d892700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94680c2530 con 0x7f9468151590 2024-09-13T18:48:22.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f946d892700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9468154c30 con 0x7f946806db40 2024-09-13T18:48:22.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f946d892700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94681ee510 con 0x7f9468157fd0 2024-09-13T18:48:22.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f9467fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9468151590 0x7f94681ea7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:22.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f946c890700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f946806db40 0x7f94680c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:22.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f9467fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9468151590 0x7f94681ea7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39640/0 (socket says 172.21.15.146:39640) 2024-09-13T18:48:22.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f946c890700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f946806db40 0x7f94680c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:52356/0 (socket says 172.21.15.146:52356) 2024-09-13T18:48:22.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f9467fff700 1 -- 172.21.15.146:0/3440824484 learned_addr learned my addr 172.21.15.146:0/3440824484 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:22.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f946d091700 1 --2- 172.21.15.146:0/3440824484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9468157fd0 0x7f94680c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:22.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f9467fff700 1 -- 172.21.15.146:0/3440824484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9468157fd0 msgr2=0x7f94680c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:22.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f9467fff700 1 --2- 172.21.15.146:0/3440824484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9468157fd0 0x7f94680c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f9467fff700 1 -- 172.21.15.146:0/3440824484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f946806db40 msgr2=0x7f94680c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:22.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f9467fff700 1 --2- 172.21.15.146:0/3440824484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f946806db40 0x7f94680c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:22.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f9467fff700 1 -- 172.21.15.146:0/3440824484 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c009570 con 0x7f9468151590 2024-09-13T18:48:22.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f9467fff700 1 --2- 172.21.15.146:0/3440824484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9468151590 0x7f94681ea7b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9454009a70 tx=0x7f9454009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:22.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f94657fa700 1 -- 172.21.15.146:0/3440824484 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454015070 con 0x7f9468151590 2024-09-13T18:48:22.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f94657fa700 1 -- 172.21.15.146:0/3440824484 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94540102e0 con 0x7f9468151590 2024-09-13T18:48:22.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.885+0000 7f946d892700 1 -- 172.21.15.146:0/3440824484 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9468149b10 con 0x7f9468151590 2024-09-13T18:48:22.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.889+0000 7f94657fa700 1 -- 172.21.15.146:0/3440824484 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945400e700 con 0x7f9468151590 2024-09-13T18:48:22.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.889+0000 7f946d892700 1 -- 172.21.15.146:0/3440824484 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94681edec0 con 0x7f9468151590 2024-09-13T18:48:22.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.889+0000 7f946d892700 1 -- 172.21.15.146:0/3440824484 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94681edd20 con 0x7f9468151590 2024-09-13T18:48:22.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.889+0000 7f94657fa700 1 -- 172.21.15.146:0/3440824484 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f945400e860 con 0x7f9468151590 2024-09-13T18:48:22.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.889+0000 7f94657fa700 1 --2- 172.21.15.146:0/3440824484 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f944c066730 0x7f944c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:22.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.889+0000 7f94657fa700 1 -- 172.21.15.146:0/3440824484 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f9454088320 con 0x7f9468151590 2024-09-13T18:48:22.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.889+0000 7f946c890700 1 --2- 172.21.15.146:0/3440824484 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f944c066730 0x7f944c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:22.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.889+0000 7f946c890700 1 --2- 172.21.15.146:0/3440824484 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f944c066730 0x7f944c068bf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f945c009c30 tx=0x7f945c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:22.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:22.893+0000 7f94657fa700 1 -- 172.21.15.146:0/3440824484 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f94681edd20 con 0x7f9468151590 2024-09-13T18:48:23.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.125+0000 7f946d892700 1 -- 172.21.15.146:0/3440824484 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94681edd20 con 0x7f9468151590 2024-09-13T18:48:23.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.125+0000 7f94657fa700 1 -- 172.21.15.146:0/3440824484 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+5518 (secure 0 0 0) 0x7f94681edd20 con 0x7f9468151590 2024-09-13T18:48:23.140 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:23.140 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:20.845594+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:23.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.129+0000 7f946d892700 1 -- 172.21.15.146:0/3440824484 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f944c066730 msgr2=0x7f944c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:23.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.129+0000 7f946d892700 1 --2- 172.21.15.146:0/3440824484 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f944c066730 0x7f944c068bf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f945c009c30 tx=0x7f945c008030 comp rx=0 tx=0).stop 2024-09-13T18:48:23.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.129+0000 7f946d892700 1 -- 172.21.15.146:0/3440824484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9468151590 msgr2=0x7f94681ea7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:23.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.129+0000 7f946d892700 1 --2- 172.21.15.146:0/3440824484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9468151590 0x7f94681ea7b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9454009a70 tx=0x7f9454009d80 comp rx=0 tx=0).stop 2024-09-13T18:48:23.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.129+0000 7f946d892700 1 -- 172.21.15.146:0/3440824484 shutdown_connections 2024-09-13T18:48:23.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.129+0000 7f946d892700 1 --2- 172.21.15.146:0/3440824484 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f944c066730 0x7f944c068bf0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:23.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.129+0000 7f946d892700 1 --2- 172.21.15.146:0/3440824484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9468157fd0 0x7f94680c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:23.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.129+0000 7f946d892700 1 --2- 172.21.15.146:0/3440824484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9468151590 0x7f94681ea7b0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:23.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.129+0000 7f946d892700 1 --2- 172.21.15.146:0/3440824484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f946806db40 0x7f94680c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:23.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.129+0000 7f946d892700 1 -- 172.21.15.146:0/3440824484 >> 172.21.15.146:0/3440824484 conn(0x7f9468066b00 msgr2=0x7f94681560e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:23.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.129+0000 7f946d892700 1 -- 172.21.15.146:0/3440824484 shutdown_connections 2024-09-13T18:48:23.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:23.129+0000 7f946d892700 1 -- 172.21.15.146:0/3440824484 wait complete. 2024-09-13T18:48:23.145 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 86 2024-09-13T18:48:23.158 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 86, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 86, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6459": { "addr": "172.21.15.146:6835/294014284", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 294014284, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 294014284, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6459, "incarnation": 74, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 13 }, "gid_6714": { "addr": "172.21.15.146:6837/2169954140", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 2169954140, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 2169954140, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6714, "incarnation": 86, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 39, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:48:20.845594+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6459, "mds_1": 6714 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.146:6833/3023779168", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3023779168, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3023779168, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 69, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6187, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:48:25.159 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:25.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f9098915700 1 Processor -- start 2024-09-13T18:48:25.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f9098915700 1 -- start start 2024-09-13T18:48:25.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f9098915700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909406e7c0 0x7f90940b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:25.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f9098915700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f909406eba0 0x7f90940c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:25.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f9098915700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90940c1020 con 0x7f90941598f0 2024-09-13T18:48:25.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f9098915700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90941593f0 con 0x7f909406eba0 2024-09-13T18:48:25.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f9098915700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90940c6a70 con 0x7f909406e7c0 2024-09-13T18:48:25.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f908a59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f909406eba0 0x7f90940c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:25.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f908a59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f909406eba0 0x7f90940c2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39650/0 (socket says 172.21.15.146:39650) 2024-09-13T18:48:25.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f908a59c700 1 -- 172.21.15.146:0/2309408322 learned_addr learned my addr 172.21.15.146:0/2309408322 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:25.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f9092d9d700 1 --2- 172.21.15.146:0/2309408322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909406e7c0 0x7f90940b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:25.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f908a59c700 1 -- 172.21.15.146:0/2309408322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909406e7c0 msgr2=0x7f90940b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:25.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f908a59c700 1 --2- 172.21.15.146:0/2309408322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909406e7c0 0x7f90940b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:25.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f908a59c700 1 -- 172.21.15.146:0/2309408322 >> v1:172.21.15.146:6789/0 conn(0x7f90941598f0 legacy=0x7f90941531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:25.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f908a59c700 1 -- 172.21.15.146:0/2309408322 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90940c1290 con 0x7f909406eba0 2024-09-13T18:48:25.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f908a59c700 1 --2- 172.21.15.146:0/2309408322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909406eba0 0x7f90940c2ce0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f907c009910 tx=0x7f907c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd22a04aa4114b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:25.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f909259c700 1 -- 172.21.15.146:0/2309408322 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c00ba60 con 0x7f909406eba0 2024-09-13T18:48:25.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f909259c700 1 -- 172.21.15.146:0/2309408322 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c00bbc0 con 0x7f909406eba0 2024-09-13T18:48:25.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f909259c700 1 -- 172.21.15.146:0/2309408322 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c009e60 con 0x7f909406eba0 2024-09-13T18:48:25.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f9098915700 1 -- 172.21.15.146:0/2309408322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909406eba0 msgr2=0x7f90940c2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:25.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.261+0000 7f9098915700 1 --2- 172.21.15.146:0/2309408322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909406eba0 0x7f90940c2ce0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f907c009910 tx=0x7f907c009c20 comp rx=0 tx=0).stop 2024-09-13T18:48:25.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 -- 172.21.15.146:0/2309408322 shutdown_connections 2024-09-13T18:48:25.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 --2- 172.21.15.146:0/2309408322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909406eba0 0x7f90940c2ce0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:25.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 --2- 172.21.15.146:0/2309408322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909406e7c0 0x7f90940b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:25.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 -- 172.21.15.146:0/2309408322 >> 172.21.15.146:0/2309408322 conn(0x7f9094071ed0 msgr2=0x7f90940722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:25.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 -- 172.21.15.146:0/2309408322 shutdown_connections 2024-09-13T18:48:25.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 -- 172.21.15.146:0/2309408322 wait complete. 2024-09-13T18:48:25.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 Processor -- start 2024-09-13T18:48:25.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 -- start start 2024-09-13T18:48:25.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909406e7c0 0x7f90941eef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:25.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f909406eba0 0x7f90941f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:25.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90941598f0 0x7f90940c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:25.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90940c6f60 con 0x7f909406e7c0 2024-09-13T18:48:25.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90941f24c0 con 0x7f909406eba0 2024-09-13T18:48:25.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90940c9400 con 0x7f90941598f0 2024-09-13T18:48:25.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9092d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909406e7c0 0x7f90941eef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:25.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9092d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909406e7c0 0x7f90941eef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39662/0 (socket says 172.21.15.146:39662) 2024-09-13T18:48:25.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9092d9d700 1 -- 172.21.15.146:0/1532208976 learned_addr learned my addr 172.21.15.146:0/1532208976 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:25.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f909359e700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90941598f0 0x7f90940c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:25.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f908a59c700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f909406eba0 0x7f90941f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:25.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9092d9d700 1 -- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90941598f0 msgr2=0x7f90940c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:25.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9092d9d700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90941598f0 0x7f90940c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:25.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9092d9d700 1 -- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f909406eba0 msgr2=0x7f90941f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:25.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9092d9d700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f909406eba0 0x7f90941f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:25.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9092d9d700 1 -- 172.21.15.146:0/1532208976 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c009570 con 0x7f909406e7c0 2024-09-13T18:48:25.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9092d9d700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909406e7c0 0x7f90941eef50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f908400d800 tx=0x7f908400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:25.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f908bfff700 1 -- 172.21.15.146:0/1532208976 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90840125f0 con 0x7f909406e7c0 2024-09-13T18:48:25.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f908bfff700 1 -- 172.21.15.146:0/1532208976 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9084012750 con 0x7f909406e7c0 2024-09-13T18:48:25.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 -- 172.21.15.146:0/1532208976 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90940c7170 con 0x7f909406e7c0 2024-09-13T18:48:25.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f908bfff700 1 -- 172.21.15.146:0/1532208976 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084012ac0 con 0x7f909406e7c0 2024-09-13T18:48:25.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.265+0000 7f9098915700 1 -- 172.21.15.146:0/1532208976 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90941f2a60 con 0x7f909406e7c0 2024-09-13T18:48:25.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.269+0000 7f9098915700 1 -- 172.21.15.146:0/1532208976 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9094075a30 con 0x7f909406e7c0 2024-09-13T18:48:25.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.269+0000 7f908bfff700 1 -- 172.21.15.146:0/1532208976 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9084012c20 con 0x7f909406e7c0 2024-09-13T18:48:25.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.269+0000 7f908bfff700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9074066760 0x7f9074068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:25.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.269+0000 7f908bfff700 1 -- 172.21.15.146:0/1532208976 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f9084085770 con 0x7f909406e7c0 2024-09-13T18:48:25.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.269+0000 7f908a59c700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9074066760 0x7f9074068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:25.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.269+0000 7f908a59c700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9074066760 0x7f9074068c20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f907c009910 tx=0x7f907c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:25.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.277+0000 7f908bfff700 1 -- 172.21.15.146:0/1532208976 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9084052370 con 0x7f909406e7c0 2024-09-13T18:48:25.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.501+0000 7f9098915700 1 -- 172.21.15.146:0/1532208976 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90940cc890 con 0x7f909406e7c0 2024-09-13T18:48:25.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.501+0000 7f908bfff700 1 -- 172.21.15.146:0/1532208976 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+5518 (secure 0 0 0) 0x7f9084012ed0 con 0x7f909406e7c0 2024-09-13T18:48:25.517 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:25.517 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:20.845594+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:25.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.505+0000 7f9098915700 1 -- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9074066760 msgr2=0x7f9074068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:25.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.505+0000 7f9098915700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9074066760 0x7f9074068c20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f907c009910 tx=0x7f907c008030 comp rx=0 tx=0).stop 2024-09-13T18:48:25.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.505+0000 7f9098915700 1 -- 172.21.15.146:0/1532208976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909406e7c0 msgr2=0x7f90941eef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:25.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.505+0000 7f9098915700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909406e7c0 0x7f90941eef50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f908400d800 tx=0x7f908400dbc0 comp rx=0 tx=0).stop 2024-09-13T18:48:25.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.505+0000 7f9098915700 1 -- 172.21.15.146:0/1532208976 shutdown_connections 2024-09-13T18:48:25.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.505+0000 7f9098915700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9074066760 0x7f9074068c20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:25.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.505+0000 7f9098915700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90941598f0 0x7f90940c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:25.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.505+0000 7f9098915700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f909406eba0 0x7f90941f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:25.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.505+0000 7f9098915700 1 --2- 172.21.15.146:0/1532208976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909406e7c0 0x7f90941eef50 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:25.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.505+0000 7f9098915700 1 -- 172.21.15.146:0/1532208976 >> 172.21.15.146:0/1532208976 conn(0x7f9094071ed0 msgr2=0x7f909406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:25.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.505+0000 7f9098915700 1 -- 172.21.15.146:0/1532208976 shutdown_connections 2024-09-13T18:48:25.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:25.505+0000 7f9098915700 1 -- 172.21.15.146:0/1532208976 wait complete. 2024-09-13T18:48:25.521 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 86 2024-09-13T18:48:25.535 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:48:27.536 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:27.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.641+0000 7f9e72594700 1 Processor -- start 2024-09-13T18:48:27.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.641+0000 7f9e72594700 1 -- start start 2024-09-13T18:48:27.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9e6c1514b0 0x7f9e6c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:27.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9e6c0b0160 0x7f9e6c0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:27.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e6c0b27e0 0x7f9e6c0b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:27.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6c06ae30 con 0x7f9e6c0b0160 2024-09-13T18:48:27.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6c0caa70 con 0x7f9e6c1514b0 2024-09-13T18:48:27.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0b0b70 con 0x7f9e6c0b27e0 2024-09-13T18:48:27.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e70d91700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9e6c0b0160 0x7f9e6c0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:27.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e70d91700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9e6c0b0160 0x7f9e6c0b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39674/0 (socket says 172.21.15.146:39674) 2024-09-13T18:48:27.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e70d91700 1 -- 172.21.15.146:0/4279793103 learned_addr learned my addr 172.21.15.146:0/4279793103 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:27.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e71592700 1 --2- 172.21.15.146:0/4279793103 >> v2:172.21.15.154:3300/0 conn(0x7f9e6c1514b0 0x7f9e6c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:27.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e71d93700 1 --2- 172.21.15.146:0/4279793103 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e6c0b27e0 0x7f9e6c0b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:27.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e70d91700 1 -- 172.21.15.146:0/4279793103 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e6c0b27e0 msgr2=0x7f9e6c0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:27.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e70d91700 1 --2- 172.21.15.146:0/4279793103 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e6c0b27e0 0x7f9e6c0b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:27.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e70d91700 1 -- 172.21.15.146:0/4279793103 >> v2:172.21.15.154:3300/0 conn(0x7f9e6c1514b0 msgr2=0x7f9e6c1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:27.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e70d91700 1 --2- 172.21.15.146:0/4279793103 >> v2:172.21.15.154:3300/0 conn(0x7f9e6c1514b0 0x7f9e6c1572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:27.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e70d91700 1 -- 172.21.15.146:0/4279793103 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c0b0e30 con 0x7f9e6c0b0160 2024-09-13T18:48:27.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e70d91700 1 --2- 172.21.15.146:0/4279793103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e6c0b0160 0x7f9e6c0b0540 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0676d0 tx=0x7f9e5c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257ccfe1b477f7a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:27.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e63fff700 1 -- 172.21.15.146:0/4279793103 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00b8a0 con 0x7f9e6c0b0160 2024-09-13T18:48:27.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e63fff700 1 -- 172.21.15.146:0/4279793103 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c00ba00 con 0x7f9e6c0b0160 2024-09-13T18:48:27.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e63fff700 1 -- 172.21.15.146:0/4279793103 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00bd90 con 0x7f9e6c0b0160 2024-09-13T18:48:27.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 -- 172.21.15.146:0/4279793103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e6c0b0160 msgr2=0x7f9e6c0b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:27.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 --2- 172.21.15.146:0/4279793103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e6c0b0160 0x7f9e6c0b0540 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0676d0 tx=0x7f9e5c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:48:27.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 -- 172.21.15.146:0/4279793103 shutdown_connections 2024-09-13T18:48:27.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 --2- 172.21.15.146:0/4279793103 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e6c0b27e0 0x7f9e6c0b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:27.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 --2- 172.21.15.146:0/4279793103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e6c0b0160 0x7f9e6c0b0540 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:27.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 --2- 172.21.15.146:0/4279793103 >> v2:172.21.15.154:3300/0 conn(0x7f9e6c1514b0 0x7f9e6c1572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:27.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 -- 172.21.15.146:0/4279793103 >> 172.21.15.146:0/4279793103 conn(0x7f9e6c06c2d0 msgr2=0x7f9e6c06c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:27.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 -- 172.21.15.146:0/4279793103 shutdown_connections 2024-09-13T18:48:27.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.645+0000 7f9e72594700 1 -- 172.21.15.146:0/4279793103 wait complete. 2024-09-13T18:48:27.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e72594700 1 Processor -- start 2024-09-13T18:48:27.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e72594700 1 -- start start 2024-09-13T18:48:27.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e72594700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e6c0b0160 0x7f9e6c0c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:27.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e72594700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e6c0b27e0 0x7f9e6c0cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:27.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e72594700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e6c1514b0 0x7f9e6c0c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:27.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e72594700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0cbbe0 con 0x7f9e6c0b0160 2024-09-13T18:48:27.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e72594700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0cc3f0 con 0x7f9e6c1514b0 2024-09-13T18:48:27.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e72594700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c1ef470 con 0x7f9e6c0b27e0 2024-09-13T18:48:27.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e71592700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e6c0b0160 0x7f9e6c0c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:27.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e71592700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e6c0b0160 0x7f9e6c0c3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39686/0 (socket says 172.21.15.146:39686) 2024-09-13T18:48:27.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e71592700 1 -- 172.21.15.146:0/4120650692 learned_addr learned my addr 172.21.15.146:0/4120650692 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:27.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e70d91700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e6c0b27e0 0x7f9e6c0cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:27.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e71d93700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e6c1514b0 0x7f9e6c0c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:27.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e71592700 1 -- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e6c0b27e0 msgr2=0x7f9e6c0cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:27.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e71592700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e6c0b27e0 0x7f9e6c0cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:27.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e71592700 1 -- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e6c1514b0 msgr2=0x7f9e6c0c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:27.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e71592700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e6c1514b0 0x7f9e6c0c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:27.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e71592700 1 -- 172.21.15.146:0/4120650692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c009570 con 0x7f9e6c0b0160 2024-09-13T18:48:27.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e71592700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e6c0b0160 0x7f9e6c0c3b70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800d830 tx=0x7f9e6800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:27.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e61ffb700 1 -- 172.21.15.146:0/4120650692 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e680125f0 con 0x7f9e6c0b0160 2024-09-13T18:48:27.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e61ffb700 1 -- 172.21.15.146:0/4120650692 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e68012750 con 0x7f9e6c0b0160 2024-09-13T18:48:27.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e72594700 1 -- 172.21.15.146:0/4120650692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6c0cce60 con 0x7f9e6c0b0160 2024-09-13T18:48:27.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e61ffb700 1 -- 172.21.15.146:0/4120650692 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68012ac0 con 0x7f9e6c0b0160 2024-09-13T18:48:27.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.649+0000 7f9e72594700 1 -- 172.21.15.146:0/4120650692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c0cbe30 con 0x7f9e6c0b0160 2024-09-13T18:48:27.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.653+0000 7f9e72594700 1 -- 172.21.15.146:0/4120650692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6c0b0540 con 0x7f9e6c0b0160 2024-09-13T18:48:27.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.653+0000 7f9e61ffb700 1 -- 172.21.15.146:0/4120650692 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e68012c20 con 0x7f9e6c0b0160 2024-09-13T18:48:27.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.653+0000 7f9e61ffb700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9e50066730 0x7f9e50068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:27.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.653+0000 7f9e61ffb700 1 -- 172.21.15.146:0/4120650692 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f9e68084970 con 0x7f9e6c0b0160 2024-09-13T18:48:27.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.653+0000 7f9e70d91700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9e50066730 0x7f9e50068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:27.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.653+0000 7f9e70d91700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9e50066730 0x7f9e50068bf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c000c00 tx=0x7f9e5c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:27.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.661+0000 7f9e61ffb700 1 -- 172.21.15.146:0/4120650692 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9e68051660 con 0x7f9e6c0b0160 2024-09-13T18:48:27.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.869+0000 7f9e72594700 1 -- 172.21.15.146:0/4120650692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e6c0c7cc0 con 0x7f9e6c0b0160 2024-09-13T18:48:27.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.873+0000 7f9e61ffb700 1 -- 172.21.15.146:0/4120650692 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+5520 (secure 0 0 0) 0x7f9e68012ed0 con 0x7f9e6c0b0160 2024-09-13T18:48:27.887 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:27.887 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:25.886578+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:resolve","state_seq":32,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:27.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.873+0000 7f9e72594700 1 -- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9e50066730 msgr2=0x7f9e50068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:27.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.873+0000 7f9e72594700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9e50066730 0x7f9e50068bf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c000c00 tx=0x7f9e5c008030 comp rx=0 tx=0).stop 2024-09-13T18:48:27.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.873+0000 7f9e72594700 1 -- 172.21.15.146:0/4120650692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e6c0b0160 msgr2=0x7f9e6c0c3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:27.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.873+0000 7f9e72594700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e6c0b0160 0x7f9e6c0c3b70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800d830 tx=0x7f9e6800db40 comp rx=0 tx=0).stop 2024-09-13T18:48:27.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.873+0000 7f9e72594700 1 -- 172.21.15.146:0/4120650692 shutdown_connections 2024-09-13T18:48:27.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.873+0000 7f9e72594700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9e50066730 0x7f9e50068bf0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:27.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.873+0000 7f9e72594700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e6c1514b0 0x7f9e6c0c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:27.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.873+0000 7f9e72594700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e6c0b27e0 0x7f9e6c0cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:27.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.873+0000 7f9e72594700 1 --2- 172.21.15.146:0/4120650692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e6c0b0160 0x7f9e6c0c3b70 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:27.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.873+0000 7f9e72594700 1 -- 172.21.15.146:0/4120650692 >> 172.21.15.146:0/4120650692 conn(0x7f9e6c06c2d0 msgr2=0x7f9e6c0aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:27.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.873+0000 7f9e72594700 1 -- 172.21.15.146:0/4120650692 shutdown_connections 2024-09-13T18:48:27.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:27.873+0000 7f9e72594700 1 -- 172.21.15.146:0/4120650692 wait complete. 2024-09-13T18:48:27.891 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 87 2024-09-13T18:48:27.901 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:48:29.902 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:30.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.001+0000 7fdf0e6ea700 1 Processor -- start 2024-09-13T18:48:30.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.001+0000 7fdf0e6ea700 1 -- start start 2024-09-13T18:48:30.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0e6ea700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdf08154a40 0x7fdf080721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:30.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0e6ea700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdf0806f7d0 0x7fdf08157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:30.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0e6ea700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf080c8c70 con 0x7fdf080cac60 2024-09-13T18:48:30.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0e6ea700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf08072810 con 0x7fdf0806f7d0 2024-09-13T18:48:30.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0e6ea700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf08157910 con 0x7fdf08154a40 2024-09-13T18:48:30.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0cee7700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdf0806f7d0 0x7fdf08157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:30.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0cee7700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdf0806f7d0 0x7fdf08157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39690/0 (socket says 172.21.15.146:39690) 2024-09-13T18:48:30.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0cee7700 1 -- 172.21.15.146:0/3336247645 learned_addr learned my addr 172.21.15.146:0/3336247645 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:30.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0d6e8700 1 --2- 172.21.15.146:0/3336247645 >> v2:172.21.15.154:3300/0 conn(0x7fdf08154a40 0x7fdf080721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:30.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0cee7700 1 -- 172.21.15.146:0/3336247645 >> v2:172.21.15.154:3300/0 conn(0x7fdf08154a40 msgr2=0x7fdf080721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:30.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0cee7700 1 --2- 172.21.15.146:0/3336247645 >> v2:172.21.15.154:3300/0 conn(0x7fdf08154a40 0x7fdf080721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:30.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0cee7700 1 -- 172.21.15.146:0/3336247645 >> v1:172.21.15.154:6789/0 conn(0x7fdf080cac60 legacy=0x7fdf0815a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:30.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0cee7700 1 -- 172.21.15.146:0/3336247645 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf080c8f50 con 0x7fdf0806f7d0 2024-09-13T18:48:30.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0d6e8700 1 --2- 172.21.15.146:0/3336247645 >> v2:172.21.15.154:3300/0 conn(0x7fdf08154a40 0x7fdf080721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:48:30.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0cee7700 1 --2- 172.21.15.146:0/3336247645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf0806f7d0 0x7fdf08157d30 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fdf080cbbe0 tx=0x7fdef8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd271a52cfc977eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:30.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdefffff700 1 -- 172.21.15.146:0/3336247645 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef800b8a0 con 0x7fdf0806f7d0 2024-09-13T18:48:30.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdefffff700 1 -- 172.21.15.146:0/3336247645 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef800ba00 con 0x7fdf0806f7d0 2024-09-13T18:48:30.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdefffff700 1 -- 172.21.15.146:0/3336247645 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef800bd90 con 0x7fdf0806f7d0 2024-09-13T18:48:30.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/3336247645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf0806f7d0 msgr2=0x7fdf08157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:30.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0e6ea700 1 --2- 172.21.15.146:0/3336247645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf0806f7d0 0x7fdf08157d30 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fdf080cbbe0 tx=0x7fdef8009bf0 comp rx=0 tx=0).stop 2024-09-13T18:48:30.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/3336247645 shutdown_connections 2024-09-13T18:48:30.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0e6ea700 1 --2- 172.21.15.146:0/3336247645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf0806f7d0 0x7fdf08157d30 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:30.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0e6ea700 1 --2- 172.21.15.146:0/3336247645 >> v2:172.21.15.154:3300/0 conn(0x7fdf08154a40 0x7fdf080721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:30.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/3336247645 >> 172.21.15.146:0/3336247645 conn(0x7fdf08066090 msgr2=0x7fdf08066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:30.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/3336247645 shutdown_connections 2024-09-13T18:48:30.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.005+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/3336247645 wait complete. 2024-09-13T18:48:30.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0e6ea700 1 Processor -- start 2024-09-13T18:48:30.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0e6ea700 1 -- start start 2024-09-13T18:48:30.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0e6ea700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf0806f7d0 0x7fdf0806d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:30.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0e6ea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf080cac60 0x7fdf081f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:30.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0e6ea700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf08154a40 0x7fdf081f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:30.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0e6ea700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf080c2de0 con 0x7fdf08154a40 2024-09-13T18:48:30.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0e6ea700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf080c4230 con 0x7fdf080cac60 2024-09-13T18:48:30.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0e6ea700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0814c1c0 con 0x7fdf0806f7d0 2024-09-13T18:48:30.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0dee9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf08154a40 0x7fdf081f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:30.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0dee9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf08154a40 0x7fdf081f2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39706/0 (socket says 172.21.15.146:39706) 2024-09-13T18:48:30.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0dee9700 1 -- 172.21.15.146:0/2486779402 learned_addr learned my addr 172.21.15.146:0/2486779402 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:30.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0d6e8700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf0806f7d0 0x7fdf0806d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:30.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0cee7700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf080cac60 0x7fdf081f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:30.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0dee9700 1 -- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf0806f7d0 msgr2=0x7fdf0806d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:30.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0dee9700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf0806f7d0 0x7fdf0806d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:30.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0dee9700 1 -- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf080cac60 msgr2=0x7fdf081f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:30.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0dee9700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf080cac60 0x7fdf081f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:30.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0dee9700 1 -- 172.21.15.146:0/2486779402 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef8009570 con 0x7fdf08154a40 2024-09-13T18:48:30.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0dee9700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf08154a40 0x7fdf081f2cb0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000cb50 tx=0x7fdf0000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:30.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdefdffb700 1 -- 172.21.15.146:0/2486779402 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00003c20 con 0x7fdf08154a40 2024-09-13T18:48:30.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdefdffb700 1 -- 172.21.15.146:0/2486779402 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf00003d80 con 0x7fdf08154a40 2024-09-13T18:48:30.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/2486779402 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf080c3050 con 0x7fdf08154a40 2024-09-13T18:48:30.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdefdffb700 1 -- 172.21.15.146:0/2486779402 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0000aa20 con 0x7fdf08154a40 2024-09-13T18:48:30.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.009+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/2486779402 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf080c1b00 con 0x7fdf08154a40 2024-09-13T18:48:30.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.013+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/2486779402 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf080c9e00 con 0x7fdf08154a40 2024-09-13T18:48:30.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.013+0000 7fdefdffb700 1 -- 172.21.15.146:0/2486779402 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf0000ab80 con 0x7fdf08154a40 2024-09-13T18:48:30.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.013+0000 7fdefdffb700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdeec066760 0x7fdeec068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:30.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.013+0000 7fdefdffb700 1 -- 172.21.15.146:0/2486779402 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7fdf00084980 con 0x7fdf08154a40 2024-09-13T18:48:30.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.013+0000 7fdf0d6e8700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdeec066760 0x7fdeec068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:30.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.013+0000 7fdf0d6e8700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdeec066760 0x7fdeec068c20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c960 tx=0x7fdf04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:30.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.021+0000 7fdefdffb700 1 -- 172.21.15.146:0/2486779402 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf000514a0 con 0x7fdf08154a40 2024-09-13T18:48:30.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.257+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/2486779402 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf08154e20 con 0x7fdf08154a40 2024-09-13T18:48:30.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.257+0000 7fdefdffb700 1 -- 172.21.15.146:0/2486779402 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+5520 (secure 0 0 0) 0x7fdf00054ac0 con 0x7fdf08154a40 2024-09-13T18:48:30.272 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:30.272 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:25.886578+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:resolve","state_seq":32,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:30.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.257+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdeec066760 msgr2=0x7fdeec068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:30.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.257+0000 7fdf0e6ea700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdeec066760 0x7fdeec068c20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c960 tx=0x7fdf04008040 comp rx=0 tx=0).stop 2024-09-13T18:48:30.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.257+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/2486779402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf08154a40 msgr2=0x7fdf081f2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:30.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.257+0000 7fdf0e6ea700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf08154a40 0x7fdf081f2cb0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000cb50 tx=0x7fdf0000ce60 comp rx=0 tx=0).stop 2024-09-13T18:48:30.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.257+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/2486779402 shutdown_connections 2024-09-13T18:48:30.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.257+0000 7fdf0e6ea700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdeec066760 0x7fdeec068c20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:30.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.257+0000 7fdf0e6ea700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf08154a40 0x7fdf081f2cb0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:30.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.257+0000 7fdf0e6ea700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf080cac60 0x7fdf081f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:30.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.257+0000 7fdf0e6ea700 1 --2- 172.21.15.146:0/2486779402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf0806f7d0 0x7fdf0806d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:30.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.257+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/2486779402 >> 172.21.15.146:0/2486779402 conn(0x7fdf08066090 msgr2=0x7fdf081560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:30.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.257+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/2486779402 shutdown_connections 2024-09-13T18:48:30.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:30.261+0000 7fdf0e6ea700 1 -- 172.21.15.146:0/2486779402 wait complete. 2024-09-13T18:48:30.276 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 87 2024-09-13T18:48:30.286 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:48:32.287 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:32.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2f59e700 1 Processor -- start 2024-09-13T18:48:32.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2f59e700 1 -- start start 2024-09-13T18:48:32.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2f59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbb3014fb50 0x7fbb300c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:32.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2f59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbb300c7b00 0x7fbb300c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:32.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb300c6060 0x7fbb300c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:32.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2f59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb300c6a30 con 0x7fbb3014fb50 2024-09-13T18:48:32.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2f59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb30156810 con 0x7fbb300c7b00 2024-09-13T18:48:32.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb301559b0 con 0x7fbb300c6060 2024-09-13T18:48:32.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2e59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbb3014fb50 0x7fbb300c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:32.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2e59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbb3014fb50 0x7fbb300c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58034/0 (socket says 172.21.15.146:58034) 2024-09-13T18:48:32.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2e59c700 1 -- 172.21.15.146:0/3564964610 learned_addr learned my addr 172.21.15.146:0/3564964610 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:32.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2ed9d700 1 --2- 172.21.15.146:0/3564964610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb300c6060 0x7fbb300c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:32.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2dd9b700 1 --2- 172.21.15.146:0/3564964610 >> v2:172.21.15.154:3300/0 conn(0x7fbb300c7b00 0x7fbb300c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:32.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2e59c700 1 -- 172.21.15.146:0/3564964610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb300c6060 msgr2=0x7fbb300c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:32.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2e59c700 1 --2- 172.21.15.146:0/3564964610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb300c6060 0x7fbb300c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:32.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2e59c700 1 -- 172.21.15.146:0/3564964610 >> v2:172.21.15.154:3300/0 conn(0x7fbb300c7b00 msgr2=0x7fbb300c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:32.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2e59c700 1 --2- 172.21.15.146:0/3564964610 >> v2:172.21.15.154:3300/0 conn(0x7fbb300c7b00 0x7fbb300c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:32.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2e59c700 1 -- 172.21.15.146:0/3564964610 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb30155c90 con 0x7fbb3014fb50 2024-09-13T18:48:32.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2e59c700 1 --2- 172.21.15.146:0/3564964610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb3014fb50 0x7fbb300c7fd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009910 tx=0x7fbb24009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64472b62e7c7a113 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:32.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2d59a700 1 -- 172.21.15.146:0/3564964610 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24011830 con 0x7fbb3014fb50 2024-09-13T18:48:32.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2d59a700 1 -- 172.21.15.146:0/3564964610 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb24011990 con 0x7fbb3014fb50 2024-09-13T18:48:32.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.397+0000 7fbb2d59a700 1 -- 172.21.15.146:0/3564964610 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24011d40 con 0x7fbb3014fb50 2024-09-13T18:48:32.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 -- 172.21.15.146:0/3564964610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb3014fb50 msgr2=0x7fbb300c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:32.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 --2- 172.21.15.146:0/3564964610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb3014fb50 0x7fbb300c7fd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009910 tx=0x7fbb24009c20 comp rx=0 tx=0).stop 2024-09-13T18:48:32.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 -- 172.21.15.146:0/3564964610 shutdown_connections 2024-09-13T18:48:32.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 --2- 172.21.15.146:0/3564964610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb300c6060 0x7fbb300c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:32.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 --2- 172.21.15.146:0/3564964610 >> v2:172.21.15.154:3300/0 conn(0x7fbb300c7b00 0x7fbb300c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:32.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 --2- 172.21.15.146:0/3564964610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb3014fb50 0x7fbb300c7fd0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:32.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 -- 172.21.15.146:0/3564964610 >> 172.21.15.146:0/3564964610 conn(0x7fbb30066b00 msgr2=0x7fbb3006aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:32.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 -- 172.21.15.146:0/3564964610 shutdown_connections 2024-09-13T18:48:32.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 -- 172.21.15.146:0/3564964610 wait complete. 2024-09-13T18:48:32.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 Processor -- start 2024-09-13T18:48:32.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 -- start start 2024-09-13T18:48:32.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb300c6060 0x7fbb30150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:32.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb300c7b00 0x7fbb301eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:32.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb3014fb50 0x7fbb300c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:32.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb300c1590 con 0x7fbb3014fb50 2024-09-13T18:48:32.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb300c1850 con 0x7fbb300c7b00 2024-09-13T18:48:32.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb300c1b30 con 0x7fbb300c6060 2024-09-13T18:48:32.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2ed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb3014fb50 0x7fbb300c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:32.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb300c7b00 0x7fbb301eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:32.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2ed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb3014fb50 0x7fbb300c4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58048/0 (socket says 172.21.15.146:58048) 2024-09-13T18:48:32.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2ed9d700 1 -- 172.21.15.146:0/995796156 learned_addr learned my addr 172.21.15.146:0/995796156 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:32.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2e59c700 1 --2- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb300c6060 0x7fbb30150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:32.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2e59c700 1 -- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb300c7b00 msgr2=0x7fbb301eecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:32.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2e59c700 1 --2- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb300c7b00 0x7fbb301eecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:32.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2e59c700 1 -- 172.21.15.146:0/995796156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb3014fb50 msgr2=0x7fbb300c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:32.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2e59c700 1 --2- 172.21.15.146:0/995796156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb3014fb50 0x7fbb300c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:32.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2e59c700 1 -- 172.21.15.146:0/995796156 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb24009570 con 0x7fbb300c6060 2024-09-13T18:48:32.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2e59c700 1 --2- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb300c6060 0x7fbb30150c60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbb24016000 tx=0x7fbb24004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:32.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb1effd700 1 -- 172.21.15.146:0/995796156 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24004450 con 0x7fbb300c6060 2024-09-13T18:48:32.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb1effd700 1 -- 172.21.15.146:0/995796156 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2401f070 con 0x7fbb300c6060 2024-09-13T18:48:32.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.401+0000 7fbb2f59e700 1 -- 172.21.15.146:0/995796156 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb301501a0 con 0x7fbb300c6060 2024-09-13T18:48:32.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.405+0000 7fbb1effd700 1 -- 172.21.15.146:0/995796156 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24003900 con 0x7fbb300c6060 2024-09-13T18:48:32.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.405+0000 7fbb2f59e700 1 -- 172.21.15.146:0/995796156 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb30150760 con 0x7fbb300c6060 2024-09-13T18:48:32.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.405+0000 7fbb2f59e700 1 -- 172.21.15.146:0/995796156 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3014e190 con 0x7fbb300c6060 2024-09-13T18:48:32.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.405+0000 7fbb1effd700 1 -- 172.21.15.146:0/995796156 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb24003a60 con 0x7fbb300c6060 2024-09-13T18:48:32.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.405+0000 7fbb1effd700 1 --2- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbb10066730 0x7fbb10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:32.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.405+0000 7fbb1effd700 1 -- 172.21.15.146:0/995796156 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7fbb24021070 con 0x7fbb300c6060 2024-09-13T18:48:32.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.405+0000 7fbb2dd9b700 1 --2- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbb10066730 0x7fbb10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:32.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.405+0000 7fbb2dd9b700 1 --2- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbb10066730 0x7fbb10068bf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbb3014db90 tx=0x7fbb20009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:32.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.413+0000 7fbb1effd700 1 -- 172.21.15.146:0/995796156 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbb24052920 con 0x7fbb300c6060 2024-09-13T18:48:32.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.665+0000 7fbb2f59e700 1 -- 172.21.15.146:0/995796156 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb300c64f0 con 0x7fbb300c6060 2024-09-13T18:48:32.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.669+0000 7fbb1effd700 1 -- 172.21.15.146:0/995796156 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+5522 (secure 0 0 0) 0x7fbb24015090 con 0x7fbb300c6060 2024-09-13T18:48:32.683 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:32.683 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:30.909275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:reconnect","state_seq":34,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:32.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.669+0000 7fbb2f59e700 1 -- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbb10066730 msgr2=0x7fbb10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:32.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.669+0000 7fbb2f59e700 1 --2- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbb10066730 0x7fbb10068bf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbb3014db90 tx=0x7fbb20009240 comp rx=0 tx=0).stop 2024-09-13T18:48:32.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.669+0000 7fbb2f59e700 1 -- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb300c6060 msgr2=0x7fbb30150c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:32.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.669+0000 7fbb2f59e700 1 --2- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb300c6060 0x7fbb30150c60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbb24016000 tx=0x7fbb24004770 comp rx=0 tx=0).stop 2024-09-13T18:48:32.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.669+0000 7fbb2f59e700 1 -- 172.21.15.146:0/995796156 shutdown_connections 2024-09-13T18:48:32.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.669+0000 7fbb2f59e700 1 --2- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbb10066730 0x7fbb10068bf0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:32.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.669+0000 7fbb2f59e700 1 --2- 172.21.15.146:0/995796156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb3014fb50 0x7fbb300c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:32.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.669+0000 7fbb2f59e700 1 --2- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb300c7b00 0x7fbb301eecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:32.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.669+0000 7fbb2f59e700 1 --2- 172.21.15.146:0/995796156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb300c6060 0x7fbb30150c60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:32.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.669+0000 7fbb2f59e700 1 -- 172.21.15.146:0/995796156 >> 172.21.15.146:0/995796156 conn(0x7fbb30066b00 msgr2=0x7fbb3006aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:32.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.669+0000 7fbb2f59e700 1 -- 172.21.15.146:0/995796156 shutdown_connections 2024-09-13T18:48:32.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:32.669+0000 7fbb2f59e700 1 -- 172.21.15.146:0/995796156 wait complete. 2024-09-13T18:48:32.687 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 88 2024-09-13T18:48:32.699 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:48:34.701 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:34.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13c864700 1 Processor -- start 2024-09-13T18:48:34.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13c864700 1 -- start start 2024-09-13T18:48:34.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13c864700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1380c5fc0 0x7fa1380c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:34.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13c864700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1380c63a0 0x7fa1380c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:34.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13c864700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1380c8c70 con 0x7fa13806f5c0 2024-09-13T18:48:34.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13c864700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1380c9bd0 con 0x7fa1380c63a0 2024-09-13T18:48:34.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13c864700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1380c8f50 con 0x7fa1380c5fc0 2024-09-13T18:48:34.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1380c63a0 0x7fa1380c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:34.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa136d9d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1380c5fc0 0x7fa1380c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:34.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1380c63a0 0x7fa1380c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:50236/0 (socket says 172.21.15.146:50236) 2024-09-13T18:48:34.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13759e700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa13806f5c0 0x7fa138159660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:36334/0 (socket says 172.21.15.146:36334) 2024-09-13T18:48:34.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13659c700 1 -- 172.21.15.146:0/395101876 learned_addr learned my addr 172.21.15.146:0/395101876 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:34.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13659c700 1 -- 172.21.15.146:0/395101876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1380c5fc0 msgr2=0x7fa1380c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:34.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13659c700 1 --2- 172.21.15.146:0/395101876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1380c5fc0 0x7fa1380c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:34.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13659c700 1 -- 172.21.15.146:0/395101876 >> v1:172.21.15.154:6789/0 conn(0x7fa13806f5c0 legacy=0x7fa138159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:34.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa135d9b700 1 -- 172.21.15.146:0/395101876 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203825031 0 0) 0x7fa1380c8c70 con 0x7fa13806f5c0 2024-09-13T18:48:34.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13659c700 1 -- 172.21.15.146:0/395101876 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1380c9230 con 0x7fa1380c63a0 2024-09-13T18:48:34.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa13659c700 1 --2- 172.21.15.146:0/395101876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1380c63a0 0x7fa1380c7140 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009910 tx=0x7fa12c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=966d5ca7f2092668 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:34.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.817+0000 7fa135d9b700 1 -- 172.21.15.146:0/395101876 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c011830 con 0x7fa1380c63a0 2024-09-13T18:48:34.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.821+0000 7fa135d9b700 1 -- 172.21.15.146:0/395101876 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c011990 con 0x7fa1380c63a0 2024-09-13T18:48:34.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.821+0000 7fa135d9b700 1 -- 172.21.15.146:0/395101876 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c011d40 con 0x7fa1380c63a0 2024-09-13T18:48:34.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.821+0000 7fa13c864700 1 -- 172.21.15.146:0/395101876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1380c63a0 msgr2=0x7fa1380c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:34.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.821+0000 7fa13c864700 1 --2- 172.21.15.146:0/395101876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1380c63a0 0x7fa1380c7140 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009910 tx=0x7fa12c009c20 comp rx=0 tx=0).stop 2024-09-13T18:48:34.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.821+0000 7fa13c864700 1 -- 172.21.15.146:0/395101876 shutdown_connections 2024-09-13T18:48:34.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.821+0000 7fa13c864700 1 --2- 172.21.15.146:0/395101876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1380c63a0 0x7fa1380c7140 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:34.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.821+0000 7fa13c864700 1 --2- 172.21.15.146:0/395101876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1380c5fc0 0x7fa1380c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:34.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.821+0000 7fa13c864700 1 -- 172.21.15.146:0/395101876 >> 172.21.15.146:0/395101876 conn(0x7fa138066090 msgr2=0x7fa138066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:34.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.821+0000 7fa13c864700 1 -- 172.21.15.146:0/395101876 shutdown_connections 2024-09-13T18:48:34.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.821+0000 7fa13c864700 1 -- 172.21.15.146:0/395101876 wait complete. 2024-09-13T18:48:34.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.821+0000 7fa13c864700 1 Processor -- start 2024-09-13T18:48:34.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.821+0000 7fa13c864700 1 -- start start 2024-09-13T18:48:34.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13c864700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa13806f5c0 0x7fa1380c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:34.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13c864700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1380c5fc0 0x7fa1380c10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:34.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13c864700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1380c63a0 0x7fa1380ab660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:34.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13c864700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13814b310 con 0x7fa1380c63a0 2024-09-13T18:48:34.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13c864700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1380aa740 con 0x7fa13806f5c0 2024-09-13T18:48:34.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13c864700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13814b050 con 0x7fa1380c5fc0 2024-09-13T18:48:34.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1380c63a0 0x7fa1380ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:34.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1380c63a0 0x7fa1380ab660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58064/0 (socket says 172.21.15.146:58064) 2024-09-13T18:48:34.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13759e700 1 -- 172.21.15.146:0/1581337290 learned_addr learned my addr 172.21.15.146:0/1581337290 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:34.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13659c700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1380c5fc0 0x7fa1380c10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:34.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa136d9d700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa13806f5c0 0x7fa1380c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:34.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13759e700 1 -- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1380c5fc0 msgr2=0x7fa1380c10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:34.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13759e700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1380c5fc0 0x7fa1380c10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:34.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13759e700 1 -- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa13806f5c0 msgr2=0x7fa1380c2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:34.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13759e700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa13806f5c0 0x7fa1380c2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:34.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13759e700 1 -- 172.21.15.146:0/1581337290 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c009570 con 0x7fa1380c63a0 2024-09-13T18:48:34.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13759e700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1380c63a0 0x7fa1380ab660 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa13000cb60 tx=0x7fa13000ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:34.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa11f7fe700 1 -- 172.21.15.146:0/1581337290 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13000a9e0 con 0x7fa1380c63a0 2024-09-13T18:48:34.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa11f7fe700 1 -- 172.21.15.146:0/1581337290 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13000ab40 con 0x7fa1380c63a0 2024-09-13T18:48:34.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13c864700 1 -- 172.21.15.146:0/1581337290 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1380ad430 con 0x7fa1380c63a0 2024-09-13T18:48:34.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa11f7fe700 1 -- 172.21.15.146:0/1581337290 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1300070d0 con 0x7fa1380c63a0 2024-09-13T18:48:34.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13c864700 1 -- 172.21.15.146:0/1581337290 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1380aac30 con 0x7fa1380c63a0 2024-09-13T18:48:34.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.825+0000 7fa13c864700 1 -- 172.21.15.146:0/1581337290 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1380cb790 con 0x7fa1380c63a0 2024-09-13T18:48:34.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.829+0000 7fa11f7fe700 1 -- 172.21.15.146:0/1581337290 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa130007230 con 0x7fa1380c63a0 2024-09-13T18:48:34.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.829+0000 7fa11f7fe700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa124066730 0x7fa124068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:34.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.829+0000 7fa11f7fe700 1 -- 172.21.15.146:0/1581337290 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7fa130084210 con 0x7fa1380c63a0 2024-09-13T18:48:34.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.829+0000 7fa136d9d700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa124066730 0x7fa124068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:34.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.829+0000 7fa136d9d700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa124066730 0x7fa124068bf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa128009bf0 tx=0x7fa128009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:34.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:34.833+0000 7fa11f7fe700 1 -- 172.21.15.146:0/1581337290 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa130050dd0 con 0x7fa1380c63a0 2024-09-13T18:48:35.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.065+0000 7fa13c864700 1 -- 172.21.15.146:0/1581337290 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1380ad620 con 0x7fa1380c63a0 2024-09-13T18:48:35.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.065+0000 7fa11f7fe700 1 -- 172.21.15.146:0/1581337290 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+5522 (secure 0 0 0) 0x7fa1300543f0 con 0x7fa1380c63a0 2024-09-13T18:48:35.082 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:35.082 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:30.909275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:reconnect","state_seq":34,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:35.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.069+0000 7fa13c864700 1 -- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa124066730 msgr2=0x7fa124068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:35.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.069+0000 7fa13c864700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa124066730 0x7fa124068bf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa128009bf0 tx=0x7fa128009240 comp rx=0 tx=0).stop 2024-09-13T18:48:35.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.069+0000 7fa13c864700 1 -- 172.21.15.146:0/1581337290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1380c63a0 msgr2=0x7fa1380ab660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:35.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.069+0000 7fa13c864700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1380c63a0 0x7fa1380ab660 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa13000cb60 tx=0x7fa13000ce70 comp rx=0 tx=0).stop 2024-09-13T18:48:35.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.069+0000 7fa13c864700 1 -- 172.21.15.146:0/1581337290 shutdown_connections 2024-09-13T18:48:35.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.069+0000 7fa13c864700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa124066730 0x7fa124068bf0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:35.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.069+0000 7fa13c864700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1380c63a0 0x7fa1380ab660 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:35.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.069+0000 7fa13c864700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1380c5fc0 0x7fa1380c10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:35.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.069+0000 7fa13c864700 1 --2- 172.21.15.146:0/1581337290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa13806f5c0 0x7fa1380c2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:35.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.069+0000 7fa13c864700 1 -- 172.21.15.146:0/1581337290 >> 172.21.15.146:0/1581337290 conn(0x7fa138066090 msgr2=0x7fa138071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:35.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.069+0000 7fa13c864700 1 -- 172.21.15.146:0/1581337290 shutdown_connections 2024-09-13T18:48:35.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:35.069+0000 7fa13c864700 1 -- 172.21.15.146:0/1581337290 wait complete. 2024-09-13T18:48:35.086 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 88 2024-09-13T18:48:35.097 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 88, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 88, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6459": { "addr": "172.21.15.146:6835/294014284", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 294014284, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 294014284, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6459, "incarnation": 74, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 13 }, "gid_6714": { "addr": "172.21.15.146:6837/2169954140", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 2169954140, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 2169954140, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6714, "incarnation": 86, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:reconnect", "state_seq": 34 } }, "last_failure": 0, "last_failure_osd_epoch": 39, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:48:30.909275+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6459, "mds_1": 6714 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.146:6833/3023779168", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3023779168, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3023779168, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 69, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6187, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:48:37.099 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:37.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.197+0000 7fd5311dd700 1 Processor -- start 2024-09-13T18:48:37.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.197+0000 7fd5311dd700 1 -- start start 2024-09-13T18:48:37.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5311dd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd52c0c8380 0x7fd52c162b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:37.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5311dd700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52c0c9890 con 0x7fd52c151840 2024-09-13T18:48:37.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5311dd700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52c0c9510 con 0x7fd52c157fc0 2024-09-13T18:48:37.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5311dd700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52c0c36f0 con 0x7fd52c0c8380 2024-09-13T18:48:37.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd52bfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd52c151840 0x7fd52c0c1f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48272/0 (socket says 172.21.15.146:48272) 2024-09-13T18:48:37.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd52bfff700 1 -- 172.21.15.146:0/1629482028 learned_addr learned my addr 172.21.15.146:0/1629482028 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:48:37.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5309dc700 1 --2- 172.21.15.146:0/1629482028 >> v2:172.21.15.146:3300/0 conn(0x7fd52c0c8380 0x7fd52c162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:37.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd52affd700 1 -- 172.21.15.146:0/1629482028 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531427943 0 0) 0x7fd52c0c9890 con 0x7fd52c151840 2024-09-13T18:48:37.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd52affd700 1 -- 172.21.15.146:0/1629482028 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5180035d0 con 0x7fd52c151840 2024-09-13T18:48:37.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5309dc700 1 -- 172.21.15.146:0/1629482028 >> v1:172.21.15.154:6789/0 conn(0x7fd52c157fc0 legacy=0x7fd52c06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:37.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5309dc700 1 -- 172.21.15.146:0/1629482028 >> v1:172.21.15.146:6789/0 conn(0x7fd52c151840 legacy=0x7fd52c0c1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:37.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5309dc700 1 -- 172.21.15.146:0/1629482028 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c0c7490 con 0x7fd52c0c8380 2024-09-13T18:48:37.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5309dc700 1 --2- 172.21.15.146:0/1629482028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd52c0c8380 0x7fd52c162b80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd524009d90 tx=0x7fd52400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c135d33265851a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:37.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd52affd700 1 -- 172.21.15.146:0/1629482028 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52400e070 con 0x7fd52c0c8380 2024-09-13T18:48:37.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd52affd700 1 -- 172.21.15.146:0/1629482028 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52400bb20 con 0x7fd52c0c8380 2024-09-13T18:48:37.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd52affd700 1 -- 172.21.15.146:0/1629482028 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52400be90 con 0x7fd52c0c8380 2024-09-13T18:48:37.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5311dd700 1 -- 172.21.15.146:0/1629482028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd52c0c8380 msgr2=0x7fd52c162b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:37.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5311dd700 1 --2- 172.21.15.146:0/1629482028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd52c0c8380 0x7fd52c162b80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd524009d90 tx=0x7fd52400b330 comp rx=0 tx=0).stop 2024-09-13T18:48:37.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5311dd700 1 -- 172.21.15.146:0/1629482028 shutdown_connections 2024-09-13T18:48:37.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5311dd700 1 --2- 172.21.15.146:0/1629482028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd52c0c8380 0x7fd52c162b80 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:37.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5311dd700 1 -- 172.21.15.146:0/1629482028 >> 172.21.15.146:0/1629482028 conn(0x7fd52c0ccf70 msgr2=0x7fd52c0cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:37.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5311dd700 1 -- 172.21.15.146:0/1629482028 shutdown_connections 2024-09-13T18:48:37.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.201+0000 7fd5311dd700 1 -- 172.21.15.146:0/1629482028 wait complete. 2024-09-13T18:48:37.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd5311dd700 1 Processor -- start 2024-09-13T18:48:37.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd5311dd700 1 -- start start 2024-09-13T18:48:37.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd5311dd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd52c0c8380 0x7fd52c155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:37.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd5311dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd52c151840 0x7fd52c1f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:37.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd5311dd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd52c157fc0 0x7fd52c06afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:37.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd5311dd700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c1f0a50 con 0x7fd52c0c8380 2024-09-13T18:48:37.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd5311dd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c1de950 con 0x7fd52c151840 2024-09-13T18:48:37.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd5311dd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c1f0420 con 0x7fd52c157fc0 2024-09-13T18:48:37.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd52bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd52c0c8380 0x7fd52c155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:37.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd52bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd52c0c8380 0x7fd52c155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58080/0 (socket says 172.21.15.146:58080) 2024-09-13T18:48:37.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd52bfff700 1 -- 172.21.15.146:0/428383581 learned_addr learned my addr 172.21.15.146:0/428383581 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:37.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd52b7fe700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd52c151840 0x7fd52c1f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:37.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd5309dc700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd52c157fc0 0x7fd52c06afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:37.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd52bfff700 1 -- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd52c157fc0 msgr2=0x7fd52c06afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:37.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd52bfff700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd52c157fc0 0x7fd52c06afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:37.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd52bfff700 1 -- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd52c151840 msgr2=0x7fd52c1f0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:37.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd52bfff700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd52c151840 0x7fd52c1f0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:37.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd52bfff700 1 -- 172.21.15.146:0/428383581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd524009a70 con 0x7fd52c0c8380 2024-09-13T18:48:37.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd52b7fe700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd52c151840 0x7fd52c1f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:48:37.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd52bfff700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd52c0c8380 0x7fd52c155110 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c700 tx=0x7fd51c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:37.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd528ff9700 1 -- 172.21.15.146:0/428383581 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c005d90 con 0x7fd52c0c8380 2024-09-13T18:48:37.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd528ff9700 1 -- 172.21.15.146:0/428383581 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c0070d0 con 0x7fd52c0c8380 2024-09-13T18:48:37.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd5311dd700 1 -- 172.21.15.146:0/428383581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c0cc3d0 con 0x7fd52c0c8380 2024-09-13T18:48:37.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd528ff9700 1 -- 172.21.15.146:0/428383581 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c007480 con 0x7fd52c0c8380 2024-09-13T18:48:37.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.205+0000 7fd5311dd700 1 -- 172.21.15.146:0/428383581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c0ccaa0 con 0x7fd52c0c8380 2024-09-13T18:48:37.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.209+0000 7fd5311dd700 1 -- 172.21.15.146:0/428383581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c067a60 con 0x7fd52c0c8380 2024-09-13T18:48:37.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.209+0000 7fd528ff9700 1 -- 172.21.15.146:0/428383581 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd51c0075e0 con 0x7fd52c0c8380 2024-09-13T18:48:37.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.209+0000 7fd528ff9700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd50c066730 0x7fd50c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:37.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.209+0000 7fd528ff9700 1 -- 172.21.15.146:0/428383581 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7fd51c083d20 con 0x7fd52c0c8380 2024-09-13T18:48:37.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.209+0000 7fd52b7fe700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd50c066730 0x7fd50c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:37.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.209+0000 7fd52b7fe700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd50c066730 0x7fd50c068bf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0cbbb0 tx=0x7fd514009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:37.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.217+0000 7fd528ff9700 1 -- 172.21.15.146:0/428383581 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd51c050960 con 0x7fd52c0c8380 2024-09-13T18:48:37.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.449+0000 7fd5311dd700 1 -- 172.21.15.146:0/428383581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd52c157aa0 con 0x7fd52c0c8380 2024-09-13T18:48:37.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.449+0000 7fd528ff9700 1 -- 172.21.15.146:0/428383581 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5519 (secure 0 0 0) 0x7fd51c053f80 con 0x7fd52c0c8380 2024-09-13T18:48:37.466 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:37.467 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:36.122000+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:rejoin","state_seq":36,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:37.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.453+0000 7fd5311dd700 1 -- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd50c066730 msgr2=0x7fd50c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:37.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.453+0000 7fd5311dd700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd50c066730 0x7fd50c068bf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0cbbb0 tx=0x7fd514009040 comp rx=0 tx=0).stop 2024-09-13T18:48:37.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.453+0000 7fd5311dd700 1 -- 172.21.15.146:0/428383581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd52c0c8380 msgr2=0x7fd52c155110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:37.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.453+0000 7fd5311dd700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd52c0c8380 0x7fd52c155110 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c700 tx=0x7fd51c00ca10 comp rx=0 tx=0).stop 2024-09-13T18:48:37.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.453+0000 7fd5311dd700 1 -- 172.21.15.146:0/428383581 shutdown_connections 2024-09-13T18:48:37.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.453+0000 7fd5311dd700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd50c066730 0x7fd50c068bf0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:37.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.453+0000 7fd5311dd700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd52c157fc0 0x7fd52c06afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:37.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.453+0000 7fd5311dd700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd52c151840 0x7fd52c1f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:37.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.453+0000 7fd5311dd700 1 --2- 172.21.15.146:0/428383581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd52c0c8380 0x7fd52c155110 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:37.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.453+0000 7fd5311dd700 1 -- 172.21.15.146:0/428383581 >> 172.21.15.146:0/428383581 conn(0x7fd52c0ccf70 msgr2=0x7fd52c0c1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:37.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.453+0000 7fd5311dd700 1 -- 172.21.15.146:0/428383581 shutdown_connections 2024-09-13T18:48:37.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:37.453+0000 7fd5311dd700 1 -- 172.21.15.146:0/428383581 wait complete. 2024-09-13T18:48:37.476 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 89 2024-09-13T18:48:37.482 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:48:39.483 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:39.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1486e1e700 1 Processor -- start 2024-09-13T18:48:39.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1486e1e700 1 -- start start 2024-09-13T18:48:39.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1486e1e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480151590 0x7f14801587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:39.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1486e1e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f148006db40 0x7f1480161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:39.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1486e1e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1480158d10 con 0x7f1480157fd0 2024-09-13T18:48:39.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1486e1e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14800c69b0 con 0x7f148006db40 2024-09-13T18:48:39.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1486e1e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14801572e0 con 0x7f1480151590 2024-09-13T18:48:39.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f148561b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1480157fd0 0x7f14800c4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48276/0 (socket says 172.21.15.146:48276) 2024-09-13T18:48:39.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f148561b700 1 -- 172.21.15.146:0/550448027 learned_addr learned my addr 172.21.15.146:0/550448027 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:48:39.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1485e1c700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480151590 0x7f14801587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:39.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f148661d700 1 --2- 172.21.15.146:0/550448027 >> v2:172.21.15.154:3300/0 conn(0x7f148006db40 0x7f1480161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:39.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119940993 0 0) 0x7f1480158d10 con 0x7f1480157fd0 2024-09-13T18:48:39.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f146c0035d0 con 0x7f1480157fd0 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 811313164 0 0) 0x7f146c0035d0 con 0x7f1480157fd0 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480151590 msgr2=0x7f14801587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480151590 0x7f14801587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 >> v2:172.21.15.154:3300/0 conn(0x7f148006db40 msgr2=0x7f1480161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 --2- 172.21.15.146:0/550448027 >> v2:172.21.15.154:3300/0 conn(0x7f148006db40 0x7f1480161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1480066500 con 0x7f1480157fd0 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f1470001670 con 0x7f1480157fd0 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 >> v1:172.21.15.146:6789/0 conn(0x7f1480157fd0 legacy=0x7f14800c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f146c003a90 0x7f146c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f146c004870 0x7f146c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480151590 0x7f146c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14800c69b0 con 0x7f146c003a90 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801572e0 con 0x7f146c004870 2024-09-13T18:48:39.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c00ba80 con 0x7f1480151590 2024-09-13T18:48:39.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.585+0000 7f1485e1c700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f146c003a90 0x7f146c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:39.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f148661d700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480151590 0x7f146c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:39.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f148561b700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f146c004870 0x7f146c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:39.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1485e1c700 1 -- 172.21.15.146:0/550448027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480151590 msgr2=0x7f146c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:39.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1485e1c700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480151590 0x7f146c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:39.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1485e1c700 1 -- 172.21.15.146:0/550448027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f146c004870 msgr2=0x7f146c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:39.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1485e1c700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f146c004870 0x7f146c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:39.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1485e1c700 1 -- 172.21.15.146:0/550448027 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0035d0 con 0x7f146c003a90 2024-09-13T18:48:39.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f148661d700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480151590 0x7f146c005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:48:39.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f148561b700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f146c004870 0x7f146c004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:48:39.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1485e1c700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f146c003a90 0x7f146c003eb0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f147c009b70 tx=0x7f146c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f417e29f47e2ed1f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:39.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c00e6c0 con 0x7f146c003a90 2024-09-13T18:48:39.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c00e820 con 0x7f146c003a90 2024-09-13T18:48:39.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1484e1a700 1 -- 172.21.15.146:0/550448027 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c00eb90 con 0x7f146c003a90 2024-09-13T18:48:39.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1486e1e700 1 -- 172.21.15.146:0/550448027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f146c003a90 msgr2=0x7f146c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:39.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1486e1e700 1 --2- 172.21.15.146:0/550448027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f146c003a90 0x7f146c003eb0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f147c009b70 tx=0x7f146c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:48:39.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f148661d700 1 -- 172.21.15.146:0/550448027 reap_dead start 2024-09-13T18:48:39.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1486e1e700 1 -- 172.21.15.146:0/550448027 shutdown_connections 2024-09-13T18:48:39.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1486e1e700 1 -- 172.21.15.146:0/550448027 >> 172.21.15.146:0/550448027 conn(0x7f1480066b00 msgr2=0x7f148006aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:39.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1486e1e700 1 -- 172.21.15.146:0/550448027 shutdown_connections 2024-09-13T18:48:39.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1486e1e700 1 -- 172.21.15.146:0/550448027 wait complete. 2024-09-13T18:48:39.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1486e1e700 1 Processor -- start 2024-09-13T18:48:39.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.589+0000 7f1486e1e700 1 -- start start 2024-09-13T18:48:39.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f1486e1e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f146c003a90 0x7f1480150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:39.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f1486e1e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14800c21b0 0x7f14801532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:39.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f1486e1e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14801537e0 0x7f148014c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:39.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f1486e1e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1480152e10 con 0x7f14800c21b0 2024-09-13T18:48:39.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f1486e1e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801e0a50 con 0x7f14801537e0 2024-09-13T18:48:39.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f1486e1e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1480153c60 con 0x7f146c003a90 2024-09-13T18:48:39.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f148561b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14800c21b0 0x7f14801532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:39.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f148561b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14800c21b0 0x7f14801532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58112/0 (socket says 172.21.15.146:58112) 2024-09-13T18:48:39.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f148561b700 1 -- 172.21.15.146:0/3774836133 learned_addr learned my addr 172.21.15.146:0/3774836133 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:39.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f1485e1c700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f146c003a90 0x7f1480150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:39.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f148661d700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14801537e0 0x7f148014c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:39.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f148561b700 1 -- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f146c003a90 msgr2=0x7f1480150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:39.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f148561b700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f146c003a90 0x7f1480150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:39.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f148561b700 1 -- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14801537e0 msgr2=0x7f148014c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:39.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f148561b700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14801537e0 0x7f148014c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:39.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f148561b700 1 -- 172.21.15.146:0/3774836133 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14801e07b0 con 0x7f14800c21b0 2024-09-13T18:48:39.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f148661d700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14801537e0 0x7f148014c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:48:39.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f148561b700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14800c21b0 0x7f14801532a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f147000d8e0 tx=0x7f147000dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:39.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f14767fc700 1 -- 172.21.15.146:0/3774836133 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470007560 con 0x7f14800c21b0 2024-09-13T18:48:39.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f14767fc700 1 -- 172.21.15.146:0/3774836133 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14700076c0 con 0x7f14800c21b0 2024-09-13T18:48:39.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f1486e1e700 1 -- 172.21.15.146:0/3774836133 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14801e1740 con 0x7f14800c21b0 2024-09-13T18:48:39.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f14767fc700 1 -- 172.21.15.146:0/3774836133 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470004ad0 con 0x7f14800c21b0 2024-09-13T18:48:39.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f1486e1e700 1 -- 172.21.15.146:0/3774836133 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14801e1d00 con 0x7f14800c21b0 2024-09-13T18:48:39.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f1486e1e700 1 -- 172.21.15.146:0/3774836133 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14801e30d0 con 0x7f14800c21b0 2024-09-13T18:48:39.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f14767fc700 1 -- 172.21.15.146:0/3774836133 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1470004c30 con 0x7f14800c21b0 2024-09-13T18:48:39.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.593+0000 7f14767fc700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1464066760 0x7f1464068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:39.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.597+0000 7f14767fc700 1 -- 172.21.15.146:0/3774836133 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f1470084400 con 0x7f14800c21b0 2024-09-13T18:48:39.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.597+0000 7f1485e1c700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1464066760 0x7f1464068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:39.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.597+0000 7f1485e1c700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1464066760 0x7f1464068c20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f147c0090b0 tx=0x7f147c01e040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:39.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.601+0000 7f14767fc700 1 -- 172.21.15.146:0/3774836133 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1470051040 con 0x7f14800c21b0 2024-09-13T18:48:39.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.829+0000 7f1486e1e700 1 -- 172.21.15.146:0/3774836133 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14801e1e90 con 0x7f14800c21b0 2024-09-13T18:48:39.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.833+0000 7f14767fc700 1 -- 172.21.15.146:0/3774836133 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5519 (secure 0 0 0) 0x7f1470054660 con 0x7f14800c21b0 2024-09-13T18:48:39.847 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:39.847 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:36.122000+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:rejoin","state_seq":36,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:39.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.833+0000 7f1486e1e700 1 -- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1464066760 msgr2=0x7f1464068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:39.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.833+0000 7f1486e1e700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1464066760 0x7f1464068c20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f147c0090b0 tx=0x7f147c01e040 comp rx=0 tx=0).stop 2024-09-13T18:48:39.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.833+0000 7f1486e1e700 1 -- 172.21.15.146:0/3774836133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14800c21b0 msgr2=0x7f14801532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:39.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.833+0000 7f1486e1e700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14800c21b0 0x7f14801532a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f147000d8e0 tx=0x7f147000dbf0 comp rx=0 tx=0).stop 2024-09-13T18:48:39.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.833+0000 7f1486e1e700 1 -- 172.21.15.146:0/3774836133 shutdown_connections 2024-09-13T18:48:39.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.833+0000 7f1486e1e700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1464066760 0x7f1464068c20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:39.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.833+0000 7f1486e1e700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14801537e0 0x7f148014c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:39.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.833+0000 7f1486e1e700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14800c21b0 0x7f14801532a0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:39.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.833+0000 7f1486e1e700 1 --2- 172.21.15.146:0/3774836133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f146c003a90 0x7f1480150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:39.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.833+0000 7f1486e1e700 1 -- 172.21.15.146:0/3774836133 >> 172.21.15.146:0/3774836133 conn(0x7f1480066b00 msgr2=0x7f1480154f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:39.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.833+0000 7f1486e1e700 1 -- 172.21.15.146:0/3774836133 shutdown_connections 2024-09-13T18:48:39.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:39.833+0000 7f1486e1e700 1 -- 172.21.15.146:0/3774836133 wait complete. 2024-09-13T18:48:39.851 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 89 2024-09-13T18:48:39.864 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:48:41.006 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:48:41.008 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:48:41.865 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:41.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.965+0000 7f48fb59e700 1 Processor -- start 2024-09-13T18:48:41.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.965+0000 7f48fb59e700 1 -- start start 2024-09-13T18:48:41.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48fb59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48fc06eba0 0x7f48fc0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:41.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48fb59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48fc0c1020 con 0x7f48fc15a000 2024-09-13T18:48:41.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48fb59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48fc1593f0 con 0x7f48fc06e7c0 2024-09-13T18:48:41.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48fb59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0c6a70 con 0x7f48fc06eba0 2024-09-13T18:48:41.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48fad9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f48fc15a000 0x7f48fc1531a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:38256/0 (socket says 172.21.15.146:38256) 2024-09-13T18:48:41.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48fad9d700 1 -- 172.21.15.146:0/435690419 learned_addr learned my addr 172.21.15.146:0/435690419 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:48:41.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48f3fff700 1 --2- 172.21.15.146:0/435690419 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48fc06eba0 0x7f48fc0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:41.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48f9d9b700 1 -- 172.21.15.146:0/435690419 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147136598 0 0) 0x7f48fc0c1020 con 0x7f48fc15a000 2024-09-13T18:48:41.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48f9d9b700 1 -- 172.21.15.146:0/435690419 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e80035d0 con 0x7f48fc15a000 2024-09-13T18:48:41.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48f9d9b700 1 -- 172.21.15.146:0/435690419 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049638144 0 0) 0x7f48fc1593f0 con 0x7f48fc06e7c0 2024-09-13T18:48:41.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48f9d9b700 1 -- 172.21.15.146:0/435690419 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48fc0c1020 con 0x7f48fc06e7c0 2024-09-13T18:48:41.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48f9d9b700 1 -- 172.21.15.146:0/435690419 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3916037845 0 0) 0x7f48e80035d0 con 0x7f48fc15a000 2024-09-13T18:48:41.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48f3fff700 1 -- 172.21.15.146:0/435690419 >> v1:172.21.15.154:6789/0 conn(0x7f48fc06e7c0 legacy=0x7f48fc1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:41.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48f3fff700 1 -- 172.21.15.146:0/435690419 >> v1:172.21.15.146:6789/0 conn(0x7f48fc15a000 legacy=0x7f48fc1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:41.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48f3fff700 1 -- 172.21.15.146:0/435690419 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc0c1290 con 0x7f48fc06eba0 2024-09-13T18:48:41.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48f3fff700 1 --2- 172.21.15.146:0/435690419 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48fc06eba0 0x7f48fc0b11a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009880 tx=0x7f48e4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34976e3c312f8c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:41.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48f9d9b700 1 -- 172.21.15.146:0/435690419 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40019f0 con 0x7f48fc06eba0 2024-09-13T18:48:41.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48f9d9b700 1 -- 172.21.15.146:0/435690419 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e4004030 con 0x7f48fc06eba0 2024-09-13T18:48:41.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48f9d9b700 1 -- 172.21.15.146:0/435690419 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40043a0 con 0x7f48fc06eba0 2024-09-13T18:48:41.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48fb59e700 1 -- 172.21.15.146:0/435690419 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48fc06eba0 msgr2=0x7f48fc0b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:41.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48fb59e700 1 --2- 172.21.15.146:0/435690419 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48fc06eba0 0x7f48fc0b11a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009880 tx=0x7f48e4009b90 comp rx=0 tx=0).stop 2024-09-13T18:48:41.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48fb59e700 1 -- 172.21.15.146:0/435690419 shutdown_connections 2024-09-13T18:48:41.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48fb59e700 1 --2- 172.21.15.146:0/435690419 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48fc06eba0 0x7f48fc0b11a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:41.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48fb59e700 1 -- 172.21.15.146:0/435690419 >> 172.21.15.146:0/435690419 conn(0x7f48fc071ed0 msgr2=0x7f48fc0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:41.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48fb59e700 1 -- 172.21.15.146:0/435690419 shutdown_connections 2024-09-13T18:48:41.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.969+0000 7f48fb59e700 1 -- 172.21.15.146:0/435690419 wait complete. 2024-09-13T18:48:41.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.973+0000 7f48fb59e700 1 Processor -- start 2024-09-13T18:48:41.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.973+0000 7f48fb59e700 1 -- start start 2024-09-13T18:48:41.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.973+0000 7f48fb59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48fc06e7c0 0x7f48fc158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:41.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.973+0000 7f48fb59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48fc06eba0 0x7f48fc1e1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:41.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.973+0000 7f48fb59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48fc15a000 0x7f48fc151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:41.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.973+0000 7f48fb59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc1efc40 con 0x7f48fc15a000 2024-09-13T18:48:41.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.973+0000 7f48fb59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc159350 con 0x7f48fc06eba0 2024-09-13T18:48:41.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.973+0000 7f48fb59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc159070 con 0x7f48fc06e7c0 2024-09-13T18:48:41.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.973+0000 7f48fad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48fc15a000 0x7f48fc151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:41.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.973+0000 7f48fad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48fc15a000 0x7f48fc151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35508/0 (socket says 172.21.15.146:35508) 2024-09-13T18:48:41.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.973+0000 7f48fad9d700 1 -- 172.21.15.146:0/3586681470 learned_addr learned my addr 172.21.15.146:0/3586681470 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:41.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48fa59c700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48fc06e7c0 0x7f48fc158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:41.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48f3fff700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48fc06eba0 0x7f48fc1e1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:41.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48fad9d700 1 -- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48fc06e7c0 msgr2=0x7f48fc158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:41.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48fad9d700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48fc06e7c0 0x7f48fc158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:41.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48fad9d700 1 -- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48fc06eba0 msgr2=0x7f48fc1e1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:41.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48fad9d700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48fc06eba0 0x7f48fc1e1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:41.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48fad9d700 1 -- 172.21.15.146:0/3586681470 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e40094b0 con 0x7f48fc15a000 2024-09-13T18:48:41.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48fad9d700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48fc15a000 0x7f48fc151ef0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f48f400cbc0 tx=0x7f48f400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:41.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48f2ffd700 1 -- 172.21.15.146:0/3586681470 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f400aaa0 con 0x7f48fc15a000 2024-09-13T18:48:41.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48f2ffd700 1 -- 172.21.15.146:0/3586681470 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f400ac20 con 0x7f48fc15a000 2024-09-13T18:48:41.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48fb59e700 1 -- 172.21.15.146:0/3586681470 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc1eee80 con 0x7f48fc15a000 2024-09-13T18:48:41.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48f2ffd700 1 -- 172.21.15.146:0/3586681470 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40070d0 con 0x7f48fc15a000 2024-09-13T18:48:41.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48fb59e700 1 -- 172.21.15.146:0/3586681470 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc1ef440 con 0x7f48fc15a000 2024-09-13T18:48:41.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48f2ffd700 1 -- 172.21.15.146:0/3586681470 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f48f4007230 con 0x7f48fc15a000 2024-09-13T18:48:41.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.977+0000 7f48fb59e700 1 -- 172.21.15.146:0/3586681470 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48fc0b11a0 con 0x7f48fc15a000 2024-09-13T18:48:41.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.981+0000 7f48f2ffd700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48dc066730 0x7f48dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:41.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.981+0000 7f48f2ffd700 1 -- 172.21.15.146:0/3586681470 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f48f4084700 con 0x7f48fc15a000 2024-09-13T18:48:41.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.981+0000 7f48fa59c700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48dc066730 0x7f48dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:41.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.981+0000 7f48fa59c700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48dc066730 0x7f48dc068bf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0011e0 tx=0x7f48ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:42.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:41.985+0000 7f48f2ffd700 1 -- 172.21.15.146:0/3586681470 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f48f4051220 con 0x7f48fc15a000 2024-09-13T18:48:42.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.229+0000 7f48fb59e700 1 -- 172.21.15.146:0/3586681470 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48fc06e310 con 0x7f48fc15a000 2024-09-13T18:48:42.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.229+0000 7f48f2ffd700 1 -- 172.21.15.146:0/3586681470 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5519 (secure 0 0 0) 0x7f48f4018090 con 0x7f48fc15a000 2024-09-13T18:48:42.244 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:42.244 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:41.442913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:42.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.229+0000 7f48fb59e700 1 -- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48dc066730 msgr2=0x7f48dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:42.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.229+0000 7f48fb59e700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48dc066730 0x7f48dc068bf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0011e0 tx=0x7f48ec009040 comp rx=0 tx=0).stop 2024-09-13T18:48:42.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.233+0000 7f48fb59e700 1 -- 172.21.15.146:0/3586681470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48fc15a000 msgr2=0x7f48fc151ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:42.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.233+0000 7f48fb59e700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48fc15a000 0x7f48fc151ef0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f48f400cbc0 tx=0x7f48f400ced0 comp rx=0 tx=0).stop 2024-09-13T18:48:42.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.233+0000 7f48fb59e700 1 -- 172.21.15.146:0/3586681470 shutdown_connections 2024-09-13T18:48:42.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.233+0000 7f48fb59e700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48dc066730 0x7f48dc068bf0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:42.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.233+0000 7f48fb59e700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48fc15a000 0x7f48fc151ef0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:42.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.233+0000 7f48fb59e700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48fc06eba0 0x7f48fc1e1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:42.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.233+0000 7f48fb59e700 1 --2- 172.21.15.146:0/3586681470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48fc06e7c0 0x7f48fc158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:42.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.233+0000 7f48fb59e700 1 -- 172.21.15.146:0/3586681470 >> 172.21.15.146:0/3586681470 conn(0x7f48fc071ed0 msgr2=0x7f48fc0cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:42.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.233+0000 7f48fb59e700 1 -- 172.21.15.146:0/3586681470 shutdown_connections 2024-09-13T18:48:42.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:42.233+0000 7f48fb59e700 1 -- 172.21.15.146:0/3586681470 wait complete. 2024-09-13T18:48:42.248 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 90 2024-09-13T18:48:42.258 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=1, replacing gid=6370 2024-09-13T18:48:42.258 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.f 2024-09-13T18:48:47.259 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2024-09-13T18:48:47.259 INFO:tasks.ceph.mds.f:Restarting daemon 2024-09-13T18:48:47.259 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-09-13T18:48:47.262 INFO:tasks.ceph.mds.f:Started 2024-09-13T18:48:47.262 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:48:47.350 INFO:tasks.ceph.mds.f.smithi154.stdout:starting mds.f at 2024-09-13T18:48:47.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.361+0000 7ff776ceb700 1 Processor -- start 2024-09-13T18:48:47.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.361+0000 7ff776ceb700 1 -- start start 2024-09-13T18:48:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.361+0000 7ff776ceb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff7700c6590 0x7ff77015d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff776ceb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff7700c5440 0x7ff770153340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff776ceb700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff770066780 con 0x7ff7700c61b0 2024-09-13T18:48:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff776ceb700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7701594a0 con 0x7ff7700c5440 2024-09-13T18:48:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff776ceb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77015d960 con 0x7ff7700c6590 2024-09-13T18:48:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff7764ea700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff7700c5440 0x7ff770153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff7764ea700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff7700c5440 0x7ff770153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35524/0 (socket says 172.21.15.146:35524) 2024-09-13T18:48:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff7764ea700 1 -- 172.21.15.146:0/3916801952 learned_addr learned my addr 172.21.15.146:0/3916801952 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff775ce9700 1 --1- 172.21.15.146:0/3916801952 >> v1:172.21.15.154:6789/0 conn(0x7ff7700c61b0 0x7ff77006cd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:48532/0 (socket says 172.21.15.146:48532) 2024-09-13T18:48:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff7754e8700 1 --2- 172.21.15.146:0/3916801952 >> v2:172.21.15.154:3300/0 conn(0x7ff7700c6590 0x7ff77015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff7764ea700 1 -- 172.21.15.146:0/3916801952 >> v2:172.21.15.154:3300/0 conn(0x7ff7700c6590 msgr2=0x7ff77015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:48:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff7764ea700 1 --2- 172.21.15.146:0/3916801952 >> v2:172.21.15.154:3300/0 conn(0x7ff7700c6590 0x7ff77015d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff7764ea700 1 -- 172.21.15.146:0/3916801952 >> v1:172.21.15.154:6789/0 conn(0x7ff7700c61b0 legacy=0x7ff77006cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff7764ea700 1 -- 172.21.15.146:0/3916801952 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77006eec0 con 0x7ff7700c5440 2024-09-13T18:48:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff7764ea700 1 --2- 172.21.15.146:0/3916801952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7700c5440 0x7ff770153340 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff77006ebb0 tx=0x7ff76800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1182eb710ebc18 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff774ce7700 1 -- 172.21.15.146:0/3916801952 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76800e070 con 0x7ff7700c5440 2024-09-13T18:48:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff774ce7700 1 -- 172.21.15.146:0/3916801952 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76800b910 con 0x7ff7700c5440 2024-09-13T18:48:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff774ce7700 1 -- 172.21.15.146:0/3916801952 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76800bcc0 con 0x7ff7700c5440 2024-09-13T18:48:47.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff776ceb700 1 -- 172.21.15.146:0/3916801952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7700c5440 msgr2=0x7ff770153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:47.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff776ceb700 1 --2- 172.21.15.146:0/3916801952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7700c5440 0x7ff770153340 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff77006ebb0 tx=0x7ff76800b330 comp rx=0 tx=0).stop 2024-09-13T18:48:47.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff776ceb700 1 -- 172.21.15.146:0/3916801952 shutdown_connections 2024-09-13T18:48:47.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff776ceb700 1 --2- 172.21.15.146:0/3916801952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7700c5440 0x7ff770153340 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:47.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff776ceb700 1 --2- 172.21.15.146:0/3916801952 >> v2:172.21.15.154:3300/0 conn(0x7ff7700c6590 0x7ff77015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:47.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff776ceb700 1 -- 172.21.15.146:0/3916801952 >> 172.21.15.146:0/3916801952 conn(0x7ff770066b00 msgr2=0x7ff77006aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:47.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff776ceb700 1 -- 172.21.15.146:0/3916801952 shutdown_connections 2024-09-13T18:48:47.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.365+0000 7ff776ceb700 1 -- 172.21.15.146:0/3916801952 wait complete. 2024-09-13T18:48:47.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff776ceb700 1 Processor -- start 2024-09-13T18:48:47.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff776ceb700 1 -- start start 2024-09-13T18:48:47.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff776ceb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7700c5440 0x7ff7700c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:47.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff776ceb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7700c61b0 0x7ff7701f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:47.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff776ceb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7700c6590 0x7ff7701ef660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:47.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff776ceb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7700c42f0 con 0x7ff7700c61b0 2024-09-13T18:48:47.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff776ceb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77015c480 con 0x7ff7700c5440 2024-09-13T18:48:47.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff776ceb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7701f2680 con 0x7ff7700c6590 2024-09-13T18:48:47.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff7754e8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7700c61b0 0x7ff7701f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:47.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff7754e8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7700c61b0 0x7ff7701f2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35536/0 (socket says 172.21.15.146:35536) 2024-09-13T18:48:47.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff7754e8700 1 -- 172.21.15.146:0/1820703649 learned_addr learned my addr 172.21.15.146:0/1820703649 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:48:47.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff775ce9700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7700c5440 0x7ff7700c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:47.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff7764ea700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7700c6590 0x7ff7701ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:47.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff7754e8700 1 -- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7700c6590 msgr2=0x7ff7701ef660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:47.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff7754e8700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7700c6590 0x7ff7701ef660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:47.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff7754e8700 1 -- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7700c5440 msgr2=0x7ff7700c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:47.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff7754e8700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7700c5440 0x7ff7700c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:47.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff7754e8700 1 -- 172.21.15.146:0/1820703649 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff768009b30 con 0x7ff7700c61b0 2024-09-13T18:48:47.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff7754e8700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7700c61b0 0x7ff7701f2df0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff76000e9d0 tx=0x7ff76000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:47.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff7667fc700 1 -- 172.21.15.146:0/1820703649 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff760009720 con 0x7ff7700c61b0 2024-09-13T18:48:47.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff7667fc700 1 -- 172.21.15.146:0/1820703649 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76000ee70 con 0x7ff7700c61b0 2024-09-13T18:48:47.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff776ceb700 1 -- 172.21.15.146:0/1820703649 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7701f2960 con 0x7ff7700c61b0 2024-09-13T18:48:47.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.369+0000 7ff7667fc700 1 -- 172.21.15.146:0/1820703649 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76000ca50 con 0x7ff7700c61b0 2024-09-13T18:48:47.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.373+0000 7ff776ceb700 1 -- 172.21.15.146:0/1820703649 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7701e4480 con 0x7ff7700c61b0 2024-09-13T18:48:47.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.373+0000 7ff776ceb700 1 -- 172.21.15.146:0/1820703649 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff770159f50 con 0x7ff7700c61b0 2024-09-13T18:48:47.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.373+0000 7ff7667fc700 1 -- 172.21.15.146:0/1820703649 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff76000cbb0 con 0x7ff7700c61b0 2024-09-13T18:48:47.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.373+0000 7ff7667fc700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff754066730 0x7ff754068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:48:47.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.373+0000 7ff7667fc700 1 -- 172.21.15.146:0/1820703649 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7ff760084d10 con 0x7ff7700c61b0 2024-09-13T18:48:47.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.373+0000 7ff775ce9700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff754066730 0x7ff754068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:48:47.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.373+0000 7ff775ce9700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff754066730 0x7ff754068bf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff76c000f60 tx=0x7ff76c007ef0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:48:47.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.381+0000 7ff7667fc700 1 -- 172.21.15.146:0/1820703649 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff760051830 con 0x7ff7700c61b0 2024-09-13T18:48:47.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.609+0000 7ff776ceb700 1 -- 172.21.15.146:0/1820703649 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7701e56c0 con 0x7ff7700c61b0 2024-09-13T18:48:47.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.609+0000 7ff7667fc700 1 -- 172.21.15.146:0/1820703649 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+6279 (secure 0 0 0) 0x7ff760021090 con 0x7ff7700c61b0 2024-09-13T18:48:47.626 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:48:47.626 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:41.442913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:48:47.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.613+0000 7ff776ceb700 1 -- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff754066730 msgr2=0x7ff754068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:47.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.613+0000 7ff776ceb700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff754066730 0x7ff754068bf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff76c000f60 tx=0x7ff76c007ef0 comp rx=0 tx=0).stop 2024-09-13T18:48:47.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.613+0000 7ff776ceb700 1 -- 172.21.15.146:0/1820703649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7700c61b0 msgr2=0x7ff7701f2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:48:47.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.613+0000 7ff776ceb700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7700c61b0 0x7ff7701f2df0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff76000e9d0 tx=0x7ff76000ece0 comp rx=0 tx=0).stop 2024-09-13T18:48:47.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.613+0000 7ff776ceb700 1 -- 172.21.15.146:0/1820703649 shutdown_connections 2024-09-13T18:48:47.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.613+0000 7ff776ceb700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ff754066730 0x7ff754068bf0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:47.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.613+0000 7ff776ceb700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7700c6590 0x7ff7701ef660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:47.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.613+0000 7ff776ceb700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7700c61b0 0x7ff7701f2df0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:47.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.613+0000 7ff776ceb700 1 --2- 172.21.15.146:0/1820703649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7700c5440 0x7ff7700c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:48:47.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.613+0000 7ff776ceb700 1 -- 172.21.15.146:0/1820703649 >> 172.21.15.146:0/1820703649 conn(0x7ff770066b00 msgr2=0x7ff77014fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:48:47.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.613+0000 7ff776ceb700 1 -- 172.21.15.146:0/1820703649 shutdown_connections 2024-09-13T18:48:47.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:48:47.613+0000 7ff776ceb700 1 -- 172.21.15.146:0/1820703649 wait complete. 2024-09-13T18:48:47.630 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 91 2024-09-13T18:48:47.641 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2024-09-13T18:48:47.642 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 106 secs before thrashing 2024-09-13T18:49:11.457 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:49:11.460 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:49:41.499 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:49:42.221 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:50:12.269 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:50:12.272 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:50:33.643 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:50:33.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190b59e700 1 Processor -- start 2024-09-13T18:50:33.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190b59e700 1 -- start start 2024-09-13T18:50:33.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190b59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f190c0c9ed0 0x7f190c159700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:33.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190b59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f190c0cf540 0x7f190c0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:33.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190b59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c0cc2d0 con 0x7f190c153a60 2024-09-13T18:50:33.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190b59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f190c0b3990 con 0x7f190c0cf540 2024-09-13T18:50:33.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190b59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f190c153e40 con 0x7f190c0c9ed0 2024-09-13T18:50:33.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190a59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f190c153a60 0x7f190c0b3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50700/0 (socket says 172.21.15.146:50700) 2024-09-13T18:50:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190a59c700 1 -- 172.21.15.146:0/298944710 learned_addr learned my addr 172.21.15.146:0/298944710 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:50:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190ad9d700 1 --2- 172.21.15.146:0/298944710 >> v2:172.21.15.146:3300/0 conn(0x7f190c0cf540 0x7f190c0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f1909d9b700 1 --2- 172.21.15.146:0/298944710 >> v2:172.21.15.154:3300/0 conn(0x7f190c0c9ed0 0x7f190c159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190959a700 1 -- 172.21.15.146:0/298944710 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296425440 0 0) 0x7f190c0cc2d0 con 0x7f190c153a60 2024-09-13T18:50:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190959a700 1 -- 172.21.15.146:0/298944710 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18f40035d0 con 0x7f190c153a60 2024-09-13T18:50:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190ad9d700 1 -- 172.21.15.146:0/298944710 >> v2:172.21.15.154:3300/0 conn(0x7f190c0c9ed0 msgr2=0x7f190c159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190ad9d700 1 --2- 172.21.15.146:0/298944710 >> v2:172.21.15.154:3300/0 conn(0x7f190c0c9ed0 0x7f190c159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190ad9d700 1 -- 172.21.15.146:0/298944710 >> v1:172.21.15.146:6789/0 conn(0x7f190c153a60 legacy=0x7f190c0b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190ad9d700 1 -- 172.21.15.146:0/298944710 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c0cb3e0 con 0x7f190c0cf540 2024-09-13T18:50:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.750+0000 7f190ad9d700 1 --2- 172.21.15.146:0/298944710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f190c0cf540 0x7f190c0cf990 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f190c0c60b0 tx=0x7f190400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40092b5233d2c3fc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.754+0000 7f190959a700 1 -- 172.21.15.146:0/298944710 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190400e070 con 0x7f190c0cf540 2024-09-13T18:50:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.754+0000 7f190959a700 1 -- 172.21.15.146:0/298944710 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190400bb20 con 0x7f190c0cf540 2024-09-13T18:50:33.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.754+0000 7f190959a700 1 -- 172.21.15.146:0/298944710 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190400be90 con 0x7f190c0cf540 2024-09-13T18:50:33.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.754+0000 7f190b59e700 1 -- 172.21.15.146:0/298944710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f190c0cf540 msgr2=0x7f190c0cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:33.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.754+0000 7f190b59e700 1 --2- 172.21.15.146:0/298944710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f190c0cf540 0x7f190c0cf990 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f190c0c60b0 tx=0x7f190400b330 comp rx=0 tx=0).stop 2024-09-13T18:50:33.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.754+0000 7f190b59e700 1 -- 172.21.15.146:0/298944710 shutdown_connections 2024-09-13T18:50:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.754+0000 7f190b59e700 1 --2- 172.21.15.146:0/298944710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f190c0cf540 0x7f190c0cf990 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.754+0000 7f190b59e700 1 --2- 172.21.15.146:0/298944710 >> v2:172.21.15.154:3300/0 conn(0x7f190c0c9ed0 0x7f190c159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.754+0000 7f190b59e700 1 -- 172.21.15.146:0/298944710 >> 172.21.15.146:0/298944710 conn(0x7f190c071ec0 msgr2=0x7f190c0722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.754+0000 7f190b59e700 1 -- 172.21.15.146:0/298944710 shutdown_connections 2024-09-13T18:50:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.754+0000 7f190b59e700 1 -- 172.21.15.146:0/298944710 wait complete. 2024-09-13T18:50:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.754+0000 7f190b59e700 1 Processor -- start 2024-09-13T18:50:33.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.754+0000 7f190b59e700 1 -- start start 2024-09-13T18:50:33.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f190c0cf540 0x7f190c1f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:33.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190b59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f190c153a60 0x7f190c14af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:33.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f190c1f2c40 0x7f190c14b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:33.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190b59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0c82f0 con 0x7f190c1f2c40 2024-09-13T18:50:33.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190b59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0c8820 con 0x7f190c153a60 2024-09-13T18:50:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190c1f31c0 con 0x7f190c0cf540 2024-09-13T18:50:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f190c1f2c40 0x7f190c14b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f190c1f2c40 0x7f190c14b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49524/0 (socket says 172.21.15.146:49524) 2024-09-13T18:50:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190ad9d700 1 -- 172.21.15.146:0/300699604 learned_addr learned my addr 172.21.15.146:0/300699604 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190a59c700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f190c0cf540 0x7f190c1f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f1909d9b700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f190c153a60 0x7f190c14af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190ad9d700 1 -- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f190c0cf540 msgr2=0x7f190c1f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190ad9d700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f190c0cf540 0x7f190c1f2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190ad9d700 1 -- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f190c153a60 msgr2=0x7f190c14af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190ad9d700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f190c153a60 0x7f190c14af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190ad9d700 1 -- 172.21.15.146:0/300699604 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1904009a70 con 0x7f190c1f2c40 2024-09-13T18:50:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190ad9d700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f190c1f2c40 0x7f190c14b4b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1904014040 tx=0x7f1904004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f18faffd700 1 -- 172.21.15.146:0/300699604 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904007000 con 0x7f190c1f2c40 2024-09-13T18:50:33.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f18faffd700 1 -- 172.21.15.146:0/300699604 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1904012630 con 0x7f190c1f2c40 2024-09-13T18:50:33.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190b59e700 1 -- 172.21.15.146:0/300699604 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190c0c8540 con 0x7f190c1f2c40 2024-09-13T18:50:33.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f18faffd700 1 -- 172.21.15.146:0/300699604 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19040129c0 con 0x7f190c1f2c40 2024-09-13T18:50:33.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190b59e700 1 -- 172.21.15.146:0/300699604 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190c0c8ea0 con 0x7f190c1f2c40 2024-09-13T18:50:33.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.758+0000 7f190b59e700 1 -- 172.21.15.146:0/300699604 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190c0c8a20 con 0x7f190c1f2c40 2024-09-13T18:50:33.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.762+0000 7f18faffd700 1 -- 172.21.15.146:0/300699604 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1904012b20 con 0x7f190c1f2c40 2024-09-13T18:50:33.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.762+0000 7f18faffd700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f18ec066730 0x7f18ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:33.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.762+0000 7f18faffd700 1 -- 172.21.15.146:0/300699604 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f19040865e0 con 0x7f190c1f2c40 2024-09-13T18:50:33.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.762+0000 7f190a59c700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f18ec066730 0x7f18ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:33.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.762+0000 7f190a59c700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f18ec066730 0x7f18ec068bf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1900007fd0 tx=0x7f1900009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:33.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:33.766+0000 7f18faffd700 1 -- 172.21.15.146:0/300699604 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f190c0c8a20 con 0x7f190c1f2c40 2024-09-13T18:50:34.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.002+0000 7f190b59e700 1 -- 172.21.15.146:0/300699604 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f190c0c8a20 con 0x7f190c1f2c40 2024-09-13T18:50:34.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.002+0000 7f18faffd700 1 -- 172.21.15.146:0/300699604 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+6279 (secure 0 0 0) 0x7f190c0c8a20 con 0x7f190c1f2c40 2024-09-13T18:50:34.021 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:50:34.021 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:41.442913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:50:34.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.006+0000 7f190b59e700 1 -- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f18ec066730 msgr2=0x7f18ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:34.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.006+0000 7f190b59e700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f18ec066730 0x7f18ec068bf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1900007fd0 tx=0x7f1900009040 comp rx=0 tx=0).stop 2024-09-13T18:50:34.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.006+0000 7f190b59e700 1 -- 172.21.15.146:0/300699604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f190c1f2c40 msgr2=0x7f190c14b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:34.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.006+0000 7f190b59e700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f190c1f2c40 0x7f190c14b4b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1904014040 tx=0x7f1904004af0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.006+0000 7f190b59e700 1 -- 172.21.15.146:0/300699604 shutdown_connections 2024-09-13T18:50:34.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.006+0000 7f190b59e700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f18ec066730 0x7f18ec068bf0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.006+0000 7f190b59e700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f190c1f2c40 0x7f190c14b4b0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.006+0000 7f190b59e700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f190c153a60 0x7f190c14af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.006+0000 7f190b59e700 1 --2- 172.21.15.146:0/300699604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f190c0cf540 0x7f190c1f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.006+0000 7f190b59e700 1 -- 172.21.15.146:0/300699604 >> 172.21.15.146:0/300699604 conn(0x7f190c071ec0 msgr2=0x7f190c0ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:34.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.006+0000 7f190b59e700 1 -- 172.21.15.146:0/300699604 shutdown_connections 2024-09-13T18:50:34.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.006+0000 7f190b59e700 1 -- 172.21.15.146:0/300699604 wait complete. 2024-09-13T18:50:34.025 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 91 2024-09-13T18:50:34.036 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2024-09-13T18:50:34.036 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T18:50:34.036 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:50:34.051 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T18:50:34.051 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:50:34.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4c1888700 1 Processor -- start 2024-09-13T18:50:34.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4c1888700 1 -- start start 2024-09-13T18:50:34.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4c1888700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4b404ccb0 0x7fe4b400be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:34.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4c1888700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4b400d530 0x7fe4b400c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:34.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4c1888700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b40470f0 con 0x7fe4b40d69b0 2024-09-13T18:50:34.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4c1888700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b4048050 con 0x7fe4b400d530 2024-09-13T18:50:34.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4c1888700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b4046ee0 con 0x7fe4b404ccb0 2024-09-13T18:50:34.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4bbfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4b404ccb0 0x7fe4b400be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:34.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4c1087700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4b400d530 0x7fe4b400c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:34.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4bbfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4b404ccb0 0x7fe4b400be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:46050/0 (socket says 172.21.15.146:46050) 2024-09-13T18:50:34.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4c1087700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4b400d530 0x7fe4b400c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:56100/0 (socket says 172.21.15.146:56100) 2024-09-13T18:50:34.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4bbfff700 1 -- 172.21.15.146:0/194123410 learned_addr learned my addr 172.21.15.146:0/194123410 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:34.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4bbfff700 1 -- 172.21.15.146:0/194123410 >> v2:172.21.15.154:3300/0 conn(0x7fe4b400d530 msgr2=0x7fe4b400c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:34.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4bbfff700 1 --2- 172.21.15.146:0/194123410 >> v2:172.21.15.154:3300/0 conn(0x7fe4b400d530 0x7fe4b400c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4bbfff700 1 -- 172.21.15.146:0/194123410 >> v1:172.21.15.154:6789/0 conn(0x7fe4b40d69b0 legacy=0x7fe4b404e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:34.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4bbfff700 1 -- 172.21.15.146:0/194123410 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b40473d0 con 0x7fe4b404ccb0 2024-09-13T18:50:34.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4c1087700 1 --2- 172.21.15.146:0/194123410 >> v2:172.21.15.154:3300/0 conn(0x7fe4b400d530 0x7fe4b400c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:50:34.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4bbfff700 1 --2- 172.21.15.146:0/194123410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4b404ccb0 0x7fe4b400be60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009910 tx=0x7fe4a8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60455eb8d38775d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:34.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4bb7fe700 1 -- 172.21.15.146:0/194123410 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8011830 con 0x7fe4b404ccb0 2024-09-13T18:50:34.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4bb7fe700 1 -- 172.21.15.146:0/194123410 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a8011990 con 0x7fe4b404ccb0 2024-09-13T18:50:34.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.150+0000 7fe4bb7fe700 1 -- 172.21.15.146:0/194123410 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8011d40 con 0x7fe4b404ccb0 2024-09-13T18:50:34.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 -- 172.21.15.146:0/194123410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4b404ccb0 msgr2=0x7fe4b400be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:34.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 --2- 172.21.15.146:0/194123410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4b404ccb0 0x7fe4b400be60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009910 tx=0x7fe4a8009c20 comp rx=0 tx=0).stop 2024-09-13T18:50:34.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 -- 172.21.15.146:0/194123410 shutdown_connections 2024-09-13T18:50:34.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 --2- 172.21.15.146:0/194123410 >> v2:172.21.15.154:3300/0 conn(0x7fe4b400d530 0x7fe4b400c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 --2- 172.21.15.146:0/194123410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4b404ccb0 0x7fe4b400be60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 -- 172.21.15.146:0/194123410 >> 172.21.15.146:0/194123410 conn(0x7fe4b400a130 msgr2=0x7fe4b400dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:34.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 -- 172.21.15.146:0/194123410 shutdown_connections 2024-09-13T18:50:34.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 -- 172.21.15.146:0/194123410 wait complete. 2024-09-13T18:50:34.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 Processor -- start 2024-09-13T18:50:34.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 -- start start 2024-09-13T18:50:34.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4b400d530 0x7fe4b4051210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:34.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4b404ccb0 0x7fe4b4171cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:34.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4b40d69b0 0x7fe4b4172210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:34.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b40da9a0 con 0x7fe4b40d69b0 2024-09-13T18:50:34.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b404fe80 con 0x7fe4b404ccb0 2024-09-13T18:50:34.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1888700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b4050bb0 con 0x7fe4b400d530 2024-09-13T18:50:34.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.154+0000 7fe4c1087700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4b40d69b0 0x7fe4b4172210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:34.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4c1087700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4b40d69b0 0x7fe4b4172210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49538/0 (socket says 172.21.15.146:49538) 2024-09-13T18:50:34.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4c1087700 1 -- 172.21.15.146:0/4157054507 learned_addr learned my addr 172.21.15.146:0/4157054507 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:34.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4c0886700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4b400d530 0x7fe4b4051210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:34.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4bbfff700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4b404ccb0 0x7fe4b4171cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:34.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4c1087700 1 -- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4b400d530 msgr2=0x7fe4b4051210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:34.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4c1087700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4b400d530 0x7fe4b4051210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4c1087700 1 -- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4b404ccb0 msgr2=0x7fe4b4171cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:34.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4c1087700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4b404ccb0 0x7fe4b4171cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4c1087700 1 -- 172.21.15.146:0/4157054507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc04f5c0 con 0x7fe4b40d69b0 2024-09-13T18:50:34.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4c1087700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4b40d69b0 0x7fe4b4172210 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0641b0 tx=0x7fe4bc075d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:34.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4b97fa700 1 -- 172.21.15.146:0/4157054507 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc090070 con 0x7fe4b40d69b0 2024-09-13T18:50:34.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4c1888700 1 -- 172.21.15.146:0/4157054507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a8009570 con 0x7fe4b40d69b0 2024-09-13T18:50:34.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4c1888700 1 -- 172.21.15.146:0/4157054507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b40da6c0 con 0x7fe4b40d69b0 2024-09-13T18:50:34.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4b97fa700 1 -- 172.21.15.146:0/4157054507 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc073930 con 0x7fe4b40d69b0 2024-09-13T18:50:34.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4b97fa700 1 -- 172.21.15.146:0/4157054507 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc067e50 con 0x7fe4b40d69b0 2024-09-13T18:50:34.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4c1888700 1 -- 172.21.15.146:0/4157054507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49c005310 con 0x7fe4b40d69b0 2024-09-13T18:50:34.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4b97fa700 1 -- 172.21.15.146:0/4157054507 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe4bc073c10 con 0x7fe4b40d69b0 2024-09-13T18:50:34.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.158+0000 7fe4b97fa700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe4a0066730 0x7fe4a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:34.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.162+0000 7fe4b97fa700 1 -- 172.21.15.146:0/4157054507 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7fe4bc0ffa30 con 0x7fe4b40d69b0 2024-09-13T18:50:34.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.162+0000 7fe4c0886700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe4a0066730 0x7fe4a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:34.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.162+0000 7fe4c0886700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe4a0066730 0x7fe4a0068bf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0001800 tx=0x7fe4b0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:34.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.166+0000 7fe4b97fa700 1 -- 172.21.15.146:0/4157054507 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe4bc0cc670 con 0x7fe4b40d69b0 2024-09-13T18:50:34.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.390+0000 7fe4c1888700 1 -- 172.21.15.146:0/4157054507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe49c0055c0 con 0x7fe4b40d69b0 2024-09-13T18:50:34.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.390+0000 7fe4b97fa700 1 -- 172.21.15.146:0/4157054507 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+6279 (secure 0 0 0) 0x7fe4bc0cfc90 con 0x7fe4b40d69b0 2024-09-13T18:50:34.409 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:50:34.409 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:41.442913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:50:34.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.394+0000 7fe4c1888700 1 -- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe4a0066730 msgr2=0x7fe4a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:34.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.394+0000 7fe4c1888700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe4a0066730 0x7fe4a0068bf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0001800 tx=0x7fe4b0009040 comp rx=0 tx=0).stop 2024-09-13T18:50:34.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.394+0000 7fe4c1888700 1 -- 172.21.15.146:0/4157054507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4b40d69b0 msgr2=0x7fe4b4172210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:34.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.394+0000 7fe4c1888700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4b40d69b0 0x7fe4b4172210 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0641b0 tx=0x7fe4bc075d10 comp rx=0 tx=0).stop 2024-09-13T18:50:34.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.394+0000 7fe4c1888700 1 -- 172.21.15.146:0/4157054507 shutdown_connections 2024-09-13T18:50:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.394+0000 7fe4c1888700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe4a0066730 0x7fe4a0068bf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.394+0000 7fe4c1888700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4b40d69b0 0x7fe4b4172210 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.394+0000 7fe4c1888700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4b404ccb0 0x7fe4b4171cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.394+0000 7fe4c1888700 1 --2- 172.21.15.146:0/4157054507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4b400d530 0x7fe4b4051210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.394+0000 7fe4c1888700 1 -- 172.21.15.146:0/4157054507 >> 172.21.15.146:0/4157054507 conn(0x7fe4b400a130 msgr2=0x7fe4b4005000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.394+0000 7fe4c1888700 1 -- 172.21.15.146:0/4157054507 shutdown_connections 2024-09-13T18:50:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:34.394+0000 7fe4c1888700 1 -- 172.21.15.146:0/4157054507 wait complete. 2024-09-13T18:50:34.412 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 91 2024-09-13T18:50:34.424 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:50:36.425 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:50:36.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.514+0000 7fb355ada700 1 Processor -- start 2024-09-13T18:50:36.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.514+0000 7fb355ada700 1 -- start start 2024-09-13T18:50:36.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.514+0000 7fb355ada700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb3480d58d0 0x7fb348045cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:36.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.514+0000 7fb355ada700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb348008210 con 0x7fb34804be20 2024-09-13T18:50:36.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.514+0000 7fb355ada700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34804d150 con 0x7fb3480dbb80 2024-09-13T18:50:36.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.514+0000 7fb355ada700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3480341b0 con 0x7fb3480d58d0 2024-09-13T18:50:36.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.514+0000 7fb34ffff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb34804be20 0x7fb3480db470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50702/0 (socket says 172.21.15.146:50702) 2024-09-13T18:50:36.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.514+0000 7fb34ffff700 1 -- 172.21.15.146:0/3961971635 learned_addr learned my addr 172.21.15.146:0/3961971635 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:50:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315867850 0 0) 0x7fb348008210 con 0x7fb34804be20 2024-09-13T18:50:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb354ad8700 1 --2- 172.21.15.146:0/3961971635 >> v2:172.21.15.154:3300/0 conn(0x7fb3480d58d0 0x7fb348045cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3400035d0 con 0x7fb34804be20 2024-09-13T18:50:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434948761 0 0) 0x7fb3400035d0 con 0x7fb34804be20 2024-09-13T18:50:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 >> v2:172.21.15.154:3300/0 conn(0x7fb3480d58d0 msgr2=0x7fb348045cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:36.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 --2- 172.21.15.146:0/3961971635 >> v2:172.21.15.154:3300/0 conn(0x7fb3480d58d0 0x7fb348045cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 >> v1:172.21.15.154:6789/0 conn(0x7fb3480dbb80 legacy=0x7fb3480e5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34804d430 con 0x7fb34804be20 2024-09-13T18:50:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7fb33c001670 con 0x7fb34804be20 2024-09-13T18:50:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 >> v1:172.21.15.146:6789/0 conn(0x7fb34804be20 legacy=0x7fb3480db470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 --2- 172.21.15.146:0/3961971635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb340003a20 0x7fb340003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 --2- 172.21.15.146:0/3961971635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340004870 0x7fb340004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 --2- 172.21.15.146:0/3961971635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb340005230 0x7fb34000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34000c1a0 con 0x7fb340005230 2024-09-13T18:50:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34000c310 con 0x7fb340004870 2024-09-13T18:50:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34000c4d0 con 0x7fb340003a20 2024-09-13T18:50:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb3552d9700 1 --2- 172.21.15.146:0/3961971635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb340005230 0x7fb34000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:36.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34ffff700 1 --2- 172.21.15.146:0/3961971635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340004870 0x7fb340004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb354ad8700 1 --2- 172.21.15.146:0/3961971635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb340003a20 0x7fb340003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb3552d9700 1 -- 172.21.15.146:0/3961971635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb340003a20 msgr2=0x7fb340003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb3552d9700 1 --2- 172.21.15.146:0/3961971635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb340003a20 0x7fb340003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb3552d9700 1 -- 172.21.15.146:0/3961971635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340004870 msgr2=0x7fb340004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb3552d9700 1 --2- 172.21.15.146:0/3961971635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340004870 0x7fb340004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb3552d9700 1 -- 172.21.15.146:0/3961971635 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400035d0 con 0x7fb340005230 2024-09-13T18:50:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb3552d9700 1 -- 172.21.15.146:0/3961971635 reap_dead start 2024-09-13T18:50:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb3552d9700 1 --2- 172.21.15.146:0/3961971635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb340005230 0x7fb34000ba30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb35004ff70 tx=0x7fb3400058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be93ceeab99ee19b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35006f220 con 0x7fb340005230 2024-09-13T18:50:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3500737c0 con 0x7fb340005230 2024-09-13T18:50:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb34f7fe700 1 -- 172.21.15.146:0/3961971635 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350075a30 con 0x7fb340005230 2024-09-13T18:50:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb355ada700 1 -- 172.21.15.146:0/3961971635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb340005230 msgr2=0x7fb34000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:36.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb355ada700 1 --2- 172.21.15.146:0/3961971635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb340005230 0x7fb34000ba30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb35004ff70 tx=0x7fb3400058a0 comp rx=0 tx=0).stop 2024-09-13T18:50:36.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb355ada700 1 -- 172.21.15.146:0/3961971635 shutdown_connections 2024-09-13T18:50:36.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb355ada700 1 --2- 172.21.15.146:0/3961971635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb340005230 0x7fb34000ba30 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:36.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb355ada700 1 -- 172.21.15.146:0/3961971635 >> 172.21.15.146:0/3961971635 conn(0x7fb348050e10 msgr2=0x7fb348051220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:36.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb355ada700 1 -- 172.21.15.146:0/3961971635 shutdown_connections 2024-09-13T18:50:36.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.518+0000 7fb355ada700 1 -- 172.21.15.146:0/3961971635 wait complete. 2024-09-13T18:50:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb355ada700 1 Processor -- start 2024-09-13T18:50:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb355ada700 1 -- start start 2024-09-13T18:50:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb355ada700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb340005230 0x7fb3480d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb355ada700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3480d28a0 0x7fb3480d1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb355ada700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3481774f0 0x7fb348036610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb355ada700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3480d1ab0 con 0x7fb3481774f0 2024-09-13T18:50:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb355ada700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3480cfb70 con 0x7fb3480d28a0 2024-09-13T18:50:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb355ada700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3480d0130 con 0x7fb340005230 2024-09-13T18:50:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb3552d9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3481774f0 0x7fb348036610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb3552d9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3481774f0 0x7fb348036610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49552/0 (socket says 172.21.15.146:49552) 2024-09-13T18:50:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb3552d9700 1 -- 172.21.15.146:0/3178488486 learned_addr learned my addr 172.21.15.146:0/3178488486 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb354ad8700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb340005230 0x7fb3480d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb34ffff700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3480d28a0 0x7fb3480d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb3552d9700 1 -- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb340005230 msgr2=0x7fb3480d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb3552d9700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb340005230 0x7fb3480d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb3552d9700 1 -- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3480d28a0 msgr2=0x7fb3480d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb3552d9700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3480d28a0 0x7fb3480d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb3552d9700 1 -- 172.21.15.146:0/3178488486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34804aa70 con 0x7fb3481774f0 2024-09-13T18:50:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb34ffff700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3480d28a0 0x7fb3480d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:50:36.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb3552d9700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3481774f0 0x7fb348036610 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb350072e30 tx=0x7fb350068890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:36.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb34d7fa700 1 -- 172.21.15.146:0/3178488486 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35009c070 con 0x7fb3481774f0 2024-09-13T18:50:36.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb34d7fa700 1 -- 172.21.15.146:0/3178488486 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb350067750 con 0x7fb3481774f0 2024-09-13T18:50:36.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb355ada700 1 -- 172.21.15.146:0/3178488486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34804acf0 con 0x7fb3481774f0 2024-09-13T18:50:36.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb34d7fa700 1 -- 172.21.15.146:0/3178488486 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350067c50 con 0x7fb3481774f0 2024-09-13T18:50:36.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.522+0000 7fb355ada700 1 -- 172.21.15.146:0/3178488486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3480cfdc0 con 0x7fb3481774f0 2024-09-13T18:50:36.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.526+0000 7fb355ada700 1 -- 172.21.15.146:0/3178488486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb348000ed0 con 0x7fb3481774f0 2024-09-13T18:50:36.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.526+0000 7fb34d7fa700 1 -- 172.21.15.146:0/3178488486 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb350067db0 con 0x7fb3481774f0 2024-09-13T18:50:36.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.526+0000 7fb34d7fa700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb334066760 0x7fb334068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:36.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.526+0000 7fb34d7fa700 1 -- 172.21.15.146:0/3178488486 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7fb350100090 con 0x7fb3481774f0 2024-09-13T18:50:36.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.526+0000 7fb354ad8700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb334066760 0x7fb334068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:36.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.526+0000 7fb354ad8700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb334066760 0x7fb334068c20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb3440045f0 tx=0x7fb344008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:36.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.534+0000 7fb34d7fa700 1 -- 172.21.15.146:0/3178488486 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb3500ccdf0 con 0x7fb3481774f0 2024-09-13T18:50:36.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.810+0000 7fb355ada700 1 -- 172.21.15.146:0/3178488486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb34804c3d0 con 0x7fb3481774f0 2024-09-13T18:50:36.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.810+0000 7fb34d7fa700 1 -- 172.21.15.146:0/3178488486 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+6279 (secure 0 0 0) 0x7fb3500d0410 con 0x7fb3481774f0 2024-09-13T18:50:36.828 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:50:36.828 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:41.442913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:50:36.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.814+0000 7fb355ada700 1 -- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb334066760 msgr2=0x7fb334068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:36.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.814+0000 7fb355ada700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb334066760 0x7fb334068c20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb3440045f0 tx=0x7fb344008040 comp rx=0 tx=0).stop 2024-09-13T18:50:36.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.814+0000 7fb355ada700 1 -- 172.21.15.146:0/3178488486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3481774f0 msgr2=0x7fb348036610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:36.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.814+0000 7fb355ada700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3481774f0 0x7fb348036610 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb350072e30 tx=0x7fb350068890 comp rx=0 tx=0).stop 2024-09-13T18:50:36.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.814+0000 7fb355ada700 1 -- 172.21.15.146:0/3178488486 shutdown_connections 2024-09-13T18:50:36.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.814+0000 7fb355ada700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb334066760 0x7fb334068c20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:36.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.814+0000 7fb355ada700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3481774f0 0x7fb348036610 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:36.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.814+0000 7fb355ada700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3480d28a0 0x7fb3480d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:36.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.814+0000 7fb355ada700 1 --2- 172.21.15.146:0/3178488486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb340005230 0x7fb3480d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:36.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.814+0000 7fb355ada700 1 -- 172.21.15.146:0/3178488486 >> 172.21.15.146:0/3178488486 conn(0x7fb348050e10 msgr2=0x7fb348046590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:36.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.814+0000 7fb355ada700 1 -- 172.21.15.146:0/3178488486 shutdown_connections 2024-09-13T18:50:36.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:36.814+0000 7fb355ada700 1 -- 172.21.15.146:0/3178488486 wait complete. 2024-09-13T18:50:36.832 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 91 2024-09-13T18:50:36.842 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:50:38.843 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:50:38.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.950+0000 7f0f43fff700 1 Processor -- start 2024-09-13T18:50:38.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.950+0000 7f0f43fff700 1 -- start start 2024-09-13T18:50:38.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.950+0000 7f0f43fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0f440cac60 0x7f0f440cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:38.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.950+0000 7f0f43fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f440c8c70 con 0x7f0f440721f0 2024-09-13T18:50:38.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.950+0000 7f0f43fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f440c8f50 con 0x7f0f44154a40 2024-09-13T18:50:38.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.950+0000 7f0f43fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f44072810 con 0x7f0f440cac60 2024-09-13T18:50:38.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.950+0000 7f0f427fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0f440721f0 0x7f0f44157d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50716/0 (socket says 172.21.15.146:50716) 2024-09-13T18:50:38.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.950+0000 7f0f427fc700 1 -- 172.21.15.146:0/636998408 learned_addr learned my addr 172.21.15.146:0/636998408 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:50:38.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.950+0000 7f0f437fe700 1 --2- 172.21.15.146:0/636998408 >> v2:172.21.15.146:3300/0 conn(0x7f0f440cac60 0x7f0f440cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:38.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.950+0000 7f0f41ffb700 1 -- 172.21.15.146:0/636998408 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092366089 0 0) 0x7f0f440c8c70 con 0x7f0f440721f0 2024-09-13T18:50:38.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.950+0000 7f0f41ffb700 1 -- 172.21.15.146:0/636998408 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f2c0035d0 con 0x7f0f440721f0 2024-09-13T18:50:38.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f437fe700 1 -- 172.21.15.146:0/636998408 >> v1:172.21.15.154:6789/0 conn(0x7f0f44154a40 legacy=0x7f0f4406f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:38.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f437fe700 1 -- 172.21.15.146:0/636998408 >> v1:172.21.15.146:6789/0 conn(0x7f0f440721f0 legacy=0x7f0f44157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:38.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f41ffb700 1 -- 172.21.15.146:0/636998408 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2038916458 0 0) 0x7f0f2c0035d0 con 0x7f0f440721f0 2024-09-13T18:50:38.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f437fe700 1 -- 172.21.15.146:0/636998408 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f44157530 con 0x7f0f440cac60 2024-09-13T18:50:38.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f437fe700 1 --2- 172.21.15.146:0/636998408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f440cac60 0x7f0f440cb0f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009d90 tx=0x7f0f3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f96124a0cbee218 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:38.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f41ffb700 1 -- 172.21.15.146:0/636998408 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c00e070 con 0x7f0f440cac60 2024-09-13T18:50:38.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f41ffb700 1 -- 172.21.15.146:0/636998408 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c00bb20 con 0x7f0f440cac60 2024-09-13T18:50:38.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f41ffb700 1 -- 172.21.15.146:0/636998408 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c00bed0 con 0x7f0f440cac60 2024-09-13T18:50:38.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f43fff700 1 -- 172.21.15.146:0/636998408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f440cac60 msgr2=0x7f0f440cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:38.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f43fff700 1 --2- 172.21.15.146:0/636998408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f440cac60 0x7f0f440cb0f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009d90 tx=0x7f0f3c00b330 comp rx=0 tx=0).stop 2024-09-13T18:50:38.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f43fff700 1 -- 172.21.15.146:0/636998408 shutdown_connections 2024-09-13T18:50:38.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f43fff700 1 --2- 172.21.15.146:0/636998408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f440cac60 0x7f0f440cb0f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:38.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f43fff700 1 -- 172.21.15.146:0/636998408 >> 172.21.15.146:0/636998408 conn(0x7f0f44066090 msgr2=0x7f0f44066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:38.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f43fff700 1 -- 172.21.15.146:0/636998408 shutdown_connections 2024-09-13T18:50:38.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f43fff700 1 -- 172.21.15.146:0/636998408 wait complete. 2024-09-13T18:50:38.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f43fff700 1 Processor -- start 2024-09-13T18:50:38.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.954+0000 7f0f43fff700 1 -- start start 2024-09-13T18:50:38.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f440721f0 0x7f0f44156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:38.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f440cac60 0x7f0f441f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:38.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f44154a40 0x7f0f441f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:38.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f43fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f44157910 con 0x7f0f44154a40 2024-09-13T18:50:38.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f43fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f441e2020 con 0x7f0f440721f0 2024-09-13T18:50:38.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f43fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f441f1e50 con 0x7f0f440cac60 2024-09-13T18:50:38.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f44154a40 0x7f0f441f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:38.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f44154a40 0x7f0f441f2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49572/0 (socket says 172.21.15.146:49572) 2024-09-13T18:50:38.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f437fe700 1 -- 172.21.15.146:0/3824520846 learned_addr learned my addr 172.21.15.146:0/3824520846 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:38.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f427fc700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f440cac60 0x7f0f441f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:38.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f42ffd700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f440721f0 0x7f0f44156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:38.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f437fe700 1 -- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f440cac60 msgr2=0x7f0f441f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:38.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f437fe700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f440cac60 0x7f0f441f2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:38.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f437fe700 1 -- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f440721f0 msgr2=0x7f0f44156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:38.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f437fe700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f440721f0 0x7f0f44156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:38.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f437fe700 1 -- 172.21.15.146:0/3824520846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c009a70 con 0x7f0f44154a40 2024-09-13T18:50:38.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f427fc700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f440cac60 0x7f0f441f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:50:38.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f42ffd700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f440721f0 0x7f0f44156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:50:38.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f437fe700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f44154a40 0x7f0f441f2870 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c014040 tx=0x7f0f3c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:38.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f2b7fe700 1 -- 172.21.15.146:0/3824520846 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c01b810 con 0x7f0f44154a40 2024-09-13T18:50:38.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f2b7fe700 1 -- 172.21.15.146:0/3824520846 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c020070 con 0x7f0f44154a40 2024-09-13T18:50:38.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f43fff700 1 -- 172.21.15.146:0/3824520846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f441e2270 con 0x7f0f44154a40 2024-09-13T18:50:38.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f2b7fe700 1 -- 172.21.15.146:0/3824520846 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c01bbb0 con 0x7f0f44154a40 2024-09-13T18:50:38.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f43fff700 1 -- 172.21.15.146:0/3824520846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f441e2830 con 0x7f0f44154a40 2024-09-13T18:50:38.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.958+0000 7f0f43fff700 1 -- 172.21.15.146:0/3824520846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f441f2010 con 0x7f0f44154a40 2024-09-13T18:50:38.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.962+0000 7f0f2b7fe700 1 -- 172.21.15.146:0/3824520846 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f3c01bd10 con 0x7f0f44154a40 2024-09-13T18:50:38.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.962+0000 7f0f2b7fe700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0f30066730 0x7f0f30068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:38.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.962+0000 7f0f2b7fe700 1 -- 172.21.15.146:0/3824520846 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f0f3c022070 con 0x7f0f44154a40 2024-09-13T18:50:38.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.962+0000 7f0f42ffd700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0f30066730 0x7f0f30068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:38.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.962+0000 7f0f42ffd700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0f30066730 0x7f0f30068bf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0f38000f60 tx=0x7f0f38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:38.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:38.974+0000 7f0f2b7fe700 1 -- 172.21.15.146:0/3824520846 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0f3c057000 con 0x7f0f44154a40 2024-09-13T18:50:39.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.182+0000 7f0f43fff700 1 -- 172.21.15.146:0/3824520846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f440c8c70 con 0x7f0f44154a40 2024-09-13T18:50:39.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.186+0000 7f0f2b7fe700 1 -- 172.21.15.146:0/3824520846 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+6279 (secure 0 0 0) 0x7f0f3c056b90 con 0x7f0f44154a40 2024-09-13T18:50:39.202 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:50:39.202 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:41.442913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:50:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.186+0000 7f0f43fff700 1 -- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0f30066730 msgr2=0x7f0f30068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.186+0000 7f0f43fff700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0f30066730 0x7f0f30068bf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0f38000f60 tx=0x7f0f38009040 comp rx=0 tx=0).stop 2024-09-13T18:50:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.186+0000 7f0f43fff700 1 -- 172.21.15.146:0/3824520846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f44154a40 msgr2=0x7f0f441f2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.186+0000 7f0f43fff700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f44154a40 0x7f0f441f2870 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c014040 tx=0x7f0f3c004af0 comp rx=0 tx=0).stop 2024-09-13T18:50:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.186+0000 7f0f43fff700 1 -- 172.21.15.146:0/3824520846 shutdown_connections 2024-09-13T18:50:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.186+0000 7f0f43fff700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0f30066730 0x7f0f30068bf0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:39.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.186+0000 7f0f43fff700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f44154a40 0x7f0f441f2870 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:39.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.186+0000 7f0f43fff700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f440cac60 0x7f0f441f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:39.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.186+0000 7f0f43fff700 1 --2- 172.21.15.146:0/3824520846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f440721f0 0x7f0f44156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:39.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.186+0000 7f0f43fff700 1 -- 172.21.15.146:0/3824520846 >> 172.21.15.146:0/3824520846 conn(0x7f0f44066090 msgr2=0x7f0f440c53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:39.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.186+0000 7f0f43fff700 1 -- 172.21.15.146:0/3824520846 shutdown_connections 2024-09-13T18:50:39.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:39.186+0000 7f0f43fff700 1 -- 172.21.15.146:0/3824520846 wait complete. 2024-09-13T18:50:39.206 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 91 2024-09-13T18:50:39.217 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:50:41.218 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:50:41.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.322+0000 7f3fbeef6700 1 Processor -- start 2024-09-13T18:50:41.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.322+0000 7f3fbeef6700 1 -- start start 2024-09-13T18:50:41.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3fb8154a40 0x7f3fb80721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:41.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb806f7d0 0x7f3fb8157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:41.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3fb80cac60 0x7f3fb80cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:41.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb8072810 con 0x7f3fb8154a40 2024-09-13T18:50:41.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb8157910 con 0x7f3fb80cac60 2024-09-13T18:50:41.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb80c8c70 con 0x7f3fb806f7d0 2024-09-13T18:50:41.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbdef4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3fb8154a40 0x7f3fb80721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:41.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbdef4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3fb8154a40 0x7f3fb80721f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48534/0 (socket says 172.21.15.146:48534) 2024-09-13T18:50:41.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbdef4700 1 -- 172.21.15.146:0/1539176516 learned_addr learned my addr 172.21.15.146:0/1539176516 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:41.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbe6f5700 1 --2- 172.21.15.146:0/1539176516 >> v2:172.21.15.154:3300/0 conn(0x7f3fb80cac60 0x7f3fb80cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:41.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbd6f3700 1 --2- 172.21.15.146:0/1539176516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb806f7d0 0x7f3fb8157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:41.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbdef4700 1 -- 172.21.15.146:0/1539176516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb806f7d0 msgr2=0x7f3fb8157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:41.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbdef4700 1 --2- 172.21.15.146:0/1539176516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb806f7d0 0x7f3fb8157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:41.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbdef4700 1 -- 172.21.15.146:0/1539176516 >> v2:172.21.15.154:3300/0 conn(0x7f3fb80cac60 msgr2=0x7f3fb80cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:41.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbdef4700 1 --2- 172.21.15.146:0/1539176516 >> v2:172.21.15.154:3300/0 conn(0x7f3fb80cac60 0x7f3fb80cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:41.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbdef4700 1 -- 172.21.15.146:0/1539176516 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb80c8f50 con 0x7f3fb8154a40 2024-09-13T18:50:41.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbe6f5700 1 --2- 172.21.15.146:0/1539176516 >> v2:172.21.15.154:3300/0 conn(0x7f3fb80cac60 0x7f3fb80cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:50:41.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbdef4700 1 --2- 172.21.15.146:0/1539176516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fb8154a40 0x7f3fb80721f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3fb806c490 tx=0x7f3fb0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e1ae86c78bec6e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:41.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbcef2700 1 -- 172.21.15.146:0/1539176516 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0011830 con 0x7f3fb8154a40 2024-09-13T18:50:41.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbcef2700 1 -- 172.21.15.146:0/1539176516 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb0011990 con 0x7f3fb8154a40 2024-09-13T18:50:41.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbcef2700 1 -- 172.21.15.146:0/1539176516 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0011d20 con 0x7f3fb8154a40 2024-09-13T18:50:41.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 -- 172.21.15.146:0/1539176516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fb8154a40 msgr2=0x7f3fb80721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:41.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 --2- 172.21.15.146:0/1539176516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fb8154a40 0x7f3fb80721f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3fb806c490 tx=0x7f3fb0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:50:41.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 -- 172.21.15.146:0/1539176516 shutdown_connections 2024-09-13T18:50:41.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 --2- 172.21.15.146:0/1539176516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb806f7d0 0x7f3fb8157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:41.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 --2- 172.21.15.146:0/1539176516 >> v2:172.21.15.154:3300/0 conn(0x7f3fb80cac60 0x7f3fb80cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:41.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 --2- 172.21.15.146:0/1539176516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fb8154a40 0x7f3fb80721f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:41.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 -- 172.21.15.146:0/1539176516 >> 172.21.15.146:0/1539176516 conn(0x7f3fb8066090 msgr2=0x7f3fb8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:41.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 -- 172.21.15.146:0/1539176516 shutdown_connections 2024-09-13T18:50:41.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.326+0000 7f3fbeef6700 1 -- 172.21.15.146:0/1539176516 wait complete. 2024-09-13T18:50:41.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.330+0000 7f3fbeef6700 1 Processor -- start 2024-09-13T18:50:41.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.330+0000 7f3fbeef6700 1 -- start start 2024-09-13T18:50:41.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.330+0000 7f3fbeef6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb806f7d0 0x7f3fb80c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:41.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbeef6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb80cac60 0x7f3fb815b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:41.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbeef6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fb8154a40 0x7f3fb8151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:41.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbeef6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb80c4970 con 0x7f3fb8154a40 2024-09-13T18:50:41.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbeef6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb815a580 con 0x7f3fb80cac60 2024-09-13T18:50:41.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbeef6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb815b670 con 0x7f3fb806f7d0 2024-09-13T18:50:41.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbe6f5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fb8154a40 0x7f3fb8151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:41.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbe6f5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fb8154a40 0x7f3fb8151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48536/0 (socket says 172.21.15.146:48536) 2024-09-13T18:50:41.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbe6f5700 1 -- 172.21.15.146:0/3813926822 learned_addr learned my addr 172.21.15.146:0/3813926822 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:41.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbdef4700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb806f7d0 0x7f3fb80c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:41.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbd6f3700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb80cac60 0x7f3fb815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:41.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbe6f5700 1 -- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb806f7d0 msgr2=0x7f3fb80c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:41.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbe6f5700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb806f7d0 0x7f3fb80c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:41.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbe6f5700 1 -- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb80cac60 msgr2=0x7f3fb815b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:41.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbe6f5700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb80cac60 0x7f3fb815b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:41.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbe6f5700 1 -- 172.21.15.146:0/3813926822 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb4009a70 con 0x7f3fb8154a40 2024-09-13T18:50:41.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbe6f5700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fb8154a40 0x7f3fb8151eb0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400efc0 tx=0x7f3fb400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:41.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fae7fc700 1 -- 172.21.15.146:0/3813926822 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4009cd0 con 0x7f3fb8154a40 2024-09-13T18:50:41.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fae7fc700 1 -- 172.21.15.146:0/3813926822 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb400f040 con 0x7f3fb8154a40 2024-09-13T18:50:41.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbeef6700 1 -- 172.21.15.146:0/3813926822 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb0009570 con 0x7f3fb8154a40 2024-09-13T18:50:41.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fae7fc700 1 -- 172.21.15.146:0/3813926822 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb400cd60 con 0x7f3fb8154a40 2024-09-13T18:50:41.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbeef6700 1 -- 172.21.15.146:0/3813926822 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb8159fc0 con 0x7f3fb8154a40 2024-09-13T18:50:41.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbeef6700 1 -- 172.21.15.146:0/3813926822 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb80c9e00 con 0x7f3fb8154a40 2024-09-13T18:50:41.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fae7fc700 1 -- 172.21.15.146:0/3813926822 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3fb4014480 con 0x7f3fb8154a40 2024-09-13T18:50:41.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fae7fc700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f9c066760 0x7f3f9c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:41.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fae7fc700 1 -- 172.21.15.146:0/3813926822 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f3fb40850a0 con 0x7f3fb8154a40 2024-09-13T18:50:41.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.334+0000 7f3fbdef4700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f9c066760 0x7f3f9c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:41.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.338+0000 7f3fbdef4700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f9c066760 0x7f3f9c068c20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0000c00 tx=0x7f3fb0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:41.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.342+0000 7f3fae7fc700 1 -- 172.21.15.146:0/3813926822 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3fb4051bc0 con 0x7f3fb8154a40 2024-09-13T18:50:41.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.546+0000 7f3fbeef6700 1 -- 172.21.15.146:0/3813926822 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fb80ca310 con 0x7f3fb8154a40 2024-09-13T18:50:41.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.546+0000 7f3fae7fc700 1 -- 172.21.15.146:0/3813926822 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+6279 (secure 0 0 0) 0x7f3fb40551e0 con 0x7f3fb8154a40 2024-09-13T18:50:41.565 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:50:41.565 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:41.442913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:50:41.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.550+0000 7f3fbeef6700 1 -- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f9c066760 msgr2=0x7f3f9c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:41.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.550+0000 7f3fbeef6700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f9c066760 0x7f3f9c068c20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0000c00 tx=0x7f3fb0008030 comp rx=0 tx=0).stop 2024-09-13T18:50:41.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.550+0000 7f3fbeef6700 1 -- 172.21.15.146:0/3813926822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fb8154a40 msgr2=0x7f3fb8151eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:41.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.550+0000 7f3fbeef6700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fb8154a40 0x7f3fb8151eb0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400efc0 tx=0x7f3fb400c3a0 comp rx=0 tx=0).stop 2024-09-13T18:50:41.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.550+0000 7f3fbeef6700 1 -- 172.21.15.146:0/3813926822 shutdown_connections 2024-09-13T18:50:41.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.550+0000 7f3fbeef6700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3f9c066760 0x7f3f9c068c20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:41.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.550+0000 7f3fbeef6700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fb8154a40 0x7f3fb8151eb0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:41.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.550+0000 7f3fbeef6700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb80cac60 0x7f3fb815b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:41.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.550+0000 7f3fbeef6700 1 --2- 172.21.15.146:0/3813926822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb806f7d0 0x7f3fb80c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:41.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.550+0000 7f3fbeef6700 1 -- 172.21.15.146:0/3813926822 >> 172.21.15.146:0/3813926822 conn(0x7f3fb8066090 msgr2=0x7f3fb8153d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:41.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.550+0000 7f3fbeef6700 1 -- 172.21.15.146:0/3813926822 shutdown_connections 2024-09-13T18:50:41.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:41.550+0000 7f3fbeef6700 1 -- 172.21.15.146:0/3813926822 wait complete. 2024-09-13T18:50:41.569 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 91 2024-09-13T18:50:41.579 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:50:42.316 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:50:42.319 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:50:43.580 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:50:43.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ece789700 1 Processor -- start 2024-09-13T18:50:43.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ece789700 1 -- start start 2024-09-13T18:50:43.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ece789700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6ec80cf450 0x7f6ec80cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:43.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ece789700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec80c5bd0 con 0x7f6ec80c57f0 2024-09-13T18:50:43.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ece789700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec80c9bb0 con 0x7f6ec80ca020 2024-09-13T18:50:43.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ece789700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec815a160 con 0x7f6ec80cf450 2024-09-13T18:50:43.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6eccd26700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6ec80cf450 0x7f6ec80cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:43.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ec7fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6ec80c57f0 0x7f6ec80cb930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43440/0 (socket says 172.21.15.146:43440) 2024-09-13T18:50:43.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ec7fff700 1 -- 172.21.15.146:0/3660312380 learned_addr learned my addr 172.21.15.146:0/3660312380 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:50:43.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6eccd26700 1 -- 172.21.15.146:0/3660312380 >> v1:172.21.15.154:6789/0 conn(0x7f6ec80ca020 legacy=0x7f6ec80ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:43.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6eccd26700 1 -- 172.21.15.146:0/3660312380 >> v1:172.21.15.146:6789/0 conn(0x7f6ec80c57f0 legacy=0x7f6ec80cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:43.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6eccd26700 1 -- 172.21.15.146:0/3660312380 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec8159830 con 0x7f6ec80cf450 2024-09-13T18:50:43.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ec6ffd700 1 -- 172.21.15.146:0/3660312380 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077837632 0 0) 0x7f6ec80c5bd0 con 0x7f6ec80c57f0 2024-09-13T18:50:43.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6eccd26700 1 --2- 172.21.15.146:0/3660312380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cf450 0x7f6ec80cf8a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009d90 tx=0x7f6ec000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b60fb2c8ac8a58 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:43.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ec6ffd700 1 -- 172.21.15.146:0/3660312380 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000e070 con 0x7f6ec80cf450 2024-09-13T18:50:43.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ec6ffd700 1 -- 172.21.15.146:0/3660312380 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec000bbd0 con 0x7f6ec80cf450 2024-09-13T18:50:43.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ec6ffd700 1 -- 172.21.15.146:0/3660312380 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0012430 con 0x7f6ec80cf450 2024-09-13T18:50:43.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ece789700 1 -- 172.21.15.146:0/3660312380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cf450 msgr2=0x7f6ec80cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:43.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.682+0000 7f6ece789700 1 --2- 172.21.15.146:0/3660312380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cf450 0x7f6ec80cf8a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009d90 tx=0x7f6ec000b330 comp rx=0 tx=0).stop 2024-09-13T18:50:43.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ece789700 1 -- 172.21.15.146:0/3660312380 shutdown_connections 2024-09-13T18:50:43.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ece789700 1 --2- 172.21.15.146:0/3660312380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cf450 0x7f6ec80cf8a0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:43.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ece789700 1 -- 172.21.15.146:0/3660312380 >> 172.21.15.146:0/3660312380 conn(0x7f6ec8071d30 msgr2=0x7f6ec8072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:43.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ece789700 1 -- 172.21.15.146:0/3660312380 shutdown_connections 2024-09-13T18:50:43.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ece789700 1 -- 172.21.15.146:0/3660312380 wait complete. 2024-09-13T18:50:43.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ece789700 1 Processor -- start 2024-09-13T18:50:43.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ece789700 1 -- start start 2024-09-13T18:50:43.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ece789700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec80c57f0 0x7f6ec8158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:43.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ece789700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec80ca020 0x7f6ec8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:43.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ece789700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cf450 0x7f6ec81f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:43.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ece789700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80c9170 con 0x7f6ec80cf450 2024-09-13T18:50:43.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ece789700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80c3f10 con 0x7f6ec80ca020 2024-09-13T18:50:43.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ece789700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80b4410 con 0x7f6ec80c57f0 2024-09-13T18:50:43.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6eccd26700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cf450 0x7f6ec81f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.686+0000 7f6ec7fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec80c57f0 0x7f6ec8158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ec77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec80ca020 0x7f6ec8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6eccd26700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cf450 0x7f6ec81f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48558/0 (socket says 172.21.15.146:48558) 2024-09-13T18:50:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ec7fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec80c57f0 0x7f6ec8158ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:42288/0 (socket says 172.21.15.146:42288) 2024-09-13T18:50:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ec77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec80ca020 0x7f6ec8158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:40558/0 (socket says 172.21.15.146:40558) 2024-09-13T18:50:43.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6eccd26700 1 -- 172.21.15.146:0/1927844508 learned_addr learned my addr 172.21.15.146:0/1927844508 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:43.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ec7fff700 1 -- 172.21.15.146:0/1927844508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec80ca020 msgr2=0x7f6ec8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:43.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ec7fff700 1 --2- 172.21.15.146:0/1927844508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec80ca020 0x7f6ec8158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:43.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ec7fff700 1 -- 172.21.15.146:0/1927844508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cf450 msgr2=0x7f6ec81f2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:43.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ec7fff700 1 --2- 172.21.15.146:0/1927844508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cf450 0x7f6ec81f2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:43.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ec7fff700 1 -- 172.21.15.146:0/1927844508 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec0009a70 con 0x7f6ec80c57f0 2024-09-13T18:50:43.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6eccd26700 1 --2- 172.21.15.146:0/1927844508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cf450 0x7f6ec81f2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:50:43.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ec7fff700 1 --2- 172.21.15.146:0/1927844508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec80c57f0 0x7f6ec8158ab0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000c550 tx=0x7f6eb000c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:43.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ec4ff9700 1 -- 172.21.15.146:0/1927844508 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0004ca0 con 0x7f6ec80c57f0 2024-09-13T18:50:43.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ec4ff9700 1 -- 172.21.15.146:0/1927844508 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb0004e00 con 0x7f6ec80c57f0 2024-09-13T18:50:43.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ece789700 1 -- 172.21.15.146:0/1927844508 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec80b46f0 con 0x7f6ec80c57f0 2024-09-13T18:50:43.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ec4ff9700 1 -- 172.21.15.146:0/1927844508 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0007310 con 0x7f6ec80c57f0 2024-09-13T18:50:43.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ece789700 1 -- 172.21.15.146:0/1927844508 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec80c8e00 con 0x7f6ec80c57f0 2024-09-13T18:50:43.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.690+0000 7f6ece789700 1 -- 172.21.15.146:0/1927844508 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec815a560 con 0x7f6ec80c57f0 2024-09-13T18:50:43.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.694+0000 7f6ec4ff9700 1 -- 172.21.15.146:0/1927844508 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6eb0007470 con 0x7f6ec80c57f0 2024-09-13T18:50:43.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.694+0000 7f6ec4ff9700 1 --2- 172.21.15.146:0/1927844508 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6eac066730 0x7f6eac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:43.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.694+0000 7f6ec4ff9700 1 -- 172.21.15.146:0/1927844508 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f6eb0084460 con 0x7f6ec80c57f0 2024-09-13T18:50:43.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.694+0000 7f6ec77fe700 1 --2- 172.21.15.146:0/1927844508 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6eac066730 0x7f6eac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:43.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.694+0000 7f6ec77fe700 1 --2- 172.21.15.146:0/1927844508 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6eac066730 0x7f6eac068bf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6ec81f2de0 tx=0x7f6eb8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:43.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.706+0000 7f6ec4ff9700 1 -- 172.21.15.146:0/1927844508 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6eb0051010 con 0x7f6ec80c57f0 2024-09-13T18:50:43.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.914+0000 7f6ece789700 1 -- 172.21.15.146:0/1927844508 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ec8067c50 con 0x7f6ec80c57f0 2024-09-13T18:50:43.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.914+0000 7f6ec4ff9700 1 -- 172.21.15.146:0/1927844508 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+6279 (secure 0 0 0) 0x7f6eb0054630 con 0x7f6ec80c57f0 2024-09-13T18:50:43.932 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:50:43.932 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:41.442913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:50:43.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.914+0000 7f6ece789700 1 -- 172.21.15.146:0/1927844508 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6eac066730 msgr2=0x7f6eac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:43.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.914+0000 7f6ece789700 1 --2- 172.21.15.146:0/1927844508 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6eac066730 0x7f6eac068bf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6ec81f2de0 tx=0x7f6eb8009040 comp rx=0 tx=0).stop 2024-09-13T18:50:43.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.914+0000 7f6ece789700 1 -- 172.21.15.146:0/1927844508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec80c57f0 msgr2=0x7f6ec8158ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:43.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.914+0000 7f6ece789700 1 --2- 172.21.15.146:0/1927844508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec80c57f0 0x7f6ec8158ab0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000c550 tx=0x7f6eb000c860 comp rx=0 tx=0).stop 2024-09-13T18:50:43.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.918+0000 7f6ece789700 1 -- 172.21.15.146:0/1927844508 shutdown_connections 2024-09-13T18:50:43.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.918+0000 7f6ece789700 1 --2- 172.21.15.146:0/1927844508 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6eac066730 0x7f6eac068bf0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:43.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.918+0000 7f6ece789700 1 --2- 172.21.15.146:0/1927844508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cf450 0x7f6ec81f2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:43.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.918+0000 7f6ece789700 1 --2- 172.21.15.146:0/1927844508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec80ca020 0x7f6ec8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:43.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.918+0000 7f6ece789700 1 --2- 172.21.15.146:0/1927844508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ec80c57f0 0x7f6ec8158ab0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:43.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.918+0000 7f6ece789700 1 -- 172.21.15.146:0/1927844508 >> 172.21.15.146:0/1927844508 conn(0x7f6ec8071d30 msgr2=0x7f6ec80cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:43.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.918+0000 7f6ece789700 1 -- 172.21.15.146:0/1927844508 shutdown_connections 2024-09-13T18:50:43.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:43.918+0000 7f6ece789700 1 -- 172.21.15.146:0/1927844508 wait complete. 2024-09-13T18:50:43.936 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 91 2024-09-13T18:50:43.947 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:50:45.948 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:50:46.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.050+0000 7f9636600700 1 Processor -- start 2024-09-13T18:50:46.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.050+0000 7f9636600700 1 -- start start 2024-09-13T18:50:46.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.050+0000 7f9636600700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96300cac60 0x7f96300cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:46.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.050+0000 7f9636600700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96300c8c70 con 0x7f96300721f0 2024-09-13T18:50:46.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.050+0000 7f9636600700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96300c8f50 con 0x7f9630154a40 2024-09-13T18:50:46.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.050+0000 7f9636600700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9630072810 con 0x7f96300cac60 2024-09-13T18:50:46.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.050+0000 7f9635dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96300cac60 0x7f96300cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:46.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.050+0000 7f9635dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96300cac60 0x7f96300cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48570/0 (socket says 172.21.15.146:48570) 2024-09-13T18:50:46.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.050+0000 7f9634dfd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f96300721f0 0x7f9630157d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43450/0 (socket says 172.21.15.146:43450) 2024-09-13T18:50:46.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.050+0000 7f9635dff700 1 -- 172.21.15.146:0/3662410657 learned_addr learned my addr 172.21.15.146:0/3662410657 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:46.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f9635dff700 1 -- 172.21.15.146:0/3662410657 >> v1:172.21.15.154:6789/0 conn(0x7f9630154a40 legacy=0x7f963006f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:46.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f9635dff700 1 -- 172.21.15.146:0/3662410657 >> v1:172.21.15.146:6789/0 conn(0x7f96300721f0 legacy=0x7f9630157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:46.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f962ffff700 1 -- 172.21.15.146:0/3662410657 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785158387 0 0) 0x7f96300c8c70 con 0x7f96300721f0 2024-09-13T18:50:46.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f9635dff700 1 -- 172.21.15.146:0/3662410657 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9630157530 con 0x7f96300cac60 2024-09-13T18:50:46.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f9635dff700 1 --2- 172.21.15.146:0/3662410657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300cac60 0x7f96300cb0f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9628009890 tx=0x7f9628009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c1c92307f37453 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:46.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f962ffff700 1 -- 172.21.15.146:0/3662410657 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628004230 con 0x7f96300cac60 2024-09-13T18:50:46.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f962ffff700 1 -- 172.21.15.146:0/3662410657 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9628004390 con 0x7f96300cac60 2024-09-13T18:50:46.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f962ffff700 1 -- 172.21.15.146:0/3662410657 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96280036c0 con 0x7f96300cac60 2024-09-13T18:50:46.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f9636600700 1 -- 172.21.15.146:0/3662410657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300cac60 msgr2=0x7f96300cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f9636600700 1 --2- 172.21.15.146:0/3662410657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300cac60 0x7f96300cb0f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9628009890 tx=0x7f9628009ba0 comp rx=0 tx=0).stop 2024-09-13T18:50:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f9636600700 1 -- 172.21.15.146:0/3662410657 shutdown_connections 2024-09-13T18:50:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f9636600700 1 --2- 172.21.15.146:0/3662410657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300cac60 0x7f96300cb0f0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f9636600700 1 -- 172.21.15.146:0/3662410657 >> 172.21.15.146:0/3662410657 conn(0x7f9630066090 msgr2=0x7f9630066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f9636600700 1 -- 172.21.15.146:0/3662410657 shutdown_connections 2024-09-13T18:50:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f9636600700 1 -- 172.21.15.146:0/3662410657 wait complete. 2024-09-13T18:50:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f9636600700 1 Processor -- start 2024-09-13T18:50:46.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.054+0000 7f9636600700 1 -- start start 2024-09-13T18:50:46.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f9636600700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300721f0 0x7f96301f0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:46.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f9636600700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300cac60 0x7f963006dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:46.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f9636600700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9630154a40 0x7f96300adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:46.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f9636600700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9630157910 con 0x7f96300721f0 2024-09-13T18:50:46.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f9636600700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96300c2740 con 0x7f9630154a40 2024-09-13T18:50:46.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f9636600700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96301f0430 con 0x7f96300cac60 2024-09-13T18:50:46.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f96355fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300721f0 0x7f96301f0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:46.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f96355fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300721f0 0x7f96301f0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48576/0 (socket says 172.21.15.146:48576) 2024-09-13T18:50:46.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f96355fe700 1 -- 172.21.15.146:0/3061762922 learned_addr learned my addr 172.21.15.146:0/3061762922 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:46.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f9634dfd700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300cac60 0x7f963006dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:46.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f9635dff700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9630154a40 0x7f96300adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:46.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f96355fe700 1 -- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300cac60 msgr2=0x7f963006dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:46.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f96355fe700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300cac60 0x7f963006dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:46.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f96355fe700 1 -- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9630154a40 msgr2=0x7f96300adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:46.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f96355fe700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9630154a40 0x7f96300adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:46.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f96355fe700 1 -- 172.21.15.146:0/3061762922 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9628009570 con 0x7f96300721f0 2024-09-13T18:50:46.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f96355fe700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300721f0 0x7f96301f0850 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9620001fd0 tx=0x7f962000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:46.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f962dffb700 1 -- 172.21.15.146:0/3061762922 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620003e50 con 0x7f96300721f0 2024-09-13T18:50:46.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f962dffb700 1 -- 172.21.15.146:0/3061762922 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962000a880 con 0x7f96300721f0 2024-09-13T18:50:46.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f9636600700 1 -- 172.21.15.146:0/3061762922 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96301f0d90 con 0x7f96300721f0 2024-09-13T18:50:46.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f962dffb700 1 -- 172.21.15.146:0/3061762922 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962000ac30 con 0x7f96300721f0 2024-09-13T18:50:46.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f9636600700 1 -- 172.21.15.146:0/3061762922 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96300c2460 con 0x7f96300721f0 2024-09-13T18:50:46.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.058+0000 7f9636600700 1 -- 172.21.15.146:0/3061762922 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96301f0570 con 0x7f96300721f0 2024-09-13T18:50:46.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.062+0000 7f962dffb700 1 -- 172.21.15.146:0/3061762922 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9620005070 con 0x7f96300721f0 2024-09-13T18:50:46.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.062+0000 7f962dffb700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f961c066760 0x7f961c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:46.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.062+0000 7f962dffb700 1 -- 172.21.15.146:0/3061762922 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f9620084410 con 0x7f96300721f0 2024-09-13T18:50:46.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.062+0000 7f9634dfd700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f961c066760 0x7f961c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:46.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.062+0000 7f9634dfd700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f961c066760 0x7f961c068c20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f96300c6b30 tx=0x7f9624007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:46.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.070+0000 7f962dffb700 1 -- 172.21.15.146:0/3061762922 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f96301f0570 con 0x7f96300721f0 2024-09-13T18:50:46.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.278+0000 7f9636600700 1 -- 172.21.15.146:0/3061762922 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96301f0570 con 0x7f96300721f0 2024-09-13T18:50:46.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.278+0000 7f962dffb700 1 -- 172.21.15.146:0/3061762922 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+6279 (secure 0 0 0) 0x7f96301f0570 con 0x7f96300721f0 2024-09-13T18:50:46.296 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:50:46.296 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:41.442913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:50:46.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.278+0000 7f9636600700 1 -- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f961c066760 msgr2=0x7f961c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:46.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.278+0000 7f9636600700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f961c066760 0x7f961c068c20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f96300c6b30 tx=0x7f9624007e90 comp rx=0 tx=0).stop 2024-09-13T18:50:46.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.278+0000 7f9636600700 1 -- 172.21.15.146:0/3061762922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300721f0 msgr2=0x7f96301f0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:46.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.278+0000 7f9636600700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300721f0 0x7f96301f0850 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9620001fd0 tx=0x7f962000a340 comp rx=0 tx=0).stop 2024-09-13T18:50:46.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.282+0000 7f9636600700 1 -- 172.21.15.146:0/3061762922 shutdown_connections 2024-09-13T18:50:46.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.282+0000 7f9636600700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f961c066760 0x7f961c068c20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:46.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.282+0000 7f9636600700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9630154a40 0x7f96300adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:46.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.282+0000 7f9636600700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96300cac60 0x7f963006dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:46.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.282+0000 7f9636600700 1 --2- 172.21.15.146:0/3061762922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96300721f0 0x7f96301f0850 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:46.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.282+0000 7f9636600700 1 -- 172.21.15.146:0/3061762922 >> 172.21.15.146:0/3061762922 conn(0x7f9630066090 msgr2=0x7f9630066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:46.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.282+0000 7f9636600700 1 -- 172.21.15.146:0/3061762922 shutdown_connections 2024-09-13T18:50:46.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:46.282+0000 7f9636600700 1 -- 172.21.15.146:0/3061762922 wait complete. 2024-09-13T18:50:46.300 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 91 2024-09-13T18:50:46.310 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:50:48.311 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:50:48.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 Processor -- start 2024-09-13T18:50:48.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 -- start start 2024-09-13T18:50:48.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f62180cbd50 0x7f62180b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:48.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6218071390 0x7f62180cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:48.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62180cc670 con 0x7f62180ca210 2024-09-13T18:50:48.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62180caa30 con 0x7f62180cbd50 2024-09-13T18:50:48.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62180cb810 con 0x7f6218071390 2024-09-13T18:50:48.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621dd01700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f62180cbd50 0x7f62180b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:48.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621dd01700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f62180cbd50 0x7f62180b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48590/0 (socket says 172.21.15.146:48590) 2024-09-13T18:50:48.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621dd01700 1 -- 172.21.15.146:0/3977701507 learned_addr learned my addr 172.21.15.146:0/3977701507 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:48.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621e502700 1 --2- 172.21.15.146:0/3977701507 >> v2:172.21.15.154:3300/0 conn(0x7f6218071390 0x7f62180cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:48.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621dd01700 1 -- 172.21.15.146:0/3977701507 >> v2:172.21.15.154:3300/0 conn(0x7f6218071390 msgr2=0x7f62180cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:48.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621dd01700 1 --2- 172.21.15.146:0/3977701507 >> v2:172.21.15.154:3300/0 conn(0x7f6218071390 0x7f62180cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:48.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621dd01700 1 -- 172.21.15.146:0/3977701507 >> v1:172.21.15.154:6789/0 conn(0x7f62180ca210 legacy=0x7f62180701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:48.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621dd01700 1 -- 172.21.15.146:0/3977701507 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621815da20 con 0x7f62180cbd50 2024-09-13T18:50:48.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621dd01700 1 --2- 172.21.15.146:0/3977701507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62180cbd50 0x7f62180b1c40 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f62180cce40 tx=0x7f6214009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d8c07a1d26c8b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:48.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ccff700 1 -- 172.21.15.146:0/3977701507 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621400b8a0 con 0x7f62180cbd50 2024-09-13T18:50:48.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ccff700 1 -- 172.21.15.146:0/3977701507 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621400ba00 con 0x7f62180cbd50 2024-09-13T18:50:48.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ccff700 1 -- 172.21.15.146:0/3977701507 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621400bd90 con 0x7f62180cbd50 2024-09-13T18:50:48.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 -- 172.21.15.146:0/3977701507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62180cbd50 msgr2=0x7f62180b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:48.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 --2- 172.21.15.146:0/3977701507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62180cbd50 0x7f62180b1c40 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f62180cce40 tx=0x7f6214009bf0 comp rx=0 tx=0).stop 2024-09-13T18:50:48.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 -- 172.21.15.146:0/3977701507 shutdown_connections 2024-09-13T18:50:48.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 --2- 172.21.15.146:0/3977701507 >> v2:172.21.15.154:3300/0 conn(0x7f6218071390 0x7f62180cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:48.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 --2- 172.21.15.146:0/3977701507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62180cbd50 0x7f62180b1c40 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:48.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 -- 172.21.15.146:0/3977701507 >> 172.21.15.146:0/3977701507 conn(0x7f6218066270 msgr2=0x7f6218066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:48.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 -- 172.21.15.146:0/3977701507 shutdown_connections 2024-09-13T18:50:48.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.418+0000 7f621ed03700 1 -- 172.21.15.146:0/3977701507 wait complete. 2024-09-13T18:50:48.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621ed03700 1 Processor -- start 2024-09-13T18:50:48.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621ed03700 1 -- start start 2024-09-13T18:50:48.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621ed03700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6218071390 0x7f62181f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:48.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621ed03700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62180ca210 0x7f62181562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:48.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621ed03700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180cbd50 0x7f62181567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:48.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621ed03700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62181f3630 con 0x7f6218071390 2024-09-13T18:50:48.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621ed03700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62180c8e20 con 0x7f62180ca210 2024-09-13T18:50:48.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621ed03700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62180c9100 con 0x7f62180cbd50 2024-09-13T18:50:48.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621dd01700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6218071390 0x7f62181f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:48.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621dd01700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6218071390 0x7f62181f2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48594/0 (socket says 172.21.15.146:48594) 2024-09-13T18:50:48.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621dd01700 1 -- 172.21.15.146:0/1605376402 learned_addr learned my addr 172.21.15.146:0/1605376402 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:48.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621e502700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180cbd50 0x7f62181567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:48.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621d500700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62180ca210 0x7f62181562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:48.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621e502700 1 -- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62180ca210 msgr2=0x7f62181562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:48.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621e502700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62180ca210 0x7f62181562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:48.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621e502700 1 -- 172.21.15.146:0/1605376402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6218071390 msgr2=0x7f62181f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:48.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621e502700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6218071390 0x7f62181f2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:48.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621e502700 1 -- 172.21.15.146:0/1605376402 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6214009570 con 0x7f62180cbd50 2024-09-13T18:50:48.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621dd01700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6218071390 0x7f62181f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:50:48.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621e502700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180cbd50 0x7f62181567f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6210007d90 tx=0x7f621000e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:48.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f620e7fc700 1 -- 172.21.15.146:0/1605376402 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210018070 con 0x7f62180cbd50 2024-09-13T18:50:48.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f620e7fc700 1 -- 172.21.15.146:0/1605376402 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621000eb30 con 0x7f62180cbd50 2024-09-13T18:50:48.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.422+0000 7f621ed03700 1 -- 172.21.15.146:0/1605376402 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62180c6a60 con 0x7f62180cbd50 2024-09-13T18:50:48.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.426+0000 7f620e7fc700 1 -- 172.21.15.146:0/1605376402 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210003a40 con 0x7f62180cbd50 2024-09-13T18:50:48.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.426+0000 7f621ed03700 1 -- 172.21.15.146:0/1605376402 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621814fd80 con 0x7f62180cbd50 2024-09-13T18:50:48.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.426+0000 7f621ed03700 1 -- 172.21.15.146:0/1605376402 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621815d500 con 0x7f62180cbd50 2024-09-13T18:50:48.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.426+0000 7f620e7fc700 1 -- 172.21.15.146:0/1605376402 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6210014420 con 0x7f62180cbd50 2024-09-13T18:50:48.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.426+0000 7f620e7fc700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61fc066730 0x7f61fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:48.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.426+0000 7f620e7fc700 1 -- 172.21.15.146:0/1605376402 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f6210084970 con 0x7f62180cbd50 2024-09-13T18:50:48.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.426+0000 7f621dd01700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61fc066730 0x7f61fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:48.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.430+0000 7f621dd01700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61fc066730 0x7f61fc068bf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6214009150 tx=0x7f6214008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:48.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.434+0000 7f620e7fc700 1 -- 172.21.15.146:0/1605376402 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f62100513f0 con 0x7f62180cbd50 2024-09-13T18:50:48.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.642+0000 7f621ed03700 1 -- 172.21.15.146:0/1605376402 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f621815cc30 con 0x7f62180cbd50 2024-09-13T18:50:48.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.642+0000 7f620e7fc700 1 -- 172.21.15.146:0/1605376402 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+6279 (secure 0 0 0) 0x7f6210054a10 con 0x7f62180cbd50 2024-09-13T18:50:48.662 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:50:48.662 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6187,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:48:41.442913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6459,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6459":{"gid":6459,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":13,"addr":"172.21.15.146:6835/294014284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":294014284},{"type":"v1","addr":"172.21.15.146:6835","nonce":294014284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:50:48.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.646+0000 7f621ed03700 1 -- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61fc066730 msgr2=0x7f61fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:48.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.646+0000 7f621ed03700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61fc066730 0x7f61fc068bf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6214009150 tx=0x7f6214008030 comp rx=0 tx=0).stop 2024-09-13T18:50:48.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.646+0000 7f621ed03700 1 -- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180cbd50 msgr2=0x7f62181567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:48.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.646+0000 7f621ed03700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180cbd50 0x7f62181567f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6210007d90 tx=0x7f621000e3c0 comp rx=0 tx=0).stop 2024-09-13T18:50:48.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.650+0000 7f621ed03700 1 -- 172.21.15.146:0/1605376402 shutdown_connections 2024-09-13T18:50:48.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.650+0000 7f621ed03700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f61fc066730 0x7f61fc068bf0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:48.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.650+0000 7f621ed03700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180cbd50 0x7f62181567f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:48.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.650+0000 7f621ed03700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62180ca210 0x7f62181562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:48.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.650+0000 7f621ed03700 1 --2- 172.21.15.146:0/1605376402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6218071390 0x7f62181f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:48.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.650+0000 7f621ed03700 1 -- 172.21.15.146:0/1605376402 >> 172.21.15.146:0/1605376402 conn(0x7f6218066270 msgr2=0x7f62180cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:48.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.650+0000 7f621ed03700 1 -- 172.21.15.146:0/1605376402 shutdown_connections 2024-09-13T18:50:48.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:48.650+0000 7f621ed03700 1 -- 172.21.15.146:0/1605376402 wait complete. 2024-09-13T18:50:48.668 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 91 2024-09-13T18:50:48.682 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T18:50:50.683 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:50:50.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 Processor -- start 2024-09-13T18:50:50.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 -- start start 2024-09-13T18:50:50.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f900c153bc0 0x7f900c0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:50.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900c0c6a80 0x7f900c160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:50.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900c153fa0 con 0x7f900c0c6610 2024-09-13T18:50:50.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c0aa500 con 0x7f900c153bc0 2024-09-13T18:50:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c15a7e0 con 0x7f900c0c6a80 2024-09-13T18:50:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f900b7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f900c0c6610 0x7f900c0cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43466/0 (socket says 172.21.15.146:43466) 2024-09-13T18:50:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f900b7fe700 1 -- 172.21.15.146:0/3953466197 learned_addr learned my addr 172.21.15.146:0/3953466197 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:50:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f9010e5a700 1 --2- 172.21.15.146:0/3953466197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900c0c6a80 0x7f900c160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f900bfff700 1 --2- 172.21.15.146:0/3953466197 >> v2:172.21.15.154:3300/0 conn(0x7f900c153bc0 0x7f900c0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f900affd700 1 -- 172.21.15.146:0/3953466197 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327190971 0 0) 0x7f900c153fa0 con 0x7f900c0c6610 2024-09-13T18:50:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f900affd700 1 -- 172.21.15.146:0/3953466197 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ff40035d0 con 0x7f900c0c6610 2024-09-13T18:50:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f9010e5a700 1 -- 172.21.15.146:0/3953466197 >> v2:172.21.15.154:3300/0 conn(0x7f900c153bc0 msgr2=0x7f900c0c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f9010e5a700 1 --2- 172.21.15.146:0/3953466197 >> v2:172.21.15.154:3300/0 conn(0x7f900c153bc0 0x7f900c0c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f9010e5a700 1 -- 172.21.15.146:0/3953466197 >> v1:172.21.15.146:6789/0 conn(0x7f900c0c6610 legacy=0x7f900c0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f900affd700 1 -- 172.21.15.146:0/3953466197 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3248889116 0 0) 0x7f8ff40035d0 con 0x7f900c0c6610 2024-09-13T18:50:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f9010e5a700 1 -- 172.21.15.146:0/3953466197 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c0c9320 con 0x7f900c0c6a80 2024-09-13T18:50:50.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f9010e5a700 1 --2- 172.21.15.146:0/3953466197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900c0c6a80 0x7f900c160c20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9004009d90 tx=0x7f900400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0327814a20ceb88 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f900affd700 1 -- 172.21.15.146:0/3953466197 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900400e070 con 0x7f900c0c6a80 2024-09-13T18:50:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f900affd700 1 -- 172.21.15.146:0/3953466197 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900400bb20 con 0x7f900c0c6a80 2024-09-13T18:50:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f900affd700 1 -- 172.21.15.146:0/3953466197 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900400be90 con 0x7f900c0c6a80 2024-09-13T18:50:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 -- 172.21.15.146:0/3953466197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900c0c6a80 msgr2=0x7f900c160c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 --2- 172.21.15.146:0/3953466197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900c0c6a80 0x7f900c160c20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9004009d90 tx=0x7f900400b330 comp rx=0 tx=0).stop 2024-09-13T18:50:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 -- 172.21.15.146:0/3953466197 shutdown_connections 2024-09-13T18:50:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 --2- 172.21.15.146:0/3953466197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900c0c6a80 0x7f900c160c20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 --2- 172.21.15.146:0/3953466197 >> v2:172.21.15.154:3300/0 conn(0x7f900c153bc0 0x7f900c0c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 -- 172.21.15.146:0/3953466197 >> 172.21.15.146:0/3953466197 conn(0x7f900c066090 msgr2=0x7f900c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 -- 172.21.15.146:0/3953466197 shutdown_connections 2024-09-13T18:50:50.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.790+0000 7f901165b700 1 -- 172.21.15.146:0/3953466197 wait complete. 2024-09-13T18:50:50.812 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f901165b700 1 Processor -- start 2024-09-13T18:50:50.812 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f901165b700 1 -- start start 2024-09-13T18:50:50.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f901165b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c0c6610 0x7f900c0c3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:50.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f901165b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900c0c6a80 0x7f900c1e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:50.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f901165b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900c153bc0 0x7f900c0aa770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:50.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f901165b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0ad4c0 con 0x7f900c0c6610 2024-09-13T18:50:50.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f901165b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0c4dd0 con 0x7f900c0c6a80 2024-09-13T18:50:50.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f901165b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0c50b0 con 0x7f900c153bc0 2024-09-13T18:50:50.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f900bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c0c6610 0x7f900c0c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:50.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f900bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c0c6610 0x7f900c0c3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48596/0 (socket says 172.21.15.146:48596) 2024-09-13T18:50:50.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f900bfff700 1 -- 172.21.15.146:0/2753332707 learned_addr learned my addr 172.21.15.146:0/2753332707 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:50.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f900b7fe700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900c0c6a80 0x7f900c1e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:50.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f9010e5a700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900c153bc0 0x7f900c0aa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:50.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f900bfff700 1 -- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900c153bc0 msgr2=0x7f900c0aa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:50.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f900bfff700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900c153bc0 0x7f900c0aa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:50.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f900bfff700 1 -- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900c0c6a80 msgr2=0x7f900c1e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:50.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f900bfff700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900c0c6a80 0x7f900c1e6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:50.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f900bfff700 1 -- 172.21.15.146:0/2753332707 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9004009a70 con 0x7f900c0c6610 2024-09-13T18:50:50.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f900b7fe700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900c0c6a80 0x7f900c1e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:50:50.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.794+0000 7f9010e5a700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900c153bc0 0x7f900c0aa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:50:50.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.798+0000 7f900bfff700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c0c6610 0x7f900c0c3110 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f900000e840 tx=0x7f900000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:50.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.798+0000 7f9008ff9700 1 -- 172.21.15.146:0/2753332707 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000095a0 con 0x7f900c0c6610 2024-09-13T18:50:50.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.798+0000 7f9008ff9700 1 -- 172.21.15.146:0/2753332707 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900000c690 con 0x7f900c0c6610 2024-09-13T18:50:50.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.798+0000 7f901165b700 1 -- 172.21.15.146:0/2753332707 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c1ece70 con 0x7f900c0c6610 2024-09-13T18:50:50.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.798+0000 7f9008ff9700 1 -- 172.21.15.146:0/2753332707 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900000ca00 con 0x7f900c0c6610 2024-09-13T18:50:50.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.798+0000 7f901165b700 1 -- 172.21.15.146:0/2753332707 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c1ed430 con 0x7f900c0c6610 2024-09-13T18:50:50.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.798+0000 7f901165b700 1 -- 172.21.15.146:0/2753332707 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c159a00 con 0x7f900c0c6610 2024-09-13T18:50:50.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.798+0000 7f9008ff9700 1 -- 172.21.15.146:0/2753332707 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f900000cb60 con 0x7f900c0c6610 2024-09-13T18:50:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.802+0000 7f9008ff9700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ff8066730 0x7f8ff8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.802+0000 7f9008ff9700 1 -- 172.21.15.146:0/2753332707 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f9000084c90 con 0x7f900c0c6610 2024-09-13T18:50:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.802+0000 7f900b7fe700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ff8066730 0x7f8ff8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.802+0000 7f900b7fe700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ff8066730 0x7f8ff8068bf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f900c0a9660 tx=0x7f8ffc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:50.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:50.810+0000 7f9008ff9700 1 -- 172.21.15.146:0/2753332707 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f900c159a00 con 0x7f900c0c6610 2024-09-13T18:50:51.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.022+0000 7f901165b700 1 -- 172.21.15.146:0/2753332707 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f900c159a00 con 0x7f900c0c6610 2024-09-13T18:50:51.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.022+0000 7f9008ff9700 1 -- 172.21.15.146:0/2753332707 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+5521 (secure 0 0 0) 0x7f900c159a00 con 0x7f900c0c6610 2024-09-13T18:50:51.042 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:50:51.042 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":92,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:50:50.877814+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:50:51.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.026+0000 7f901165b700 1 -- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ff8066730 msgr2=0x7f8ff8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:51.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.026+0000 7f901165b700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ff8066730 0x7f8ff8068bf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f900c0a9660 tx=0x7f8ffc009040 comp rx=0 tx=0).stop 2024-09-13T18:50:51.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.026+0000 7f901165b700 1 -- 172.21.15.146:0/2753332707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c0c6610 msgr2=0x7f900c0c3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:51.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.026+0000 7f901165b700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c0c6610 0x7f900c0c3110 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f900000e840 tx=0x7f900000ec00 comp rx=0 tx=0).stop 2024-09-13T18:50:51.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.026+0000 7f901165b700 1 -- 172.21.15.146:0/2753332707 shutdown_connections 2024-09-13T18:50:51.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.026+0000 7f901165b700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ff8066730 0x7f8ff8068bf0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.026+0000 7f901165b700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900c153bc0 0x7f900c0aa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.026+0000 7f901165b700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900c0c6a80 0x7f900c1e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.026+0000 7f901165b700 1 --2- 172.21.15.146:0/2753332707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c0c6610 0x7f900c0c3110 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.026+0000 7f901165b700 1 -- 172.21.15.146:0/2753332707 >> 172.21.15.146:0/2753332707 conn(0x7f900c066090 msgr2=0x7f900c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:51.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.026+0000 7f901165b700 1 -- 172.21.15.146:0/2753332707 shutdown_connections 2024-09-13T18:50:51.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.026+0000 7f901165b700 1 -- 172.21.15.146:0/2753332707 wait complete. 2024-09-13T18:50:51.046 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 92 2024-09-13T18:50:51.057 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-09-13T18:50:51.057 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:50:51.058 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:50:51.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36bf4d5700 1 Processor -- start 2024-09-13T18:50:51.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36bf4d5700 1 -- start start 2024-09-13T18:50:51.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36bf4d5700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f36b80c8f70 0x7f36b80a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:51.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36bf4d5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f36b80c9350 0x7f36b80723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:51.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36bf4d5700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b8159f80 con 0x7f36b80c7530 2024-09-13T18:50:51.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36bf4d5700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b80ca1d0 con 0x7f36b80c8f70 2024-09-13T18:50:51.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36bf4d5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b8159670 con 0x7f36b80c9350 2024-09-13T18:50:51.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36becd4700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f36b80c7530 0x7f36b80cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:44446/0 (socket says 172.21.15.146:44446) 2024-09-13T18:50:51.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36becd4700 1 -- 172.21.15.146:0/4125473347 learned_addr learned my addr 172.21.15.146:0/4125473347 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:50:51.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36be4d3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f36b80c8f70 0x7f36b80a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:51.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36bdcd2700 1 --2- 172.21.15.146:0/4125473347 >> v2:172.21.15.154:3300/0 conn(0x7f36b80c9350 0x7f36b80723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:51.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36bd4d1700 1 -- 172.21.15.146:0/4125473347 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052121989 0 0) 0x7f36b8159f80 con 0x7f36b80c7530 2024-09-13T18:50:51.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36bd4d1700 1 -- 172.21.15.146:0/4125473347 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36a40035d0 con 0x7f36b80c7530 2024-09-13T18:50:51.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36be4d3700 1 -- 172.21.15.146:0/4125473347 >> v2:172.21.15.154:3300/0 conn(0x7f36b80c9350 msgr2=0x7f36b80723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:51.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36be4d3700 1 --2- 172.21.15.146:0/4125473347 >> v2:172.21.15.154:3300/0 conn(0x7f36b80c9350 0x7f36b80723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36be4d3700 1 -- 172.21.15.146:0/4125473347 >> v1:172.21.15.146:6789/0 conn(0x7f36b80c7530 legacy=0x7f36b80cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:51.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36be4d3700 1 -- 172.21.15.146:0/4125473347 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b80aa3a0 con 0x7f36b80c8f70 2024-09-13T18:50:51.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36be4d3700 1 --2- 172.21.15.146:0/4125473347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36b80c8f70 0x7f36b80a9cf0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f36b80c8810 tx=0x7f36b4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a496667a8a03546 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:51.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.146+0000 7f36bd4d1700 1 -- 172.21.15.146:0/4125473347 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b4011830 con 0x7f36b80c8f70 2024-09-13T18:50:51.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bd4d1700 1 -- 172.21.15.146:0/4125473347 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b4011990 con 0x7f36b80c8f70 2024-09-13T18:50:51.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bd4d1700 1 -- 172.21.15.146:0/4125473347 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b4011d00 con 0x7f36b80c8f70 2024-09-13T18:50:51.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 -- 172.21.15.146:0/4125473347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36b80c8f70 msgr2=0x7f36b80a9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:51.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 --2- 172.21.15.146:0/4125473347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36b80c8f70 0x7f36b80a9cf0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f36b80c8810 tx=0x7f36b4009b30 comp rx=0 tx=0).stop 2024-09-13T18:50:51.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 -- 172.21.15.146:0/4125473347 shutdown_connections 2024-09-13T18:50:51.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 --2- 172.21.15.146:0/4125473347 >> v2:172.21.15.154:3300/0 conn(0x7f36b80c9350 0x7f36b80723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 --2- 172.21.15.146:0/4125473347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36b80c8f70 0x7f36b80a9cf0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 -- 172.21.15.146:0/4125473347 >> 172.21.15.146:0/4125473347 conn(0x7f36b8066090 msgr2=0x7f36b8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:51.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 -- 172.21.15.146:0/4125473347 shutdown_connections 2024-09-13T18:50:51.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 -- 172.21.15.146:0/4125473347 wait complete. 2024-09-13T18:50:51.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 Processor -- start 2024-09-13T18:50:51.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 -- start start 2024-09-13T18:50:51.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36b80c7530 0x7f36b80adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:51.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b80c8f70 0x7f36b8150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:51.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36b80c9350 0x7f36b8151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:51.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b81ef470 con 0x7f36b80c9350 2024-09-13T18:50:51.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b81ef9a0 con 0x7f36b80c8f70 2024-09-13T18:50:51.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36bf4d5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b80ae090 con 0x7f36b80c7530 2024-09-13T18:50:51.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36becd4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36b80c9350 0x7f36b8151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:51.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36becd4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36b80c9350 0x7f36b8151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34620/0 (socket says 172.21.15.146:34620) 2024-09-13T18:50:51.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.150+0000 7f36becd4700 1 -- 172.21.15.146:0/1024933734 learned_addr learned my addr 172.21.15.146:0/1024933734 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:51.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36bdcd2700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b80c8f70 0x7f36b8150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:51.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36be4d3700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36b80c7530 0x7f36b80adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:51.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36becd4700 1 -- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36b80c7530 msgr2=0x7f36b80adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:51.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36becd4700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36b80c7530 0x7f36b80adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36becd4700 1 -- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b80c8f70 msgr2=0x7f36b8150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:51.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36becd4700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b80c8f70 0x7f36b8150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36becd4700 1 -- 172.21.15.146:0/1024933734 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b40094b0 con 0x7f36b80c9350 2024-09-13T18:50:51.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36be4d3700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36b80c7530 0x7f36b80adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:50:51.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36becd4700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36b80c9350 0x7f36b8151a10 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f36b000cba0 tx=0x7f36b000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:51.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36aeffd700 1 -- 172.21.15.146:0/1024933734 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b000a770 con 0x7f36b80c9350 2024-09-13T18:50:51.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36aeffd700 1 -- 172.21.15.146:0/1024933734 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b000a8d0 con 0x7f36b80c9350 2024-09-13T18:50:51.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36aeffd700 1 -- 172.21.15.146:0/1024933734 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b000ac60 con 0x7f36b80c9350 2024-09-13T18:50:51.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36bf4d5700 1 -- 172.21.15.146:0/1024933734 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b81ef6c0 con 0x7f36b80c9350 2024-09-13T18:50:51.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36bf4d5700 1 -- 172.21.15.146:0/1024933734 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b80aae00 con 0x7f36b80c9350 2024-09-13T18:50:51.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36bf4d5700 1 -- 172.21.15.146:0/1024933734 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b80c9cf0 con 0x7f36b80c9350 2024-09-13T18:50:51.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36aeffd700 1 -- 172.21.15.146:0/1024933734 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f36b001b3b0 con 0x7f36b80c9350 2024-09-13T18:50:51.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.154+0000 7f36aeffd700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f369c066730 0x7f369c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:51.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.158+0000 7f36aeffd700 1 -- 172.21.15.146:0/1024933734 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7f36b0084380 con 0x7f36b80c9350 2024-09-13T18:50:51.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.158+0000 7f36be4d3700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f369c066730 0x7f369c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:51.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.158+0000 7f36be4d3700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f369c066730 0x7f369c068bf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f36b4000c00 tx=0x7f36b4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:51.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.166+0000 7f36aeffd700 1 -- 172.21.15.146:0/1024933734 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f36b0050ee0 con 0x7f36b80c9350 2024-09-13T18:50:51.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.366+0000 7f36bf4d5700 1 -- 172.21.15.146:0/1024933734 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36b80c6830 con 0x7f36b80c9350 2024-09-13T18:50:51.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.370+0000 7f36aeffd700 1 -- 172.21.15.146:0/1024933734 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+5521 (secure 0 0 0) 0x7f36b0054500 con 0x7f36b80c9350 2024-09-13T18:50:51.387 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:50:51.387 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":92,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:50:50.877814+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:50:51.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.374+0000 7f36bf4d5700 1 -- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f369c066730 msgr2=0x7f369c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:51.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.374+0000 7f36bf4d5700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f369c066730 0x7f369c068bf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f36b4000c00 tx=0x7f36b4008030 comp rx=0 tx=0).stop 2024-09-13T18:50:51.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.374+0000 7f36bf4d5700 1 -- 172.21.15.146:0/1024933734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36b80c9350 msgr2=0x7f36b8151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:51.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.374+0000 7f36bf4d5700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36b80c9350 0x7f36b8151a10 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f36b000cba0 tx=0x7f36b000cbd0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.374+0000 7f36bf4d5700 1 -- 172.21.15.146:0/1024933734 shutdown_connections 2024-09-13T18:50:51.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.374+0000 7f36bf4d5700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f369c066730 0x7f369c068bf0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.374+0000 7f36bf4d5700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36b80c9350 0x7f36b8151a10 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.374+0000 7f36bf4d5700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b80c8f70 0x7f36b8150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.374+0000 7f36bf4d5700 1 --2- 172.21.15.146:0/1024933734 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36b80c7530 0x7f36b80adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:51.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.374+0000 7f36bf4d5700 1 -- 172.21.15.146:0/1024933734 >> 172.21.15.146:0/1024933734 conn(0x7f36b8066090 msgr2=0x7f36b8158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:51.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.374+0000 7f36bf4d5700 1 -- 172.21.15.146:0/1024933734 shutdown_connections 2024-09-13T18:50:51.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:51.374+0000 7f36bf4d5700 1 -- 172.21.15.146:0/1024933734 wait complete. 2024-09-13T18:50:51.393 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 92 2024-09-13T18:50:51.408 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 92, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 92, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6187": { "addr": "172.21.15.146:6833/3023779168", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3023779168, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3023779168, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6187, "incarnation": 92, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:replay", "state_seq": 1 }, "gid_6714": { "addr": "172.21.15.146:6837/2169954140", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 2169954140, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 2169954140, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6714, "incarnation": 86, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 38 } }, "last_failure": 0, "last_failure_osd_epoch": 40, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:50:50.877814+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6187, "mds_1": 6714 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6839/2053734262", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 2053734262, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2053734262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 91, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6032, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:50:53.410 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:50:53.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454f59e700 1 Processor -- start 2024-09-13T18:50:53.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454f59e700 1 -- start start 2024-09-13T18:50:53.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454f59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f455015a110 0x7f455006ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:53.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454f59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455006fdf0 con 0x7f45501538e0 2024-09-13T18:50:53.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454f59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4550070630 con 0x7f455006b2e0 2024-09-13T18:50:53.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454f59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455006b6f0 con 0x7f455015a110 2024-09-13T18:50:53.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454e59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f45501538e0 0x7f455006cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:44452/0 (socket says 172.21.15.146:44452) 2024-09-13T18:50:53.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454e59c700 1 -- 172.21.15.146:0/3082130296 learned_addr learned my addr 172.21.15.146:0/3082130296 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:50:53.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454dd9b700 1 --2- 172.21.15.146:0/3082130296 >> v2:172.21.15.154:3300/0 conn(0x7f455015a110 0x7f455006ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:53.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454d59a700 1 -- 172.21.15.146:0/3082130296 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077499064 0 0) 0x7f455006fdf0 con 0x7f45501538e0 2024-09-13T18:50:53.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454d59a700 1 -- 172.21.15.146:0/3082130296 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f453c0035d0 con 0x7f45501538e0 2024-09-13T18:50:53.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454dd9b700 1 -- 172.21.15.146:0/3082130296 >> v1:172.21.15.154:6789/0 conn(0x7f455006b2e0 legacy=0x7f45500c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:53.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454dd9b700 1 -- 172.21.15.146:0/3082130296 >> v1:172.21.15.146:6789/0 conn(0x7f45501538e0 legacy=0x7f455006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:53.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454d59a700 1 -- 172.21.15.146:0/3082130296 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3914231599 0 0) 0x7f453c0035d0 con 0x7f45501538e0 2024-09-13T18:50:53.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454dd9b700 1 -- 172.21.15.146:0/3082130296 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4550070070 con 0x7f455015a110 2024-09-13T18:50:53.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454dd9b700 1 --2- 172.21.15.146:0/3082130296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f455015a110 0x7f455006ada0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4540009d90 tx=0x7f454000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=612a31e1adba7b98 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:53.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454d59a700 1 -- 172.21.15.146:0/3082130296 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000e070 con 0x7f455015a110 2024-09-13T18:50:53.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454d59a700 1 -- 172.21.15.146:0/3082130296 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454000bb20 con 0x7f455015a110 2024-09-13T18:50:53.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.518+0000 7f454d59a700 1 -- 172.21.15.146:0/3082130296 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000be90 con 0x7f455015a110 2024-09-13T18:50:53.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.522+0000 7f454f59e700 1 -- 172.21.15.146:0/3082130296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f455015a110 msgr2=0x7f455006ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:53.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.522+0000 7f454f59e700 1 --2- 172.21.15.146:0/3082130296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f455015a110 0x7f455006ada0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4540009d90 tx=0x7f454000b330 comp rx=0 tx=0).stop 2024-09-13T18:50:53.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.522+0000 7f454f59e700 1 -- 172.21.15.146:0/3082130296 shutdown_connections 2024-09-13T18:50:53.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.522+0000 7f454f59e700 1 --2- 172.21.15.146:0/3082130296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f455015a110 0x7f455006ada0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:53.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.522+0000 7f454f59e700 1 -- 172.21.15.146:0/3082130296 >> 172.21.15.146:0/3082130296 conn(0x7f45500723d0 msgr2=0x7f4550066920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:53.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.522+0000 7f454f59e700 1 -- 172.21.15.146:0/3082130296 shutdown_connections 2024-09-13T18:50:53.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.522+0000 7f454f59e700 1 -- 172.21.15.146:0/3082130296 wait complete. 2024-09-13T18:50:53.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454f59e700 1 Processor -- start 2024-09-13T18:50:53.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454f59e700 1 -- start start 2024-09-13T18:50:53.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454f59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f455006b2e0 0x7f45501f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:53.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45501538e0 0x7f45501551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:53.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f455015a110 0x7f45500bc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:53.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454f59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45501ecd00 con 0x7f45501538e0 2024-09-13T18:50:53.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454f59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4550154a10 con 0x7f455006b2e0 2024-09-13T18:50:53.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45501f2fc0 con 0x7f455015a110 2024-09-13T18:50:53.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454dd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45501538e0 0x7f45501551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:53.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454dd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45501538e0 0x7f45501551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34632/0 (socket says 172.21.15.146:34632) 2024-09-13T18:50:53.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454dd9b700 1 -- 172.21.15.146:0/890992547 learned_addr learned my addr 172.21.15.146:0/890992547 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:53.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454e59c700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f455006b2e0 0x7f45501f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:53.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454ed9d700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f455015a110 0x7f45500bc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:53.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454dd9b700 1 -- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f455015a110 msgr2=0x7f45500bc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:53.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454dd9b700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f455015a110 0x7f45500bc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:53.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454dd9b700 1 -- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f455006b2e0 msgr2=0x7f45501f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:53.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454dd9b700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f455006b2e0 0x7f45501f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:53.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454dd9b700 1 -- 172.21.15.146:0/890992547 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4540009a70 con 0x7f45501538e0 2024-09-13T18:50:53.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454e59c700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f455006b2e0 0x7f45501f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:50:53.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454dd9b700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45501538e0 0x7f45501551f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4540014040 tx=0x7f4540004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:53.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f4546ffd700 1 -- 172.21.15.146:0/890992547 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4540004810 con 0x7f45501538e0 2024-09-13T18:50:53.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f4546ffd700 1 -- 172.21.15.146:0/890992547 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4540021070 con 0x7f45501538e0 2024-09-13T18:50:53.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454f59e700 1 -- 172.21.15.146:0/890992547 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4550156120 con 0x7f45501538e0 2024-09-13T18:50:53.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f4546ffd700 1 -- 172.21.15.146:0/890992547 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4540026670 con 0x7f45501538e0 2024-09-13T18:50:53.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.526+0000 7f454f59e700 1 -- 172.21.15.146:0/890992547 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45501564b0 con 0x7f45501538e0 2024-09-13T18:50:53.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.530+0000 7f454f59e700 1 -- 172.21.15.146:0/890992547 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455015a510 con 0x7f45501538e0 2024-09-13T18:50:53.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.530+0000 7f4546ffd700 1 -- 172.21.15.146:0/890992547 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4540023030 con 0x7f45501538e0 2024-09-13T18:50:53.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.530+0000 7f4546ffd700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4530066730 0x7f4530068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:53.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.530+0000 7f4546ffd700 1 -- 172.21.15.146:0/890992547 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7f4540086a00 con 0x7f45501538e0 2024-09-13T18:50:53.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.530+0000 7f454e59c700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4530066730 0x7f4530068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:53.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.530+0000 7f454e59c700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4530066730 0x7f4530068bf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4538005930 tx=0x7f4538009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:53.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.538+0000 7f4546ffd700 1 -- 172.21.15.146:0/890992547 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4540053700 con 0x7f45501538e0 2024-09-13T18:50:53.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f454f59e700 1 -- 172.21.15.146:0/890992547 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4550066a50 con 0x7f45501538e0 2024-09-13T18:50:53.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f4546ffd700 1 -- 172.21.15.146:0/890992547 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+5521 (secure 0 0 0) 0x7f4540056d20 con 0x7f45501538e0 2024-09-13T18:50:53.763 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:50:53.764 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":92,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:50:50.877814+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:50:53.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f454f59e700 1 -- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4530066730 msgr2=0x7f4530068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:53.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f454f59e700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4530066730 0x7f4530068bf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4538005930 tx=0x7f4538009040 comp rx=0 tx=0).stop 2024-09-13T18:50:53.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f454f59e700 1 -- 172.21.15.146:0/890992547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45501538e0 msgr2=0x7f45501551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:53.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f454f59e700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45501538e0 0x7f45501551f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4540014040 tx=0x7f4540004af0 comp rx=0 tx=0).stop 2024-09-13T18:50:53.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f454f59e700 1 -- 172.21.15.146:0/890992547 shutdown_connections 2024-09-13T18:50:53.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f454f59e700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4530066730 0x7f4530068bf0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:53.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f454f59e700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f455015a110 0x7f45500bc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:53.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f454f59e700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45501538e0 0x7f45501551f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:53.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f454f59e700 1 --2- 172.21.15.146:0/890992547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f455006b2e0 0x7f45501f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:53.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f454f59e700 1 -- 172.21.15.146:0/890992547 >> 172.21.15.146:0/890992547 conn(0x7f45500723d0 msgr2=0x7f455014e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:53.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f454f59e700 1 -- 172.21.15.146:0/890992547 shutdown_connections 2024-09-13T18:50:53.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:53.746+0000 7f454f59e700 1 -- 172.21.15.146:0/890992547 wait complete. 2024-09-13T18:50:53.767 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 92 2024-09-13T18:50:53.779 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:50:55.781 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:50:55.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.890+0000 7f34c295b700 1 Processor -- start 2024-09-13T18:50:55.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.890+0000 7f34c295b700 1 -- start start 2024-09-13T18:50:55.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c295b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34bc0c6f90 0x7f34bc156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:55.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c295b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34bc150850 con 0x7f34bc06fcc0 2024-09-13T18:50:55.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c295b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34bc0c2420 con 0x7f34bc150470 2024-09-13T18:50:55.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c295b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34bc0c2ff0 con 0x7f34bc0c6f90 2024-09-13T18:50:55.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c1158700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f34bc06fcc0 0x7f34bc0c6880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:44466/0 (socket says 172.21.15.146:44466) 2024-09-13T18:50:55.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c1158700 1 -- 172.21.15.146:0/1642780930 learned_addr learned my addr 172.21.15.146:0/1642780930 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:50:55.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c215a700 1 --2- 172.21.15.146:0/1642780930 >> v2:172.21.15.154:3300/0 conn(0x7f34bc0c6f90 0x7f34bc156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:55.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c0957700 1 -- 172.21.15.146:0/1642780930 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219262348 0 0) 0x7f34bc150850 con 0x7f34bc06fcc0 2024-09-13T18:50:55.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c0957700 1 -- 172.21.15.146:0/1642780930 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34a80035d0 con 0x7f34bc06fcc0 2024-09-13T18:50:55.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c215a700 1 -- 172.21.15.146:0/1642780930 >> v1:172.21.15.154:6789/0 conn(0x7f34bc150470 legacy=0x7f34bc070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:55.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c215a700 1 -- 172.21.15.146:0/1642780930 >> v1:172.21.15.146:6789/0 conn(0x7f34bc06fcc0 legacy=0x7f34bc0c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:55.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c215a700 1 -- 172.21.15.146:0/1642780930 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc0c26c0 con 0x7f34bc0c6f90 2024-09-13T18:50:55.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c215a700 1 --2- 172.21.15.146:0/1642780930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34bc0c6f90 0x7f34bc156460 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009d90 tx=0x7f34b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=400779a455695dae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:55.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c0957700 1 -- 172.21.15.146:0/1642780930 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b400e070 con 0x7f34bc0c6f90 2024-09-13T18:50:55.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c0957700 1 -- 172.21.15.146:0/1642780930 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b400bb20 con 0x7f34bc0c6f90 2024-09-13T18:50:55.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c0957700 1 -- 172.21.15.146:0/1642780930 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b400be90 con 0x7f34bc0c6f90 2024-09-13T18:50:55.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c295b700 1 -- 172.21.15.146:0/1642780930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34bc0c6f90 msgr2=0x7f34bc156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:55.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c295b700 1 --2- 172.21.15.146:0/1642780930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34bc0c6f90 0x7f34bc156460 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009d90 tx=0x7f34b400b330 comp rx=0 tx=0).stop 2024-09-13T18:50:55.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c295b700 1 -- 172.21.15.146:0/1642780930 shutdown_connections 2024-09-13T18:50:55.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c295b700 1 --2- 172.21.15.146:0/1642780930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34bc0c6f90 0x7f34bc156460 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:55.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c295b700 1 -- 172.21.15.146:0/1642780930 >> 172.21.15.146:0/1642780930 conn(0x7f34bc0cbe80 msgr2=0x7f34bc0cc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:55.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c295b700 1 -- 172.21.15.146:0/1642780930 shutdown_connections 2024-09-13T18:50:55.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.894+0000 7f34c295b700 1 -- 172.21.15.146:0/1642780930 wait complete. 2024-09-13T18:50:55.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c295b700 1 Processor -- start 2024-09-13T18:50:55.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c295b700 1 -- start start 2024-09-13T18:50:55.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c295b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34bc06fcc0 0x7f34bc154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:55.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c295b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34bc0c6f90 0x7f34bc155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:55.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c295b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34bc150470 0x7f34bc140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:55.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c295b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc0caf00 con 0x7f34bc06fcc0 2024-09-13T18:50:55.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c295b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc0cb710 con 0x7f34bc150470 2024-09-13T18:50:55.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c295b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1542f0 con 0x7f34bc0c6f90 2024-09-13T18:50:55.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c1959700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34bc06fcc0 0x7f34bc154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:55.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c1959700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34bc06fcc0 0x7f34bc154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34634/0 (socket says 172.21.15.146:34634) 2024-09-13T18:50:55.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c1959700 1 -- 172.21.15.146:0/465417501 learned_addr learned my addr 172.21.15.146:0/465417501 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:55.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c1158700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34bc0c6f90 0x7f34bc155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:55.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c215a700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34bc150470 0x7f34bc140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:55.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c1959700 1 -- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34bc0c6f90 msgr2=0x7f34bc155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:55.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c1959700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34bc0c6f90 0x7f34bc155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:55.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c1959700 1 -- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34bc150470 msgr2=0x7f34bc140a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:55.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c1959700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34bc150470 0x7f34bc140a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:55.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c1959700 1 -- 172.21.15.146:0/465417501 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b4009a70 con 0x7f34bc06fcc0 2024-09-13T18:50:55.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c1959700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34bc06fcc0 0x7f34bc154c40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f34b800c500 tx=0x7f34b800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:55.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34b27fc700 1 -- 172.21.15.146:0/465417501 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b80052d0 con 0x7f34bc06fcc0 2024-09-13T18:50:55.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34b27fc700 1 -- 172.21.15.146:0/465417501 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b8005430 con 0x7f34bc06fcc0 2024-09-13T18:50:55.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c295b700 1 -- 172.21.15.146:0/465417501 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34bc0cb150 con 0x7f34bc06fcc0 2024-09-13T18:50:55.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34b27fc700 1 -- 172.21.15.146:0/465417501 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b80055a0 con 0x7f34bc06fcc0 2024-09-13T18:50:55.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.898+0000 7f34c295b700 1 -- 172.21.15.146:0/465417501 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34bc1df770 con 0x7f34bc06fcc0 2024-09-13T18:50:55.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.902+0000 7f34c295b700 1 -- 172.21.15.146:0/465417501 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34bc0c6880 con 0x7f34bc06fcc0 2024-09-13T18:50:55.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.902+0000 7f34b27fc700 1 -- 172.21.15.146:0/465417501 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f34b80038e0 con 0x7f34bc06fcc0 2024-09-13T18:50:55.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.902+0000 7f34b27fc700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f34a0066730 0x7f34a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:55.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.902+0000 7f34b27fc700 1 -- 172.21.15.146:0/465417501 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7f34b8083c90 con 0x7f34bc06fcc0 2024-09-13T18:50:55.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.902+0000 7f34c1158700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f34a0066730 0x7f34a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:55.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.906+0000 7f34c1158700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f34a0066730 0x7f34a0068bf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f34bc155c00 tx=0x7f34ac007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:55.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:55.910+0000 7f34b27fc700 1 -- 172.21.15.146:0/465417501 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f34b8050870 con 0x7f34bc06fcc0 2024-09-13T18:50:56.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.118+0000 7f34c295b700 1 -- 172.21.15.146:0/465417501 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34bc0c85b0 con 0x7f34bc06fcc0 2024-09-13T18:50:56.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.118+0000 7f34b27fc700 1 -- 172.21.15.146:0/465417501 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+5521 (secure 0 0 0) 0x7f34b8017020 con 0x7f34bc06fcc0 2024-09-13T18:50:56.137 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:50:56.138 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":92,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:50:50.877814+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:50:56.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.122+0000 7f34c295b700 1 -- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f34a0066730 msgr2=0x7f34a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:56.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.122+0000 7f34c295b700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f34a0066730 0x7f34a0068bf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f34bc155c00 tx=0x7f34ac007de0 comp rx=0 tx=0).stop 2024-09-13T18:50:56.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.122+0000 7f34c295b700 1 -- 172.21.15.146:0/465417501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34bc06fcc0 msgr2=0x7f34bc154c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:56.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.122+0000 7f34c295b700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34bc06fcc0 0x7f34bc154c40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f34b800c500 tx=0x7f34b800c810 comp rx=0 tx=0).stop 2024-09-13T18:50:56.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.122+0000 7f34c295b700 1 -- 172.21.15.146:0/465417501 shutdown_connections 2024-09-13T18:50:56.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.122+0000 7f34c295b700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f34a0066730 0x7f34a0068bf0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:56.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.122+0000 7f34c295b700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34bc150470 0x7f34bc140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:56.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.122+0000 7f34c295b700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34bc0c6f90 0x7f34bc155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:56.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.122+0000 7f34c295b700 1 --2- 172.21.15.146:0/465417501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34bc06fcc0 0x7f34bc154c40 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:56.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.122+0000 7f34c295b700 1 -- 172.21.15.146:0/465417501 >> 172.21.15.146:0/465417501 conn(0x7f34bc0cbe80 msgr2=0x7f34bc0c8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:56.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.122+0000 7f34c295b700 1 -- 172.21.15.146:0/465417501 shutdown_connections 2024-09-13T18:50:56.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:56.122+0000 7f34c295b700 1 -- 172.21.15.146:0/465417501 wait complete. 2024-09-13T18:50:56.141 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 92 2024-09-13T18:50:56.152 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:50:58.154 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:50:58.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.258+0000 7f98360c0700 1 Processor -- start 2024-09-13T18:50:58.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.258+0000 7f98360c0700 1 -- start start 2024-09-13T18:50:58.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98360c0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9830071610 0x7f9830159b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:58.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98360c0700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9830071410 con 0x7f9830150610 2024-09-13T18:50:58.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98360c0700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9830156090 con 0x7f9830150230 2024-09-13T18:50:58.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98360c0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98301596b0 con 0x7f9830071610 2024-09-13T18:50:58.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98348bd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9830150610 0x7f983006d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:44478/0 (socket says 172.21.15.146:44478) 2024-09-13T18:50:58.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98348bd700 1 -- 172.21.15.146:0/1537079909 learned_addr learned my addr 172.21.15.146:0/1537079909 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:50:58.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98358bf700 1 --2- 172.21.15.146:0/1537079909 >> v2:172.21.15.154:3300/0 conn(0x7f9830071610 0x7f9830159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:58.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264441680 0 0) 0x7f9830071410 con 0x7f9830150610 2024-09-13T18:50:58.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f981c0035d0 con 0x7f9830150610 2024-09-13T18:50:58.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1277548463 0 0) 0x7f981c0035d0 con 0x7f9830150610 2024-09-13T18:50:58.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 >> v2:172.21.15.154:3300/0 conn(0x7f9830071610 msgr2=0x7f9830159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:58.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 --2- 172.21.15.146:0/1537079909 >> v2:172.21.15.154:3300/0 conn(0x7f9830071610 0x7f9830159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:58.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 >> v1:172.21.15.154:6789/0 conn(0x7f9830150230 legacy=0x7f983006ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:58.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9830156350 con 0x7f9830150610 2024-09-13T18:50:58.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f9820001670 con 0x7f9830150610 2024-09-13T18:50:58.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 >> v1:172.21.15.146:6789/0 conn(0x7f9830150610 legacy=0x7f983006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:58.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 --2- 172.21.15.146:0/1537079909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f981c003a90 0x7f981c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:58.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 --2- 172.21.15.146:0/1537079909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f981c004870 0x7f981c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:58.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 --2- 172.21.15.146:0/1537079909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f981c005230 0x7f981c00dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:58.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98301596b0 con 0x7f981c005230 2024-09-13T18:50:58.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c00e370 con 0x7f981c003a90 2024-09-13T18:50:58.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c00e530 con 0x7f981c004870 2024-09-13T18:50:58.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98358bf700 1 --2- 172.21.15.146:0/1537079909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f981c005230 0x7f981c00dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:58.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98350be700 1 --2- 172.21.15.146:0/1537079909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f981c003a90 0x7f981c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:58.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98348bd700 1 --2- 172.21.15.146:0/1537079909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f981c004870 0x7f981c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:58.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98358bf700 1 -- 172.21.15.146:0/1537079909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f981c004870 msgr2=0x7f981c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:58.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98358bf700 1 --2- 172.21.15.146:0/1537079909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f981c004870 0x7f981c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:58.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98358bf700 1 -- 172.21.15.146:0/1537079909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f981c003a90 msgr2=0x7f981c003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:50:58.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98358bf700 1 --2- 172.21.15.146:0/1537079909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f981c003a90 0x7f981c003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:58.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98358bf700 1 -- 172.21.15.146:0/1537079909 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c0035d0 con 0x7f981c005230 2024-09-13T18:50:58.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98358bf700 1 -- 172.21.15.146:0/1537079909 reap_dead start 2024-09-13T18:50:58.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f98358bf700 1 --2- 172.21.15.146:0/1537079909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f981c005230 0x7f981c00dc40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9828009f90 tx=0x7f981c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731ada4783776d7b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:58.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828003a40 con 0x7f981c005230 2024-09-13T18:50:58.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982800f040 con 0x7f981c005230 2024-09-13T18:50:58.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.262+0000 7f9827fff700 1 -- 172.21.15.146:0/1537079909 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828003de0 con 0x7f981c005230 2024-09-13T18:50:58.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 -- 172.21.15.146:0/1537079909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f981c005230 msgr2=0x7f981c00dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:58.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 --2- 172.21.15.146:0/1537079909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f981c005230 0x7f981c00dc40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9828009f90 tx=0x7f981c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:50:58.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 -- 172.21.15.146:0/1537079909 shutdown_connections 2024-09-13T18:50:58.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 --2- 172.21.15.146:0/1537079909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f981c005230 0x7f981c00dc40 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:58.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 -- 172.21.15.146:0/1537079909 >> 172.21.15.146:0/1537079909 conn(0x7f9830071ed0 msgr2=0x7f98300722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:58.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 -- 172.21.15.146:0/1537079909 shutdown_connections 2024-09-13T18:50:58.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 -- 172.21.15.146:0/1537079909 wait complete. 2024-09-13T18:50:58.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 Processor -- start 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 -- start start 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f981c005230 0x7f9830154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9830155960 0x7f98301ee660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98301eeba0 0x7f98300ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9830142440 con 0x7f9830155960 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98301543d0 con 0x7f98301eeba0 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98360c0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9830154900 con 0x7f981c005230 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98348bd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9830155960 0x7f98301ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98348bd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9830155960 0x7f98301ee660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34652/0 (socket says 172.21.15.146:34652) 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98348bd700 1 -- 172.21.15.146:0/694505097 learned_addr learned my addr 172.21.15.146:0/694505097 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98350be700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f981c005230 0x7f9830154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.266+0000 7f98358bf700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98301eeba0 0x7f98300ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.270+0000 7f98348bd700 1 -- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f981c005230 msgr2=0x7f9830154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:58.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.270+0000 7f98348bd700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f981c005230 0x7f9830154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:58.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.270+0000 7f98348bd700 1 -- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98301eeba0 msgr2=0x7f98300ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:58.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.270+0000 7f98348bd700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98301eeba0 0x7f98300ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:58.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.270+0000 7f98348bd700 1 -- 172.21.15.146:0/694505097 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98300c9f20 con 0x7f9830155960 2024-09-13T18:50:58.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.270+0000 7f98358bf700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98301eeba0 0x7f98300ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:50:58.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.270+0000 7f98348bd700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9830155960 0x7f98301ee660 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f9820007290 tx=0x7f98200075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:58.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.270+0000 7f9825ffb700 1 -- 172.21.15.146:0/694505097 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820012620 con 0x7f9830155960 2024-09-13T18:50:58.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.270+0000 7f98360c0700 1 -- 172.21.15.146:0/694505097 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98301ef160 con 0x7f9830155960 2024-09-13T18:50:58.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.270+0000 7f9825ffb700 1 -- 172.21.15.146:0/694505097 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9820012780 con 0x7f9830155960 2024-09-13T18:50:58.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.270+0000 7f9825ffb700 1 -- 172.21.15.146:0/694505097 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820012b10 con 0x7f9830155960 2024-09-13T18:50:58.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.270+0000 7f98360c0700 1 -- 172.21.15.146:0/694505097 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9830143070 con 0x7f9830155960 2024-09-13T18:50:58.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.270+0000 7f98360c0700 1 -- 172.21.15.146:0/694505097 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9830071130 con 0x7f9830155960 2024-09-13T18:50:58.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.274+0000 7f9825ffb700 1 -- 172.21.15.146:0/694505097 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9820012c70 con 0x7f9830155960 2024-09-13T18:50:58.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.274+0000 7f9825ffb700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9814066760 0x7f9814068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:50:58.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.274+0000 7f9825ffb700 1 -- 172.21.15.146:0/694505097 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7f9820085400 con 0x7f9830155960 2024-09-13T18:50:58.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.274+0000 7f98350be700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9814066760 0x7f9814068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:50:58.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.274+0000 7f98350be700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9814066760 0x7f9814068c20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f982c000c00 tx=0x7f982c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:50:58.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.278+0000 7f9825ffb700 1 -- 172.21.15.146:0/694505097 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9820051ec0 con 0x7f9830155960 2024-09-13T18:50:58.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.486+0000 7f98360c0700 1 -- 172.21.15.146:0/694505097 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f983006ebd0 con 0x7f9830155960 2024-09-13T18:50:58.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.490+0000 7f9825ffb700 1 -- 172.21.15.146:0/694505097 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+5524 (secure 0 0 0) 0x7f98200554e0 con 0x7f9830155960 2024-09-13T18:50:58.507 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:50:58.507 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:50:56.908526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:resolve","state_seq":110,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:50:58.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.490+0000 7f98360c0700 1 -- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9814066760 msgr2=0x7f9814068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:58.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.490+0000 7f98360c0700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9814066760 0x7f9814068c20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f982c000c00 tx=0x7f982c009040 comp rx=0 tx=0).stop 2024-09-13T18:50:58.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.490+0000 7f98360c0700 1 -- 172.21.15.146:0/694505097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9830155960 msgr2=0x7f98301ee660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:50:58.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.490+0000 7f98360c0700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9830155960 0x7f98301ee660 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f9820007290 tx=0x7f98200075a0 comp rx=0 tx=0).stop 2024-09-13T18:50:58.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.490+0000 7f98360c0700 1 -- 172.21.15.146:0/694505097 shutdown_connections 2024-09-13T18:50:58.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.490+0000 7f98360c0700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9814066760 0x7f9814068c20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:58.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.490+0000 7f98360c0700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98301eeba0 0x7f98300ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:58.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.490+0000 7f98360c0700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9830155960 0x7f98301ee660 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:58.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.490+0000 7f98360c0700 1 --2- 172.21.15.146:0/694505097 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f981c005230 0x7f9830154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:50:58.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.490+0000 7f98360c0700 1 -- 172.21.15.146:0/694505097 >> 172.21.15.146:0/694505097 conn(0x7f9830071ed0 msgr2=0x7f98300c56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:50:58.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.490+0000 7f98360c0700 1 -- 172.21.15.146:0/694505097 shutdown_connections 2024-09-13T18:50:58.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:50:58.490+0000 7f98360c0700 1 -- 172.21.15.146:0/694505097 wait complete. 2024-09-13T18:50:58.510 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 93 2024-09-13T18:50:58.522 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:51:00.523 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:51:00.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faf00260700 1 Processor -- start 2024-09-13T18:51:00.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faf00260700 1 -- start start 2024-09-13T18:51:00.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faf00260700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faef806e330 0x7faef80c0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:00.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faf00260700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faef80c1440 0x7faef80c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:00.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faf00260700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef8156bf0 con 0x7faef806df50 2024-09-13T18:51:00.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faf00260700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef80cc020 con 0x7faef80c1440 2024-09-13T18:51:00.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faf00260700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef8070560 con 0x7faef806e330 2024-09-13T18:51:00.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefdffc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7faef806df50 0x7faef806fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:44482/0 (socket says 172.21.15.146:44482) 2024-09-13T18:51:00.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefdffc700 1 -- 172.21.15.146:0/2368660429 learned_addr learned my addr 172.21.15.146:0/2368660429 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:51:00.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefe7fd700 1 --2- 172.21.15.146:0/2368660429 >> v2:172.21.15.154:3300/0 conn(0x7faef80c1440 0x7faef80c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:00.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefd7fb700 1 --2- 172.21.15.146:0/2368660429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faef806e330 0x7faef80c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:00.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefcffa700 1 -- 172.21.15.146:0/2368660429 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130419752 0 0) 0x7faef8156bf0 con 0x7faef806df50 2024-09-13T18:51:00.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefcffa700 1 -- 172.21.15.146:0/2368660429 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee40035d0 con 0x7faef806df50 2024-09-13T18:51:00.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefe7fd700 1 -- 172.21.15.146:0/2368660429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faef806e330 msgr2=0x7faef80c0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:51:00.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefe7fd700 1 --2- 172.21.15.146:0/2368660429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faef806e330 0x7faef80c0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:00.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefe7fd700 1 -- 172.21.15.146:0/2368660429 >> v1:172.21.15.146:6789/0 conn(0x7faef806df50 legacy=0x7faef806fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:00.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefe7fd700 1 -- 172.21.15.146:0/2368660429 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef80c1db0 con 0x7faef80c1440 2024-09-13T18:51:00.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefe7fd700 1 --2- 172.21.15.146:0/2368660429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faef80c1440 0x7faef80c1870 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faef0009d90 tx=0x7faef000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3db1c009cad1f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:00.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefcffa700 1 -- 172.21.15.146:0/2368660429 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef000e070 con 0x7faef80c1440 2024-09-13T18:51:00.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefcffa700 1 -- 172.21.15.146:0/2368660429 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef000bb20 con 0x7faef80c1440 2024-09-13T18:51:00.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.630+0000 7faefcffa700 1 -- 172.21.15.146:0/2368660429 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef000be90 con 0x7faef80c1440 2024-09-13T18:51:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.634+0000 7faf00260700 1 -- 172.21.15.146:0/2368660429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faef80c1440 msgr2=0x7faef80c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.634+0000 7faf00260700 1 --2- 172.21.15.146:0/2368660429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faef80c1440 0x7faef80c1870 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faef0009d90 tx=0x7faef000b330 comp rx=0 tx=0).stop 2024-09-13T18:51:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.634+0000 7faf00260700 1 -- 172.21.15.146:0/2368660429 shutdown_connections 2024-09-13T18:51:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.634+0000 7faf00260700 1 --2- 172.21.15.146:0/2368660429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faef80c1440 0x7faef80c1870 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.634+0000 7faf00260700 1 --2- 172.21.15.146:0/2368660429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faef806e330 0x7faef80c0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.634+0000 7faf00260700 1 -- 172.21.15.146:0/2368660429 >> 172.21.15.146:0/2368660429 conn(0x7faef80cb1d0 msgr2=0x7faef80cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.634+0000 7faf00260700 1 -- 172.21.15.146:0/2368660429 shutdown_connections 2024-09-13T18:51:00.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.634+0000 7faf00260700 1 -- 172.21.15.146:0/2368660429 wait complete. 2024-09-13T18:51:00.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.634+0000 7faf00260700 1 Processor -- start 2024-09-13T18:51:00.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.634+0000 7faf00260700 1 -- start start 2024-09-13T18:51:00.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faf00260700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faef806df50 0x7faef81644a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:00.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faf00260700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef806e330 0x7faef80affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:00.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faf00260700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faef80c1440 0x7faef8163520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:00.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faf00260700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef8153500 con 0x7faef806e330 2024-09-13T18:51:00.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faf00260700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef81537a0 con 0x7faef806df50 2024-09-13T18:51:00.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faf00260700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef8162480 con 0x7faef80c1440 2024-09-13T18:51:00.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faefd7fb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef806e330 0x7faef80affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:00.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faefd7fb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef806e330 0x7faef80affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34658/0 (socket says 172.21.15.146:34658) 2024-09-13T18:51:00.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faefd7fb700 1 -- 172.21.15.146:0/2435987415 learned_addr learned my addr 172.21.15.146:0/2435987415 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:51:00.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faefdffc700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faef806df50 0x7faef81644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:00.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faefe7fd700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faef80c1440 0x7faef8163520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:00.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faefd7fb700 1 -- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faef80c1440 msgr2=0x7faef8163520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:00.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faefd7fb700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faef80c1440 0x7faef8163520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:00.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faefd7fb700 1 -- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faef806df50 msgr2=0x7faef81644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:00.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faefd7fb700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faef806df50 0x7faef81644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:00.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faefd7fb700 1 -- 172.21.15.146:0/2435987415 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef0009a70 con 0x7faef806e330 2024-09-13T18:51:00.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faefd7fb700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef806e330 0x7faef80affb0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7faee800e9d0 tx=0x7faee800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:00.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faeee7fc700 1 -- 172.21.15.146:0/2435987415 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8009720 con 0x7faef806e330 2024-09-13T18:51:00.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faeee7fc700 1 -- 172.21.15.146:0/2435987415 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee800ee70 con 0x7faef806e330 2024-09-13T18:51:00.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faf00260700 1 -- 172.21.15.146:0/2435987415 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef8162760 con 0x7faef806e330 2024-09-13T18:51:00.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faeee7fc700 1 -- 172.21.15.146:0/2435987415 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee800ca50 con 0x7faef806e330 2024-09-13T18:51:00.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.638+0000 7faf00260700 1 -- 172.21.15.146:0/2435987415 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef8162d20 con 0x7faef806e330 2024-09-13T18:51:00.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.642+0000 7faf00260700 1 -- 172.21.15.146:0/2435987415 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef8070820 con 0x7faef806e330 2024-09-13T18:51:00.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.642+0000 7faeee7fc700 1 -- 172.21.15.146:0/2435987415 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faee800cbb0 con 0x7faef806e330 2024-09-13T18:51:00.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.642+0000 7faeee7fc700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faedc066730 0x7faedc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:00.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.642+0000 7faeee7fc700 1 -- 172.21.15.146:0/2435987415 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7faee80849d0 con 0x7faef806e330 2024-09-13T18:51:00.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.642+0000 7faefdffc700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faedc066730 0x7faedc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:00.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.642+0000 7faefdffc700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faedc066730 0x7faedc068bf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7faef4007fd0 tx=0x7faef4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:00.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.654+0000 7faeee7fc700 1 -- 172.21.15.146:0/2435987415 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faee8051520 con 0x7faef806e330 2024-09-13T18:51:00.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.862+0000 7faf00260700 1 -- 172.21.15.146:0/2435987415 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faef806ee40 con 0x7faef806e330 2024-09-13T18:51:00.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.866+0000 7faeee7fc700 1 -- 172.21.15.146:0/2435987415 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+5524 (secure 0 0 0) 0x7faee8017090 con 0x7faef806e330 2024-09-13T18:51:00.883 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:51:00.883 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:50:56.908526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:resolve","state_seq":110,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:51:00.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.866+0000 7faf00260700 1 -- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faedc066730 msgr2=0x7faedc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:00.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.866+0000 7faf00260700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faedc066730 0x7faedc068bf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7faef4007fd0 tx=0x7faef4009040 comp rx=0 tx=0).stop 2024-09-13T18:51:00.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.866+0000 7faf00260700 1 -- 172.21.15.146:0/2435987415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef806e330 msgr2=0x7faef80affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:00.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.866+0000 7faf00260700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef806e330 0x7faef80affb0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7faee800e9d0 tx=0x7faee800ece0 comp rx=0 tx=0).stop 2024-09-13T18:51:00.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.866+0000 7faf00260700 1 -- 172.21.15.146:0/2435987415 shutdown_connections 2024-09-13T18:51:00.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.866+0000 7faf00260700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faedc066730 0x7faedc068bf0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:00.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.866+0000 7faf00260700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faef80c1440 0x7faef8163520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:00.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.866+0000 7faf00260700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef806e330 0x7faef80affb0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:00.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.866+0000 7faf00260700 1 --2- 172.21.15.146:0/2435987415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faef806df50 0x7faef81644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:00.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.866+0000 7faf00260700 1 -- 172.21.15.146:0/2435987415 >> 172.21.15.146:0/2435987415 conn(0x7faef80cb1d0 msgr2=0x7faef80c74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:00.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.866+0000 7faf00260700 1 -- 172.21.15.146:0/2435987415 shutdown_connections 2024-09-13T18:51:00.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:00.866+0000 7faf00260700 1 -- 172.21.15.146:0/2435987415 wait complete. 2024-09-13T18:51:00.887 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 93 2024-09-13T18:51:00.899 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:51:02.899 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:51:03.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 Processor -- start 2024-09-13T18:51:03.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 -- start start 2024-09-13T18:51:03.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f65ec06df50 0x7f65ec0c0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:03.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65ec0c1440 0x7f65ec0c1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:03.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65ec156bf0 con 0x7f65ec06e330 2024-09-13T18:51:03.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ec0cc020 con 0x7f65ec06df50 2024-09-13T18:51:03.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec0c1dd0 con 0x7f65ec0c1440 2024-09-13T18:51:03.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65eb7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f65ec06df50 0x7f65ec0c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:03.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65eb7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f65ec06df50 0x7f65ec0c0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40094/0 (socket says 172.21.15.146:40094) 2024-09-13T18:51:03.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65eb7fe700 1 -- 172.21.15.146:0/2077084120 learned_addr learned my addr 172.21.15.146:0/2077084120 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:51:03.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65ebfff700 1 --2- 172.21.15.146:0/2077084120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65ec0c1440 0x7f65ec0c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:03.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65eb7fe700 1 -- 172.21.15.146:0/2077084120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65ec0c1440 msgr2=0x7f65ec0c1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:51:03.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65eb7fe700 1 --2- 172.21.15.146:0/2077084120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65ec0c1440 0x7f65ec0c1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:03.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65eb7fe700 1 -- 172.21.15.146:0/2077084120 >> v1:172.21.15.154:6789/0 conn(0x7f65ec06e330 legacy=0x7f65ec06fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:03.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65eb7fe700 1 -- 172.21.15.146:0/2077084120 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec070560 con 0x7f65ec06df50 2024-09-13T18:51:03.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65eb7fe700 1 --2- 172.21.15.146:0/2077084120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ec06df50 0x7f65ec0c0260 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009910 tx=0x7f65e0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c41cda4d758956 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:03.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65ea7fc700 1 -- 172.21.15.146:0/2077084120 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e0011830 con 0x7f65ec06df50 2024-09-13T18:51:03.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65ea7fc700 1 -- 172.21.15.146:0/2077084120 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e0011990 con 0x7f65ec06df50 2024-09-13T18:51:03.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65ea7fc700 1 -- 172.21.15.146:0/2077084120 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e0011d40 con 0x7f65ec06df50 2024-09-13T18:51:03.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 -- 172.21.15.146:0/2077084120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ec06df50 msgr2=0x7f65ec0c0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:03.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 --2- 172.21.15.146:0/2077084120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ec06df50 0x7f65ec0c0260 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009910 tx=0x7f65e0009c20 comp rx=0 tx=0).stop 2024-09-13T18:51:03.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 -- 172.21.15.146:0/2077084120 shutdown_connections 2024-09-13T18:51:03.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 --2- 172.21.15.146:0/2077084120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65ec0c1440 0x7f65ec0c1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:03.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 --2- 172.21.15.146:0/2077084120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ec06df50 0x7f65ec0c0260 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:03.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 -- 172.21.15.146:0/2077084120 >> 172.21.15.146:0/2077084120 conn(0x7f65ec0cb1d0 msgr2=0x7f65ec0cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:03.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 -- 172.21.15.146:0/2077084120 shutdown_connections 2024-09-13T18:51:03.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.010+0000 7f65f2205700 1 -- 172.21.15.146:0/2077084120 wait complete. 2024-09-13T18:51:03.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65f2205700 1 Processor -- start 2024-09-13T18:51:03.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65f2205700 1 -- start start 2024-09-13T18:51:03.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65f2205700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65ec06e330 0x7f65ec1f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:03.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65f2205700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ec0c1440 0x7f65ec162800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:03.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65f2205700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65ec150820 0x7f65ec150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:03.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65f2205700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec1445e0 con 0x7f65ec0c1440 2024-09-13T18:51:03.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65f2205700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec1514f0 con 0x7f65ec06e330 2024-09-13T18:51:03.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65f2205700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec1548b0 con 0x7f65ec150820 2024-09-13T18:51:03.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65eaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ec0c1440 0x7f65ec162800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:03.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65eaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ec0c1440 0x7f65ec162800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40108/0 (socket says 172.21.15.146:40108) 2024-09-13T18:51:03.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65eaffd700 1 -- 172.21.15.146:0/4167751328 learned_addr learned my addr 172.21.15.146:0/4167751328 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:51:03.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65eb7fe700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65ec06e330 0x7f65ec1f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:03.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65ebfff700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65ec150820 0x7f65ec150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:03.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65eaffd700 1 -- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65ec150820 msgr2=0x7f65ec150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:03.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65eaffd700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65ec150820 0x7f65ec150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:03.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65eaffd700 1 -- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65ec06e330 msgr2=0x7f65ec1f0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:03.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65eaffd700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65ec06e330 0x7f65ec1f0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:03.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65eaffd700 1 -- 172.21.15.146:0/4167751328 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e0009570 con 0x7f65ec0c1440 2024-09-13T18:51:03.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65ebfff700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65ec150820 0x7f65ec150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:51:03.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65eaffd700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ec0c1440 0x7f65ec162800 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00c530 tx=0x7f65dc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:03.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65d3fff700 1 -- 172.21.15.146:0/4167751328 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0052d0 con 0x7f65ec0c1440 2024-09-13T18:51:03.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65d3fff700 1 -- 172.21.15.146:0/4167751328 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc005430 con 0x7f65ec0c1440 2024-09-13T18:51:03.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65f2205700 1 -- 172.21.15.146:0/4167751328 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ec151710 con 0x7f65ec0c1440 2024-09-13T18:51:03.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65d3fff700 1 -- 172.21.15.146:0/4167751328 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0055a0 con 0x7f65ec0c1440 2024-09-13T18:51:03.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.014+0000 7f65f2205700 1 -- 172.21.15.146:0/4167751328 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec151cd0 con 0x7f65ec0c1440 2024-09-13T18:51:03.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.018+0000 7f65f2205700 1 -- 172.21.15.146:0/4167751328 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ec070820 con 0x7f65ec0c1440 2024-09-13T18:51:03.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.018+0000 7f65d3fff700 1 -- 172.21.15.146:0/4167751328 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f65dc0038c0 con 0x7f65ec0c1440 2024-09-13T18:51:03.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.018+0000 7f65d3fff700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f65d8066730 0x7f65d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:03.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.018+0000 7f65d3fff700 1 -- 172.21.15.146:0/4167751328 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7f65dc083ea0 con 0x7f65ec0c1440 2024-09-13T18:51:03.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.018+0000 7f65eb7fe700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f65d8066730 0x7f65d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:03.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.018+0000 7f65eb7fe700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f65d8066730 0x7f65d8068bf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0675a0 tx=0x7f65e0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:03.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.026+0000 7f65d3fff700 1 -- 172.21.15.146:0/4167751328 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f65dc0508e0 con 0x7f65ec0c1440 2024-09-13T18:51:03.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.234+0000 7f65f2205700 1 -- 172.21.15.146:0/4167751328 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65ec06ee40 con 0x7f65ec0c1440 2024-09-13T18:51:03.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.234+0000 7f65d3fff700 1 -- 172.21.15.146:0/4167751328 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+5526 (secure 0 0 0) 0x7f65dc017020 con 0x7f65ec0c1440 2024-09-13T18:51:03.252 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:51:03.253 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:51:01.935418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:reconnect","state_seq":112,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:51:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.238+0000 7f65f2205700 1 -- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f65d8066730 msgr2=0x7f65d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.238+0000 7f65f2205700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f65d8066730 0x7f65d8068bf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0675a0 tx=0x7f65e0008030 comp rx=0 tx=0).stop 2024-09-13T18:51:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.238+0000 7f65f2205700 1 -- 172.21.15.146:0/4167751328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ec0c1440 msgr2=0x7f65ec162800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.238+0000 7f65f2205700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ec0c1440 0x7f65ec162800 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00c530 tx=0x7f65dc00c840 comp rx=0 tx=0).stop 2024-09-13T18:51:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.238+0000 7f65f2205700 1 -- 172.21.15.146:0/4167751328 shutdown_connections 2024-09-13T18:51:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.238+0000 7f65f2205700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f65d8066730 0x7f65d8068bf0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.238+0000 7f65f2205700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65ec150820 0x7f65ec150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.238+0000 7f65f2205700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ec0c1440 0x7f65ec162800 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.238+0000 7f65f2205700 1 --2- 172.21.15.146:0/4167751328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65ec06e330 0x7f65ec1f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.238+0000 7f65f2205700 1 -- 172.21.15.146:0/4167751328 >> 172.21.15.146:0/4167751328 conn(0x7f65ec0cb1d0 msgr2=0x7f65ec0ad560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.238+0000 7f65f2205700 1 -- 172.21.15.146:0/4167751328 shutdown_connections 2024-09-13T18:51:03.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:03.238+0000 7f65f2205700 1 -- 172.21.15.146:0/4167751328 wait complete. 2024-09-13T18:51:03.256 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 94 2024-09-13T18:51:03.269 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 94, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 94, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6187": { "addr": "172.21.15.146:6833/3023779168", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 3023779168, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 3023779168, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6187, "incarnation": 92, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:reconnect", "state_seq": 112 }, "gid_6714": { "addr": "172.21.15.146:6837/2169954140", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 2169954140, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 2169954140, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6714, "incarnation": 86, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 38 } }, "last_failure": 0, "last_failure_osd_epoch": 40, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:51:01.935418+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6187, "mds_1": 6714 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6839/2053734262", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 2053734262, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2053734262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 91, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6032, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:51:05.270 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:51:05.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.374+0000 7f216480d700 1 Processor -- start 2024-09-13T18:51:05.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.374+0000 7f216480d700 1 -- start start 2024-09-13T18:51:05.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f216480d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f21600caed0 0x7f21600723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:05.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f216480d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21600a9840 0x7f216015e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:05.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f216480d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2160161550 con 0x7f21600cb2b0 2024-09-13T18:51:05.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f216480d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2160072a30 con 0x7f21600caed0 2024-09-13T18:51:05.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f216480d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21600aa800 con 0x7f21600a9840 2024-09-13T18:51:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f215ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21600a9840 0x7f216015e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f215f7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f21600caed0 0x7f21600723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f215ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21600a9840 0x7f216015e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:59312/0 (socket says 172.21.15.146:59312) 2024-09-13T18:51:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f215ffff700 1 -- 172.21.15.146:0/2961284084 learned_addr learned my addr 172.21.15.146:0/2961284084 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:51:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f215f7fe700 1 --2- 172.21.15.146:0/2961284084 >> v2:172.21.15.154:3300/0 conn(0x7f21600caed0 0x7f21600723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:40436/0 (socket says 172.21.15.146:40436) 2024-09-13T18:51:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f215ffff700 1 -- 172.21.15.146:0/2961284084 >> v2:172.21.15.154:3300/0 conn(0x7f21600caed0 msgr2=0x7f21600723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:51:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f215ffff700 1 --2- 172.21.15.146:0/2961284084 >> v2:172.21.15.154:3300/0 conn(0x7f21600caed0 0x7f21600723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f215ffff700 1 -- 172.21.15.146:0/2961284084 >> v1:172.21.15.154:6789/0 conn(0x7f21600cb2b0 legacy=0x7f21601619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:05.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f215ffff700 1 -- 172.21.15.146:0/2961284084 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2160161220 con 0x7f21600a9840 2024-09-13T18:51:05.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f215ffff700 1 --2- 172.21.15.146:0/2961284084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21600a9840 0x7f216015e100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2158009910 tx=0x7f2158009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efba3dad996964b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:05.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f215e7fc700 1 -- 172.21.15.146:0/2961284084 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158011830 con 0x7f21600a9840 2024-09-13T18:51:05.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f215e7fc700 1 -- 172.21.15.146:0/2961284084 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2158011990 con 0x7f21600a9840 2024-09-13T18:51:05.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f215e7fc700 1 -- 172.21.15.146:0/2961284084 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158011d40 con 0x7f21600a9840 2024-09-13T18:51:05.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f216480d700 1 -- 172.21.15.146:0/2961284084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21600a9840 msgr2=0x7f216015e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:05.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f216480d700 1 --2- 172.21.15.146:0/2961284084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21600a9840 0x7f216015e100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2158009910 tx=0x7f2158009c20 comp rx=0 tx=0).stop 2024-09-13T18:51:05.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f216480d700 1 -- 172.21.15.146:0/2961284084 shutdown_connections 2024-09-13T18:51:05.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f216480d700 1 --2- 172.21.15.146:0/2961284084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21600a9840 0x7f216015e100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:05.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f216480d700 1 --2- 172.21.15.146:0/2961284084 >> v2:172.21.15.154:3300/0 conn(0x7f21600caed0 0x7f21600723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:05.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f216480d700 1 -- 172.21.15.146:0/2961284084 >> 172.21.15.146:0/2961284084 conn(0x7f2160066090 msgr2=0x7f2160066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:05.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f216480d700 1 -- 172.21.15.146:0/2961284084 shutdown_connections 2024-09-13T18:51:05.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.378+0000 7f216480d700 1 -- 172.21.15.146:0/2961284084 wait complete. 2024-09-13T18:51:05.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f216480d700 1 Processor -- start 2024-09-13T18:51:05.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f216480d700 1 -- start start 2024-09-13T18:51:05.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f216480d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21600a9840 0x7f21600af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:05.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f216480d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21600caed0 0x7f21600c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:05.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f216480d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21600cb2b0 0x7f21600ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:05.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f216480d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601249d0 con 0x7f21600a9840 2024-09-13T18:51:05.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f216480d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21600aff20 con 0x7f21600caed0 2024-09-13T18:51:05.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f216480d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21600af960 con 0x7f21600cb2b0 2024-09-13T18:51:05.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f215f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21600a9840 0x7f21600af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:05.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f215f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21600a9840 0x7f21600af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40122/0 (socket says 172.21.15.146:40122) 2024-09-13T18:51:05.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f215ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21600cb2b0 0x7f21600ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:05.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f215f7fe700 1 -- 172.21.15.146:0/2389304460 learned_addr learned my addr 172.21.15.146:0/2389304460 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:51:05.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f215effd700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21600caed0 0x7f21600c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:05.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f215f7fe700 1 -- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21600cb2b0 msgr2=0x7f21600ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:05.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f215f7fe700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21600cb2b0 0x7f21600ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:05.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f215f7fe700 1 -- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21600caed0 msgr2=0x7f21600c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:05.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f215f7fe700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21600caed0 0x7f21600c5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:05.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f215f7fe700 1 -- 172.21.15.146:0/2389304460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2158009570 con 0x7f21600a9840 2024-09-13T18:51:05.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f215effd700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21600caed0 0x7f21600c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:51:05.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f215f7fe700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21600a9840 0x7f21600af2b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f215000efc0 tx=0x7f215000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:05.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f2157fff700 1 -- 172.21.15.146:0/2389304460 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215000cb50 con 0x7f21600a9840 2024-09-13T18:51:05.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f2157fff700 1 -- 172.21.15.146:0/2389304460 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215000f040 con 0x7f21600a9840 2024-09-13T18:51:05.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f216480d700 1 -- 172.21.15.146:0/2389304460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21600afc40 con 0x7f21600a9840 2024-09-13T18:51:05.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.382+0000 7f2157fff700 1 -- 172.21.15.146:0/2389304460 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2150014480 con 0x7f21600a9840 2024-09-13T18:51:05.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.386+0000 7f216480d700 1 -- 172.21.15.146:0/2389304460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21601f4e30 con 0x7f21600a9840 2024-09-13T18:51:05.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.386+0000 7f216480d700 1 -- 172.21.15.146:0/2389304460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216006f5c0 con 0x7f21600a9840 2024-09-13T18:51:05.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.386+0000 7f2157fff700 1 -- 172.21.15.146:0/2389304460 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f21500145e0 con 0x7f21600a9840 2024-09-13T18:51:05.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.386+0000 7f2157fff700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2140066730 0x7f2140068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:05.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.386+0000 7f2157fff700 1 -- 172.21.15.146:0/2389304460 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7f2150085220 con 0x7f21600a9840 2024-09-13T18:51:05.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.386+0000 7f215effd700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2140066730 0x7f2140068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:05.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.390+0000 7f215effd700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2140066730 0x7f2140068bf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f216015a0e0 tx=0x7f2148009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:05.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.398+0000 7f2157fff700 1 -- 172.21.15.146:0/2389304460 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2150051ce0 con 0x7f21600a9840 2024-09-13T18:51:05.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.610+0000 7f216480d700 1 -- 172.21.15.146:0/2389304460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f216006e8c0 con 0x7f21600a9840 2024-09-13T18:51:05.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.610+0000 7f2157fff700 1 -- 172.21.15.146:0/2389304460 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+5526 (secure 0 0 0) 0x7f2150055300 con 0x7f21600a9840 2024-09-13T18:51:05.630 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:51:05.631 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:51:01.935418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:reconnect","state_seq":112,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:51:05.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.614+0000 7f216480d700 1 -- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2140066730 msgr2=0x7f2140068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:05.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.614+0000 7f216480d700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2140066730 0x7f2140068bf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f216015a0e0 tx=0x7f2148009040 comp rx=0 tx=0).stop 2024-09-13T18:51:05.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.614+0000 7f216480d700 1 -- 172.21.15.146:0/2389304460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21600a9840 msgr2=0x7f21600af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:05.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.614+0000 7f216480d700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21600a9840 0x7f21600af2b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f215000efc0 tx=0x7f215000c0c0 comp rx=0 tx=0).stop 2024-09-13T18:51:05.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.614+0000 7f216480d700 1 -- 172.21.15.146:0/2389304460 shutdown_connections 2024-09-13T18:51:05.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.614+0000 7f216480d700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2140066730 0x7f2140068bf0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:05.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.614+0000 7f216480d700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21600cb2b0 0x7f21600ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:05.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.614+0000 7f216480d700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21600caed0 0x7f21600c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:05.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.614+0000 7f216480d700 1 --2- 172.21.15.146:0/2389304460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21600a9840 0x7f21600af2b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:05.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.614+0000 7f216480d700 1 -- 172.21.15.146:0/2389304460 >> 172.21.15.146:0/2389304460 conn(0x7f2160066090 msgr2=0x7f216006fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:05.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.614+0000 7f216480d700 1 -- 172.21.15.146:0/2389304460 shutdown_connections 2024-09-13T18:51:05.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:05.614+0000 7f216480d700 1 -- 172.21.15.146:0/2389304460 wait complete. 2024-09-13T18:51:05.635 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 94 2024-09-13T18:51:05.646 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:51:07.647 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:51:07.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.750+0000 7f56c991d700 1 Processor -- start 2024-09-13T18:51:07.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.750+0000 7f56c991d700 1 -- start start 2024-09-13T18:51:07.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c991d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56c4072580 0x7f56c4152740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:07.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c991d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c40c9510 con 0x7f56c40cb2b0 2024-09-13T18:51:07.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c991d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c406efa0 con 0x7f56c40caed0 2024-09-13T18:51:07.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c991d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c406f280 con 0x7f56c4072580 2024-09-13T18:51:07.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c911c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f56c40cb2b0 0x7f56c4155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58508/0 (socket says 172.21.15.146:58508) 2024-09-13T18:51:07.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c911c700 1 -- 172.21.15.146:0/3444001412 learned_addr learned my addr 172.21.15.146:0/3444001412 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:51:07.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c3fff700 1 --2- 172.21.15.146:0/3444001412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56c4072580 0x7f56c4152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:07.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c37fe700 1 -- 172.21.15.146:0/3444001412 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132195156 0 0) 0x7f56c40c9510 con 0x7f56c40cb2b0 2024-09-13T18:51:07.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c37fe700 1 -- 172.21.15.146:0/3444001412 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56b00035d0 con 0x7f56c40cb2b0 2024-09-13T18:51:07.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c3fff700 1 -- 172.21.15.146:0/3444001412 >> v1:172.21.15.154:6789/0 conn(0x7f56c40caed0 legacy=0x7f56c406f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:07.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c3fff700 1 -- 172.21.15.146:0/3444001412 >> v1:172.21.15.146:6789/0 conn(0x7f56c40cb2b0 legacy=0x7f56c4155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:07.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c3fff700 1 -- 172.21.15.146:0/3444001412 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c40c8c70 con 0x7f56c4072580 2024-09-13T18:51:07.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c3fff700 1 --2- 172.21.15.146:0/3444001412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56c4072580 0x7f56c4152740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009dc0 tx=0x7f56b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d196345c72391ad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:07.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c37fe700 1 -- 172.21.15.146:0/3444001412 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b400e070 con 0x7f56c4072580 2024-09-13T18:51:07.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c37fe700 1 -- 172.21.15.146:0/3444001412 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b400bb20 con 0x7f56c4072580 2024-09-13T18:51:07.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c37fe700 1 -- 172.21.15.146:0/3444001412 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4012430 con 0x7f56c4072580 2024-09-13T18:51:07.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c991d700 1 -- 172.21.15.146:0/3444001412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56c4072580 msgr2=0x7f56c4152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:07.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c991d700 1 --2- 172.21.15.146:0/3444001412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56c4072580 0x7f56c4152740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009dc0 tx=0x7f56b400b330 comp rx=0 tx=0).stop 2024-09-13T18:51:07.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c991d700 1 -- 172.21.15.146:0/3444001412 shutdown_connections 2024-09-13T18:51:07.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c991d700 1 --2- 172.21.15.146:0/3444001412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56c4072580 0x7f56c4152740 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:07.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c991d700 1 -- 172.21.15.146:0/3444001412 >> 172.21.15.146:0/3444001412 conn(0x7f56c4066090 msgr2=0x7f56c4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:07.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c991d700 1 -- 172.21.15.146:0/3444001412 shutdown_connections 2024-09-13T18:51:07.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.754+0000 7f56c991d700 1 -- 172.21.15.146:0/3444001412 wait complete. 2024-09-13T18:51:07.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c991d700 1 Processor -- start 2024-09-13T18:51:07.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c991d700 1 -- start start 2024-09-13T18:51:07.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c991d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56c4072580 0x7f56c41eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:07.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c991d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56c40caed0 0x7f56c41ed720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:07.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c991d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56c40cb2b0 0x7f56c414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:07.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c991d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c41ec530 con 0x7f56c40cb2b0 2024-09-13T18:51:07.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c991d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c40aac40 con 0x7f56c40caed0 2024-09-13T18:51:07.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c991d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c41ef100 con 0x7f56c4072580 2024-09-13T18:51:07.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c911c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56c40cb2b0 0x7f56c414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:07.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c891b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56c4072580 0x7f56c41eebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:07.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56c40caed0 0x7f56c41ed720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:07.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c911c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56c40cb2b0 0x7f56c414e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40134/0 (socket says 172.21.15.146:40134) 2024-09-13T18:51:07.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c911c700 1 -- 172.21.15.146:0/2620637743 learned_addr learned my addr 172.21.15.146:0/2620637743 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:51:07.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c911c700 1 -- 172.21.15.146:0/2620637743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56c4072580 msgr2=0x7f56c41eebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:07.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c911c700 1 --2- 172.21.15.146:0/2620637743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56c4072580 0x7f56c41eebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:07.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c911c700 1 -- 172.21.15.146:0/2620637743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56c40caed0 msgr2=0x7f56c41ed720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:07.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c911c700 1 --2- 172.21.15.146:0/2620637743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56c40caed0 0x7f56c41ed720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:07.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c911c700 1 -- 172.21.15.146:0/2620637743 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b4009a70 con 0x7f56c40cb2b0 2024-09-13T18:51:07.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c911c700 1 --2- 172.21.15.146:0/2620637743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56c40cb2b0 0x7f56c414e690 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c430 tx=0x7f56bc00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:07.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c17fa700 1 -- 172.21.15.146:0/2620637743 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc005b20 con 0x7f56c40cb2b0 2024-09-13T18:51:07.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c17fa700 1 -- 172.21.15.146:0/2620637743 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc005ca0 con 0x7f56c40cb2b0 2024-09-13T18:51:07.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c991d700 1 -- 172.21.15.146:0/2620637743 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c41563a0 con 0x7f56c40cb2b0 2024-09-13T18:51:07.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c17fa700 1 -- 172.21.15.146:0/2620637743 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc007310 con 0x7f56c40cb2b0 2024-09-13T18:51:07.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.758+0000 7f56c991d700 1 -- 172.21.15.146:0/2620637743 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c4156660 con 0x7f56c40cb2b0 2024-09-13T18:51:07.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.762+0000 7f56c991d700 1 -- 172.21.15.146:0/2620637743 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c40ca080 con 0x7f56c40cb2b0 2024-09-13T18:51:07.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.762+0000 7f56c17fa700 1 -- 172.21.15.146:0/2620637743 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f56bc007470 con 0x7f56c40cb2b0 2024-09-13T18:51:07.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.762+0000 7f56c17fa700 1 --2- 172.21.15.146:0/2620637743 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56a4066730 0x7f56a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:07.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.762+0000 7f56c17fa700 1 -- 172.21.15.146:0/2620637743 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7f56bc083db0 con 0x7f56c40cb2b0 2024-09-13T18:51:07.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.762+0000 7f56c891b700 1 --2- 172.21.15.146:0/2620637743 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56a4066730 0x7f56a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:07.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.762+0000 7f56c891b700 1 --2- 172.21.15.146:0/2620637743 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56a4066730 0x7f56a4068bf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f56ac000f60 tx=0x7f56ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:07.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.770+0000 7f56c17fa700 1 -- 172.21.15.146:0/2620637743 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f56bc088070 con 0x7f56c40cb2b0 2024-09-13T18:51:08.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.986+0000 7f56c991d700 1 -- 172.21.15.146:0/2620637743 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56c41ec670 con 0x7f56c40cb2b0 2024-09-13T18:51:08.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.986+0000 7f56c17fa700 1 -- 172.21.15.146:0/2620637743 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+5523 (secure 0 0 0) 0x7f56bc017070 con 0x7f56c40cb2b0 2024-09-13T18:51:08.005 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:51:08.005 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":95,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:51:06.964427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:rejoin","state_seq":114,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:51:08.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.990+0000 7f56c991d700 1 -- 172.21.15.146:0/2620637743 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56a4066730 msgr2=0x7f56a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:08.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.990+0000 7f56c991d700 1 --2- 172.21.15.146:0/2620637743 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56a4066730 0x7f56a4068bf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f56ac000f60 tx=0x7f56ac009040 comp rx=0 tx=0).stop 2024-09-13T18:51:08.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.990+0000 7f56c991d700 1 -- 172.21.15.146:0/2620637743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56c40cb2b0 msgr2=0x7f56c414e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:08.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.990+0000 7f56c991d700 1 --2- 172.21.15.146:0/2620637743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56c40cb2b0 0x7f56c414e690 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c430 tx=0x7f56bc00c740 comp rx=0 tx=0).stop 2024-09-13T18:51:08.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.990+0000 7f56c991d700 1 -- 172.21.15.146:0/2620637743 shutdown_connections 2024-09-13T18:51:08.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.990+0000 7f56c991d700 1 --2- 172.21.15.146:0/2620637743 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56a4066730 0x7f56a4068bf0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:08.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.990+0000 7f56c991d700 1 --2- 172.21.15.146:0/2620637743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56c40cb2b0 0x7f56c414e690 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:08.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.990+0000 7f56c991d700 1 --2- 172.21.15.146:0/2620637743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56c40caed0 0x7f56c41ed720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:08.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.990+0000 7f56c991d700 1 --2- 172.21.15.146:0/2620637743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56c4072580 0x7f56c41eebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:08.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.990+0000 7f56c991d700 1 -- 172.21.15.146:0/2620637743 >> 172.21.15.146:0/2620637743 conn(0x7f56c4066090 msgr2=0x7f56c4152510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:08.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.990+0000 7f56c991d700 1 -- 172.21.15.146:0/2620637743 shutdown_connections 2024-09-13T18:51:08.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:07.990+0000 7f56c991d700 1 -- 172.21.15.146:0/2620637743 wait complete. 2024-09-13T18:51:08.009 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 95 2024-09-13T18:51:08.021 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:51:10.022 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:51:10.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.126+0000 7f2c324ec700 1 Processor -- start 2024-09-13T18:51:10.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.126+0000 7f2c324ec700 1 -- start start 2024-09-13T18:51:10.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c324ec700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2c2c0ca210 0x7f2c2c0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:10.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c324ec700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2c2c071390 0x7f2c2c0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:10.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c324ec700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2c0cc670 con 0x7f2c2c0cbd50 2024-09-13T18:51:10.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c324ec700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2c0caa30 con 0x7f2c2c071390 2024-09-13T18:51:10.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c324ec700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2c0cb810 con 0x7f2c2c0ca210 2024-09-13T18:51:10.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c31ceb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2c2c071390 0x7f2c2c0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:10.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c314ea700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f2c2c0cbd50 0x7f2c2c0701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58516/0 (socket says 172.21.15.146:58516) 2024-09-13T18:51:10.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c314ea700 1 -- 172.21.15.146:0/556160581 learned_addr learned my addr 172.21.15.146:0/556160581 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:51:10.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c30ce9700 1 --2- 172.21.15.146:0/556160581 >> v2:172.21.15.154:3300/0 conn(0x7f2c2c0ca210 0x7f2c2c0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:10.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c31ceb700 1 -- 172.21.15.146:0/556160581 >> v2:172.21.15.154:3300/0 conn(0x7f2c2c0ca210 msgr2=0x7f2c2c0b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:51:10.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c31ceb700 1 --2- 172.21.15.146:0/556160581 >> v2:172.21.15.154:3300/0 conn(0x7f2c2c0ca210 0x7f2c2c0b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:10.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c31ceb700 1 -- 172.21.15.146:0/556160581 >> v1:172.21.15.146:6789/0 conn(0x7f2c2c0cbd50 legacy=0x7f2c2c0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:10.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c31ceb700 1 -- 172.21.15.146:0/556160581 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c15da20 con 0x7f2c2c071390 2024-09-13T18:51:10.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c31ceb700 1 --2- 172.21.15.146:0/556160581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c071390 0x7f2c2c0cc130 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0b18a0 tx=0x7f2c24009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2d3bb756a39d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:10.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c2bfff700 1 -- 172.21.15.146:0/556160581 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24011830 con 0x7f2c2c071390 2024-09-13T18:51:10.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c2bfff700 1 -- 172.21.15.146:0/556160581 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c24011990 con 0x7f2c2c071390 2024-09-13T18:51:10.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c2bfff700 1 -- 172.21.15.146:0/556160581 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24011d40 con 0x7f2c2c071390 2024-09-13T18:51:10.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c324ec700 1 -- 172.21.15.146:0/556160581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c071390 msgr2=0x7f2c2c0cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:10.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c324ec700 1 --2- 172.21.15.146:0/556160581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c071390 0x7f2c2c0cc130 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0b18a0 tx=0x7f2c24009b30 comp rx=0 tx=0).stop 2024-09-13T18:51:10.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c324ec700 1 -- 172.21.15.146:0/556160581 shutdown_connections 2024-09-13T18:51:10.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c324ec700 1 --2- 172.21.15.146:0/556160581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c071390 0x7f2c2c0cc130 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:10.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c324ec700 1 --2- 172.21.15.146:0/556160581 >> v2:172.21.15.154:3300/0 conn(0x7f2c2c0ca210 0x7f2c2c0b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:10.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c324ec700 1 -- 172.21.15.146:0/556160581 >> 172.21.15.146:0/556160581 conn(0x7f2c2c066270 msgr2=0x7f2c2c066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:10.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c324ec700 1 -- 172.21.15.146:0/556160581 shutdown_connections 2024-09-13T18:51:10.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.130+0000 7f2c324ec700 1 -- 172.21.15.146:0/556160581 wait complete. 2024-09-13T18:51:10.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c324ec700 1 Processor -- start 2024-09-13T18:51:10.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c324ec700 1 -- start start 2024-09-13T18:51:10.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c324ec700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2c2c071390 0x7f2c2c1f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:10.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c324ec700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c0ca210 0x7f2c2c1effb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:10.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c324ec700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c2c0cbd50 0x7f2c2c1505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:10.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c324ec700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c15a530 con 0x7f2c2c0ca210 2024-09-13T18:51:10.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c324ec700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c1f3ad0 con 0x7f2c2c0cbd50 2024-09-13T18:51:10.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c324ec700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c15c650 con 0x7f2c2c071390 2024-09-13T18:51:10.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c30ce9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c0ca210 0x7f2c2c1effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:10.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c30ce9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c0ca210 0x7f2c2c1effb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40144/0 (socket says 172.21.15.146:40144) 2024-09-13T18:51:10.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c30ce9700 1 -- 172.21.15.146:0/3194320159 learned_addr learned my addr 172.21.15.146:0/3194320159 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:51:10.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c314ea700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2c2c071390 0x7f2c2c1f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:10.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c31ceb700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c2c0cbd50 0x7f2c2c1505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:10.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c30ce9700 1 -- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2c2c071390 msgr2=0x7f2c2c1f3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:10.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c30ce9700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2c2c071390 0x7f2c2c1f3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:10.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c30ce9700 1 -- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c2c0cbd50 msgr2=0x7f2c2c1505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:10.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c30ce9700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c2c0cbd50 0x7f2c2c1505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:10.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c30ce9700 1 -- 172.21.15.146:0/3194320159 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c240094b0 con 0x7f2c2c0ca210 2024-09-13T18:51:10.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c30ce9700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c0ca210 0x7f2c2c1effb0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2c20007ec0 tx=0x7f2c2000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:10.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c29ffb700 1 -- 172.21.15.146:0/3194320159 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2000eea0 con 0x7f2c2c0ca210 2024-09-13T18:51:10.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c29ffb700 1 -- 172.21.15.146:0/3194320159 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c20014420 con 0x7f2c2c0ca210 2024-09-13T18:51:10.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c324ec700 1 -- 172.21.15.146:0/3194320159 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c1efb20 con 0x7f2c2c0ca210 2024-09-13T18:51:10.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c29ffb700 1 -- 172.21.15.146:0/3194320159 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200147b0 con 0x7f2c2c0ca210 2024-09-13T18:51:10.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.134+0000 7f2c324ec700 1 -- 172.21.15.146:0/3194320159 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c15c090 con 0x7f2c2c0ca210 2024-09-13T18:51:10.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.138+0000 7f2c324ec700 1 -- 172.21.15.146:0/3194320159 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2c1efcb0 con 0x7f2c2c0ca210 2024-09-13T18:51:10.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.138+0000 7f2c29ffb700 1 -- 172.21.15.146:0/3194320159 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2c20014910 con 0x7f2c2c0ca210 2024-09-13T18:51:10.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.138+0000 7f2c29ffb700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2c18066760 0x7f2c18068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:10.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.138+0000 7f2c29ffb700 1 -- 172.21.15.146:0/3194320159 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7f2c200852e0 con 0x7f2c2c0ca210 2024-09-13T18:51:10.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.138+0000 7f2c314ea700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2c18066760 0x7f2c18068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:10.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.138+0000 7f2c314ea700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2c18066760 0x7f2c18068c20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c000f60 tx=0x7f2c1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:10.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.146+0000 7f2c29ffb700 1 -- 172.21.15.146:0/3194320159 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2c20051d20 con 0x7f2c2c0ca210 2024-09-13T18:51:10.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.354+0000 7f2c324ec700 1 -- 172.21.15.146:0/3194320159 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c2c15d500 con 0x7f2c2c0ca210 2024-09-13T18:51:10.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.358+0000 7f2c29ffb700 1 -- 172.21.15.146:0/3194320159 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+5523 (secure 0 0 0) 0x7f2c20055340 con 0x7f2c2c0ca210 2024-09-13T18:51:10.375 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:51:10.375 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":95,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:51:06.964427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:rejoin","state_seq":114,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:51:10.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.358+0000 7f2c324ec700 1 -- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2c18066760 msgr2=0x7f2c18068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:10.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.358+0000 7f2c324ec700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2c18066760 0x7f2c18068c20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c000f60 tx=0x7f2c1c009040 comp rx=0 tx=0).stop 2024-09-13T18:51:10.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.358+0000 7f2c324ec700 1 -- 172.21.15.146:0/3194320159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c0ca210 msgr2=0x7f2c2c1effb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:10.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.358+0000 7f2c324ec700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c0ca210 0x7f2c2c1effb0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2c20007ec0 tx=0x7f2c2000e3c0 comp rx=0 tx=0).stop 2024-09-13T18:51:10.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.358+0000 7f2c324ec700 1 -- 172.21.15.146:0/3194320159 shutdown_connections 2024-09-13T18:51:10.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.358+0000 7f2c324ec700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2c18066760 0x7f2c18068c20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:10.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.358+0000 7f2c324ec700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c2c0cbd50 0x7f2c2c1505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:10.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.358+0000 7f2c324ec700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c0ca210 0x7f2c2c1effb0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:10.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.358+0000 7f2c324ec700 1 --2- 172.21.15.146:0/3194320159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2c2c071390 0x7f2c2c1f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:10.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.358+0000 7f2c324ec700 1 -- 172.21.15.146:0/3194320159 >> 172.21.15.146:0/3194320159 conn(0x7f2c2c066270 msgr2=0x7f2c2c149c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:10.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.358+0000 7f2c324ec700 1 -- 172.21.15.146:0/3194320159 shutdown_connections 2024-09-13T18:51:10.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:10.358+0000 7f2c324ec700 1 -- 172.21.15.146:0/3194320159 wait complete. 2024-09-13T18:51:10.378 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 95 2024-09-13T18:51:10.390 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:51:12.383 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:51:12.388 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:51:12.391 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:51:12.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 Processor -- start 2024-09-13T18:51:12.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 -- start start 2024-09-13T18:51:12.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd2d0157e10 0x7fd2d00c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:12.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd2d006fe60 0x7fd2d0070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:12.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d00adfe0 con 0x7fd2d00c5bd0 2024-09-13T18:51:12.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d015e690 con 0x7fd2d0157e10 2024-09-13T18:51:12.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d00c7570 con 0x7fd2d006fe60 2024-09-13T18:51:12.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2cf7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd2d0157e10 0x7fd2d00c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:12.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2cf7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd2d0157e10 0x7fd2d00c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49924/0 (socket says 172.21.15.146:49924) 2024-09-13T18:51:12.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2cf7fe700 1 -- 172.21.15.146:0/1771570635 learned_addr learned my addr 172.21.15.146:0/1771570635 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:51:12.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2ceffd700 1 --2- 172.21.15.146:0/1771570635 >> v2:172.21.15.154:3300/0 conn(0x7fd2d006fe60 0x7fd2d0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:12.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2cf7fe700 1 -- 172.21.15.146:0/1771570635 >> v2:172.21.15.154:3300/0 conn(0x7fd2d006fe60 msgr2=0x7fd2d0070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:51:12.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2cf7fe700 1 --2- 172.21.15.146:0/1771570635 >> v2:172.21.15.154:3300/0 conn(0x7fd2d006fe60 0x7fd2d0070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:12.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2cf7fe700 1 -- 172.21.15.146:0/1771570635 >> v1:172.21.15.154:6789/0 conn(0x7fd2d00c5bd0 legacy=0x7fd2d00c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:12.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2cf7fe700 1 -- 172.21.15.146:0/1771570635 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d015eb80 con 0x7fd2d0157e10 2024-09-13T18:51:12.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2cf7fe700 1 --2- 172.21.15.146:0/1771570635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2d0157e10 0x7fd2d00c6df0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd2d015b820 tx=0x7fd2c4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d912de13671b59b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:12.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2ce7fc700 1 -- 172.21.15.146:0/1771570635 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4011830 con 0x7fd2d0157e10 2024-09-13T18:51:12.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2ce7fc700 1 -- 172.21.15.146:0/1771570635 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c4011990 con 0x7fd2d0157e10 2024-09-13T18:51:12.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2ce7fc700 1 -- 172.21.15.146:0/1771570635 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4011d20 con 0x7fd2d0157e10 2024-09-13T18:51:12.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1771570635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2d0157e10 msgr2=0x7fd2d00c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:12.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 --2- 172.21.15.146:0/1771570635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2d0157e10 0x7fd2d00c6df0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd2d015b820 tx=0x7fd2c4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:51:12.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1771570635 shutdown_connections 2024-09-13T18:51:12.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 --2- 172.21.15.146:0/1771570635 >> v2:172.21.15.154:3300/0 conn(0x7fd2d006fe60 0x7fd2d0070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:12.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 --2- 172.21.15.146:0/1771570635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2d0157e10 0x7fd2d00c6df0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:12.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1771570635 >> 172.21.15.146:0/1771570635 conn(0x7fd2d00cb1d0 msgr2=0x7fd2d00cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:12.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1771570635 shutdown_connections 2024-09-13T18:51:12.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.494+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1771570635 wait complete. 2024-09-13T18:51:12.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.498+0000 7fd2d5b5f700 1 Processor -- start 2024-09-13T18:51:12.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.498+0000 7fd2d5b5f700 1 -- start start 2024-09-13T18:51:12.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.498+0000 7fd2d5b5f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2d006fe60 0x7fd2d0161910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:12.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.498+0000 7fd2d5b5f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2d00c5bd0 0x7fd2d01f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:12.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.498+0000 7fd2d5b5f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2d0157e10 0x7fd2d01f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:12.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.498+0000 7fd2d5b5f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d01f3690 con 0x7fd2d006fe60 2024-09-13T18:51:12.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.498+0000 7fd2d5b5f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d015fa90 con 0x7fd2d00c5bd0 2024-09-13T18:51:12.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.498+0000 7fd2d5b5f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d015ef50 con 0x7fd2d0157e10 2024-09-13T18:51:12.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.498+0000 7fd2cf7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2d006fe60 0x7fd2d0161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:12.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.498+0000 7fd2cf7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2d006fe60 0x7fd2d0161910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49936/0 (socket says 172.21.15.146:49936) 2024-09-13T18:51:12.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.498+0000 7fd2cf7fe700 1 -- 172.21.15.146:0/1968818762 learned_addr learned my addr 172.21.15.146:0/1968818762 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:51:12.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.498+0000 7fd2ceffd700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2d00c5bd0 0x7fd2d01f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:12.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.498+0000 7fd2cffff700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2d0157e10 0x7fd2d01f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:12.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.502+0000 7fd2cf7fe700 1 -- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2d0157e10 msgr2=0x7fd2d01f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:12.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.502+0000 7fd2cf7fe700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2d0157e10 0x7fd2d01f7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:12.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.502+0000 7fd2cf7fe700 1 -- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2d00c5bd0 msgr2=0x7fd2d01f7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:12.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.502+0000 7fd2cf7fe700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2d00c5bd0 0x7fd2d01f7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:12.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.502+0000 7fd2cf7fe700 1 -- 172.21.15.146:0/1968818762 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c4009570 con 0x7fd2d006fe60 2024-09-13T18:51:12.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.502+0000 7fd2cf7fe700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2d006fe60 0x7fd2d0161910 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00675d0 tx=0x7fd2c4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:12.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.502+0000 7fd2b7fff700 1 -- 172.21.15.146:0/1968818762 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4004450 con 0x7fd2d006fe60 2024-09-13T18:51:12.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.502+0000 7fd2b7fff700 1 -- 172.21.15.146:0/1968818762 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c401f070 con 0x7fd2d006fe60 2024-09-13T18:51:12.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.502+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1968818762 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d015af80 con 0x7fd2d006fe60 2024-09-13T18:51:12.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.502+0000 7fd2b7fff700 1 -- 172.21.15.146:0/1968818762 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4003900 con 0x7fd2d006fe60 2024-09-13T18:51:12.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.502+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1968818762 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d015f4d0 con 0x7fd2d006fe60 2024-09-13T18:51:12.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.502+0000 7fd2b7fff700 1 -- 172.21.15.146:0/1968818762 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd2c4003a60 con 0x7fd2d006fe60 2024-09-13T18:51:12.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.502+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1968818762 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d01581f0 con 0x7fd2d006fe60 2024-09-13T18:51:12.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.506+0000 7fd2b7fff700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd2bc066760 0x7fd2bc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:12.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.506+0000 7fd2b7fff700 1 -- 172.21.15.146:0/1968818762 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7fd2c4021070 con 0x7fd2d006fe60 2024-09-13T18:51:12.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.506+0000 7fd2ceffd700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd2bc066760 0x7fd2bc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:12.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.506+0000 7fd2ceffd700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd2bc066760 0x7fd2bc068c20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00c10e0 tx=0x7fd2c800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:12.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.510+0000 7fd2b7fff700 1 -- 172.21.15.146:0/1968818762 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd2c4052910 con 0x7fd2d006fe60 2024-09-13T18:51:12.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.722+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1968818762 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2d00c1f70 con 0x7fd2d006fe60 2024-09-13T18:51:12.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.722+0000 7fd2b7fff700 1 -- 172.21.15.146:0/1968818762 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5523 (secure 0 0 0) 0x7fd2c4015020 con 0x7fd2d006fe60 2024-09-13T18:51:12.741 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:51:12.741 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:51:11.991743+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:active","state_seq":116,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:51:12.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.726+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd2bc066760 msgr2=0x7fd2bc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:12.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.726+0000 7fd2d5b5f700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd2bc066760 0x7fd2bc068c20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00c10e0 tx=0x7fd2c800b040 comp rx=0 tx=0).stop 2024-09-13T18:51:12.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.726+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1968818762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2d006fe60 msgr2=0x7fd2d0161910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:12.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.726+0000 7fd2d5b5f700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2d006fe60 0x7fd2d0161910 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00675d0 tx=0x7fd2c4004770 comp rx=0 tx=0).stop 2024-09-13T18:51:12.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.726+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1968818762 shutdown_connections 2024-09-13T18:51:12.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.726+0000 7fd2d5b5f700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd2bc066760 0x7fd2bc068c20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:12.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.726+0000 7fd2d5b5f700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2d0157e10 0x7fd2d01f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:12.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.726+0000 7fd2d5b5f700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2d00c5bd0 0x7fd2d01f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:12.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.726+0000 7fd2d5b5f700 1 --2- 172.21.15.146:0/1968818762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2d006fe60 0x7fd2d0161910 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:12.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.726+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1968818762 >> 172.21.15.146:0/1968818762 conn(0x7fd2d00cb1d0 msgr2=0x7fd2d00c0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:12.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.726+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1968818762 shutdown_connections 2024-09-13T18:51:12.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:12.726+0000 7fd2d5b5f700 1 -- 172.21.15.146:0/1968818762 wait complete. 2024-09-13T18:51:12.745 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 96 2024-09-13T18:51:12.757 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=0, replacing gid=6459 2024-09-13T18:51:12.757 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.a 2024-09-13T18:51:18.758 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-09-13T18:51:18.758 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T18:51:18.759 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T18:51:18.761 INFO:tasks.ceph.mds.a:Started 2024-09-13T18:51:18.761 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:51:18.844 INFO:tasks.ceph.mds.a.smithi146.stdout:starting mds.a at 2024-09-13T18:51:18.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.866+0000 7fe9a3174700 1 Processor -- start 2024-09-13T18:51:18.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.866+0000 7fe9a3174700 1 -- start start 2024-09-13T18:51:18.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a3174700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe99c1572e0 0x7fe99c0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:18.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a3174700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c0b2d20 0x7fe99c06b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:18.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a3174700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99c06ae30 con 0x7fe99c1514b0 2024-09-13T18:51:18.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a3174700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99c1576c0 con 0x7fe99c1572e0 2024-09-13T18:51:18.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a3174700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c0caa70 con 0x7fe99c0b2d20 2024-09-13T18:51:18.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a1971700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe99c1572e0 0x7fe99c0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:18.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a1971700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe99c1572e0 0x7fe99c0b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49956/0 (socket says 172.21.15.146:49956) 2024-09-13T18:51:18.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a1971700 1 -- 172.21.15.146:0/265246194 learned_addr learned my addr 172.21.15.146:0/265246194 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:51:18.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a2973700 1 --2- 172.21.15.146:0/265246194 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c0b2d20 0x7fe99c06b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:18.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a1971700 1 -- 172.21.15.146:0/265246194 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c0b2d20 msgr2=0x7fe99c06b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:51:18.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a1971700 1 --2- 172.21.15.146:0/265246194 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c0b2d20 0x7fe99c06b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:18.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a1971700 1 -- 172.21.15.146:0/265246194 >> v1:172.21.15.154:6789/0 conn(0x7fe99c1514b0 legacy=0x7fe99c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:51:18.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a1971700 1 -- 172.21.15.146:0/265246194 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c0b0b70 con 0x7fe99c1572e0 2024-09-13T18:51:18.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a2973700 1 --2- 172.21.15.146:0/265246194 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c0b2d20 0x7fe99c06b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:51:18.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a1971700 1 --2- 172.21.15.146:0/265246194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1572e0 0x7fe99c0b27e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009910 tx=0x7fe98c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b82ee3ef3c47fad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:18.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a1170700 1 -- 172.21.15.146:0/265246194 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c00ba60 con 0x7fe99c1572e0 2024-09-13T18:51:18.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a1170700 1 -- 172.21.15.146:0/265246194 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c00bbc0 con 0x7fe99c1572e0 2024-09-13T18:51:18.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a1170700 1 -- 172.21.15.146:0/265246194 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c009e60 con 0x7fe99c1572e0 2024-09-13T18:51:18.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a3174700 1 -- 172.21.15.146:0/265246194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1572e0 msgr2=0x7fe99c0b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:18.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a3174700 1 --2- 172.21.15.146:0/265246194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1572e0 0x7fe99c0b27e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009910 tx=0x7fe98c009c20 comp rx=0 tx=0).stop 2024-09-13T18:51:18.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a3174700 1 -- 172.21.15.146:0/265246194 shutdown_connections 2024-09-13T18:51:18.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a3174700 1 --2- 172.21.15.146:0/265246194 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c0b2d20 0x7fe99c06b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:18.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a3174700 1 --2- 172.21.15.146:0/265246194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1572e0 0x7fe99c0b27e0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:18.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a3174700 1 -- 172.21.15.146:0/265246194 >> 172.21.15.146:0/265246194 conn(0x7fe99c06c2d0 msgr2=0x7fe99c06c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:18.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a3174700 1 -- 172.21.15.146:0/265246194 shutdown_connections 2024-09-13T18:51:18.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.870+0000 7fe9a3174700 1 -- 172.21.15.146:0/265246194 wait complete. 2024-09-13T18:51:18.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a3174700 1 Processor -- start 2024-09-13T18:51:18.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a3174700 1 -- start start 2024-09-13T18:51:18.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a3174700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c0b2d20 0x7fe99c0af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:18.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a3174700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c1514b0 0x7fe99c153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:18.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a3174700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1572e0 0x7fe99c153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:18.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a3174700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c1e4770 con 0x7fe99c1572e0 2024-09-13T18:51:18.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a3174700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c1e11a0 con 0x7fe99c0b2d20 2024-09-13T18:51:18.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a3174700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c1e1410 con 0x7fe99c1514b0 2024-09-13T18:51:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a2973700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1572e0 0x7fe99c153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a2973700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1572e0 0x7fe99c153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49972/0 (socket says 172.21.15.146:49972) 2024-09-13T18:51:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a2973700 1 -- 172.21.15.146:0/788482225 learned_addr learned my addr 172.21.15.146:0/788482225 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:51:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a1971700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c1514b0 0x7fe99c153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a2172700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c0b2d20 0x7fe99c0af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a2973700 1 -- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c1514b0 msgr2=0x7fe99c153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a2973700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c1514b0 0x7fe99c153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a2973700 1 -- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c0b2d20 msgr2=0x7fe99c0af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a2973700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c0b2d20 0x7fe99c0af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a2973700 1 -- 172.21.15.146:0/788482225 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c009570 con 0x7fe99c1572e0 2024-09-13T18:51:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a2172700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c0b2d20 0x7fe99c0af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:51:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a2973700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1572e0 0x7fe99c153a70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe99400cef0 tx=0x7fe99400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:18.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe992ffd700 1 -- 172.21.15.146:0/788482225 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99400adb0 con 0x7fe99c1572e0 2024-09-13T18:51:18.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe992ffd700 1 -- 172.21.15.146:0/788482225 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe994015440 con 0x7fe99c1572e0 2024-09-13T18:51:18.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a3174700 1 -- 172.21.15.146:0/788482225 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99c1e16f0 con 0x7fe99c1572e0 2024-09-13T18:51:18.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe992ffd700 1 -- 172.21.15.146:0/788482225 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940157f0 con 0x7fe99c1572e0 2024-09-13T18:51:18.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.874+0000 7fe9a3174700 1 -- 172.21.15.146:0/788482225 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99c1e1cb0 con 0x7fe99c1572e0 2024-09-13T18:51:18.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.878+0000 7fe992ffd700 1 -- 172.21.15.146:0/788482225 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe99400d430 con 0x7fe99c1572e0 2024-09-13T18:51:18.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.878+0000 7fe9a3174700 1 -- 172.21.15.146:0/788482225 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99c151890 con 0x7fe99c1572e0 2024-09-13T18:51:18.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.878+0000 7fe992ffd700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe980066730 0x7fe980068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:18.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.878+0000 7fe992ffd700 1 -- 172.21.15.146:0/788482225 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7fe994084e20 con 0x7fe99c1572e0 2024-09-13T18:51:18.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.878+0000 7fe9a2172700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe980066730 0x7fe980068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:18.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.878+0000 7fe9a2172700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe980066730 0x7fe980068bf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe998000f60 tx=0x7fe998009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:18.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:18.886+0000 7fe992ffd700 1 -- 172.21.15.146:0/788482225 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe994051a00 con 0x7fe99c1572e0 2024-09-13T18:51:19.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.138+0000 7fe9a3174700 1 -- 172.21.15.146:0/788482225 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe99c0c7cc0 con 0x7fe99c1572e0 2024-09-13T18:51:19.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.138+0000 7fe992ffd700 1 -- 172.21.15.146:0/788482225 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5523 (secure 0 0 0) 0x7fe99401a070 con 0x7fe99c1572e0 2024-09-13T18:51:19.156 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:51:19.156 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:51:11.991743+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:active","state_seq":116,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:51:19.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.142+0000 7fe9a3174700 1 -- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe980066730 msgr2=0x7fe980068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:19.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.142+0000 7fe9a3174700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe980066730 0x7fe980068bf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe998000f60 tx=0x7fe998009040 comp rx=0 tx=0).stop 2024-09-13T18:51:19.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.142+0000 7fe9a3174700 1 -- 172.21.15.146:0/788482225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1572e0 msgr2=0x7fe99c153a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:19.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.142+0000 7fe9a3174700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1572e0 0x7fe99c153a70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe99400cef0 tx=0x7fe99400cf20 comp rx=0 tx=0).stop 2024-09-13T18:51:19.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.142+0000 7fe9a3174700 1 -- 172.21.15.146:0/788482225 shutdown_connections 2024-09-13T18:51:19.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.142+0000 7fe9a3174700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe980066730 0x7fe980068bf0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:19.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.142+0000 7fe9a3174700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1572e0 0x7fe99c153a70 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:19.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.142+0000 7fe9a3174700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c1514b0 0x7fe99c153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:19.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.142+0000 7fe9a3174700 1 --2- 172.21.15.146:0/788482225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c0b2d20 0x7fe99c0af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:19.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.142+0000 7fe9a3174700 1 -- 172.21.15.146:0/788482225 >> 172.21.15.146:0/788482225 conn(0x7fe99c06c2d0 msgr2=0x7fe99c156090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:19.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.142+0000 7fe9a3174700 1 -- 172.21.15.146:0/788482225 shutdown_connections 2024-09-13T18:51:19.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:19.142+0000 7fe9a3174700 1 -- 172.21.15.146:0/788482225 wait complete. 2024-09-13T18:51:19.161 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 96 2024-09-13T18:51:19.174 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-09-13T18:51:21.174 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:51:21.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.282+0000 7f456b59e700 1 Processor -- start 2024-09-13T18:51:21.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.282+0000 7f456b59e700 1 -- start start 2024-09-13T18:51:21.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f456c150230 0x7f456c06ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:21.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456b59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f456c071610 0x7f456c159b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:21.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456b59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456c071410 con 0x7f456c150610 2024-09-13T18:51:21.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456b59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456c1596b0 con 0x7f456c071610 2024-09-13T18:51:21.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456c156090 con 0x7f456c150230 2024-09-13T18:51:21.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456ad9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f456c071610 0x7f456c159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:21.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456a59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f456c150230 0x7f456c06ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:21.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456ad9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f456c071610 0x7f456c159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:50166/0 (socket says 172.21.15.146:50166) 2024-09-13T18:51:21.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456ad9d700 1 -- 172.21.15.146:0/3209136927 learned_addr learned my addr 172.21.15.146:0/3209136927 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:51:21.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456a59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f456c150230 0x7f456c06ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:50512/0 (socket says 172.21.15.146:50512) 2024-09-13T18:51:21.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456a59c700 1 -- 172.21.15.146:0/3209136927 >> v2:172.21.15.154:3300/0 conn(0x7f456c071610 msgr2=0x7f456c159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:51:21.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456a59c700 1 --2- 172.21.15.146:0/3209136927 >> v2:172.21.15.154:3300/0 conn(0x7f456c071610 0x7f456c159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:21.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456a59c700 1 -- 172.21.15.146:0/3209136927 >> v1:172.21.15.154:6789/0 conn(0x7f456c150610 legacy=0x7f456c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:21.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456a59c700 1 -- 172.21.15.146:0/3209136927 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c156350 con 0x7f456c150230 2024-09-13T18:51:21.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456a59c700 1 --2- 172.21.15.146:0/3209136927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f456c150230 0x7f456c06ebd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f455c009850 tx=0x7f455c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6096e5b5a32bf249 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:21.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456959a700 1 -- 172.21.15.146:0/3209136927 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c011830 con 0x7f456c150230 2024-09-13T18:51:21.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456959a700 1 -- 172.21.15.146:0/3209136927 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455c011990 con 0x7f456c150230 2024-09-13T18:51:21.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456959a700 1 -- 172.21.15.146:0/3209136927 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c011d40 con 0x7f456c150230 2024-09-13T18:51:21.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456b59e700 1 -- 172.21.15.146:0/3209136927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f456c150230 msgr2=0x7f456c06ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:21.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456b59e700 1 --2- 172.21.15.146:0/3209136927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f456c150230 0x7f456c06ebd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f455c009850 tx=0x7f455c009b60 comp rx=0 tx=0).stop 2024-09-13T18:51:21.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456b59e700 1 -- 172.21.15.146:0/3209136927 shutdown_connections 2024-09-13T18:51:21.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456b59e700 1 --2- 172.21.15.146:0/3209136927 >> v2:172.21.15.154:3300/0 conn(0x7f456c071610 0x7f456c159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:21.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456b59e700 1 --2- 172.21.15.146:0/3209136927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f456c150230 0x7f456c06ebd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:21.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456b59e700 1 -- 172.21.15.146:0/3209136927 >> 172.21.15.146:0/3209136927 conn(0x7f456c071ed0 msgr2=0x7f456c0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:21.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456b59e700 1 -- 172.21.15.146:0/3209136927 shutdown_connections 2024-09-13T18:51:21.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.286+0000 7f456b59e700 1 -- 172.21.15.146:0/3209136927 wait complete. 2024-09-13T18:51:21.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456b59e700 1 Processor -- start 2024-09-13T18:51:21.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456b59e700 1 -- start start 2024-09-13T18:51:21.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456b59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f456c071610 0x7f456c1f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:21.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f456c150230 0x7f456c1512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:21.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f456c152010 0x7f456c0c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:21.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456b59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c149d30 con 0x7f456c152010 2024-09-13T18:51:21.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456b59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c155c90 con 0x7f456c071610 2024-09-13T18:51:21.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456c157320 con 0x7f456c150230 2024-09-13T18:51:21.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f456c152010 0x7f456c0c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:21.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f456c152010 0x7f456c0c68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40206/0 (socket says 172.21.15.146:40206) 2024-09-13T18:51:21.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456ad9d700 1 -- 172.21.15.146:0/868821573 learned_addr learned my addr 172.21.15.146:0/868821573 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:51:21.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456a59c700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f456c071610 0x7f456c1f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:21.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f4569d9b700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f456c150230 0x7f456c1512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:21.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456ad9d700 1 -- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f456c150230 msgr2=0x7f456c1512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:21.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456ad9d700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f456c150230 0x7f456c1512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:21.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456ad9d700 1 -- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f456c071610 msgr2=0x7f456c1f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:21.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456ad9d700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f456c071610 0x7f456c1f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:21.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456ad9d700 1 -- 172.21.15.146:0/868821573 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c0094b0 con 0x7f456c152010 2024-09-13T18:51:21.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f4569d9b700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f456c150230 0x7f456c1512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:51:21.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456ad9d700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f456c152010 0x7f456c0c68c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f456400efc0 tx=0x7f456400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:21.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f455affd700 1 -- 172.21.15.146:0/868821573 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564003c40 con 0x7f456c152010 2024-09-13T18:51:21.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f455affd700 1 -- 172.21.15.146:0/868821573 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456400f040 con 0x7f456c152010 2024-09-13T18:51:21.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456b59e700 1 -- 172.21.15.146:0/868821573 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456c1575c0 con 0x7f456c152010 2024-09-13T18:51:21.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f455affd700 1 -- 172.21.15.146:0/868821573 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564009e60 con 0x7f456c152010 2024-09-13T18:51:21.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.290+0000 7f456b59e700 1 -- 172.21.15.146:0/868821573 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456c150d80 con 0x7f456c152010 2024-09-13T18:51:21.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.294+0000 7f456b59e700 1 -- 172.21.15.146:0/868821573 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f456c150fa0 con 0x7f456c152010 2024-09-13T18:51:21.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.294+0000 7f455affd700 1 -- 172.21.15.146:0/868821573 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f456400c930 con 0x7f456c152010 2024-09-13T18:51:21.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.294+0000 7f455affd700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f454c066730 0x7f454c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:51:21.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.294+0000 7f455affd700 1 -- 172.21.15.146:0/868821573 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7f4564084bf0 con 0x7f456c152010 2024-09-13T18:51:21.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.294+0000 7f456a59c700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f454c066730 0x7f454c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:51:21.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.294+0000 7f456a59c700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f454c066730 0x7f454c068bf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f455c0090b0 tx=0x7f455c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:51:21.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.298+0000 7f455affd700 1 -- 172.21.15.146:0/868821573 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f45640517d0 con 0x7f456c152010 2024-09-13T18:51:21.508 INFO:tasks.ceph.mds.e.smithi146.stderr:2024-09-13T18:51:21.490+0000 7f43b84e6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:51:21.508 INFO:tasks.ceph.mds.e.smithi146.stderr:2024-09-13T18:51:21.490+0000 7f43b84e6700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-13T18:51:21.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.530+0000 7f456b59e700 1 -- 172.21.15.146:0/868821573 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f456c06ebd0 con 0x7f456c152010 2024-09-13T18:51:21.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.530+0000 7f455affd700 1 -- 172.21.15.146:0/868821573 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+6283 (secure 0 0 0) 0x7f4564012340 con 0x7f456c152010 2024-09-13T18:51:21.548 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:51:21.548 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:51:11.991743+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:active","state_seq":116,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:51:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.534+0000 7f456b59e700 1 -- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f454c066730 msgr2=0x7f454c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.534+0000 7f456b59e700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f454c066730 0x7f454c068bf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f455c0090b0 tx=0x7f455c008030 comp rx=0 tx=0).stop 2024-09-13T18:51:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.534+0000 7f456b59e700 1 -- 172.21.15.146:0/868821573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f456c152010 msgr2=0x7f456c0c68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:51:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.534+0000 7f456b59e700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f456c152010 0x7f456c0c68c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f456400efc0 tx=0x7f456400c3a0 comp rx=0 tx=0).stop 2024-09-13T18:51:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.534+0000 7f456b59e700 1 -- 172.21.15.146:0/868821573 shutdown_connections 2024-09-13T18:51:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.534+0000 7f456b59e700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f454c066730 0x7f454c068bf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.534+0000 7f456b59e700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f456c152010 0x7f456c0c68c0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.534+0000 7f456b59e700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f456c150230 0x7f456c1512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.534+0000 7f456b59e700 1 --2- 172.21.15.146:0/868821573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f456c071610 0x7f456c1f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:51:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.534+0000 7f456b59e700 1 -- 172.21.15.146:0/868821573 >> 172.21.15.146:0/868821573 conn(0x7f456c071ed0 msgr2=0x7f456c14aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:51:21.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.534+0000 7f456b59e700 1 -- 172.21.15.146:0/868821573 shutdown_connections 2024-09-13T18:51:21.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:51:21.534+0000 7f456b59e700 1 -- 172.21.15.146:0/868821573 wait complete. 2024-09-13T18:51:21.552 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 97 2024-09-13T18:51:21.566 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-09-13T18:51:21.566 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 54 secs before thrashing 2024-09-13T18:51:25.724 INFO:tasks.workunit.client.0.smithi146.stdout:fs setup took 1265 secs 2024-09-13T18:51:26.841 INFO:tasks.workunit.client.0.smithi146.stdout:Syncing()...1 sec 2024-09-13T18:51:26.842 INFO:tasks.workunit.client.0.smithi146.stdout:Starting Actual Benchmark At: Fri Sep 13 18:51:26 2024 2024-09-13T18:51:26.843 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:51:42.526 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:51:42.529 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:52:12.788 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:52:12.791 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:52:15.568 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:18.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.293+0000 7fc2d0a9a700 1 Processor -- start 2024-09-13T18:52:18.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.297+0000 7fc2d0a9a700 1 -- start start 2024-09-13T18:52:18.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.297+0000 7fc2d0a9a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc2cc151510 0x7fc2cc0cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.297+0000 7fc2d0a9a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2cc158040 con 0x7fc2cc0c84f0 2024-09-13T18:52:18.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.297+0000 7fc2d0a9a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2cc0c7ae0 con 0x7fc2cc1518f0 2024-09-13T18:52:18.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.297+0000 7fc2d0a9a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2cc0afc80 con 0x7fc2cc151510 2024-09-13T18:52:18.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.297+0000 7fc2c9d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc2cc1518f0 0x7fc2cc0c7de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:58676/0 (socket says 172.21.15.146:58676) 2024-09-13T18:52:18.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.297+0000 7fc2c9d9b700 1 -- 172.21.15.146:0/2784351140 learned_addr learned my addr 172.21.15.146:0/2784351140 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:52:18.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.297+0000 7fc2ca59c700 1 --2- 172.21.15.146:0/2784351140 >> v2:172.21.15.146:3300/0 conn(0x7fc2cc151510 0x7fc2cc0cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.301+0000 7fc2ca59c700 1 -- 172.21.15.146:0/2784351140 >> v1:172.21.15.154:6789/0 conn(0x7fc2cc1518f0 legacy=0x7fc2cc0c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.301+0000 7fc2ca59c700 1 -- 172.21.15.146:0/2784351140 >> v1:172.21.15.146:6789/0 conn(0x7fc2cc0c84f0 legacy=0x7fc2cc161ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:52:18.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.301+0000 7fc2ca59c700 1 -- 172.21.15.146:0/2784351140 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2cc0aff60 con 0x7fc2cc151510 2024-09-13T18:52:18.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.301+0000 7fc2ca59c700 1 --2- 172.21.15.146:0/2784351140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc151510 0x7fc2cc0cd9d0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009a30 tx=0x7fc2bc009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d0c185c95fdede server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:18.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.321+0000 7fc2c959a700 1 -- 172.21.15.146:0/2784351140 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc004230 con 0x7fc2cc151510 2024-09-13T18:52:18.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.321+0000 7fc2c959a700 1 -- 172.21.15.146:0/2784351140 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc004390 con 0x7fc2cc151510 2024-09-13T18:52:18.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.321+0000 7fc2d0a9a700 1 -- 172.21.15.146:0/2784351140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc151510 msgr2=0x7fc2cc0cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.321+0000 7fc2d0a9a700 1 --2- 172.21.15.146:0/2784351140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc151510 0x7fc2cc0cd9d0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009a30 tx=0x7fc2bc009d40 comp rx=0 tx=0).stop 2024-09-13T18:52:18.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.321+0000 7fc2d0a9a700 1 -- 172.21.15.146:0/2784351140 shutdown_connections 2024-09-13T18:52:18.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.321+0000 7fc2d0a9a700 1 --2- 172.21.15.146:0/2784351140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc151510 0x7fc2cc0cd9d0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.321+0000 7fc2d0a9a700 1 -- 172.21.15.146:0/2784351140 >> 172.21.15.146:0/2784351140 conn(0x7fc2cc066090 msgr2=0x7fc2cc066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:18.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.321+0000 7fc2d0a9a700 1 -- 172.21.15.146:0/2784351140 shutdown_connections 2024-09-13T18:52:18.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.325+0000 7fc2d0a9a700 1 -- 172.21.15.146:0/2784351140 wait complete. 2024-09-13T18:52:18.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.329+0000 7fc2d0a9a700 1 Processor -- start 2024-09-13T18:52:18.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.329+0000 7fc2d0a9a700 1 -- start start 2024-09-13T18:52:18.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.329+0000 7fc2d0a9a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2cc0c84f0 0x7fc2cc0ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.329+0000 7fc2d0a9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc151510 0x7fc2cc0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.329+0000 7fc2d0a9a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2cc1518f0 0x7fc2cc0b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.329+0000 7fc2d0a9a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc0b0630 con 0x7fc2cc151510 2024-09-13T18:52:18.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.329+0000 7fc2d0a9a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc0b2210 con 0x7fc2cc0c84f0 2024-09-13T18:52:18.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.329+0000 7fc2d0a9a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc0c5230 con 0x7fc2cc1518f0 2024-09-13T18:52:18.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.329+0000 7fc2c9d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc151510 0x7fc2cc0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.329+0000 7fc2c9d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc151510 0x7fc2cc0b0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59974/0 (socket says 172.21.15.146:59974) 2024-09-13T18:52:18.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.329+0000 7fc2c9d9b700 1 -- 172.21.15.146:0/3536090803 learned_addr learned my addr 172.21.15.146:0/3536090803 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:18.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.333+0000 7fc2c9d9b700 1 -- 172.21.15.146:0/3536090803 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2cc1518f0 msgr2=0x7fc2cc0b1000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:52:18.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.333+0000 7fc2c9d9b700 1 --2- 172.21.15.146:0/3536090803 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2cc1518f0 0x7fc2cc0b1000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.333+0000 7fc2c9d9b700 1 -- 172.21.15.146:0/3536090803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2cc0c84f0 msgr2=0x7fc2cc0ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.333+0000 7fc2c9d9b700 1 --2- 172.21.15.146:0/3536090803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2cc0c84f0 0x7fc2cc0ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.333+0000 7fc2c9d9b700 1 -- 172.21.15.146:0/3536090803 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc0094b0 con 0x7fc2cc151510 2024-09-13T18:52:18.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.333+0000 7fc2c9d9b700 1 --2- 172.21.15.146:0/3536090803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc151510 0x7fc2cc0b0ac0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400a0e0 tx=0x7fc2c400a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:18.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.341+0000 7fc2baffd700 1 -- 172.21.15.146:0/3536090803 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400add0 con 0x7fc2cc151510 2024-09-13T18:52:18.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.341+0000 7fc2d0a9a700 1 -- 172.21.15.146:0/3536090803 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2cc0b2470 con 0x7fc2cc151510 2024-09-13T18:52:18.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.341+0000 7fc2d0a9a700 1 -- 172.21.15.146:0/3536090803 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2cc0b2a30 con 0x7fc2cc151510 2024-09-13T18:52:18.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.341+0000 7fc2baffd700 1 -- 172.21.15.146:0/3536090803 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c40070d0 con 0x7fc2cc151510 2024-09-13T18:52:18.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.341+0000 7fc2baffd700 1 -- 172.21.15.146:0/3536090803 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4007460 con 0x7fc2cc151510 2024-09-13T18:52:18.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.341+0000 7fc2b8ff9700 1 -- 172.21.15.146:0/3536090803 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b0005310 con 0x7fc2cc151510 2024-09-13T18:52:18.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.349+0000 7fc2baffd700 1 -- 172.21.15.146:0/3536090803 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc2c40075c0 con 0x7fc2cc151510 2024-09-13T18:52:18.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.349+0000 7fc2baffd700 1 --2- 172.21.15.146:0/3536090803 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2ac066800 0x7fc2ac068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.349+0000 7fc2baffd700 1 -- 172.21.15.146:0/3536090803 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7fc2c4084a60 con 0x7fc2cc151510 2024-09-13T18:52:18.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.349+0000 7fc2ca59c700 1 --2- 172.21.15.146:0/3536090803 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2ac066800 0x7fc2ac068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.353+0000 7fc2ca59c700 1 --2- 172.21.15.146:0/3536090803 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2ac066800 0x7fc2ac068cc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009a30 tx=0x7fc2bc010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:18.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:18.361+0000 7fc2baffd700 1 -- 172.21.15.146:0/3536090803 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc2c4051620 con 0x7fc2cc151510 2024-09-13T18:52:19.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.185+0000 7fc2b8ff9700 1 -- 172.21.15.146:0/3536090803 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2b0005b40 con 0x7fc2cc151510 2024-09-13T18:52:19.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.185+0000 7fc2baffd700 1 -- 172.21.15.146:0/3536090803 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+6283 (secure 0 0 0) 0x7fc2c4018030 con 0x7fc2cc151510 2024-09-13T18:52:19.202 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:19.202 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:51:11.991743+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:active","state_seq":116,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:19.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.193+0000 7fc2d0a9a700 1 -- 172.21.15.146:0/3536090803 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2ac066800 msgr2=0x7fc2ac068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:19.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.193+0000 7fc2d0a9a700 1 --2- 172.21.15.146:0/3536090803 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2ac066800 0x7fc2ac068cc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009a30 tx=0x7fc2bc010000 comp rx=0 tx=0).stop 2024-09-13T18:52:19.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.193+0000 7fc2d0a9a700 1 -- 172.21.15.146:0/3536090803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc151510 msgr2=0x7fc2cc0b0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:19.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.193+0000 7fc2d0a9a700 1 --2- 172.21.15.146:0/3536090803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc151510 0x7fc2cc0b0ac0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400a0e0 tx=0x7fc2c400a4a0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.193+0000 7fc2d0a9a700 1 -- 172.21.15.146:0/3536090803 shutdown_connections 2024-09-13T18:52:19.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.193+0000 7fc2d0a9a700 1 --2- 172.21.15.146:0/3536090803 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2ac066800 0x7fc2ac068cc0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.193+0000 7fc2d0a9a700 1 --2- 172.21.15.146:0/3536090803 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2cc1518f0 0x7fc2cc0b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.193+0000 7fc2d0a9a700 1 --2- 172.21.15.146:0/3536090803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc151510 0x7fc2cc0b0ac0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.193+0000 7fc2d0a9a700 1 --2- 172.21.15.146:0/3536090803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2cc0c84f0 0x7fc2cc0ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.193+0000 7fc2d0a9a700 1 -- 172.21.15.146:0/3536090803 >> 172.21.15.146:0/3536090803 conn(0x7fc2cc066090 msgr2=0x7fc2cc0aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:19.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.193+0000 7fc2d0a9a700 1 -- 172.21.15.146:0/3536090803 shutdown_connections 2024-09-13T18:52:19.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.193+0000 7fc2d0a9a700 1 -- 172.21.15.146:0/3536090803 wait complete. 2024-09-13T18:52:19.213 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 97 2024-09-13T18:52:19.230 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=0) 2024-09-13T18:52:19.230 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-09-13T18:52:19.230 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:52:19.244 INFO:tasks.ceph.mds.e:Stopped 2024-09-13T18:52:19.245 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:19.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.613+0000 7f9d426f2700 1 Processor -- start 2024-09-13T18:52:19.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.613+0000 7f9d426f2700 1 -- start start 2024-09-13T18:52:19.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.613+0000 7f9d426f2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0caed0 0x7f9d3c072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:19.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.613+0000 7f9d426f2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9d3c0cb2b0 0x7f9d3c0c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:19.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.613+0000 7f9d426f2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3c161060 con 0x7f9d3c0c8230 2024-09-13T18:52:19.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.613+0000 7f9d426f2700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d3c0c78d0 con 0x7f9d3c0cb2b0 2024-09-13T18:52:19.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.613+0000 7f9d426f2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c161300 con 0x7f9d3c0caed0 2024-09-13T18:52:19.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.613+0000 7f9d3bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0caed0 0x7f9d3c072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:19.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.613+0000 7f9d40c8f700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9d3c0c8230 0x7f9d3c15e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37932/0 (socket says 172.21.15.146:37932) 2024-09-13T18:52:19.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.613+0000 7f9d40c8f700 1 -- 172.21.15.146:0/2261657523 learned_addr learned my addr 172.21.15.146:0/2261657523 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:52:19.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.617+0000 7f9d3bfff700 1 -- 172.21.15.146:0/2261657523 >> v2:172.21.15.146:3300/0 conn(0x7f9d3c0cb2b0 msgr2=0x7f9d3c0c7cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:52:19.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.617+0000 7f9d3bfff700 1 --2- 172.21.15.146:0/2261657523 >> v2:172.21.15.146:3300/0 conn(0x7f9d3c0cb2b0 0x7f9d3c0c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.617+0000 7f9d3bfff700 1 -- 172.21.15.146:0/2261657523 >> v1:172.21.15.146:6789/0 conn(0x7f9d3c0c8230 legacy=0x7f9d3c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:19.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.617+0000 7f9d3bfff700 1 -- 172.21.15.146:0/2261657523 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c1615c0 con 0x7f9d3c0caed0 2024-09-13T18:52:19.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.621+0000 7f9d3bfff700 1 --2- 172.21.15.146:0/2261657523 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0caed0 0x7f9d3c072380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009850 tx=0x7f9d2c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=925049d060fd6965 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:19.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.625+0000 7f9d3affd700 1 -- 172.21.15.146:0/2261657523 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c011830 con 0x7f9d3c0caed0 2024-09-13T18:52:19.640 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.625+0000 7f9d3affd700 1 -- 172.21.15.146:0/2261657523 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c011990 con 0x7f9d3c0caed0 2024-09-13T18:52:19.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.625+0000 7f9d426f2700 1 -- 172.21.15.146:0/2261657523 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0caed0 msgr2=0x7f9d3c072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:19.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.625+0000 7f9d426f2700 1 --2- 172.21.15.146:0/2261657523 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0caed0 0x7f9d3c072380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009850 tx=0x7f9d2c009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:19.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.625+0000 7f9d426f2700 1 -- 172.21.15.146:0/2261657523 shutdown_connections 2024-09-13T18:52:19.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.625+0000 7f9d426f2700 1 --2- 172.21.15.146:0/2261657523 >> v2:172.21.15.146:3300/0 conn(0x7f9d3c0cb2b0 0x7f9d3c0c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.625+0000 7f9d426f2700 1 --2- 172.21.15.146:0/2261657523 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0caed0 0x7f9d3c072380 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.625+0000 7f9d426f2700 1 -- 172.21.15.146:0/2261657523 >> 172.21.15.146:0/2261657523 conn(0x7f9d3c066090 msgr2=0x7f9d3c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:19.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.629+0000 7f9d426f2700 1 -- 172.21.15.146:0/2261657523 shutdown_connections 2024-09-13T18:52:19.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.629+0000 7f9d426f2700 1 -- 172.21.15.146:0/2261657523 wait complete. 2024-09-13T18:52:19.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.629+0000 7f9d426f2700 1 Processor -- start 2024-09-13T18:52:19.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.629+0000 7f9d426f2700 1 -- start start 2024-09-13T18:52:19.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d426f2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0c8230 0x7f9d3c0ab340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:19.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d426f2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d3c0cb2b0 0x7f9d3c14d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:19.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d426f2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d3c0ac620 0x7f9d3c14c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:19.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d426f2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c0aa8f0 con 0x7f9d3c0ac620 2024-09-13T18:52:19.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d426f2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c0aaeb0 con 0x7f9d3c0cb2b0 2024-09-13T18:52:19.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d426f2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c0ad670 con 0x7f9d3c0c8230 2024-09-13T18:52:19.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d3bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0c8230 0x7f9d3c0ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:19.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d3b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d3c0cb2b0 0x7f9d3c14d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:19.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d3bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0c8230 0x7f9d3c0ab340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:43638/0 (socket says 172.21.15.146:43638) 2024-09-13T18:52:19.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d3b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d3c0cb2b0 0x7f9d3c14d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:33722/0 (socket says 172.21.15.146:33722) 2024-09-13T18:52:19.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d3bfff700 1 -- 172.21.15.146:0/1680967017 learned_addr learned my addr 172.21.15.146:0/1680967017 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:19.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d3bfff700 1 -- 172.21.15.146:0/1680967017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d3c0cb2b0 msgr2=0x7f9d3c14d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:19.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d3bfff700 1 --2- 172.21.15.146:0/1680967017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d3c0cb2b0 0x7f9d3c14d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d3bfff700 1 -- 172.21.15.146:0/1680967017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d3c0ac620 msgr2=0x7f9d3c14c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:19.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d3bfff700 1 --2- 172.21.15.146:0/1680967017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d3c0ac620 0x7f9d3c14c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d3bfff700 1 -- 172.21.15.146:0/1680967017 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c0094b0 con 0x7f9d3c0c8230 2024-09-13T18:52:19.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d3bfff700 1 --2- 172.21.15.146:0/1680967017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0c8230 0x7f9d3c0ab340 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c014040 tx=0x7f9d2c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:19.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d38ff9700 1 -- 172.21.15.146:0/1680967017 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0038c0 con 0x7f9d3c0c8230 2024-09-13T18:52:19.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d426f2700 1 -- 172.21.15.146:0/1680967017 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3c0adb70 con 0x7f9d3c0c8230 2024-09-13T18:52:19.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d426f2700 1 -- 172.21.15.146:0/1680967017 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3c0ae280 con 0x7f9d3c0c8230 2024-09-13T18:52:19.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.633+0000 7f9d38ff9700 1 -- 172.21.15.146:0/1680967017 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c01e070 con 0x7f9d3c0c8230 2024-09-13T18:52:19.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.637+0000 7f9d38ff9700 1 -- 172.21.15.146:0/1680967017 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c003c40 con 0x7f9d3c0c8230 2024-09-13T18:52:19.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.645+0000 7f9d38ff9700 1 -- 172.21.15.146:0/1680967017 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d2c01a420 con 0x7f9d3c0c8230 2024-09-13T18:52:19.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.645+0000 7f9d38ff9700 1 --2- 172.21.15.146:0/1680967017 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d28066800 0x7f9d28068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:19.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.645+0000 7f9d3b7fe700 1 --2- 172.21.15.146:0/1680967017 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d28066800 0x7f9d28068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:19.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.645+0000 7f9d38ff9700 1 -- 172.21.15.146:0/1680967017 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7f9d2c085700 con 0x7f9d3c0c8230 2024-09-13T18:52:19.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.653+0000 7f9d3b7fe700 1 --2- 172.21.15.146:0/1680967017 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d28066800 0x7f9d28068cc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400a400 tx=0x7f9d3400d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:19.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.657+0000 7f9d426f2700 1 -- 172.21.15.146:0/1680967017 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1c005310 con 0x7f9d3c0c8230 2024-09-13T18:52:19.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:19.669+0000 7f9d38ff9700 1 -- 172.21.15.146:0/1680967017 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9d2c0522c0 con 0x7f9d3c0c8230 2024-09-13T18:52:20.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.453+0000 7f9d426f2700 1 -- 172.21.15.146:0/1680967017 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d1c005b40 con 0x7f9d3c0c8230 2024-09-13T18:52:20.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.453+0000 7f9d38ff9700 1 -- 172.21.15.146:0/1680967017 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+6283 (secure 0 0 0) 0x7f9d2c028020 con 0x7f9d3c0c8230 2024-09-13T18:52:20.471 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:20.471 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:51:11.991743+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:active","state_seq":116,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:20.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.461+0000 7f9d426f2700 1 -- 172.21.15.146:0/1680967017 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d28066800 msgr2=0x7f9d28068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:20.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.461+0000 7f9d426f2700 1 --2- 172.21.15.146:0/1680967017 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d28066800 0x7f9d28068cc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400a400 tx=0x7f9d3400d040 comp rx=0 tx=0).stop 2024-09-13T18:52:20.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.461+0000 7f9d426f2700 1 -- 172.21.15.146:0/1680967017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0c8230 msgr2=0x7f9d3c0ab340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:20.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.461+0000 7f9d426f2700 1 --2- 172.21.15.146:0/1680967017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0c8230 0x7f9d3c0ab340 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c014040 tx=0x7f9d2c004770 comp rx=0 tx=0).stop 2024-09-13T18:52:20.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.461+0000 7f9d426f2700 1 -- 172.21.15.146:0/1680967017 shutdown_connections 2024-09-13T18:52:20.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.461+0000 7f9d426f2700 1 --2- 172.21.15.146:0/1680967017 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9d28066800 0x7f9d28068cc0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.461+0000 7f9d426f2700 1 --2- 172.21.15.146:0/1680967017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d3c0ac620 0x7f9d3c14c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.461+0000 7f9d426f2700 1 --2- 172.21.15.146:0/1680967017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d3c0cb2b0 0x7f9d3c14d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.461+0000 7f9d426f2700 1 --2- 172.21.15.146:0/1680967017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0c8230 0x7f9d3c0ab340 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.461+0000 7f9d426f2700 1 -- 172.21.15.146:0/1680967017 >> 172.21.15.146:0/1680967017 conn(0x7f9d3c066090 msgr2=0x7f9d3c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:20.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.461+0000 7f9d426f2700 1 -- 172.21.15.146:0/1680967017 shutdown_connections 2024-09-13T18:52:20.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:20.461+0000 7f9d426f2700 1 -- 172.21.15.146:0/1680967017 wait complete. 2024-09-13T18:52:20.482 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 97 2024-09-13T18:52:20.500 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:52:22.501 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:23.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabbd7f6700 1 Processor -- start 2024-09-13T18:52:23.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabbd7f6700 1 -- start start 2024-09-13T18:52:23.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabbd7f6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fabb80caed0 0x7fabb8072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabbd7f6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabb815e170 0x7fabb815e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabbd7f6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb8161060 con 0x7fabb80cb2b0 2024-09-13T18:52:23.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabbd7f6700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb80c78d0 con 0x7fabb80caed0 2024-09-13T18:52:23.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabbd7f6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb815eb40 con 0x7fabb815e170 2024-09-13T18:52:23.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabb6ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fabb80caed0 0x7fabb8072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabb6ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fabb80caed0 0x7fabb8072380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:56236/0 (socket says 172.21.15.146:56236) 2024-09-13T18:52:23.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabb6ffd700 1 -- 172.21.15.146:0/3288355547 learned_addr learned my addr 172.21.15.146:0/3288355547 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:23.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabb6ffd700 1 -- 172.21.15.146:0/3288355547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabb815e170 msgr2=0x7fabb815e600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:52:23.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabb6ffd700 1 --2- 172.21.15.146:0/3288355547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabb815e170 0x7fabb815e600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabb6ffd700 1 -- 172.21.15.146:0/3288355547 >> v1:172.21.15.154:6789/0 conn(0x7fabb80cb2b0 legacy=0x7fabb80c7cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:52:23.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabb6ffd700 1 -- 172.21.15.146:0/3288355547 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb8161300 con 0x7fabb80caed0 2024-09-13T18:52:23.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.013+0000 7fabb6ffd700 1 --2- 172.21.15.146:0/3288355547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb80caed0 0x7fabb8072380 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7faba8009ba0 tx=0x7faba8009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7c2597012287f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.025+0000 7fabb5ffb700 1 -- 172.21.15.146:0/3288355547 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8004230 con 0x7fabb80caed0 2024-09-13T18:52:23.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.025+0000 7fabb5ffb700 1 -- 172.21.15.146:0/3288355547 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba8004390 con 0x7fabb80caed0 2024-09-13T18:52:23.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.025+0000 7fabb5ffb700 1 -- 172.21.15.146:0/3288355547 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80036c0 con 0x7fabb80caed0 2024-09-13T18:52:23.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.025+0000 7fabbd7f6700 1 -- 172.21.15.146:0/3288355547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb80caed0 msgr2=0x7fabb8072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.025+0000 7fabbd7f6700 1 --2- 172.21.15.146:0/3288355547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb80caed0 0x7fabb8072380 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7faba8009ba0 tx=0x7faba8009eb0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 -- 172.21.15.146:0/3288355547 shutdown_connections 2024-09-13T18:52:23.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 --2- 172.21.15.146:0/3288355547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabb815e170 0x7fabb815e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 --2- 172.21.15.146:0/3288355547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb80caed0 0x7fabb8072380 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 -- 172.21.15.146:0/3288355547 >> 172.21.15.146:0/3288355547 conn(0x7fabb8066090 msgr2=0x7fabb8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 -- 172.21.15.146:0/3288355547 shutdown_connections 2024-09-13T18:52:23.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 -- 172.21.15.146:0/3288355547 wait complete. 2024-09-13T18:52:23.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 Processor -- start 2024-09-13T18:52:23.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 -- start start 2024-09-13T18:52:23.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabb80caed0 0x7fabb80acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb80cb2b0 0x7fabb81fedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabb815e170 0x7fabb815d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb80ac860 con 0x7fabb80cb2b0 2024-09-13T18:52:23.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb815e730 con 0x7fabb80caed0 2024-09-13T18:52:23.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.029+0000 7fabbd7f6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb80c70c0 con 0x7fabb815e170 2024-09-13T18:52:23.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.033+0000 7fabb77fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabb815e170 0x7fabb815d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.033+0000 7fabb77fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabb815e170 0x7fabb815d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:36062/0 (socket says 172.21.15.146:36062) 2024-09-13T18:52:23.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.033+0000 7fabb77fe700 1 -- 172.21.15.146:0/2301134137 learned_addr learned my addr 172.21.15.146:0/2301134137 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:23.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.033+0000 7fabb67fc700 1 --2- 172.21.15.146:0/2301134137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb80cb2b0 0x7fabb81fedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.033+0000 7fabb77fe700 1 -- 172.21.15.146:0/2301134137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabb80caed0 msgr2=0x7fabb80acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.033+0000 7fabb77fe700 1 --2- 172.21.15.146:0/2301134137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabb80caed0 0x7fabb80acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.033+0000 7fabb77fe700 1 -- 172.21.15.146:0/2301134137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb80cb2b0 msgr2=0x7fabb81fedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.033+0000 7fabb77fe700 1 --2- 172.21.15.146:0/2301134137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb80cb2b0 0x7fabb81fedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.033+0000 7fabb77fe700 1 -- 172.21.15.146:0/2301134137 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba8009570 con 0x7fabb815e170 2024-09-13T18:52:23.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.033+0000 7fabb77fe700 1 --2- 172.21.15.146:0/2301134137 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabb815e170 0x7fabb815d110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fabac00b440 tx=0x7fabac00b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.045+0000 7fab9f7fe700 1 -- 172.21.15.146:0/2301134137 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac011830 con 0x7fabb815e170 2024-09-13T18:52:23.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.045+0000 7fabbd7f6700 1 -- 172.21.15.146:0/2301134137 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb81fca50 con 0x7fabb815e170 2024-09-13T18:52:23.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.049+0000 7fabbd7f6700 1 -- 172.21.15.146:0/2301134137 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb81ff830 con 0x7fabb815e170 2024-09-13T18:52:23.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.049+0000 7fab9f7fe700 1 -- 172.21.15.146:0/2301134137 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac0119b0 con 0x7fabb815e170 2024-09-13T18:52:23.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.049+0000 7fab9f7fe700 1 -- 172.21.15.146:0/2301134137 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac011d60 con 0x7fabb815e170 2024-09-13T18:52:23.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.049+0000 7fabbd7f6700 1 -- 172.21.15.146:0/2301134137 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab98005310 con 0x7fabb815e170 2024-09-13T18:52:23.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.053+0000 7fab9f7fe700 1 -- 172.21.15.146:0/2301134137 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fabac01a400 con 0x7fabb815e170 2024-09-13T18:52:23.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.053+0000 7fab9f7fe700 1 --2- 172.21.15.146:0/2301134137 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faba4066800 0x7faba4068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.053+0000 7fab9f7fe700 1 -- 172.21.15.146:0/2301134137 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7fabac084f20 con 0x7fabb815e170 2024-09-13T18:52:23.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.053+0000 7fabb6ffd700 1 --2- 172.21.15.146:0/2301134137 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faba4066800 0x7faba4068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.053+0000 7fabb6ffd700 1 --2- 172.21.15.146:0/2301134137 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faba4066800 0x7faba4068cc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7faba8009ba0 tx=0x7faba8010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.061+0000 7fab9f7fe700 1 -- 172.21.15.146:0/2301134137 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fabac051a50 con 0x7fabb815e170 2024-09-13T18:52:23.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.941+0000 7fabbd7f6700 1 -- 172.21.15.146:0/2301134137 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab98005890 con 0x7fabb815e170 2024-09-13T18:52:23.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.941+0000 7fab9f7fe700 1 -- 172.21.15.146:0/2301134137 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+6283 (secure 0 0 0) 0x7fabac015020 con 0x7fabb815e170 2024-09-13T18:52:23.966 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:23.967 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:51:11.991743+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:active","state_seq":116,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:23.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.973+0000 7fabbd7f6700 1 -- 172.21.15.146:0/2301134137 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faba4066800 msgr2=0x7faba4068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.973+0000 7fabbd7f6700 1 --2- 172.21.15.146:0/2301134137 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faba4066800 0x7faba4068cc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7faba8009ba0 tx=0x7faba8010000 comp rx=0 tx=0).stop 2024-09-13T18:52:23.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.973+0000 7fabbd7f6700 1 -- 172.21.15.146:0/2301134137 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabb815e170 msgr2=0x7fabb815d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.973+0000 7fabbd7f6700 1 --2- 172.21.15.146:0/2301134137 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabb815e170 0x7fabb815d110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fabac00b440 tx=0x7fabac00b800 comp rx=0 tx=0).stop 2024-09-13T18:52:23.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.973+0000 7fabbd7f6700 1 -- 172.21.15.146:0/2301134137 shutdown_connections 2024-09-13T18:52:23.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.973+0000 7fabbd7f6700 1 --2- 172.21.15.146:0/2301134137 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7faba4066800 0x7faba4068cc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.973+0000 7fabbd7f6700 1 --2- 172.21.15.146:0/2301134137 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fabb815e170 0x7fabb815d110 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.973+0000 7fabbd7f6700 1 --2- 172.21.15.146:0/2301134137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb80cb2b0 0x7fabb81fedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.973+0000 7fabbd7f6700 1 --2- 172.21.15.146:0/2301134137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabb80caed0 0x7fabb80acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.973+0000 7fabbd7f6700 1 -- 172.21.15.146:0/2301134137 >> 172.21.15.146:0/2301134137 conn(0x7fabb8066090 msgr2=0x7fabb80c1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.973+0000 7fabbd7f6700 1 -- 172.21.15.146:0/2301134137 shutdown_connections 2024-09-13T18:52:23.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:23.973+0000 7fabbd7f6700 1 -- 172.21.15.146:0/2301134137 wait complete. 2024-09-13T18:52:23.994 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 97 2024-09-13T18:52:24.044 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:52:26.045 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:26.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.685+0000 7fc2931b4700 1 Processor -- start 2024-09-13T18:52:26.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2931b4700 1 -- start start 2024-09-13T18:52:26.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2931b4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc28c0caed0 0x7fc28c072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2931b4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc28c15e170 0x7fc28c15e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2931b4700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28c161060 con 0x7fc28c0cb2b0 2024-09-13T18:52:26.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2931b4700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc28c0c78d0 con 0x7fc28c15e170 2024-09-13T18:52:26.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2931b4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc28c161300 con 0x7fc28c0caed0 2024-09-13T18:52:26.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2929b3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc28c15e170 0x7fc28c15e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2929b3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc28c15e170 0x7fc28c15e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:56254/0 (socket says 172.21.15.146:56254) 2024-09-13T18:52:26.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2929b3700 1 -- 172.21.15.146:0/1822527278 learned_addr learned my addr 172.21.15.146:0/1822527278 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:26.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2921b2700 1 --2- 172.21.15.146:0/1822527278 >> v2:172.21.15.154:3300/0 conn(0x7fc28c0caed0 0x7fc28c072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2921b2700 1 -- 172.21.15.146:0/1822527278 >> v2:172.21.15.146:3300/0 conn(0x7fc28c15e170 msgr2=0x7fc28c15e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:26.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2921b2700 1 --2- 172.21.15.146:0/1822527278 >> v2:172.21.15.146:3300/0 conn(0x7fc28c15e170 0x7fc28c15e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2921b2700 1 -- 172.21.15.146:0/1822527278 >> v1:172.21.15.146:6789/0 conn(0x7fc28c0cb2b0 legacy=0x7fc28c0c7cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:52:26.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.689+0000 7fc2921b2700 1 -- 172.21.15.146:0/1822527278 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c15eb20 con 0x7fc28c0caed0 2024-09-13T18:52:26.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.693+0000 7fc2921b2700 1 --2- 172.21.15.146:0/1822527278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c0caed0 0x7fc28c072380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc284009d90 tx=0x7fc28400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77fdcc820f0566e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:26.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.693+0000 7fc2911b0700 1 -- 172.21.15.146:0/1822527278 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28400e070 con 0x7fc28c0caed0 2024-09-13T18:52:26.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.693+0000 7fc2911b0700 1 -- 172.21.15.146:0/1822527278 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28400bb50 con 0x7fc28c0caed0 2024-09-13T18:52:26.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.693+0000 7fc2931b4700 1 -- 172.21.15.146:0/1822527278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c0caed0 msgr2=0x7fc28c072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.693+0000 7fc2931b4700 1 --2- 172.21.15.146:0/1822527278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c0caed0 0x7fc28c072380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc284009d90 tx=0x7fc28400b330 comp rx=0 tx=0).stop 2024-09-13T18:52:26.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.697+0000 7fc2931b4700 1 -- 172.21.15.146:0/1822527278 shutdown_connections 2024-09-13T18:52:26.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.697+0000 7fc2931b4700 1 --2- 172.21.15.146:0/1822527278 >> v2:172.21.15.146:3300/0 conn(0x7fc28c15e170 0x7fc28c15e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.697+0000 7fc2931b4700 1 --2- 172.21.15.146:0/1822527278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c0caed0 0x7fc28c072380 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.697+0000 7fc2931b4700 1 -- 172.21.15.146:0/1822527278 >> 172.21.15.146:0/1822527278 conn(0x7fc28c066090 msgr2=0x7fc28c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:26.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.701+0000 7fc2931b4700 1 -- 172.21.15.146:0/1822527278 shutdown_connections 2024-09-13T18:52:26.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.701+0000 7fc2931b4700 1 -- 172.21.15.146:0/1822527278 wait complete. 2024-09-13T18:52:26.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.701+0000 7fc2931b4700 1 Processor -- start 2024-09-13T18:52:26.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2931b4700 1 -- start start 2024-09-13T18:52:26.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2931b4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc28c0caed0 0x7fc28c14eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2931b4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c0cb2b0 0x7fc28c1488a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2931b4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc28c15e170 0x7fc28c148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2931b4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c14f910 con 0x7fc28c15e170 2024-09-13T18:52:26.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2931b4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c0ace70 con 0x7fc28c0cb2b0 2024-09-13T18:52:26.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2931b4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c15f530 con 0x7fc28c0caed0 2024-09-13T18:52:26.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2921b2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc28c0caed0 0x7fc28c14eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2921b2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc28c0caed0 0x7fc28c14eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:36066/0 (socket says 172.21.15.146:36066) 2024-09-13T18:52:26.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2921b2700 1 -- 172.21.15.146:0/2844127957 learned_addr learned my addr 172.21.15.146:0/2844127957 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:26.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2919b1700 1 --2- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c0cb2b0 0x7fc28c1488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2921b2700 1 -- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c0cb2b0 msgr2=0x7fc28c1488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2921b2700 1 --2- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c0cb2b0 0x7fc28c1488a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2921b2700 1 -- 172.21.15.146:0/2844127957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc28c15e170 msgr2=0x7fc28c148de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:52:26.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2921b2700 1 --2- 172.21.15.146:0/2844127957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc28c15e170 0x7fc28c148de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2921b2700 1 -- 172.21.15.146:0/2844127957 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc284009a70 con 0x7fc28c0caed0 2024-09-13T18:52:26.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.705+0000 7fc2921b2700 1 --2- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc28c0caed0 0x7fc28c14eec0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc28400b730 tx=0x7fc284004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:26.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.721+0000 7fc282ffd700 1 -- 172.21.15.146:0/2844127957 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284012630 con 0x7fc28c0caed0 2024-09-13T18:52:26.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.721+0000 7fc282ffd700 1 -- 172.21.15.146:0/2844127957 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2840127b0 con 0x7fc28c0caed0 2024-09-13T18:52:26.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.721+0000 7fc282ffd700 1 -- 172.21.15.146:0/2844127957 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284012b60 con 0x7fc28c0caed0 2024-09-13T18:52:26.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.721+0000 7fc2931b4700 1 -- 172.21.15.146:0/2844127957 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28c0acbd0 con 0x7fc28c0caed0 2024-09-13T18:52:26.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.721+0000 7fc2931b4700 1 -- 172.21.15.146:0/2844127957 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28c0ad230 con 0x7fc28c0caed0 2024-09-13T18:52:26.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.721+0000 7fc282ffd700 1 -- 172.21.15.146:0/2844127957 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc284012cc0 con 0x7fc28c0caed0 2024-09-13T18:52:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.725+0000 7fc282ffd700 1 --2- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2700667b0 0x7fc270068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.725+0000 7fc282ffd700 1 -- 172.21.15.146:0/2844127957 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7fc284086b10 con 0x7fc28c0caed0 2024-09-13T18:52:26.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.729+0000 7fc2919b1700 1 --2- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2700667b0 0x7fc270068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.733+0000 7fc2919b1700 1 --2- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2700667b0 0x7fc270068c70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc28c152130 tx=0x7fc27c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:26.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.733+0000 7fc2931b4700 1 -- 172.21.15.146:0/2844127957 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc274005310 con 0x7fc28c0caed0 2024-09-13T18:52:26.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:26.745+0000 7fc282ffd700 1 -- 172.21.15.146:0/2844127957 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc284053640 con 0x7fc28c0caed0 2024-09-13T18:52:27.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.553+0000 7fc2931b4700 1 -- 172.21.15.146:0/2844127957 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc274005b40 con 0x7fc28c0caed0 2024-09-13T18:52:27.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.553+0000 7fc282ffd700 1 -- 172.21.15.146:0/2844127957 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+6283 (secure 0 0 0) 0x7fc284022020 con 0x7fc28c0caed0 2024-09-13T18:52:27.569 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:27.569 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:51:11.991743+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:active","state_seq":116,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:27.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.569+0000 7fc2931b4700 1 -- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2700667b0 msgr2=0x7fc270068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.569+0000 7fc2931b4700 1 --2- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2700667b0 0x7fc270068c70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc28c152130 tx=0x7fc27c008040 comp rx=0 tx=0).stop 2024-09-13T18:52:27.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.569+0000 7fc2931b4700 1 -- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc28c0caed0 msgr2=0x7fc28c14eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.569+0000 7fc2931b4700 1 --2- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc28c0caed0 0x7fc28c14eec0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc28400b730 tx=0x7fc284004af0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.573+0000 7fc2931b4700 1 -- 172.21.15.146:0/2844127957 shutdown_connections 2024-09-13T18:52:27.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.573+0000 7fc2931b4700 1 --2- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2700667b0 0x7fc270068c70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.573+0000 7fc2931b4700 1 --2- 172.21.15.146:0/2844127957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc28c15e170 0x7fc28c148de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.573+0000 7fc2931b4700 1 --2- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c0cb2b0 0x7fc28c1488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.573+0000 7fc2931b4700 1 --2- 172.21.15.146:0/2844127957 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc28c0caed0 0x7fc28c14eec0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.573+0000 7fc2931b4700 1 -- 172.21.15.146:0/2844127957 >> 172.21.15.146:0/2844127957 conn(0x7fc28c066090 msgr2=0x7fc28c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:27.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.573+0000 7fc2931b4700 1 -- 172.21.15.146:0/2844127957 shutdown_connections 2024-09-13T18:52:27.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:27.577+0000 7fc2931b4700 1 -- 172.21.15.146:0/2844127957 wait complete. 2024-09-13T18:52:27.617 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 97 2024-09-13T18:52:27.655 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:52:29.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:30.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.065+0000 7fdd56d0a700 1 Processor -- start 2024-09-13T18:52:30.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.065+0000 7fdd56d0a700 1 -- start start 2024-09-13T18:52:30.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.065+0000 7fdd56d0a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd500cac60 0x7fdd500cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.065+0000 7fdd56d0a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd500a9920 con 0x7fdd5008baa0 2024-09-13T18:52:30.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.065+0000 7fdd56d0a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd50070070 con 0x7fdd5016e0c0 2024-09-13T18:52:30.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.065+0000 7fdd56d0a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500723f0 con 0x7fdd500cac60 2024-09-13T18:52:30.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.065+0000 7fdd4ffff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fdd5008baa0 0x7fdd5006f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35330/0 (socket says 172.21.15.146:35330) 2024-09-13T18:52:30.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.065+0000 7fdd4ffff700 1 -- 172.21.15.146:0/3776567230 learned_addr learned my addr 172.21.15.146:0/3776567230 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:52:30.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.069+0000 7fdd54aa6700 1 --2- 172.21.15.146:0/3776567230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd500cac60 0x7fdd500cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.069+0000 7fdd4f7fe700 1 -- 172.21.15.146:0/3776567230 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671157295 0 0) 0x7fdd500a9920 con 0x7fdd5008baa0 2024-09-13T18:52:30.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.069+0000 7fdd4f7fe700 1 -- 172.21.15.146:0/3776567230 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd380035d0 con 0x7fdd5008baa0 2024-09-13T18:52:30.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.069+0000 7fdd54aa6700 1 -- 172.21.15.146:0/3776567230 >> v1:172.21.15.154:6789/0 conn(0x7fdd5016e0c0 legacy=0x7fdd501704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.069+0000 7fdd54aa6700 1 -- 172.21.15.146:0/3776567230 >> v1:172.21.15.146:6789/0 conn(0x7fdd5008baa0 legacy=0x7fdd5006f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.069+0000 7fdd54aa6700 1 -- 172.21.15.146:0/3776567230 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd500725e0 con 0x7fdd500cac60 2024-09-13T18:52:30.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.073+0000 7fdd54aa6700 1 --2- 172.21.15.146:0/3776567230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd500cac60 0x7fdd500cb040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd400088b0 tx=0x7fdd4000d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75e27d7779ae2c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:30.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.073+0000 7fdd4f7fe700 1 -- 172.21.15.146:0/3776567230 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40010070 con 0x7fdd500cac60 2024-09-13T18:52:30.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.073+0000 7fdd4f7fe700 1 -- 172.21.15.146:0/3776567230 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd40008a00 con 0x7fdd500cac60 2024-09-13T18:52:30.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.077+0000 7fdd56d0a700 1 -- 172.21.15.146:0/3776567230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd500cac60 msgr2=0x7fdd500cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.077+0000 7fdd56d0a700 1 --2- 172.21.15.146:0/3776567230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd500cac60 0x7fdd500cb040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd400088b0 tx=0x7fdd4000d330 comp rx=0 tx=0).stop 2024-09-13T18:52:30.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.077+0000 7fdd56d0a700 1 -- 172.21.15.146:0/3776567230 shutdown_connections 2024-09-13T18:52:30.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.077+0000 7fdd56d0a700 1 --2- 172.21.15.146:0/3776567230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd500cac60 0x7fdd500cb040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.077+0000 7fdd56d0a700 1 -- 172.21.15.146:0/3776567230 >> 172.21.15.146:0/3776567230 conn(0x7fdd50066090 msgr2=0x7fdd50066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:30.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.077+0000 7fdd56d0a700 1 -- 172.21.15.146:0/3776567230 shutdown_connections 2024-09-13T18:52:30.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.077+0000 7fdd56d0a700 1 -- 172.21.15.146:0/3776567230 wait complete. 2024-09-13T18:52:30.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.077+0000 7fdd56d0a700 1 Processor -- start 2024-09-13T18:52:30.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd56d0a700 1 -- start start 2024-09-13T18:52:30.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd56d0a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd5008baa0 0x7fdd501ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd56d0a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd500cac60 0x7fdd50164940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd56d0a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd5016e0c0 0x7fdd50169660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd56d0a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd50200190 con 0x7fdd5008baa0 2024-09-13T18:52:30.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd56d0a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd501fc5e0 con 0x7fdd5016e0c0 2024-09-13T18:52:30.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd56d0a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd501fbcd0 con 0x7fdd500cac60 2024-09-13T18:52:30.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd4ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd500cac60 0x7fdd50164940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd4ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd500cac60 0x7fdd50164940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:36078/0 (socket says 172.21.15.146:36078) 2024-09-13T18:52:30.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd4ffff700 1 -- 172.21.15.146:0/1645596070 learned_addr learned my addr 172.21.15.146:0/1645596070 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:30.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd552a7700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd5016e0c0 0x7fdd50169660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd54aa6700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd5008baa0 0x7fdd501ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd4ffff700 1 -- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd5016e0c0 msgr2=0x7fdd50169660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd4ffff700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd5016e0c0 0x7fdd50169660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd4ffff700 1 -- 172.21.15.146:0/1645596070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd5008baa0 msgr2=0x7fdd501ffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd4ffff700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd5008baa0 0x7fdd501ffc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd4ffff700 1 -- 172.21.15.146:0/1645596070 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd40008540 con 0x7fdd500cac60 2024-09-13T18:52:30.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd552a7700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd5016e0c0 0x7fdd50169660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:30.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd4ffff700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd500cac60 0x7fdd50164940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800a180 tx=0x7fdd4800a490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:30.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd4d7fa700 1 -- 172.21.15.146:0/1645596070 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48007370 con 0x7fdd500cac60 2024-09-13T18:52:30.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd56d0a700 1 -- 172.21.15.146:0/1645596070 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5016c8f0 con 0x7fdd500cac60 2024-09-13T18:52:30.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.081+0000 7fdd56d0a700 1 -- 172.21.15.146:0/1645596070 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd501fbf90 con 0x7fdd500cac60 2024-09-13T18:52:30.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.085+0000 7fdd4d7fa700 1 -- 172.21.15.146:0/1645596070 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd480074f0 con 0x7fdd500cac60 2024-09-13T18:52:30.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.085+0000 7fdd4d7fa700 1 -- 172.21.15.146:0/1645596070 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd480078a0 con 0x7fdd500cac60 2024-09-13T18:52:30.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.085+0000 7fdd56d0a700 1 -- 172.21.15.146:0/1645596070 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd500c9da0 con 0x7fdd500cac60 2024-09-13T18:52:30.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.085+0000 7fdd4d7fa700 1 -- 172.21.15.146:0/1645596070 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdd48001770 con 0x7fdd500cac60 2024-09-13T18:52:30.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.085+0000 7fdd4d7fa700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdd3c06ef40 0x7fdd3c071400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.085+0000 7fdd4d7fa700 1 -- 172.21.15.146:0/1645596070 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7fdd48084a30 con 0x7fdd500cac60 2024-09-13T18:52:30.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.085+0000 7fdd54aa6700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdd3c06ef40 0x7fdd3c071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.085+0000 7fdd54aa6700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdd3c06ef40 0x7fdd3c071400 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdd40008880 tx=0x7fdd4000a040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:30.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.089+0000 7fdd4d7fa700 1 -- 172.21.15.146:0/1645596070 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdd480516c0 con 0x7fdd500cac60 2024-09-13T18:52:30.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.457+0000 7fdd56d0a700 1 -- 172.21.15.146:0/1645596070 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd500aa2b0 con 0x7fdd500cac60 2024-09-13T18:52:30.475 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:30.475 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:51:11.991743+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:active","state_seq":116,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:30.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.461+0000 7fdd4d7fa700 1 -- 172.21.15.146:0/1645596070 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+6283 (secure 0 0 0) 0x7fdd48017020 con 0x7fdd500cac60 2024-09-13T18:52:30.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.461+0000 7fdd56d0a700 1 -- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdd3c06ef40 msgr2=0x7fdd3c071400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.461+0000 7fdd56d0a700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdd3c06ef40 0x7fdd3c071400 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdd40008880 tx=0x7fdd4000a040 comp rx=0 tx=0).stop 2024-09-13T18:52:30.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.461+0000 7fdd56d0a700 1 -- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd500cac60 msgr2=0x7fdd50164940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.461+0000 7fdd56d0a700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd500cac60 0x7fdd50164940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800a180 tx=0x7fdd4800a490 comp rx=0 tx=0).stop 2024-09-13T18:52:30.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.461+0000 7fdd56d0a700 1 -- 172.21.15.146:0/1645596070 shutdown_connections 2024-09-13T18:52:30.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.461+0000 7fdd56d0a700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdd3c06ef40 0x7fdd3c071400 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.461+0000 7fdd56d0a700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd5016e0c0 0x7fdd50169660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.461+0000 7fdd56d0a700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd500cac60 0x7fdd50164940 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.461+0000 7fdd56d0a700 1 --2- 172.21.15.146:0/1645596070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd5008baa0 0x7fdd501ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.461+0000 7fdd56d0a700 1 -- 172.21.15.146:0/1645596070 >> 172.21.15.146:0/1645596070 conn(0x7fdd50066090 msgr2=0x7fdd500c5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:30.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.461+0000 7fdd56d0a700 1 -- 172.21.15.146:0/1645596070 shutdown_connections 2024-09-13T18:52:30.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:30.461+0000 7fdd56d0a700 1 -- 172.21.15.146:0/1645596070 wait complete. 2024-09-13T18:52:30.502 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 97 2024-09-13T18:52:30.524 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:52:32.525 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:33.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.129+0000 7fc2eb2c6700 1 Processor -- start 2024-09-13T18:52:33.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.129+0000 7fc2eb2c6700 1 -- start start 2024-09-13T18:52:33.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.129+0000 7fc2eb2c6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2e40caed0 0x7fc2e40723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:33.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.129+0000 7fc2eb2c6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc2e40cb2b0 0x7fc2e41619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:33.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.129+0000 7fc2eb2c6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e4161550 con 0x7fc2e40a9840 2024-09-13T18:52:33.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.129+0000 7fc2eb2c6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e4072a30 con 0x7fc2e40cb2b0 2024-09-13T18:52:33.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.129+0000 7fc2eb2c6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e40aa800 con 0x7fc2e40caed0 2024-09-13T18:52:33.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2ea2c4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2e40caed0 0x7fc2e40723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:33.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2ea2c4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2e40caed0 0x7fc2e40723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:54652/0 (socket says 172.21.15.146:54652) 2024-09-13T18:52:33.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2ea2c4700 1 -- 172.21.15.146:0/4137530442 learned_addr learned my addr 172.21.15.146:0/4137530442 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:33.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2e9ac3700 1 --2- 172.21.15.146:0/4137530442 >> v2:172.21.15.154:3300/0 conn(0x7fc2e40cb2b0 0x7fc2e41619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:33.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2e9ac3700 1 -- 172.21.15.146:0/4137530442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2e40caed0 msgr2=0x7fc2e40723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:33.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2e9ac3700 1 --2- 172.21.15.146:0/4137530442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2e40caed0 0x7fc2e40723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:33.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2e9ac3700 1 -- 172.21.15.146:0/4137530442 >> v1:172.21.15.154:6789/0 conn(0x7fc2e40a9840 legacy=0x7fc2e415e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:33.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2e9ac3700 1 -- 172.21.15.146:0/4137530442 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e4161220 con 0x7fc2e40cb2b0 2024-09-13T18:52:33.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2e9ac3700 1 --2- 172.21.15.146:0/4137530442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2e40cb2b0 0x7fc2e41619e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009850 tx=0x7fc2d4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc675fc27be597e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:33.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2e92c2700 1 -- 172.21.15.146:0/4137530442 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4004230 con 0x7fc2e40cb2b0 2024-09-13T18:52:33.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2e92c2700 1 -- 172.21.15.146:0/4137530442 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d4004390 con 0x7fc2e40cb2b0 2024-09-13T18:52:33.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2e92c2700 1 -- 172.21.15.146:0/4137530442 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d40036c0 con 0x7fc2e40cb2b0 2024-09-13T18:52:33.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/4137530442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2e40cb2b0 msgr2=0x7fc2e41619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:33.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.133+0000 7fc2eb2c6700 1 --2- 172.21.15.146:0/4137530442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2e40cb2b0 0x7fc2e41619e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009850 tx=0x7fc2d4009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:33.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/4137530442 shutdown_connections 2024-09-13T18:52:33.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 --2- 172.21.15.146:0/4137530442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2e40cb2b0 0x7fc2e41619e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:33.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 --2- 172.21.15.146:0/4137530442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2e40caed0 0x7fc2e40723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:33.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/4137530442 >> 172.21.15.146:0/4137530442 conn(0x7fc2e4066090 msgr2=0x7fc2e4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:33.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/4137530442 shutdown_connections 2024-09-13T18:52:33.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/4137530442 wait complete. 2024-09-13T18:52:33.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 Processor -- start 2024-09-13T18:52:33.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 -- start start 2024-09-13T18:52:33.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2e40a9840 0x7fc2e41ea4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:33.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2e40caed0 0x7fc2e40aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:33.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2e40cb2b0 0x7fc2e40ab140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:33.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e40afa10 con 0x7fc2e40caed0 2024-09-13T18:52:33.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e4157320 con 0x7fc2e40cb2b0 2024-09-13T18:52:33.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.137+0000 7fc2eb2c6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e40ae700 con 0x7fc2e40a9840 2024-09-13T18:52:33.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.141+0000 7fc2e9ac3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2e40caed0 0x7fc2e40aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:33.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.141+0000 7fc2e9ac3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2e40caed0 0x7fc2e40aac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40822/0 (socket says 172.21.15.146:40822) 2024-09-13T18:52:33.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.141+0000 7fc2e9ac3700 1 -- 172.21.15.146:0/647644327 learned_addr learned my addr 172.21.15.146:0/647644327 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:33.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.141+0000 7fc2ea2c4700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2e40a9840 0x7fc2e41ea4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:33.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.141+0000 7fc2e9ac3700 1 -- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2e40a9840 msgr2=0x7fc2e41ea4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:33.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.141+0000 7fc2e9ac3700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2e40a9840 0x7fc2e41ea4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:33.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.141+0000 7fc2e9ac3700 1 -- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2e40cb2b0 msgr2=0x7fc2e40ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:33.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.141+0000 7fc2eaac5700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2e40cb2b0 0x7fc2e40ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:33.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.141+0000 7fc2e9ac3700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2e40cb2b0 0x7fc2e40ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:33.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.141+0000 7fc2e9ac3700 1 -- 172.21.15.146:0/647644327 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d40094b0 con 0x7fc2e40caed0 2024-09-13T18:52:33.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.141+0000 7fc2eaac5700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2e40cb2b0 0x7fc2e40ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:33.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.145+0000 7fc2e9ac3700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2e40caed0 0x7fc2e40aac00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40090b0 tx=0x7fc2d4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:33.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.145+0000 7fc2daffd700 1 -- 172.21.15.146:0/647644327 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4006ea0 con 0x7fc2e40caed0 2024-09-13T18:52:33.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.149+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/647644327 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e40ae9a0 con 0x7fc2e40caed0 2024-09-13T18:52:33.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.149+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/647644327 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e40af0f0 con 0x7fc2e40caed0 2024-09-13T18:52:33.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.149+0000 7fc2daffd700 1 -- 172.21.15.146:0/647644327 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d401f070 con 0x7fc2e40caed0 2024-09-13T18:52:33.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.149+0000 7fc2daffd700 1 -- 172.21.15.146:0/647644327 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4011a30 con 0x7fc2e40caed0 2024-09-13T18:52:33.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.149+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/647644327 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e40ca4f0 con 0x7fc2e40caed0 2024-09-13T18:52:33.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.153+0000 7fc2daffd700 1 -- 172.21.15.146:0/647644327 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc2d4021030 con 0x7fc2e40caed0 2024-09-13T18:52:33.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.157+0000 7fc2daffd700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2c8066800 0x7fc2c8068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:33.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.157+0000 7fc2ea2c4700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2c8066800 0x7fc2c8068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:33.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.157+0000 7fc2daffd700 1 -- 172.21.15.146:0/647644327 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7fc2d4086030 con 0x7fc2e40caed0 2024-09-13T18:52:33.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.157+0000 7fc2ea2c4700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2c8066800 0x7fc2c8068cc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009c40 tx=0x7fc2dc00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:33.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.161+0000 7fc2daffd700 1 -- 172.21.15.146:0/647644327 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc2d40b7db0 con 0x7fc2e40caed0 2024-09-13T18:52:33.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.557+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/647644327 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2e40ca4f0 con 0x7fc2e40caed0 2024-09-13T18:52:33.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.557+0000 7fc2daffd700 1 -- 172.21.15.146:0/647644327 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+6283 (secure 0 0 0) 0x7fc2d4052bf0 con 0x7fc2e40caed0 2024-09-13T18:52:33.572 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:33.572 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:51:11.991743+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6187,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6187":{"gid":6187,"name":"e","rank":0,"incarnation":92,"state":"up:active","state_seq":116,"addr":"172.21.15.146:6833/3023779168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3023779168},{"type":"v1","addr":"172.21.15.146:6833","nonce":3023779168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:33.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.561+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2c8066800 msgr2=0x7fc2c8068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:33.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.561+0000 7fc2eb2c6700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2c8066800 0x7fc2c8068cc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009c40 tx=0x7fc2dc00b040 comp rx=0 tx=0).stop 2024-09-13T18:52:33.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.561+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/647644327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2e40caed0 msgr2=0x7fc2e40aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:33.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.561+0000 7fc2eb2c6700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2e40caed0 0x7fc2e40aac00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40090b0 tx=0x7fc2d4004770 comp rx=0 tx=0).stop 2024-09-13T18:52:33.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.561+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/647644327 shutdown_connections 2024-09-13T18:52:33.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.561+0000 7fc2eb2c6700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc2c8066800 0x7fc2c8068cc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:33.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.561+0000 7fc2eb2c6700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2e40cb2b0 0x7fc2e40ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:33.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.561+0000 7fc2eb2c6700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2e40caed0 0x7fc2e40aac00 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:33.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.561+0000 7fc2eb2c6700 1 --2- 172.21.15.146:0/647644327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2e40a9840 0x7fc2e41ea4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:33.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.561+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/647644327 >> 172.21.15.146:0/647644327 conn(0x7fc2e4066090 msgr2=0x7fc2e415f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:33.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.561+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/647644327 shutdown_connections 2024-09-13T18:52:33.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:33.561+0000 7fc2eb2c6700 1 -- 172.21.15.146:0/647644327 wait complete. 2024-09-13T18:52:33.582 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 97 2024-09-13T18:52:33.596 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-09-13T18:52:35.597 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:35.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.757+0000 7efcb37eb700 1 Processor -- start 2024-09-13T18:52:35.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.757+0000 7efcb37eb700 1 -- start start 2024-09-13T18:52:35.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.757+0000 7efcb37eb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac06c7e0 0x7efcac0c01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:35.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.757+0000 7efcb37eb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efcac156470 0x7efcac153120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:35.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.757+0000 7efcb37eb700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcac159060 con 0x7efcac06e9c0 2024-09-13T18:52:35.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.757+0000 7efcb37eb700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcac159f50 con 0x7efcac156470 2024-09-13T18:52:35.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.757+0000 7efcb37eb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac1592d0 con 0x7efcac06c7e0 2024-09-13T18:52:35.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb0d86700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7efcac06e9c0 0x7efcac155d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48472/0 (socket says 172.21.15.146:48472) 2024-09-13T18:52:35.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb0d86700 1 -- 172.21.15.146:0/2273155370 learned_addr learned my addr 172.21.15.146:0/2273155370 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:52:35.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb1587700 1 --2- 172.21.15.146:0/2273155370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac06c7e0 0x7efcac0c01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:35.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb1d88700 1 --2- 172.21.15.146:0/2273155370 >> v2:172.21.15.146:3300/0 conn(0x7efcac156470 0x7efcac153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:35.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb1d88700 1 -- 172.21.15.146:0/2273155370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac06c7e0 msgr2=0x7efcac0c01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:35.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb1d88700 1 --2- 172.21.15.146:0/2273155370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac06c7e0 0x7efcac0c01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:35.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb1d88700 1 -- 172.21.15.146:0/2273155370 >> v1:172.21.15.146:6789/0 conn(0x7efcac06e9c0 legacy=0x7efcac155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:35.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb1d88700 1 -- 172.21.15.146:0/2273155370 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac1595b0 con 0x7efcac156470 2024-09-13T18:52:35.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb1587700 1 --2- 172.21.15.146:0/2273155370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac06c7e0 0x7efcac0c01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:35.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb1d88700 1 --2- 172.21.15.146:0/2273155370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcac156470 0x7efcac153120 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009ab0 tx=0x7efc9c009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2a4b92bee8deab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:35.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efca3fff700 1 -- 172.21.15.146:0/2273155370 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c004230 con 0x7efcac156470 2024-09-13T18:52:35.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efca3fff700 1 -- 172.21.15.146:0/2273155370 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c004390 con 0x7efcac156470 2024-09-13T18:52:35.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efca3fff700 1 -- 172.21.15.146:0/2273155370 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c0036c0 con 0x7efcac156470 2024-09-13T18:52:35.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb37eb700 1 -- 172.21.15.146:0/2273155370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcac156470 msgr2=0x7efcac153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:35.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb37eb700 1 --2- 172.21.15.146:0/2273155370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcac156470 0x7efcac153120 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009ab0 tx=0x7efc9c009dc0 comp rx=0 tx=0).stop 2024-09-13T18:52:35.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb37eb700 1 -- 172.21.15.146:0/2273155370 shutdown_connections 2024-09-13T18:52:35.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb37eb700 1 --2- 172.21.15.146:0/2273155370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcac156470 0x7efcac153120 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:35.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb37eb700 1 --2- 172.21.15.146:0/2273155370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac06c7e0 0x7efcac0c01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:35.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb37eb700 1 -- 172.21.15.146:0/2273155370 >> 172.21.15.146:0/2273155370 conn(0x7efcac0cb1d0 msgr2=0x7efcac0cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:35.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb37eb700 1 -- 172.21.15.146:0/2273155370 shutdown_connections 2024-09-13T18:52:35.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.761+0000 7efcb37eb700 1 -- 172.21.15.146:0/2273155370 wait complete. 2024-09-13T18:52:35.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb37eb700 1 Processor -- start 2024-09-13T18:52:35.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb37eb700 1 -- start start 2024-09-13T18:52:35.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb37eb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac06c7e0 0x7efcac0c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:35.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb37eb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac06e9c0 0x7efcac1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:35.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb37eb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcac156470 0x7efcac1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:35.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb37eb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac1498d0 con 0x7efcac156470 2024-09-13T18:52:35.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb37eb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0c9f70 con 0x7efcac06e9c0 2024-09-13T18:52:35.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb37eb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0ca6e0 con 0x7efcac06c7e0 2024-09-13T18:52:35.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb0d86700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac06e9c0 0x7efcac1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:35.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb0d86700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac06e9c0 0x7efcac1544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:34108/0 (socket says 172.21.15.146:34108) 2024-09-13T18:52:35.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb0d86700 1 -- 172.21.15.146:0/889359169 learned_addr learned my addr 172.21.15.146:0/889359169 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:35.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb1587700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac06c7e0 0x7efcac0c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:35.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb1d88700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcac156470 0x7efcac1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:35.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb0d86700 1 -- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac06c7e0 msgr2=0x7efcac0c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:35.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb0d86700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac06c7e0 0x7efcac0c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:35.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb0d86700 1 -- 172.21.15.146:0/889359169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcac156470 msgr2=0x7efcac1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:35.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb0d86700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcac156470 0x7efcac1549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:35.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb0d86700 1 -- 172.21.15.146:0/889359169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca80094b0 con 0x7efcac06e9c0 2024-09-13T18:52:35.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb1587700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac06c7e0 0x7efcac0c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:35.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb1d88700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcac156470 0x7efcac1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:35.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb0d86700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac06e9c0 0x7efcac1544a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efca4002050 tx=0x7efca400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:35.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efca1ffb700 1 -- 172.21.15.146:0/889359169 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca400ac90 con 0x7efcac06e9c0 2024-09-13T18:52:35.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efca1ffb700 1 -- 172.21.15.146:0/889359169 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca400adf0 con 0x7efcac06e9c0 2024-09-13T18:52:35.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb37eb700 1 -- 172.21.15.146:0/889359169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc9c0094b0 con 0x7efcac06e9c0 2024-09-13T18:52:35.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efca1ffb700 1 -- 172.21.15.146:0/889359169 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca4007310 con 0x7efcac06e9c0 2024-09-13T18:52:35.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.765+0000 7efcb37eb700 1 -- 172.21.15.146:0/889359169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcac150200 con 0x7efcac06e9c0 2024-09-13T18:52:35.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.769+0000 7efca1ffb700 1 -- 172.21.15.146:0/889359169 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efca4007470 con 0x7efcac06e9c0 2024-09-13T18:52:35.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.769+0000 7efcb37eb700 1 -- 172.21.15.146:0/889359169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcac0c01f0 con 0x7efcac06e9c0 2024-09-13T18:52:35.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.769+0000 7efca1ffb700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc90066760 0x7efc90068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:35.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.769+0000 7efca1ffb700 1 -- 172.21.15.146:0/889359169 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7781+0+0 (secure 0 0 0) 0x7efca40845d0 con 0x7efcac06e9c0 2024-09-13T18:52:35.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.769+0000 7efcb1587700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc90066760 0x7efc90068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:35.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.769+0000 7efcb1587700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc90066760 0x7efc90068c20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efca80114b0 tx=0x7efca8009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:35.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:35.781+0000 7efca1ffb700 1 -- 172.21.15.146:0/889359169 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efca40510c0 con 0x7efcac06e9c0 2024-09-13T18:52:36.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.137+0000 7efcb37eb700 1 -- 172.21.15.146:0/889359169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcac0c1f70 con 0x7efcac06e9c0 2024-09-13T18:52:36.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.137+0000 7efca1ffb700 1 -- 172.21.15.146:0/889359169 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+5521 (secure 0 0 0) 0x7efca4018090 con 0x7efcac06e9c0 2024-09-13T18:52:36.154 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:36.155 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:52:35.904588+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6984,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:36.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.141+0000 7efcb37eb700 1 -- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc90066760 msgr2=0x7efc90068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.141+0000 7efcb37eb700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc90066760 0x7efc90068c20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efca80114b0 tx=0x7efca8009240 comp rx=0 tx=0).stop 2024-09-13T18:52:36.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.141+0000 7efcb37eb700 1 -- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac06e9c0 msgr2=0x7efcac1544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.141+0000 7efcb37eb700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac06e9c0 0x7efcac1544a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efca4002050 tx=0x7efca400a340 comp rx=0 tx=0).stop 2024-09-13T18:52:36.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.141+0000 7efcb37eb700 1 -- 172.21.15.146:0/889359169 shutdown_connections 2024-09-13T18:52:36.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.141+0000 7efcb37eb700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efc90066760 0x7efc90068c20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.141+0000 7efcb37eb700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcac156470 0x7efcac1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.141+0000 7efcb37eb700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac06e9c0 0x7efcac1544a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.141+0000 7efcb37eb700 1 --2- 172.21.15.146:0/889359169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac06c7e0 0x7efcac0c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.141+0000 7efcb37eb700 1 -- 172.21.15.146:0/889359169 >> 172.21.15.146:0/889359169 conn(0x7efcac0cb1d0 msgr2=0x7efcac0c0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:36.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.141+0000 7efcb37eb700 1 -- 172.21.15.146:0/889359169 shutdown_connections 2024-09-13T18:52:36.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.141+0000 7efcb37eb700 1 -- 172.21.15.146:0/889359169 wait complete. 2024-09-13T18:52:36.159 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 98 2024-09-13T18:52:36.174 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2024-09-13T18:52:36.174 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:52:36.174 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:36.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.333+0000 7f13684fc700 1 Processor -- start 2024-09-13T18:52:36.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.333+0000 7f13684fc700 1 -- start start 2024-09-13T18:52:36.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.333+0000 7f13684fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f13600cacd0 0x7f13600cb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:36.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.333+0000 7f13684fc700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13600c98e0 con 0x7f13600a9490 2024-09-13T18:52:36.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.333+0000 7f13684fc700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136006f5c0 con 0x7f13600c9500 2024-09-13T18:52:36.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.333+0000 7f13684fc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1360155420 con 0x7f13600cacd0 2024-09-13T18:52:36.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.333+0000 7f1366a99700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f13600cacd0 0x7f13600cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:36.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.333+0000 7f1366a99700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f13600cacd0 0x7f13600cb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:34116/0 (socket says 172.21.15.146:34116) 2024-09-13T18:52:36.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.333+0000 7f1366a99700 1 -- 172.21.15.146:0/3941571830 learned_addr learned my addr 172.21.15.146:0/3941571830 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:36.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.337+0000 7f1366a99700 1 -- 172.21.15.146:0/3941571830 >> v1:172.21.15.154:6789/0 conn(0x7f13600c9500 legacy=0x7f136014f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.337+0000 7f1366a99700 1 -- 172.21.15.146:0/3941571830 >> v1:172.21.15.146:6789/0 conn(0x7f13600a9490 legacy=0x7f1360155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.337+0000 7f1366a99700 1 -- 172.21.15.146:0/3941571830 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13600c7b70 con 0x7f13600cacd0 2024-09-13T18:52:36.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.337+0000 7f1366a99700 1 --2- 172.21.15.146:0/3941571830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13600cacd0 0x7f13600cb140 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f135c009940 tx=0x7f135c009c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6b8458c7cdebee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:36.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.337+0000 7f1365296700 1 -- 172.21.15.146:0/3941571830 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c00ba20 con 0x7f13600cacd0 2024-09-13T18:52:36.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.337+0000 7f1365296700 1 -- 172.21.15.146:0/3941571830 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c009e90 con 0x7f13600cacd0 2024-09-13T18:52:36.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.337+0000 7f1365296700 1 -- 172.21.15.146:0/3941571830 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c00bd80 con 0x7f13600cacd0 2024-09-13T18:52:36.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.337+0000 7f13684fc700 1 -- 172.21.15.146:0/3941571830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13600cacd0 msgr2=0x7f13600cb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.337+0000 7f13684fc700 1 --2- 172.21.15.146:0/3941571830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13600cacd0 0x7f13600cb140 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f135c009940 tx=0x7f135c009c50 comp rx=0 tx=0).stop 2024-09-13T18:52:36.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.337+0000 7f13684fc700 1 -- 172.21.15.146:0/3941571830 shutdown_connections 2024-09-13T18:52:36.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.337+0000 7f13684fc700 1 --2- 172.21.15.146:0/3941571830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13600cacd0 0x7f13600cb140 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.337+0000 7f13684fc700 1 -- 172.21.15.146:0/3941571830 >> 172.21.15.146:0/3941571830 conn(0x7f1360066090 msgr2=0x7f1360066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:36.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.341+0000 7f13684fc700 1 -- 172.21.15.146:0/3941571830 shutdown_connections 2024-09-13T18:52:36.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.341+0000 7f13684fc700 1 -- 172.21.15.146:0/3941571830 wait complete. 2024-09-13T18:52:36.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.341+0000 7f13684fc700 1 Processor -- start 2024-09-13T18:52:36.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f13684fc700 1 -- start start 2024-09-13T18:52:36.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f13684fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13600a9490 0x7f13600c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:36.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f13684fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13600c9500 0x7f13601fce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:36.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f13684fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13600cacd0 0x7f13601fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:36.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f13684fc700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13600ad240 con 0x7f13600c9500 2024-09-13T18:52:36.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f13684fc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13601fc6f0 con 0x7f13600a9490 2024-09-13T18:52:36.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f13684fc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13600ab990 con 0x7f13600cacd0 2024-09-13T18:52:36.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1366a99700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13600cacd0 0x7f13601fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:36.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1366298700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13600a9490 0x7f13600c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:36.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1366298700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13600a9490 0x7f13600c48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:34118/0 (socket says 172.21.15.146:34118) 2024-09-13T18:52:36.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1366298700 1 -- 172.21.15.146:0/1110623051 learned_addr learned my addr 172.21.15.146:0/1110623051 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:36.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1365a97700 1 --2- 172.21.15.146:0/1110623051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13600c9500 0x7f13601fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:36.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1366298700 1 -- 172.21.15.146:0/1110623051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13600cacd0 msgr2=0x7f13601fb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1366298700 1 --2- 172.21.15.146:0/1110623051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13600cacd0 0x7f13601fb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1366298700 1 -- 172.21.15.146:0/1110623051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13600c9500 msgr2=0x7f13601fce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1366298700 1 --2- 172.21.15.146:0/1110623051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13600c9500 0x7f13601fce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1366298700 1 -- 172.21.15.146:0/1110623051 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c009570 con 0x7f13600a9490 2024-09-13T18:52:36.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1366298700 1 --2- 172.21.15.146:0/1110623051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13600a9490 0x7f13600c48e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f135800ce50 tx=0x7f135800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:36.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1356ffd700 1 -- 172.21.15.146:0/1110623051 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135800ce80 con 0x7f13600a9490 2024-09-13T18:52:36.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1356ffd700 1 -- 172.21.15.146:0/1110623051 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135800aab0 con 0x7f13600a9490 2024-09-13T18:52:36.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f13684fc700 1 -- 172.21.15.146:0/1110623051 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13600abc70 con 0x7f13600a9490 2024-09-13T18:52:36.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f13684fc700 1 -- 172.21.15.146:0/1110623051 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13600ac230 con 0x7f13600a9490 2024-09-13T18:52:36.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.345+0000 7f1356ffd700 1 -- 172.21.15.146:0/1110623051 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135800ae20 con 0x7f13600a9490 2024-09-13T18:52:36.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.349+0000 7f1356ffd700 1 -- 172.21.15.146:0/1110623051 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1358003a20 con 0x7f13600a9490 2024-09-13T18:52:36.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.349+0000 7f1356ffd700 1 --2- 172.21.15.146:0/1110623051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1344066730 0x7f1344068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:36.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.349+0000 7f13684fc700 1 -- 172.21.15.146:0/1110623051 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13600ac440 con 0x7f13600a9490 2024-09-13T18:52:36.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.349+0000 7f1356ffd700 1 -- 172.21.15.146:0/1110623051 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f13580a6980 con 0x7f13600a9490 2024-09-13T18:52:36.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.353+0000 7f1365a97700 1 --2- 172.21.15.146:0/1110623051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1344066730 0x7f1344068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:36.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.353+0000 7f1365a97700 1 --2- 172.21.15.146:0/1110623051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1344066730 0x7f1344068bf0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1360152fd0 tx=0x7f1350009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:36.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.357+0000 7f1356ffd700 1 -- 172.21.15.146:0/1110623051 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1358073370 con 0x7f13600a9490 2024-09-13T18:52:36.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.737+0000 7f13684fc700 1 -- 172.21.15.146:0/1110623051 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13600a98e0 con 0x7f13600a9490 2024-09-13T18:52:36.754 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:36.754 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:52:35.904588+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6984,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:36.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.737+0000 7f1356ffd700 1 -- 172.21.15.146:0/1110623051 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+5521 (secure 0 0 0) 0x7f1358076990 con 0x7f13600a9490 2024-09-13T18:52:36.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.741+0000 7f13684fc700 1 -- 172.21.15.146:0/1110623051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1344066730 msgr2=0x7f1344068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.741+0000 7f13684fc700 1 --2- 172.21.15.146:0/1110623051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1344066730 0x7f1344068bf0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1360152fd0 tx=0x7f1350009040 comp rx=0 tx=0).stop 2024-09-13T18:52:36.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.741+0000 7f13684fc700 1 -- 172.21.15.146:0/1110623051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13600a9490 msgr2=0x7f13600c48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.741+0000 7f13684fc700 1 --2- 172.21.15.146:0/1110623051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13600a9490 0x7f13600c48e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f135800ce50 tx=0x7f135800a340 comp rx=0 tx=0).stop 2024-09-13T18:52:36.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.741+0000 7f13684fc700 1 -- 172.21.15.146:0/1110623051 shutdown_connections 2024-09-13T18:52:36.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.741+0000 7f13684fc700 1 --2- 172.21.15.146:0/1110623051 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1344066730 0x7f1344068bf0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.741+0000 7f13684fc700 1 --2- 172.21.15.146:0/1110623051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13600cacd0 0x7f13601fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.741+0000 7f13684fc700 1 --2- 172.21.15.146:0/1110623051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13600c9500 0x7f13601fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.741+0000 7f13684fc700 1 --2- 172.21.15.146:0/1110623051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13600a9490 0x7f13600c48e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.741+0000 7f13684fc700 1 -- 172.21.15.146:0/1110623051 >> 172.21.15.146:0/1110623051 conn(0x7f1360066090 msgr2=0x7f13600c16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:36.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.741+0000 7f13684fc700 1 -- 172.21.15.146:0/1110623051 shutdown_connections 2024-09-13T18:52:36.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:36.741+0000 7f13684fc700 1 -- 172.21.15.146:0/1110623051 wait complete. 2024-09-13T18:52:36.760 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 98 2024-09-13T18:52:36.779 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 98, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 98, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6714": { "addr": "172.21.15.146:6837/2169954140", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 2169954140, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 2169954140, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6714, "incarnation": 86, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 38 }, "gid_6984": { "addr": "172.21.15.146:6835/2934322065", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 2934322065, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 2934322065, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6984, "incarnation": 98, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 41, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:52:35.904588+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6984, "mds_1": 6714 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6839/2053734262", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 2053734262, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2053734262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 91, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6032, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:52:38.780 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:38.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce23fff700 1 Processor -- start 2024-09-13T18:52:38.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce23fff700 1 -- start start 2024-09-13T18:52:38.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce23fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fce24153bc0 0x7fce240c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:38.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce23fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce240c6a80 0x7fce24160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:38.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce23fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce24153fa0 con 0x7fce240c6610 2024-09-13T18:52:38.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce23fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce240aa500 con 0x7fce24153bc0 2024-09-13T18:52:38.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce23fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2415a7e0 con 0x7fce240c6a80 2024-09-13T18:52:38.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce227fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fce240c6610 0x7fce240cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48490/0 (socket says 172.21.15.146:48490) 2024-09-13T18:52:38.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce227fc700 1 -- 172.21.15.146:0/2042182707 learned_addr learned my addr 172.21.15.146:0/2042182707 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:52:38.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce237fe700 1 --2- 172.21.15.146:0/2042182707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce240c6a80 0x7fce24160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:38.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce22ffd700 1 --2- 172.21.15.146:0/2042182707 >> v2:172.21.15.154:3300/0 conn(0x7fce24153bc0 0x7fce240c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:38.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce21ffb700 1 -- 172.21.15.146:0/2042182707 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550701526 0 0) 0x7fce24153fa0 con 0x7fce240c6610 2024-09-13T18:52:38.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce21ffb700 1 -- 172.21.15.146:0/2042182707 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce0c0035d0 con 0x7fce240c6610 2024-09-13T18:52:38.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce237fe700 1 -- 172.21.15.146:0/2042182707 >> v2:172.21.15.154:3300/0 conn(0x7fce24153bc0 msgr2=0x7fce240c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:38.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce237fe700 1 --2- 172.21.15.146:0/2042182707 >> v2:172.21.15.154:3300/0 conn(0x7fce24153bc0 0x7fce240c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:38.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce237fe700 1 -- 172.21.15.146:0/2042182707 >> v1:172.21.15.146:6789/0 conn(0x7fce240c6610 legacy=0x7fce240cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:38.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce237fe700 1 -- 172.21.15.146:0/2042182707 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce240c9320 con 0x7fce240c6a80 2024-09-13T18:52:38.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce22ffd700 1 --2- 172.21.15.146:0/2042182707 >> v2:172.21.15.154:3300/0 conn(0x7fce24153bc0 0x7fce240c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:38.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce237fe700 1 --2- 172.21.15.146:0/2042182707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce240c6a80 0x7fce24160c20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009d90 tx=0x7fce1c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74b4b05332777298 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:38.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce21ffb700 1 -- 172.21.15.146:0/2042182707 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00bd20 con 0x7fce240c6a80 2024-09-13T18:52:38.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce21ffb700 1 -- 172.21.15.146:0/2042182707 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c00be80 con 0x7fce240c6a80 2024-09-13T18:52:38.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.881+0000 7fce21ffb700 1 -- 172.21.15.146:0/2042182707 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c0116a0 con 0x7fce240c6a80 2024-09-13T18:52:38.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 -- 172.21.15.146:0/2042182707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce240c6a80 msgr2=0x7fce24160c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:38.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 --2- 172.21.15.146:0/2042182707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce240c6a80 0x7fce24160c20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009d90 tx=0x7fce1c00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:38.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 -- 172.21.15.146:0/2042182707 shutdown_connections 2024-09-13T18:52:38.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 --2- 172.21.15.146:0/2042182707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce240c6a80 0x7fce24160c20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:38.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 --2- 172.21.15.146:0/2042182707 >> v2:172.21.15.154:3300/0 conn(0x7fce24153bc0 0x7fce240c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:38.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 -- 172.21.15.146:0/2042182707 >> 172.21.15.146:0/2042182707 conn(0x7fce24066090 msgr2=0x7fce24066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:38.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 -- 172.21.15.146:0/2042182707 shutdown_connections 2024-09-13T18:52:38.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 -- 172.21.15.146:0/2042182707 wait complete. 2024-09-13T18:52:38.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 Processor -- start 2024-09-13T18:52:38.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 -- start start 2024-09-13T18:52:38.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce240c6610 0x7fce240c1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:38.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240c6a80 0x7fce240aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:38.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce24153bc0 0x7fce241eee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:38.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce240c27b0 con 0x7fce240c6a80 2024-09-13T18:52:38.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce241f2dd0 con 0x7fce24153bc0 2024-09-13T18:52:38.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce23fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce240c1c80 con 0x7fce240c6610 2024-09-13T18:52:38.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce227fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240c6a80 0x7fce240aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:38.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce227fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240c6a80 0x7fce240aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40848/0 (socket says 172.21.15.146:40848) 2024-09-13T18:52:38.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce227fc700 1 -- 172.21.15.146:0/79551762 learned_addr learned my addr 172.21.15.146:0/79551762 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:38.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce22ffd700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce240c6610 0x7fce240c1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:38.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce237fe700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce24153bc0 0x7fce241eee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:38.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce227fc700 1 -- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce240c6610 msgr2=0x7fce240c1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:38.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce227fc700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce240c6610 0x7fce240c1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:38.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce227fc700 1 -- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce24153bc0 msgr2=0x7fce241eee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:38.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce227fc700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce24153bc0 0x7fce241eee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:38.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce227fc700 1 -- 172.21.15.146:0/79551762 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce140094b0 con 0x7fce240c6a80 2024-09-13T18:52:38.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.885+0000 7fce227fc700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240c6a80 0x7fce240aef40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c700 tx=0x7fce1800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:38.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.889+0000 7fce0bfff700 1 -- 172.21.15.146:0/79551762 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18005d70 con 0x7fce240c6a80 2024-09-13T18:52:38.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.889+0000 7fce0bfff700 1 -- 172.21.15.146:0/79551762 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce18018b10 con 0x7fce240c6a80 2024-09-13T18:52:38.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.889+0000 7fce23fff700 1 -- 172.21.15.146:0/79551762 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c009a70 con 0x7fce240c6a80 2024-09-13T18:52:38.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.889+0000 7fce23fff700 1 -- 172.21.15.146:0/79551762 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce241f2780 con 0x7fce240c6a80 2024-09-13T18:52:38.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.889+0000 7fce0bfff700 1 -- 172.21.15.146:0/79551762 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce180070d0 con 0x7fce240c6a80 2024-09-13T18:52:38.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.889+0000 7fce0bfff700 1 -- 172.21.15.146:0/79551762 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fce18007230 con 0x7fce240c6a80 2024-09-13T18:52:38.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.889+0000 7fce0bfff700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fce10066780 0x7fce10068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:38.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.889+0000 7fce0bfff700 1 -- 172.21.15.146:0/79551762 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7fce1808c8c0 con 0x7fce240c6a80 2024-09-13T18:52:38.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.889+0000 7fce22ffd700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fce10066780 0x7fce10068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:38.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.893+0000 7fce22ffd700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fce10066780 0x7fce10068c40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fce14009c70 tx=0x7fce14008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:38.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.893+0000 7fce23fff700 1 -- 172.21.15.146:0/79551762 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce04005310 con 0x7fce240c6a80 2024-09-13T18:52:38.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:38.897+0000 7fce0bfff700 1 -- 172.21.15.146:0/79551762 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fce18059410 con 0x7fce240c6a80 2024-09-13T18:52:39.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.181+0000 7fce23fff700 1 -- 172.21.15.146:0/79551762 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce040055c0 con 0x7fce240c6a80 2024-09-13T18:52:39.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.185+0000 7fce0bfff700 1 -- 172.21.15.146:0/79551762 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+5521 (secure 0 0 0) 0x7fce1805ca30 con 0x7fce240c6a80 2024-09-13T18:52:39.199 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:39.199 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:52:35.904588+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6984,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:39.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.185+0000 7fce23fff700 1 -- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fce10066780 msgr2=0x7fce10068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.185+0000 7fce23fff700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fce10066780 0x7fce10068c40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fce14009c70 tx=0x7fce14008040 comp rx=0 tx=0).stop 2024-09-13T18:52:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.185+0000 7fce23fff700 1 -- 172.21.15.146:0/79551762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240c6a80 msgr2=0x7fce240aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.185+0000 7fce23fff700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240c6a80 0x7fce240aef40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c700 tx=0x7fce1800ca10 comp rx=0 tx=0).stop 2024-09-13T18:52:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.185+0000 7fce23fff700 1 -- 172.21.15.146:0/79551762 shutdown_connections 2024-09-13T18:52:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.185+0000 7fce23fff700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fce10066780 0x7fce10068c40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.185+0000 7fce23fff700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce24153bc0 0x7fce241eee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.185+0000 7fce23fff700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240c6a80 0x7fce240aef40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.185+0000 7fce23fff700 1 --2- 172.21.15.146:0/79551762 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce240c6610 0x7fce240c1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.185+0000 7fce23fff700 1 -- 172.21.15.146:0/79551762 >> 172.21.15.146:0/79551762 conn(0x7fce24066090 msgr2=0x7fce240cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.185+0000 7fce23fff700 1 -- 172.21.15.146:0/79551762 shutdown_connections 2024-09-13T18:52:39.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:39.185+0000 7fce23fff700 1 -- 172.21.15.146:0/79551762 wait complete. 2024-09-13T18:52:39.204 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 98 2024-09-13T18:52:39.218 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:52:41.218 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:41.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 Processor -- start 2024-09-13T18:52:41.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 -- start start 2024-09-13T18:52:41.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc03c151840 0x7fc03c0c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:41.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc03c157fc0 0x7fc03c06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:41.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc03c0c9890 con 0x7fc03c0c8380 2024-09-13T18:52:41.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03c0c36f0 con 0x7fc03c151840 2024-09-13T18:52:41.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03c1572d0 con 0x7fc03c157fc0 2024-09-13T18:52:41.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc03bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc03c151840 0x7fc03c0c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:41.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc03bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc03c151840 0x7fc03c0c1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40216/0 (socket says 172.21.15.146:40216) 2024-09-13T18:52:41.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc03bfff700 1 -- 172.21.15.146:0/2440478246 learned_addr learned my addr 172.21.15.146:0/2440478246 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:41.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc03bfff700 1 -- 172.21.15.146:0/2440478246 >> v2:172.21.15.154:3300/0 conn(0x7fc03c157fc0 msgr2=0x7fc03c06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:41.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc03bfff700 1 --2- 172.21.15.146:0/2440478246 >> v2:172.21.15.154:3300/0 conn(0x7fc03c157fc0 0x7fc03c06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc03bfff700 1 -- 172.21.15.146:0/2440478246 >> v1:172.21.15.154:6789/0 conn(0x7fc03c0c8380 legacy=0x7fc03c15a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:41.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc03bfff700 1 -- 172.21.15.146:0/2440478246 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c0c9510 con 0x7fc03c151840 2024-09-13T18:52:41.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc03bfff700 1 --2- 172.21.15.146:0/2440478246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc03c151840 0x7fc03c0c1f90 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc03c15b8e0 tx=0x7fc02c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e76c6c816107d8c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:41.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc03affd700 1 -- 172.21.15.146:0/2440478246 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c004230 con 0x7fc03c151840 2024-09-13T18:52:41.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc03affd700 1 -- 172.21.15.146:0/2440478246 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c004390 con 0x7fc03c151840 2024-09-13T18:52:41.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc03affd700 1 -- 172.21.15.146:0/2440478246 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0036c0 con 0x7fc03c151840 2024-09-13T18:52:41.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 -- 172.21.15.146:0/2440478246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc03c151840 msgr2=0x7fc03c0c1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:41.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 --2- 172.21.15.146:0/2440478246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc03c151840 0x7fc03c0c1f90 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc03c15b8e0 tx=0x7fc02c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 -- 172.21.15.146:0/2440478246 shutdown_connections 2024-09-13T18:52:41.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 --2- 172.21.15.146:0/2440478246 >> v2:172.21.15.154:3300/0 conn(0x7fc03c157fc0 0x7fc03c06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 --2- 172.21.15.146:0/2440478246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc03c151840 0x7fc03c0c1f90 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 -- 172.21.15.146:0/2440478246 >> 172.21.15.146:0/2440478246 conn(0x7fc03c0ccf70 msgr2=0x7fc03c0cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:41.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 -- 172.21.15.146:0/2440478246 shutdown_connections 2024-09-13T18:52:41.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.313+0000 7fc04177d700 1 -- 172.21.15.146:0/2440478246 wait complete. 2024-09-13T18:52:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc04177d700 1 Processor -- start 2024-09-13T18:52:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc04177d700 1 -- start start 2024-09-13T18:52:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc04177d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc03c0c8380 0x7fc03c14aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc04177d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc03c151840 0x7fc03c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc04177d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc03c157fc0 0x7fc03c1ed5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc04177d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c14a910 con 0x7fc03c157fc0 2024-09-13T18:52:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc04177d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c14abb0 con 0x7fc03c151840 2024-09-13T18:52:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc04177d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c15a360 con 0x7fc03c0c8380 2024-09-13T18:52:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc040f7c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc03c157fc0 0x7fc03c1ed5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:41.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc040f7c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc03c157fc0 0x7fc03c1ed5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40230/0 (socket says 172.21.15.146:40230) 2024-09-13T18:52:41.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc040f7c700 1 -- 172.21.15.146:0/144184240 learned_addr learned my addr 172.21.15.146:0/144184240 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:41.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc03bfff700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc03c0c8380 0x7fc03c14aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:41.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc040f7c700 1 -- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc03c0c8380 msgr2=0x7fc03c14aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:41.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc040f7c700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc03c0c8380 0x7fc03c14aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc03b7fe700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc03c151840 0x7fc03c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:41.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc040f7c700 1 -- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc03c151840 msgr2=0x7fc03c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:41.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc040f7c700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc03c151840 0x7fc03c0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc040f7c700 1 -- 172.21.15.146:0/144184240 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c009570 con 0x7fc03c157fc0 2024-09-13T18:52:41.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc03b7fe700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc03c151840 0x7fc03c0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:41.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc040f7c700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc03c157fc0 0x7fc03c1ed5d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc03400cc10 tx=0x7fc03400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:41.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc038ff9700 1 -- 172.21.15.146:0/144184240 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03400abb0 con 0x7fc03c157fc0 2024-09-13T18:52:41.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc038ff9700 1 -- 172.21.15.146:0/144184240 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03400ad10 con 0x7fc03c157fc0 2024-09-13T18:52:41.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc038ff9700 1 -- 172.21.15.146:0/144184240 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034012660 con 0x7fc03c157fc0 2024-09-13T18:52:41.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc04177d700 1 -- 172.21.15.146:0/144184240 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03c15a640 con 0x7fc03c157fc0 2024-09-13T18:52:41.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc04177d700 1 -- 172.21.15.146:0/144184240 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c15ac00 con 0x7fc03c157fc0 2024-09-13T18:52:41.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc04177d700 1 -- 172.21.15.146:0/144184240 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c067a60 con 0x7fc03c157fc0 2024-09-13T18:52:41.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc038ff9700 1 -- 172.21.15.146:0/144184240 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc0340127c0 con 0x7fc03c157fc0 2024-09-13T18:52:41.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc038ff9700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc028066760 0x7fc028068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:41.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.317+0000 7fc038ff9700 1 -- 172.21.15.146:0/144184240 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7fc0340850d0 con 0x7fc03c157fc0 2024-09-13T18:52:41.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.321+0000 7fc03bfff700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc028066760 0x7fc028068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:41.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.321+0000 7fc03bfff700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc028066760 0x7fc028068c20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc03c15b8e0 tx=0x7fc02c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:41.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.325+0000 7fc038ff9700 1 -- 172.21.15.146:0/144184240 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc034051c60 con 0x7fc03c157fc0 2024-09-13T18:52:41.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.513+0000 7fc04177d700 1 -- 172.21.15.146:0/144184240 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc03c0c7470 con 0x7fc03c157fc0 2024-09-13T18:52:41.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.513+0000 7fc038ff9700 1 -- 172.21.15.146:0/144184240 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+5521 (secure 0 0 0) 0x7fc03c0c7470 con 0x7fc03c157fc0 2024-09-13T18:52:41.529 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:41.529 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:52:35.904588+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6984,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.517+0000 7fc04177d700 1 -- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc028066760 msgr2=0x7fc028068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.517+0000 7fc04177d700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc028066760 0x7fc028068c20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc03c15b8e0 tx=0x7fc02c008030 comp rx=0 tx=0).stop 2024-09-13T18:52:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.517+0000 7fc04177d700 1 -- 172.21.15.146:0/144184240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc03c157fc0 msgr2=0x7fc03c1ed5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.517+0000 7fc04177d700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc03c157fc0 0x7fc03c1ed5d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc03400cc10 tx=0x7fc03400a060 comp rx=0 tx=0).stop 2024-09-13T18:52:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.517+0000 7fc04177d700 1 -- 172.21.15.146:0/144184240 shutdown_connections 2024-09-13T18:52:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.517+0000 7fc04177d700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc028066760 0x7fc028068c20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.517+0000 7fc04177d700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc03c157fc0 0x7fc03c1ed5d0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.517+0000 7fc04177d700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc03c151840 0x7fc03c0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.517+0000 7fc04177d700 1 --2- 172.21.15.146:0/144184240 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc03c0c8380 0x7fc03c14aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.517+0000 7fc04177d700 1 -- 172.21.15.146:0/144184240 >> 172.21.15.146:0/144184240 conn(0x7fc03c0ccf70 msgr2=0x7fc03c156b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.517+0000 7fc04177d700 1 -- 172.21.15.146:0/144184240 shutdown_connections 2024-09-13T18:52:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:41.517+0000 7fc04177d700 1 -- 172.21.15.146:0/144184240 wait complete. 2024-09-13T18:52:41.533 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 98 2024-09-13T18:52:41.543 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:52:43.544 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:43.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f394a730700 1 Processor -- start 2024-09-13T18:52:43.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f394a730700 1 -- start start 2024-09-13T18:52:43.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f394a730700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3944154a40 0x7f39440721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f394a730700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f394406f7d0 0x7f3944157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f394a730700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f39440cac60 0x7f39440cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f394a730700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3944072810 con 0x7f39440cac60 2024-09-13T18:52:43.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f394a730700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3944157910 con 0x7f3944154a40 2024-09-13T18:52:43.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f394a730700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39440c8c70 con 0x7f394406f7d0 2024-09-13T18:52:43.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f3949f2f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f39440cac60 0x7f39440cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f3949f2f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f39440cac60 0x7f39440cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40244/0 (socket says 172.21.15.146:40244) 2024-09-13T18:52:43.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f3949f2f700 1 -- 172.21.15.146:0/3166087395 learned_addr learned my addr 172.21.15.146:0/3166087395 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:43.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f3948f2d700 1 --2- 172.21.15.146:0/3166087395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f394406f7d0 0x7f3944157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f394972e700 1 --2- 172.21.15.146:0/3166087395 >> v2:172.21.15.154:3300/0 conn(0x7f3944154a40 0x7f39440721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f3949f2f700 1 -- 172.21.15.146:0/3166087395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f394406f7d0 msgr2=0x7f3944157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:43.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f3949f2f700 1 --2- 172.21.15.146:0/3166087395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f394406f7d0 0x7f3944157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f3949f2f700 1 -- 172.21.15.146:0/3166087395 >> v2:172.21.15.154:3300/0 conn(0x7f3944154a40 msgr2=0x7f39440721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:43.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f3949f2f700 1 --2- 172.21.15.146:0/3166087395 >> v2:172.21.15.154:3300/0 conn(0x7f3944154a40 0x7f39440721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f3949f2f700 1 -- 172.21.15.146:0/3166087395 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39440c8f50 con 0x7f39440cac60 2024-09-13T18:52:43.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f394972e700 1 --2- 172.21.15.146:0/3166087395 >> v2:172.21.15.154:3300/0 conn(0x7f3944154a40 0x7f39440721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:43.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f3949f2f700 1 --2- 172.21.15.146:0/3166087395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39440cac60 0x7f39440cb0f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f39440cbbe0 tx=0x7f393c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776cb97528567f36 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:43.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f393bfff700 1 -- 172.21.15.146:0/3166087395 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c011830 con 0x7f39440cac60 2024-09-13T18:52:43.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.649+0000 7f393bfff700 1 -- 172.21.15.146:0/3166087395 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c011990 con 0x7f39440cac60 2024-09-13T18:52:43.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f393bfff700 1 -- 172.21.15.146:0/3166087395 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c011d20 con 0x7f39440cac60 2024-09-13T18:52:43.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 -- 172.21.15.146:0/3166087395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39440cac60 msgr2=0x7f39440cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 --2- 172.21.15.146:0/3166087395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39440cac60 0x7f39440cb0f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f39440cbbe0 tx=0x7f393c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 -- 172.21.15.146:0/3166087395 shutdown_connections 2024-09-13T18:52:43.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 --2- 172.21.15.146:0/3166087395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f394406f7d0 0x7f3944157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 --2- 172.21.15.146:0/3166087395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39440cac60 0x7f39440cb0f0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 --2- 172.21.15.146:0/3166087395 >> v2:172.21.15.154:3300/0 conn(0x7f3944154a40 0x7f39440721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 -- 172.21.15.146:0/3166087395 >> 172.21.15.146:0/3166087395 conn(0x7f3944066090 msgr2=0x7f3944066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:43.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 -- 172.21.15.146:0/3166087395 shutdown_connections 2024-09-13T18:52:43.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 -- 172.21.15.146:0/3166087395 wait complete. 2024-09-13T18:52:43.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 Processor -- start 2024-09-13T18:52:43.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 -- start start 2024-09-13T18:52:43.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f394406f7d0 0x7f39440c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39440cac60 0x7f39440ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3944154a40 0x7f39440ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39440ac020 con 0x7f39440cac60 2024-09-13T18:52:43.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39440c5080 con 0x7f3944154a40 2024-09-13T18:52:43.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394a730700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39440aa010 con 0x7f394406f7d0 2024-09-13T18:52:43.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f3948f2d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39440cac60 0x7f39440ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f3948f2d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39440cac60 0x7f39440ab5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40250/0 (socket says 172.21.15.146:40250) 2024-09-13T18:52:43.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f3948f2d700 1 -- 172.21.15.146:0/1375581844 learned_addr learned my addr 172.21.15.146:0/1375581844 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:43.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.653+0000 7f394972e700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f394406f7d0 0x7f39440c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f3949f2f700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3944154a40 0x7f39440ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f3948f2d700 1 -- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f394406f7d0 msgr2=0x7f39440c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f3948f2d700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f394406f7d0 0x7f39440c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f3948f2d700 1 -- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3944154a40 msgr2=0x7f39440ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f3948f2d700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3944154a40 0x7f39440ac4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f3948f2d700 1 -- 172.21.15.146:0/1375581844 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c009570 con 0x7f39440cac60 2024-09-13T18:52:43.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f3948f2d700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39440cac60 0x7f39440ab5d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f394000eed0 tx=0x7f394000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:43.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f3939ffb700 1 -- 172.21.15.146:0/1375581844 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940010040 con 0x7f39440cac60 2024-09-13T18:52:43.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f3939ffb700 1 -- 172.21.15.146:0/1375581844 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394000bb20 con 0x7f39440cac60 2024-09-13T18:52:43.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f394a730700 1 -- 172.21.15.146:0/1375581844 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39440aa2f0 con 0x7f39440cac60 2024-09-13T18:52:43.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f3939ffb700 1 -- 172.21.15.146:0/1375581844 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940014400 con 0x7f39440cac60 2024-09-13T18:52:43.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f394a730700 1 -- 172.21.15.146:0/1375581844 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39440aa8b0 con 0x7f39440cac60 2024-09-13T18:52:43.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f394a730700 1 -- 172.21.15.146:0/1375581844 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39440c9e00 con 0x7f39440cac60 2024-09-13T18:52:43.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.657+0000 7f3939ffb700 1 -- 172.21.15.146:0/1375581844 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3940014560 con 0x7f39440cac60 2024-09-13T18:52:43.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.661+0000 7f3939ffb700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3928066760 0x7f3928068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.661+0000 7f3939ffb700 1 -- 172.21.15.146:0/1375581844 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f3940084ca0 con 0x7f39440cac60 2024-09-13T18:52:43.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.661+0000 7f394972e700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3928066760 0x7f3928068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.661+0000 7f394972e700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3928066760 0x7f3928068c20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f393400c920 tx=0x7f3934008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:43.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.665+0000 7f3939ffb700 1 -- 172.21.15.146:0/1375581844 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3940051830 con 0x7f39440cac60 2024-09-13T18:52:43.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.873+0000 7f394a730700 1 -- 172.21.15.146:0/1375581844 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39440ac160 con 0x7f39440cac60 2024-09-13T18:52:43.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.873+0000 7f3939ffb700 1 -- 172.21.15.146:0/1375581844 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+5523 (secure 0 0 0) 0x7f39440ac160 con 0x7f39440cac60 2024-09-13T18:52:43.889 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:43.889 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:52:41.690823+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6984,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:resolve","state_seq":22,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:43.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.873+0000 7f394a730700 1 -- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3928066760 msgr2=0x7f3928068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.877+0000 7f394a730700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3928066760 0x7f3928068c20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f393400c920 tx=0x7f3934008040 comp rx=0 tx=0).stop 2024-09-13T18:52:43.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.877+0000 7f394a730700 1 -- 172.21.15.146:0/1375581844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39440cac60 msgr2=0x7f39440ab5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.877+0000 7f394a730700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39440cac60 0x7f39440ab5d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f394000eed0 tx=0x7f394000ef00 comp rx=0 tx=0).stop 2024-09-13T18:52:43.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.877+0000 7f394a730700 1 -- 172.21.15.146:0/1375581844 shutdown_connections 2024-09-13T18:52:43.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.877+0000 7f394a730700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3928066760 0x7f3928068c20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.877+0000 7f394a730700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3944154a40 0x7f39440ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.877+0000 7f394a730700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39440cac60 0x7f39440ab5d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.877+0000 7f394a730700 1 --2- 172.21.15.146:0/1375581844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f394406f7d0 0x7f39440c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.877+0000 7f394a730700 1 -- 172.21.15.146:0/1375581844 >> 172.21.15.146:0/1375581844 conn(0x7f3944066090 msgr2=0x7f3944066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:43.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.877+0000 7f394a730700 1 -- 172.21.15.146:0/1375581844 shutdown_connections 2024-09-13T18:52:43.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:43.877+0000 7f394a730700 1 -- 172.21.15.146:0/1375581844 wait complete. 2024-09-13T18:52:43.893 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 99 2024-09-13T18:52:43.904 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:52:45.787 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:52:45.790 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:52:45.905 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:46.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.009+0000 7fea3b59e700 1 Processor -- start 2024-09-13T18:52:46.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.009+0000 7fea3b59e700 1 -- start start 2024-09-13T18:52:46.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.009+0000 7fea3b59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fea3c0caed0 0x7fea3c0723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.009+0000 7fea3b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3c0a9840 0x7fea3c15e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.009+0000 7fea3b59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3c161550 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.009+0000 7fea3b59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3c072a30 con 0x7fea3c0caed0 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.009+0000 7fea3b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c0aa800 con 0x7fea3c0a9840 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.009+0000 7fea39d9b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fea3c0cb2b0 0x7fea3c1619e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:44482/0 (socket says 172.21.15.146:44482) 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.009+0000 7fea39d9b700 1 -- 172.21.15.146:0/3083302432 learned_addr learned my addr 172.21.15.146:0/3083302432 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.009+0000 7fea3ad9d700 1 --2- 172.21.15.146:0/3083302432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3c0a9840 0x7fea3c15e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.009+0000 7fea3a59c700 1 --2- 172.21.15.146:0/3083302432 >> v2:172.21.15.154:3300/0 conn(0x7fea3c0caed0 0x7fea3c0723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3959a700 1 -- 172.21.15.146:0/3083302432 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654460113 0 0) 0x7fea3c161550 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3959a700 1 -- 172.21.15.146:0/3083302432 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea240035d0 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3ad9d700 1 -- 172.21.15.146:0/3083302432 >> v2:172.21.15.154:3300/0 conn(0x7fea3c0caed0 msgr2=0x7fea3c0723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3ad9d700 1 --2- 172.21.15.146:0/3083302432 >> v2:172.21.15.154:3300/0 conn(0x7fea3c0caed0 0x7fea3c0723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3ad9d700 1 -- 172.21.15.146:0/3083302432 >> v1:172.21.15.146:6789/0 conn(0x7fea3c0cb2b0 legacy=0x7fea3c1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3959a700 1 -- 172.21.15.146:0/3083302432 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 263307594 0 0) 0x7fea240035d0 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3ad9d700 1 -- 172.21.15.146:0/3083302432 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c161220 con 0x7fea3c0a9840 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3ad9d700 1 --2- 172.21.15.146:0/3083302432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3c0a9840 0x7fea3c15e100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009850 tx=0x7fea2c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d1b986248f63c8c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:46.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3959a700 1 -- 172.21.15.146:0/3083302432 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30001670 con 0x7fea3c0a9840 2024-09-13T18:52:46.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3959a700 1 -- 172.21.15.146:0/3083302432 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c004030 con 0x7fea3c0a9840 2024-09-13T18:52:46.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3959a700 1 -- 172.21.15.146:0/3083302432 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0043a0 con 0x7fea3c0a9840 2024-09-13T18:52:46.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3b59e700 1 -- 172.21.15.146:0/3083302432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3c0a9840 msgr2=0x7fea3c15e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3b59e700 1 --2- 172.21.15.146:0/3083302432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3c0a9840 0x7fea3c15e100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009850 tx=0x7fea2c009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:46.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3b59e700 1 -- 172.21.15.146:0/3083302432 shutdown_connections 2024-09-13T18:52:46.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3b59e700 1 --2- 172.21.15.146:0/3083302432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3c0a9840 0x7fea3c15e100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3b59e700 1 --2- 172.21.15.146:0/3083302432 >> v2:172.21.15.154:3300/0 conn(0x7fea3c0caed0 0x7fea3c0723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3b59e700 1 -- 172.21.15.146:0/3083302432 >> 172.21.15.146:0/3083302432 conn(0x7fea3c066090 msgr2=0x7fea3c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:46.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3b59e700 1 -- 172.21.15.146:0/3083302432 shutdown_connections 2024-09-13T18:52:46.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3b59e700 1 -- 172.21.15.146:0/3083302432 wait complete. 2024-09-13T18:52:46.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3b59e700 1 Processor -- start 2024-09-13T18:52:46.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.013+0000 7fea3b59e700 1 -- start start 2024-09-13T18:52:46.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3c0a9840 0x7fea3c0af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3b59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea3c0caed0 0x7fea3c15a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea3c0cb2b0 0x7fea3c15a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3b59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c0ad930 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3b59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c1f3190 con 0x7fea3c0caed0 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c0acda0 con 0x7fea3c0a9840 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea3c0cb2b0 0x7fea3c15a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea3c0cb2b0 0x7fea3c15a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40260/0 (socket says 172.21.15.146:40260) 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3ad9d700 1 -- 172.21.15.146:0/2624245846 learned_addr learned my addr 172.21.15.146:0/2624245846 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3a59c700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3c0a9840 0x7fea3c0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea39d9b700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea3c0caed0 0x7fea3c15a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3ad9d700 1 -- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3c0a9840 msgr2=0x7fea3c0af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3ad9d700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3c0a9840 0x7fea3c0af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3ad9d700 1 -- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea3c0caed0 msgr2=0x7fea3c15a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3ad9d700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea3c0caed0 0x7fea3c15a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3ad9d700 1 -- 172.21.15.146:0/2624245846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c0094b0 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3ad9d700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea3c0cb2b0 0x7fea3c15a790 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0aa940 tx=0x7fea2c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:46.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea2affd700 1 -- 172.21.15.146:0/2624245846 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c003ac0 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea2affd700 1 -- 172.21.15.146:0/2624245846 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c01f070 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea2affd700 1 -- 172.21.15.146:0/2624245846 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c003e80 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3b59e700 1 -- 172.21.15.146:0/2624245846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3c0ad080 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3b59e700 1 -- 172.21.15.146:0/2624245846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3c1f7510 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea3b59e700 1 -- 172.21.15.146:0/2624245846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3c1f7730 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.017+0000 7fea2affd700 1 -- 172.21.15.146:0/2624245846 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fea2c021030 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.021+0000 7fea2affd700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea1c066730 0x7fea1c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.021+0000 7fea2affd700 1 -- 172.21.15.146:0/2624245846 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7fea2c085cf0 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.021+0000 7fea3a59c700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea1c066730 0x7fea1c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.021+0000 7fea3a59c700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea1c066730 0x7fea1c068bf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fea34011340 tx=0x7fea3400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:46.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.025+0000 7fea2affd700 1 -- 172.21.15.146:0/2624245846 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fea3c1f7730 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.229+0000 7fea3b59e700 1 -- 172.21.15.146:0/2624245846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea3c1f7730 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.229+0000 7fea2affd700 1 -- 172.21.15.146:0/2624245846 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+5523 (secure 0 0 0) 0x7fea3c1f7730 con 0x7fea3c0cb2b0 2024-09-13T18:52:46.249 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:46.249 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:52:41.690823+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6984,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:resolve","state_seq":22,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.233+0000 7fea3b59e700 1 -- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea1c066730 msgr2=0x7fea1c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.233+0000 7fea3b59e700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea1c066730 0x7fea1c068bf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fea34011340 tx=0x7fea3400b040 comp rx=0 tx=0).stop 2024-09-13T18:52:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.233+0000 7fea3b59e700 1 -- 172.21.15.146:0/2624245846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea3c0cb2b0 msgr2=0x7fea3c15a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.233+0000 7fea3b59e700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea3c0cb2b0 0x7fea3c15a790 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0aa940 tx=0x7fea2c004770 comp rx=0 tx=0).stop 2024-09-13T18:52:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.233+0000 7fea3b59e700 1 -- 172.21.15.146:0/2624245846 shutdown_connections 2024-09-13T18:52:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.233+0000 7fea3b59e700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fea1c066730 0x7fea1c068bf0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.233+0000 7fea3b59e700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea3c0cb2b0 0x7fea3c15a790 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.233+0000 7fea3b59e700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea3c0caed0 0x7fea3c15a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.233+0000 7fea3b59e700 1 --2- 172.21.15.146:0/2624245846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3c0a9840 0x7fea3c0af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.233+0000 7fea3b59e700 1 -- 172.21.15.146:0/2624245846 >> 172.21.15.146:0/2624245846 conn(0x7fea3c066090 msgr2=0x7fea3c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.233+0000 7fea3b59e700 1 -- 172.21.15.146:0/2624245846 shutdown_connections 2024-09-13T18:52:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:46.233+0000 7fea3b59e700 1 -- 172.21.15.146:0/2624245846 wait complete. 2024-09-13T18:52:46.251 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 99 2024-09-13T18:52:46.263 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:52:48.264 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:48.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0ed4ae9700 1 Processor -- start 2024-09-13T18:52:48.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0ed4ae9700 1 -- start start 2024-09-13T18:52:48.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0ed4ae9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ed00c5fc0 0x7f0ed00c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0ed4ae9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0ed00c63a0 0x7f0ed00c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0ed4ae9700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed00c8c70 con 0x7f0ed006f5c0 2024-09-13T18:52:48.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0ed4ae9700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed00c9bd0 con 0x7f0ed00c63a0 2024-09-13T18:52:48.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0ed4ae9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed00c8f50 con 0x7f0ed00c5fc0 2024-09-13T18:52:48.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0eceffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0ed00c63a0 0x7f0ed00c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0eceffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0ed00c63a0 0x7f0ed00c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40262/0 (socket says 172.21.15.146:40262) 2024-09-13T18:52:48.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0ecffff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0ed006f5c0 0x7f0ed0159660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:44488/0 (socket says 172.21.15.146:44488) 2024-09-13T18:52:48.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0eceffd700 1 -- 172.21.15.146:0/159273469 learned_addr learned my addr 172.21.15.146:0/159273469 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:48.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0ecf7fe700 1 --2- 172.21.15.146:0/159273469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ed00c5fc0 0x7f0ed00c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0eceffd700 1 -- 172.21.15.146:0/159273469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ed00c5fc0 msgr2=0x7f0ed00c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0eceffd700 1 --2- 172.21.15.146:0/159273469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ed00c5fc0 0x7f0ed00c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0ece7fc700 1 -- 172.21.15.146:0/159273469 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784107267 0 0) 0x7f0ed00c8c70 con 0x7f0ed006f5c0 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0eceffd700 1 -- 172.21.15.146:0/159273469 >> v1:172.21.15.146:6789/0 conn(0x7f0ed006f5c0 legacy=0x7f0ed0159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0eceffd700 1 -- 172.21.15.146:0/159273469 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed00c9230 con 0x7f0ed00c63a0 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0eceffd700 1 --2- 172.21.15.146:0/159273469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed00c63a0 0x7f0ed00c7140 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009910 tx=0x7f0eb8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e16dd9f060b472 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0ece7fc700 1 -- 172.21.15.146:0/159273469 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8011830 con 0x7f0ed00c63a0 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0ece7fc700 1 -- 172.21.15.146:0/159273469 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb8011990 con 0x7f0ed00c63a0 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.369+0000 7f0ece7fc700 1 -- 172.21.15.146:0/159273469 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8011d40 con 0x7f0ed00c63a0 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/159273469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed00c63a0 msgr2=0x7f0ed00c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 --2- 172.21.15.146:0/159273469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed00c63a0 0x7f0ed00c7140 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009910 tx=0x7f0eb8009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:48.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/159273469 shutdown_connections 2024-09-13T18:52:48.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 --2- 172.21.15.146:0/159273469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed00c63a0 0x7f0ed00c7140 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 --2- 172.21.15.146:0/159273469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ed00c5fc0 0x7f0ed00c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/159273469 >> 172.21.15.146:0/159273469 conn(0x7f0ed0066090 msgr2=0x7f0ed0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:48.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/159273469 shutdown_connections 2024-09-13T18:52:48.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/159273469 wait complete. 2024-09-13T18:52:48.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 Processor -- start 2024-09-13T18:52:48.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 -- start start 2024-09-13T18:52:48.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ed006f5c0 0x7f0ed00ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ed00c5fc0 0x7f0ed0151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed00c63a0 0x7f0ed01505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.373+0000 7f0ed4ae9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed00aaac0 con 0x7f0ed00c63a0 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ed4ae9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed00ad4d0 con 0x7f0ed006f5c0 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ed4ae9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0066920 con 0x7f0ed00c5fc0 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ecffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed00c63a0 0x7f0ed01505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ecffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed00c63a0 0x7f0ed01505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40276/0 (socket says 172.21.15.146:40276) 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ecffff700 1 -- 172.21.15.146:0/654579686 learned_addr learned my addr 172.21.15.146:0/654579686 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0eceffd700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ed00c5fc0 0x7f0ed0151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ecf7fe700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ed006f5c0 0x7f0ed00ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ecffff700 1 -- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ed00c5fc0 msgr2=0x7f0ed0151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ecffff700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ed00c5fc0 0x7f0ed0151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ecffff700 1 -- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ed006f5c0 msgr2=0x7f0ed00ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ecffff700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ed006f5c0 0x7f0ed00ac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ecffff700 1 -- 172.21.15.146:0/654579686 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb8009570 con 0x7f0ed00c63a0 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ecffff700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed00c63a0 0x7f0ed01505f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0001eb0 tx=0x7f0ec000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ec7fff700 1 -- 172.21.15.146:0/654579686 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0004ca0 con 0x7f0ed00c63a0 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ec7fff700 1 -- 172.21.15.146:0/654579686 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec0004e00 con 0x7f0ed00c63a0 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/654579686 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed00ad790 con 0x7f0ed00c63a0 2024-09-13T18:52:48.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ec7fff700 1 -- 172.21.15.146:0/654579686 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0007310 con 0x7f0ed00c63a0 2024-09-13T18:52:48.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/654579686 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed00ac1f0 con 0x7f0ed00c63a0 2024-09-13T18:52:48.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ec7fff700 1 -- 172.21.15.146:0/654579686 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0ec0007470 con 0x7f0ed00c63a0 2024-09-13T18:52:48.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.377+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/654579686 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed00cb790 con 0x7f0ed00c63a0 2024-09-13T18:52:48.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.381+0000 7f0ec7fff700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0eb0066760 0x7f0eb0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.381+0000 7f0ec7fff700 1 -- 172.21.15.146:0/654579686 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f0ec0084580 con 0x7f0ed00c63a0 2024-09-13T18:52:48.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.381+0000 7f0ecf7fe700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0eb0066760 0x7f0eb0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.381+0000 7f0ecf7fe700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0eb0066760 0x7f0eb0068c20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800bfd0 tx=0x7f0ec800d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:48.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.389+0000 7f0ec7fff700 1 -- 172.21.15.146:0/654579686 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0ec0050f70 con 0x7f0ed00c63a0 2024-09-13T18:52:48.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.601+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/654579686 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ed00a9540 con 0x7f0ed00c63a0 2024-09-13T18:52:48.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.601+0000 7f0ec7fff700 1 -- 172.21.15.146:0/654579686 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+5527 (secure 0 0 0) 0x7f0ec0054590 con 0x7f0ed00c63a0 2024-09-13T18:52:48.617 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:48.617 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:52:47.711074+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6984,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:reconnect","state_seq":24,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:48.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.601+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0eb0066760 msgr2=0x7f0eb0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.605+0000 7f0ed4ae9700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0eb0066760 0x7f0eb0068c20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800bfd0 tx=0x7f0ec800d040 comp rx=0 tx=0).stop 2024-09-13T18:52:48.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.605+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/654579686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed00c63a0 msgr2=0x7f0ed01505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.605+0000 7f0ed4ae9700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed00c63a0 0x7f0ed01505f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0001eb0 tx=0x7f0ec000c730 comp rx=0 tx=0).stop 2024-09-13T18:52:48.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.605+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/654579686 shutdown_connections 2024-09-13T18:52:48.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.605+0000 7f0ed4ae9700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0eb0066760 0x7f0eb0068c20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.605+0000 7f0ed4ae9700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed00c63a0 0x7f0ed01505f0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.605+0000 7f0ed4ae9700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ed00c5fc0 0x7f0ed0151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.605+0000 7f0ed4ae9700 1 --2- 172.21.15.146:0/654579686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ed006f5c0 0x7f0ed00ac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.605+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/654579686 >> 172.21.15.146:0/654579686 conn(0x7f0ed0066090 msgr2=0x7f0ed0157160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:48.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.605+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/654579686 shutdown_connections 2024-09-13T18:52:48.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:48.605+0000 7f0ed4ae9700 1 -- 172.21.15.146:0/654579686 wait complete. 2024-09-13T18:52:48.621 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 100 2024-09-13T18:52:48.633 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 100, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 100, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6714": { "addr": "172.21.15.146:6837/2169954140", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 2169954140, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 2169954140, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6714, "incarnation": 86, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 38 }, "gid_6984": { "addr": "172.21.15.146:6835/2934322065", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 2934322065, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 2934322065, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6984, "incarnation": 98, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 24 } }, "last_failure": 0, "last_failure_osd_epoch": 41, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:52:47.711074+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6984, "mds_1": 6714 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6839/2053734262", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 2053734262, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2053734262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 91, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6032, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:52:50.634 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:50.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd57a247700 1 Processor -- start 2024-09-13T18:52:50.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd57a247700 1 -- start start 2024-09-13T18:52:50.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd57a247700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd57406db40 0x7fd5741587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd57a247700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd574158d10 con 0x7fd574151590 2024-09-13T18:52:50.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd57a247700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd574066500 con 0x7fd574157fd0 2024-09-13T18:52:50.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd57a247700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5740c69b0 con 0x7fd57406db40 2024-09-13T18:52:50.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd579245700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd574151590 0x7fd5740c4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:44490/0 (socket says 172.21.15.146:44490) 2024-09-13T18:52:50.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd579245700 1 -- 172.21.15.146:0/136508486 learned_addr learned my addr 172.21.15.146:0/136508486 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:52:50.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd579a46700 1 --2- 172.21.15.146:0/136508486 >> v2:172.21.15.154:3300/0 conn(0x7fd57406db40 0x7fd5741587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3601868693 0 0) 0x7fd574158d10 con 0x7fd574151590 2024-09-13T18:52:50.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5600035d0 con 0x7fd574151590 2024-09-13T18:52:50.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2042443214 0 0) 0x7fd5600035d0 con 0x7fd574151590 2024-09-13T18:52:50.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 >> v2:172.21.15.154:3300/0 conn(0x7fd57406db40 msgr2=0x7fd5741587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:50.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 --2- 172.21.15.146:0/136508486 >> v2:172.21.15.154:3300/0 conn(0x7fd57406db40 0x7fd5741587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 >> v1:172.21.15.154:6789/0 conn(0x7fd574157fd0 legacy=0x7fd57415fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57406f740 con 0x7fd574151590 2024-09-13T18:52:50.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7fd564001670 con 0x7fd574151590 2024-09-13T18:52:50.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 >> v1:172.21.15.146:6789/0 conn(0x7fd574151590 legacy=0x7fd5740c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 --2- 172.21.15.146:0/136508486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd560003a90 0x7fd560003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 --2- 172.21.15.146:0/136508486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560004870 0x7fd560004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 --2- 172.21.15.146:0/136508486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd560005230 0x7fd56000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5740c69b0 con 0x7fd560005230 2024-09-13T18:52:50.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56000c140 con 0x7fd560004870 2024-09-13T18:52:50.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56000c300 con 0x7fd560003a90 2024-09-13T18:52:50.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd579a46700 1 --2- 172.21.15.146:0/136508486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd560005230 0x7fd56000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd578a44700 1 --2- 172.21.15.146:0/136508486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560004870 0x7fd560004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd579245700 1 --2- 172.21.15.146:0/136508486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd560003a90 0x7fd560003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd579a46700 1 -- 172.21.15.146:0/136508486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd560003a90 msgr2=0x7fd560003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:50.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd579a46700 1 --2- 172.21.15.146:0/136508486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd560003a90 0x7fd560003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd579a46700 1 -- 172.21.15.146:0/136508486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560004870 msgr2=0x7fd560004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:50.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd579a46700 1 --2- 172.21.15.146:0/136508486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560004870 0x7fd560004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd579a46700 1 -- 172.21.15.146:0/136508486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5600035d0 con 0x7fd560005230 2024-09-13T18:52:50.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.741+0000 7fd579245700 1 --2- 172.21.15.146:0/136508486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd560003a90 0x7fd560003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:50.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.745+0000 7fd579a46700 1 -- 172.21.15.146:0/136508486 reap_dead start 2024-09-13T18:52:50.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.745+0000 7fd579a46700 1 --2- 172.21.15.146:0/136508486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd560005230 0x7fd56000ba30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009b70 tx=0x7fd5600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ac85ffb4742bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:50.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.745+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c004020 con 0x7fd560005230 2024-09-13T18:52:50.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.745+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56c004180 con 0x7fd560005230 2024-09-13T18:52:50.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.745+0000 7fd56bfff700 1 -- 172.21.15.146:0/136508486 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c004530 con 0x7fd560005230 2024-09-13T18:52:50.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.745+0000 7fd57a247700 1 -- 172.21.15.146:0/136508486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd560005230 msgr2=0x7fd56000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.745+0000 7fd57a247700 1 --2- 172.21.15.146:0/136508486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd560005230 0x7fd56000ba30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009b70 tx=0x7fd5600058a0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.745+0000 7fd57a247700 1 -- 172.21.15.146:0/136508486 shutdown_connections 2024-09-13T18:52:50.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.745+0000 7fd57a247700 1 --2- 172.21.15.146:0/136508486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd560005230 0x7fd56000ba30 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.745+0000 7fd57a247700 1 -- 172.21.15.146:0/136508486 >> 172.21.15.146:0/136508486 conn(0x7fd574066b00 msgr2=0x7fd57406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:50.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.745+0000 7fd57a247700 1 -- 172.21.15.146:0/136508486 shutdown_connections 2024-09-13T18:52:50.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.745+0000 7fd57a247700 1 -- 172.21.15.146:0/136508486 wait complete. 2024-09-13T18:52:50.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd57a247700 1 Processor -- start 2024-09-13T18:52:50.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd57a247700 1 -- start start 2024-09-13T18:52:50.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd57a247700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560005230 0x7fd574152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd57a247700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd574150070 0x7fd5741504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd57a247700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd574140f90 0x7fd574141410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd57a247700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd574141950 con 0x7fd574150070 2024-09-13T18:52:50.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd57a247700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5740c8400 con 0x7fd560005230 2024-09-13T18:52:50.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd57a247700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5740c86e0 con 0x7fd574140f90 2024-09-13T18:52:50.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd578a44700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd574150070 0x7fd5741504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd578a44700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd574150070 0x7fd5741504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40288/0 (socket says 172.21.15.146:40288) 2024-09-13T18:52:50.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd578a44700 1 -- 172.21.15.146:0/1601890266 learned_addr learned my addr 172.21.15.146:0/1601890266 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:50.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd579245700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560005230 0x7fd574152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd579a46700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd574140f90 0x7fd574141410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd578a44700 1 -- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd574140f90 msgr2=0x7fd574141410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd578a44700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd574140f90 0x7fd574141410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd578a44700 1 -- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560005230 msgr2=0x7fd574152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd578a44700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560005230 0x7fd574152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd578a44700 1 -- 172.21.15.146:0/1601890266 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5740c89c0 con 0x7fd574150070 2024-09-13T18:52:50.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd578a44700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd574150070 0x7fd5741504f0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fd570002060 tx=0x7fd57000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:50.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd569ffb700 1 -- 172.21.15.146:0/1601890266 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570003c30 con 0x7fd574150070 2024-09-13T18:52:50.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd569ffb700 1 -- 172.21.15.146:0/1601890266 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd570003d90 con 0x7fd574150070 2024-09-13T18:52:50.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd57a247700 1 -- 172.21.15.146:0/1601890266 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5740c8ca0 con 0x7fd574150070 2024-09-13T18:52:50.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd569ffb700 1 -- 172.21.15.146:0/1601890266 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000dbd0 con 0x7fd574150070 2024-09-13T18:52:50.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.749+0000 7fd57a247700 1 -- 172.21.15.146:0/1601890266 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5741585a0 con 0x7fd574150070 2024-09-13T18:52:50.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.753+0000 7fd57a247700 1 -- 172.21.15.146:0/1601890266 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd574153040 con 0x7fd574150070 2024-09-13T18:52:50.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.753+0000 7fd569ffb700 1 -- 172.21.15.146:0/1601890266 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd57000dd30 con 0x7fd574150070 2024-09-13T18:52:50.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.753+0000 7fd569ffb700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd558066730 0x7fd558068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.753+0000 7fd569ffb700 1 -- 172.21.15.146:0/1601890266 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7fd570084e00 con 0x7fd574150070 2024-09-13T18:52:50.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.753+0000 7fd579245700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd558066730 0x7fd558068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.753+0000 7fd579245700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd558066730 0x7fd558068bf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd564001eb0 tx=0x7fd564007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:50.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.761+0000 7fd569ffb700 1 -- 172.21.15.146:0/1601890266 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd5700518f0 con 0x7fd574150070 2024-09-13T18:52:50.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.969+0000 7fd57a247700 1 -- 172.21.15.146:0/1601890266 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd574158730 con 0x7fd574150070 2024-09-13T18:52:50.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.973+0000 7fd569ffb700 1 -- 172.21.15.146:0/1601890266 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+5527 (secure 0 0 0) 0x7fd570054f10 con 0x7fd574150070 2024-09-13T18:52:50.988 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:50.988 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:52:47.711074+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6984,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:reconnect","state_seq":24,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:50.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.977+0000 7fd57a247700 1 -- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd558066730 msgr2=0x7fd558068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.977+0000 7fd57a247700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd558066730 0x7fd558068bf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd564001eb0 tx=0x7fd564007040 comp rx=0 tx=0).stop 2024-09-13T18:52:50.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.977+0000 7fd57a247700 1 -- 172.21.15.146:0/1601890266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd574150070 msgr2=0x7fd5741504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.977+0000 7fd57a247700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd574150070 0x7fd5741504f0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fd570002060 tx=0x7fd57000d430 comp rx=0 tx=0).stop 2024-09-13T18:52:50.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.977+0000 7fd57a247700 1 -- 172.21.15.146:0/1601890266 shutdown_connections 2024-09-13T18:52:50.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.977+0000 7fd57a247700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd558066730 0x7fd558068bf0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.977+0000 7fd57a247700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd574140f90 0x7fd574141410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.977+0000 7fd57a247700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd574150070 0x7fd5741504f0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.977+0000 7fd57a247700 1 --2- 172.21.15.146:0/1601890266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd560005230 0x7fd574152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.977+0000 7fd57a247700 1 -- 172.21.15.146:0/1601890266 >> 172.21.15.146:0/1601890266 conn(0x7fd574066b00 msgr2=0x7fd574154f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:50.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.977+0000 7fd57a247700 1 -- 172.21.15.146:0/1601890266 shutdown_connections 2024-09-13T18:52:50.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:50.977+0000 7fd57a247700 1 -- 172.21.15.146:0/1601890266 wait complete. 2024-09-13T18:52:50.994 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 100 2024-09-13T18:52:51.007 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:52:53.007 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:53.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb8a50a700 1 Processor -- start 2024-09-13T18:52:53.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb8a50a700 1 -- start start 2024-09-13T18:52:53.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb8a50a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb7c0dce40 0x7fdb7c045e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb8a50a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdb7c04a730 0x7fdb7c04ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb8a50a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdb7c0e2a70 0x7fdb7c0e2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb8a50a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7c04b070 con 0x7fdb7c04a730 2024-09-13T18:52:53.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb8a50a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7c045a00 con 0x7fdb7c0e2a70 2024-09-13T18:52:53.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb8a50a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c0dd220 con 0x7fdb7c0dce40 2024-09-13T18:52:53.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb88d07700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdb7c04a730 0x7fdb7c04ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb88d07700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdb7c04a730 0x7fdb7c04ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38140/0 (socket says 172.21.15.146:38140) 2024-09-13T18:52:53.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb89508700 1 --2- 172.21.15.146:0/4147566867 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb7c0dce40 0x7fdb7c045e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb88d07700 1 -- 172.21.15.146:0/4147566867 learned_addr learned my addr 172.21.15.146:0/4147566867 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:53.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb89d09700 1 --2- 172.21.15.146:0/4147566867 >> v2:172.21.15.154:3300/0 conn(0x7fdb7c0e2a70 0x7fdb7c0e2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb88d07700 1 -- 172.21.15.146:0/4147566867 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb7c0dce40 msgr2=0x7fdb7c045e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:53.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb88d07700 1 --2- 172.21.15.146:0/4147566867 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb7c0dce40 0x7fdb7c045e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb88d07700 1 -- 172.21.15.146:0/4147566867 >> v2:172.21.15.154:3300/0 conn(0x7fdb7c0e2a70 msgr2=0x7fdb7c0e2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:53.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb88d07700 1 --2- 172.21.15.146:0/4147566867 >> v2:172.21.15.154:3300/0 conn(0x7fdb7c0e2a70 0x7fdb7c0e2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb88d07700 1 -- 172.21.15.146:0/4147566867 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c0e38a0 con 0x7fdb7c04a730 2024-09-13T18:52:53.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb88d07700 1 --2- 172.21.15.146:0/4147566867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7c04a730 0x7fdb7c04ab30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c04dbb0 tx=0x7fdb74009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0b3a0e0ff44f70 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:53.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb7bfff700 1 -- 172.21.15.146:0/4147566867 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb74011830 con 0x7fdb7c04a730 2024-09-13T18:52:53.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb7bfff700 1 -- 172.21.15.146:0/4147566867 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb74011990 con 0x7fdb7c04a730 2024-09-13T18:52:53.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb7bfff700 1 -- 172.21.15.146:0/4147566867 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb74011d20 con 0x7fdb7c04a730 2024-09-13T18:52:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb8a50a700 1 -- 172.21.15.146:0/4147566867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7c04a730 msgr2=0x7fdb7c04ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.117+0000 7fdb8a50a700 1 --2- 172.21.15.146:0/4147566867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7c04a730 0x7fdb7c04ab30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c04dbb0 tx=0x7fdb74009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.121+0000 7fdb8a50a700 1 -- 172.21.15.146:0/4147566867 shutdown_connections 2024-09-13T18:52:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.121+0000 7fdb8a50a700 1 --2- 172.21.15.146:0/4147566867 >> v2:172.21.15.154:3300/0 conn(0x7fdb7c0e2a70 0x7fdb7c0e2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.121+0000 7fdb8a50a700 1 --2- 172.21.15.146:0/4147566867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7c04a730 0x7fdb7c04ab30 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.121+0000 7fdb8a50a700 1 --2- 172.21.15.146:0/4147566867 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb7c0dce40 0x7fdb7c045e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.121+0000 7fdb8a50a700 1 -- 172.21.15.146:0/4147566867 >> 172.21.15.146:0/4147566867 conn(0x7fdb7c04fd00 msgr2=0x7fdb7c050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:53.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.121+0000 7fdb8a50a700 1 -- 172.21.15.146:0/4147566867 shutdown_connections 2024-09-13T18:52:53.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.121+0000 7fdb8a50a700 1 -- 172.21.15.146:0/4147566867 wait complete. 2024-09-13T18:52:53.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.121+0000 7fdb8a50a700 1 Processor -- start 2024-09-13T18:52:53.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.121+0000 7fdb8a50a700 1 -- start start 2024-09-13T18:52:53.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb8a50a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb7c04a730 0x7fdb7c17be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb8a50a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7c0dce40 0x7fdb7c17c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb8a50a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb7c0e2a70 0x7fdb7c0e1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb8a50a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c0e25a0 con 0x7fdb7c0dce40 2024-09-13T18:52:53.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb8a50a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c0e27d0 con 0x7fdb7c0e2a70 2024-09-13T18:52:53.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb8a50a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c0d79a0 con 0x7fdb7c04a730 2024-09-13T18:52:53.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb88d07700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7c0dce40 0x7fdb7c17c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb89508700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb7c04a730 0x7fdb7c17be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb88d07700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7c0dce40 0x7fdb7c17c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38152/0 (socket says 172.21.15.146:38152) 2024-09-13T18:52:53.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb88d07700 1 -- 172.21.15.146:0/3432030123 learned_addr learned my addr 172.21.15.146:0/3432030123 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:53.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb89d09700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb7c0e2a70 0x7fdb7c0e1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb88d07700 1 -- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb7c04a730 msgr2=0x7fdb7c17be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb88d07700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb7c04a730 0x7fdb7c17be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb88d07700 1 -- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb7c0e2a70 msgr2=0x7fdb7c0e1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb88d07700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb7c0e2a70 0x7fdb7c0e1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb88d07700 1 -- 172.21.15.146:0/3432030123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb74009570 con 0x7fdb7c0dce40 2024-09-13T18:52:53.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb89508700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb7c04a730 0x7fdb7c17be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:53.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb88d07700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7c0dce40 0x7fdb7c17c340 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fdb74013000 tx=0x7fdb74004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:53.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb79ffb700 1 -- 172.21.15.146:0/3432030123 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb74004240 con 0x7fdb7c0dce40 2024-09-13T18:52:53.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb79ffb700 1 -- 172.21.15.146:0/3432030123 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7401f070 con 0x7fdb7c0dce40 2024-09-13T18:52:53.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb8a50a700 1 -- 172.21.15.146:0/3432030123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7c0e4230 con 0x7fdb7c0dce40 2024-09-13T18:52:53.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb79ffb700 1 -- 172.21.15.146:0/3432030123 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb74004600 con 0x7fdb7c0dce40 2024-09-13T18:52:53.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb8a50a700 1 -- 172.21.15.146:0/3432030123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7c0e3c70 con 0x7fdb7c0dce40 2024-09-13T18:52:53.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb8a50a700 1 -- 172.21.15.146:0/3432030123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7c04ba40 con 0x7fdb7c0dce40 2024-09-13T18:52:53.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.125+0000 7fdb79ffb700 1 -- 172.21.15.146:0/3432030123 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdb74021030 con 0x7fdb7c0dce40 2024-09-13T18:52:53.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.129+0000 7fdb79ffb700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdb68066730 0x7fdb68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.129+0000 7fdb79ffb700 1 -- 172.21.15.146:0/3432030123 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7fdb740855e0 con 0x7fdb7c0dce40 2024-09-13T18:52:53.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.129+0000 7fdb89508700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdb68066730 0x7fdb68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.129+0000 7fdb89508700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdb68066730 0x7fdb68068bf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdb800112c0 tx=0x7fdb80009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:53.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.137+0000 7fdb79ffb700 1 -- 172.21.15.146:0/3432030123 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdb74052110 con 0x7fdb7c0dce40 2024-09-13T18:52:53.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.345+0000 7fdb8a50a700 1 -- 172.21.15.146:0/3432030123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb7c0e61e0 con 0x7fdb7c0dce40 2024-09-13T18:52:53.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.345+0000 7fdb79ffb700 1 -- 172.21.15.146:0/3432030123 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5524 (secure 0 0 0) 0x7fdb74015020 con 0x7fdb7c0dce40 2024-09-13T18:52:53.362 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:53.362 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:52:52.734663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6984,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:rejoin","state_seq":26,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:53.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.349+0000 7fdb8a50a700 1 -- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdb68066730 msgr2=0x7fdb68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.349+0000 7fdb8a50a700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdb68066730 0x7fdb68068bf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdb800112c0 tx=0x7fdb80009240 comp rx=0 tx=0).stop 2024-09-13T18:52:53.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.349+0000 7fdb8a50a700 1 -- 172.21.15.146:0/3432030123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7c0dce40 msgr2=0x7fdb7c17c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.349+0000 7fdb8a50a700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7c0dce40 0x7fdb7c17c340 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fdb74013000 tx=0x7fdb74004770 comp rx=0 tx=0).stop 2024-09-13T18:52:53.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.349+0000 7fdb8a50a700 1 -- 172.21.15.146:0/3432030123 shutdown_connections 2024-09-13T18:52:53.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.349+0000 7fdb8a50a700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fdb68066730 0x7fdb68068bf0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.349+0000 7fdb8a50a700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb7c0e2a70 0x7fdb7c0e1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.349+0000 7fdb8a50a700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7c0dce40 0x7fdb7c17c340 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.349+0000 7fdb8a50a700 1 --2- 172.21.15.146:0/3432030123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb7c04a730 0x7fdb7c17be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.349+0000 7fdb8a50a700 1 -- 172.21.15.146:0/3432030123 >> 172.21.15.146:0/3432030123 conn(0x7fdb7c04fd00 msgr2=0x7fdb7c0059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:53.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.349+0000 7fdb8a50a700 1 -- 172.21.15.146:0/3432030123 shutdown_connections 2024-09-13T18:52:53.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:53.349+0000 7fdb8a50a700 1 -- 172.21.15.146:0/3432030123 wait complete. 2024-09-13T18:52:53.366 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 101 2024-09-13T18:52:53.377 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:52:55.378 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:55.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.481+0000 7f0c62959700 1 Processor -- start 2024-09-13T18:52:55.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.481+0000 7f0c62959700 1 -- start start 2024-09-13T18:52:55.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.481+0000 7f0c62959700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c5c070940 0x7f0c5c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.481+0000 7f0c62959700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5c071430 con 0x7f0c5c06bb10 2024-09-13T18:52:55.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.481+0000 7f0c62959700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5c15da60 con 0x7f0c5c1530d0 2024-09-13T18:52:55.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.481+0000 7f0c62959700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c15a530 con 0x7f0c5c070940 2024-09-13T18:52:55.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.481+0000 7f0c61156700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0c5c06bb10 0x7f0c5c070d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48552/0 (socket says 172.21.15.146:48552) 2024-09-13T18:52:55.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.481+0000 7f0c61156700 1 -- 172.21.15.146:0/3022888786 learned_addr learned my addr 172.21.15.146:0/3022888786 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:52:55.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.481+0000 7f0c61957700 1 --2- 172.21.15.146:0/3022888786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c5c070940 0x7f0c5c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c60955700 1 -- 172.21.15.146:0/3022888786 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395401335 0 0) 0x7f0c5c071430 con 0x7f0c5c06bb10 2024-09-13T18:52:55.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c60955700 1 -- 172.21.15.146:0/3022888786 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c480035d0 con 0x7f0c5c06bb10 2024-09-13T18:52:55.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c60955700 1 -- 172.21.15.146:0/3022888786 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884730578 0 0) 0x7f0c5c15da60 con 0x7f0c5c1530d0 2024-09-13T18:52:55.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c60955700 1 -- 172.21.15.146:0/3022888786 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c5c071430 con 0x7f0c5c1530d0 2024-09-13T18:52:55.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c61957700 1 -- 172.21.15.146:0/3022888786 >> v1:172.21.15.154:6789/0 conn(0x7f0c5c1530d0 legacy=0x7f0c5c153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c61957700 1 -- 172.21.15.146:0/3022888786 >> v1:172.21.15.146:6789/0 conn(0x7f0c5c06bb10 legacy=0x7f0c5c070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c61957700 1 -- 172.21.15.146:0/3022888786 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c15cc00 con 0x7f0c5c070940 2024-09-13T18:52:55.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c61957700 1 --2- 172.21.15.146:0/3022888786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c5c070940 0x7f0c5c159ff0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009dc0 tx=0x7f0c5400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c03042723ffc84e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:55.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c60955700 1 -- 172.21.15.146:0/3022888786 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400e070 con 0x7f0c5c070940 2024-09-13T18:52:55.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c60955700 1 -- 172.21.15.146:0/3022888786 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5400bb20 con 0x7f0c5c070940 2024-09-13T18:52:55.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c60955700 1 -- 172.21.15.146:0/3022888786 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54012430 con 0x7f0c5c070940 2024-09-13T18:52:55.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c62959700 1 -- 172.21.15.146:0/3022888786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c5c070940 msgr2=0x7f0c5c159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c62959700 1 --2- 172.21.15.146:0/3022888786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c5c070940 0x7f0c5c159ff0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009dc0 tx=0x7f0c5400b330 comp rx=0 tx=0).stop 2024-09-13T18:52:55.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c62959700 1 -- 172.21.15.146:0/3022888786 shutdown_connections 2024-09-13T18:52:55.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c62959700 1 --2- 172.21.15.146:0/3022888786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c5c070940 0x7f0c5c159ff0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c62959700 1 -- 172.21.15.146:0/3022888786 >> 172.21.15.146:0/3022888786 conn(0x7f0c5c06c2d0 msgr2=0x7f0c5c06c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:55.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c62959700 1 -- 172.21.15.146:0/3022888786 shutdown_connections 2024-09-13T18:52:55.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.485+0000 7f0c62959700 1 -- 172.21.15.146:0/3022888786 wait complete. 2024-09-13T18:52:55.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62959700 1 Processor -- start 2024-09-13T18:52:55.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62959700 1 -- start start 2024-09-13T18:52:55.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62959700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c06bb10 0x7f0c5c1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62959700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c5c070940 0x7f0c5c0c2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62959700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c5c1530d0 0x7f0c5c0c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62959700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0cb630 con 0x7f0c5c1530d0 2024-09-13T18:52:55.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62959700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c1598f0 con 0x7f0c5c06bb10 2024-09-13T18:52:55.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62959700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c159b20 con 0x7f0c5c070940 2024-09-13T18:52:55.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62158700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c5c1530d0 0x7f0c5c0c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c61156700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c5c070940 0x7f0c5c0c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c61957700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c06bb10 0x7f0c5c1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62158700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c5c1530d0 0x7f0c5c0c1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38166/0 (socket says 172.21.15.146:38166) 2024-09-13T18:52:55.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62158700 1 -- 172.21.15.146:0/129794501 learned_addr learned my addr 172.21.15.146:0/129794501 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:55.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62158700 1 -- 172.21.15.146:0/129794501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c5c070940 msgr2=0x7f0c5c0c2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62158700 1 --2- 172.21.15.146:0/129794501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c5c070940 0x7f0c5c0c2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62158700 1 -- 172.21.15.146:0/129794501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c06bb10 msgr2=0x7f0c5c1573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62158700 1 --2- 172.21.15.146:0/129794501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c06bb10 0x7f0c5c1573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62158700 1 -- 172.21.15.146:0/129794501 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c54009a70 con 0x7f0c5c1530d0 2024-09-13T18:52:55.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62158700 1 --2- 172.21.15.146:0/129794501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c5c1530d0 0x7f0c5c0c1b30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800c410 tx=0x7f0c5800c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:55.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c527fc700 1 -- 172.21.15.146:0/129794501 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c580052d0 con 0x7f0c5c1530d0 2024-09-13T18:52:55.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c527fc700 1 -- 172.21.15.146:0/129794501 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c58005430 con 0x7f0c5c1530d0 2024-09-13T18:52:55.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62959700 1 -- 172.21.15.146:0/129794501 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5c0c30d0 con 0x7f0c5c1530d0 2024-09-13T18:52:55.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c527fc700 1 -- 172.21.15.146:0/129794501 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c580055a0 con 0x7f0c5c1530d0 2024-09-13T18:52:55.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.489+0000 7f0c62959700 1 -- 172.21.15.146:0/129794501 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c1f0040 con 0x7f0c5c1530d0 2024-09-13T18:52:55.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.493+0000 7f0c62959700 1 -- 172.21.15.146:0/129794501 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5c1efea0 con 0x7f0c5c1530d0 2024-09-13T18:52:55.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.493+0000 7f0c527fc700 1 -- 172.21.15.146:0/129794501 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0c580038e0 con 0x7f0c5c1530d0 2024-09-13T18:52:55.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.497+0000 7f0c527fc700 1 --2- 172.21.15.146:0/129794501 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c40066760 0x7f0c40068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.497+0000 7f0c527fc700 1 -- 172.21.15.146:0/129794501 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f0c58083f50 con 0x7f0c5c1530d0 2024-09-13T18:52:55.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.497+0000 7f0c61957700 1 --2- 172.21.15.146:0/129794501 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c40066760 0x7f0c40068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.497+0000 7f0c61957700 1 --2- 172.21.15.146:0/129794501 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c40066760 0x7f0c40068c20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0c54014040 tx=0x7f0c5401e040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:55.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.501+0000 7f0c527fc700 1 -- 172.21.15.146:0/129794501 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0c58050b80 con 0x7f0c5c1530d0 2024-09-13T18:52:55.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.705+0000 7f0c62959700 1 -- 172.21.15.146:0/129794501 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c5c1efea0 con 0x7f0c5c1530d0 2024-09-13T18:52:55.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.709+0000 7f0c527fc700 1 -- 172.21.15.146:0/129794501 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5524 (secure 0 0 0) 0x7f0c5c1efea0 con 0x7f0c5c1530d0 2024-09-13T18:52:55.723 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:55.723 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:52:52.734663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6984,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:rejoin","state_seq":26,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:55.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.709+0000 7f0c62959700 1 -- 172.21.15.146:0/129794501 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c40066760 msgr2=0x7f0c40068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.709+0000 7f0c62959700 1 --2- 172.21.15.146:0/129794501 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c40066760 0x7f0c40068c20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0c54014040 tx=0x7f0c5401e040 comp rx=0 tx=0).stop 2024-09-13T18:52:55.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.709+0000 7f0c62959700 1 -- 172.21.15.146:0/129794501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c5c1530d0 msgr2=0x7f0c5c0c1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.709+0000 7f0c62959700 1 --2- 172.21.15.146:0/129794501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c5c1530d0 0x7f0c5c0c1b30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800c410 tx=0x7f0c5800c720 comp rx=0 tx=0).stop 2024-09-13T18:52:55.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.709+0000 7f0c62959700 1 -- 172.21.15.146:0/129794501 shutdown_connections 2024-09-13T18:52:55.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.709+0000 7f0c62959700 1 --2- 172.21.15.146:0/129794501 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0c40066760 0x7f0c40068c20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.709+0000 7f0c62959700 1 --2- 172.21.15.146:0/129794501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c5c1530d0 0x7f0c5c0c1b30 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.709+0000 7f0c62959700 1 --2- 172.21.15.146:0/129794501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c5c070940 0x7f0c5c0c2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.709+0000 7f0c62959700 1 --2- 172.21.15.146:0/129794501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c06bb10 0x7f0c5c1573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.709+0000 7f0c62959700 1 -- 172.21.15.146:0/129794501 >> 172.21.15.146:0/129794501 conn(0x7f0c5c06c2d0 msgr2=0x7f0c5c15beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:55.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.709+0000 7f0c62959700 1 -- 172.21.15.146:0/129794501 shutdown_connections 2024-09-13T18:52:55.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:55.709+0000 7f0c62959700 1 -- 172.21.15.146:0/129794501 wait complete. 2024-09-13T18:52:55.727 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 101 2024-09-13T18:52:55.739 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:52:57.740 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:57.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd9273700 1 Processor -- start 2024-09-13T18:52:57.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd9273700 1 -- start start 2024-09-13T18:52:57.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd9273700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3cd40c5fc0 0x7f3cd40c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd9273700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3cd4159d70 0x7f3cd40c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd9273700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd40cad60 con 0x7f3cd40c63a0 2024-09-13T18:52:57.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd9273700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd40c9bd0 con 0x7f3cd4159d70 2024-09-13T18:52:57.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd9273700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd40c8c70 con 0x7f3cd40c5fc0 2024-09-13T18:52:57.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd8a72700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3cd4159d70 0x7f3cd40c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd8a72700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3cd4159d70 0x7f3cd40c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38168/0 (socket says 172.21.15.146:38168) 2024-09-13T18:52:57.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd8a72700 1 -- 172.21.15.146:0/4051712161 learned_addr learned my addr 172.21.15.146:0/4051712161 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:57.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd3fff700 1 --2- 172.21.15.146:0/4051712161 >> v2:172.21.15.154:3300/0 conn(0x7f3cd40c5fc0 0x7f3cd40c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd8a72700 1 -- 172.21.15.146:0/4051712161 >> v2:172.21.15.154:3300/0 conn(0x7f3cd40c5fc0 msgr2=0x7f3cd40c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:57.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd8a72700 1 --2- 172.21.15.146:0/4051712161 >> v2:172.21.15.154:3300/0 conn(0x7f3cd40c5fc0 0x7f3cd40c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd8a72700 1 -- 172.21.15.146:0/4051712161 >> v1:172.21.15.154:6789/0 conn(0x7f3cd40c63a0 legacy=0x7f3cd4159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd8a72700 1 -- 172.21.15.146:0/4051712161 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd40c8f50 con 0x7f3cd4159d70 2024-09-13T18:52:57.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd3fff700 1 --2- 172.21.15.146:0/4051712161 >> v2:172.21.15.154:3300/0 conn(0x7f3cd40c5fc0 0x7f3cd40c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:57.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd8a72700 1 --2- 172.21.15.146:0/4051712161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cd4159d70 0x7f3cd40c7140 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40729c0 tx=0x7f3ccc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c526761eab931b08 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:57.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd2ffd700 1 -- 172.21.15.146:0/4051712161 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc00f8a0 con 0x7f3cd4159d70 2024-09-13T18:52:57.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd2ffd700 1 -- 172.21.15.146:0/4051712161 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc00fa00 con 0x7f3cd4159d70 2024-09-13T18:52:57.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.849+0000 7f3cd2ffd700 1 -- 172.21.15.146:0/4051712161 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc00fd90 con 0x7f3cd4159d70 2024-09-13T18:52:57.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.853+0000 7f3cd9273700 1 -- 172.21.15.146:0/4051712161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cd4159d70 msgr2=0x7f3cd40c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.853+0000 7f3cd9273700 1 --2- 172.21.15.146:0/4051712161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cd4159d70 0x7f3cd40c7140 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40729c0 tx=0x7f3ccc009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.853+0000 7f3cd9273700 1 -- 172.21.15.146:0/4051712161 shutdown_connections 2024-09-13T18:52:57.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.853+0000 7f3cd9273700 1 --2- 172.21.15.146:0/4051712161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cd4159d70 0x7f3cd40c7140 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.853+0000 7f3cd9273700 1 --2- 172.21.15.146:0/4051712161 >> v2:172.21.15.154:3300/0 conn(0x7f3cd40c5fc0 0x7f3cd40c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.853+0000 7f3cd9273700 1 -- 172.21.15.146:0/4051712161 >> 172.21.15.146:0/4051712161 conn(0x7f3cd4066090 msgr2=0x7f3cd4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:57.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.853+0000 7f3cd9273700 1 -- 172.21.15.146:0/4051712161 shutdown_connections 2024-09-13T18:52:57.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.853+0000 7f3cd9273700 1 -- 172.21.15.146:0/4051712161 wait complete. 2024-09-13T18:52:57.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.853+0000 7f3cd9273700 1 Processor -- start 2024-09-13T18:52:57.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.853+0000 7f3cd9273700 1 -- start start 2024-09-13T18:52:57.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd9273700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cd40c5fc0 0x7f3cd41efe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd9273700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd40c63a0 0x7f3cd40c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd9273700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cd4159d70 0x7f3cd41f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd9273700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40c46b0 con 0x7f3cd40c5fc0 2024-09-13T18:52:57.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd9273700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41f2af0 con 0x7f3cd40c63a0 2024-09-13T18:52:57.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd9273700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40c33f0 con 0x7f3cd4159d70 2024-09-13T18:52:57.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cd40c5fc0 0x7f3cd41efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cd40c5fc0 0x7f3cd41efe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38180/0 (socket says 172.21.15.146:38180) 2024-09-13T18:52:57.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd3fff700 1 -- 172.21.15.146:0/591556218 learned_addr learned my addr 172.21.15.146:0/591556218 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:52:57.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd8a72700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cd4159d70 0x7f3cd41f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd37fe700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd40c63a0 0x7f3cd40c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd3fff700 1 -- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cd4159d70 msgr2=0x7f3cd41f2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd3fff700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cd4159d70 0x7f3cd41f2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd3fff700 1 -- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd40c63a0 msgr2=0x7f3cd40c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd3fff700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd40c63a0 0x7f3cd40c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd3fff700 1 -- 172.21.15.146:0/591556218 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc009570 con 0x7f3cd40c5fc0 2024-09-13T18:52:57.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd3fff700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cd40c5fc0 0x7f3cd41efe50 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800e760 tx=0x7f3cc800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:57.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd0ff9700 1 -- 172.21.15.146:0/591556218 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc800c070 con 0x7f3cd40c5fc0 2024-09-13T18:52:57.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd0ff9700 1 -- 172.21.15.146:0/591556218 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc8004250 con 0x7f3cd40c5fc0 2024-09-13T18:52:57.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd9273700 1 -- 172.21.15.146:0/591556218 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd40c3690 con 0x7f3cd40c5fc0 2024-09-13T18:52:57.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd0ff9700 1 -- 172.21.15.146:0/591556218 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80036c0 con 0x7f3cd40c5fc0 2024-09-13T18:52:57.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd9273700 1 -- 172.21.15.146:0/591556218 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd40c3c50 con 0x7f3cd40c5fc0 2024-09-13T18:52:57.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.857+0000 7f3cd9273700 1 -- 172.21.15.146:0/591556218 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd40cb790 con 0x7f3cd40c5fc0 2024-09-13T18:52:57.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.861+0000 7f3cd0ff9700 1 -- 172.21.15.146:0/591556218 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3cc8003820 con 0x7f3cd40c5fc0 2024-09-13T18:52:57.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.861+0000 7f3cd0ff9700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3cc0066730 0x7f3cc0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.861+0000 7f3cd0ff9700 1 -- 172.21.15.146:0/591556218 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f3cc8084780 con 0x7f3cd40c5fc0 2024-09-13T18:52:57.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.861+0000 7f3cd37fe700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3cc0066730 0x7f3cc0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.861+0000 7f3cd37fe700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3cc0066730 0x7f3cc0068bf0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40afac0 tx=0x7f3cc4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:57.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:57.869+0000 7f3cd0ff9700 1 -- 172.21.15.146:0/591556218 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3cc8051150 con 0x7f3cd40c5fc0 2024-09-13T18:52:58.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.081+0000 7f3cd9273700 1 -- 172.21.15.146:0/591556218 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cd40c1f40 con 0x7f3cd40c5fc0 2024-09-13T18:52:58.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.081+0000 7f3cd0ff9700 1 -- 172.21.15.146:0/591556218 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5524 (secure 0 0 0) 0x7f3cc80094e0 con 0x7f3cd40c5fc0 2024-09-13T18:52:58.098 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:52:58.098 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":102,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:52:57.765909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6984,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:58.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.085+0000 7f3cd9273700 1 -- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3cc0066730 msgr2=0x7f3cc0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.085+0000 7f3cd9273700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3cc0066730 0x7f3cc0068bf0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40afac0 tx=0x7f3cc4009040 comp rx=0 tx=0).stop 2024-09-13T18:52:58.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.085+0000 7f3cd9273700 1 -- 172.21.15.146:0/591556218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cd40c5fc0 msgr2=0x7f3cd41efe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.085+0000 7f3cd9273700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cd40c5fc0 0x7f3cd41efe50 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800e760 tx=0x7f3cc800ea70 comp rx=0 tx=0).stop 2024-09-13T18:52:58.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.085+0000 7f3cd9273700 1 -- 172.21.15.146:0/591556218 shutdown_connections 2024-09-13T18:52:58.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.085+0000 7f3cd9273700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3cc0066730 0x7f3cc0068bf0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.085+0000 7f3cd9273700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cd4159d70 0x7f3cd41f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.085+0000 7f3cd9273700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd40c63a0 0x7f3cd40c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.085+0000 7f3cd9273700 1 --2- 172.21.15.146:0/591556218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cd40c5fc0 0x7f3cd41efe50 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.085+0000 7f3cd9273700 1 -- 172.21.15.146:0/591556218 >> 172.21.15.146:0/591556218 conn(0x7f3cd4066090 msgr2=0x7f3cd4072160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:58.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.085+0000 7f3cd9273700 1 -- 172.21.15.146:0/591556218 shutdown_connections 2024-09-13T18:52:58.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:52:58.085+0000 7f3cd9273700 1 -- 172.21.15.146:0/591556218 wait complete. 2024-09-13T18:52:58.102 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 102 2024-09-13T18:52:58.114 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=6187 2024-09-13T18:52:58.114 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.e 2024-09-13T18:53:05.115 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2024-09-13T18:53:05.115 INFO:tasks.ceph.mds.e:Restarting daemon 2024-09-13T18:53:05.116 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-09-13T18:53:05.118 INFO:tasks.ceph.mds.e:Started 2024-09-13T18:53:05.118 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:05.593 INFO:tasks.ceph.mds.e.smithi146.stdout:starting mds.e at 2024-09-13T18:53:05.621 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.597+0000 7f211e1b2700 1 Processor -- start 2024-09-13T18:53:05.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.597+0000 7f211e1b2700 1 -- start start 2024-09-13T18:53:05.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.597+0000 7f211e1b2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21180cb2b0 0x7f21180c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.597+0000 7f211e1b2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2118161060 con 0x7f21180c8230 2024-09-13T18:53:05.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.597+0000 7f211e1b2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2118161300 con 0x7f21180caed0 2024-09-13T18:53:05.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.597+0000 7f211e1b2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21180c78d0 con 0x7f21180cb2b0 2024-09-13T18:53:05.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.597+0000 7f2117fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f21180c8230 0x7f211815e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49764/0 (socket says 172.21.15.146:49764) 2024-09-13T18:53:05.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.597+0000 7f2117fff700 1 -- 172.21.15.146:0/374099387 learned_addr learned my addr 172.21.15.146:0/374099387 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:53:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552070055 0 0) 0x7f2118161300 con 0x7f21180caed0 2024-09-13T18:53:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21000035d0 con 0x7f21180caed0 2024-09-13T18:53:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041630123 0 0) 0x7f2118161060 con 0x7f21180c8230 2024-09-13T18:53:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2118161300 con 0x7f21180c8230 2024-09-13T18:53:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2148629666 0 0) 0x7f21000035d0 con 0x7f21180caed0 2024-09-13T18:53:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21180cb2b0 msgr2=0x7f21180c7cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:53:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 --2- 172.21.15.146:0/374099387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21180cb2b0 0x7f21180c7cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 >> v1:172.21.15.146:6789/0 conn(0x7f21180c8230 legacy=0x7f211815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21181615c0 con 0x7f21180caed0 2024-09-13T18:53:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f21080015a0 con 0x7f21180caed0 2024-09-13T18:53:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 >> v1:172.21.15.154:6789/0 conn(0x7f21180caed0 legacy=0x7f2118072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 --2- 172.21.15.146:0/374099387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2100003c20 0x7f2100004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 --2- 172.21.15.146:0/374099387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2100029a20 0x7f210002be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 --2- 172.21.15.146:0/374099387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f210002c530 0x7f210002e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21180c78d0 con 0x7f210002c530 2024-09-13T18:53:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210002f2a0 con 0x7f2100003c20 2024-09-13T18:53:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210002f460 con 0x7f2100029a20 2024-09-13T18:53:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f2117fff700 1 --2- 172.21.15.146:0/374099387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f210002c530 0x7f210002e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f2117fff700 1 -- 172.21.15.146:0/374099387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2100029a20 msgr2=0x7f210002be10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:53:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f2117fff700 1 --2- 172.21.15.146:0/374099387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2100029a20 0x7f210002be10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f2117fff700 1 -- 172.21.15.146:0/374099387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2100003c20 msgr2=0x7f2100004020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f2117fff700 1 --2- 172.21.15.146:0/374099387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2100003c20 0x7f2100004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f2117fff700 1 -- 172.21.15.146:0/374099387 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21181615c0 con 0x7f210002c530 2024-09-13T18:53:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f2117fff700 1 --2- 172.21.15.146:0/374099387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f210002c530 0x7f210002e9c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f210c001cc0 tx=0x7f21000058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3cad8e765ad7c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.601+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c004b50 con 0x7f210002c530 2024-09-13T18:53:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.605+0000 7f21167fc700 1 -- 172.21.15.146:0/374099387 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c004cb0 con 0x7f210002c530 2024-09-13T18:53:05.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.605+0000 7f211e1b2700 1 -- 172.21.15.146:0/374099387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f210002c530 msgr2=0x7f210002e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.605+0000 7f211e1b2700 1 --2- 172.21.15.146:0/374099387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f210002c530 0x7f210002e9c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f210c001cc0 tx=0x7f21000058a0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.605+0000 7f2117fff700 1 -- 172.21.15.146:0/374099387 reap_dead start 2024-09-13T18:53:05.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.609+0000 7f211e1b2700 1 -- 172.21.15.146:0/374099387 shutdown_connections 2024-09-13T18:53:05.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.609+0000 7f211e1b2700 1 -- 172.21.15.146:0/374099387 >> 172.21.15.146:0/374099387 conn(0x7f2118066090 msgr2=0x7f2118066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:05.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.621+0000 7f211e1b2700 1 -- 172.21.15.146:0/374099387 shutdown_connections 2024-09-13T18:53:05.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.621+0000 7f211e1b2700 1 -- 172.21.15.146:0/374099387 wait complete. 2024-09-13T18:53:05.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.621+0000 7f211e1b2700 1 Processor -- start 2024-09-13T18:53:05.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.621+0000 7f211e1b2700 1 -- start start 2024-09-13T18:53:05.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.625+0000 7f211e1b2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f210002c530 0x7f2118123e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.625+0000 7f211e1b2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2118126a70 0x7f211806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.625+0000 7f211e1b2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2118158f80 0x7f21181593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.625+0000 7f211e1b2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2118124390 con 0x7f2118158f80 2024-09-13T18:53:05.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.625+0000 7f211e1b2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2118127d50 con 0x7f210002c530 2024-09-13T18:53:05.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.625+0000 7f211e1b2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21180ad480 con 0x7f2118126a70 2024-09-13T18:53:05.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.625+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2118126a70 0x7f211806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.625+0000 7f21177fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f210002c530 0x7f2118123e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.625+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2118126a70 0x7f211806f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:51590/0 (socket says 172.21.15.146:51590) 2024-09-13T18:53:05.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.625+0000 7f2116ffd700 1 -- 172.21.15.146:0/2820649961 learned_addr learned my addr 172.21.15.146:0/2820649961 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:05.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.625+0000 7f2116ffd700 1 -- 172.21.15.146:0/2820649961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f210002c530 msgr2=0x7f2118123e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.625+0000 7f2116ffd700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f210002c530 0x7f2118123e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.625+0000 7f2116ffd700 1 -- 172.21.15.146:0/2820649961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2118158f80 msgr2=0x7f21181593e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:53:05.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.629+0000 7f2117fff700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2118158f80 0x7f21181593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.629+0000 7f2116ffd700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2118158f80 0x7f21181593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.629+0000 7f2116ffd700 1 -- 172.21.15.146:0/2820649961 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21180ad720 con 0x7f2118126a70 2024-09-13T18:53:05.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.629+0000 7f2117fff700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2118158f80 0x7f21181593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:53:05.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.629+0000 7f2116ffd700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2118126a70 0x7f211806f830 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f211000be00 tx=0x7f2110009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:05.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.629+0000 7f20fffff700 1 -- 172.21.15.146:0/2820649961 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211000cc10 con 0x7f2118126a70 2024-09-13T18:53:05.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.629+0000 7f20fffff700 1 -- 172.21.15.146:0/2820649961 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211000cdb0 con 0x7f2118126a70 2024-09-13T18:53:05.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.629+0000 7f20fffff700 1 -- 172.21.15.146:0/2820649961 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110014680 con 0x7f2118126a70 2024-09-13T18:53:05.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.629+0000 7f211e1b2700 1 -- 172.21.15.146:0/2820649961 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2118127a70 con 0x7f2118126a70 2024-09-13T18:53:05.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.629+0000 7f211e1b2700 1 -- 172.21.15.146:0/2820649961 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2118126270 con 0x7f2118126a70 2024-09-13T18:53:05.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.633+0000 7f211e1b2700 1 -- 172.21.15.146:0/2820649961 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21180a9490 con 0x7f2118126a70 2024-09-13T18:53:05.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.633+0000 7f20fffff700 1 -- 172.21.15.146:0/2820649961 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f21100147e0 con 0x7f2118126a70 2024-09-13T18:53:05.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.641+0000 7f20fffff700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2104066800 0x7f2104068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.641+0000 7f20fffff700 1 -- 172.21.15.146:0/2820649961 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f2110084f90 con 0x7f2118126a70 2024-09-13T18:53:05.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.641+0000 7f21177fe700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2104066800 0x7f2104068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.641+0000 7f20fffff700 1 -- 172.21.15.146:0/2820649961 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2110051480 con 0x7f2118126a70 2024-09-13T18:53:05.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:05.653+0000 7f21177fe700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2104066800 0x7f2104068cc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2108007d40 tx=0x7f2108009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:06.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.361+0000 7f211e1b2700 1 -- 172.21.15.146:0/2820649961 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f211806e4f0 con 0x7f2118126a70 2024-09-13T18:53:06.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.361+0000 7f20fffff700 1 -- 172.21.15.146:0/2820649961 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+6282 (secure 0 0 0) 0x7f2110019090 con 0x7f2118126a70 2024-09-13T18:53:06.385 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:06.385 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":7098,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:52:57.765909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6984,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:06.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.377+0000 7f20fdffb700 1 -- 172.21.15.146:0/2820649961 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2104066800 msgr2=0x7f2104068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.377+0000 7f20fdffb700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2104066800 0x7f2104068cc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2108007d40 tx=0x7f2108009040 comp rx=0 tx=0).stop 2024-09-13T18:53:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.377+0000 7f20fdffb700 1 -- 172.21.15.146:0/2820649961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2118126a70 msgr2=0x7f211806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.381+0000 7f20fdffb700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2118126a70 0x7f211806f830 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f211000be00 tx=0x7f2110009ef0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.381+0000 7f20fdffb700 1 -- 172.21.15.146:0/2820649961 shutdown_connections 2024-09-13T18:53:06.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.381+0000 7f20fdffb700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2104066800 0x7f2104068cc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.381+0000 7f20fdffb700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2118158f80 0x7f21181593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.381+0000 7f20fdffb700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2118126a70 0x7f211806f830 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.381+0000 7f20fdffb700 1 --2- 172.21.15.146:0/2820649961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f210002c530 0x7f2118123e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.381+0000 7f20fdffb700 1 -- 172.21.15.146:0/2820649961 >> 172.21.15.146:0/2820649961 conn(0x7f2118066090 msgr2=0x7f21180c5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:06.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.381+0000 7f20fdffb700 1 -- 172.21.15.146:0/2820649961 shutdown_connections 2024-09-13T18:53:06.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:06.381+0000 7f20fdffb700 1 -- 172.21.15.146:0/2820649961 wait complete. 2024-09-13T18:53:06.399 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 103 2024-09-13T18:53:06.416 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2024-09-13T18:53:06.416 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 3 secs before thrashing 2024-09-13T18:53:07.739 INFO:tasks.ceph.mds.a.smithi146.stderr:2024-09-13T18:53:07.725+0000 7f40aeb2a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:53:07.739 INFO:tasks.ceph.mds.a.smithi146.stderr:2024-09-13T18:53:07.725+0000 7f40aeb2a700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-13T18:53:09.418 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:10.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.217+0000 7f1dd9267700 1 Processor -- start 2024-09-13T18:53:10.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.217+0000 7f1dd9267700 1 -- start start 2024-09-13T18:53:10.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd9267700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 0x7f1dd4072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:10.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd9267700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1dd415e170 0x7f1dd415e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:10.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd9267700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd415eb60 con 0x7f1dd40cb2b0 2024-09-13T18:53:10.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd9267700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd40c78d0 con 0x7f1dd415e170 2024-09-13T18:53:10.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd9267700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4161060 con 0x7f1dd40caed0 2024-09-13T18:53:10.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd37fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1dd415e170 0x7f1dd415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:10.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd2ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 0x7f1dd4072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:10.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd37fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1dd415e170 0x7f1dd415e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:53080/0 (socket says 172.21.15.146:53080) 2024-09-13T18:53:10.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd2ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 0x7f1dd4072380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:51598/0 (socket says 172.21.15.146:51598) 2024-09-13T18:53:10.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd37fe700 1 -- 172.21.15.146:0/2921662960 learned_addr learned my addr 172.21.15.146:0/2921662960 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:10.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd37fe700 1 -- 172.21.15.146:0/2921662960 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 msgr2=0x7f1dd4072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:10.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd37fe700 1 --2- 172.21.15.146:0/2921662960 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 0x7f1dd4072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd37fe700 1 -- 172.21.15.146:0/2921662960 >> v1:172.21.15.154:6789/0 conn(0x7f1dd40cb2b0 legacy=0x7f1dd40c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:10.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd37fe700 1 -- 172.21.15.146:0/2921662960 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd4161300 con 0x7f1dd415e170 2024-09-13T18:53:10.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.221+0000 7f1dd2ffd700 1 --2- 172.21.15.146:0/2921662960 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 0x7f1dd4072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:10.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.225+0000 7f1dd37fe700 1 --2- 172.21.15.146:0/2921662960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd415e170 0x7f1dd415e620 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009910 tx=0x7f1dc4009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b00c0013079a8e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:10.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.225+0000 7f1dd1ffb700 1 -- 172.21.15.146:0/2921662960 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4011830 con 0x7f1dd415e170 2024-09-13T18:53:10.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.225+0000 7f1dd1ffb700 1 -- 172.21.15.146:0/2921662960 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc4011990 con 0x7f1dd415e170 2024-09-13T18:53:10.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.225+0000 7f1dd1ffb700 1 -- 172.21.15.146:0/2921662960 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4011d40 con 0x7f1dd415e170 2024-09-13T18:53:10.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.229+0000 7f1dd9267700 1 -- 172.21.15.146:0/2921662960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd415e170 msgr2=0x7f1dd415e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:10.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.229+0000 7f1dd9267700 1 --2- 172.21.15.146:0/2921662960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd415e170 0x7f1dd415e620 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009910 tx=0x7f1dc4009c20 comp rx=0 tx=0).stop 2024-09-13T18:53:10.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.241+0000 7f1dd9267700 1 -- 172.21.15.146:0/2921662960 shutdown_connections 2024-09-13T18:53:10.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.241+0000 7f1dd9267700 1 --2- 172.21.15.146:0/2921662960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd415e170 0x7f1dd415e620 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.241+0000 7f1dd9267700 1 --2- 172.21.15.146:0/2921662960 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 0x7f1dd4072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.241+0000 7f1dd9267700 1 -- 172.21.15.146:0/2921662960 >> 172.21.15.146:0/2921662960 conn(0x7f1dd4066090 msgr2=0x7f1dd4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:10.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.241+0000 7f1dd9267700 1 -- 172.21.15.146:0/2921662960 shutdown_connections 2024-09-13T18:53:10.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.241+0000 7f1dd9267700 1 -- 172.21.15.146:0/2921662960 wait complete. 2024-09-13T18:53:10.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.245+0000 7f1dd9267700 1 Processor -- start 2024-09-13T18:53:10.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd9267700 1 -- start start 2024-09-13T18:53:10.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd9267700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 0x7f1dd406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:10.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd9267700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd40cb2b0 0x7f1dd414cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:10.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd9267700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dd415e170 0x7f1dd414d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:10.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd9267700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd414e660 con 0x7f1dd415e170 2024-09-13T18:53:10.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd9267700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd406cbc0 con 0x7f1dd40cb2b0 2024-09-13T18:53:10.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd9267700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4145e60 con 0x7f1dd40caed0 2024-09-13T18:53:10.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd2ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 0x7f1dd406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:10.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd2ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 0x7f1dd406c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:51606/0 (socket says 172.21.15.146:51606) 2024-09-13T18:53:10.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd2ffd700 1 -- 172.21.15.146:0/2685126274 learned_addr learned my addr 172.21.15.146:0/2685126274 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:10.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd37fe700 1 --2- 172.21.15.146:0/2685126274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dd415e170 0x7f1dd414d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:10.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd2ffd700 1 -- 172.21.15.146:0/2685126274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd40cb2b0 msgr2=0x7f1dd414cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:10.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd2ffd700 1 --2- 172.21.15.146:0/2685126274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd40cb2b0 0x7f1dd414cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd2ffd700 1 -- 172.21.15.146:0/2685126274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dd415e170 msgr2=0x7f1dd414d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:10.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd2ffd700 1 --2- 172.21.15.146:0/2685126274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dd415e170 0x7f1dd414d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.249+0000 7f1dd2ffd700 1 -- 172.21.15.146:0/2685126274 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc80094b0 con 0x7f1dd40caed0 2024-09-13T18:53:10.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.253+0000 7f1dd2ffd700 1 --2- 172.21.15.146:0/2685126274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 0x7f1dd406c680 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800ec10 tx=0x7f1dc8009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:10.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.253+0000 7f1dc37fe700 1 -- 172.21.15.146:0/2685126274 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc800c940 con 0x7f1dd40caed0 2024-09-13T18:53:10.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.253+0000 7f1dc37fe700 1 -- 172.21.15.146:0/2685126274 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc800caa0 con 0x7f1dd40caed0 2024-09-13T18:53:10.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.257+0000 7f1dc37fe700 1 -- 172.21.15.146:0/2685126274 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc800ce30 con 0x7f1dd40caed0 2024-09-13T18:53:10.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.257+0000 7f1dd9267700 1 -- 172.21.15.146:0/2685126274 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc4009570 con 0x7f1dd40caed0 2024-09-13T18:53:10.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.257+0000 7f1dd9267700 1 -- 172.21.15.146:0/2685126274 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd406cfb0 con 0x7f1dd40caed0 2024-09-13T18:53:10.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.265+0000 7f1dc37fe700 1 -- 172.21.15.146:0/2685126274 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1dc8013470 con 0x7f1dd40caed0 2024-09-13T18:53:10.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.269+0000 7f1dc17fa700 1 -- 172.21.15.146:0/2685126274 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd4146010 con 0x7f1dd40caed0 2024-09-13T18:53:10.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.269+0000 7f1dc37fe700 1 --2- 172.21.15.146:0/2685126274 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1dbc0669f0 0x7f1dbc068eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:10.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.269+0000 7f1dc37fe700 1 -- 172.21.15.146:0/2685126274 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f1dc8084b20 con 0x7f1dd40caed0 2024-09-13T18:53:10.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.269+0000 7f1dd27fc700 1 --2- 172.21.15.146:0/2685126274 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1dbc0669f0 0x7f1dbc068eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:10.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.277+0000 7f1dd27fc700 1 --2- 172.21.15.146:0/2685126274 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1dbc0669f0 0x7f1dbc068eb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0012e0 tx=0x7f1dcc001780 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:10.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:10.281+0000 7f1dc37fe700 1 -- 172.21.15.146:0/2685126274 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1dd4146010 con 0x7f1dd40caed0 2024-09-13T18:53:11.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.125+0000 7f1dc17fa700 1 -- 172.21.15.146:0/2685126274 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dd415e6d0 con 0x7f1dd40caed0 2024-09-13T18:53:11.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.125+0000 7f1dc37fe700 1 -- 172.21.15.146:0/2685126274 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+6282 (secure 0 0 0) 0x7f1dc8021090 con 0x7f1dd40caed0 2024-09-13T18:53:11.147 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:11.147 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":7098,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:52:57.765909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6984,"mds_1":6714},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:11.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.141+0000 7f1dd9267700 1 -- 172.21.15.146:0/2685126274 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1dbc0669f0 msgr2=0x7f1dbc068eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:11.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.141+0000 7f1dd9267700 1 --2- 172.21.15.146:0/2685126274 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1dbc0669f0 0x7f1dbc068eb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0012e0 tx=0x7f1dcc001780 comp rx=0 tx=0).stop 2024-09-13T18:53:11.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.141+0000 7f1dd9267700 1 -- 172.21.15.146:0/2685126274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 msgr2=0x7f1dd406c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:11.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.141+0000 7f1dd9267700 1 --2- 172.21.15.146:0/2685126274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 0x7f1dd406c680 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800ec10 tx=0x7f1dc8009ed0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.141+0000 7f1dd9267700 1 -- 172.21.15.146:0/2685126274 shutdown_connections 2024-09-13T18:53:11.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.141+0000 7f1dd9267700 1 --2- 172.21.15.146:0/2685126274 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f1dbc0669f0 0x7f1dbc068eb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.141+0000 7f1dd9267700 1 --2- 172.21.15.146:0/2685126274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dd415e170 0x7f1dd414d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.141+0000 7f1dd9267700 1 --2- 172.21.15.146:0/2685126274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd40cb2b0 0x7f1dd414cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.145+0000 7f1dd9267700 1 --2- 172.21.15.146:0/2685126274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dd40caed0 0x7f1dd406c680 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.145+0000 7f1dd9267700 1 -- 172.21.15.146:0/2685126274 >> 172.21.15.146:0/2685126274 conn(0x7f1dd4066090 msgr2=0x7f1dd406f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:11.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.149+0000 7f1dd9267700 1 -- 172.21.15.146:0/2685126274 shutdown_connections 2024-09-13T18:53:11.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.149+0000 7f1dd9267700 1 -- 172.21.15.146:0/2685126274 wait complete. 2024-09-13T18:53:11.172 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 103 2024-09-13T18:53:11.202 INFO:tasks.mds_thrash.fs.[cephfs]:thrashing max_mds: 2 -> 3 2024-09-13T18:53:11.202 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-13T18:53:11.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.594+0000 7f20aaa5a700 1 Processor -- start 2024-09-13T18:53:11.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.594+0000 7f20aaa5a700 1 -- start start 2024-09-13T18:53:11.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.598+0000 7f20aaa5a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f20a40c8f70 0x7f20a415a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:11.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.598+0000 7f20aaa5a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f20a40a9490 0x7f20a40c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:11.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.598+0000 7f20aaa5a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a40c8650 con 0x7f20a40c9350 2024-09-13T18:53:11.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.598+0000 7f20aaa5a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a40aa1a0 con 0x7f20a40c8f70 2024-09-13T18:53:11.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.598+0000 7f20aaa5a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a40aa480 con 0x7f20a40a9490 2024-09-13T18:53:11.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.598+0000 7f20a3fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f20a40c8f70 0x7f20a415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:11.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.598+0000 7f20a3fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f20a40c8f70 0x7f20a415a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55254/0 (socket says 172.21.15.146:55254) 2024-09-13T18:53:11.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.598+0000 7f20a3fff700 1 -- 172.21.15.146:0/1574570500 learned_addr learned my addr 172.21.15.146:0/1574570500 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:11.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.598+0000 7f20a3fff700 1 -- 172.21.15.146:0/1574570500 >> v2:172.21.15.154:3300/0 conn(0x7f20a40a9490 msgr2=0x7f20a40c76c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:53:11.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.598+0000 7f20a3fff700 1 --2- 172.21.15.146:0/1574570500 >> v2:172.21.15.154:3300/0 conn(0x7f20a40a9490 0x7f20a40c76c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.598+0000 7f20a3fff700 1 -- 172.21.15.146:0/1574570500 >> v1:172.21.15.146:6789/0 conn(0x7f20a40c9350 legacy=0x7f20a40cac60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:53:11.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.598+0000 7f20a3fff700 1 -- 172.21.15.146:0/1574570500 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a4159890 con 0x7f20a40c8f70 2024-09-13T18:53:11.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.602+0000 7f20a3fff700 1 --2- 172.21.15.146:0/1574570500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20a40c8f70 0x7f20a415a370 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f20a415dd90 tx=0x7f209c014800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5066fbb7f0bf52a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:11.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.622+0000 7f20a2ffd700 1 -- 172.21.15.146:0/1574570500 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c003c40 con 0x7f20a40c8f70 2024-09-13T18:53:11.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.622+0000 7f20a2ffd700 1 -- 172.21.15.146:0/1574570500 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c003da0 con 0x7f20a40c8f70 2024-09-13T18:53:11.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.622+0000 7f20aaa5a700 1 -- 172.21.15.146:0/1574570500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20a40c8f70 msgr2=0x7f20a415a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:11.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.622+0000 7f20aaa5a700 1 --2- 172.21.15.146:0/1574570500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20a40c8f70 0x7f20a415a370 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f20a415dd90 tx=0x7f209c014800 comp rx=0 tx=0).stop 2024-09-13T18:53:11.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.622+0000 7f20aaa5a700 1 -- 172.21.15.146:0/1574570500 shutdown_connections 2024-09-13T18:53:11.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.622+0000 7f20aaa5a700 1 --2- 172.21.15.146:0/1574570500 >> v2:172.21.15.154:3300/0 conn(0x7f20a40a9490 0x7f20a40c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.622+0000 7f20aaa5a700 1 --2- 172.21.15.146:0/1574570500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20a40c8f70 0x7f20a415a370 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.622+0000 7f20aaa5a700 1 -- 172.21.15.146:0/1574570500 >> 172.21.15.146:0/1574570500 conn(0x7f20a4066090 msgr2=0x7f20a4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:11.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.626+0000 7f20aaa5a700 1 -- 172.21.15.146:0/1574570500 shutdown_connections 2024-09-13T18:53:11.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.634+0000 7f20aaa5a700 1 -- 172.21.15.146:0/1574570500 wait complete. 2024-09-13T18:53:11.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.634+0000 7f20aaa5a700 1 Processor -- start 2024-09-13T18:53:11.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.634+0000 7f20aaa5a700 1 -- start start 2024-09-13T18:53:11.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20aaa5a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20a40a9490 0x7f20a40c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:11.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20aaa5a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20a40c9350 0x7f20a41fb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:11.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20aaa5a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20a41fbd70 0x7f20a40c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:11.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20aaa5a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a415a850 con 0x7f20a41fbd70 2024-09-13T18:53:11.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20aaa5a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a41ff130 con 0x7f20a40a9490 2024-09-13T18:53:11.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20aaa5a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a41ff620 con 0x7f20a40c9350 2024-09-13T18:53:11.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20a8ff7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20a41fbd70 0x7f20a40c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:11.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20a3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20a40a9490 0x7f20a40c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:11.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20a3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20a40a9490 0x7f20a40c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:34632/0 (socket says 172.21.15.146:34632) 2024-09-13T18:53:11.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20a3fff700 1 -- 172.21.15.146:0/2991058302 learned_addr learned my addr 172.21.15.146:0/2991058302 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:11.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20a3fff700 1 -- 172.21.15.146:0/2991058302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20a40c9350 msgr2=0x7f20a41fb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:11.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20a3fff700 1 --2- 172.21.15.146:0/2991058302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20a40c9350 0x7f20a41fb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20a3fff700 1 -- 172.21.15.146:0/2991058302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20a41fbd70 msgr2=0x7f20a40c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:11.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20a3fff700 1 --2- 172.21.15.146:0/2991058302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20a41fbd70 0x7f20a40c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20a3fff700 1 -- 172.21.15.146:0/2991058302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c014180 con 0x7f20a40a9490 2024-09-13T18:53:11.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.638+0000 7f20a3fff700 1 --2- 172.21.15.146:0/2991058302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20a40a9490 0x7f20a40c75c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f209c014150 tx=0x7f209c004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:11.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.654+0000 7f20a0ff9700 1 -- 172.21.15.146:0/2991058302 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c004440 con 0x7f20a40a9490 2024-09-13T18:53:11.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.654+0000 7f20aaa5a700 1 -- 172.21.15.146:0/2991058302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a415a5b0 con 0x7f20a40a9490 2024-09-13T18:53:11.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.654+0000 7f20aaa5a700 1 -- 172.21.15.146:0/2991058302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a40b0010 con 0x7f20a40a9490 2024-09-13T18:53:11.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.658+0000 7f20a0ff9700 1 -- 172.21.15.146:0/2991058302 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c0045a0 con 0x7f20a40a9490 2024-09-13T18:53:11.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.658+0000 7f20a0ff9700 1 -- 172.21.15.146:0/2991058302 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c004930 con 0x7f20a40a9490 2024-09-13T18:53:11.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.658+0000 7f20aaa5a700 1 -- 172.21.15.146:0/2991058302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2084005310 con 0x7f20a40a9490 2024-09-13T18:53:11.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.662+0000 7f20a0ff9700 1 -- 172.21.15.146:0/2991058302 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f209c0185d0 con 0x7f20a40a9490 2024-09-13T18:53:11.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.662+0000 7f20a0ff9700 1 --2- 172.21.15.146:0/2991058302 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2090066800 0x7f2090068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:11.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.666+0000 7f20a37fe700 1 --2- 172.21.15.146:0/2991058302 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2090066800 0x7f2090068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:11.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.666+0000 7f20a0ff9700 1 -- 172.21.15.146:0/2991058302 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f209c08ed10 con 0x7f20a40a9490 2024-09-13T18:53:11.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.666+0000 7f20a37fe700 1 --2- 172.21.15.146:0/2991058302 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2090066800 0x7f2090068cc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f20a408be50 tx=0x7f2098006c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:11.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:11.674+0000 7f20a0ff9700 1 -- 172.21.15.146:0/2991058302 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f209c05b910 con 0x7f20a40a9490 2024-09-13T18:53:12.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.434+0000 7f20aaa5a700 1 -- 172.21.15.146:0/2991058302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f2084005b40 con 0x7f20a40a9490 2024-09-13T18:53:12.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.822+0000 7f20a0ff9700 1 -- 172.21.15.146:0/2991058302 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v104) v1 ==== 105+0+0 (secure 0 0 0) 0x7f209c033020 con 0x7f20a40a9490 2024-09-13T18:53:12.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.826+0000 7f20aaa5a700 1 -- 172.21.15.146:0/2991058302 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2090066800 msgr2=0x7f2090068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.830+0000 7f20aaa5a700 1 --2- 172.21.15.146:0/2991058302 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2090066800 0x7f2090068cc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f20a408be50 tx=0x7f2098006c90 comp rx=0 tx=0).stop 2024-09-13T18:53:12.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.830+0000 7f20aaa5a700 1 -- 172.21.15.146:0/2991058302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20a40a9490 msgr2=0x7f20a40c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.830+0000 7f20aaa5a700 1 --2- 172.21.15.146:0/2991058302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20a40a9490 0x7f20a40c75c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f209c014150 tx=0x7f209c004af0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.830+0000 7f20aaa5a700 1 -- 172.21.15.146:0/2991058302 shutdown_connections 2024-09-13T18:53:12.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.830+0000 7f20aaa5a700 1 --2- 172.21.15.146:0/2991058302 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2090066800 0x7f2090068cc0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.830+0000 7f20aaa5a700 1 --2- 172.21.15.146:0/2991058302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20a41fbd70 0x7f20a40c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.830+0000 7f20aaa5a700 1 --2- 172.21.15.146:0/2991058302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20a40c9350 0x7f20a41fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.830+0000 7f20aaa5a700 1 --2- 172.21.15.146:0/2991058302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20a40a9490 0x7f20a40c75c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.830+0000 7f20aaa5a700 1 -- 172.21.15.146:0/2991058302 >> 172.21.15.146:0/2991058302 conn(0x7f20a4066090 msgr2=0x7f20a41583a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:12.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.830+0000 7f20aaa5a700 1 -- 172.21.15.146:0/2991058302 shutdown_connections 2024-09-13T18:53:12.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:12.830+0000 7f20aaa5a700 1 -- 172.21.15.146:0/2991058302 wait complete. 2024-09-13T18:53:12.881 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:53:12.882 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:13.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.178+0000 7f95b0851700 1 Processor -- start 2024-09-13T18:53:13.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.178+0000 7f95b0851700 1 -- start start 2024-09-13T18:53:13.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.178+0000 7f95b0851700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f95ac166a10 0x7f95ac163380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.178+0000 7f95b0851700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac0ca330 con 0x7f95ac0c12f0 2024-09-13T18:53:13.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.178+0000 7f95b0851700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac0ca610 con 0x7f95ac0c98d0 2024-09-13T18:53:13.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.178+0000 7f95b0851700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac0c9d20 con 0x7f95ac166a10 2024-09-13T18:53:13.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.178+0000 7f95abfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f95ac166a10 0x7f95ac163380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.178+0000 7f95aaffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f95ac0c98d0 0x7f95ac166300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:40796/0 (socket says 172.21.15.146:40796) 2024-09-13T18:53:13.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.178+0000 7f95aaffd700 1 -- 172.21.15.146:0/589005268 learned_addr learned my addr 172.21.15.146:0/589005268 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:53:13.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.178+0000 7f95ab7fe700 1 --1- 172.21.15.146:0/589005268 >> v1:172.21.15.146:6789/0 conn(0x7f95ac0c12f0 0x7f95ac0c16d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:51500/0 (socket says 172.21.15.146:51500) 2024-09-13T18:53:13.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.178+0000 7f95abfff700 1 -- 172.21.15.146:0/589005268 >> v1:172.21.15.154:6789/0 conn(0x7f95ac0c98d0 legacy=0x7f95ac166300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.178+0000 7f95abfff700 1 -- 172.21.15.146:0/589005268 >> v1:172.21.15.146:6789/0 conn(0x7f95ac0c12f0 legacy=0x7f95ac0c16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:13.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.182+0000 7f95abfff700 1 -- 172.21.15.146:0/589005268 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac16a570 con 0x7f95ac166a10 2024-09-13T18:53:13.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.182+0000 7f95abfff700 1 --2- 172.21.15.146:0/589005268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac166a10 0x7f95ac163380 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f95a0007a00 tx=0x7f95a0007220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56d014c99304315 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:13.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.186+0000 7f95aa7fc700 1 -- 172.21.15.146:0/589005268 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0004230 con 0x7f95ac166a10 2024-09-13T18:53:13.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.186+0000 7f95aa7fc700 1 -- 172.21.15.146:0/589005268 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a0004390 con 0x7f95ac166a10 2024-09-13T18:53:13.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.186+0000 7f95aa7fc700 1 -- 172.21.15.146:0/589005268 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a000a7c0 con 0x7f95ac166a10 2024-09-13T18:53:13.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.186+0000 7f95b0851700 1 -- 172.21.15.146:0/589005268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac166a10 msgr2=0x7f95ac163380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.186+0000 7f95b0851700 1 --2- 172.21.15.146:0/589005268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac166a10 0x7f95ac163380 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f95a0007a00 tx=0x7f95a0007220 comp rx=0 tx=0).stop 2024-09-13T18:53:13.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.190+0000 7f95b0851700 1 -- 172.21.15.146:0/589005268 shutdown_connections 2024-09-13T18:53:13.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.190+0000 7f95b0851700 1 --2- 172.21.15.146:0/589005268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac166a10 0x7f95ac163380 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.190+0000 7f95b0851700 1 -- 172.21.15.146:0/589005268 >> 172.21.15.146:0/589005268 conn(0x7f95ac0cac60 msgr2=0x7f95ac0cb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:13.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95b0851700 1 -- 172.21.15.146:0/589005268 shutdown_connections 2024-09-13T18:53:13.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95b0851700 1 -- 172.21.15.146:0/589005268 wait complete. 2024-09-13T18:53:13.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95b0851700 1 Processor -- start 2024-09-13T18:53:13.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95b0851700 1 -- start start 2024-09-13T18:53:13.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95b0851700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95ac0c12f0 0x7f95ac162200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95b0851700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac0c98d0 0x7f95ac06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95b0851700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95ac166a10 0x7f95ac06da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95b0851700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0c2ff0 con 0x7f95ac0c98d0 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95b0851700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0c29b0 con 0x7f95ac166a10 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95b0851700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0c46e0 con 0x7f95ac0c12f0 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95aaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac0c98d0 0x7f95ac06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95aaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac0c98d0 0x7f95ac06d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55276/0 (socket says 172.21.15.146:55276) 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95aaffd700 1 -- 172.21.15.146:0/1553140469 learned_addr learned my addr 172.21.15.146:0/1553140469 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95abfff700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95ac166a10 0x7f95ac06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95aaffd700 1 -- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95ac0c12f0 msgr2=0x7f95ac162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95aaffd700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95ac0c12f0 0x7f95ac162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95aaffd700 1 -- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95ac166a10 msgr2=0x7f95ac06da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95aaffd700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95ac166a10 0x7f95ac06da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.198+0000 7f95aaffd700 1 -- 172.21.15.146:0/1553140469 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a0007480 con 0x7f95ac0c98d0 2024-09-13T18:53:13.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.202+0000 7f95abfff700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95ac166a10 0x7f95ac06da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:13.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.202+0000 7f95aaffd700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac0c98d0 0x7f95ac06d540 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f95a40020e0 tx=0x7f95a400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:13.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.214+0000 7f959bfff700 1 -- 172.21.15.146:0/1553140469 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4001f10 con 0x7f95ac0c98d0 2024-09-13T18:53:13.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.222+0000 7f95b0851700 1 -- 172.21.15.146:0/1553140469 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac0c4950 con 0x7f95ac0c98d0 2024-09-13T18:53:13.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.222+0000 7f95b0851700 1 -- 172.21.15.146:0/1553140469 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac0c2c10 con 0x7f95ac0c98d0 2024-09-13T18:53:13.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.226+0000 7f959bfff700 1 -- 172.21.15.146:0/1553140469 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a400aba0 con 0x7f95ac0c98d0 2024-09-13T18:53:13.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.226+0000 7f959bfff700 1 -- 172.21.15.146:0/1553140469 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a40070d0 con 0x7f95ac0c98d0 2024-09-13T18:53:13.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.230+0000 7f959bfff700 1 -- 172.21.15.146:0/1553140469 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f95a4007230 con 0x7f95ac0c98d0 2024-09-13T18:53:13.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.230+0000 7f959bfff700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9590066800 0x7f9590068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.230+0000 7f959bfff700 1 -- 172.21.15.146:0/1553140469 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f95a4095730 con 0x7f95ac0c98d0 2024-09-13T18:53:13.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.230+0000 7f95b0851700 1 -- 172.21.15.146:0/1553140469 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9588005310 con 0x7f95ac0c98d0 2024-09-13T18:53:13.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.230+0000 7f95ab7fe700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9590066800 0x7f9590068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.234+0000 7f95ab7fe700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9590066800 0x7f9590068cc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f959c005070 tx=0x7f959c007a90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:13.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:13.242+0000 7f959bfff700 1 -- 172.21.15.146:0/1553140469 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f95a405ddb0 con 0x7f95ac0c98d0 2024-09-13T18:53:14.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.202+0000 7f95b0851700 1 -- 172.21.15.146:0/1553140469 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9588005b40 con 0x7f95ac0c98d0 2024-09-13T18:53:14.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.202+0000 7f959bfff700 1 -- 172.21.15.146:0/1553140469 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+6297 (secure 0 0 0) 0x7f95a4029070 con 0x7f95ac0c98d0 2024-09-13T18:53:14.222 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:14.222 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:12.828927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6714,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:starting","state_seq":1,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:14.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.214+0000 7f9599ffb700 1 -- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9590066800 msgr2=0x7f9590068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.214+0000 7f9599ffb700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9590066800 0x7f9590068cc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f959c005070 tx=0x7f959c007a90 comp rx=0 tx=0).stop 2024-09-13T18:53:14.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.214+0000 7f9599ffb700 1 -- 172.21.15.146:0/1553140469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac0c98d0 msgr2=0x7f95ac06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.214+0000 7f9599ffb700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac0c98d0 0x7f95ac06d540 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f95a40020e0 tx=0x7f95a400a340 comp rx=0 tx=0).stop 2024-09-13T18:53:14.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.214+0000 7f9599ffb700 1 -- 172.21.15.146:0/1553140469 shutdown_connections 2024-09-13T18:53:14.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.214+0000 7f9599ffb700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f9590066800 0x7f9590068cc0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.214+0000 7f9599ffb700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95ac166a10 0x7f95ac06da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.214+0000 7f9599ffb700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac0c98d0 0x7f95ac06d540 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.214+0000 7f9599ffb700 1 --2- 172.21.15.146:0/1553140469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95ac0c12f0 0x7f95ac162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.214+0000 7f9599ffb700 1 -- 172.21.15.146:0/1553140469 >> 172.21.15.146:0/1553140469 conn(0x7f95ac0cac60 msgr2=0x7f95ac0cb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:14.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.214+0000 7f9599ffb700 1 -- 172.21.15.146:0/1553140469 shutdown_connections 2024-09-13T18:53:14.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:14.214+0000 7f9599ffb700 1 -- 172.21.15.146:0/1553140469 wait complete. 2024-09-13T18:53:14.241 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 105 2024-09-13T18:53:14.261 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 105, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 105, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6714": { "addr": "172.21.15.146:6837/2169954140", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 2169954140, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 2169954140, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6714, "incarnation": 86, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 38 }, "gid_6984": { "addr": "172.21.15.146:6835/2934322065", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 2934322065, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 2934322065, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6984, "incarnation": 98, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 28 }, "gid_7098": { "addr": "172.21.15.146:6833/354253522", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 354253522, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 354253522, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7098, "incarnation": 105, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:starting", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 41, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:53:12.828927+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6984, "mds_1": 6714, "mds_2": 7098 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6839/2053734262", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 2053734262, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2053734262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 91, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6032, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:53:16.176 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:53:16.179 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:53:16.262 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:16.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.558+0000 7f802c954700 1 Processor -- start 2024-09-13T18:53:16.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.558+0000 7f802c954700 1 -- start start 2024-09-13T18:53:16.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.558+0000 7f802c954700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f802806ec50 0x7f802806fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:16.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.558+0000 7f802c954700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f802806f030 0x7f8028070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:16.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.558+0000 7f802c954700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8028166870 con 0x7f8028169ad0 2024-09-13T18:53:16.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.558+0000 7f802c954700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802806d180 con 0x7f802806f030 2024-09-13T18:53:16.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.558+0000 7f802c954700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8028166b30 con 0x7f802806ec50 2024-09-13T18:53:16.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.558+0000 7f8025d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f802806f030 0x7f8028070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:16.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.558+0000 7f802659c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f802806ec50 0x7f802806fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:16.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.558+0000 7f802659c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f802806ec50 0x7f802806fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41774/0 (socket says 172.21.15.146:41774) 2024-09-13T18:53:16.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.558+0000 7f8025d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f802806f030 0x7f8028070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:34656/0 (socket says 172.21.15.146:34656) 2024-09-13T18:53:16.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.558+0000 7f802659c700 1 -- 172.21.15.146:0/3084675044 learned_addr learned my addr 172.21.15.146:0/3084675044 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:16.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.602+0000 7f8025d9b700 1 -- 172.21.15.146:0/3084675044 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f802806ec50 msgr2=0x7f802806fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:16.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.602+0000 7f8025d9b700 1 --2- 172.21.15.146:0/3084675044 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f802806ec50 0x7f802806fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:16.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.602+0000 7f8025d9b700 1 -- 172.21.15.146:0/3084675044 >> v1:172.21.15.146:6789/0 conn(0x7f8028169ad0 legacy=0x7f80280c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:16.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.558+0000 7f8026d9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f8028169ad0 0x7f80280c6df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:51516/0 (socket says 172.21.15.146:51516) 2024-09-13T18:53:16.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.606+0000 7f8025d9b700 1 -- 172.21.15.146:0/3084675044 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8028166e10 con 0x7f802806f030 2024-09-13T18:53:16.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.606+0000 7f802659c700 1 --2- 172.21.15.146:0/3084675044 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f802806ec50 0x7f802806fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:16.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.606+0000 7f8025d9b700 1 --2- 172.21.15.146:0/3084675044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f802806f030 0x7f8028070300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f801800dc70 tx=0x7f801800df80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49315e2aeb768b1b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:16.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.606+0000 7f802559a700 1 -- 172.21.15.146:0/3084675044 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018012070 con 0x7f802806f030 2024-09-13T18:53:16.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.606+0000 7f802559a700 1 -- 172.21.15.146:0/3084675044 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801800d2e0 con 0x7f802806f030 2024-09-13T18:53:16.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.606+0000 7f802559a700 1 -- 172.21.15.146:0/3084675044 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801800f8e0 con 0x7f802806f030 2024-09-13T18:53:16.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.606+0000 7f802c954700 1 -- 172.21.15.146:0/3084675044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f802806f030 msgr2=0x7f8028070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:16.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.606+0000 7f802c954700 1 --2- 172.21.15.146:0/3084675044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f802806f030 0x7f8028070300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f801800dc70 tx=0x7f801800df80 comp rx=0 tx=0).stop 2024-09-13T18:53:16.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.606+0000 7f802c954700 1 -- 172.21.15.146:0/3084675044 shutdown_connections 2024-09-13T18:53:16.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.606+0000 7f802c954700 1 --2- 172.21.15.146:0/3084675044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f802806f030 0x7f8028070300 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:16.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.606+0000 7f802c954700 1 --2- 172.21.15.146:0/3084675044 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f802806ec50 0x7f802806fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:16.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.606+0000 7f802c954700 1 -- 172.21.15.146:0/3084675044 >> 172.21.15.146:0/3084675044 conn(0x7f80280cb1d0 msgr2=0x7f80280cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:16.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.606+0000 7f802c954700 1 -- 172.21.15.146:0/3084675044 shutdown_connections 2024-09-13T18:53:16.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.610+0000 7f802c954700 1 -- 172.21.15.146:0/3084675044 wait complete. 2024-09-13T18:53:16.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.614+0000 7f802c954700 1 Processor -- start 2024-09-13T18:53:16.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.618+0000 7f802c954700 1 -- start start 2024-09-13T18:53:16.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f802c954700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f802806ec50 0x7f80281f02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:16.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f802c954700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f802806f030 0x7f80280ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:16.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f802c954700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8028169ad0 0x7f8028161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:16.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f802c954700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80281fbf50 con 0x7f802806f030 2024-09-13T18:53:16.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f802c954700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80281ef8b0 con 0x7f802806ec50 2024-09-13T18:53:16.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f802c954700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802815d560 con 0x7f8028169ad0 2024-09-13T18:53:16.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f8025d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f802806f030 0x7f80280ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:16.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f8025d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f802806f030 0x7f80280ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55288/0 (socket says 172.21.15.146:55288) 2024-09-13T18:53:16.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f8025d9b700 1 -- 172.21.15.146:0/3052292456 learned_addr learned my addr 172.21.15.146:0/3052292456 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:16.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f802659c700 1 --2- 172.21.15.146:0/3052292456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f802806ec50 0x7f80281f02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:16.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f8025d9b700 1 -- 172.21.15.146:0/3052292456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8028169ad0 msgr2=0x7f8028161370 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:53:16.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f8025d9b700 1 --2- 172.21.15.146:0/3052292456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8028169ad0 0x7f8028161370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:16.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f8025d9b700 1 -- 172.21.15.146:0/3052292456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f802806ec50 msgr2=0x7f80281f02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:16.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f8025d9b700 1 --2- 172.21.15.146:0/3052292456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f802806ec50 0x7f80281f02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:16.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f8025d9b700 1 -- 172.21.15.146:0/3052292456 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802000aa90 con 0x7f802806f030 2024-09-13T18:53:16.642 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f8025d9b700 1 --2- 172.21.15.146:0/3052292456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f802806f030 0x7f80280ca320 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f80280675d0 tx=0x7f8018003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:16.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.622+0000 7f8016ffd700 1 -- 172.21.15.146:0/3052292456 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801800f8e0 con 0x7f802806f030 2024-09-13T18:53:16.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.634+0000 7f8016ffd700 1 -- 172.21.15.146:0/3052292456 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801800fa40 con 0x7f802806f030 2024-09-13T18:53:16.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.634+0000 7f8016ffd700 1 -- 172.21.15.146:0/3052292456 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801800fdf0 con 0x7f802806f030 2024-09-13T18:53:16.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.634+0000 7f802c954700 1 -- 172.21.15.146:0/3052292456 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801800d950 con 0x7f802806f030 2024-09-13T18:53:16.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.634+0000 7f802c954700 1 -- 172.21.15.146:0/3052292456 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80281629d0 con 0x7f802806f030 2024-09-13T18:53:16.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.638+0000 7f802c954700 1 -- 172.21.15.146:0/3052292456 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c005310 con 0x7f802806f030 2024-09-13T18:53:16.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.638+0000 7f8016ffd700 1 -- 172.21.15.146:0/3052292456 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f801800b450 con 0x7f802806f030 2024-09-13T18:53:16.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.642+0000 7f8016ffd700 1 --2- 172.21.15.146:0/3052292456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8008066800 0x7f8008068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:16.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.642+0000 7f8016ffd700 1 -- 172.21.15.146:0/3052292456 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f801808a2a0 con 0x7f802806f030 2024-09-13T18:53:16.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.642+0000 7f802659c700 1 --2- 172.21.15.146:0/3052292456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8008066800 0x7f8008068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:16.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.646+0000 7f8016ffd700 1 -- 172.21.15.146:0/3052292456 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8018056d80 con 0x7f802806f030 2024-09-13T18:53:16.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:16.650+0000 7f802659c700 1 --2- 172.21.15.146:0/3052292456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8008066800 0x7f8008068cc0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8020007290 tx=0x7f802000a040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:17.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.434+0000 7f802c954700 1 -- 172.21.15.146:0/3052292456 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f800c005b40 con 0x7f802806f030 2024-09-13T18:53:17.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.434+0000 7f8016ffd700 1 -- 172.21.15.146:0/3052292456 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+6297 (secure 0 0 0) 0x7f801805a3a0 con 0x7f802806f030 2024-09-13T18:53:17.454 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:17.454 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:12.828927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6714,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:starting","state_seq":1,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:17.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.438+0000 7f802c954700 1 -- 172.21.15.146:0/3052292456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8008066800 msgr2=0x7f8008068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:17.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.438+0000 7f802c954700 1 --2- 172.21.15.146:0/3052292456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8008066800 0x7f8008068cc0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8020007290 tx=0x7f802000a040 comp rx=0 tx=0).stop 2024-09-13T18:53:17.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.442+0000 7f802c954700 1 -- 172.21.15.146:0/3052292456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f802806f030 msgr2=0x7f80280ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:17.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.442+0000 7f802c954700 1 --2- 172.21.15.146:0/3052292456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f802806f030 0x7f80280ca320 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f80280675d0 tx=0x7f8018003bf0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.442+0000 7f802c954700 1 -- 172.21.15.146:0/3052292456 shutdown_connections 2024-09-13T18:53:17.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.442+0000 7f802c954700 1 --2- 172.21.15.146:0/3052292456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8008066800 0x7f8008068cc0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.442+0000 7f802c954700 1 --2- 172.21.15.146:0/3052292456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8028169ad0 0x7f8028161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.442+0000 7f802c954700 1 --2- 172.21.15.146:0/3052292456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f802806f030 0x7f80280ca320 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.442+0000 7f802c954700 1 --2- 172.21.15.146:0/3052292456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f802806ec50 0x7f80281f02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.442+0000 7f802c954700 1 -- 172.21.15.146:0/3052292456 >> 172.21.15.146:0/3052292456 conn(0x7f80280cb1d0 msgr2=0x7f80280c6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:17.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.442+0000 7f802c954700 1 -- 172.21.15.146:0/3052292456 shutdown_connections 2024-09-13T18:53:17.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:17.442+0000 7f802c954700 1 -- 172.21.15.146:0/3052292456 wait complete. 2024-09-13T18:53:17.470 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 105 2024-09-13T18:53:17.483 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:53:19.484 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:19.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.934+0000 7f0d31fdd700 1 Processor -- start 2024-09-13T18:53:19.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.934+0000 7f0d31fdd700 1 -- start start 2024-09-13T18:53:19.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.934+0000 7f0d31fdd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0d2c0cac60 0x7f0d2c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:19.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.934+0000 7f0d31fdd700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2c0a9920 con 0x7f0d2c08baa0 2024-09-13T18:53:19.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.934+0000 7f0d31fdd700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2c070070 con 0x7f0d2c16e0c0 2024-09-13T18:53:19.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.934+0000 7f0d31fdd700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d2c0723f0 con 0x7f0d2c0cac60 2024-09-13T18:53:19.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d30fdb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0d2c0cac60 0x7f0d2c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:19.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d2bfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0d2c08baa0 0x7f0d2c06f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:51526/0 (socket says 172.21.15.146:51526) 2024-09-13T18:53:19.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d2bfff700 1 -- 172.21.15.146:0/1858828424 learned_addr learned my addr 172.21.15.146:0/1858828424 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:53:19.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d2b7fe700 1 -- 172.21.15.146:0/1858828424 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076795057 0 0) 0x7f0d2c070070 con 0x7f0d2c16e0c0 2024-09-13T18:53:19.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d2b7fe700 1 -- 172.21.15.146:0/1858828424 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d140079e0 con 0x7f0d2c16e0c0 2024-09-13T18:53:19.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d2b7fe700 1 -- 172.21.15.146:0/1858828424 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545236383 0 0) 0x7f0d2c0a9920 con 0x7f0d2c08baa0 2024-09-13T18:53:19.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d2b7fe700 1 -- 172.21.15.146:0/1858828424 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d2c070070 con 0x7f0d2c08baa0 2024-09-13T18:53:19.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d30fdb700 1 -- 172.21.15.146:0/1858828424 >> v1:172.21.15.154:6789/0 conn(0x7f0d2c16e0c0 legacy=0x7f0d2c1704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:19.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d30fdb700 1 -- 172.21.15.146:0/1858828424 >> v1:172.21.15.146:6789/0 conn(0x7f0d2c08baa0 legacy=0x7f0d2c06f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:19.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d30fdb700 1 -- 172.21.15.146:0/1858828424 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c0725e0 con 0x7f0d2c0cac60 2024-09-13T18:53:19.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d30fdb700 1 --2- 172.21.15.146:0/1858828424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d2c0cac60 0x7f0d2c0cb040 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009910 tx=0x7f0d1c0070f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80b23430646d80d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:19.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d2b7fe700 1 -- 172.21.15.146:0/1858828424 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c014c30 con 0x7f0d2c0cac60 2024-09-13T18:53:19.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d2b7fe700 1 -- 172.21.15.146:0/1858828424 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c014d90 con 0x7f0d2c0cac60 2024-09-13T18:53:19.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d31fdd700 1 -- 172.21.15.146:0/1858828424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d2c0cac60 msgr2=0x7f0d2c0cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:19.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d31fdd700 1 --2- 172.21.15.146:0/1858828424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d2c0cac60 0x7f0d2c0cb040 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009910 tx=0x7f0d1c0070f0 comp rx=0 tx=0).stop 2024-09-13T18:53:19.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d31fdd700 1 -- 172.21.15.146:0/1858828424 shutdown_connections 2024-09-13T18:53:19.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d31fdd700 1 --2- 172.21.15.146:0/1858828424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d2c0cac60 0x7f0d2c0cb040 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:19.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.938+0000 7f0d31fdd700 1 -- 172.21.15.146:0/1858828424 >> 172.21.15.146:0/1858828424 conn(0x7f0d2c066090 msgr2=0x7f0d2c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:19.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.950+0000 7f0d31fdd700 1 -- 172.21.15.146:0/1858828424 shutdown_connections 2024-09-13T18:53:19.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.954+0000 7f0d31fdd700 1 -- 172.21.15.146:0/1858828424 wait complete. 2024-09-13T18:53:19.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.958+0000 7f0d31fdd700 1 Processor -- start 2024-09-13T18:53:19.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.958+0000 7f0d31fdd700 1 -- start start 2024-09-13T18:53:19.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d31fdd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d2c08baa0 0x7f0d2c16ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:19.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d31fdd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d2c16e0c0 0x7f0d2c0aa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:19.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d31fdd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d2c0af6e0 0x7f0d2c0aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:19.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d31fdd700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c161b50 con 0x7f0d2c08baa0 2024-09-13T18:53:19.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d31fdd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0aaed0 con 0x7f0d2c0af6e0 2024-09-13T18:53:19.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d31fdd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0b0120 con 0x7f0d2c16e0c0 2024-09-13T18:53:19.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d30fdb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d2c08baa0 0x7f0d2c16ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:19.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d30fdb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d2c08baa0 0x7f0d2c16ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55304/0 (socket says 172.21.15.146:55304) 2024-09-13T18:53:19.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d30fdb700 1 -- 172.21.15.146:0/953008638 learned_addr learned my addr 172.21.15.146:0/953008638 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:19.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d30fdb700 1 -- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d2c16e0c0 msgr2=0x7f0d2c0aa990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:53:19.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d30fdb700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d2c16e0c0 0x7f0d2c0aa990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:19.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d30fdb700 1 -- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d2c0af6e0 msgr2=0x7f0d2c0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:19.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d317dc700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d2c0af6e0 0x7f0d2c0aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:19.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d30fdb700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d2c0af6e0 0x7f0d2c0aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:19.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d30fdb700 1 -- 172.21.15.146:0/953008638 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0095e0 con 0x7f0d2c08baa0 2024-09-13T18:53:19.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d317dc700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d2c0af6e0 0x7f0d2c0aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:53:19.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d30fdb700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d2c08baa0 0x7f0d2c16ce60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c007130 tx=0x7f0d1c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:19.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d297fa700 1 -- 172.21.15.146:0/953008638 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c0143f0 con 0x7f0d2c08baa0 2024-09-13T18:53:19.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d297fa700 1 -- 172.21.15.146:0/953008638 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c014550 con 0x7f0d2c08baa0 2024-09-13T18:53:19.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d297fa700 1 -- 172.21.15.146:0/953008638 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c01d470 con 0x7f0d2c08baa0 2024-09-13T18:53:19.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d31fdd700 1 -- 172.21.15.146:0/953008638 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2c15b7a0 con 0x7f0d2c08baa0 2024-09-13T18:53:19.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.962+0000 7f0d31fdd700 1 -- 172.21.15.146:0/953008638 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2c0afe40 con 0x7f0d2c08baa0 2024-09-13T18:53:19.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.966+0000 7f0d12ffd700 1 -- 172.21.15.146:0/953008638 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2c0c9da0 con 0x7f0d2c08baa0 2024-09-13T18:53:19.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.966+0000 7f0d297fa700 1 -- 172.21.15.146:0/953008638 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0d1c01a070 con 0x7f0d2c08baa0 2024-09-13T18:53:19.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.970+0000 7f0d297fa700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0d18066800 0x7f0d18068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:19.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.970+0000 7f0d297fa700 1 -- 172.21.15.146:0/953008638 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f0d1c085c50 con 0x7f0d2c08baa0 2024-09-13T18:53:19.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.970+0000 7f0d2bfff700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0d18066800 0x7f0d18068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:19.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.970+0000 7f0d2bfff700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0d18066800 0x7f0d18068cc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0d24001560 tx=0x7f0d240016d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:20.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:19.986+0000 7f0d297fa700 1 -- 172.21.15.146:0/953008638 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0d1c052b70 con 0x7f0d2c08baa0 2024-09-13T18:53:21.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.074+0000 7f0d12ffd700 1 -- 172.21.15.146:0/953008638 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d2c0aa2b0 con 0x7f0d2c08baa0 2024-09-13T18:53:21.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.074+0000 7f0d297fa700 1 -- 172.21.15.146:0/953008638 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+6297 (secure 0 0 0) 0x7f0d1c056190 con 0x7f0d2c08baa0 2024-09-13T18:53:21.093 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:21.093 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:12.828927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6714,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:starting","state_seq":1,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:21.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.082+0000 7f0d31fdd700 1 -- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0d18066800 msgr2=0x7f0d18068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.082+0000 7f0d31fdd700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0d18066800 0x7f0d18068cc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0d24001560 tx=0x7f0d240016d0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.082+0000 7f0d31fdd700 1 -- 172.21.15.146:0/953008638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d2c08baa0 msgr2=0x7f0d2c16ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.082+0000 7f0d31fdd700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d2c08baa0 0x7f0d2c16ce60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c007130 tx=0x7f0d1c004770 comp rx=0 tx=0).stop 2024-09-13T18:53:21.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.082+0000 7f0d31fdd700 1 -- 172.21.15.146:0/953008638 shutdown_connections 2024-09-13T18:53:21.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.082+0000 7f0d31fdd700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0d18066800 0x7f0d18068cc0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.082+0000 7f0d31fdd700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d2c0af6e0 0x7f0d2c0aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.082+0000 7f0d31fdd700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d2c16e0c0 0x7f0d2c0aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.082+0000 7f0d31fdd700 1 --2- 172.21.15.146:0/953008638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d2c08baa0 0x7f0d2c16ce60 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.082+0000 7f0d31fdd700 1 -- 172.21.15.146:0/953008638 >> 172.21.15.146:0/953008638 conn(0x7f0d2c066090 msgr2=0x7f0d2c06c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:21.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.082+0000 7f0d31fdd700 1 -- 172.21.15.146:0/953008638 shutdown_connections 2024-09-13T18:53:21.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:21.082+0000 7f0d31fdd700 1 -- 172.21.15.146:0/953008638 wait complete. 2024-09-13T18:53:21.105 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 105 2024-09-13T18:53:21.137 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:53:23.138 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:23.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.522+0000 7f59f991e700 1 Processor -- start 2024-09-13T18:53:23.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.522+0000 7f59f991e700 1 -- start start 2024-09-13T18:53:23.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.522+0000 7f59f991e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f59f415e170 0x7f59f415e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.522+0000 7f59f991e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f4161060 con 0x7f59f40cb2b0 2024-09-13T18:53:23.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.522+0000 7f59f991e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f4161300 con 0x7f59f40caed0 2024-09-13T18:53:23.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.522+0000 7f59f991e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f40c78d0 con 0x7f59f415e170 2024-09-13T18:53:23.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.526+0000 7f59f27fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f59f40cb2b0 0x7f59f40c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:53806/0 (socket says 172.21.15.146:53806) 2024-09-13T18:53:23.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.526+0000 7f59f27fc700 1 -- 172.21.15.146:0/159436263 learned_addr learned my addr 172.21.15.146:0/159436263 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:53:23.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.526+0000 7f59f37fe700 1 --2- 172.21.15.146:0/159436263 >> v2:172.21.15.154:3300/0 conn(0x7f59f415e170 0x7f59f415e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.526+0000 7f59f1ffb700 1 -- 172.21.15.146:0/159436263 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048275819 0 0) 0x7f59f4161060 con 0x7f59f40cb2b0 2024-09-13T18:53:23.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.526+0000 7f59f37fe700 1 -- 172.21.15.146:0/159436263 >> v1:172.21.15.154:6789/0 conn(0x7f59f40caed0 legacy=0x7f59f4072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.526+0000 7f59f37fe700 1 -- 172.21.15.146:0/159436263 >> v1:172.21.15.146:6789/0 conn(0x7f59f40cb2b0 legacy=0x7f59f40c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.526+0000 7f59f37fe700 1 -- 172.21.15.146:0/159436263 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f415eb20 con 0x7f59f415e170 2024-09-13T18:53:23.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.526+0000 7f59f37fe700 1 --2- 172.21.15.146:0/159436263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f415e170 0x7f59f415e5e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009880 tx=0x7f59e4009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18a3b2bee3c9c323 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:23.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.526+0000 7f59f1ffb700 1 -- 172.21.15.146:0/159436263 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e4011830 con 0x7f59f415e170 2024-09-13T18:53:23.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.526+0000 7f59f1ffb700 1 -- 172.21.15.146:0/159436263 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e4011990 con 0x7f59f415e170 2024-09-13T18:53:23.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 -- 172.21.15.146:0/159436263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f415e170 msgr2=0x7f59f415e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 --2- 172.21.15.146:0/159436263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f415e170 0x7f59f415e5e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009880 tx=0x7f59e4009b90 comp rx=0 tx=0).stop 2024-09-13T18:53:23.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 -- 172.21.15.146:0/159436263 shutdown_connections 2024-09-13T18:53:23.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 --2- 172.21.15.146:0/159436263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f415e170 0x7f59f415e5e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 -- 172.21.15.146:0/159436263 >> 172.21.15.146:0/159436263 conn(0x7f59f4066090 msgr2=0x7f59f4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:23.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 -- 172.21.15.146:0/159436263 shutdown_connections 2024-09-13T18:53:23.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 -- 172.21.15.146:0/159436263 wait complete. 2024-09-13T18:53:23.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 Processor -- start 2024-09-13T18:53:23.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 -- start start 2024-09-13T18:53:23.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59f40caed0 0x7f59f415ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f40cb2b0 0x7f59f40aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59f40aae30 0x7f59f413f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f4143250 con 0x7f59f40aae30 2024-09-13T18:53:23.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f41467e0 con 0x7f59f40cb2b0 2024-09-13T18:53:23.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f991e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f4143760 con 0x7f59f40caed0 2024-09-13T18:53:23.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59f40aae30 0x7f59f413f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59f40aae30 0x7f59f413f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45520/0 (socket says 172.21.15.146:45520) 2024-09-13T18:53:23.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f37fe700 1 -- 172.21.15.146:0/2401062244 learned_addr learned my addr 172.21.15.146:0/2401062244 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:23.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f2ffd700 1 --2- 172.21.15.146:0/2401062244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59f40caed0 0x7f59f415ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f37fe700 1 -- 172.21.15.146:0/2401062244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59f40caed0 msgr2=0x7f59f415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f37fe700 1 --2- 172.21.15.146:0/2401062244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59f40caed0 0x7f59f415ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f37fe700 1 -- 172.21.15.146:0/2401062244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f40cb2b0 msgr2=0x7f59f40aa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f37fe700 1 --2- 172.21.15.146:0/2401062244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f40cb2b0 0x7f59f40aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.530+0000 7f59f37fe700 1 -- 172.21.15.146:0/2401062244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e40094b0 con 0x7f59f40aae30 2024-09-13T18:53:23.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.534+0000 7f59f37fe700 1 --2- 172.21.15.146:0/2401062244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59f40aae30 0x7f59f413f530 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f59f40cac30 tx=0x7f59e4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:23.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.534+0000 7f59dbfff700 1 -- 172.21.15.146:0/2401062244 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e401aaf0 con 0x7f59f40aae30 2024-09-13T18:53:23.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.534+0000 7f59f991e700 1 -- 172.21.15.146:0/2401062244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f4145ef0 con 0x7f59f40aae30 2024-09-13T18:53:23.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.534+0000 7f59f991e700 1 -- 172.21.15.146:0/2401062244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f4143480 con 0x7f59f40aae30 2024-09-13T18:53:23.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.534+0000 7f59f991e700 1 -- 172.21.15.146:0/2401062244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f4146950 con 0x7f59f40aae30 2024-09-13T18:53:23.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.542+0000 7f59dbfff700 1 -- 172.21.15.146:0/2401062244 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e401ac50 con 0x7f59f40aae30 2024-09-13T18:53:23.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.542+0000 7f59dbfff700 1 -- 172.21.15.146:0/2401062244 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e4024420 con 0x7f59f40aae30 2024-09-13T18:53:23.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.542+0000 7f59dbfff700 1 -- 172.21.15.146:0/2401062244 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59e4022070 con 0x7f59f40aae30 2024-09-13T18:53:23.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.542+0000 7f59dbfff700 1 --2- 172.21.15.146:0/2401062244 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f59e0066880 0x7f59e0068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.542+0000 7f59dbfff700 1 -- 172.21.15.146:0/2401062244 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f59e4085b40 con 0x7f59f40aae30 2024-09-13T18:53:23.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.542+0000 7f59f2ffd700 1 --2- 172.21.15.146:0/2401062244 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f59e0066880 0x7f59e0068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.542+0000 7f59f2ffd700 1 --2- 172.21.15.146:0/2401062244 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f59e0066880 0x7f59e0068d40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f59e8007fd0 tx=0x7f59e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:23.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:23.554+0000 7f59dbfff700 1 -- 172.21.15.146:0/2401062244 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f59e40526b0 con 0x7f59f40aae30 2024-09-13T18:53:24.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.338+0000 7f59f991e700 1 -- 172.21.15.146:0/2401062244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59f40c74f0 con 0x7f59f40aae30 2024-09-13T18:53:24.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.338+0000 7f59dbfff700 1 -- 172.21.15.146:0/2401062244 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+6297 (secure 0 0 0) 0x7f59e4029090 con 0x7f59f40aae30 2024-09-13T18:53:24.358 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:24.358 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:12.828927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6714,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:starting","state_seq":1,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:24.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.342+0000 7f59f991e700 1 -- 172.21.15.146:0/2401062244 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f59e0066880 msgr2=0x7f59e0068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:24.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.342+0000 7f59f991e700 1 --2- 172.21.15.146:0/2401062244 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f59e0066880 0x7f59e0068d40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f59e8007fd0 tx=0x7f59e8009040 comp rx=0 tx=0).stop 2024-09-13T18:53:24.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.342+0000 7f59f991e700 1 -- 172.21.15.146:0/2401062244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59f40aae30 msgr2=0x7f59f413f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:24.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.342+0000 7f59f991e700 1 --2- 172.21.15.146:0/2401062244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59f40aae30 0x7f59f413f530 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f59f40cac30 tx=0x7f59e4004770 comp rx=0 tx=0).stop 2024-09-13T18:53:24.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.342+0000 7f59f991e700 1 -- 172.21.15.146:0/2401062244 shutdown_connections 2024-09-13T18:53:24.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.342+0000 7f59f991e700 1 --2- 172.21.15.146:0/2401062244 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f59e0066880 0x7f59e0068d40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:24.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.342+0000 7f59f991e700 1 --2- 172.21.15.146:0/2401062244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59f40aae30 0x7f59f413f530 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:24.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.342+0000 7f59f991e700 1 --2- 172.21.15.146:0/2401062244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f40cb2b0 0x7f59f40aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:24.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.342+0000 7f59f991e700 1 --2- 172.21.15.146:0/2401062244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59f40caed0 0x7f59f415ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:24.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.342+0000 7f59f991e700 1 -- 172.21.15.146:0/2401062244 >> 172.21.15.146:0/2401062244 conn(0x7f59f4066090 msgr2=0x7f59f414a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:24.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.342+0000 7f59f991e700 1 -- 172.21.15.146:0/2401062244 shutdown_connections 2024-09-13T18:53:24.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:24.342+0000 7f59f991e700 1 -- 172.21.15.146:0/2401062244 wait complete. 2024-09-13T18:53:24.370 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 105 2024-09-13T18:53:24.412 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:53:26.413 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:26.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.830+0000 7f6b26002700 1 Processor -- start 2024-09-13T18:53:26.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.830+0000 7f6b26002700 1 -- start start 2024-09-13T18:53:26.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.834+0000 7f6b26002700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b2014f630 0x7f6b200c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:26.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.834+0000 7f6b26002700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6b200cb120 0x7f6b200cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:26.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.834+0000 7f6b26002700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2006f5c0 con 0x7f6b200a9490 2024-09-13T18:53:26.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.834+0000 7f6b26002700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b20155420 con 0x7f6b200cb120 2024-09-13T18:53:26.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.834+0000 7f6b26002700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b200c7b70 con 0x7f6b2014f630 2024-09-13T18:53:26.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.838+0000 7f6b25000700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b2014f630 0x7f6b200c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:26.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.838+0000 7f6b25801700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6b200cb120 0x7f6b200cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:26.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.838+0000 7f6b25801700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6b200cb120 0x7f6b200cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45530/0 (socket says 172.21.15.146:45530) 2024-09-13T18:53:26.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.838+0000 7f6b25801700 1 -- 172.21.15.146:0/2583243054 learned_addr learned my addr 172.21.15.146:0/2583243054 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:26.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.838+0000 7f6b25801700 1 -- 172.21.15.146:0/2583243054 >> v2:172.21.15.154:3300/0 conn(0x7f6b2014f630 msgr2=0x7f6b200c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:26.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.838+0000 7f6b25801700 1 --2- 172.21.15.146:0/2583243054 >> v2:172.21.15.154:3300/0 conn(0x7f6b2014f630 0x7f6b200c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:26.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.838+0000 7f6b25801700 1 -- 172.21.15.146:0/2583243054 >> v1:172.21.15.154:6789/0 conn(0x7f6b200a9490 legacy=0x7f6b20155c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:53:26.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.838+0000 7f6b25801700 1 -- 172.21.15.146:0/2583243054 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b200c7e50 con 0x7f6b200cb120 2024-09-13T18:53:26.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.842+0000 7f6b25801700 1 --2- 172.21.15.146:0/2583243054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b200cb120 0x7f6b200cb5b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6b200c8810 tx=0x7f6b1000d690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85e4a620b55773a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:26.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.842+0000 7f6b1f7fe700 1 -- 172.21.15.146:0/2583243054 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b10010070 con 0x7f6b200cb120 2024-09-13T18:53:26.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.842+0000 7f6b1f7fe700 1 -- 172.21.15.146:0/2583243054 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1000dc30 con 0x7f6b200cb120 2024-09-13T18:53:26.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.842+0000 7f6b1f7fe700 1 -- 172.21.15.146:0/2583243054 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b10004030 con 0x7f6b200cb120 2024-09-13T18:53:26.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.842+0000 7f6b26002700 1 -- 172.21.15.146:0/2583243054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b200cb120 msgr2=0x7f6b200cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:26.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.842+0000 7f6b26002700 1 --2- 172.21.15.146:0/2583243054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b200cb120 0x7f6b200cb5b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6b200c8810 tx=0x7f6b1000d690 comp rx=0 tx=0).stop 2024-09-13T18:53:26.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.862+0000 7f6b26002700 1 -- 172.21.15.146:0/2583243054 shutdown_connections 2024-09-13T18:53:26.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.862+0000 7f6b26002700 1 --2- 172.21.15.146:0/2583243054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b200cb120 0x7f6b200cb5b0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:26.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.862+0000 7f6b26002700 1 --2- 172.21.15.146:0/2583243054 >> v2:172.21.15.154:3300/0 conn(0x7f6b2014f630 0x7f6b200c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:26.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.862+0000 7f6b26002700 1 -- 172.21.15.146:0/2583243054 >> 172.21.15.146:0/2583243054 conn(0x7f6b20066090 msgr2=0x7f6b20066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:26.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.862+0000 7f6b26002700 1 -- 172.21.15.146:0/2583243054 shutdown_connections 2024-09-13T18:53:26.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.862+0000 7f6b26002700 1 -- 172.21.15.146:0/2583243054 wait complete. 2024-09-13T18:53:26.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.862+0000 7f6b26002700 1 Processor -- start 2024-09-13T18:53:26.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.862+0000 7f6b26002700 1 -- start start 2024-09-13T18:53:26.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.862+0000 7f6b26002700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b200a9490 0x7f6b2006e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:26.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b26002700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b200cb120 0x7f6b200accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:26.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b26002700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2014f630 0x7f6b20157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b26002700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2006e880 con 0x7f6b200a9490 2024-09-13T18:53:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b26002700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b201566b0 con 0x7f6b200cb120 2024-09-13T18:53:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b26002700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b20156950 con 0x7f6b2014f630 2024-09-13T18:53:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b25000700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b200a9490 0x7f6b2006e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b25000700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b200a9490 0x7f6b2006e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45546/0 (socket says 172.21.15.146:45546) 2024-09-13T18:53:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b25000700 1 -- 172.21.15.146:0/3419050993 learned_addr learned my addr 172.21.15.146:0/3419050993 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b1ffff700 1 --2- 172.21.15.146:0/3419050993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b200cb120 0x7f6b200accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b25000700 1 -- 172.21.15.146:0/3419050993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2014f630 msgr2=0x7f6b20157e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:53:26.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b25000700 1 --2- 172.21.15.146:0/3419050993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2014f630 0x7f6b20157e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:26.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b25000700 1 -- 172.21.15.146:0/3419050993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b200cb120 msgr2=0x7f6b200accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:26.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b25000700 1 --2- 172.21.15.146:0/3419050993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b200cb120 0x7f6b200accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:26.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b25000700 1 -- 172.21.15.146:0/3419050993 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b100085f0 con 0x7f6b200a9490 2024-09-13T18:53:26.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.866+0000 7f6b25000700 1 --2- 172.21.15.146:0/3419050993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b200a9490 0x7f6b2006e1f0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009ec0 tx=0x7f6b1400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:26.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.878+0000 7f6b1d7fa700 1 -- 172.21.15.146:0/3419050993 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400c700 con 0x7f6b200a9490 2024-09-13T18:53:26.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.882+0000 7f6b26002700 1 -- 172.21.15.146:0/3419050993 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b20156c30 con 0x7f6b200a9490 2024-09-13T18:53:26.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.882+0000 7f6b26002700 1 -- 172.21.15.146:0/3419050993 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b201571f0 con 0x7f6b200a9490 2024-09-13T18:53:26.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.882+0000 7f6b1d7fa700 1 -- 172.21.15.146:0/3419050993 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1400c860 con 0x7f6b200a9490 2024-09-13T18:53:26.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.882+0000 7f6b1d7fa700 1 -- 172.21.15.146:0/3419050993 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400cbd0 con 0x7f6b200a9490 2024-09-13T18:53:26.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.886+0000 7f6b1d7fa700 1 -- 172.21.15.146:0/3419050993 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b1400cd30 con 0x7f6b200a9490 2024-09-13T18:53:26.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.886+0000 7f6b1d7fa700 1 --2- 172.21.15.146:0/3419050993 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6b04066830 0x7f6b04068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:26.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.886+0000 7f6b1d7fa700 1 -- 172.21.15.146:0/3419050993 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f6b140a6870 con 0x7f6b200a9490 2024-09-13T18:53:26.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.890+0000 7f6b1ffff700 1 --2- 172.21.15.146:0/3419050993 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6b04066830 0x7f6b04068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:26.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.894+0000 7f6b0effd700 1 -- 172.21.15.146:0/3419050993 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6afc005310 con 0x7f6b200a9490 2024-09-13T18:53:26.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.894+0000 7f6b1ffff700 1 --2- 172.21.15.146:0/3419050993 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6b04066830 0x7f6b04068cf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800b450 tx=0x7f6b1800d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:26.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:26.906+0000 7f6b1d7fa700 1 -- 172.21.15.146:0/3419050993 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6b14073350 con 0x7f6b200a9490 2024-09-13T18:53:27.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.822+0000 7f6b0effd700 1 -- 172.21.15.146:0/3419050993 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6afc005b40 con 0x7f6b200a9490 2024-09-13T18:53:27.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.826+0000 7f6b1d7fa700 1 -- 172.21.15.146:0/3419050993 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+6297 (secure 0 0 0) 0x7f6b14043070 con 0x7f6b200a9490 2024-09-13T18:53:27.842 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:27.843 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:12.828927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6714,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:starting","state_seq":1,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:27.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.826+0000 7f6b0effd700 1 -- 172.21.15.146:0/3419050993 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6b04066830 msgr2=0x7f6b04068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:27.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.826+0000 7f6b0effd700 1 --2- 172.21.15.146:0/3419050993 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6b04066830 0x7f6b04068cf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800b450 tx=0x7f6b1800d040 comp rx=0 tx=0).stop 2024-09-13T18:53:27.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.826+0000 7f6b0effd700 1 -- 172.21.15.146:0/3419050993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b200a9490 msgr2=0x7f6b2006e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:27.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.826+0000 7f6b0effd700 1 --2- 172.21.15.146:0/3419050993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b200a9490 0x7f6b2006e1f0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009ec0 tx=0x7f6b1400eec0 comp rx=0 tx=0).stop 2024-09-13T18:53:27.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.826+0000 7f6b0effd700 1 -- 172.21.15.146:0/3419050993 shutdown_connections 2024-09-13T18:53:27.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.826+0000 7f6b0effd700 1 --2- 172.21.15.146:0/3419050993 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6b04066830 0x7f6b04068cf0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:27.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.826+0000 7f6b0effd700 1 --2- 172.21.15.146:0/3419050993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2014f630 0x7f6b20157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:27.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.830+0000 7f6b0effd700 1 --2- 172.21.15.146:0/3419050993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b200cb120 0x7f6b200accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:27.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.830+0000 7f6b0effd700 1 --2- 172.21.15.146:0/3419050993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b200a9490 0x7f6b2006e1f0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:27.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.830+0000 7f6b0effd700 1 -- 172.21.15.146:0/3419050993 >> 172.21.15.146:0/3419050993 conn(0x7f6b20066090 msgr2=0x7f6b2006f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:27.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.830+0000 7f6b0effd700 1 -- 172.21.15.146:0/3419050993 shutdown_connections 2024-09-13T18:53:27.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:27.830+0000 7f6b0effd700 1 -- 172.21.15.146:0/3419050993 wait complete. 2024-09-13T18:53:27.851 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 105 2024-09-13T18:53:27.871 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:53:29.872 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:30.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e17d2c700 1 Processor -- start 2024-09-13T18:53:30.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e17d2c700 1 -- start start 2024-09-13T18:53:30.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e17d2c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7e10153bc0 0x7f7e100c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:30.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e17d2c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e100c6a80 0x7f7e10160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:30.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e17d2c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e10153fa0 con 0x7f7e100c6610 2024-09-13T18:53:30.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e17d2c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e100aa500 con 0x7f7e10153bc0 2024-09-13T18:53:30.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e17d2c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1015a7e0 con 0x7f7e100c6a80 2024-09-13T18:53:30.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e162c9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e100c6a80 0x7f7e10160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:30.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e162c9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e100c6a80 0x7f7e10160c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:35380/0 (socket says 172.21.15.146:35380) 2024-09-13T18:53:30.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e162c9700 1 -- 172.21.15.146:0/3941317725 learned_addr learned my addr 172.21.15.146:0/3941317725 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:30.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e162c9700 1 -- 172.21.15.146:0/3941317725 >> v2:172.21.15.154:3300/0 conn(0x7f7e10153bc0 msgr2=0x7f7e100c8de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:53:30.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e162c9700 1 --2- 172.21.15.146:0/3941317725 >> v2:172.21.15.154:3300/0 conn(0x7f7e10153bc0 0x7f7e100c8de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:30.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e162c9700 1 -- 172.21.15.146:0/3941317725 >> v1:172.21.15.146:6789/0 conn(0x7f7e100c6610 legacy=0x7f7e100cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:30.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e162c9700 1 -- 172.21.15.146:0/3941317725 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e100c9320 con 0x7f7e100c6a80 2024-09-13T18:53:30.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e162c9700 1 --2- 172.21.15.146:0/3941317725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e100c6a80 0x7f7e10160c20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009e50 tx=0x7f7e0800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e269c6921333d695 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:30.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.274+0000 7f7e14ac6700 1 -- 172.21.15.146:0/3941317725 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0800e070 con 0x7f7e100c6a80 2024-09-13T18:53:30.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.278+0000 7f7e14ac6700 1 -- 172.21.15.146:0/3941317725 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0800b9c0 con 0x7f7e100c6a80 2024-09-13T18:53:30.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.278+0000 7f7e17d2c700 1 -- 172.21.15.146:0/3941317725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e100c6a80 msgr2=0x7f7e10160c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:30.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.278+0000 7f7e17d2c700 1 --2- 172.21.15.146:0/3941317725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e100c6a80 0x7f7e10160c20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009e50 tx=0x7f7e0800b330 comp rx=0 tx=0).stop 2024-09-13T18:53:30.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 -- 172.21.15.146:0/3941317725 shutdown_connections 2024-09-13T18:53:30.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 --2- 172.21.15.146:0/3941317725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e100c6a80 0x7f7e10160c20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:30.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 --2- 172.21.15.146:0/3941317725 >> v2:172.21.15.154:3300/0 conn(0x7f7e10153bc0 0x7f7e100c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:30.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 -- 172.21.15.146:0/3941317725 >> 172.21.15.146:0/3941317725 conn(0x7f7e10066090 msgr2=0x7f7e10066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:30.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 -- 172.21.15.146:0/3941317725 shutdown_connections 2024-09-13T18:53:30.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 -- 172.21.15.146:0/3941317725 wait complete. 2024-09-13T18:53:30.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 Processor -- start 2024-09-13T18:53:30.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 -- start start 2024-09-13T18:53:30.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e100c6610 0x7f7e100c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:30.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e100c6a80 0x7f7e100ac550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:30.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e10153bc0 0x7f7e1006dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:30.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e100c45e0 con 0x7f7e10153bc0 2024-09-13T18:53:30.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e100aca90 con 0x7f7e100c6610 2024-09-13T18:53:30.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e17d2c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e10158ed0 con 0x7f7e100c6a80 2024-09-13T18:53:30.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e152c7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e100c6a80 0x7f7e100ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:30.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e162c9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e10153bc0 0x7f7e1006dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:30.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e162c9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e10153bc0 0x7f7e1006dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45562/0 (socket says 172.21.15.146:45562) 2024-09-13T18:53:30.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.282+0000 7f7e162c9700 1 -- 172.21.15.146:0/3015832622 learned_addr learned my addr 172.21.15.146:0/3015832622 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:30.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.286+0000 7f7e162c9700 1 -- 172.21.15.146:0/3015832622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e100c6a80 msgr2=0x7f7e100ac550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:30.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.286+0000 7f7e162c9700 1 --2- 172.21.15.146:0/3015832622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e100c6a80 0x7f7e100ac550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:30.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.286+0000 7f7e162c9700 1 -- 172.21.15.146:0/3015832622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e100c6610 msgr2=0x7f7e100c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:30.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.286+0000 7f7e162c9700 1 --2- 172.21.15.146:0/3015832622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e100c6610 0x7f7e100c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:30.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.286+0000 7f7e162c9700 1 -- 172.21.15.146:0/3015832622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e08009b30 con 0x7f7e10153bc0 2024-09-13T18:53:30.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.286+0000 7f7e162c9700 1 --2- 172.21.15.146:0/3015832622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e10153bc0 0x7f7e1006dcd0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800b730 tx=0x7f7e08004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:30.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.286+0000 7f7e067fc700 1 -- 172.21.15.146:0/3015832622 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0801c850 con 0x7f7e10153bc0 2024-09-13T18:53:30.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.286+0000 7f7e17d2c700 1 -- 172.21.15.146:0/3015832622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e100c5020 con 0x7f7e10153bc0 2024-09-13T18:53:30.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.286+0000 7f7e17d2c700 1 -- 172.21.15.146:0/3015832622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e100c4d40 con 0x7f7e10153bc0 2024-09-13T18:53:30.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.286+0000 7f7e067fc700 1 -- 172.21.15.146:0/3015832622 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0801c9b0 con 0x7f7e10153bc0 2024-09-13T18:53:30.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.286+0000 7f7e067fc700 1 -- 172.21.15.146:0/3015832622 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0801cd20 con 0x7f7e10153bc0 2024-09-13T18:53:30.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.286+0000 7f7e067fc700 1 -- 172.21.15.146:0/3015832622 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7e080123f0 con 0x7f7e10153bc0 2024-09-13T18:53:30.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.294+0000 7f7e067fc700 1 --2- 172.21.15.146:0/3015832622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7df4066880 0x7f7df4068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:30.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.294+0000 7f7e15ac8700 1 --2- 172.21.15.146:0/3015832622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7df4066880 0x7f7df4068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:30.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.294+0000 7f7e067fc700 1 -- 172.21.15.146:0/3015832622 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f7e08089170 con 0x7f7e10153bc0 2024-09-13T18:53:30.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.298+0000 7f7e15ac8700 1 --2- 172.21.15.146:0/3015832622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7df4066880 0x7f7df4068d40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7e000096f0 tx=0x7f7e00006c50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:30.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.302+0000 7f7e17d2c700 1 -- 172.21.15.146:0/3015832622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df8005310 con 0x7f7e10153bc0 2024-09-13T18:53:30.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.318+0000 7f7e067fc700 1 -- 172.21.15.146:0/3015832622 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7e0808a060 con 0x7f7e10153bc0 2024-09-13T18:53:31.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.990+0000 7f7e17d2c700 1 -- 172.21.15.146:0/3015832622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7df80055c0 con 0x7f7e10153bc0 2024-09-13T18:53:31.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:30.994+0000 7f7e067fc700 1 -- 172.21.15.146:0/3015832622 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+6297 (secure 0 0 0) 0x7f7e08052560 con 0x7f7e10153bc0 2024-09-13T18:53:31.010 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:31.011 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:12.828927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6714,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:starting","state_seq":1,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:31.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:31.006+0000 7f7df3fff700 1 -- 172.21.15.146:0/3015832622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7df4066880 msgr2=0x7f7df4068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:31.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:31.006+0000 7f7df3fff700 1 --2- 172.21.15.146:0/3015832622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7df4066880 0x7f7df4068d40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7e000096f0 tx=0x7f7e00006c50 comp rx=0 tx=0).stop 2024-09-13T18:53:31.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:31.006+0000 7f7df3fff700 1 -- 172.21.15.146:0/3015832622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e10153bc0 msgr2=0x7f7e1006dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:31.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:31.006+0000 7f7df3fff700 1 --2- 172.21.15.146:0/3015832622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e10153bc0 0x7f7e1006dcd0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800b730 tx=0x7f7e08004af0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:31.006+0000 7f7df3fff700 1 -- 172.21.15.146:0/3015832622 shutdown_connections 2024-09-13T18:53:31.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:31.006+0000 7f7df3fff700 1 --2- 172.21.15.146:0/3015832622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7df4066880 0x7f7df4068d40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:31.006+0000 7f7df3fff700 1 --2- 172.21.15.146:0/3015832622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e10153bc0 0x7f7e1006dcd0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:31.006+0000 7f7df3fff700 1 --2- 172.21.15.146:0/3015832622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e100c6a80 0x7f7e100ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:31.006+0000 7f7df3fff700 1 --2- 172.21.15.146:0/3015832622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e100c6610 0x7f7e100c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:31.006+0000 7f7df3fff700 1 -- 172.21.15.146:0/3015832622 >> 172.21.15.146:0/3015832622 conn(0x7f7e10066090 msgr2=0x7f7e100cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:31.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:31.010+0000 7f7df3fff700 1 -- 172.21.15.146:0/3015832622 shutdown_connections 2024-09-13T18:53:31.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:31.010+0000 7f7df3fff700 1 -- 172.21.15.146:0/3015832622 wait complete. 2024-09-13T18:53:31.031 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 105 2024-09-13T18:53:31.071 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 105, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 105, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6714": { "addr": "172.21.15.146:6837/2169954140", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 2169954140, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 2169954140, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6714, "incarnation": 86, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 38 }, "gid_6984": { "addr": "172.21.15.146:6835/2934322065", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 2934322065, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 2934322065, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6984, "incarnation": 98, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 28 }, "gid_7098": { "addr": "172.21.15.146:6833/354253522", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 354253522, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 354253522, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7098, "incarnation": 105, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:starting", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 41, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:53:12.828927+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6984, "mds_1": 6714, "mds_2": 7098 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6839/2053734262", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 2053734262, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2053734262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 91, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6032, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:53:33.071 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:33.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c7d22700 1 Processor -- start 2024-09-13T18:53:33.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c7d22700 1 -- start start 2024-09-13T18:53:33.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c7d22700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f93c0159cb0 0x7f93c00c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:33.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c7d22700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f93c006f5c0 0x7f93c00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:33.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c7d22700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c00a9b60 con 0x7f93c00c62b0 2024-09-13T18:53:33.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c7d22700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c00c9c10 con 0x7f93c006f5c0 2024-09-13T18:53:33.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c7d22700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c00c8b70 con 0x7f93c0159cb0 2024-09-13T18:53:33.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c52bd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f93c006f5c0 0x7f93c00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:33.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c52bd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f93c006f5c0 0x7f93c00c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33958/0 (socket says 172.21.15.146:33958) 2024-09-13T18:53:33.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c52bd700 1 -- 172.21.15.146:0/3084223959 learned_addr learned my addr 172.21.15.146:0/3084223959 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:33.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c52bd700 1 -- 172.21.15.146:0/3084223959 >> v2:172.21.15.154:3300/0 conn(0x7f93c0159cb0 msgr2=0x7f93c00c5d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:53:33.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c52bd700 1 --2- 172.21.15.146:0/3084223959 >> v2:172.21.15.154:3300/0 conn(0x7f93c0159cb0 0x7f93c00c5d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:33.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c52bd700 1 -- 172.21.15.146:0/3084223959 >> v1:172.21.15.146:6789/0 conn(0x7f93c00c62b0 legacy=0x7f93c0071980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:53:33.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c52bd700 1 -- 172.21.15.146:0/3084223959 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c00c8e50 con 0x7f93c006f5c0 2024-09-13T18:53:33.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c52bd700 1 --2- 172.21.15.146:0/3084223959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93c006f5c0 0x7f93c00c7760 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f93c0072990 tx=0x7f93b0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f1a7249862b494 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:33.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c4abc700 1 -- 172.21.15.146:0/3084223959 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0004230 con 0x7f93c006f5c0 2024-09-13T18:53:33.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c4abc700 1 -- 172.21.15.146:0/3084223959 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b0004390 con 0x7f93c006f5c0 2024-09-13T18:53:33.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.450+0000 7f93c4abc700 1 -- 172.21.15.146:0/3084223959 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00036c0 con 0x7f93c006f5c0 2024-09-13T18:53:33.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.454+0000 7f93c7d22700 1 -- 172.21.15.146:0/3084223959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93c006f5c0 msgr2=0x7f93c00c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:33.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.454+0000 7f93c7d22700 1 --2- 172.21.15.146:0/3084223959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93c006f5c0 0x7f93c00c7760 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f93c0072990 tx=0x7f93b0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:53:33.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.454+0000 7f93c7d22700 1 -- 172.21.15.146:0/3084223959 shutdown_connections 2024-09-13T18:53:33.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.454+0000 7f93c7d22700 1 --2- 172.21.15.146:0/3084223959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93c006f5c0 0x7f93c00c7760 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:33.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.454+0000 7f93c7d22700 1 --2- 172.21.15.146:0/3084223959 >> v2:172.21.15.154:3300/0 conn(0x7f93c0159cb0 0x7f93c00c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:33.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.454+0000 7f93c7d22700 1 -- 172.21.15.146:0/3084223959 >> 172.21.15.146:0/3084223959 conn(0x7f93c0066090 msgr2=0x7f93c0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:33.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.454+0000 7f93c7d22700 1 -- 172.21.15.146:0/3084223959 shutdown_connections 2024-09-13T18:53:33.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.454+0000 7f93c7d22700 1 -- 172.21.15.146:0/3084223959 wait complete. 2024-09-13T18:53:33.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.458+0000 7f93c7d22700 1 Processor -- start 2024-09-13T18:53:33.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.458+0000 7f93c7d22700 1 -- start start 2024-09-13T18:53:33.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.458+0000 7f93c7d22700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c006f5c0 0x7f93c00c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:33.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.458+0000 7f93c7d22700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93c00c62b0 0x7f93c014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:33.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.458+0000 7f93c7d22700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c0159cb0 0x7f93c01edad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:33.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.458+0000 7f93c7d22700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c01f0dd0 con 0x7f93c00c62b0 2024-09-13T18:53:33.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.458+0000 7f93c7d22700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c01f1020 con 0x7f93c006f5c0 2024-09-13T18:53:33.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.458+0000 7f93c7d22700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c01f1300 con 0x7f93c0159cb0 2024-09-13T18:53:33.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.458+0000 7f93c62bf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c0159cb0 0x7f93c01edad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:33.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.458+0000 7f93c62bf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c0159cb0 0x7f93c01edad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:44740/0 (socket says 172.21.15.146:44740) 2024-09-13T18:53:33.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.458+0000 7f93c5abe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c006f5c0 0x7f93c00c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:33.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.458+0000 7f93c62bf700 1 -- 172.21.15.146:0/2136789153 learned_addr learned my addr 172.21.15.146:0/2136789153 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:33.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.462+0000 7f93c5abe700 1 -- 172.21.15.146:0/2136789153 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c0159cb0 msgr2=0x7f93c01edad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:33.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.462+0000 7f93c5abe700 1 --2- 172.21.15.146:0/2136789153 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c0159cb0 0x7f93c01edad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:33.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.462+0000 7f93c5abe700 1 -- 172.21.15.146:0/2136789153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93c00c62b0 msgr2=0x7f93c014cee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:53:33.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.462+0000 7f93c5abe700 1 --2- 172.21.15.146:0/2136789153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93c00c62b0 0x7f93c014cee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:33.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.462+0000 7f93c5abe700 1 -- 172.21.15.146:0/2136789153 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b0009570 con 0x7f93c006f5c0 2024-09-13T18:53:33.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.462+0000 7f93c5abe700 1 --2- 172.21.15.146:0/2136789153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c006f5c0 0x7f93c00c20d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009f70 tx=0x7f93b800be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:33.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.462+0000 7f93be7fc700 1 -- 172.21.15.146:0/2136789153 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8015710 con 0x7f93c006f5c0 2024-09-13T18:53:33.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.462+0000 7f93c7d22700 1 -- 172.21.15.146:0/2136789153 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c01f15e0 con 0x7f93c006f5c0 2024-09-13T18:53:33.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.462+0000 7f93c7d22700 1 -- 172.21.15.146:0/2136789153 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c01f1ba0 con 0x7f93c006f5c0 2024-09-13T18:53:33.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.462+0000 7f93be7fc700 1 -- 172.21.15.146:0/2136789153 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b8013050 con 0x7f93c006f5c0 2024-09-13T18:53:33.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.466+0000 7f93be7fc700 1 -- 172.21.15.146:0/2136789153 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8015af0 con 0x7f93c006f5c0 2024-09-13T18:53:33.486 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.470+0000 7f93be7fc700 1 -- 172.21.15.146:0/2136789153 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f93b8015c50 con 0x7f93c006f5c0 2024-09-13T18:53:33.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.470+0000 7f93be7fc700 1 --2- 172.21.15.146:0/2136789153 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f93ac066830 0x7f93ac068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:33.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.470+0000 7f93be7fc700 1 -- 172.21.15.146:0/2136789153 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f93b8089580 con 0x7f93c006f5c0 2024-09-13T18:53:33.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.482+0000 7f93c7d22700 1 -- 172.21.15.146:0/2136789153 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a0005310 con 0x7f93c006f5c0 2024-09-13T18:53:33.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.490+0000 7f93c52bd700 1 --2- 172.21.15.146:0/2136789153 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f93ac066830 0x7f93ac068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:33.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.490+0000 7f93c52bd700 1 --2- 172.21.15.146:0/2136789153 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f93ac066830 0x7f93ac068cf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f93c0072990 tx=0x7f93b0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:33.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:33.498+0000 7f93be7fc700 1 -- 172.21.15.146:0/2136789153 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f93b8056060 con 0x7f93c006f5c0 2024-09-13T18:53:34.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.330+0000 7f93c7d22700 1 -- 172.21.15.146:0/2136789153 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93a0005b40 con 0x7f93c006f5c0 2024-09-13T18:53:34.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.330+0000 7f93be7fc700 1 -- 172.21.15.146:0/2136789153 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6295 (secure 0 0 0) 0x7f93b8059680 con 0x7f93c006f5c0 2024-09-13T18:53:34.351 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:34.352 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:33.990688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6714,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:34.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.334+0000 7f93c7d22700 1 -- 172.21.15.146:0/2136789153 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f93ac066830 msgr2=0x7f93ac068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:34.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.334+0000 7f93c7d22700 1 --2- 172.21.15.146:0/2136789153 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f93ac066830 0x7f93ac068cf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f93c0072990 tx=0x7f93b0008030 comp rx=0 tx=0).stop 2024-09-13T18:53:34.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.334+0000 7f93c7d22700 1 -- 172.21.15.146:0/2136789153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c006f5c0 msgr2=0x7f93c00c20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:34.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.334+0000 7f93c7d22700 1 --2- 172.21.15.146:0/2136789153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c006f5c0 0x7f93c00c20d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009f70 tx=0x7f93b800be20 comp rx=0 tx=0).stop 2024-09-13T18:53:34.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.334+0000 7f93c7d22700 1 -- 172.21.15.146:0/2136789153 shutdown_connections 2024-09-13T18:53:34.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.334+0000 7f93c7d22700 1 --2- 172.21.15.146:0/2136789153 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f93ac066830 0x7f93ac068cf0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:34.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.334+0000 7f93c7d22700 1 --2- 172.21.15.146:0/2136789153 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c0159cb0 0x7f93c01edad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:34.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.334+0000 7f93c7d22700 1 --2- 172.21.15.146:0/2136789153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93c00c62b0 0x7f93c014cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:34.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.334+0000 7f93c7d22700 1 --2- 172.21.15.146:0/2136789153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c006f5c0 0x7f93c00c20d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:34.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.334+0000 7f93c7d22700 1 -- 172.21.15.146:0/2136789153 >> 172.21.15.146:0/2136789153 conn(0x7f93c0066090 msgr2=0x7f93c006ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:34.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.334+0000 7f93c7d22700 1 -- 172.21.15.146:0/2136789153 shutdown_connections 2024-09-13T18:53:34.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.334+0000 7f93c7d22700 1 -- 172.21.15.146:0/2136789153 wait complete. 2024-09-13T18:53:34.356 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 106 2024-09-13T18:53:34.372 INFO:tasks.mds_thrash.fs.[cephfs]:mds cluster has 3 alive and active, now stable! 2024-09-13T18:53:34.372 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=1) 2024-09-13T18:53:34.372 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T18:53:34.372 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:53:34.390 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T18:53:34.391 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:34.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.846+0000 7f3ffd588700 1 Processor -- start 2024-09-13T18:53:34.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.850+0000 7f3ffd588700 1 -- start start 2024-09-13T18:53:34.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.850+0000 7f3ffd588700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3ff80caed0 0x7f3ff8072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:34.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.850+0000 7f3ffd588700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ff80cb2b0 0x7f3ff80c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:34.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.850+0000 7f3ffd588700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3ff80c8230 0x7f3ff815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:34.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.850+0000 7f3ffd588700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff80c78d0 con 0x7f3ff80c8230 2024-09-13T18:53:34.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.850+0000 7f3ffd588700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff8161060 con 0x7f3ff80caed0 2024-09-13T18:53:34.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.850+0000 7f3ffd588700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff8161300 con 0x7f3ff80cb2b0 2024-09-13T18:53:34.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.854+0000 7f3ffcd87700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3ff80c8230 0x7f3ff815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:34.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.854+0000 7f3ffcd87700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3ff80c8230 0x7f3ff815e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33962/0 (socket says 172.21.15.146:33962) 2024-09-13T18:53:34.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.854+0000 7f3ffcd87700 1 -- 172.21.15.146:0/3018527026 learned_addr learned my addr 172.21.15.146:0/3018527026 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:34.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.854+0000 7f3ff77fe700 1 --2- 172.21.15.146:0/3018527026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ff80cb2b0 0x7f3ff80c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:34.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.854+0000 7f3ffcd87700 1 -- 172.21.15.146:0/3018527026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ff80cb2b0 msgr2=0x7f3ff80c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:34.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.854+0000 7f3ffcd87700 1 --2- 172.21.15.146:0/3018527026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ff80cb2b0 0x7f3ff80c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:34.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.854+0000 7f3ffcd87700 1 -- 172.21.15.146:0/3018527026 >> v2:172.21.15.154:3300/0 conn(0x7f3ff80caed0 msgr2=0x7f3ff8072380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:53:34.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.854+0000 7f3ffcd87700 1 --2- 172.21.15.146:0/3018527026 >> v2:172.21.15.154:3300/0 conn(0x7f3ff80caed0 0x7f3ff8072380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:34.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.854+0000 7f3ffcd87700 1 -- 172.21.15.146:0/3018527026 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff81615c0 con 0x7f3ff80c8230 2024-09-13T18:53:34.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.854+0000 7f3ffcd87700 1 --2- 172.21.15.146:0/3018527026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ff80c8230 0x7f3ff815e170 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80c87e0 tx=0x7f3fe8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f343e68a13ff0d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:34.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.858+0000 7f3ff6ffd700 1 -- 172.21.15.146:0/3018527026 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8011830 con 0x7f3ff80c8230 2024-09-13T18:53:34.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.858+0000 7f3ff6ffd700 1 -- 172.21.15.146:0/3018527026 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe8011990 con 0x7f3ff80c8230 2024-09-13T18:53:34.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.862+0000 7f3ffd588700 1 -- 172.21.15.146:0/3018527026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ff80c8230 msgr2=0x7f3ff815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:34.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.862+0000 7f3ffd588700 1 --2- 172.21.15.146:0/3018527026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ff80c8230 0x7f3ff815e170 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80c87e0 tx=0x7f3fe8009bf0 comp rx=0 tx=0).stop 2024-09-13T18:53:34.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.862+0000 7f3ffd588700 1 -- 172.21.15.146:0/3018527026 shutdown_connections 2024-09-13T18:53:34.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.862+0000 7f3ffd588700 1 --2- 172.21.15.146:0/3018527026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ff80cb2b0 0x7f3ff80c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:34.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.862+0000 7f3ffd588700 1 --2- 172.21.15.146:0/3018527026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ff80c8230 0x7f3ff815e170 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:34.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.862+0000 7f3ffd588700 1 --2- 172.21.15.146:0/3018527026 >> v2:172.21.15.154:3300/0 conn(0x7f3ff80caed0 0x7f3ff8072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:34.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.862+0000 7f3ffd588700 1 -- 172.21.15.146:0/3018527026 >> 172.21.15.146:0/3018527026 conn(0x7f3ff8066090 msgr2=0x7f3ff8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:34.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.866+0000 7f3ffd588700 1 -- 172.21.15.146:0/3018527026 shutdown_connections 2024-09-13T18:53:34.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.866+0000 7f3ffd588700 1 -- 172.21.15.146:0/3018527026 wait complete. 2024-09-13T18:53:34.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ffd588700 1 Processor -- start 2024-09-13T18:53:34.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ffd588700 1 -- start start 2024-09-13T18:53:34.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ffd588700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ff80caed0 0x7f3ff806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:34.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ffd588700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ff80cb2b0 0x7f3ff80aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:34.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ffd588700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ff80ab3f0 0x7f3ff80ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:34.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ffd588700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff80b0260 con 0x7f3ff80caed0 2024-09-13T18:53:34.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ffd588700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff814dc80 con 0x7f3ff80ab3f0 2024-09-13T18:53:34.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ffd588700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff80ae010 con 0x7f3ff80cb2b0 2024-09-13T18:53:34.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ff7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ff80caed0 0x7f3ff806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:34.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ff7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ff80caed0 0x7f3ff806f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33968/0 (socket says 172.21.15.146:33968) 2024-09-13T18:53:34.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ff7fff700 1 -- 172.21.15.146:0/3797458632 learned_addr learned my addr 172.21.15.146:0/3797458632 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:34.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ff77fe700 1 --2- 172.21.15.146:0/3797458632 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ff80cb2b0 0x7f3ff80aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:34.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ff7fff700 1 -- 172.21.15.146:0/3797458632 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ff80cb2b0 msgr2=0x7f3ff80aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:34.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ff7fff700 1 --2- 172.21.15.146:0/3797458632 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ff80cb2b0 0x7f3ff80aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:34.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ff7fff700 1 -- 172.21.15.146:0/3797458632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ff80ab3f0 msgr2=0x7f3ff80ae4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:53:34.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ff7fff700 1 --2- 172.21.15.146:0/3797458632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ff80ab3f0 0x7f3ff80ae4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:34.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.870+0000 7f3ff7fff700 1 -- 172.21.15.146:0/3797458632 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe8009570 con 0x7f3ff80caed0 2024-09-13T18:53:34.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.874+0000 7f3ff7fff700 1 --2- 172.21.15.146:0/3797458632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ff80caed0 0x7f3ff806f830 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00b7b0 tx=0x7f3fec00bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:34.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.874+0000 7f3ff4ff9700 1 -- 172.21.15.146:0/3797458632 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec010680 con 0x7f3ff80caed0 2024-09-13T18:53:34.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.874+0000 7f3ffd588700 1 -- 172.21.15.146:0/3797458632 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff80aba10 con 0x7f3ff80caed0 2024-09-13T18:53:34.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.874+0000 7f3ffd588700 1 -- 172.21.15.146:0/3797458632 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff80aff80 con 0x7f3ff80caed0 2024-09-13T18:53:34.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.874+0000 7f3ff4ff9700 1 -- 172.21.15.146:0/3797458632 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fec010800 con 0x7f3ff80caed0 2024-09-13T18:53:34.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.874+0000 7f3ff4ff9700 1 -- 172.21.15.146:0/3797458632 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec010bb0 con 0x7f3ff80caed0 2024-09-13T18:53:34.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.878+0000 7f3ff4ff9700 1 -- 172.21.15.146:0/3797458632 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3fec010d10 con 0x7f3ff80caed0 2024-09-13T18:53:34.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.878+0000 7f3ff4ff9700 1 --2- 172.21.15.146:0/3797458632 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3fe4066730 0x7f3fe4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:34.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.878+0000 7f3ff4ff9700 1 -- 172.21.15.146:0/3797458632 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f3fec084900 con 0x7f3ff80caed0 2024-09-13T18:53:34.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.878+0000 7f3ff77fe700 1 --2- 172.21.15.146:0/3797458632 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3fe4066730 0x7f3fe4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:34.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.878+0000 7f3ff77fe700 1 --2- 172.21.15.146:0/3797458632 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3fe4066730 0x7f3fe4068bf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8070ac0 tx=0x7f3ff000d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:34.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.890+0000 7f3ffd588700 1 -- 172.21.15.146:0/3797458632 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd8005310 con 0x7f3ff80caed0 2024-09-13T18:53:34.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:34.906+0000 7f3ff4ff9700 1 -- 172.21.15.146:0/3797458632 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3fec051360 con 0x7f3ff80caed0 2024-09-13T18:53:35.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.678+0000 7f3ffd588700 1 -- 172.21.15.146:0/3797458632 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fd8005b40 con 0x7f3ff80caed0 2024-09-13T18:53:35.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.678+0000 7f3ff4ff9700 1 -- 172.21.15.146:0/3797458632 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6295 (secure 0 0 0) 0x7f3fec01e070 con 0x7f3ff80caed0 2024-09-13T18:53:35.700 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:35.701 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:33.990688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6714,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:35.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.682+0000 7f3fde7fc700 1 -- 172.21.15.146:0/3797458632 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3fe4066730 msgr2=0x7f3fe4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:35.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.682+0000 7f3fde7fc700 1 --2- 172.21.15.146:0/3797458632 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3fe4066730 0x7f3fe4068bf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8070ac0 tx=0x7f3ff000d040 comp rx=0 tx=0).stop 2024-09-13T18:53:35.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.682+0000 7f3fde7fc700 1 -- 172.21.15.146:0/3797458632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ff80caed0 msgr2=0x7f3ff806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:35.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.682+0000 7f3fde7fc700 1 --2- 172.21.15.146:0/3797458632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ff80caed0 0x7f3ff806f830 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00b7b0 tx=0x7f3fec00bac0 comp rx=0 tx=0).stop 2024-09-13T18:53:35.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.682+0000 7f3fde7fc700 1 -- 172.21.15.146:0/3797458632 shutdown_connections 2024-09-13T18:53:35.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.682+0000 7f3fde7fc700 1 --2- 172.21.15.146:0/3797458632 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3fe4066730 0x7f3fe4068bf0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:35.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.682+0000 7f3fde7fc700 1 --2- 172.21.15.146:0/3797458632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ff80ab3f0 0x7f3ff80ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:35.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.682+0000 7f3fde7fc700 1 --2- 172.21.15.146:0/3797458632 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3ff80cb2b0 0x7f3ff80aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:35.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.682+0000 7f3fde7fc700 1 --2- 172.21.15.146:0/3797458632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ff80caed0 0x7f3ff806f830 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:35.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.682+0000 7f3fde7fc700 1 -- 172.21.15.146:0/3797458632 >> 172.21.15.146:0/3797458632 conn(0x7f3ff8066090 msgr2=0x7f3ff806c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:35.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.682+0000 7f3fde7fc700 1 -- 172.21.15.146:0/3797458632 shutdown_connections 2024-09-13T18:53:35.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:35.682+0000 7f3fde7fc700 1 -- 172.21.15.146:0/3797458632 wait complete. 2024-09-13T18:53:35.702 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 106 2024-09-13T18:53:35.718 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:53:37.719 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:37.746 INFO:tasks.ceph.mds.a.smithi146.stderr:2024-09-13T18:53:37.730+0000 7f40aeb2a700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-13T18:53:38.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.042+0000 7f217a91a700 1 Processor -- start 2024-09-13T18:53:38.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.042+0000 7f217a91a700 1 -- start start 2024-09-13T18:53:38.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f217a91a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f21740c9350 0x7f21740a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:38.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f217a91a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f21740c7530 0x7f21740cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:38.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f217a91a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2174159f80 con 0x7f21740c8f70 2024-09-13T18:53:38.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f217a91a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21740ca1d0 con 0x7f21740c9350 2024-09-13T18:53:38.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f217a91a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2174159670 con 0x7f21740c7530 2024-09-13T18:53:38.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f2179117700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f21740c9350 0x7f21740a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:38.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f2179117700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f21740c9350 0x7f21740a9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33978/0 (socket says 172.21.15.146:33978) 2024-09-13T18:53:38.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f2179918700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f21740c8f70 0x7f21740723f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35208/0 (socket says 172.21.15.146:35208) 2024-09-13T18:53:38.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f2179117700 1 -- 172.21.15.146:0/288178073 learned_addr learned my addr 172.21.15.146:0/288178073 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:38.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f217a119700 1 --2- 172.21.15.146:0/288178073 >> v2:172.21.15.154:3300/0 conn(0x7f21740c7530 0x7f21740cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:38.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f2179117700 1 -- 172.21.15.146:0/288178073 >> v2:172.21.15.154:3300/0 conn(0x7f21740c7530 msgr2=0x7f21740cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:38.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f2179117700 1 --2- 172.21.15.146:0/288178073 >> v2:172.21.15.154:3300/0 conn(0x7f21740c7530 0x7f21740cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:38.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f2179117700 1 -- 172.21.15.146:0/288178073 >> v1:172.21.15.146:6789/0 conn(0x7f21740c8f70 legacy=0x7f21740723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:38.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f2179117700 1 -- 172.21.15.146:0/288178073 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21740aa3a0 con 0x7f21740c9350 2024-09-13T18:53:38.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f2179117700 1 --2- 172.21.15.146:0/288178073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21740c9350 0x7f21740a9cf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f217406edb0 tx=0x7f216c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc5b313fc00c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:38.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f2178916700 1 -- 172.21.15.146:0/288178073 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c00e070 con 0x7f21740c9350 2024-09-13T18:53:38.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.046+0000 7f2178916700 1 -- 172.21.15.146:0/288178073 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c003a40 con 0x7f21740c9350 2024-09-13T18:53:38.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.050+0000 7f217a91a700 1 -- 172.21.15.146:0/288178073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21740c9350 msgr2=0x7f21740a9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:38.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.050+0000 7f217a91a700 1 --2- 172.21.15.146:0/288178073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21740c9350 0x7f21740a9cf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f217406edb0 tx=0x7f216c00b330 comp rx=0 tx=0).stop 2024-09-13T18:53:38.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.074+0000 7f217a91a700 1 -- 172.21.15.146:0/288178073 shutdown_connections 2024-09-13T18:53:38.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.074+0000 7f217a91a700 1 --2- 172.21.15.146:0/288178073 >> v2:172.21.15.154:3300/0 conn(0x7f21740c7530 0x7f21740cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:38.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.074+0000 7f217a91a700 1 --2- 172.21.15.146:0/288178073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21740c9350 0x7f21740a9cf0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:38.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.074+0000 7f217a91a700 1 -- 172.21.15.146:0/288178073 >> 172.21.15.146:0/288178073 conn(0x7f2174066090 msgr2=0x7f2174066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:38.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.082+0000 7f217a91a700 1 -- 172.21.15.146:0/288178073 shutdown_connections 2024-09-13T18:53:38.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.082+0000 7f217a91a700 1 -- 172.21.15.146:0/288178073 wait complete. 2024-09-13T18:53:38.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.082+0000 7f217a91a700 1 Processor -- start 2024-09-13T18:53:38.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.086+0000 7f217a91a700 1 -- start start 2024-09-13T18:53:38.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.086+0000 7f217a91a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21740c7530 0x7f217406fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:38.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.086+0000 7f217a91a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21740c8f70 0x7f21742001f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:38.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.086+0000 7f217a91a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21740c9350 0x7f2174200730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:38.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.086+0000 7f217a91a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21740b0360 con 0x7f21740c9350 2024-09-13T18:53:38.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.086+0000 7f217a91a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21741ffda0 con 0x7f21740c7530 2024-09-13T18:53:38.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.086+0000 7f217a91a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21740b0080 con 0x7f21740c8f70 2024-09-13T18:53:38.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.086+0000 7f217a119700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21740c9350 0x7f2174200730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:38.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.086+0000 7f217a119700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21740c9350 0x7f2174200730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33992/0 (socket says 172.21.15.146:33992) 2024-09-13T18:53:38.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.086+0000 7f217a119700 1 -- 172.21.15.146:0/1977149963 learned_addr learned my addr 172.21.15.146:0/1977149963 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:38.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.090+0000 7f217a119700 1 -- 172.21.15.146:0/1977149963 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21740c8f70 msgr2=0x7f21742001f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:53:38.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.090+0000 7f217a119700 1 --2- 172.21.15.146:0/1977149963 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21740c8f70 0x7f21742001f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:38.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.090+0000 7f217a119700 1 -- 172.21.15.146:0/1977149963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21740c7530 msgr2=0x7f217406fdf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:53:38.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.090+0000 7f217a119700 1 --2- 172.21.15.146:0/1977149963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21740c7530 0x7f217406fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:38.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.090+0000 7f217a119700 1 -- 172.21.15.146:0/1977149963 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c009b30 con 0x7f21740c9350 2024-09-13T18:53:38.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.090+0000 7f217a119700 1 --2- 172.21.15.146:0/1977149963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21740c9350 0x7f2174200730 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f217000eb40 tx=0x7f217000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:38.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.090+0000 7f216a7fc700 1 -- 172.21.15.146:0/1977149963 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217000c7b0 con 0x7f21740c9350 2024-09-13T18:53:38.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.090+0000 7f216a7fc700 1 -- 172.21.15.146:0/1977149963 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217000c910 con 0x7f21740c9350 2024-09-13T18:53:38.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.090+0000 7f217a91a700 1 -- 172.21.15.146:0/1977149963 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2174203d20 con 0x7f21740c9350 2024-09-13T18:53:38.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.090+0000 7f217a91a700 1 -- 172.21.15.146:0/1977149963 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2174204000 con 0x7f21740c9350 2024-09-13T18:53:38.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.090+0000 7f216a7fc700 1 -- 172.21.15.146:0/1977149963 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217000cca0 con 0x7f21740c9350 2024-09-13T18:53:38.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.090+0000 7f216a7fc700 1 -- 172.21.15.146:0/1977149963 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2170013470 con 0x7f21740c9350 2024-09-13T18:53:38.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.090+0000 7f216a7fc700 1 --2- 172.21.15.146:0/1977149963 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f21580667b0 0x7f2158068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:38.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.094+0000 7f216a7fc700 1 -- 172.21.15.146:0/1977149963 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f2170084c00 con 0x7f21740c9350 2024-09-13T18:53:38.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.094+0000 7f217a91a700 1 -- 172.21.15.146:0/1977149963 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f215c005310 con 0x7f21740c9350 2024-09-13T18:53:38.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.102+0000 7f216a7fc700 1 -- 172.21.15.146:0/1977149963 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2170021080 con 0x7f21740c9350 2024-09-13T18:53:38.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.102+0000 7f2179918700 1 --2- 172.21.15.146:0/1977149963 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f21580667b0 0x7f2158068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:38.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.114+0000 7f2179918700 1 --2- 172.21.15.146:0/1977149963 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f21580667b0 0x7f2158068c70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2164001700 tx=0x7f21640079f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:38.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.934+0000 7f217a91a700 1 -- 172.21.15.146:0/1977149963 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f215c005b40 con 0x7f21740c9350 2024-09-13T18:53:38.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.938+0000 7f216a7fc700 1 -- 172.21.15.146:0/1977149963 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6295 (secure 0 0 0) 0x7f2170017020 con 0x7f21740c9350 2024-09-13T18:53:38.958 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:38.958 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:33.990688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6714,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:38.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.958+0000 7f2157fff700 1 -- 172.21.15.146:0/1977149963 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f21580667b0 msgr2=0x7f2158068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:38.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.958+0000 7f2157fff700 1 --2- 172.21.15.146:0/1977149963 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f21580667b0 0x7f2158068c70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2164001700 tx=0x7f21640079f0 comp rx=0 tx=0).stop 2024-09-13T18:53:38.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.958+0000 7f2157fff700 1 -- 172.21.15.146:0/1977149963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21740c9350 msgr2=0x7f2174200730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:38.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.958+0000 7f2157fff700 1 --2- 172.21.15.146:0/1977149963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21740c9350 0x7f2174200730 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f217000eb40 tx=0x7f217000ef00 comp rx=0 tx=0).stop 2024-09-13T18:53:38.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.958+0000 7f2157fff700 1 -- 172.21.15.146:0/1977149963 shutdown_connections 2024-09-13T18:53:38.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.958+0000 7f2157fff700 1 --2- 172.21.15.146:0/1977149963 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f21580667b0 0x7f2158068c70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:38.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.958+0000 7f2157fff700 1 --2- 172.21.15.146:0/1977149963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21740c9350 0x7f2174200730 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:38.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.958+0000 7f2157fff700 1 --2- 172.21.15.146:0/1977149963 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21740c8f70 0x7f21742001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:38.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.958+0000 7f2157fff700 1 --2- 172.21.15.146:0/1977149963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21740c7530 0x7f217406fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:38.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.958+0000 7f2157fff700 1 -- 172.21.15.146:0/1977149963 >> 172.21.15.146:0/1977149963 conn(0x7f2174066090 msgr2=0x7f217406d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:38.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.958+0000 7f2157fff700 1 -- 172.21.15.146:0/1977149963 shutdown_connections 2024-09-13T18:53:38.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:38.962+0000 7f2157fff700 1 -- 172.21.15.146:0/1977149963 wait complete. 2024-09-13T18:53:38.982 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 106 2024-09-13T18:53:39.009 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:53:41.010 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:41.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.343+0000 7f7a58370700 1 Processor -- start 2024-09-13T18:53:41.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.343+0000 7f7a58370700 1 -- start start 2024-09-13T18:53:41.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.343+0000 7f7a58370700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7a500caed0 0x7f7a50072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:41.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.343+0000 7f7a58370700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a500cb2b0 0x7f7a500c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:41.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.343+0000 7f7a58370700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7a500c8230 0x7f7a5015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:41.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.343+0000 7f7a58370700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a500c78d0 con 0x7f7a500c8230 2024-09-13T18:53:41.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.343+0000 7f7a58370700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a50161060 con 0x7f7a500caed0 2024-09-13T18:53:41.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.343+0000 7f7a58370700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50161300 con 0x7f7a500cb2b0 2024-09-13T18:53:41.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.347+0000 7f7a5690d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7a500c8230 0x7f7a5015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:41.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.347+0000 7f7a5690d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7a500c8230 0x7f7a5015e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39132/0 (socket says 172.21.15.146:39132) 2024-09-13T18:53:41.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.347+0000 7f7a5690d700 1 -- 172.21.15.146:0/972719021 learned_addr learned my addr 172.21.15.146:0/972719021 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:41.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.347+0000 7f7a5690d700 1 -- 172.21.15.146:0/972719021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a500cb2b0 msgr2=0x7f7a500c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:41.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.347+0000 7f7a5690d700 1 --2- 172.21.15.146:0/972719021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a500cb2b0 0x7f7a500c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:41.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.347+0000 7f7a5690d700 1 -- 172.21.15.146:0/972719021 >> v2:172.21.15.154:3300/0 conn(0x7f7a500caed0 msgr2=0x7f7a50072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:41.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.347+0000 7f7a5690d700 1 --2- 172.21.15.146:0/972719021 >> v2:172.21.15.154:3300/0 conn(0x7f7a500caed0 0x7f7a50072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:41.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.347+0000 7f7a5690d700 1 -- 172.21.15.146:0/972719021 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a501615c0 con 0x7f7a500c8230 2024-09-13T18:53:41.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.347+0000 7f7a5690d700 1 --2- 172.21.15.146:0/972719021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a500c8230 0x7f7a5015e170 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7a500c87e0 tx=0x7f7a4c009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129c27f8c9984db8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:41.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.351+0000 7f7a5510a700 1 -- 172.21.15.146:0/972719021 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00b910 con 0x7f7a500c8230 2024-09-13T18:53:41.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.351+0000 7f7a5510a700 1 -- 172.21.15.146:0/972719021 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c00ba70 con 0x7f7a500c8230 2024-09-13T18:53:41.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.351+0000 7f7a5510a700 1 -- 172.21.15.146:0/972719021 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00be00 con 0x7f7a500c8230 2024-09-13T18:53:41.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.355+0000 7f7a58370700 1 -- 172.21.15.146:0/972719021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a500c8230 msgr2=0x7f7a5015e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:41.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.355+0000 7f7a58370700 1 --2- 172.21.15.146:0/972719021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a500c8230 0x7f7a5015e170 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7a500c87e0 tx=0x7f7a4c009e50 comp rx=0 tx=0).stop 2024-09-13T18:53:41.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.363+0000 7f7a58370700 1 -- 172.21.15.146:0/972719021 shutdown_connections 2024-09-13T18:53:41.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.363+0000 7f7a58370700 1 --2- 172.21.15.146:0/972719021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a500cb2b0 0x7f7a500c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:41.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.363+0000 7f7a58370700 1 --2- 172.21.15.146:0/972719021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a500c8230 0x7f7a5015e170 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:41.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.363+0000 7f7a58370700 1 --2- 172.21.15.146:0/972719021 >> v2:172.21.15.154:3300/0 conn(0x7f7a500caed0 0x7f7a50072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:41.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.363+0000 7f7a58370700 1 -- 172.21.15.146:0/972719021 >> 172.21.15.146:0/972719021 conn(0x7f7a50066090 msgr2=0x7f7a50066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:41.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.363+0000 7f7a58370700 1 -- 172.21.15.146:0/972719021 shutdown_connections 2024-09-13T18:53:41.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.363+0000 7f7a58370700 1 -- 172.21.15.146:0/972719021 wait complete. 2024-09-13T18:53:41.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.367+0000 7f7a58370700 1 Processor -- start 2024-09-13T18:53:41.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.371+0000 7f7a58370700 1 -- start start 2024-09-13T18:53:41.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a58370700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a500c8230 0x7f7a50167ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:41.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a58370700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a500caed0 0x7f7a500aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:41.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a58370700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a500cb2b0 0x7f7a500acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:41.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a58370700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50163c50 con 0x7f7a500c8230 2024-09-13T18:53:41.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a58370700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5015e590 con 0x7f7a500cb2b0 2024-09-13T18:53:41.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a58370700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50163ec0 con 0x7f7a500caed0 2024-09-13T18:53:41.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a5610c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a500c8230 0x7f7a50167ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:41.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a5610c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a500c8230 0x7f7a50167ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39148/0 (socket says 172.21.15.146:39148) 2024-09-13T18:53:41.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a5610c700 1 -- 172.21.15.146:0/3724403141 learned_addr learned my addr 172.21.15.146:0/3724403141 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:41.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a5610c700 1 -- 172.21.15.146:0/3724403141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a500caed0 msgr2=0x7f7a500aca80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:53:41.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a5610c700 1 --2- 172.21.15.146:0/3724403141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a500caed0 0x7f7a500aca80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:41.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a5610c700 1 -- 172.21.15.146:0/3724403141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a500cb2b0 msgr2=0x7f7a500acfc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:53:41.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a5610c700 1 --2- 172.21.15.146:0/3724403141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a500cb2b0 0x7f7a500acfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:41.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a5610c700 1 -- 172.21.15.146:0/3724403141 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c009570 con 0x7f7a500c8230 2024-09-13T18:53:41.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.375+0000 7f7a5610c700 1 --2- 172.21.15.146:0/3724403141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a500c8230 0x7f7a50167ba0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000b550 tx=0x7f7a4000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:41.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.379+0000 7f7a46ffd700 1 -- 172.21.15.146:0/3724403141 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40011830 con 0x7f7a500c8230 2024-09-13T18:53:41.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.379+0000 7f7a58370700 1 -- 172.21.15.146:0/3724403141 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a501641a0 con 0x7f7a500c8230 2024-09-13T18:53:41.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.379+0000 7f7a58370700 1 -- 172.21.15.146:0/3724403141 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5020e4c0 con 0x7f7a500c8230 2024-09-13T18:53:41.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.379+0000 7f7a46ffd700 1 -- 172.21.15.146:0/3724403141 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a40011990 con 0x7f7a500c8230 2024-09-13T18:53:41.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.379+0000 7f7a46ffd700 1 -- 172.21.15.146:0/3724403141 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40011d40 con 0x7f7a500c8230 2024-09-13T18:53:41.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.383+0000 7f7a46ffd700 1 -- 172.21.15.146:0/3724403141 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a4001a400 con 0x7f7a500c8230 2024-09-13T18:53:41.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.383+0000 7f7a46ffd700 1 --2- 172.21.15.146:0/3724403141 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7a3c0669e0 0x7f7a3c068ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:41.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.383+0000 7f7a46ffd700 1 -- 172.21.15.146:0/3724403141 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f7a40084a40 con 0x7f7a500c8230 2024-09-13T18:53:41.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.383+0000 7f7a5590b700 1 --2- 172.21.15.146:0/3724403141 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7a3c0669e0 0x7f7a3c068ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:41.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.383+0000 7f7a5590b700 1 --2- 172.21.15.146:0/3724403141 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7a3c0669e0 0x7f7a3c068ea0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7a5015e500 tx=0x7f7a48009250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:41.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.387+0000 7f7a58370700 1 -- 172.21.15.146:0/3724403141 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a38005310 con 0x7f7a500c8230 2024-09-13T18:53:41.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:41.395+0000 7f7a46ffd700 1 -- 172.21.15.146:0/3724403141 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7a40015080 con 0x7f7a500c8230 2024-09-13T18:53:42.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.203+0000 7f7a58370700 1 -- 172.21.15.146:0/3724403141 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a38005b40 con 0x7f7a500c8230 2024-09-13T18:53:42.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.203+0000 7f7a46ffd700 1 -- 172.21.15.146:0/3724403141 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6295 (secure 0 0 0) 0x7f7a40051520 con 0x7f7a500c8230 2024-09-13T18:53:42.223 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:42.223 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:33.990688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6714,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:42.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.207+0000 7f7a58370700 1 -- 172.21.15.146:0/3724403141 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7a3c0669e0 msgr2=0x7f7a3c068ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:42.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.207+0000 7f7a58370700 1 --2- 172.21.15.146:0/3724403141 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7a3c0669e0 0x7f7a3c068ea0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7a5015e500 tx=0x7f7a48009250 comp rx=0 tx=0).stop 2024-09-13T18:53:42.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.207+0000 7f7a58370700 1 -- 172.21.15.146:0/3724403141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a500c8230 msgr2=0x7f7a50167ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:42.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.207+0000 7f7a58370700 1 --2- 172.21.15.146:0/3724403141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a500c8230 0x7f7a50167ba0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000b550 tx=0x7f7a4000b910 comp rx=0 tx=0).stop 2024-09-13T18:53:42.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.207+0000 7f7a58370700 1 -- 172.21.15.146:0/3724403141 shutdown_connections 2024-09-13T18:53:42.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.207+0000 7f7a58370700 1 --2- 172.21.15.146:0/3724403141 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7a3c0669e0 0x7f7a3c068ea0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:42.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.207+0000 7f7a58370700 1 --2- 172.21.15.146:0/3724403141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a500cb2b0 0x7f7a500acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:42.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.207+0000 7f7a58370700 1 --2- 172.21.15.146:0/3724403141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a500caed0 0x7f7a500aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:42.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.207+0000 7f7a58370700 1 --2- 172.21.15.146:0/3724403141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a500c8230 0x7f7a50167ba0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:42.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.207+0000 7f7a58370700 1 -- 172.21.15.146:0/3724403141 >> 172.21.15.146:0/3724403141 conn(0x7f7a50066090 msgr2=0x7f7a501601d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:42.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.207+0000 7f7a58370700 1 -- 172.21.15.146:0/3724403141 shutdown_connections 2024-09-13T18:53:42.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:42.211+0000 7f7a58370700 1 -- 172.21.15.146:0/3724403141 wait complete. 2024-09-13T18:53:42.233 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 106 2024-09-13T18:53:42.262 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:53:44.262 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.607+0000 7fe8b03f5700 1 Processor -- start 2024-09-13T18:53:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8b03f5700 1 -- start start 2024-09-13T18:53:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8b03f5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 0x7fe8a815db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8b03f5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe8a81610f0 0x7fe8a80700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8b03f5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a806f280 con 0x7fe8a80c6df0 2024-09-13T18:53:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8b03f5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a815e410 con 0x7fe8a81610f0 2024-09-13T18:53:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8b03f5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a815e6f0 con 0x7fe8a8158080 2024-09-13T18:53:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8ae191700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 0x7fe8a815db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8ae191700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 0x7fe8a815db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:51424/0 (socket says 172.21.15.146:51424) 2024-09-13T18:53:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8ae191700 1 -- 172.21.15.146:0/1392351017 learned_addr learned my addr 172.21.15.146:0/1392351017 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8ad990700 1 --2- 172.21.15.146:0/1392351017 >> v2:172.21.15.154:3300/0 conn(0x7fe8a81610f0 0x7fe8a80700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8ae191700 1 -- 172.21.15.146:0/1392351017 >> v2:172.21.15.154:3300/0 conn(0x7fe8a81610f0 msgr2=0x7fe8a80700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:44.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8ae191700 1 --2- 172.21.15.146:0/1392351017 >> v2:172.21.15.154:3300/0 conn(0x7fe8a81610f0 0x7fe8a80700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:44.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8ae191700 1 -- 172.21.15.146:0/1392351017 >> v1:172.21.15.154:6789/0 conn(0x7fe8a80c6df0 legacy=0x7fe8a80c5bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:53:44.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8ae191700 1 -- 172.21.15.146:0/1392351017 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a815e9d0 con 0x7fe8a8158080 2024-09-13T18:53:44.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8ad990700 1 --2- 172.21.15.146:0/1392351017 >> v2:172.21.15.154:3300/0 conn(0x7fe8a81610f0 0x7fe8a80700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:44.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.611+0000 7fe8ae191700 1 --2- 172.21.15.146:0/1392351017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 0x7fe8a815db80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009e50 tx=0x7fe8a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc0721c020586345 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:44.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.615+0000 7fe8ad18f700 1 -- 172.21.15.146:0/1392351017 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000e070 con 0x7fe8a8158080 2024-09-13T18:53:44.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.615+0000 7fe8ad18f700 1 -- 172.21.15.146:0/1392351017 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0003a40 con 0x7fe8a8158080 2024-09-13T18:53:44.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.615+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1392351017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 msgr2=0x7fe8a815db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:44.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.615+0000 7fe8b03f5700 1 --2- 172.21.15.146:0/1392351017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 0x7fe8a815db80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009e50 tx=0x7fe8a000b330 comp rx=0 tx=0).stop 2024-09-13T18:53:44.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.639+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1392351017 shutdown_connections 2024-09-13T18:53:44.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.639+0000 7fe8b03f5700 1 --2- 172.21.15.146:0/1392351017 >> v2:172.21.15.154:3300/0 conn(0x7fe8a81610f0 0x7fe8a80700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:44.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.639+0000 7fe8b03f5700 1 --2- 172.21.15.146:0/1392351017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 0x7fe8a815db80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:44.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.639+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1392351017 >> 172.21.15.146:0/1392351017 conn(0x7fe8a80cb1d0 msgr2=0x7fe8a80cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:44.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.639+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1392351017 shutdown_connections 2024-09-13T18:53:44.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.639+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1392351017 wait complete. 2024-09-13T18:53:44.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.643+0000 7fe8b03f5700 1 Processor -- start 2024-09-13T18:53:44.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.643+0000 7fe8b03f5700 1 -- start start 2024-09-13T18:53:44.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8b03f5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8a80c6df0 0x7fe8a80c1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:44.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8b03f5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 0x7fe8a80c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:44.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8b03f5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8a81610f0 0x7fe8a8156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:44.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8b03f5700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a814db10 con 0x7fe8a81610f0 2024-09-13T18:53:44.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8b03f5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a80c2c60 con 0x7fe8a80c6df0 2024-09-13T18:53:44.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8b03f5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a815a640 con 0x7fe8a8158080 2024-09-13T18:53:44.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8ad990700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 0x7fe8a80c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:44.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8ad990700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 0x7fe8a80c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:51438/0 (socket says 172.21.15.146:51438) 2024-09-13T18:53:44.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8ad990700 1 -- 172.21.15.146:0/1089723164 learned_addr learned my addr 172.21.15.146:0/1089723164 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:44.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8ae191700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8a80c6df0 0x7fe8a80c1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:44.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8ae992700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8a81610f0 0x7fe8a8156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:44.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8ad990700 1 -- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8a80c6df0 msgr2=0x7fe8a80c1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:44.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8ad990700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8a80c6df0 0x7fe8a80c1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:44.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8ad990700 1 -- 172.21.15.146:0/1089723164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8a81610f0 msgr2=0x7fe8a8156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:44.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8ad990700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8a81610f0 0x7fe8a8156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:44.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8ad990700 1 -- 172.21.15.146:0/1089723164 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a0009b30 con 0x7fe8a8158080 2024-09-13T18:53:44.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8ae992700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8a81610f0 0x7fe8a8156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:53:44.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8ad990700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 0x7fe8a80c2f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe89800d800 tx=0x7fe89800db10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:44.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe89effd700 1 -- 172.21.15.146:0/1089723164 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8980125f0 con 0x7fe8a8158080 2024-09-13T18:53:44.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe89effd700 1 -- 172.21.15.146:0/1089723164 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe898012750 con 0x7fe8a8158080 2024-09-13T18:53:44.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe89effd700 1 -- 172.21.15.146:0/1089723164 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898012b00 con 0x7fe8a8158080 2024-09-13T18:53:44.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1089723164 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a80c5010 con 0x7fe8a8158080 2024-09-13T18:53:44.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.647+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1089723164 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a80c52d0 con 0x7fe8a8158080 2024-09-13T18:53:44.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.651+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1089723164 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a815a780 con 0x7fe8a8158080 2024-09-13T18:53:44.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.651+0000 7fe89effd700 1 -- 172.21.15.146:0/1089723164 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe898012c60 con 0x7fe8a8158080 2024-09-13T18:53:44.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.651+0000 7fe89effd700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe88c066780 0x7fe88c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:44.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.651+0000 7fe89effd700 1 -- 172.21.15.146:0/1089723164 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7fe898084d80 con 0x7fe8a8158080 2024-09-13T18:53:44.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.651+0000 7fe8ae191700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe88c066780 0x7fe88c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:44.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.659+0000 7fe8ae191700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe88c066780 0x7fe88c068c40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009850 tx=0x7fe8a0009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:44.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:44.659+0000 7fe89effd700 1 -- 172.21.15.146:0/1089723164 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe8980518d0 con 0x7fe8a8158080 2024-09-13T18:53:45.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.403+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1089723164 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8a8070a40 con 0x7fe8a8158080 2024-09-13T18:53:45.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.403+0000 7fe89effd700 1 -- 172.21.15.146:0/1089723164 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6295 (secure 0 0 0) 0x7fe898086070 con 0x7fe8a8158080 2024-09-13T18:53:45.419 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:45.420 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:33.990688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6714,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:45.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.431+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe88c066780 msgr2=0x7fe88c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:45.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.431+0000 7fe8b03f5700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe88c066780 0x7fe88c068c40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009850 tx=0x7fe8a0009000 comp rx=0 tx=0).stop 2024-09-13T18:53:45.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.431+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 msgr2=0x7fe8a80c2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:45.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.431+0000 7fe8b03f5700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 0x7fe8a80c2f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe89800d800 tx=0x7fe89800db10 comp rx=0 tx=0).stop 2024-09-13T18:53:45.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.431+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1089723164 shutdown_connections 2024-09-13T18:53:45.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.431+0000 7fe8b03f5700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe88c066780 0x7fe88c068c40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:45.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.431+0000 7fe8b03f5700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8a81610f0 0x7fe8a8156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:45.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.431+0000 7fe8b03f5700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8a8158080 0x7fe8a80c2f20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:45.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.431+0000 7fe8b03f5700 1 --2- 172.21.15.146:0/1089723164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8a80c6df0 0x7fe8a80c1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:45.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.431+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1089723164 >> 172.21.15.146:0/1089723164 conn(0x7fe8a80cb1d0 msgr2=0x7fe8a80caa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:45.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.431+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1089723164 shutdown_connections 2024-09-13T18:53:45.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:45.431+0000 7fe8b03f5700 1 -- 172.21.15.146:0/1089723164 wait complete. 2024-09-13T18:53:45.458 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 106 2024-09-13T18:53:45.473 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:53:46.772 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:53:46.775 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:53:47.473 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:47.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.863+0000 7f21c9252700 1 Processor -- start 2024-09-13T18:53:47.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.863+0000 7f21c9252700 1 -- start start 2024-09-13T18:53:47.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.867+0000 7f21c9252700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f21c40c9500 0x7f21c414f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:47.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.867+0000 7f21c9252700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21c40cacd0 0x7f21c40cb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:47.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.867+0000 7f21c259c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f21c40a9490 0x7f21c4155c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:44670/0 (socket says 172.21.15.146:44670) 2024-09-13T18:53:47.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.867+0000 7f21c259c700 1 -- 172.21.15.146:0/3398870932 learned_addr learned my addr 172.21.15.146:0/3398870932 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:53:47.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.867+0000 7f21c359e700 1 --2- 172.21.15.146:0/3398870932 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21c40cacd0 0x7f21c40cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:47.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.867+0000 7f21c9252700 1 -- 172.21.15.146:0/3398870932 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c40c98e0 con 0x7f21c40a9490 2024-09-13T18:53:47.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.867+0000 7f21c9252700 1 -- 172.21.15.146:0/3398870932 --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c406f5c0 con 0x7f21c40c9500 2024-09-13T18:53:47.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.867+0000 7f21c9252700 1 -- 172.21.15.146:0/3398870932 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c4155420 con 0x7f21c40cacd0 2024-09-13T18:53:47.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.871+0000 7f21c359e700 1 -- 172.21.15.146:0/3398870932 >> v2:172.21.15.146:3300/0 conn(0x7f21c40c9500 msgr2=0x7f21c414f630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:53:47.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.879+0000 7f21c1d9b700 1 -- 172.21.15.146:0/3398870932 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141961463 0 0) 0x7f21c40c98e0 con 0x7f21c40a9490 2024-09-13T18:53:47.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.883+0000 7f21c2d9d700 1 --2- 172.21.15.146:0/3398870932 >> v2:172.21.15.146:3300/0 conn(0x7f21c40c9500 0x7f21c414f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:47.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.883+0000 7f21c359e700 1 --2- 172.21.15.146:0/3398870932 >> v2:172.21.15.146:3300/0 conn(0x7f21c40c9500 0x7f21c414f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:47.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.883+0000 7f21c359e700 1 -- 172.21.15.146:0/3398870932 >> v1:172.21.15.154:6789/0 conn(0x7f21c40a9490 legacy=0x7f21c4155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:47.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.883+0000 7f21c359e700 1 -- 172.21.15.146:0/3398870932 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c414fb70 con 0x7f21c40cacd0 2024-09-13T18:53:47.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.887+0000 7f21c359e700 1 --2- 172.21.15.146:0/3398870932 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21c40cacd0 0x7f21c40cb160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009850 tx=0x7f21b8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fabf4bc35f14c45a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:47.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.887+0000 7f21c2d9d700 1 --2- 172.21.15.146:0/3398870932 >> v2:172.21.15.146:3300/0 conn(0x7f21c40c9500 0x7f21c414f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:53:47.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.887+0000 7f21c1d9b700 1 -- 172.21.15.146:0/3398870932 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8011830 con 0x7f21c40cacd0 2024-09-13T18:53:47.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.887+0000 7f21c1d9b700 1 -- 172.21.15.146:0/3398870932 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8011990 con 0x7f21c40cacd0 2024-09-13T18:53:47.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.887+0000 7f21c9252700 1 -- 172.21.15.146:0/3398870932 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21c40cacd0 msgr2=0x7f21c40cb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:47.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.887+0000 7f21c9252700 1 --2- 172.21.15.146:0/3398870932 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21c40cacd0 0x7f21c40cb160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009850 tx=0x7f21b8009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:47.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.895+0000 7f21c9252700 1 -- 172.21.15.146:0/3398870932 shutdown_connections 2024-09-13T18:53:47.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.899+0000 7f21c9252700 1 --2- 172.21.15.146:0/3398870932 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21c40cacd0 0x7f21c40cb160 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:47.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.899+0000 7f21c9252700 1 --2- 172.21.15.146:0/3398870932 >> v2:172.21.15.146:3300/0 conn(0x7f21c40c9500 0x7f21c414f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:47.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.899+0000 7f21c9252700 1 -- 172.21.15.146:0/3398870932 >> 172.21.15.146:0/3398870932 conn(0x7f21c4066090 msgr2=0x7f21c4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:47.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.903+0000 7f21c9252700 1 -- 172.21.15.146:0/3398870932 shutdown_connections 2024-09-13T18:53:47.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.907+0000 7f21c9252700 1 -- 172.21.15.146:0/3398870932 wait complete. 2024-09-13T18:53:47.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.907+0000 7f21c9252700 1 Processor -- start 2024-09-13T18:53:47.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.907+0000 7f21c9252700 1 -- start start 2024-09-13T18:53:47.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c9252700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21c40a9490 0x7f21c40c1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:47.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c9252700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21c40c9500 0x7f21c41465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:47.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c9252700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21c4153fd0 0x7f21c4146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:47.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c9252700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40ab380 con 0x7f21c4153fd0 2024-09-13T18:53:47.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c9252700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c414efc0 con 0x7f21c40a9490 2024-09-13T18:53:47.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c9252700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40af370 con 0x7f21c40c9500 2024-09-13T18:53:47.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c2d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21c40a9490 0x7f21c40c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:47.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c2d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21c40a9490 0x7f21c40c1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:52408/0 (socket says 172.21.15.146:52408) 2024-09-13T18:53:47.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c2d9d700 1 -- 172.21.15.146:0/299704837 learned_addr learned my addr 172.21.15.146:0/299704837 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:47.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c259c700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21c40c9500 0x7f21c41465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:47.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c359e700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21c4153fd0 0x7f21c4146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:47.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c259c700 1 -- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21c40a9490 msgr2=0x7f21c40c1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:47.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c259c700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21c40a9490 0x7f21c40c1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:47.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c259c700 1 -- 172.21.15.146:0/299704837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21c4153fd0 msgr2=0x7f21c4146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:47.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c259c700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21c4153fd0 0x7f21c4146b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:47.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c259c700 1 -- 172.21.15.146:0/299704837 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b80094b0 con 0x7f21c40c9500 2024-09-13T18:53:47.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c359e700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21c4153fd0 0x7f21c4146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:47.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.911+0000 7f21c259c700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21c40c9500 0x7f21c41465c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00cc00 tx=0x7f21bc00cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:47.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.915+0000 7f21ab7fe700 1 -- 172.21.15.146:0/299704837 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc00aa20 con 0x7f21c40c9500 2024-09-13T18:53:47.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.915+0000 7f21ab7fe700 1 -- 172.21.15.146:0/299704837 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc00aba0 con 0x7f21c40c9500 2024-09-13T18:53:47.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.915+0000 7f21c9252700 1 -- 172.21.15.146:0/299704837 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c40af650 con 0x7f21c40c9500 2024-09-13T18:53:47.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.915+0000 7f21c9252700 1 -- 172.21.15.146:0/299704837 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c40c23b0 con 0x7f21c40c9500 2024-09-13T18:53:47.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.915+0000 7f21ab7fe700 1 -- 172.21.15.146:0/299704837 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc0070d0 con 0x7f21c40c9500 2024-09-13T18:53:47.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.919+0000 7f21c9252700 1 -- 172.21.15.146:0/299704837 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a4005310 con 0x7f21c40c9500 2024-09-13T18:53:47.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.919+0000 7f21ab7fe700 1 -- 172.21.15.146:0/299704837 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f21bc007230 con 0x7f21c40c9500 2024-09-13T18:53:47.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.923+0000 7f21ab7fe700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f21b0066830 0x7f21b0068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:47.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.923+0000 7f21ab7fe700 1 -- 172.21.15.146:0/299704837 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7867+0+0 (secure 0 0 0) 0x7f21bc084820 con 0x7f21c40c9500 2024-09-13T18:53:47.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.923+0000 7f21c2d9d700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f21b0066830 0x7f21b0068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:47.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.923+0000 7f21c2d9d700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f21b0066830 0x7f21b0068cf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f21b4006fd0 tx=0x7f21b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:47.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:47.931+0000 7f21ab7fe700 1 -- 172.21.15.146:0/299704837 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f21bc051300 con 0x7f21c40c9500 2024-09-13T18:53:48.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.687+0000 7f21c9252700 1 -- 172.21.15.146:0/299704837 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21a4005b40 con 0x7f21c40c9500 2024-09-13T18:53:48.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.687+0000 7f21ab7fe700 1 -- 172.21.15.146:0/299704837 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6295 (secure 0 0 0) 0x7f21bc018020 con 0x7f21c40c9500 2024-09-13T18:53:48.709 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:48.709 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6032,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:33.990688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6714,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6714":{"gid":6714,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":38,"addr":"172.21.15.146:6837/2169954140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2169954140},{"type":"v1","addr":"172.21.15.146:6837","nonce":2169954140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:48.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.699+0000 7f21a97fa700 1 -- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f21b0066830 msgr2=0x7f21b0068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:48.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.699+0000 7f21a97fa700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f21b0066830 0x7f21b0068cf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f21b4006fd0 tx=0x7f21b4008040 comp rx=0 tx=0).stop 2024-09-13T18:53:48.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.699+0000 7f21a97fa700 1 -- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21c40c9500 msgr2=0x7f21c41465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:48.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.699+0000 7f21a97fa700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21c40c9500 0x7f21c41465c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00cc00 tx=0x7f21bc00cf10 comp rx=0 tx=0).stop 2024-09-13T18:53:48.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.699+0000 7f21a97fa700 1 -- 172.21.15.146:0/299704837 shutdown_connections 2024-09-13T18:53:48.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.699+0000 7f21a97fa700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f21b0066830 0x7f21b0068cf0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:48.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.699+0000 7f21a97fa700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21c4153fd0 0x7f21c4146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:48.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.699+0000 7f21a97fa700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21c40c9500 0x7f21c41465c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:48.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.699+0000 7f21a97fa700 1 --2- 172.21.15.146:0/299704837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21c40a9490 0x7f21c40c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:48.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.699+0000 7f21a97fa700 1 -- 172.21.15.146:0/299704837 >> 172.21.15.146:0/299704837 conn(0x7f21c4066090 msgr2=0x7f21c40c5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:48.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.699+0000 7f21a97fa700 1 -- 172.21.15.146:0/299704837 shutdown_connections 2024-09-13T18:53:48.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:48.699+0000 7f21a97fa700 1 -- 172.21.15.146:0/299704837 wait complete. 2024-09-13T18:53:48.720 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 106 2024-09-13T18:53:48.777 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T18:53:50.778 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:51.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2f3f0700 1 Processor -- start 2024-09-13T18:53:51.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2f3f0700 1 -- start start 2024-09-13T18:53:51.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2f3f0700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c280caed0 0x7f5c28072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2e3ee700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c280caed0 0x7f5c28072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2e3ee700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c280caed0 0x7f5c28072380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58502/0 (socket says 172.21.15.146:58502) 2024-09-13T18:53:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2e3ee700 1 -- 172.21.15.146:0/2830250616 learned_addr learned my addr 172.21.15.146:0/2830250616 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2f3f0700 1 --2- 172.21.15.146:0/2830250616 >> v2:172.21.15.154:3300/0 conn(0x7f5c280cb2b0 0x7f5c280c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2f3f0700 1 --2- 172.21.15.146:0/2830250616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c280c8230 0x7f5c2815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/2830250616 --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c280c78d0 con 0x7f5c280caed0 2024-09-13T18:53:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/2830250616 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c28161060 con 0x7f5c280cb2b0 2024-09-13T18:53:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/2830250616 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28161300 con 0x7f5c280c8230 2024-09-13T18:53:51.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2dbed700 1 --2- 172.21.15.146:0/2830250616 >> v2:172.21.15.154:3300/0 conn(0x7f5c280cb2b0 0x7f5c280c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2ebef700 1 --2- 172.21.15.146:0/2830250616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c280c8230 0x7f5c2815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2e3ee700 1 -- 172.21.15.146:0/2830250616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c280c8230 msgr2=0x7f5c2815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:51.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2e3ee700 1 --2- 172.21.15.146:0/2830250616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c280c8230 0x7f5c2815e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2e3ee700 1 -- 172.21.15.146:0/2830250616 >> v2:172.21.15.154:3300/0 conn(0x7f5c280cb2b0 msgr2=0x7f5c280c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:51.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2e3ee700 1 --2- 172.21.15.146:0/2830250616 >> v2:172.21.15.154:3300/0 conn(0x7f5c280cb2b0 0x7f5c280c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2e3ee700 1 -- 172.21.15.146:0/2830250616 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c281615c0 con 0x7f5c280caed0 2024-09-13T18:53:51.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2ebef700 1 --2- 172.21.15.146:0/2830250616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c280c8230 0x7f5c2815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:53:51.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.127+0000 7f5c2e3ee700 1 --2- 172.21.15.146:0/2830250616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c280caed0 0x7f5c28072380 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009910 tx=0x7f5c24009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d3b7085bdc98f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:51.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.139+0000 7f5c2d3ec700 1 -- 172.21.15.146:0/2830250616 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2400ba20 con 0x7f5c280caed0 2024-09-13T18:53:51.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.139+0000 7f5c2d3ec700 1 -- 172.21.15.146:0/2830250616 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2400bb80 con 0x7f5c280caed0 2024-09-13T18:53:51.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.139+0000 7f5c2d3ec700 1 -- 172.21.15.146:0/2830250616 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24009e60 con 0x7f5c280caed0 2024-09-13T18:53:51.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.139+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/2830250616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c280caed0 msgr2=0x7f5c28072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.139+0000 7f5c2f3f0700 1 --2- 172.21.15.146:0/2830250616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c280caed0 0x7f5c28072380 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009910 tx=0x7f5c24009c20 comp rx=0 tx=0).stop 2024-09-13T18:53:51.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.139+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/2830250616 shutdown_connections 2024-09-13T18:53:51.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.139+0000 7f5c2f3f0700 1 --2- 172.21.15.146:0/2830250616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c280c8230 0x7f5c2815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.139+0000 7f5c2f3f0700 1 --2- 172.21.15.146:0/2830250616 >> v2:172.21.15.154:3300/0 conn(0x7f5c280cb2b0 0x7f5c280c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.139+0000 7f5c2f3f0700 1 --2- 172.21.15.146:0/2830250616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c280caed0 0x7f5c28072380 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.139+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/2830250616 >> 172.21.15.146:0/2830250616 conn(0x7f5c28066090 msgr2=0x7f5c28066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:51.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.139+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/2830250616 shutdown_connections 2024-09-13T18:53:51.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.139+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/2830250616 wait complete. 2024-09-13T18:53:51.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.143+0000 7f5c2f3f0700 1 Processor -- start 2024-09-13T18:53:51.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2f3f0700 1 -- start start 2024-09-13T18:53:51.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2f3f0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c280c8230 0x7f5c280bfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2f3f0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c280caed0 0x7f5c28159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2e3ee700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c280c8230 0x7f5c280bfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2e3ee700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c280c8230 0x7f5c280bfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58512/0 (socket says 172.21.15.146:58512) 2024-09-13T18:53:51.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2e3ee700 1 -- 172.21.15.146:0/180733563 learned_addr learned my addr 172.21.15.146:0/180733563 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:51.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2f3f0700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c280cb2b0 0x7f5c281f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/180733563 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c281f3a70 con 0x7f5c280c8230 2024-09-13T18:53:51.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/180733563 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280c1ce0 con 0x7f5c280cb2b0 2024-09-13T18:53:51.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/180733563 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c281f3130 con 0x7f5c280caed0 2024-09-13T18:53:51.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2dbed700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c280caed0 0x7f5c28159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2ebef700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c280cb2b0 0x7f5c281f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2e3ee700 1 -- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c280caed0 msgr2=0x7f5c28159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2e3ee700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c280caed0 0x7f5c28159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2e3ee700 1 -- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c280cb2b0 msgr2=0x7f5c281f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2e3ee700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c280cb2b0 0x7f5c281f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.151+0000 7f5c2e3ee700 1 -- 172.21.15.146:0/180733563 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c24009570 con 0x7f5c280c8230 2024-09-13T18:53:51.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.155+0000 7f5c2e3ee700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c280c8230 0x7f5c280bfcf0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5c280cac30 tx=0x7f5c24004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:51.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.155+0000 7f5c1effd700 1 -- 172.21.15.146:0/180733563 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24004510 con 0x7f5c280c8230 2024-09-13T18:53:51.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.155+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/180733563 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2806cd40 con 0x7f5c280c8230 2024-09-13T18:53:51.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.155+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/180733563 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c280c8e90 con 0x7f5c280c8230 2024-09-13T18:53:51.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.159+0000 7f5c1effd700 1 -- 172.21.15.146:0/180733563 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c240036c0 con 0x7f5c280c8230 2024-09-13T18:53:51.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.159+0000 7f5c1effd700 1 -- 172.21.15.146:0/180733563 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24003a70 con 0x7f5c280c8230 2024-09-13T18:53:51.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.159+0000 7f5c1effd700 1 -- 172.21.15.146:0/180733563 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5c24003bd0 con 0x7f5c280c8230 2024-09-13T18:53:51.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.159+0000 7f5c1effd700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c0c066880 0x7f5c0c068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.159+0000 7f5c1effd700 1 -- 172.21.15.146:0/180733563 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7f5c24085d10 con 0x7f5c280c8230 2024-09-13T18:53:51.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.159+0000 7f5c2dbed700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c0c066880 0x7f5c0c068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.159+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/180733563 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c10005310 con 0x7f5c280c8230 2024-09-13T18:53:51.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.163+0000 7f5c2dbed700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c0c066880 0x7f5c0c068d40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5c280aa840 tx=0x7f5c20003a40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:51.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.171+0000 7f5c1effd700 1 -- 172.21.15.146:0/180733563 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5c24052790 con 0x7f5c280c8230 2024-09-13T18:53:51.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.839+0000 7f5c2f3f0700 1 -- 172.21.15.146:0/180733563 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c10005b40 con 0x7f5c280c8230 2024-09-13T18:53:51.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.843+0000 7f5c1effd700 1 -- 172.21.15.146:0/180733563 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5535 (secure 0 0 0) 0x7f5c24028090 con 0x7f5c280c8230 2024-09-13T18:53:51.858 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:51.858 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:50.924878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:51.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.847+0000 7f5c1cff9700 1 -- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c0c066880 msgr2=0x7f5c0c068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.847+0000 7f5c1cff9700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c0c066880 0x7f5c0c068d40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5c280aa840 tx=0x7f5c20003a40 comp rx=0 tx=0).stop 2024-09-13T18:53:51.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.847+0000 7f5c1cff9700 1 -- 172.21.15.146:0/180733563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c280c8230 msgr2=0x7f5c280bfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.847+0000 7f5c1cff9700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c280c8230 0x7f5c280bfcf0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5c280cac30 tx=0x7f5c24004770 comp rx=0 tx=0).stop 2024-09-13T18:53:51.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.847+0000 7f5c1cff9700 1 -- 172.21.15.146:0/180733563 shutdown_connections 2024-09-13T18:53:51.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.847+0000 7f5c1cff9700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5c0c066880 0x7f5c0c068d40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.847+0000 7f5c1cff9700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c280cb2b0 0x7f5c281f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.847+0000 7f5c1cff9700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c280caed0 0x7f5c28159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.847+0000 7f5c1cff9700 1 --2- 172.21.15.146:0/180733563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c280c8230 0x7f5c280bfcf0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.847+0000 7f5c1cff9700 1 -- 172.21.15.146:0/180733563 >> 172.21.15.146:0/180733563 conn(0x7f5c28066090 msgr2=0x7f5c280700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:51.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.847+0000 7f5c1cff9700 1 -- 172.21.15.146:0/180733563 shutdown_connections 2024-09-13T18:53:51.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:51.847+0000 7f5c1cff9700 1 -- 172.21.15.146:0/180733563 wait complete. 2024-09-13T18:53:51.866 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 107 2024-09-13T18:53:51.898 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-09-13T18:53:51.898 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:53:51.898 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:52.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.351+0000 7fced6fb4700 1 Processor -- start 2024-09-13T18:53:52.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.355+0000 7fced6fb4700 1 -- start start 2024-09-13T18:53:52.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.355+0000 7fced6fb4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fced00cac60 0x7fced00cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.355+0000 7fced6fb4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fced016e0c0 0x7fced01704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.355+0000 7fced6fb4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced00a9920 con 0x7fced008baa0 2024-09-13T18:53:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.355+0000 7fced6fb4700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced00723f0 con 0x7fced016e0c0 2024-09-13T18:53:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.355+0000 7fced6fb4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced00cb580 con 0x7fced00cac60 2024-09-13T18:53:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.355+0000 7fced67b3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fced016e0c0 0x7fced01704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.355+0000 7fced67b3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fced016e0c0 0x7fced01704b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58520/0 (socket says 172.21.15.146:58520) 2024-09-13T18:53:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.355+0000 7fced67b3700 1 -- 172.21.15.146:0/2278478951 learned_addr learned my addr 172.21.15.146:0/2278478951 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.363+0000 7fced67b3700 1 -- 172.21.15.146:0/2278478951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fced00cac60 msgr2=0x7fced00cb040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:53:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.363+0000 7fced67b3700 1 --2- 172.21.15.146:0/2278478951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fced00cac60 0x7fced00cb040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.363+0000 7fced67b3700 1 -- 172.21.15.146:0/2278478951 >> v1:172.21.15.154:6789/0 conn(0x7fced008baa0 legacy=0x7fced006f960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:53:52.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.367+0000 7fced67b3700 1 -- 172.21.15.146:0/2278478951 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced0070070 con 0x7fced016e0c0 2024-09-13T18:53:52.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.367+0000 7fced67b3700 1 --2- 172.21.15.146:0/2278478951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fced016e0c0 0x7fced01704b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcec00089c0 tx=0x7fcec0008cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cab5b225939aa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:52.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.367+0000 7fced4fb0700 1 -- 172.21.15.146:0/2278478951 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec0004230 con 0x7fced016e0c0 2024-09-13T18:53:52.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.367+0000 7fced4fb0700 1 -- 172.21.15.146:0/2278478951 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec0004390 con 0x7fced016e0c0 2024-09-13T18:53:52.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.367+0000 7fced4fb0700 1 -- 172.21.15.146:0/2278478951 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec00126d0 con 0x7fced016e0c0 2024-09-13T18:53:52.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.367+0000 7fced6fb4700 1 -- 172.21.15.146:0/2278478951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fced016e0c0 msgr2=0x7fced01704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.367+0000 7fced6fb4700 1 --2- 172.21.15.146:0/2278478951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fced016e0c0 0x7fced01704b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcec00089c0 tx=0x7fcec0008cd0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.367+0000 7fced6fb4700 1 -- 172.21.15.146:0/2278478951 shutdown_connections 2024-09-13T18:53:52.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.367+0000 7fced6fb4700 1 --2- 172.21.15.146:0/2278478951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fced016e0c0 0x7fced01704b0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.367+0000 7fced6fb4700 1 --2- 172.21.15.146:0/2278478951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fced00cac60 0x7fced00cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.367+0000 7fced6fb4700 1 -- 172.21.15.146:0/2278478951 >> 172.21.15.146:0/2278478951 conn(0x7fced0066090 msgr2=0x7fced0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:52.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.367+0000 7fced6fb4700 1 -- 172.21.15.146:0/2278478951 shutdown_connections 2024-09-13T18:53:52.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.383+0000 7fced6fb4700 1 -- 172.21.15.146:0/2278478951 wait complete. 2024-09-13T18:53:52.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.387+0000 7fced6fb4700 1 Processor -- start 2024-09-13T18:53:52.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced6fb4700 1 -- start start 2024-09-13T18:53:52.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced6fb4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fced008baa0 0x7fced00ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced6fb4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fced00cac60 0x7fced00aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced6fb4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fced016e0c0 0x7fced020d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced6fb4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced00afa10 con 0x7fced016e0c0 2024-09-13T18:53:52.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced6fb4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced00ad540 con 0x7fced00cac60 2024-09-13T18:53:52.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced6fb4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced00b01e0 con 0x7fced008baa0 2024-09-13T18:53:52.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced67b3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fced016e0c0 0x7fced020d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced67b3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fced016e0c0 0x7fced020d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58522/0 (socket says 172.21.15.146:58522) 2024-09-13T18:53:52.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced67b3700 1 -- 172.21.15.146:0/802612739 learned_addr learned my addr 172.21.15.146:0/802612739 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:52.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced67b3700 1 -- 172.21.15.146:0/802612739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fced008baa0 msgr2=0x7fced00ad000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:53:52.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced67b3700 1 --2- 172.21.15.146:0/802612739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fced008baa0 0x7fced00ad000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced67b3700 1 -- 172.21.15.146:0/802612739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fced00cac60 msgr2=0x7fced00aac00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:53:52.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced67b3700 1 --2- 172.21.15.146:0/802612739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fced00cac60 0x7fced00aac00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced67b3700 1 -- 172.21.15.146:0/802612739 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec00085f0 con 0x7fced016e0c0 2024-09-13T18:53:52.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.399+0000 7fced67b3700 1 --2- 172.21.15.146:0/802612739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fced016e0c0 0x7fced020d070 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fced00cac30 tx=0x7fcec0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:52.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.403+0000 7fcec67fc700 1 -- 172.21.15.146:0/802612739 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec000b860 con 0x7fced016e0c0 2024-09-13T18:53:52.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.403+0000 7fced6fb4700 1 -- 172.21.15.146:0/802612739 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced020cbe0 con 0x7fced016e0c0 2024-09-13T18:53:52.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.407+0000 7fced6fb4700 1 -- 172.21.15.146:0/802612739 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced00afc20 con 0x7fced016e0c0 2024-09-13T18:53:52.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.407+0000 7fcec67fc700 1 -- 172.21.15.146:0/802612739 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec000b9c0 con 0x7fced016e0c0 2024-09-13T18:53:52.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.415+0000 7fcec67fc700 1 -- 172.21.15.146:0/802612739 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec000bd30 con 0x7fced016e0c0 2024-09-13T18:53:52.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.415+0000 7fcec67fc700 1 -- 172.21.15.146:0/802612739 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcec001b3f0 con 0x7fced016e0c0 2024-09-13T18:53:52.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.419+0000 7fcec67fc700 1 --2- 172.21.15.146:0/802612739 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fceb4066880 0x7fceb4068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.419+0000 7fced5fb2700 1 --2- 172.21.15.146:0/802612739 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fceb4066880 0x7fceb4068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.419+0000 7fcec67fc700 1 -- 172.21.15.146:0/802612739 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7fcec0086850 con 0x7fced016e0c0 2024-09-13T18:53:52.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.423+0000 7fced5fb2700 1 --2- 172.21.15.146:0/802612739 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fceb4066880 0x7fceb4068d40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009730 tx=0x7fcecc006c80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:52.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.427+0000 7fcebbfff700 1 -- 172.21.15.146:0/802612739 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb0005310 con 0x7fced016e0c0 2024-09-13T18:53:52.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:52.435+0000 7fcec67fc700 1 -- 172.21.15.146:0/802612739 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcec0053240 con 0x7fced016e0c0 2024-09-13T18:53:53.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.179+0000 7fcebbfff700 1 -- 172.21.15.146:0/802612739 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fceb0005b40 con 0x7fced016e0c0 2024-09-13T18:53:53.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.179+0000 7fcec67fc700 1 -- 172.21.15.146:0/802612739 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5535 (secure 0 0 0) 0x7fcec0056860 con 0x7fced016e0c0 2024-09-13T18:53:53.198 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:53.198 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:50.924878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:53.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.195+0000 7fcebbfff700 1 -- 172.21.15.146:0/802612739 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fceb4066880 msgr2=0x7fceb4068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.195+0000 7fcebbfff700 1 --2- 172.21.15.146:0/802612739 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fceb4066880 0x7fceb4068d40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009730 tx=0x7fcecc006c80 comp rx=0 tx=0).stop 2024-09-13T18:53:53.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.195+0000 7fcebbfff700 1 -- 172.21.15.146:0/802612739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fced016e0c0 msgr2=0x7fced020d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.195+0000 7fcebbfff700 1 --2- 172.21.15.146:0/802612739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fced016e0c0 0x7fced020d070 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fced00cac30 tx=0x7fcec0004770 comp rx=0 tx=0).stop 2024-09-13T18:53:53.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.195+0000 7fcebbfff700 1 -- 172.21.15.146:0/802612739 shutdown_connections 2024-09-13T18:53:53.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.195+0000 7fcebbfff700 1 --2- 172.21.15.146:0/802612739 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fceb4066880 0x7fceb4068d40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.195+0000 7fcebbfff700 1 --2- 172.21.15.146:0/802612739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fced016e0c0 0x7fced020d070 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.195+0000 7fcebbfff700 1 --2- 172.21.15.146:0/802612739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fced00cac60 0x7fced00aac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.195+0000 7fcebbfff700 1 --2- 172.21.15.146:0/802612739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fced008baa0 0x7fced00ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.195+0000 7fcebbfff700 1 -- 172.21.15.146:0/802612739 >> 172.21.15.146:0/802612739 conn(0x7fced0066090 msgr2=0x7fced006f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:53.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.195+0000 7fcebbfff700 1 -- 172.21.15.146:0/802612739 shutdown_connections 2024-09-13T18:53:53.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:53.195+0000 7fcebbfff700 1 -- 172.21.15.146:0/802612739 wait complete. 2024-09-13T18:53:53.216 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 107 2024-09-13T18:53:53.259 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 107, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 107, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6032": { "addr": "172.21.15.154:6839/2053734262", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 2053734262, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2053734262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6032, "incarnation": 107, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:replay", "state_seq": 1 }, "gid_6984": { "addr": "172.21.15.146:6835/2934322065", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 2934322065, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 2934322065, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6984, "incarnation": 98, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 28 }, "gid_7098": { "addr": "172.21.15.146:6833/354253522", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 354253522, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 354253522, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7098, "incarnation": 105, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 42, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:53:50.924878+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6984, "mds_1": 6032, "mds_2": 7098 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:53:55.260 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:55.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.727+0000 7fb4eb35f700 1 Processor -- start 2024-09-13T18:53:55.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.727+0000 7fb4eb35f700 1 -- start start 2024-09-13T18:53:55.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.731+0000 7fb4eb35f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4e408baa0 0x7fb4e40cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.731+0000 7fb4eb35f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e40a9920 con 0x7fb4e40cac60 2024-09-13T18:53:55.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.731+0000 7fb4eb35f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e4070070 con 0x7fb4e416e0c0 2024-09-13T18:53:55.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.731+0000 7fb4eb35f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40723f0 con 0x7fb4e408baa0 2024-09-13T18:53:55.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.731+0000 7fb4ea35d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb4e40cac60 0x7fb4e406f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43370/0 (socket says 172.21.15.146:43370) 2024-09-13T18:53:55.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.731+0000 7fb4ea35d700 1 -- 172.21.15.146:0/3245752489 learned_addr learned my addr 172.21.15.146:0/3245752489 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:53:55.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.731+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000334264 0 0) 0x7fb4e40a9920 con 0x7fb4e40cac60 2024-09-13T18:53:55.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.731+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4d00035b0 con 0x7fb4e40cac60 2024-09-13T18:53:55.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.731+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131926320 0 0) 0x7fb4e4070070 con 0x7fb4e416e0c0 2024-09-13T18:53:55.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.731+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4e40a9920 con 0x7fb4e416e0c0 2024-09-13T18:53:55.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.731+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3328952062 0 0) 0x7fb4d00035b0 con 0x7fb4e40cac60 2024-09-13T18:53:55.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.731+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4e408baa0 msgr2=0x7fb4e40cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:55.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e9b5c700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4e408baa0 0x7fb4e40cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e935b700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4e408baa0 0x7fb4e40cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 >> v1:172.21.15.154:6789/0 conn(0x7fb4e416e0c0 legacy=0x7fb4e41704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e40725e0 con 0x7fb4e40cac60 2024-09-13T18:53:55.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7fb4e00015f0 con 0x7fb4e40cac60 2024-09-13T18:53:55.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 >> v1:172.21.15.146:6789/0 conn(0x7fb4e40cac60 legacy=0x7fb4e406f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e935b700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4d0004ec0 0x7fb4d0003940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e935b700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4d0003f90 0x7fb4d0004410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e935b700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4d000d960 0x7fb4d000fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d40010e0 con 0x7fb4d000d960 2024-09-13T18:53:55.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0009500 con 0x7fb4d0004ec0 2024-09-13T18:53:55.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00096c0 con 0x7fb4d0003f90 2024-09-13T18:53:55.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e9b5c700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4e408baa0 0x7fb4e40cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:53:55.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4eab5e700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4d000d960 0x7fb4d000fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4ea35d700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4d0004ec0 0x7fb4d0003940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e9b5c700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4d0003f90 0x7fb4d0004410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4eab5e700 1 -- 172.21.15.146:0/3245752489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4d0003f90 msgr2=0x7fb4d0004410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:55.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4eab5e700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4d0003f90 0x7fb4d0004410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4eab5e700 1 -- 172.21.15.146:0/3245752489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4d0004ec0 msgr2=0x7fb4d0003940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:55.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4eab5e700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4d0004ec0 0x7fb4d0003940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4eab5e700 1 -- 172.21.15.146:0/3245752489 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d00035b0 con 0x7fb4d000d960 2024-09-13T18:53:55.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.735+0000 7fb4e9b5c700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4d0003f90 0x7fb4d0004410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:53:55.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.739+0000 7fb4eab5e700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4d000d960 0x7fb4d000fdf0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb4e406c370 tx=0x7fb4d0003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1648e585be86433f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:55.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.739+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4007930 con 0x7fb4d000d960 2024-09-13T18:53:55.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.739+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d400cd00 con 0x7fb4d000d960 2024-09-13T18:53:55.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.739+0000 7fb4e935b700 1 -- 172.21.15.146:0/3245752489 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d40052f0 con 0x7fb4d000d960 2024-09-13T18:53:55.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.739+0000 7fb4eb35f700 1 -- 172.21.15.146:0/3245752489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4d000d960 msgr2=0x7fb4d000fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.739+0000 7fb4eb35f700 1 --2- 172.21.15.146:0/3245752489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4d000d960 0x7fb4d000fdf0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb4e406c370 tx=0x7fb4d0003860 comp rx=0 tx=0).stop 2024-09-13T18:53:55.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.739+0000 7fb4eab5e700 1 -- 172.21.15.146:0/3245752489 reap_dead start 2024-09-13T18:53:55.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.755+0000 7fb4eb35f700 1 -- 172.21.15.146:0/3245752489 shutdown_connections 2024-09-13T18:53:55.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.755+0000 7fb4eb35f700 1 -- 172.21.15.146:0/3245752489 >> 172.21.15.146:0/3245752489 conn(0x7fb4e4066090 msgr2=0x7fb4e4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:55.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.767+0000 7fb4eb35f700 1 -- 172.21.15.146:0/3245752489 shutdown_connections 2024-09-13T18:53:55.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.771+0000 7fb4eb35f700 1 -- 172.21.15.146:0/3245752489 wait complete. 2024-09-13T18:53:55.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4eb35f700 1 Processor -- start 2024-09-13T18:53:55.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4eb35f700 1 -- start start 2024-09-13T18:53:55.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4eb35f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4d000d960 0x7fb4e40aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4eb35f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4e40abc50 0x7fb4e40c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4eb35f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4e40ae010 0x7fb4e40ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4eb35f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e41f4110 con 0x7fb4e40ae010 2024-09-13T18:53:55.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4eb35f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40ae9d0 con 0x7fb4e40abc50 2024-09-13T18:53:55.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4eb35f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40ad330 con 0x7fb4d000d960 2024-09-13T18:53:55.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4ea35d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4d000d960 0x7fb4e40aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4ea35d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4d000d960 0x7fb4e40aa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:35614/0 (socket says 172.21.15.146:35614) 2024-09-13T18:53:55.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4ea35d700 1 -- 172.21.15.146:0/4186218888 learned_addr learned my addr 172.21.15.146:0/4186218888 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:55.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4e9b5c700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4e40abc50 0x7fb4e40c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4eab5e700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4e40ae010 0x7fb4e40ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4ea35d700 1 -- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4e40abc50 msgr2=0x7fb4e40c54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4ea35d700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4e40abc50 0x7fb4e40c54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4ea35d700 1 -- 172.21.15.146:0/4186218888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4e40ae010 msgr2=0x7fb4e40ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4ea35d700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4e40ae010 0x7fb4e40ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4ea35d700 1 -- 172.21.15.146:0/4186218888 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e40aae60 con 0x7fb4d000d960 2024-09-13T18:53:55.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4e9b5c700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4e40abc50 0x7fb4e40c54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:53:55.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4eab5e700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4e40ae010 0x7fb4e40ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:55.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.775+0000 7fb4ea35d700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4d000d960 0x7fb4e40aa7d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000eaf0 tx=0x7fb4e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:55.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.779+0000 7fb4daffd700 1 -- 172.21.15.146:0/4186218888 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0005e20 con 0x7fb4d000d960 2024-09-13T18:53:55.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.779+0000 7fb4eb35f700 1 -- 172.21.15.146:0/4186218888 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e40ab140 con 0x7fb4d000d960 2024-09-13T18:53:55.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.779+0000 7fb4eb35f700 1 -- 172.21.15.146:0/4186218888 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e40ad810 con 0x7fb4d000d960 2024-09-13T18:53:55.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.783+0000 7fb4daffd700 1 -- 172.21.15.146:0/4186218888 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e000c7b0 con 0x7fb4d000d960 2024-09-13T18:53:55.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.783+0000 7fb4daffd700 1 -- 172.21.15.146:0/4186218888 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000cb40 con 0x7fb4d000d960 2024-09-13T18:53:55.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.783+0000 7fb4eb35f700 1 -- 172.21.15.146:0/4186218888 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e40aa590 con 0x7fb4d000d960 2024-09-13T18:53:55.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.783+0000 7fb4daffd700 1 -- 172.21.15.146:0/4186218888 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb4e000cca0 con 0x7fb4d000d960 2024-09-13T18:53:55.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.783+0000 7fb4daffd700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb4c8066800 0x7fb4c8068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.783+0000 7fb4daffd700 1 -- 172.21.15.146:0/4186218888 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7fb4e00853a0 con 0x7fb4d000d960 2024-09-13T18:53:55.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.787+0000 7fb4e9b5c700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb4c8066800 0x7fb4c8068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.791+0000 7fb4daffd700 1 -- 172.21.15.146:0/4186218888 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb4e0051d90 con 0x7fb4d000d960 2024-09-13T18:53:55.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:55.803+0000 7fb4e9b5c700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb4c8066800 0x7fb4c8068cc0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb4e406c800 tx=0x7fb4dc008540 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:56.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.639+0000 7fb4eb35f700 1 -- 172.21.15.146:0/4186218888 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4e406cda0 con 0x7fb4d000d960 2024-09-13T18:53:56.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.639+0000 7fb4daffd700 1 -- 172.21.15.146:0/4186218888 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5535 (secure 0 0 0) 0x7fb4e00553b0 con 0x7fb4d000d960 2024-09-13T18:53:56.662 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:53:56.662 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:50.924878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:56.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.659+0000 7fb4eb35f700 1 -- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb4c8066800 msgr2=0x7fb4c8068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.659+0000 7fb4eb35f700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb4c8066800 0x7fb4c8068cc0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb4e406c800 tx=0x7fb4dc008540 comp rx=0 tx=0).stop 2024-09-13T18:53:56.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.659+0000 7fb4eb35f700 1 -- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4d000d960 msgr2=0x7fb4e40aa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.659+0000 7fb4eb35f700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4d000d960 0x7fb4e40aa7d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000eaf0 tx=0x7fb4e000eeb0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.659+0000 7fb4eb35f700 1 -- 172.21.15.146:0/4186218888 shutdown_connections 2024-09-13T18:53:56.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.659+0000 7fb4eb35f700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb4c8066800 0x7fb4c8068cc0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.659+0000 7fb4eb35f700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4e40ae010 0x7fb4e40ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.659+0000 7fb4eb35f700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4e40abc50 0x7fb4e40c54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.659+0000 7fb4eb35f700 1 --2- 172.21.15.146:0/4186218888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4d000d960 0x7fb4e40aa7d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.659+0000 7fb4eb35f700 1 -- 172.21.15.146:0/4186218888 >> 172.21.15.146:0/4186218888 conn(0x7fb4e4066090 msgr2=0x7fb4e406d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:56.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.659+0000 7fb4eb35f700 1 -- 172.21.15.146:0/4186218888 shutdown_connections 2024-09-13T18:53:56.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:56.659+0000 7fb4eb35f700 1 -- 172.21.15.146:0/4186218888 wait complete. 2024-09-13T18:53:56.692 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 107 2024-09-13T18:53:56.713 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:53:58.715 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:59.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.271+0000 7f307271c700 1 Processor -- start 2024-09-13T18:53:59.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.271+0000 7f307271c700 1 -- start start 2024-09-13T18:53:59.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.271+0000 7f307271c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f306c0cac60 0x7f306c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.271+0000 7f307271c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c0a9920 con 0x7f306c16e0c0 2024-09-13T18:53:59.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.271+0000 7f307271c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c070070 con 0x7f306c08baa0 2024-09-13T18:53:59.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.271+0000 7f307271c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306c0723f0 con 0x7f306c0cac60 2024-09-13T18:53:59.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f3070cb9700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f306c16e0c0 0x7f306c1704b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43372/0 (socket says 172.21.15.146:43372) 2024-09-13T18:53:59.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f3070cb9700 1 -- 172.21.15.146:0/3659329247 learned_addr learned my addr 172.21.15.146:0/3659329247 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:53:59.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 -- 172.21.15.146:0/3659329247 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361598814 0 0) 0x7f306c0a9920 con 0x7f306c16e0c0 2024-09-13T18:53:59.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 -- 172.21.15.146:0/3659329247 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30540035b0 con 0x7f306c16e0c0 2024-09-13T18:53:59.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 -- 172.21.15.146:0/3659329247 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2675460303 0 0) 0x7f30540035b0 con 0x7f306c16e0c0 2024-09-13T18:53:59.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 -- 172.21.15.146:0/3659329247 >> v2:172.21.15.154:3300/0 conn(0x7f306c0cac60 msgr2=0x7f306c0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:59.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 --2- 172.21.15.146:0/3659329247 >> v2:172.21.15.154:3300/0 conn(0x7f306c0cac60 0x7f306c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 -- 172.21.15.146:0/3659329247 >> v1:172.21.15.154:6789/0 conn(0x7f306c08baa0 legacy=0x7f306c06f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:59.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 -- 172.21.15.146:0/3659329247 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c0725e0 con 0x7f306c16e0c0 2024-09-13T18:53:59.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 -- 172.21.15.146:0/3659329247 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f3060001670 con 0x7f306c16e0c0 2024-09-13T18:53:59.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 -- 172.21.15.146:0/3659329247 >> v1:172.21.15.146:6789/0 conn(0x7f306c16e0c0 legacy=0x7f306c1704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:59.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 --2- 172.21.15.146:0/3659329247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3054004cd0 0x7f3054003940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 --2- 172.21.15.146:0/3659329247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3054003fa0 0x7f305400b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 --2- 172.21.15.146:0/3659329247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305400ba50 0x7f305400de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306bfff700 1 --2- 172.21.15.146:0/3659329247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3054004cd0 0x7f3054003940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306b7fe700 1 --2- 172.21.15.146:0/3659329247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3054003fa0 0x7f305400b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f3070cb9700 1 --2- 172.21.15.146:0/3659329247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305400ba50 0x7f305400de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 -- 172.21.15.146:0/3659329247 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305400e4c0 con 0x7f305400ba50 2024-09-13T18:53:59.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 -- 172.21.15.146:0/3659329247 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305400e630 con 0x7f3054003fa0 2024-09-13T18:53:59.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f306affd700 1 -- 172.21.15.146:0/3659329247 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305400e7f0 con 0x7f3054004cd0 2024-09-13T18:53:59.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f3070cb9700 1 -- 172.21.15.146:0/3659329247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3054004cd0 msgr2=0x7f3054003940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:59.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f3070cb9700 1 --2- 172.21.15.146:0/3659329247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3054004cd0 0x7f3054003940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f3070cb9700 1 -- 172.21.15.146:0/3659329247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3054003fa0 msgr2=0x7f305400b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:59.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f3070cb9700 1 --2- 172.21.15.146:0/3659329247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3054003fa0 0x7f305400b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.275+0000 7f3070cb9700 1 -- 172.21.15.146:0/3659329247 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30540035b0 con 0x7f305400ba50 2024-09-13T18:53:59.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.279+0000 7f3070cb9700 1 --2- 172.21.15.146:0/3659329247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305400ba50 0x7f305400de40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f306c070b80 tx=0x7f3054003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38ac037a67b6005 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:59.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.279+0000 7f3070cb9700 1 -- 172.21.15.146:0/3659329247 reap_dead start 2024-09-13T18:53:59.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.279+0000 7f306affd700 1 -- 172.21.15.146:0/3659329247 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060005b80 con 0x7f305400ba50 2024-09-13T18:53:59.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.279+0000 7f306affd700 1 -- 172.21.15.146:0/3659329247 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3060016070 con 0x7f305400ba50 2024-09-13T18:53:59.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.279+0000 7f307271c700 1 -- 172.21.15.146:0/3659329247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305400ba50 msgr2=0x7f305400de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:59.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.279+0000 7f307271c700 1 --2- 172.21.15.146:0/3659329247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305400ba50 0x7f305400de40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f306c070b80 tx=0x7f3054003860 comp rx=0 tx=0).stop 2024-09-13T18:53:59.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.279+0000 7f307271c700 1 -- 172.21.15.146:0/3659329247 shutdown_connections 2024-09-13T18:53:59.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.279+0000 7f307271c700 1 --2- 172.21.15.146:0/3659329247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305400ba50 0x7f305400de40 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.279+0000 7f307271c700 1 -- 172.21.15.146:0/3659329247 >> 172.21.15.146:0/3659329247 conn(0x7f306c066090 msgr2=0x7f306c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:59.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.283+0000 7f307271c700 1 -- 172.21.15.146:0/3659329247 shutdown_connections 2024-09-13T18:53:59.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.283+0000 7f307271c700 1 -- 172.21.15.146:0/3659329247 wait complete. 2024-09-13T18:53:59.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.287+0000 7f307271c700 1 Processor -- start 2024-09-13T18:53:59.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f307271c700 1 -- start start 2024-09-13T18:53:59.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f307271c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f306c207fc0 0x7f306c0ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f307271c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f306c0aa7d0 0x7f306c0c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f307271c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f306c2083a0 0x7f306c0c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f307271c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c0c1ab0 con 0x7f306c207fc0 2024-09-13T18:53:59.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f307271c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c208c40 con 0x7f306c0aa7d0 2024-09-13T18:53:59.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f307271c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c16a060 con 0x7f306c2083a0 2024-09-13T18:53:59.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f306bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f306c207fc0 0x7f306c0ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f306bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f306c207fc0 0x7f306c0ae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58562/0 (socket says 172.21.15.146:58562) 2024-09-13T18:53:59.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f306bfff700 1 -- 172.21.15.146:0/875971622 learned_addr learned my addr 172.21.15.146:0/875971622 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:53:59.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f306b7fe700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f306c0aa7d0 0x7f306c0c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f3070cb9700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f306c2083a0 0x7f306c0c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f306bfff700 1 -- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f306c2083a0 msgr2=0x7f306c0c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:59.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f306bfff700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f306c2083a0 0x7f306c0c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f306bfff700 1 -- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f306c0aa7d0 msgr2=0x7f306c0c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:59.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f306bfff700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f306c0aa7d0 0x7f306c0c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f306bfff700 1 -- 172.21.15.146:0/875971622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c208960 con 0x7f306c207fc0 2024-09-13T18:53:59.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.291+0000 7f306bfff700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f306c207fc0 0x7f306c0ae370 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f305c00e7a0 tx=0x7f305c00eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:59.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.295+0000 7f3068ff9700 1 -- 172.21.15.146:0/875971622 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c004020 con 0x7f306c207fc0 2024-09-13T18:53:59.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.295+0000 7f307271c700 1 -- 172.21.15.146:0/875971622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c0c14f0 con 0x7f306c207fc0 2024-09-13T18:53:59.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.295+0000 7f307271c700 1 -- 172.21.15.146:0/875971622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c0c32c0 con 0x7f306c207fc0 2024-09-13T18:53:59.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.299+0000 7f307271c700 1 -- 172.21.15.146:0/875971622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306c0c9da0 con 0x7f306c207fc0 2024-09-13T18:53:59.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.303+0000 7f3068ff9700 1 -- 172.21.15.146:0/875971622 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c004180 con 0x7f306c207fc0 2024-09-13T18:53:59.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.307+0000 7f3068ff9700 1 -- 172.21.15.146:0/875971622 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c004530 con 0x7f306c207fc0 2024-09-13T18:53:59.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.307+0000 7f3068ff9700 1 -- 172.21.15.146:0/875971622 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f305c0036a0 con 0x7f306c207fc0 2024-09-13T18:53:59.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.311+0000 7f3068ff9700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f30580668b0 0x7f3058068d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.311+0000 7f3068ff9700 1 -- 172.21.15.146:0/875971622 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7f305c0843e0 con 0x7f306c207fc0 2024-09-13T18:53:59.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.311+0000 7f306b7fe700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f30580668b0 0x7f3058068d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.311+0000 7f3068ff9700 1 -- 172.21.15.146:0/875971622 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f305c050f20 con 0x7f306c207fc0 2024-09-13T18:53:59.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:53:59.311+0000 7f306b7fe700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f30580668b0 0x7f3058068d70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f306c066aa0 tx=0x7f3064009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:00.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.187+0000 7f307271c700 1 -- 172.21.15.146:0/875971622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f306c0c3120 con 0x7f306c207fc0 2024-09-13T18:54:00.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.187+0000 7f3068ff9700 1 -- 172.21.15.146:0/875971622 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+5537 (secure 0 0 0) 0x7f306c0c3120 con 0x7f306c207fc0 2024-09-13T18:54:00.204 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:54:00.204 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":108,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:58.686432+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:resolve","state_seq":79,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:54:00.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.191+0000 7f307271c700 1 -- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f30580668b0 msgr2=0x7f3058068d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:00.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.191+0000 7f307271c700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f30580668b0 0x7f3058068d70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f306c066aa0 tx=0x7f3064009040 comp rx=0 tx=0).stop 2024-09-13T18:54:00.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.191+0000 7f307271c700 1 -- 172.21.15.146:0/875971622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f306c207fc0 msgr2=0x7f306c0ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:00.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.191+0000 7f307271c700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f306c207fc0 0x7f306c0ae370 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f305c00e7a0 tx=0x7f305c00eab0 comp rx=0 tx=0).stop 2024-09-13T18:54:00.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.191+0000 7f307271c700 1 -- 172.21.15.146:0/875971622 shutdown_connections 2024-09-13T18:54:00.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.191+0000 7f307271c700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f30580668b0 0x7f3058068d70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:00.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.191+0000 7f307271c700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f306c2083a0 0x7f306c0c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:00.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.191+0000 7f307271c700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f306c0aa7d0 0x7f306c0c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:00.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.191+0000 7f307271c700 1 --2- 172.21.15.146:0/875971622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f306c207fc0 0x7f306c0ae370 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:00.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.191+0000 7f307271c700 1 -- 172.21.15.146:0/875971622 >> 172.21.15.146:0/875971622 conn(0x7f306c066090 msgr2=0x7f306c0c6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:00.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.191+0000 7f307271c700 1 -- 172.21.15.146:0/875971622 shutdown_connections 2024-09-13T18:54:00.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:00.191+0000 7f307271c700 1 -- 172.21.15.146:0/875971622 wait complete. 2024-09-13T18:54:00.217 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 108 2024-09-13T18:54:00.273 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:54:02.274 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:02.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f02923a6700 1 Processor -- start 2024-09-13T18:54:02.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f02923a6700 1 -- start start 2024-09-13T18:54:02.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f02923a6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f028c0cb2b0 0x7f028c0723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f02923a6700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028c161550 con 0x7f028c0caed0 2024-09-13T18:54:02.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f02923a6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028c0aa800 con 0x7f028c0a9840 2024-09-13T18:54:02.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f02923a6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c072a30 con 0x7f028c0cb2b0 2024-09-13T18:54:02.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f02913a4700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f028c0caed0 0x7f028c1619e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:54306/0 (socket says 172.21.15.146:54306) 2024-09-13T18:54:02.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f02913a4700 1 -- 172.21.15.146:0/1305974262 learned_addr learned my addr 172.21.15.146:0/1305974262 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:54:02.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f0290ba3700 1 --2- 172.21.15.146:0/1305974262 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f028c0cb2b0 0x7f028c0723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856129569 0 0) 0x7f028c161550 con 0x7f028c0caed0 2024-09-13T18:54:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02780035d0 con 0x7f028c0caed0 2024-09-13T18:54:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3798838570 0 0) 0x7f02780035d0 con 0x7f028c0caed0 2024-09-13T18:54:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f028c0cb2b0 msgr2=0x7f028c0723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f0283fff700 1 --2- 172.21.15.146:0/1305974262 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f028c0cb2b0 0x7f028c0723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 >> v1:172.21.15.154:6789/0 conn(0x7f028c0a9840 legacy=0x7f028c15e100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:54:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c161220 con 0x7f028c0caed0 2024-09-13T18:54:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f0288001670 con 0x7f028c0caed0 2024-09-13T18:54:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 >> v1:172.21.15.146:6789/0 conn(0x7f028c0caed0 legacy=0x7f028c1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.695+0000 7f0283fff700 1 --2- 172.21.15.146:0/1305974262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0278003ac0 0x7f0278003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f02913a4700 1 --2- 172.21.15.146:0/1305974262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0278003ac0 0x7f0278003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f0283fff700 1 --2- 172.21.15.146:0/1305974262 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0278004870 0x7f0278004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f0283fff700 1 --2- 172.21.15.146:0/1305974262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0278005280 0x7f027801ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c072a30 con 0x7f0278003ac0 2024-09-13T18:54:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027801d1c0 con 0x7f0278005280 2024-09-13T18:54:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027801d380 con 0x7f0278004870 2024-09-13T18:54:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f02913a4700 1 -- 172.21.15.146:0/1305974262 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0278004870 msgr2=0x7f0278004cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:54:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f02913a4700 1 --2- 172.21.15.146:0/1305974262 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0278004870 0x7f0278004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f02913a4700 1 -- 172.21.15.146:0/1305974262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0278005280 msgr2=0x7f027801ca40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:54:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f02913a4700 1 --2- 172.21.15.146:0/1305974262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0278005280 0x7f027801ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f02913a4700 1 -- 172.21.15.146:0/1305974262 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02780035d0 con 0x7f0278003ac0 2024-09-13T18:54:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f02913a4700 1 --2- 172.21.15.146:0/1305974262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0278003ac0 0x7f0278003ea0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0288005070 tx=0x7f02780058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877140cc1ae77571 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288005980 con 0x7f0278003ac0 2024-09-13T18:54:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0288005ae0 con 0x7f0278003ac0 2024-09-13T18:54:02.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f0283fff700 1 -- 172.21.15.146:0/1305974262 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288005e70 con 0x7f0278003ac0 2024-09-13T18:54:02.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f02923a6700 1 -- 172.21.15.146:0/1305974262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0278003ac0 msgr2=0x7f0278003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:02.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.699+0000 7f02923a6700 1 --2- 172.21.15.146:0/1305974262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0278003ac0 0x7f0278003ea0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0288005070 tx=0x7f02780058a0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.703+0000 7f0291ba5700 1 -- 172.21.15.146:0/1305974262 reap_dead start 2024-09-13T18:54:02.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.719+0000 7f02923a6700 1 -- 172.21.15.146:0/1305974262 shutdown_connections 2024-09-13T18:54:02.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.719+0000 7f02923a6700 1 -- 172.21.15.146:0/1305974262 >> 172.21.15.146:0/1305974262 conn(0x7f028c066090 msgr2=0x7f028c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:02.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.719+0000 7f02923a6700 1 -- 172.21.15.146:0/1305974262 shutdown_connections 2024-09-13T18:54:02.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.719+0000 7f02923a6700 1 -- 172.21.15.146:0/1305974262 wait complete. 2024-09-13T18:54:02.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.719+0000 7f02923a6700 1 Processor -- start 2024-09-13T18:54:02.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.723+0000 7f02923a6700 1 -- start start 2024-09-13T18:54:02.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.723+0000 7f02923a6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0278003ac0 0x7f028c1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.723+0000 7f02923a6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f028c14ee80 0x7f028c156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.723+0000 7f02923a6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f028c151820 0x7f028c151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.723+0000 7f02923a6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c06de40 con 0x7f0278003ac0 2024-09-13T18:54:02.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.723+0000 7f02923a6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c1eb630 con 0x7f028c151820 2024-09-13T18:54:02.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.723+0000 7f02923a6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c1521f0 con 0x7f028c14ee80 2024-09-13T18:54:02.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.727+0000 7f0290ba3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f028c14ee80 0x7f028c156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.727+0000 7f0290ba3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f028c14ee80 0x7f028c156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:50516/0 (socket says 172.21.15.146:50516) 2024-09-13T18:54:02.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.727+0000 7f0290ba3700 1 -- 172.21.15.146:0/2564227663 learned_addr learned my addr 172.21.15.146:0/2564227663 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:54:02.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.727+0000 7f0290ba3700 1 -- 172.21.15.146:0/2564227663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f028c151820 msgr2=0x7f028c151cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:54:02.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.727+0000 7f0290ba3700 1 --2- 172.21.15.146:0/2564227663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f028c151820 0x7f028c151cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.727+0000 7f0290ba3700 1 -- 172.21.15.146:0/2564227663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0278003ac0 msgr2=0x7f028c1564f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:54:02.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.727+0000 7f0290ba3700 1 --2- 172.21.15.146:0/2564227663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0278003ac0 0x7f028c1564f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.727+0000 7f0290ba3700 1 -- 172.21.15.146:0/2564227663 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c155d30 con 0x7f028c14ee80 2024-09-13T18:54:02.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.727+0000 7f0290ba3700 1 --2- 172.21.15.146:0/2564227663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f028c14ee80 0x7f028c156a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f028400eb90 tx=0x7f028400ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:02.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.731+0000 7f0281ffb700 1 -- 172.21.15.146:0/2564227663 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028400cb10 con 0x7f028c14ee80 2024-09-13T18:54:02.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.731+0000 7f02923a6700 1 -- 172.21.15.146:0/2564227663 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028c06db60 con 0x7f028c14ee80 2024-09-13T18:54:02.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.731+0000 7f02923a6700 1 -- 172.21.15.146:0/2564227663 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c155fd0 con 0x7f028c14ee80 2024-09-13T18:54:02.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.731+0000 7f0281ffb700 1 -- 172.21.15.146:0/2564227663 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028400f040 con 0x7f028c14ee80 2024-09-13T18:54:02.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.731+0000 7f0281ffb700 1 -- 172.21.15.146:0/2564227663 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028400ce90 con 0x7f028c14ee80 2024-09-13T18:54:02.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.735+0000 7f02923a6700 1 -- 172.21.15.146:0/2564227663 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026c005310 con 0x7f028c14ee80 2024-09-13T18:54:02.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.739+0000 7f0281ffb700 1 -- 172.21.15.146:0/2564227663 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0284014480 con 0x7f028c14ee80 2024-09-13T18:54:02.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.743+0000 7f0281ffb700 1 --2- 172.21.15.146:0/2564227663 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0270066830 0x7f0270068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.743+0000 7f0281ffb700 1 -- 172.21.15.146:0/2564227663 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7f0284085340 con 0x7f028c14ee80 2024-09-13T18:54:02.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.743+0000 7f02913a4700 1 --2- 172.21.15.146:0/2564227663 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0270066830 0x7f0270068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.743+0000 7f02913a4700 1 --2- 172.21.15.146:0/2564227663 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0270066830 0x7f0270068cf0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0288005070 tx=0x7f0288006000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:02.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:02.747+0000 7f0281ffb700 1 -- 172.21.15.146:0/2564227663 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0284051e50 con 0x7f028c14ee80 2024-09-13T18:54:03.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.619+0000 7f02923a6700 1 -- 172.21.15.146:0/2564227663 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f026c005b40 con 0x7f028c14ee80 2024-09-13T18:54:03.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.623+0000 7f0281ffb700 1 -- 172.21.15.146:0/2564227663 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+5537 (secure 0 0 0) 0x7f0284018070 con 0x7f028c14ee80 2024-09-13T18:54:03.640 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:54:03.640 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":108,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:53:58.686432+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:resolve","state_seq":79,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:54:03.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.643+0000 7f02777fe700 1 -- 172.21.15.146:0/2564227663 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0270066830 msgr2=0x7f0270068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:03.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.643+0000 7f02777fe700 1 --2- 172.21.15.146:0/2564227663 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0270066830 0x7f0270068cf0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0288005070 tx=0x7f0288006000 comp rx=0 tx=0).stop 2024-09-13T18:54:03.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.643+0000 7f02777fe700 1 -- 172.21.15.146:0/2564227663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f028c14ee80 msgr2=0x7f028c156a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:03.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.643+0000 7f02777fe700 1 --2- 172.21.15.146:0/2564227663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f028c14ee80 0x7f028c156a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f028400eb90 tx=0x7f028400ef50 comp rx=0 tx=0).stop 2024-09-13T18:54:03.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.643+0000 7f02777fe700 1 -- 172.21.15.146:0/2564227663 shutdown_connections 2024-09-13T18:54:03.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.643+0000 7f02777fe700 1 --2- 172.21.15.146:0/2564227663 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f0270066830 0x7f0270068cf0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.643+0000 7f02777fe700 1 --2- 172.21.15.146:0/2564227663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f028c151820 0x7f028c151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.643+0000 7f02777fe700 1 --2- 172.21.15.146:0/2564227663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f028c14ee80 0x7f028c156a30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.647+0000 7f02777fe700 1 --2- 172.21.15.146:0/2564227663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0278003ac0 0x7f028c1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.647+0000 7f02777fe700 1 -- 172.21.15.146:0/2564227663 >> 172.21.15.146:0/2564227663 conn(0x7f028c066090 msgr2=0x7f028c15f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:03.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.647+0000 7f02777fe700 1 -- 172.21.15.146:0/2564227663 shutdown_connections 2024-09-13T18:54:03.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:03.647+0000 7f02777fe700 1 -- 172.21.15.146:0/2564227663 wait complete. 2024-09-13T18:54:03.667 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 108 2024-09-13T18:54:03.691 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:54:05.693 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:06.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.039+0000 7f32ee8b2700 1 Processor -- start 2024-09-13T18:54:06.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.039+0000 7f32ee8b2700 1 -- start start 2024-09-13T18:54:06.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.039+0000 7f32ee8b2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f32e808baa0 0x7f32e80cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.039+0000 7f32ee8b2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e816e0c0 0x7f32e81704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.039+0000 7f32ee8b2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e80a9920 con 0x7f32e80cac60 2024-09-13T18:54:06.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.039+0000 7f32ee8b2700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e80723f0 con 0x7f32e808baa0 2024-09-13T18:54:06.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.039+0000 7f32ee8b2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e80cb580 con 0x7f32e816e0c0 2024-09-13T18:54:06.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.043+0000 7f32ed0af700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f32e808baa0 0x7f32e80cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.043+0000 7f32ed0af700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f32e808baa0 0x7f32e80cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:34016/0 (socket says 172.21.15.146:34016) 2024-09-13T18:54:06.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.043+0000 7f32ed0af700 1 -- 172.21.15.146:0/609776362 learned_addr learned my addr 172.21.15.146:0/609776362 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:54:06.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.043+0000 7f32ee0b1700 1 --2- 172.21.15.146:0/609776362 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e816e0c0 0x7f32e81704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.043+0000 7f32ed0af700 1 -- 172.21.15.146:0/609776362 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e816e0c0 msgr2=0x7f32e81704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:06.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.043+0000 7f32ed0af700 1 --2- 172.21.15.146:0/609776362 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e816e0c0 0x7f32e81704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.043+0000 7f32ed0af700 1 -- 172.21.15.146:0/609776362 >> v1:172.21.15.154:6789/0 conn(0x7f32e80cac60 legacy=0x7f32e806f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.043+0000 7f32ed0af700 1 -- 172.21.15.146:0/609776362 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e8070070 con 0x7f32e808baa0 2024-09-13T18:54:06.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.043+0000 7f32ed0af700 1 --2- 172.21.15.146:0/609776362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e808baa0 0x7f32e80cb040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f32e0008e10 tx=0x7f32e000b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3c7a53ac1460bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:06.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.051+0000 7f32ec8ae700 1 -- 172.21.15.146:0/609776362 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e000bca0 con 0x7f32e808baa0 2024-09-13T18:54:06.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.055+0000 7f32ec8ae700 1 -- 172.21.15.146:0/609776362 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e000be00 con 0x7f32e808baa0 2024-09-13T18:54:06.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.055+0000 7f32ee8b2700 1 -- 172.21.15.146:0/609776362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e808baa0 msgr2=0x7f32e80cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.055+0000 7f32ee8b2700 1 --2- 172.21.15.146:0/609776362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e808baa0 0x7f32e80cb040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f32e0008e10 tx=0x7f32e000b4b0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.055+0000 7f32ee8b2700 1 -- 172.21.15.146:0/609776362 shutdown_connections 2024-09-13T18:54:06.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.055+0000 7f32ee8b2700 1 --2- 172.21.15.146:0/609776362 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e816e0c0 0x7f32e81704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.055+0000 7f32ee8b2700 1 --2- 172.21.15.146:0/609776362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e808baa0 0x7f32e80cb040 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.055+0000 7f32ee8b2700 1 -- 172.21.15.146:0/609776362 >> 172.21.15.146:0/609776362 conn(0x7f32e8066090 msgr2=0x7f32e8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:06.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.055+0000 7f32ee8b2700 1 -- 172.21.15.146:0/609776362 shutdown_connections 2024-09-13T18:54:06.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.055+0000 7f32ee8b2700 1 -- 172.21.15.146:0/609776362 wait complete. 2024-09-13T18:54:06.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.055+0000 7f32ee8b2700 1 Processor -- start 2024-09-13T18:54:06.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.055+0000 7f32ee8b2700 1 -- start start 2024-09-13T18:54:06.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ee8b2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e80cac60 0x7f32e80ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ee8b2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e816e0c0 0x7f32e81fc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ee8b2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e81ff0a0 0x7f32e80af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ee8b2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e81fbaf0 con 0x7f32e81ff0a0 2024-09-13T18:54:06.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ee8b2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e81fca40 con 0x7f32e80cac60 2024-09-13T18:54:06.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ee8b2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e80ad0f0 con 0x7f32e816e0c0 2024-09-13T18:54:06.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ed0af700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e816e0c0 0x7f32e81fc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ed0af700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e816e0c0 0x7f32e81fc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:50520/0 (socket says 172.21.15.146:50520) 2024-09-13T18:54:06.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ed0af700 1 -- 172.21.15.146:0/1927260715 learned_addr learned my addr 172.21.15.146:0/1927260715 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:54:06.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ed8b0700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e80cac60 0x7f32e80ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ee0b1700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e81ff0a0 0x7f32e80af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ed0af700 1 -- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e80cac60 msgr2=0x7f32e80ac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ed0af700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e80cac60 0x7f32e80ac7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ed0af700 1 -- 172.21.15.146:0/1927260715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e81ff0a0 msgr2=0x7f32e80af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ed0af700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e81ff0a0 0x7f32e80af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ed0af700 1 -- 172.21.15.146:0/1927260715 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e0008a70 con 0x7f32e816e0c0 2024-09-13T18:54:06.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.059+0000 7f32ee0b1700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e81ff0a0 0x7f32e80af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:06.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.063+0000 7f32ed0af700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e816e0c0 0x7f32e81fc500 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f32e000b4f0 tx=0x7f32e0004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:06.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.067+0000 7f32de7fc700 1 -- 172.21.15.146:0/1927260715 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0019340 con 0x7f32e816e0c0 2024-09-13T18:54:06.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.067+0000 7f32ee8b2700 1 -- 172.21.15.146:0/1927260715 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e80ad3d0 con 0x7f32e816e0c0 2024-09-13T18:54:06.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.067+0000 7f32ee8b2700 1 -- 172.21.15.146:0/1927260715 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e81fbd00 con 0x7f32e816e0c0 2024-09-13T18:54:06.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.071+0000 7f32de7fc700 1 -- 172.21.15.146:0/1927260715 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e00194c0 con 0x7f32e816e0c0 2024-09-13T18:54:06.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.071+0000 7f32de7fc700 1 -- 172.21.15.146:0/1927260715 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0022a40 con 0x7f32e816e0c0 2024-09-13T18:54:06.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.071+0000 7f32ee8b2700 1 -- 172.21.15.146:0/1927260715 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d0005310 con 0x7f32e816e0c0 2024-09-13T18:54:06.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.075+0000 7f32de7fc700 1 -- 172.21.15.146:0/1927260715 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f32e0022ba0 con 0x7f32e816e0c0 2024-09-13T18:54:06.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.075+0000 7f32de7fc700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f32d4066920 0x7f32d4068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.075+0000 7f32de7fc700 1 -- 172.21.15.146:0/1927260715 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7f32e000d040 con 0x7f32e816e0c0 2024-09-13T18:54:06.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.075+0000 7f32ed8b0700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f32d4066920 0x7f32d4068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.075+0000 7f32ed8b0700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f32d4066920 0x7f32d4068de0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000f60 tx=0x7f32e4001250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:06.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:06.091+0000 7f32de7fc700 1 -- 172.21.15.146:0/1927260715 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f32e0053040 con 0x7f32e816e0c0 2024-09-13T18:54:07.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.139+0000 7f32ee8b2700 1 -- 172.21.15.146:0/1927260715 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32d0005890 con 0x7f32e816e0c0 2024-09-13T18:54:07.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.139+0000 7f32de7fc700 1 -- 172.21.15.146:0/1927260715 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+5539 (secure 0 0 0) 0x7f32e0028070 con 0x7f32e816e0c0 2024-09-13T18:54:07.161 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:54:07.161 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":109,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:54:03.721754+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:reconnect","state_seq":81,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:54:07.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.147+0000 7f32cbfff700 1 -- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f32d4066920 msgr2=0x7f32d4068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:07.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.147+0000 7f32cbfff700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f32d4066920 0x7f32d4068de0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000f60 tx=0x7f32e4001250 comp rx=0 tx=0).stop 2024-09-13T18:54:07.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.147+0000 7f32cbfff700 1 -- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e816e0c0 msgr2=0x7f32e81fc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:07.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.147+0000 7f32cbfff700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e816e0c0 0x7f32e81fc500 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f32e000b4f0 tx=0x7f32e0004af0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.147+0000 7f32cbfff700 1 -- 172.21.15.146:0/1927260715 shutdown_connections 2024-09-13T18:54:07.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.147+0000 7f32cbfff700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f32d4066920 0x7f32d4068de0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.147+0000 7f32cbfff700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32e81ff0a0 0x7f32e80af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.147+0000 7f32cbfff700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e816e0c0 0x7f32e81fc500 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.147+0000 7f32cbfff700 1 --2- 172.21.15.146:0/1927260715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e80cac60 0x7f32e80ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.147+0000 7f32cbfff700 1 -- 172.21.15.146:0/1927260715 >> 172.21.15.146:0/1927260715 conn(0x7f32e8066090 msgr2=0x7f32e816d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:07.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.147+0000 7f32cbfff700 1 -- 172.21.15.146:0/1927260715 shutdown_connections 2024-09-13T18:54:07.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:07.147+0000 7f32cbfff700 1 -- 172.21.15.146:0/1927260715 wait complete. 2024-09-13T18:54:07.164 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 109 2024-09-13T18:54:07.212 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:54:09.213 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:10.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.023+0000 7fb27f3fa700 1 Processor -- start 2024-09-13T18:54:10.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.023+0000 7fb27f3fa700 1 -- start start 2024-09-13T18:54:10.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27f3fa700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb27814f630 0x7fb2780c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:10.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27f3fa700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2780a9490 0x7fb278155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:10.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27f3fa700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb278156140 0x7fb2780cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:10.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27f3fa700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27806f5c0 con 0x7fb27814f630 2024-09-13T18:54:10.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27f3fa700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb278155420 con 0x7fb278156140 2024-09-13T18:54:10.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27f3fa700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2780c7b70 con 0x7fb2780a9490 2024-09-13T18:54:10.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27ebf9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb278156140 0x7fb2780cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:10.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27ebf9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb278156140 0x7fb2780cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:34030/0 (socket says 172.21.15.146:34030) 2024-09-13T18:54:10.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27ebf9700 1 -- 172.21.15.146:0/4212444768 learned_addr learned my addr 172.21.15.146:0/4212444768 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:54:10.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27e3f8700 1 --2- 172.21.15.146:0/4212444768 >> v2:172.21.15.146:3300/0 conn(0x7fb27814f630 0x7fb2780c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:10.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27e3f8700 1 -- 172.21.15.146:0/4212444768 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2780a9490 msgr2=0x7fb278155c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:54:10.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27e3f8700 1 --2- 172.21.15.146:0/4212444768 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2780a9490 0x7fb278155c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:10.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27e3f8700 1 -- 172.21.15.146:0/4212444768 >> v2:172.21.15.154:3300/0 conn(0x7fb278156140 msgr2=0x7fb2780cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:10.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27e3f8700 1 --2- 172.21.15.146:0/4212444768 >> v2:172.21.15.154:3300/0 conn(0x7fb278156140 0x7fb2780cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:10.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.027+0000 7fb27e3f8700 1 -- 172.21.15.146:0/4212444768 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2780c7e50 con 0x7fb27814f630 2024-09-13T18:54:10.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.035+0000 7fb27e3f8700 1 --2- 172.21.15.146:0/4212444768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb27814f630 0x7fb2780c9500 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb2780ca4a0 tx=0x7fb2740093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0675903606f47a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:10.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.035+0000 7fb27d3f6700 1 -- 172.21.15.146:0/4212444768 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274013c90 con 0x7fb27814f630 2024-09-13T18:54:10.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.035+0000 7fb27d3f6700 1 -- 172.21.15.146:0/4212444768 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb274013df0 con 0x7fb27814f630 2024-09-13T18:54:10.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.035+0000 7fb27f3fa700 1 -- 172.21.15.146:0/4212444768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb27814f630 msgr2=0x7fb2780c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:10.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.035+0000 7fb27f3fa700 1 --2- 172.21.15.146:0/4212444768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb27814f630 0x7fb2780c9500 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb2780ca4a0 tx=0x7fb2740093c0 comp rx=0 tx=0).stop 2024-09-13T18:54:10.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.047+0000 7fb27f3fa700 1 -- 172.21.15.146:0/4212444768 shutdown_connections 2024-09-13T18:54:10.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.047+0000 7fb27f3fa700 1 --2- 172.21.15.146:0/4212444768 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2780a9490 0x7fb278155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:10.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.047+0000 7fb27f3fa700 1 --2- 172.21.15.146:0/4212444768 >> v2:172.21.15.154:3300/0 conn(0x7fb278156140 0x7fb2780cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:10.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.047+0000 7fb27f3fa700 1 --2- 172.21.15.146:0/4212444768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb27814f630 0x7fb2780c9500 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:10.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.047+0000 7fb27f3fa700 1 -- 172.21.15.146:0/4212444768 >> 172.21.15.146:0/4212444768 conn(0x7fb278066090 msgr2=0x7fb278066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:10.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.051+0000 7fb27f3fa700 1 -- 172.21.15.146:0/4212444768 shutdown_connections 2024-09-13T18:54:10.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.051+0000 7fb27f3fa700 1 -- 172.21.15.146:0/4212444768 wait complete. 2024-09-13T18:54:10.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.055+0000 7fb27f3fa700 1 Processor -- start 2024-09-13T18:54:10.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27f3fa700 1 -- start start 2024-09-13T18:54:10.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27f3fa700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2780a9490 0x7fb2780af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:10.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27f3fa700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb27814f630 0x7fb278152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:10.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27f3fa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb278156140 0x7fb2781477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:10.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27f3fa700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27814c1b0 con 0x7fb2780a9490 2024-09-13T18:54:10.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27f3fa700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2780c69f0 con 0x7fb278156140 2024-09-13T18:54:10.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27f3fa700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27814ba20 con 0x7fb27814f630 2024-09-13T18:54:10.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27e3f8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2780a9490 0x7fb2780af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27e3f8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2780a9490 0x7fb2780af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43702/0 (socket says 172.21.15.146:43702) 2024-09-13T18:54:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27e3f8700 1 -- 172.21.15.146:0/2284357475 learned_addr learned my addr 172.21.15.146:0/2284357475 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:54:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27e3f8700 1 -- 172.21.15.146:0/2284357475 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb27814f630 msgr2=0x7fb278152610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:54:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27e3f8700 1 --2- 172.21.15.146:0/2284357475 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb27814f630 0x7fb278152610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27e3f8700 1 -- 172.21.15.146:0/2284357475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb278156140 msgr2=0x7fb2781477f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:54:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27e3f8700 1 --2- 172.21.15.146:0/2284357475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb278156140 0x7fb2781477f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:10.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27e3f8700 1 -- 172.21.15.146:0/2284357475 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb274009d40 con 0x7fb2780a9490 2024-09-13T18:54:10.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27e3f8700 1 --2- 172.21.15.146:0/2284357475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2780a9490 0x7fb2780af8d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb274009710 tx=0x7fb274004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:10.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb26effd700 1 -- 172.21.15.146:0/2284357475 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274013570 con 0x7fb2780a9490 2024-09-13T18:54:10.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb26effd700 1 -- 172.21.15.146:0/2284357475 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2740136d0 con 0x7fb2780a9490 2024-09-13T18:54:10.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb26effd700 1 -- 172.21.15.146:0/2284357475 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274012630 con 0x7fb2780a9490 2024-09-13T18:54:10.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27f3fa700 1 -- 172.21.15.146:0/2284357475 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb278152000 con 0x7fb2780a9490 2024-09-13T18:54:10.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.059+0000 7fb27f3fa700 1 -- 172.21.15.146:0/2284357475 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb278148bd0 con 0x7fb2780a9490 2024-09-13T18:54:10.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.063+0000 7fb26effd700 1 -- 172.21.15.146:0/2284357475 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb274012790 con 0x7fb2780a9490 2024-09-13T18:54:10.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.067+0000 7fb27f3fa700 1 -- 172.21.15.146:0/2284357475 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb260005310 con 0x7fb2780a9490 2024-09-13T18:54:10.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.067+0000 7fb26effd700 1 --2- 172.21.15.146:0/2284357475 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb25c066730 0x7fb25c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:10.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.067+0000 7fb27dbf7700 1 --2- 172.21.15.146:0/2284357475 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb25c066730 0x7fb25c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:10.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.067+0000 7fb26effd700 1 -- 172.21.15.146:0/2284357475 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7fb274086560 con 0x7fb2780a9490 2024-09-13T18:54:10.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.071+0000 7fb27dbf7700 1 --2- 172.21.15.146:0/2284357475 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb25c066730 0x7fb25c068bf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb278159850 tx=0x7fb268006c80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:10.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:10.079+0000 7fb26effd700 1 -- 172.21.15.146:0/2284357475 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb274053110 con 0x7fb2780a9490 2024-09-13T18:54:11.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.107+0000 7fb27f3fa700 1 -- 172.21.15.146:0/2284357475 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb260005b40 con 0x7fb2780a9490 2024-09-13T18:54:11.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.111+0000 7fb26effd700 1 -- 172.21.15.146:0/2284357475 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+5536 (secure 0 0 0) 0x7fb274056730 con 0x7fb2780a9490 2024-09-13T18:54:11.125 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:54:11.125 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":110,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:54:08.750236+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:rejoin","state_seq":83,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:54:11.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.111+0000 7fb26cff9700 1 -- 172.21.15.146:0/2284357475 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb25c066730 msgr2=0x7fb25c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:11.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.111+0000 7fb26cff9700 1 --2- 172.21.15.146:0/2284357475 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb25c066730 0x7fb25c068bf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb278159850 tx=0x7fb268006c80 comp rx=0 tx=0).stop 2024-09-13T18:54:11.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.111+0000 7fb26cff9700 1 -- 172.21.15.146:0/2284357475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2780a9490 msgr2=0x7fb2780af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:11.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.111+0000 7fb26cff9700 1 --2- 172.21.15.146:0/2284357475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2780a9490 0x7fb2780af8d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb274009710 tx=0x7fb274004770 comp rx=0 tx=0).stop 2024-09-13T18:54:11.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.119+0000 7fb26cff9700 1 -- 172.21.15.146:0/2284357475 shutdown_connections 2024-09-13T18:54:11.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.119+0000 7fb26cff9700 1 --2- 172.21.15.146:0/2284357475 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb25c066730 0x7fb25c068bf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:11.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.119+0000 7fb26cff9700 1 --2- 172.21.15.146:0/2284357475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb278156140 0x7fb2781477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:11.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.119+0000 7fb26cff9700 1 --2- 172.21.15.146:0/2284357475 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb27814f630 0x7fb278152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:11.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.119+0000 7fb26cff9700 1 --2- 172.21.15.146:0/2284357475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2780a9490 0x7fb2780af8d0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:11.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.123+0000 7fb26cff9700 1 -- 172.21.15.146:0/2284357475 >> 172.21.15.146:0/2284357475 conn(0x7fb278066090 msgr2=0x7fb278153d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:11.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.123+0000 7fb26cff9700 1 -- 172.21.15.146:0/2284357475 shutdown_connections 2024-09-13T18:54:11.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:11.123+0000 7fb26cff9700 1 -- 172.21.15.146:0/2284357475 wait complete. 2024-09-13T18:54:11.145 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 110 2024-09-13T18:54:11.160 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 110, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 110, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6032": { "addr": "172.21.15.154:6839/2053734262", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6837", "nonce": 2053734262, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2053734262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6032, "incarnation": 107, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:rejoin", "state_seq": 83 }, "gid_6984": { "addr": "172.21.15.146:6835/2934322065", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 2934322065, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 2934322065, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6984, "incarnation": 98, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 28 }, "gid_7098": { "addr": "172.21.15.146:6833/354253522", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 354253522, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 354253522, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7098, "incarnation": 105, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 42, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:54:08.750236+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6984, "mds_1": 6032, "mds_2": 7098 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:54:13.161 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:13.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.700+0000 7fe733ede700 1 Processor -- start 2024-09-13T18:54:13.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.700+0000 7fe733ede700 1 -- start start 2024-09-13T18:54:13.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe733ede700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe72c0adca0 0x7fe72c0cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe733ede700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe72c0c01f0 0x7fe72c1506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe733ede700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72c0c1da0 con 0x7fe72c06e580 2024-09-13T18:54:13.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe733ede700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72c06d040 con 0x7fe72c0adca0 2024-09-13T18:54:13.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe733ede700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72c06c7e0 con 0x7fe72c0c01f0 2024-09-13T18:54:13.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe73247b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe72c06e580 0x7fe72c156540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:58104/0 (socket says 172.21.15.146:58104) 2024-09-13T18:54:13.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe73247b700 1 -- 172.21.15.146:0/2625598782 learned_addr learned my addr 172.21.15.146:0/2625598782 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:54:13.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe730c78700 1 -- 172.21.15.146:0/2625598782 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081404705 0 0) 0x7fe72c0c1da0 con 0x7fe72c06e580 2024-09-13T18:54:13.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe730c78700 1 -- 172.21.15.146:0/2625598782 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7180035d0 con 0x7fe72c06e580 2024-09-13T18:54:13.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe731c7a700 1 --2- 172.21.15.146:0/2625598782 >> v2:172.21.15.146:3300/0 conn(0x7fe72c0adca0 0x7fe72c0cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe731c7a700 1 -- 172.21.15.146:0/2625598782 >> v2:172.21.15.154:3300/0 conn(0x7fe72c0c01f0 msgr2=0x7fe72c1506f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:54:13.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe731c7a700 1 --2- 172.21.15.146:0/2625598782 >> v2:172.21.15.154:3300/0 conn(0x7fe72c0c01f0 0x7fe72c1506f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe731c7a700 1 -- 172.21.15.146:0/2625598782 >> v1:172.21.15.154:6789/0 conn(0x7fe72c06e580 legacy=0x7fe72c156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe731c7a700 1 -- 172.21.15.146:0/2625598782 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c15ce20 con 0x7fe72c0adca0 2024-09-13T18:54:13.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.704+0000 7fe731c7a700 1 --2- 172.21.15.146:0/2625598782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe72c0adca0 0x7fe72c0cbcf0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe72c06ba40 tx=0x7fe72400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77453cb2e814e754 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:13.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.708+0000 7fe730c78700 1 -- 172.21.15.146:0/2625598782 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72400e070 con 0x7fe72c0adca0 2024-09-13T18:54:13.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.712+0000 7fe730c78700 1 -- 172.21.15.146:0/2625598782 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72400bb20 con 0x7fe72c0adca0 2024-09-13T18:54:13.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.712+0000 7fe733ede700 1 -- 172.21.15.146:0/2625598782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe72c0adca0 msgr2=0x7fe72c0cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.712+0000 7fe733ede700 1 --2- 172.21.15.146:0/2625598782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe72c0adca0 0x7fe72c0cbcf0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe72c06ba40 tx=0x7fe72400b330 comp rx=0 tx=0).stop 2024-09-13T18:54:13.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 -- 172.21.15.146:0/2625598782 shutdown_connections 2024-09-13T18:54:13.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 --2- 172.21.15.146:0/2625598782 >> v2:172.21.15.154:3300/0 conn(0x7fe72c0c01f0 0x7fe72c1506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 --2- 172.21.15.146:0/2625598782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe72c0adca0 0x7fe72c0cbcf0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 -- 172.21.15.146:0/2625598782 >> 172.21.15.146:0/2625598782 conn(0x7fe72c0cb1d0 msgr2=0x7fe72c0cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:13.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 -- 172.21.15.146:0/2625598782 shutdown_connections 2024-09-13T18:54:13.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 -- 172.21.15.146:0/2625598782 wait complete. 2024-09-13T18:54:13.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 Processor -- start 2024-09-13T18:54:13.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 -- start start 2024-09-13T18:54:13.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72c06e580 0x7fe72c0c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe72c0c01f0 0x7fe72c14b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0b0f80 0x7fe72c14be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c13d710 con 0x7fe72c0c01f0 2024-09-13T18:54:13.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c0c5090 con 0x7fe72c06e580 2024-09-13T18:54:13.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.720+0000 7fe733ede700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c14e400 con 0x7fe72c0b0f80 2024-09-13T18:54:13.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe731c7a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72c06e580 0x7fe72c0c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe731c7a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72c06e580 0x7fe72c0c35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:44430/0 (socket says 172.21.15.146:44430) 2024-09-13T18:54:13.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe731c7a700 1 -- 172.21.15.146:0/2826814910 learned_addr learned my addr 172.21.15.146:0/2826814910 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:54:13.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe731479700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe72c0c01f0 0x7fe72c14b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe73247b700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0b0f80 0x7fe72c14be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe731c7a700 1 -- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0b0f80 msgr2=0x7fe72c14be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe731c7a700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0b0f80 0x7fe72c14be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe731c7a700 1 -- 172.21.15.146:0/2826814910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe72c0c01f0 msgr2=0x7fe72c14b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe731c7a700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe72c0c01f0 0x7fe72c14b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe731c7a700 1 -- 172.21.15.146:0/2826814910 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe724009a70 con 0x7fe72c06e580 2024-09-13T18:54:13.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe731479700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe72c0c01f0 0x7fe72c14b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:13.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe731c7a700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72c06e580 0x7fe72c0c35a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe724014040 tx=0x7fe724004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:13.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe7227fc700 1 -- 172.21.15.146:0/2826814910 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7240265f0 con 0x7fe72c06e580 2024-09-13T18:54:13.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe733ede700 1 -- 172.21.15.146:0/2826814910 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72c0c62e0 con 0x7fe72c06e580 2024-09-13T18:54:13.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe733ede700 1 -- 172.21.15.146:0/2826814910 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72c0c6860 con 0x7fe72c06e580 2024-09-13T18:54:13.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe7227fc700 1 -- 172.21.15.146:0/2826814910 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72400bb20 con 0x7fe72c06e580 2024-09-13T18:54:13.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe7227fc700 1 -- 172.21.15.146:0/2826814910 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7240269b0 con 0x7fe72c06e580 2024-09-13T18:54:13.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.724+0000 7fe733ede700 1 -- 172.21.15.146:0/2826814910 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72c0c2230 con 0x7fe72c06e580 2024-09-13T18:54:13.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.732+0000 7fe7227fc700 1 -- 172.21.15.146:0/2826814910 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe724026b10 con 0x7fe72c06e580 2024-09-13T18:54:13.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.732+0000 7fe7227fc700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe710066730 0x7fe710068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.732+0000 7fe7227fc700 1 -- 172.21.15.146:0/2826814910 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7fe724086760 con 0x7fe72c06e580 2024-09-13T18:54:13.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.732+0000 7fe731479700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe710066730 0x7fe710068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.740+0000 7fe731479700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe710066730 0x7fe710068bf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe72c149d70 tx=0x7fe71c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:13.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:13.744+0000 7fe7227fc700 1 -- 172.21.15.146:0/2826814910 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe724053310 con 0x7fe72c06e580 2024-09-13T18:54:14.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.504+0000 7fe733ede700 1 -- 172.21.15.146:0/2826814910 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe72c140670 con 0x7fe72c06e580 2024-09-13T18:54:14.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.504+0000 7fe7227fc700 1 -- 172.21.15.146:0/2826814910 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+5536 (secure 0 0 0) 0x7fe724056930 con 0x7fe72c06e580 2024-09-13T18:54:14.524 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:54:14.524 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":110,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:54:08.750236+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:rejoin","state_seq":83,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:54:14.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.516+0000 7fe733ede700 1 -- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe710066730 msgr2=0x7fe710068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.516+0000 7fe733ede700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe710066730 0x7fe710068bf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe72c149d70 tx=0x7fe71c008040 comp rx=0 tx=0).stop 2024-09-13T18:54:14.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.520+0000 7fe733ede700 1 -- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72c06e580 msgr2=0x7fe72c0c35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.520+0000 7fe733ede700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72c06e580 0x7fe72c0c35a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe724014040 tx=0x7fe724004af0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.520+0000 7fe733ede700 1 -- 172.21.15.146:0/2826814910 shutdown_connections 2024-09-13T18:54:14.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.520+0000 7fe733ede700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe710066730 0x7fe710068bf0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.520+0000 7fe733ede700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0b0f80 0x7fe72c14be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.520+0000 7fe733ede700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe72c0c01f0 0x7fe72c14b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.520+0000 7fe733ede700 1 --2- 172.21.15.146:0/2826814910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72c06e580 0x7fe72c0c35a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.520+0000 7fe733ede700 1 -- 172.21.15.146:0/2826814910 >> 172.21.15.146:0/2826814910 conn(0x7fe72c0cb1d0 msgr2=0x7fe72c15c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:14.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.520+0000 7fe733ede700 1 -- 172.21.15.146:0/2826814910 shutdown_connections 2024-09-13T18:54:14.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:14.520+0000 7fe733ede700 1 -- 172.21.15.146:0/2826814910 wait complete. 2024-09-13T18:54:14.539 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 110 2024-09-13T18:54:14.573 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:54:16.575 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:16.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2986d700 1 Processor -- start 2024-09-13T18:54:16.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2986d700 1 -- start start 2024-09-13T18:54:16.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2986d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fce2414f630 0x7fce240c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:16.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2986d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fce240a9490 0x7fce240cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:16.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2986d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce24155c00 0x7fce240cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:16.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2986d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2406f5c0 con 0x7fce2414f630 2024-09-13T18:54:16.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2986d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce24155420 con 0x7fce240a9490 2024-09-13T18:54:16.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2986d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce240c7b70 con 0x7fce24155c00 2024-09-13T18:54:16.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2886b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fce2414f630 0x7fce240c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:16.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2886b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fce2414f630 0x7fce240c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40006/0 (socket says 172.21.15.146:40006) 2024-09-13T18:54:16.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2886b700 1 -- 172.21.15.146:0/2833421680 learned_addr learned my addr 172.21.15.146:0/2833421680 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:54:16.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2906c700 1 --2- 172.21.15.146:0/2833421680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce24155c00 0x7fce240cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:16.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2886b700 1 -- 172.21.15.146:0/2833421680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce24155c00 msgr2=0x7fce240cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:16.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.952+0000 7fce2886b700 1 --2- 172.21.15.146:0/2833421680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce24155c00 0x7fce240cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:16.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.956+0000 7fce2886b700 1 -- 172.21.15.146:0/2833421680 >> v2:172.21.15.154:3300/0 conn(0x7fce240a9490 msgr2=0x7fce240cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:16.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.956+0000 7fce23fff700 1 --2- 172.21.15.146:0/2833421680 >> v2:172.21.15.154:3300/0 conn(0x7fce240a9490 0x7fce240cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:16.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.956+0000 7fce2886b700 1 --2- 172.21.15.146:0/2833421680 >> v2:172.21.15.154:3300/0 conn(0x7fce240a9490 0x7fce240cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:16.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.956+0000 7fce2886b700 1 -- 172.21.15.146:0/2833421680 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce240c7e50 con 0x7fce2414f630 2024-09-13T18:54:16.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.956+0000 7fce2886b700 1 --2- 172.21.15.146:0/2833421680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce2414f630 0x7fce240c9500 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fce240c9a40 tx=0x7fce1c014ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ef1b3b9e0eae64 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:16.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.956+0000 7fce23fff700 1 --2- 172.21.15.146:0/2833421680 >> v2:172.21.15.154:3300/0 conn(0x7fce240a9490 0x7fce240cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:54:16.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.956+0000 7fce237fe700 1 -- 172.21.15.146:0/2833421680 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c013af0 con 0x7fce2414f630 2024-09-13T18:54:16.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.956+0000 7fce237fe700 1 -- 172.21.15.146:0/2833421680 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c013c50 con 0x7fce2414f630 2024-09-13T18:54:16.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.956+0000 7fce237fe700 1 -- 172.21.15.146:0/2833421680 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c013fe0 con 0x7fce2414f630 2024-09-13T18:54:16.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.956+0000 7fce2986d700 1 -- 172.21.15.146:0/2833421680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce2414f630 msgr2=0x7fce240c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:16.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.956+0000 7fce2986d700 1 --2- 172.21.15.146:0/2833421680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce2414f630 0x7fce240c9500 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fce240c9a40 tx=0x7fce1c014ce0 comp rx=0 tx=0).stop 2024-09-13T18:54:16.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.960+0000 7fce2986d700 1 -- 172.21.15.146:0/2833421680 shutdown_connections 2024-09-13T18:54:16.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.960+0000 7fce2986d700 1 --2- 172.21.15.146:0/2833421680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce24155c00 0x7fce240cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:16.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.960+0000 7fce2986d700 1 --2- 172.21.15.146:0/2833421680 >> v2:172.21.15.154:3300/0 conn(0x7fce240a9490 0x7fce240cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:16.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.960+0000 7fce2986d700 1 --2- 172.21.15.146:0/2833421680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce2414f630 0x7fce240c9500 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:16.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.960+0000 7fce2986d700 1 -- 172.21.15.146:0/2833421680 >> 172.21.15.146:0/2833421680 conn(0x7fce24066090 msgr2=0x7fce24066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:16.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.960+0000 7fce2986d700 1 -- 172.21.15.146:0/2833421680 shutdown_connections 2024-09-13T18:54:16.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.960+0000 7fce2986d700 1 -- 172.21.15.146:0/2833421680 wait complete. 2024-09-13T18:54:16.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.960+0000 7fce2986d700 1 Processor -- start 2024-09-13T18:54:16.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2986d700 1 -- start start 2024-09-13T18:54:16.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2986d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240a9490 0x7fce240ac090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:16.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2986d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce2414f630 0x7fce241ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:16.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2986d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce24155c00 0x7fce241ed3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:16.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2986d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce24072690 con 0x7fce240a9490 2024-09-13T18:54:16.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2986d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce241eee70 con 0x7fce2414f630 2024-09-13T18:54:16.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2986d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce241eeb90 con 0x7fce24155c00 2024-09-13T18:54:16.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2886b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240a9490 0x7fce240ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:16.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce23fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce2414f630 0x7fce241ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:16.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2886b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240a9490 0x7fce240ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40020/0 (socket says 172.21.15.146:40020) 2024-09-13T18:54:16.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2886b700 1 -- 172.21.15.146:0/2403321515 learned_addr learned my addr 172.21.15.146:0/2403321515 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:54:16.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2906c700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce24155c00 0x7fce241ed3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:16.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2886b700 1 -- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce24155c00 msgr2=0x7fce241ed3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:16.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2886b700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce24155c00 0x7fce241ed3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:16.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2886b700 1 -- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce2414f630 msgr2=0x7fce241ee1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:16.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2886b700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce2414f630 0x7fce241ee1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:16.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2886b700 1 -- 172.21.15.146:0/2403321515 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c014660 con 0x7fce240a9490 2024-09-13T18:54:16.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2906c700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce24155c00 0x7fce241ed3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:54:16.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.964+0000 7fce2886b700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240a9490 0x7fce240ac090 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fce1c016010 tx=0x7fce1c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:16.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.968+0000 7fce217fa700 1 -- 172.21.15.146:0/2403321515 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c004450 con 0x7fce240a9490 2024-09-13T18:54:16.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.968+0000 7fce2986d700 1 -- 172.21.15.146:0/2403321515 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce240c4b10 con 0x7fce240a9490 2024-09-13T18:54:16.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.968+0000 7fce2986d700 1 -- 172.21.15.146:0/2403321515 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce240aa950 con 0x7fce240a9490 2024-09-13T18:54:16.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.968+0000 7fce217fa700 1 -- 172.21.15.146:0/2403321515 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c0045b0 con 0x7fce240a9490 2024-09-13T18:54:16.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.968+0000 7fce217fa700 1 -- 172.21.15.146:0/2403321515 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c004920 con 0x7fce240a9490 2024-09-13T18:54:16.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.972+0000 7fce217fa700 1 -- 172.21.15.146:0/2403321515 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fce1c009030 con 0x7fce240a9490 2024-09-13T18:54:16.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.972+0000 7fce217fa700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fce10066800 0x7fce10068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:16.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.972+0000 7fce217fa700 1 -- 172.21.15.146:0/2403321515 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7fce1c086520 con 0x7fce240a9490 2024-09-13T18:54:16.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.976+0000 7fce23fff700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fce10066800 0x7fce10068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:16.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.976+0000 7fce2986d700 1 -- 172.21.15.146:0/2403321515 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2414fa10 con 0x7fce240a9490 2024-09-13T18:54:16.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.984+0000 7fce23fff700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fce10066800 0x7fce10068cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fce241e85a0 tx=0x7fce18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:17.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:16.992+0000 7fce217fa700 1 -- 172.21.15.146:0/2403321515 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fce1c052f10 con 0x7fce240a9490 2024-09-13T18:54:17.381 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:54:17.383 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:54:17.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce2986d700 1 -- 172.21.15.146:0/2403321515 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce240c5100 con 0x7fce240a9490 2024-09-13T18:54:17.550 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:54:17.550 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:54:14.801062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:active","state_seq":85,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:54:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce217fa700 1 -- 172.21.15.146:0/2403321515 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5536 (secure 0 0 0) 0x7fce1c021070 con 0x7fce240a9490 2024-09-13T18:54:17.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce2986d700 1 -- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fce10066800 msgr2=0x7fce10068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:17.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce2986d700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fce10066800 0x7fce10068cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fce241e85a0 tx=0x7fce18008040 comp rx=0 tx=0).stop 2024-09-13T18:54:17.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce2986d700 1 -- 172.21.15.146:0/2403321515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240a9490 msgr2=0x7fce240ac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:17.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce2986d700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240a9490 0x7fce240ac090 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fce1c016010 tx=0x7fce1c004af0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce2986d700 1 -- 172.21.15.146:0/2403321515 shutdown_connections 2024-09-13T18:54:17.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce2986d700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fce10066800 0x7fce10068cc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce2986d700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce24155c00 0x7fce241ed3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce2986d700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce2414f630 0x7fce241ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce2986d700 1 --2- 172.21.15.146:0/2403321515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce240a9490 0x7fce240ac090 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce2986d700 1 -- 172.21.15.146:0/2403321515 >> 172.21.15.146:0/2403321515 conn(0x7fce24066090 msgr2=0x7fce24073c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:17.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce2986d700 1 -- 172.21.15.146:0/2403321515 shutdown_connections 2024-09-13T18:54:17.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:17.536+0000 7fce2986d700 1 -- 172.21.15.146:0/2403321515 wait complete. 2024-09-13T18:54:17.572 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 111 2024-09-13T18:54:17.618 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=1, replacing gid=6714 2024-09-13T18:54:17.618 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.c 2024-09-13T18:54:17.754 INFO:tasks.ceph.mds.a.smithi146.stderr:2024-09-13T18:54:17.740+0000 7f40aeb2a700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-13T18:54:19.619 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-09-13T18:54:19.620 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T18:54:19.620 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T18:54:19.623 INFO:tasks.ceph.mds.c:Started 2024-09-13T18:54:19.623 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:19.854 INFO:tasks.ceph.mds.c.smithi146.stdout:starting mds.c at 2024-09-13T18:54:20.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.100+0000 7f5cdc134700 1 Processor -- start 2024-09-13T18:54:20.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.108+0000 7f5cdc134700 1 -- start start 2024-09-13T18:54:20.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.108+0000 7f5cdc134700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd40c9500 0x7f5cd4155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:20.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.108+0000 7f5cdc134700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5cd4156140 0x7f5cd40cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:20.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.108+0000 7f5cdc134700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd406f5c0 con 0x7f5cd414f630 2024-09-13T18:54:20.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.108+0000 7f5cdc134700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd40c9920 con 0x7f5cd4156140 2024-09-13T18:54:20.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.108+0000 7f5cdc134700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4155420 con 0x7f5cd40c9500 2024-09-13T18:54:20.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.112+0000 7f5cda6d1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5cd4156140 0x7f5cd40cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:20.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.112+0000 7f5cda6d1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5cd4156140 0x7f5cd40cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40028/0 (socket says 172.21.15.146:40028) 2024-09-13T18:54:20.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.112+0000 7f5cda6d1700 1 -- 172.21.15.146:0/2813985926 learned_addr learned my addr 172.21.15.146:0/2813985926 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:54:20.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.116+0000 7f5cda6d1700 1 -- 172.21.15.146:0/2813985926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd40c9500 msgr2=0x7f5cd4155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:20.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.116+0000 7f5cda6d1700 1 --2- 172.21.15.146:0/2813985926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd40c9500 0x7f5cd4155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:20.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.116+0000 7f5cda6d1700 1 -- 172.21.15.146:0/2813985926 >> v1:172.21.15.154:6789/0 conn(0x7f5cd414f630 legacy=0x7f5cd40a9490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:54:20.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.116+0000 7f5cda6d1700 1 -- 172.21.15.146:0/2813985926 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd40c7b70 con 0x7f5cd4156140 2024-09-13T18:54:20.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.116+0000 7f5cda6d1700 1 --2- 172.21.15.146:0/2813985926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd4156140 0x7f5cd40cb120 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009f90 tx=0x7f5cd0009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd21b6758c5556a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:20.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.116+0000 7f5cd8ece700 1 -- 172.21.15.146:0/2813985926 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00098b0 con 0x7f5cd4156140 2024-09-13T18:54:20.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.116+0000 7f5cd8ece700 1 -- 172.21.15.146:0/2813985926 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd0013c90 con 0x7f5cd4156140 2024-09-13T18:54:20.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.116+0000 7f5cdc134700 1 -- 172.21.15.146:0/2813985926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd4156140 msgr2=0x7f5cd40cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:20.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.116+0000 7f5cd8ece700 1 -- 172.21.15.146:0/2813985926 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00098b0 con 0x7f5cd4156140 2024-09-13T18:54:20.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.116+0000 7f5cdc134700 1 --2- 172.21.15.146:0/2813985926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd4156140 0x7f5cd40cb120 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009f90 tx=0x7f5cd0009670 comp rx=0 tx=0).stop 2024-09-13T18:54:20.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.124+0000 7f5cdc134700 1 -- 172.21.15.146:0/2813985926 shutdown_connections 2024-09-13T18:54:20.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.124+0000 7f5cdc134700 1 --2- 172.21.15.146:0/2813985926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd4156140 0x7f5cd40cb120 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:20.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.124+0000 7f5cdc134700 1 --2- 172.21.15.146:0/2813985926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd40c9500 0x7f5cd4155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:20.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.124+0000 7f5cdc134700 1 -- 172.21.15.146:0/2813985926 >> 172.21.15.146:0/2813985926 conn(0x7f5cd4066090 msgr2=0x7f5cd4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:20.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.124+0000 7f5cdc134700 1 -- 172.21.15.146:0/2813985926 shutdown_connections 2024-09-13T18:54:20.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.124+0000 7f5cdc134700 1 -- 172.21.15.146:0/2813985926 wait complete. 2024-09-13T18:54:20.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.128+0000 7f5cdc134700 1 Processor -- start 2024-09-13T18:54:20.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.132+0000 7f5cdc134700 1 -- start start 2024-09-13T18:54:20.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cdc134700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd40c9500 0x7f5cd41ee340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:20.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cdc134700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd414f630 0x7f5cd40aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:20.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cdc134700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd4156140 0x7f5cd414dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:20.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cdc134700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40ab9e0 con 0x7f5cd4156140 2024-09-13T18:54:20.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cdc134700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd406e5c0 con 0x7f5cd40c9500 2024-09-13T18:54:20.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cdc134700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4153690 con 0x7f5cd414f630 2024-09-13T18:54:20.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cda6d1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd4156140 0x7f5cd414dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:20.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cda6d1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd4156140 0x7f5cd414dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40042/0 (socket says 172.21.15.146:40042) 2024-09-13T18:54:20.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cda6d1700 1 -- 172.21.15.146:0/2577528128 learned_addr learned my addr 172.21.15.146:0/2577528128 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:54:20.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cd9ed0700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd40c9500 0x7f5cd41ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:20.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cd96cf700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd414f630 0x7f5cd40aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:20.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cda6d1700 1 -- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd414f630 msgr2=0x7f5cd40aaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:20.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cda6d1700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd414f630 0x7f5cd40aaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:20.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cda6d1700 1 -- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd40c9500 msgr2=0x7f5cd41ee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:20.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cda6d1700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd40c9500 0x7f5cd41ee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:20.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cda6d1700 1 -- 172.21.15.146:0/2577528128 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd0009e00 con 0x7f5cd4156140 2024-09-13T18:54:20.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cda6d1700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd4156140 0x7f5cd414dc40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0004000 tx=0x7f5cd0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:20.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cca7fc700 1 -- 172.21.15.146:0/2577528128 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0013e90 con 0x7f5cd4156140 2024-09-13T18:54:20.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.136+0000 7f5cdc134700 1 -- 172.21.15.146:0/2577528128 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd41533d0 con 0x7f5cd4156140 2024-09-13T18:54:20.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.140+0000 7f5cdc134700 1 -- 172.21.15.146:0/2577528128 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd406e2e0 con 0x7f5cd4156140 2024-09-13T18:54:20.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.140+0000 7f5cca7fc700 1 -- 172.21.15.146:0/2577528128 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd0003c40 con 0x7f5cd4156140 2024-09-13T18:54:20.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.140+0000 7f5cca7fc700 1 -- 172.21.15.146:0/2577528128 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0013130 con 0x7f5cd4156140 2024-09-13T18:54:20.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.140+0000 7f5cca7fc700 1 -- 172.21.15.146:0/2577528128 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5cd0013290 con 0x7f5cd4156140 2024-09-13T18:54:20.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.144+0000 7f5cca7fc700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cb8066830 0x7f5cb8068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:20.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.144+0000 7f5cca7fc700 1 -- 172.21.15.146:0/2577528128 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7f5cd00a7bc0 con 0x7f5cd4156140 2024-09-13T18:54:20.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.144+0000 7f5cdc134700 1 -- 172.21.15.146:0/2577528128 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd40a99e0 con 0x7f5cd4156140 2024-09-13T18:54:20.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.144+0000 7f5cd9ed0700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cb8066830 0x7f5cb8068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:20.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.164+0000 7f5cca7fc700 1 -- 172.21.15.146:0/2577528128 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5cd0074650 con 0x7f5cd4156140 2024-09-13T18:54:20.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:20.164+0000 7f5cd9ed0700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cb8066830 0x7f5cb8068cf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009fd0 tx=0x7f5ccc00d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:21.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.176+0000 7f5cdc134700 1 -- 172.21.15.146:0/2577528128 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cd414fa10 con 0x7f5cd4156140 2024-09-13T18:54:21.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.176+0000 7f5cca7fc700 1 -- 172.21.15.146:0/2577528128 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+6297 (secure 0 0 0) 0x7f5cd0077c70 con 0x7f5cd4156140 2024-09-13T18:54:21.196 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:54:21.196 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":112,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:54:14.801062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:active","state_seq":85,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:54:21.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.196+0000 7f5cdc134700 1 -- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cb8066830 msgr2=0x7f5cb8068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:21.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.196+0000 7f5cdc134700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cb8066830 0x7f5cb8068cf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009fd0 tx=0x7f5ccc00d040 comp rx=0 tx=0).stop 2024-09-13T18:54:21.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.196+0000 7f5cdc134700 1 -- 172.21.15.146:0/2577528128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd4156140 msgr2=0x7f5cd414dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:21.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.196+0000 7f5cdc134700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd4156140 0x7f5cd414dc40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0004000 tx=0x7f5cd0004770 comp rx=0 tx=0).stop 2024-09-13T18:54:21.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.196+0000 7f5cdc134700 1 -- 172.21.15.146:0/2577528128 shutdown_connections 2024-09-13T18:54:21.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.196+0000 7f5cdc134700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cb8066830 0x7f5cb8068cf0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:21.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.196+0000 7f5cdc134700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd4156140 0x7f5cd414dc40 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:21.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.196+0000 7f5cdc134700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd414f630 0x7f5cd40aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:21.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.196+0000 7f5cdc134700 1 --2- 172.21.15.146:0/2577528128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd40c9500 0x7f5cd41ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:21.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.196+0000 7f5cdc134700 1 -- 172.21.15.146:0/2577528128 >> 172.21.15.146:0/2577528128 conn(0x7f5cd4066090 msgr2=0x7f5cd40c5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:21.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.196+0000 7f5cdc134700 1 -- 172.21.15.146:0/2577528128 shutdown_connections 2024-09-13T18:54:21.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:54:21.196+0000 7f5cdc134700 1 -- 172.21.15.146:0/2577528128 wait complete. 2024-09-13T18:54:21.213 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 112 2024-09-13T18:54:21.264 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-09-13T18:54:21.265 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 44 secs before thrashing 2024-09-13T18:54:47.695 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:54:47.698 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:55:05.266 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:06.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.005+0000 7f8a5d633700 1 Processor -- start 2024-09-13T18:55:06.019 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.005+0000 7f8a5d633700 1 -- start start 2024-09-13T18:55:06.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.005+0000 7f8a5d633700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8a580cb2b0 0x7f8a58159e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:06.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.005+0000 7f8a5d633700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8a5815a350 0x7f8a58156a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:06.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.005+0000 7f8a5d633700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a580ca390 con 0x7f8a580caed0 2024-09-13T18:55:06.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.005+0000 7f8a5d633700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a58159980 con 0x7f8a5815a350 2024-09-13T18:55:06.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.005+0000 7f8a5d633700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a580c7480 con 0x7f8a580cb2b0 2024-09-13T18:55:06.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.005+0000 7f8a577fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8a580cb2b0 0x7f8a58159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:06.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.009+0000 7f8a577fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8a580cb2b0 0x7f8a58159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:60754/0 (socket says 172.21.15.146:60754) 2024-09-13T18:55:06.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.009+0000 7f8a577fe700 1 -- 172.21.15.146:0/959286138 learned_addr learned my addr 172.21.15.146:0/959286138 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:06.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.009+0000 7f8a5ce32700 1 --2- 172.21.15.146:0/959286138 >> v2:172.21.15.146:3300/0 conn(0x7f8a5815a350 0x7f8a58156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:06.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.009+0000 7f8a577fe700 1 -- 172.21.15.146:0/959286138 >> v2:172.21.15.146:3300/0 conn(0x7f8a5815a350 msgr2=0x7f8a58156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:06.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.009+0000 7f8a577fe700 1 --2- 172.21.15.146:0/959286138 >> v2:172.21.15.146:3300/0 conn(0x7f8a5815a350 0x7f8a58156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:06.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.009+0000 7f8a577fe700 1 -- 172.21.15.146:0/959286138 >> v1:172.21.15.154:6789/0 conn(0x7f8a580caed0 legacy=0x7f8a58072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:06.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.009+0000 7f8a577fe700 1 -- 172.21.15.146:0/959286138 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580a9c80 con 0x7f8a580cb2b0 2024-09-13T18:55:06.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.009+0000 7f8a5ce32700 1 --2- 172.21.15.146:0/959286138 >> v2:172.21.15.146:3300/0 conn(0x7f8a5815a350 0x7f8a58156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:55:06.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.009+0000 7f8a577fe700 1 --2- 172.21.15.146:0/959286138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a580cb2b0 0x7f8a58159e10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8a5806c490 tx=0x7f8a5000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5da3770fb764583 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:06.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.033+0000 7f8a56ffd700 1 -- 172.21.15.146:0/959286138 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000e070 con 0x7f8a580cb2b0 2024-09-13T18:55:06.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.033+0000 7f8a56ffd700 1 -- 172.21.15.146:0/959286138 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a50003a40 con 0x7f8a580cb2b0 2024-09-13T18:55:06.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.037+0000 7f8a5d633700 1 -- 172.21.15.146:0/959286138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a580cb2b0 msgr2=0x7f8a58159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:06.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.037+0000 7f8a5d633700 1 --2- 172.21.15.146:0/959286138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a580cb2b0 0x7f8a58159e10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8a5806c490 tx=0x7f8a5000b330 comp rx=0 tx=0).stop 2024-09-13T18:55:06.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.041+0000 7f8a5d633700 1 -- 172.21.15.146:0/959286138 shutdown_connections 2024-09-13T18:55:06.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.041+0000 7f8a5d633700 1 --2- 172.21.15.146:0/959286138 >> v2:172.21.15.146:3300/0 conn(0x7f8a5815a350 0x7f8a58156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:06.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.041+0000 7f8a5d633700 1 --2- 172.21.15.146:0/959286138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a580cb2b0 0x7f8a58159e10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:06.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.041+0000 7f8a5d633700 1 -- 172.21.15.146:0/959286138 >> 172.21.15.146:0/959286138 conn(0x7f8a58066090 msgr2=0x7f8a58066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:06.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.045+0000 7f8a5d633700 1 -- 172.21.15.146:0/959286138 shutdown_connections 2024-09-13T18:55:06.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.045+0000 7f8a5d633700 1 -- 172.21.15.146:0/959286138 wait complete. 2024-09-13T18:55:06.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.049+0000 7f8a5d633700 1 Processor -- start 2024-09-13T18:55:06.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.049+0000 7f8a5d633700 1 -- start start 2024-09-13T18:55:06.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a5d633700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a580caed0 0x7f8a581efe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:06.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a5d633700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a580cb2b0 0x7f8a5814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:06.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a5d633700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a5815a350 0x7f8a5806c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:06.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a5d633700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a581e52c0 con 0x7f8a580caed0 2024-09-13T18:55:06.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a5d633700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a581f0380 con 0x7f8a5815a350 2024-09-13T18:55:06.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a5d633700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580af440 con 0x7f8a580cb2b0 2024-09-13T18:55:06.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a577fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a580cb2b0 0x7f8a5814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:06.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a577fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a580cb2b0 0x7f8a5814e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:46202/0 (socket says 172.21.15.146:46202) 2024-09-13T18:55:06.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a577fe700 1 -- 172.21.15.146:0/3800207068 learned_addr learned my addr 172.21.15.146:0/3800207068 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:06.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a577fe700 1 -- 172.21.15.146:0/3800207068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a5815a350 msgr2=0x7f8a5806c500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:55:06.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a577fe700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a5815a350 0x7f8a5806c500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:06.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a577fe700 1 -- 172.21.15.146:0/3800207068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a580caed0 msgr2=0x7f8a581efe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:06.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a57fff700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a580caed0 0x7f8a581efe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:06.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a577fe700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a580caed0 0x7f8a581efe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:06.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a577fe700 1 -- 172.21.15.146:0/3800207068 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c0094b0 con 0x7f8a580cb2b0 2024-09-13T18:55:06.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a57fff700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a580caed0 0x7f8a581efe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:55:06.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a577fe700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a580cb2b0 0x7f8a5814e4c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000b370 tx=0x7f8a50004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:06.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a54ff9700 1 -- 172.21.15.146:0/3800207068 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500048d0 con 0x7f8a580cb2b0 2024-09-13T18:55:06.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a54ff9700 1 -- 172.21.15.146:0/3800207068 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a500092e0 con 0x7f8a580cb2b0 2024-09-13T18:55:06.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a5d633700 1 -- 172.21.15.146:0/3800207068 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a50009b30 con 0x7f8a580cb2b0 2024-09-13T18:55:06.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.053+0000 7f8a5d633700 1 -- 172.21.15.146:0/3800207068 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a580af6e0 con 0x7f8a580cb2b0 2024-09-13T18:55:06.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.057+0000 7f8a54ff9700 1 -- 172.21.15.146:0/3800207068 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50012650 con 0x7f8a580cb2b0 2024-09-13T18:55:06.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.057+0000 7f8a54ff9700 1 -- 172.21.15.146:0/3800207068 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8a500127b0 con 0x7f8a580cb2b0 2024-09-13T18:55:06.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.057+0000 7f8a54ff9700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8a44066830 0x7f8a44068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:06.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.057+0000 7f8a54ff9700 1 -- 172.21.15.146:0/3800207068 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7f8a50085f60 con 0x7f8a580cb2b0 2024-09-13T18:55:06.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.057+0000 7f8a57fff700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8a44066830 0x7f8a44068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:06.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.061+0000 7f8a57fff700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8a44066830 0x7f8a44068cf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8a48001900 tx=0x7f8a48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:06.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.065+0000 7f8a5d633700 1 -- 172.21.15.146:0/3800207068 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a38005310 con 0x7f8a580cb2b0 2024-09-13T18:55:06.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:06.081+0000 7f8a54ff9700 1 -- 172.21.15.146:0/3800207068 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8a500b1db0 con 0x7f8a580cb2b0 2024-09-13T18:55:07.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.125+0000 7f8a5d633700 1 -- 172.21.15.146:0/3800207068 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a38005b40 con 0x7f8a580cb2b0 2024-09-13T18:55:07.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.125+0000 7f8a54ff9700 1 -- 172.21.15.146:0/3800207068 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+6297 (secure 0 0 0) 0x7f8a50017030 con 0x7f8a580cb2b0 2024-09-13T18:55:07.143 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:07.143 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":112,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:54:14.801062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:active","state_seq":85,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:07.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.149+0000 7f8a3e7fc700 1 -- 172.21.15.146:0/3800207068 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8a44066830 msgr2=0x7f8a44068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:07.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.149+0000 7f8a3e7fc700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8a44066830 0x7f8a44068cf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8a48001900 tx=0x7f8a48009040 comp rx=0 tx=0).stop 2024-09-13T18:55:07.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.149+0000 7f8a3e7fc700 1 -- 172.21.15.146:0/3800207068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a580cb2b0 msgr2=0x7f8a5814e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:07.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.149+0000 7f8a3e7fc700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a580cb2b0 0x7f8a5814e4c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000b370 tx=0x7f8a50004af0 comp rx=0 tx=0).stop 2024-09-13T18:55:07.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.149+0000 7f8a3e7fc700 1 -- 172.21.15.146:0/3800207068 shutdown_connections 2024-09-13T18:55:07.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.149+0000 7f8a3e7fc700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8a44066830 0x7f8a44068cf0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:07.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.149+0000 7f8a3e7fc700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a5815a350 0x7f8a5806c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:07.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.149+0000 7f8a3e7fc700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a580cb2b0 0x7f8a5814e4c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:07.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.149+0000 7f8a3e7fc700 1 --2- 172.21.15.146:0/3800207068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a580caed0 0x7f8a581efe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:07.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.149+0000 7f8a3e7fc700 1 -- 172.21.15.146:0/3800207068 >> 172.21.15.146:0/3800207068 conn(0x7f8a58066090 msgr2=0x7f8a580c6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:07.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.149+0000 7f8a3e7fc700 1 -- 172.21.15.146:0/3800207068 shutdown_connections 2024-09-13T18:55:07.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.149+0000 7f8a3e7fc700 1 -- 172.21.15.146:0/3800207068 wait complete. 2024-09-13T18:55:07.184 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 112 2024-09-13T18:55:07.220 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=1) 2024-09-13T18:55:07.220 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-09-13T18:55:07.221 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:07.233 INFO:tasks.ceph.mds.f:Stopped 2024-09-13T18:55:07.233 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:07.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.645+0000 7ffb4acf6700 1 Processor -- start 2024-09-13T18:55:07.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.645+0000 7ffb4acf6700 1 -- start start 2024-09-13T18:55:07.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.645+0000 7ffb4acf6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffb440caed0 0x7ffb44072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:07.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.645+0000 7ffb4acf6700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb44161060 con 0x7ffb440cb2b0 2024-09-13T18:55:07.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.645+0000 7ffb4acf6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb44161300 con 0x7ffb4415e170 2024-09-13T18:55:07.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.645+0000 7ffb4acf6700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb440c78d0 con 0x7ffb440caed0 2024-09-13T18:55:07.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.645+0000 7ffb494f3700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ffb440cb2b0 0x7ffb440c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:46486/0 (socket says 172.21.15.146:46486) 2024-09-13T18:55:07.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.645+0000 7ffb494f3700 1 -- 172.21.15.146:0/473347903 learned_addr learned my addr 172.21.15.146:0/473347903 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:55:07.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.645+0000 7ffb49cf4700 1 --2- 172.21.15.146:0/473347903 >> v2:172.21.15.146:3300/0 conn(0x7ffb440caed0 0x7ffb44072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:07.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.657+0000 7ffb49cf4700 1 -- 172.21.15.146:0/473347903 >> v1:172.21.15.154:6789/0 conn(0x7ffb4415e170 legacy=0x7ffb4415e5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:55:07.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.657+0000 7ffb49cf4700 1 -- 172.21.15.146:0/473347903 >> v1:172.21.15.146:6789/0 conn(0x7ffb440cb2b0 legacy=0x7ffb440c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:07.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.657+0000 7ffb49cf4700 1 -- 172.21.15.146:0/473347903 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb441615c0 con 0x7ffb440caed0 2024-09-13T18:55:07.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.657+0000 7ffb49cf4700 1 --2- 172.21.15.146:0/473347903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb440caed0 0x7ffb44072380 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009ae0 tx=0x7ffb40009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a82306c886a442d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:07.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.665+0000 7ffb48cf2700 1 -- 172.21.15.146:0/473347903 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4000b810 con 0x7ffb440caed0 2024-09-13T18:55:07.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.665+0000 7ffb48cf2700 1 -- 172.21.15.146:0/473347903 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4000b970 con 0x7ffb440caed0 2024-09-13T18:55:07.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.665+0000 7ffb4acf6700 1 -- 172.21.15.146:0/473347903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb440caed0 msgr2=0x7ffb44072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:07.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.665+0000 7ffb4acf6700 1 --2- 172.21.15.146:0/473347903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb440caed0 0x7ffb44072380 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009ae0 tx=0x7ffb40009df0 comp rx=0 tx=0).stop 2024-09-13T18:55:07.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.673+0000 7ffb4acf6700 1 -- 172.21.15.146:0/473347903 shutdown_connections 2024-09-13T18:55:07.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.673+0000 7ffb4acf6700 1 --2- 172.21.15.146:0/473347903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb440caed0 0x7ffb44072380 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:07.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.673+0000 7ffb4acf6700 1 -- 172.21.15.146:0/473347903 >> 172.21.15.146:0/473347903 conn(0x7ffb44066090 msgr2=0x7ffb44066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:07.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.673+0000 7ffb4acf6700 1 -- 172.21.15.146:0/473347903 shutdown_connections 2024-09-13T18:55:07.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.673+0000 7ffb4acf6700 1 -- 172.21.15.146:0/473347903 wait complete. 2024-09-13T18:55:07.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.673+0000 7ffb4acf6700 1 Processor -- start 2024-09-13T18:55:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4acf6700 1 -- start start 2024-09-13T18:55:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4acf6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb440caed0 0x7ffb440ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4acf6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb440cb2b0 0x7ffb44145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4acf6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4415e170 0x7ffb4414d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4acf6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb440acd90 con 0x7ffb4415e170 2024-09-13T18:55:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4acf6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4406d220 con 0x7ffb440cb2b0 2024-09-13T18:55:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4acf6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb440ab300 con 0x7ffb440caed0 2024-09-13T18:55:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4a4f5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4415e170 0x7ffb4414d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb494f3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb440cb2b0 0x7ffb44145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4a4f5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4415e170 0x7ffb4414d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51712/0 (socket says 172.21.15.146:51712) 2024-09-13T18:55:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb494f3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb440cb2b0 0x7ffb44145ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:60764/0 (socket says 172.21.15.146:60764) 2024-09-13T18:55:07.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4a4f5700 1 -- 172.21.15.146:0/382646863 learned_addr learned my addr 172.21.15.146:0/382646863 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:07.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb49cf4700 1 --2- 172.21.15.146:0/382646863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb440caed0 0x7ffb440ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:07.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4a4f5700 1 -- 172.21.15.146:0/382646863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb440caed0 msgr2=0x7ffb440ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:07.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4a4f5700 1 --2- 172.21.15.146:0/382646863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb440caed0 0x7ffb440ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:07.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4a4f5700 1 -- 172.21.15.146:0/382646863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb440cb2b0 msgr2=0x7ffb44145ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:07.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4a4f5700 1 --2- 172.21.15.146:0/382646863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb440cb2b0 0x7ffb44145ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:07.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4a4f5700 1 -- 172.21.15.146:0/382646863 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400094b0 con 0x7ffb4415e170 2024-09-13T18:55:07.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.677+0000 7ffb4a4f5700 1 --2- 172.21.15.146:0/382646863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4415e170 0x7ffb4414d0c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ffb3400b410 tx=0x7ffb3400b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:07.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.689+0000 7ffb3a7fc700 1 -- 172.21.15.146:0/382646863 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb34011830 con 0x7ffb4415e170 2024-09-13T18:55:07.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.689+0000 7ffb4acf6700 1 -- 172.21.15.146:0/382646863 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb440ad1b0 con 0x7ffb4415e170 2024-09-13T18:55:07.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.689+0000 7ffb4acf6700 1 -- 172.21.15.146:0/382646863 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb440ad770 con 0x7ffb4415e170 2024-09-13T18:55:07.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.697+0000 7ffb3a7fc700 1 -- 172.21.15.146:0/382646863 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb340119b0 con 0x7ffb4415e170 2024-09-13T18:55:07.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.697+0000 7ffb3a7fc700 1 -- 172.21.15.146:0/382646863 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb34011d60 con 0x7ffb4415e170 2024-09-13T18:55:07.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.697+0000 7ffb3a7fc700 1 -- 172.21.15.146:0/382646863 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffb3401a400 con 0x7ffb4415e170 2024-09-13T18:55:07.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.697+0000 7ffb3a7fc700 1 --2- 172.21.15.146:0/382646863 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb28066880 0x7ffb28068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:07.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.697+0000 7ffb3a7fc700 1 -- 172.21.15.146:0/382646863 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7ffb34084ae0 con 0x7ffb4415e170 2024-09-13T18:55:07.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.697+0000 7ffb49cf4700 1 --2- 172.21.15.146:0/382646863 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb28066880 0x7ffb28068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:07.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.697+0000 7ffb4acf6700 1 -- 172.21.15.146:0/382646863 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2c005310 con 0x7ffb4415e170 2024-09-13T18:55:07.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.697+0000 7ffb49cf4700 1 --2- 172.21.15.146:0/382646863 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb28066880 0x7ffb28068d40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffb40013040 tx=0x7ffb40008000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:07.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:07.713+0000 7ffb3a7fc700 1 -- 172.21.15.146:0/382646863 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ffb34015080 con 0x7ffb4415e170 2024-09-13T18:55:08.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.201+0000 7ffb4acf6700 1 -- 172.21.15.146:0/382646863 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb2c005b40 con 0x7ffb4415e170 2024-09-13T18:55:08.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.205+0000 7ffb3a7fc700 1 -- 172.21.15.146:0/382646863 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+6297 (secure 0 0 0) 0x7ffb34051560 con 0x7ffb4415e170 2024-09-13T18:55:08.227 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:08.227 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":112,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:54:14.801062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:active","state_seq":85,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:08.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.225+0000 7ffb27fff700 1 -- 172.21.15.146:0/382646863 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb28066880 msgr2=0x7ffb28068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:08.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.225+0000 7ffb27fff700 1 --2- 172.21.15.146:0/382646863 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb28066880 0x7ffb28068d40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffb40013040 tx=0x7ffb40008000 comp rx=0 tx=0).stop 2024-09-13T18:55:08.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.225+0000 7ffb27fff700 1 -- 172.21.15.146:0/382646863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4415e170 msgr2=0x7ffb4414d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:08.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.225+0000 7ffb27fff700 1 --2- 172.21.15.146:0/382646863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4415e170 0x7ffb4414d0c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ffb3400b410 tx=0x7ffb3400b720 comp rx=0 tx=0).stop 2024-09-13T18:55:08.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.225+0000 7ffb27fff700 1 -- 172.21.15.146:0/382646863 shutdown_connections 2024-09-13T18:55:08.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.225+0000 7ffb27fff700 1 --2- 172.21.15.146:0/382646863 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7ffb28066880 0x7ffb28068d40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:08.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.225+0000 7ffb27fff700 1 --2- 172.21.15.146:0/382646863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb4415e170 0x7ffb4414d0c0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:08.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.225+0000 7ffb27fff700 1 --2- 172.21.15.146:0/382646863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb440cb2b0 0x7ffb44145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:08.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.225+0000 7ffb27fff700 1 --2- 172.21.15.146:0/382646863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb440caed0 0x7ffb440ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:08.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.225+0000 7ffb27fff700 1 -- 172.21.15.146:0/382646863 >> 172.21.15.146:0/382646863 conn(0x7ffb44066090 msgr2=0x7ffb4406dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:08.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.229+0000 7ffb27fff700 1 -- 172.21.15.146:0/382646863 shutdown_connections 2024-09-13T18:55:08.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:08.229+0000 7ffb27fff700 1 -- 172.21.15.146:0/382646863 wait complete. 2024-09-13T18:55:08.248 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 112 2024-09-13T18:55:08.272 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:55:10.273 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:10.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.637+0000 7f5d6c33b700 1 Processor -- start 2024-09-13T18:55:10.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.637+0000 7f5d6c33b700 1 -- start start 2024-09-13T18:55:10.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6c33b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5d640cac60 0x7f5d640cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:10.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6c33b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5d6416e0c0 0x7f5d641704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:10.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6c33b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d640a9920 con 0x7f5d6408baa0 2024-09-13T18:55:10.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6c33b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d640723f0 con 0x7f5d640cac60 2024-09-13T18:55:10.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6c33b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d640cb580 con 0x7f5d6416e0c0 2024-09-13T18:55:10.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d698d6700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5d6408baa0 0x7f5d6406f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:46490/0 (socket says 172.21.15.146:46490) 2024-09-13T18:55:10.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d698d6700 1 -- 172.21.15.146:0/4232567117 learned_addr learned my addr 172.21.15.146:0/4232567117 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:55:10.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6a0d7700 1 --2- 172.21.15.146:0/4232567117 >> v2:172.21.15.146:3300/0 conn(0x7f5d640cac60 0x7f5d640cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:10.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6a0d7700 1 -- 172.21.15.146:0/4232567117 >> v2:172.21.15.154:3300/0 conn(0x7f5d6416e0c0 msgr2=0x7f5d641704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:10.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6a0d7700 1 --2- 172.21.15.146:0/4232567117 >> v2:172.21.15.154:3300/0 conn(0x7f5d6416e0c0 0x7f5d641704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:10.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6a0d7700 1 -- 172.21.15.146:0/4232567117 >> v1:172.21.15.146:6789/0 conn(0x7f5d6408baa0 legacy=0x7f5d6406f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:10.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6a0d7700 1 -- 172.21.15.146:0/4232567117 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d64070070 con 0x7f5d640cac60 2024-09-13T18:55:10.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6a0d7700 1 --2- 172.21.15.146:0/4232567117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d640cac60 0x7f5d640cb040 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5d640c96e0 tx=0x7f5d6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798747375a8dc172 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:10.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d690d5700 1 -- 172.21.15.146:0/4232567117 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60004230 con 0x7f5d640cac60 2024-09-13T18:55:10.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d690d5700 1 -- 172.21.15.146:0/4232567117 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d60004390 con 0x7f5d640cac60 2024-09-13T18:55:10.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4232567117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d640cac60 msgr2=0x7f5d640cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:10.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6c33b700 1 --2- 172.21.15.146:0/4232567117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d640cac60 0x7f5d640cb040 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5d640c96e0 tx=0x7f5d6000b040 comp rx=0 tx=0).stop 2024-09-13T18:55:10.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4232567117 shutdown_connections 2024-09-13T18:55:10.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6c33b700 1 --2- 172.21.15.146:0/4232567117 >> v2:172.21.15.154:3300/0 conn(0x7f5d6416e0c0 0x7f5d641704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:10.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6c33b700 1 --2- 172.21.15.146:0/4232567117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d640cac60 0x7f5d640cb040 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:10.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4232567117 >> 172.21.15.146:0/4232567117 conn(0x7f5d64066090 msgr2=0x7f5d64066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:10.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4232567117 shutdown_connections 2024-09-13T18:55:10.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.641+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4232567117 wait complete. 2024-09-13T18:55:10.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6c33b700 1 Processor -- start 2024-09-13T18:55:10.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6c33b700 1 -- start start 2024-09-13T18:55:10.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6c33b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6408baa0 0x7f5d6416a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:10.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6c33b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d6416e0c0 0x7f5d640ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:10.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6c33b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d6416c6b0 0x7f5d641feb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:10.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6c33b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6416ba10 con 0x7f5d6408baa0 2024-09-13T18:55:10.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6c33b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d640c19b0 con 0x7f5d6416c6b0 2024-09-13T18:55:10.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6c33b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d640ad470 con 0x7f5d6416e0c0 2024-09-13T18:55:10.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6a0d7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6408baa0 0x7f5d6416a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:10.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6a0d7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6408baa0 0x7f5d6416a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51730/0 (socket says 172.21.15.146:51730) 2024-09-13T18:55:10.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6a0d7700 1 -- 172.21.15.146:0/4050308641 learned_addr learned my addr 172.21.15.146:0/4050308641 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:10.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d698d6700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d6416e0c0 0x7f5d640ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:10.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6a8d8700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d6416c6b0 0x7f5d641feb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:10.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6a0d7700 1 -- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d6416e0c0 msgr2=0x7f5d640ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:10.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6a0d7700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d6416e0c0 0x7f5d640ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:10.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6a0d7700 1 -- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d6416c6b0 msgr2=0x7f5d641feb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:10.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6a0d7700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d6416c6b0 0x7f5d641feb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:10.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6a0d7700 1 -- 172.21.15.146:0/4050308641 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d60008530 con 0x7f5d6408baa0 2024-09-13T18:55:10.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d6a0d7700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6408baa0 0x7f5d6416a810 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5d60004740 tx=0x7f5d60004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:10.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.645+0000 7f5d5affd700 1 -- 172.21.15.146:0/4050308641 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60004050 con 0x7f5d6408baa0 2024-09-13T18:55:10.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.649+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4050308641 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d640ad750 con 0x7f5d6408baa0 2024-09-13T18:55:10.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.649+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4050308641 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6416bf40 con 0x7f5d6408baa0 2024-09-13T18:55:10.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.649+0000 7f5d5affd700 1 -- 172.21.15.146:0/4050308641 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d60003ad0 con 0x7f5d6408baa0 2024-09-13T18:55:10.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.649+0000 7f5d5affd700 1 -- 172.21.15.146:0/4050308641 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60003e60 con 0x7f5d6408baa0 2024-09-13T18:55:10.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.649+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4050308641 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d640c9da0 con 0x7f5d6408baa0 2024-09-13T18:55:10.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.657+0000 7f5d5affd700 1 -- 172.21.15.146:0/4050308641 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5d600124b0 con 0x7f5d6408baa0 2024-09-13T18:55:10.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.661+0000 7f5d5affd700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d48066800 0x7f5d48068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:10.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.661+0000 7f5d698d6700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d48066800 0x7f5d48068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:10.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.665+0000 7f5d698d6700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d48066800 0x7f5d48068cc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5d6416cc90 tx=0x7f5d5c001be0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:10.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.665+0000 7f5d5affd700 1 -- 172.21.15.146:0/4050308641 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7f5d60096e40 con 0x7f5d6408baa0 2024-09-13T18:55:10.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:10.669+0000 7f5d5affd700 1 -- 172.21.15.146:0/4050308641 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5d60063950 con 0x7f5d6408baa0 2024-09-13T18:55:11.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.893+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4050308641 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d640a9fd0 con 0x7f5d6408baa0 2024-09-13T18:55:11.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.893+0000 7f5d5affd700 1 -- 172.21.15.146:0/4050308641 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+6297 (secure 0 0 0) 0x7f5d60066f70 con 0x7f5d6408baa0 2024-09-13T18:55:11.909 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:11.909 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":112,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:54:14.801062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:active","state_seq":85,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:11.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.917+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d48066800 msgr2=0x7f5d48068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:11.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.917+0000 7f5d6c33b700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d48066800 0x7f5d48068cc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5d6416cc90 tx=0x7f5d5c001be0 comp rx=0 tx=0).stop 2024-09-13T18:55:11.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.917+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4050308641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6408baa0 msgr2=0x7f5d6416a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:11.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.917+0000 7f5d6c33b700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6408baa0 0x7f5d6416a810 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5d60004740 tx=0x7f5d60004770 comp rx=0 tx=0).stop 2024-09-13T18:55:11.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.917+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4050308641 shutdown_connections 2024-09-13T18:55:11.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.917+0000 7f5d6c33b700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5d48066800 0x7f5d48068cc0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:11.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.917+0000 7f5d6c33b700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d6416c6b0 0x7f5d641feb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:11.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.917+0000 7f5d6c33b700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d6416e0c0 0x7f5d640ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:11.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.917+0000 7f5d6c33b700 1 --2- 172.21.15.146:0/4050308641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6408baa0 0x7f5d6416a810 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:11.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.917+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4050308641 >> 172.21.15.146:0/4050308641 conn(0x7f5d64066090 msgr2=0x7f5d6406e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:11.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.921+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4050308641 shutdown_connections 2024-09-13T18:55:11.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:11.921+0000 7f5d6c33b700 1 -- 172.21.15.146:0/4050308641 wait complete. 2024-09-13T18:55:11.934 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 112 2024-09-13T18:55:11.986 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:55:13.988 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:14.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.357+0000 7fde72619700 1 Processor -- start 2024-09-13T18:55:14.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.357+0000 7fde72619700 1 -- start start 2024-09-13T18:55:14.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.357+0000 7fde72619700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fde6c0c96a0 0x7fde6c06fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:14.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.357+0000 7fde72619700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fde6c0a9ba0 0x7fde6c070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:14.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.357+0000 7fde72619700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6c155770 con 0x7fde6c0cb390 2024-09-13T18:55:14.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.357+0000 7fde72619700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6c155a50 con 0x7fde6c0c96a0 2024-09-13T18:55:14.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.357+0000 7fde72619700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6c155d30 con 0x7fde6c0a9ba0 2024-09-13T18:55:14.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.357+0000 7fde70bb6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fde6c0a9ba0 0x7fde6c070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:14.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.357+0000 7fde6bfff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fde6c0cb390 0x7fde6c0a9490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:39888/0 (socket says 172.21.15.146:39888) 2024-09-13T18:55:14.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.357+0000 7fde6bfff700 1 -- 172.21.15.146:0/3884139820 learned_addr learned my addr 172.21.15.146:0/3884139820 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:55:14.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.357+0000 7fde6b7fe700 1 --2- 172.21.15.146:0/3884139820 >> v2:172.21.15.146:3300/0 conn(0x7fde6c0c96a0 0x7fde6c06fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:14.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.361+0000 7fde6b7fe700 1 -- 172.21.15.146:0/3884139820 >> v2:172.21.15.154:3300/0 conn(0x7fde6c0a9ba0 msgr2=0x7fde6c070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:14.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.361+0000 7fde6b7fe700 1 --2- 172.21.15.146:0/3884139820 >> v2:172.21.15.154:3300/0 conn(0x7fde6c0a9ba0 0x7fde6c070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:14.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.361+0000 7fde6b7fe700 1 -- 172.21.15.146:0/3884139820 >> v1:172.21.15.154:6789/0 conn(0x7fde6c0cb390 legacy=0x7fde6c0a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:14.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.361+0000 7fde6b7fe700 1 -- 172.21.15.146:0/3884139820 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c156010 con 0x7fde6c0c96a0 2024-09-13T18:55:14.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.361+0000 7fde6b7fe700 1 --2- 172.21.15.146:0/3884139820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c0c96a0 0x7fde6c06fae0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0cbd30 tx=0x7fde6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38802a41eaaf4ae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:14.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.361+0000 7fde6affd700 1 -- 172.21.15.146:0/3884139820 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64003c40 con 0x7fde6c0c96a0 2024-09-13T18:55:14.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.361+0000 7fde6affd700 1 -- 172.21.15.146:0/3884139820 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde64003da0 con 0x7fde6c0c96a0 2024-09-13T18:55:14.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.361+0000 7fde6affd700 1 -- 172.21.15.146:0/3884139820 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6400bc60 con 0x7fde6c0c96a0 2024-09-13T18:55:14.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.361+0000 7fde72619700 1 -- 172.21.15.146:0/3884139820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c0c96a0 msgr2=0x7fde6c06fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:14.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.361+0000 7fde72619700 1 --2- 172.21.15.146:0/3884139820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c0c96a0 0x7fde6c06fae0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0cbd30 tx=0x7fde6400b330 comp rx=0 tx=0).stop 2024-09-13T18:55:14.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.369+0000 7fde72619700 1 -- 172.21.15.146:0/3884139820 shutdown_connections 2024-09-13T18:55:14.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.369+0000 7fde72619700 1 --2- 172.21.15.146:0/3884139820 >> v2:172.21.15.154:3300/0 conn(0x7fde6c0a9ba0 0x7fde6c070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:14.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.369+0000 7fde72619700 1 --2- 172.21.15.146:0/3884139820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c0c96a0 0x7fde6c06fae0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:14.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.369+0000 7fde72619700 1 -- 172.21.15.146:0/3884139820 >> 172.21.15.146:0/3884139820 conn(0x7fde6c066090 msgr2=0x7fde6c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:14.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.369+0000 7fde72619700 1 -- 172.21.15.146:0/3884139820 shutdown_connections 2024-09-13T18:55:14.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.369+0000 7fde72619700 1 -- 172.21.15.146:0/3884139820 wait complete. 2024-09-13T18:55:14.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde72619700 1 Processor -- start 2024-09-13T18:55:14.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde72619700 1 -- start start 2024-09-13T18:55:14.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde72619700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde6c0a9ba0 0x7fde6c0c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:14.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde72619700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c0cb390 0x7fde6c1ee500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:14.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde72619700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c1eea40 0x7fde6c158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:14.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde6bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde6c0a9ba0 0x7fde6c0c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:14.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde6bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde6c0a9ba0 0x7fde6c0c1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:41342/0 (socket says 172.21.15.146:41342) 2024-09-13T18:55:14.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde72619700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1570a0 con 0x7fde6c0cb390 2024-09-13T18:55:14.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde72619700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1eeef0 con 0x7fde6c0a9ba0 2024-09-13T18:55:14.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde72619700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1572f0 con 0x7fde6c1eea40 2024-09-13T18:55:14.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde6bfff700 1 -- 172.21.15.146:0/3820258076 learned_addr learned my addr 172.21.15.146:0/3820258076 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:14.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde70bb6700 1 --2- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c1eea40 0x7fde6c158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde6bfff700 1 -- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c1eea40 msgr2=0x7fde6c158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde6bfff700 1 --2- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c1eea40 0x7fde6c158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde6bfff700 1 -- 172.21.15.146:0/3820258076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c0cb390 msgr2=0x7fde6c1ee500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde6bfff700 1 --2- 172.21.15.146:0/3820258076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c0cb390 0x7fde6c1ee500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde6bfff700 1 -- 172.21.15.146:0/3820258076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde64009a70 con 0x7fde6c0a9ba0 2024-09-13T18:55:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.385+0000 7fde6bfff700 1 --2- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde6c0a9ba0 0x7fde6c0c1e20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fde6000c780 tx=0x7fde6000ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:14.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.397+0000 7fde68ff9700 1 -- 172.21.15.146:0/3820258076 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde60004ca0 con 0x7fde6c0a9ba0 2024-09-13T18:55:14.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.397+0000 7fde68ff9700 1 -- 172.21.15.146:0/3820258076 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde60004e20 con 0x7fde6c0a9ba0 2024-09-13T18:55:14.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.397+0000 7fde68ff9700 1 -- 172.21.15.146:0/3820258076 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde60007310 con 0x7fde6c0a9ba0 2024-09-13T18:55:14.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.397+0000 7fde72619700 1 -- 172.21.15.146:0/3820258076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c1575d0 con 0x7fde6c0a9ba0 2024-09-13T18:55:14.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.397+0000 7fde72619700 1 -- 172.21.15.146:0/3820258076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c157b90 con 0x7fde6c0a9ba0 2024-09-13T18:55:14.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.405+0000 7fde68ff9700 1 -- 172.21.15.146:0/3820258076 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fde60007470 con 0x7fde6c0a9ba0 2024-09-13T18:55:14.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.405+0000 7fde68ff9700 1 --2- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde50066800 0x7fde50068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:14.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.405+0000 7fde68ff9700 1 -- 172.21.15.146:0/3820258076 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7fde600847f0 con 0x7fde6c0a9ba0 2024-09-13T18:55:14.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.405+0000 7fde6b7fe700 1 --2- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde50066800 0x7fde50068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:14.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.405+0000 7fde6b7fe700 1 --2- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde50066800 0x7fde50068cc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0c9570 tx=0x7fde64012000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:14.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.417+0000 7fde72619700 1 -- 172.21.15.146:0/3820258076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6c0cacd0 con 0x7fde6c0a9ba0 2024-09-13T18:55:14.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:14.429+0000 7fde68ff9700 1 -- 172.21.15.146:0/3820258076 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fde60051270 con 0x7fde6c0a9ba0 2024-09-13T18:55:15.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.213+0000 7fde72619700 1 -- 172.21.15.146:0/3820258076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde6c074440 con 0x7fde6c0a9ba0 2024-09-13T18:55:15.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.213+0000 7fde68ff9700 1 -- 172.21.15.146:0/3820258076 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+6297 (secure 0 0 0) 0x7fde60054890 con 0x7fde6c0a9ba0 2024-09-13T18:55:15.230 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:15.230 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":112,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:54:14.801062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:active","state_seq":85,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:15.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.217+0000 7fde5a7fc700 1 -- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde50066800 msgr2=0x7fde50068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:15.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.217+0000 7fde5a7fc700 1 --2- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde50066800 0x7fde50068cc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0c9570 tx=0x7fde64012000 comp rx=0 tx=0).stop 2024-09-13T18:55:15.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.221+0000 7fde5a7fc700 1 -- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde6c0a9ba0 msgr2=0x7fde6c0c1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:15.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.221+0000 7fde5a7fc700 1 --2- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde6c0a9ba0 0x7fde6c0c1e20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fde6000c780 tx=0x7fde6000ca90 comp rx=0 tx=0).stop 2024-09-13T18:55:15.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.221+0000 7fde5a7fc700 1 -- 172.21.15.146:0/3820258076 shutdown_connections 2024-09-13T18:55:15.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.221+0000 7fde5a7fc700 1 --2- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fde50066800 0x7fde50068cc0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:15.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.221+0000 7fde5a7fc700 1 --2- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde6c1eea40 0x7fde6c158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:15.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.221+0000 7fde5a7fc700 1 --2- 172.21.15.146:0/3820258076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde6c0cb390 0x7fde6c1ee500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:15.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.221+0000 7fde5a7fc700 1 --2- 172.21.15.146:0/3820258076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde6c0a9ba0 0x7fde6c0c1e20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:15.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.221+0000 7fde5a7fc700 1 -- 172.21.15.146:0/3820258076 >> 172.21.15.146:0/3820258076 conn(0x7fde6c066090 msgr2=0x7fde6c06c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:15.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.221+0000 7fde5a7fc700 1 -- 172.21.15.146:0/3820258076 shutdown_connections 2024-09-13T18:55:15.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:15.221+0000 7fde5a7fc700 1 -- 172.21.15.146:0/3820258076 wait complete. 2024-09-13T18:55:15.244 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 112 2024-09-13T18:55:15.269 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:55:17.270 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:17.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.973+0000 7f360cecb700 1 Processor -- start 2024-09-13T18:55:17.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.973+0000 7f360cecb700 1 -- start start 2024-09-13T18:55:17.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.973+0000 7f360cecb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36080cb2b0 0x7f36080c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:17.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.973+0000 7f360cecb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f36080c8230 0x7f360815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:17.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.973+0000 7f360cecb700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3608161060 con 0x7f36080caed0 2024-09-13T18:55:17.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.973+0000 7f360cecb700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36080c78d0 con 0x7f36080c8230 2024-09-13T18:55:17.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.973+0000 7f360cecb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3608161300 con 0x7f36080cb2b0 2024-09-13T18:55:17.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.973+0000 7f36077fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f36080caed0 0x7f3608072380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:39904/0 (socket says 172.21.15.146:39904) 2024-09-13T18:55:17.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.973+0000 7f36077fe700 1 -- 172.21.15.146:0/1817081161 learned_addr learned my addr 172.21.15.146:0/1817081161 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:55:17.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.973+0000 7f3607fff700 1 --2- 172.21.15.146:0/1817081161 >> v2:172.21.15.146:3300/0 conn(0x7f36080c8230 0x7f360815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:17.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.977+0000 7f3607fff700 1 -- 172.21.15.146:0/1817081161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36080cb2b0 msgr2=0x7f36080c7cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:55:17.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.977+0000 7f3607fff700 1 --2- 172.21.15.146:0/1817081161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36080cb2b0 0x7f36080c7cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:17.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.977+0000 7f3607fff700 1 -- 172.21.15.146:0/1817081161 >> v1:172.21.15.154:6789/0 conn(0x7f36080caed0 legacy=0x7f3608072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:17.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.977+0000 7f3607fff700 1 -- 172.21.15.146:0/1817081161 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36081615c0 con 0x7f36080c8230 2024-09-13T18:55:17.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.977+0000 7f3607fff700 1 --2- 172.21.15.146:0/1817081161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36080c8230 0x7f360815e170 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f360000b050 tx=0x7f360000b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555bafde9780ce6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:18.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.989+0000 7f36067fc700 1 -- 172.21.15.146:0/1817081161 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360000e070 con 0x7f36080c8230 2024-09-13T18:55:18.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.989+0000 7f36067fc700 1 -- 172.21.15.146:0/1817081161 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360000bad0 con 0x7f36080c8230 2024-09-13T18:55:18.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.989+0000 7f36067fc700 1 -- 172.21.15.146:0/1817081161 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360000be80 con 0x7f36080c8230 2024-09-13T18:55:18.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.989+0000 7f360cecb700 1 -- 172.21.15.146:0/1817081161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36080c8230 msgr2=0x7f360815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:18.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.989+0000 7f360cecb700 1 --2- 172.21.15.146:0/1817081161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36080c8230 0x7f360815e170 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f360000b050 tx=0x7f360000b360 comp rx=0 tx=0).stop 2024-09-13T18:55:18.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.997+0000 7f360cecb700 1 -- 172.21.15.146:0/1817081161 shutdown_connections 2024-09-13T18:55:18.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.997+0000 7f360cecb700 1 --2- 172.21.15.146:0/1817081161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36080c8230 0x7f360815e170 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:18.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.997+0000 7f360cecb700 1 --2- 172.21.15.146:0/1817081161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36080cb2b0 0x7f36080c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:18.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.997+0000 7f360cecb700 1 -- 172.21.15.146:0/1817081161 >> 172.21.15.146:0/1817081161 conn(0x7f3608066090 msgr2=0x7f3608066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:18.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.997+0000 7f360cecb700 1 -- 172.21.15.146:0/1817081161 shutdown_connections 2024-09-13T18:55:18.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:17.997+0000 7f360cecb700 1 -- 172.21.15.146:0/1817081161 wait complete. 2024-09-13T18:55:18.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.001+0000 7f360cecb700 1 Processor -- start 2024-09-13T18:55:18.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.001+0000 7f360cecb700 1 -- start start 2024-09-13T18:55:18.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f360cecb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36080c8230 0x7f36080adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:18.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f360cecb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36080caed0 0x7f36080ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:18.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f360cecb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36080cb2b0 0x7f36081543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:18.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f360cecb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36080ad640 con 0x7f36080caed0 2024-09-13T18:55:18.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f360cecb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36080aabb0 con 0x7f36080cb2b0 2024-09-13T18:55:18.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f360cecb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36081f75d0 con 0x7f36080c8230 2024-09-13T18:55:18.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f36077fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36080c8230 0x7f36080adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:18.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f36077fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36080c8230 0x7f36080adb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:56844/0 (socket says 172.21.15.146:56844) 2024-09-13T18:55:18.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f36077fe700 1 -- 172.21.15.146:0/1670420611 learned_addr learned my addr 172.21.15.146:0/1670420611 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:18.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f3607fff700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36080cb2b0 0x7f36081543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:18.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f3606ffd700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36080caed0 0x7f36080ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:18.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f3607fff700 1 -- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36080c8230 msgr2=0x7f36080adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:18.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f3607fff700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36080c8230 0x7f36080adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:18.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f3607fff700 1 -- 172.21.15.146:0/1670420611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36080caed0 msgr2=0x7f36080ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:18.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f3607fff700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36080caed0 0x7f36080ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:18.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f3607fff700 1 -- 172.21.15.146:0/1670420611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3600009a70 con 0x7f36080cb2b0 2024-09-13T18:55:18.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f3606ffd700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36080caed0 0x7f36080ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:55:18.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f3607fff700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36080cb2b0 0x7f36081543b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f36080c87e0 tx=0x7f3600004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:18.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f35effff700 1 -- 172.21.15.146:0/1670420611 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600012630 con 0x7f36080cb2b0 2024-09-13T18:55:18.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f35effff700 1 -- 172.21.15.146:0/1670420611 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3600012790 con 0x7f36080cb2b0 2024-09-13T18:55:18.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f35effff700 1 -- 172.21.15.146:0/1670420611 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600012b00 con 0x7f36080cb2b0 2024-09-13T18:55:18.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f360cecb700 1 -- 172.21.15.146:0/1670420611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36081f7b00 con 0x7f36080cb2b0 2024-09-13T18:55:18.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.005+0000 7f360cecb700 1 -- 172.21.15.146:0/1670420611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36080af0a0 con 0x7f36080cb2b0 2024-09-13T18:55:18.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.013+0000 7f35effff700 1 -- 172.21.15.146:0/1670420611 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3600012c60 con 0x7f36080cb2b0 2024-09-13T18:55:18.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.013+0000 7f360cecb700 1 -- 172.21.15.146:0/1670420611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36080af230 con 0x7f36080cb2b0 2024-09-13T18:55:18.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.013+0000 7f35effff700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f35f0066900 0x7f35f0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:18.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.013+0000 7f35effff700 1 -- 172.21.15.146:0/1670420611 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7f3600086a60 con 0x7f36080cb2b0 2024-09-13T18:55:18.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.013+0000 7f36077fe700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f35f0066900 0x7f35f0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:18.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.013+0000 7f36077fe700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f35f0066900 0x7f35f0068dc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f35fc007e00 tx=0x7f35fc007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:18.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:18.025+0000 7f35effff700 1 -- 172.21.15.146:0/1670420611 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f36000534f0 con 0x7f36080cb2b0 2024-09-13T18:55:18.229 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:55:18.231 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:55:19.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.053+0000 7f360cecb700 1 -- 172.21.15.146:0/1670420611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36080c7cf0 con 0x7f36080cb2b0 2024-09-13T18:55:19.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.053+0000 7f35effff700 1 -- 172.21.15.146:0/1670420611 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+6297 (secure 0 0 0) 0x7f3600056b10 con 0x7f36080cb2b0 2024-09-13T18:55:19.068 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:19.068 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":112,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:54:14.801062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:active","state_seq":85,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:19.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.061+0000 7f35edffb700 1 -- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f35f0066900 msgr2=0x7f35f0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:19.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.061+0000 7f35edffb700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f35f0066900 0x7f35f0068dc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f35fc007e00 tx=0x7f35fc007d50 comp rx=0 tx=0).stop 2024-09-13T18:55:19.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.061+0000 7f35edffb700 1 -- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36080cb2b0 msgr2=0x7f36081543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:19.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.061+0000 7f35edffb700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36080cb2b0 0x7f36081543b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f36080c87e0 tx=0x7f3600004af0 comp rx=0 tx=0).stop 2024-09-13T18:55:19.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.061+0000 7f35edffb700 1 -- 172.21.15.146:0/1670420611 shutdown_connections 2024-09-13T18:55:19.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.061+0000 7f35edffb700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f35f0066900 0x7f35f0068dc0 secure :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f35fc007e00 tx=0x7f35fc007d50 comp rx=0 tx=0).stop 2024-09-13T18:55:19.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.061+0000 7f35edffb700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36080cb2b0 0x7f36081543b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:19.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.061+0000 7f35edffb700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36080caed0 0x7f36080ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:19.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.061+0000 7f35edffb700 1 --2- 172.21.15.146:0/1670420611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36080c8230 0x7f36080adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:19.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.061+0000 7f35edffb700 1 -- 172.21.15.146:0/1670420611 >> 172.21.15.146:0/1670420611 conn(0x7f3608066090 msgr2=0x7f360806dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:19.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.061+0000 7f35edffb700 1 -- 172.21.15.146:0/1670420611 shutdown_connections 2024-09-13T18:55:19.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:19.061+0000 7f35edffb700 1 -- 172.21.15.146:0/1670420611 wait complete. 2024-09-13T18:55:19.076 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 112 2024-09-13T18:55:19.104 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:55:21.105 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:21.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.477+0000 7f3894bd8700 1 Processor -- start 2024-09-13T18:55:21.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.477+0000 7f3894bd8700 1 -- start start 2024-09-13T18:55:21.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.477+0000 7f3894bd8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f38880d46a0 0x7f38880da5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.477+0000 7f3894bd8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3888005ab0 0x7f3888034300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.477+0000 7f3894bd8700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3888032cc0 con 0x7f3888034840 2024-09-13T18:55:21.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.477+0000 7f3894bd8700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3888007400 con 0x7f38880d46a0 2024-09-13T18:55:21.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.477+0000 7f3894bd8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38880d4a80 con 0x7f3888005ab0 2024-09-13T18:55:21.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.485+0000 7f388dd9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3888005ab0 0x7f3888034300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.485+0000 7f388dd9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3888005ab0 0x7f3888034300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:38432/0 (socket says 172.21.15.146:38432) 2024-09-13T18:55:21.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.485+0000 7f388dd9b700 1 -- 172.21.15.146:0/3977072058 learned_addr learned my addr 172.21.15.146:0/3977072058 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:21.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.485+0000 7f388dd9b700 1 -- 172.21.15.146:0/3977072058 >> v2:172.21.15.146:3300/0 conn(0x7f38880d46a0 msgr2=0x7f38880da5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:55:21.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.485+0000 7f388dd9b700 1 --2- 172.21.15.146:0/3977072058 >> v2:172.21.15.146:3300/0 conn(0x7f38880d46a0 0x7f38880da5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.485+0000 7f388dd9b700 1 -- 172.21.15.146:0/3977072058 >> v1:172.21.15.154:6789/0 conn(0x7f3888034840 legacy=0x7f388804af20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:55:21.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.485+0000 7f388dd9b700 1 -- 172.21.15.146:0/3977072058 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3888006080 con 0x7f3888005ab0 2024-09-13T18:55:21.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.489+0000 7f388dd9b700 1 --2- 172.21.15.146:0/3977072058 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3888005ab0 0x7f3888034300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f38900751e0 tx=0x7f3890061ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38d48d1731a48ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:21.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.489+0000 7f388d59a700 1 -- 172.21.15.146:0/3977072058 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890073900 con 0x7f3888005ab0 2024-09-13T18:55:21.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.489+0000 7f388d59a700 1 -- 172.21.15.146:0/3977072058 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3890073a60 con 0x7f3888005ab0 2024-09-13T18:55:21.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.489+0000 7f388d59a700 1 -- 172.21.15.146:0/3977072058 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890075c30 con 0x7f3888005ab0 2024-09-13T18:55:21.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.489+0000 7f3894bd8700 1 -- 172.21.15.146:0/3977072058 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3888005ab0 msgr2=0x7f3888034300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.489+0000 7f3894bd8700 1 --2- 172.21.15.146:0/3977072058 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3888005ab0 0x7f3888034300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f38900751e0 tx=0x7f3890061ef0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 -- 172.21.15.146:0/3977072058 shutdown_connections 2024-09-13T18:55:21.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 --2- 172.21.15.146:0/3977072058 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3888005ab0 0x7f3888034300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 --2- 172.21.15.146:0/3977072058 >> v2:172.21.15.146:3300/0 conn(0x7f38880d46a0 0x7f38880da5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 -- 172.21.15.146:0/3977072058 >> 172.21.15.146:0/3977072058 conn(0x7f388804fd00 msgr2=0x7f3888050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:21.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 -- 172.21.15.146:0/3977072058 shutdown_connections 2024-09-13T18:55:21.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 -- 172.21.15.146:0/3977072058 wait complete. 2024-09-13T18:55:21.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 Processor -- start 2024-09-13T18:55:21.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 -- start start 2024-09-13T18:55:21.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3888005ab0 0x7f38880d9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3888034840 0x7f38880d9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38880d46a0 0x7f38880ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38880da680 con 0x7f3888005ab0 2024-09-13T18:55:21.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38880d90a0 con 0x7f3888034840 2024-09-13T18:55:21.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f3894bd8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38880da3a0 con 0x7f38880d46a0 2024-09-13T18:55:21.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f388dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3888034840 0x7f38880d9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f388dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3888034840 0x7f38880d9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:55508/0 (socket says 172.21.15.146:55508) 2024-09-13T18:55:21.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f388dd9b700 1 -- 172.21.15.146:0/1622493886 learned_addr learned my addr 172.21.15.146:0/1622493886 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:21.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.497+0000 7f388e59c700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3888005ab0 0x7f38880d9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.501+0000 7f388dd9b700 1 -- 172.21.15.146:0/1622493886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38880d46a0 msgr2=0x7f38880ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.501+0000 7f388dd9b700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38880d46a0 0x7f38880ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.501+0000 7f388dd9b700 1 -- 172.21.15.146:0/1622493886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3888005ab0 msgr2=0x7f38880d9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.501+0000 7f388dd9b700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3888005ab0 0x7f38880d9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.501+0000 7f388dd9b700 1 -- 172.21.15.146:0/1622493886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389004f5c0 con 0x7f3888034840 2024-09-13T18:55:21.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.501+0000 7f388e59c700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3888005ab0 0x7f38880d9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:55:21.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.505+0000 7f388dd9b700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3888034840 0x7f38880d9a70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3890067bf0 tx=0x7f389004f750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:21.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.509+0000 7f387effd700 1 -- 172.21.15.146:0/1622493886 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890067f10 con 0x7f3888034840 2024-09-13T18:55:21.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.509+0000 7f387effd700 1 -- 172.21.15.146:0/1622493886 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3890068070 con 0x7f3888034840 2024-09-13T18:55:21.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.509+0000 7f3894bd8700 1 -- 172.21.15.146:0/1622493886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38880471a0 con 0x7f3888034840 2024-09-13T18:55:21.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.509+0000 7f3894bd8700 1 -- 172.21.15.146:0/1622493886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38880cc6d0 con 0x7f3888034840 2024-09-13T18:55:21.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.509+0000 7f387effd700 1 -- 172.21.15.146:0/1622493886 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890070940 con 0x7f3888034840 2024-09-13T18:55:21.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.513+0000 7f387effd700 1 -- 172.21.15.146:0/1622493886 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3890070aa0 con 0x7f3888034840 2024-09-13T18:55:21.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.513+0000 7f387effd700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3870066730 0x7f3870068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.513+0000 7f387effd700 1 -- 172.21.15.146:0/1622493886 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7f38900fff50 con 0x7f3888034840 2024-09-13T18:55:21.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.513+0000 7f388e59c700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3870066730 0x7f3870068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.513+0000 7f3894bd8700 1 -- 172.21.15.146:0/1622493886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38880476c0 con 0x7f3888034840 2024-09-13T18:55:21.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.513+0000 7f388e59c700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3870066730 0x7f3870068bf0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3880009dc0 tx=0x7f38800090e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:21.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:21.529+0000 7f387effd700 1 -- 172.21.15.146:0/1622493886 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f38880476c0 con 0x7f3888034840 2024-09-13T18:55:22.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.217+0000 7f3894bd8700 1 -- 172.21.15.146:0/1622493886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3888006ee0 con 0x7f3888034840 2024-09-13T18:55:22.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.221+0000 7f387effd700 1 -- 172.21.15.146:0/1622493886 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+6297 (secure 0 0 0) 0x7f38880476c0 con 0x7f3888034840 2024-09-13T18:55:22.239 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:22.239 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":112,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:54:14.801062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:active","state_seq":85,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:22.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.249+0000 7f3894bd8700 1 -- 172.21.15.146:0/1622493886 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3870066730 msgr2=0x7f3870068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:22.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.249+0000 7f3894bd8700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3870066730 0x7f3870068bf0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3880009dc0 tx=0x7f38800090e0 comp rx=0 tx=0).stop 2024-09-13T18:55:22.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.249+0000 7f3894bd8700 1 -- 172.21.15.146:0/1622493886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3888034840 msgr2=0x7f38880d9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:22.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.249+0000 7f3894bd8700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3888034840 0x7f38880d9a70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3890067bf0 tx=0x7f389004f750 comp rx=0 tx=0).stop 2024-09-13T18:55:22.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.249+0000 7f3894bd8700 1 -- 172.21.15.146:0/1622493886 shutdown_connections 2024-09-13T18:55:22.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.249+0000 7f3894bd8700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3870066730 0x7f3870068bf0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:22.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.249+0000 7f3894bd8700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38880d46a0 0x7f38880ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:22.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.249+0000 7f3894bd8700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3888034840 0x7f38880d9a70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:22.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.249+0000 7f3894bd8700 1 --2- 172.21.15.146:0/1622493886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3888005ab0 0x7f38880d9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:22.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.249+0000 7f3894bd8700 1 -- 172.21.15.146:0/1622493886 >> 172.21.15.146:0/1622493886 conn(0x7f388804fd00 msgr2=0x7f38880d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:22.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.249+0000 7f3894bd8700 1 -- 172.21.15.146:0/1622493886 shutdown_connections 2024-09-13T18:55:22.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:22.249+0000 7f3894bd8700 1 -- 172.21.15.146:0/1622493886 wait complete. 2024-09-13T18:55:22.274 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 112 2024-09-13T18:55:22.311 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:55:24.313 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:24.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec713bc700 1 Processor -- start 2024-09-13T18:55:24.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec713bc700 1 -- start start 2024-09-13T18:55:24.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec713bc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fec6c159c20 0x7fec6c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:24.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec713bc700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6c0a9b60 con 0x7fec6c06f5c0 2024-09-13T18:55:24.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec713bc700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6c0c9c10 con 0x7fec6c0a95b0 2024-09-13T18:55:24.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec713bc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6c0cb830 con 0x7fec6c159c20 2024-09-13T18:55:24.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6bfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fec6c06f5c0 0x7fec6c0cb120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50018/0 (socket says 172.21.15.146:50018) 2024-09-13T18:55:24.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6bfff700 1 -- 172.21.15.146:0/3767591518 learned_addr learned my addr 172.21.15.146:0/3767591518 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:55:24.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697058418 0 0) 0x7fec6c0a9b60 con 0x7fec6c06f5c0 2024-09-13T18:55:24.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec70bbb700 1 --2- 172.21.15.146:0/3767591518 >> v2:172.21.15.154:3300/0 conn(0x7fec6c159c20 0x7fec6c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:24.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec580035d0 con 0x7fec6c06f5c0 2024-09-13T18:55:24.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1176291550 0 0) 0x7fec580035d0 con 0x7fec6c06f5c0 2024-09-13T18:55:24.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 >> v2:172.21.15.154:3300/0 conn(0x7fec6c159c20 msgr2=0x7fec6c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:24.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 --2- 172.21.15.146:0/3767591518 >> v2:172.21.15.154:3300/0 conn(0x7fec6c159c20 0x7fec6c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:24.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 >> v1:172.21.15.154:6789/0 conn(0x7fec6c0a95b0 legacy=0x7fec6c0c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:24.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0c8b70 con 0x7fec6c06f5c0 2024-09-13T18:55:24.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7fec5c0019f0 con 0x7fec6c06f5c0 2024-09-13T18:55:24.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 >> v1:172.21.15.146:6789/0 conn(0x7fec6c06f5c0 legacy=0x7fec6c0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:24.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 --2- 172.21.15.146:0/3767591518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec58003a90 0x7fec58003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:24.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 --2- 172.21.15.146:0/3767591518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec58004870 0x7fec58004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:24.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 --2- 172.21.15.146:0/3767591518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec58005280 0x7fec5800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:24.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0cb830 con 0x7fec58003a90 2024-09-13T18:55:24.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5800c1c0 con 0x7fec58004870 2024-09-13T18:55:24.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec5800c380 con 0x7fec58005280 2024-09-13T18:55:24.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.837+0000 7fec6bfff700 1 --2- 172.21.15.146:0/3767591518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec58003a90 0x7fec58003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:24.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec6bfff700 1 -- 172.21.15.146:0/3767591518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec58005280 msgr2=0x7fec5800ba30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:55:24.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec6bfff700 1 --2- 172.21.15.146:0/3767591518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec58005280 0x7fec5800ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:24.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec6bfff700 1 -- 172.21.15.146:0/3767591518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec58004870 msgr2=0x7fec58004cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:55:24.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec6bfff700 1 --2- 172.21.15.146:0/3767591518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec58004870 0x7fec58004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:24.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec6bfff700 1 -- 172.21.15.146:0/3767591518 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0c8b70 con 0x7fec58003a90 2024-09-13T18:55:24.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec6bfff700 1 --2- 172.21.15.146:0/3767591518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec58003a90 0x7fec58003e90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00ab30 tx=0x7fec580058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7dd04bdbf6721b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:24.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c012600 con 0x7fec58003a90 2024-09-13T18:55:24.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c012760 con 0x7fec58003a90 2024-09-13T18:55:24.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec6affd700 1 -- 172.21.15.146:0/3767591518 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c012af0 con 0x7fec58003a90 2024-09-13T18:55:24.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec713bc700 1 -- 172.21.15.146:0/3767591518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec58003a90 msgr2=0x7fec58003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:24.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec713bc700 1 --2- 172.21.15.146:0/3767591518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec58003a90 0x7fec58003e90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00ab30 tx=0x7fec580058a0 comp rx=0 tx=0).stop 2024-09-13T18:55:24.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec70bbb700 1 -- 172.21.15.146:0/3767591518 reap_dead start 2024-09-13T18:55:24.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec70bbb700 1 -- 172.21.15.146:0/3767591518 reap_dead start 2024-09-13T18:55:24.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec713bc700 1 -- 172.21.15.146:0/3767591518 shutdown_connections 2024-09-13T18:55:24.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec713bc700 1 -- 172.21.15.146:0/3767591518 >> 172.21.15.146:0/3767591518 conn(0x7fec6c066090 msgr2=0x7fec6c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:24.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec713bc700 1 -- 172.21.15.146:0/3767591518 shutdown_connections 2024-09-13T18:55:24.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.841+0000 7fec713bc700 1 -- 172.21.15.146:0/3767591518 wait complete. 2024-09-13T18:55:24.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.845+0000 7fec713bc700 1 Processor -- start 2024-09-13T18:55:24.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.845+0000 7fec713bc700 1 -- start start 2024-09-13T18:55:24.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.845+0000 7fec713bc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec6c0af4e0 0x7fec6c0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:24.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.845+0000 7fec713bc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec6c0ae3d0 0x7fec6c13f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:24.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.845+0000 7fec713bc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec6c0ae830 0x7fec6c13fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:24.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.845+0000 7fec713bc700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c14d290 con 0x7fec6c0ae3d0 2024-09-13T18:55:24.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.845+0000 7fec713bc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0af8c0 con 0x7fec6c0af4e0 2024-09-13T18:55:24.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.845+0000 7fec713bc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c149b40 con 0x7fec6c0ae830 2024-09-13T18:55:24.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.845+0000 7fec6b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec6c0ae3d0 0x7fec6c13f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:24.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.845+0000 7fec6b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec6c0ae3d0 0x7fec6c13f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:56576/0 (socket says 172.21.15.146:56576) 2024-09-13T18:55:24.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.845+0000 7fec6b7fe700 1 -- 172.21.15.146:0/3271872804 learned_addr learned my addr 172.21.15.146:0/3271872804 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:24.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.845+0000 7fec70bbb700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec6c0ae830 0x7fec6c13fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:24.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.849+0000 7fec70bbb700 1 -- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec6c0af4e0 msgr2=0x7fec6c0ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:24.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.849+0000 7fec6bfff700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec6c0af4e0 0x7fec6c0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:24.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.849+0000 7fec70bbb700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec6c0af4e0 0x7fec6c0ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:24.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.849+0000 7fec70bbb700 1 -- 172.21.15.146:0/3271872804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec6c0ae3d0 msgr2=0x7fec6c13f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.849+0000 7fec70bbb700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec6c0ae3d0 0x7fec6c13f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.849+0000 7fec70bbb700 1 -- 172.21.15.146:0/3271872804 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c14c4d0 con 0x7fec6c0ae830 2024-09-13T18:55:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.849+0000 7fec6bfff700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec6c0af4e0 0x7fec6c0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:55:24.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.853+0000 7fec70bbb700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec6c0ae830 0x7fec6c13fb60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0affa0 tx=0x7fec5400e790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:24.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.869+0000 7fec68ff9700 1 -- 172.21.15.146:0/3271872804 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5400c070 con 0x7fec6c0ae830 2024-09-13T18:55:24.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.869+0000 7fec713bc700 1 -- 172.21.15.146:0/3271872804 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6c14c7b0 con 0x7fec6c0ae830 2024-09-13T18:55:24.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.869+0000 7fec713bc700 1 -- 172.21.15.146:0/3271872804 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6c14cd70 con 0x7fec6c0ae830 2024-09-13T18:55:24.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.869+0000 7fec68ff9700 1 -- 172.21.15.146:0/3271872804 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec54013630 con 0x7fec6c0ae830 2024-09-13T18:55:24.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.869+0000 7fec68ff9700 1 -- 172.21.15.146:0/3271872804 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec540139e0 con 0x7fec6c0ae830 2024-09-13T18:55:24.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.869+0000 7fec6b7fe700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec6c0ae3d0 0x7fec6c13f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:55:24.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.873+0000 7fec68ff9700 1 -- 172.21.15.146:0/3271872804 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fec54013b40 con 0x7fec6c0ae830 2024-09-13T18:55:24.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.873+0000 7fec713bc700 1 -- 172.21.15.146:0/3271872804 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec50005310 con 0x7fec6c0ae830 2024-09-13T18:55:24.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.873+0000 7fec68ff9700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fec4c066830 0x7fec4c068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:24.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.873+0000 7fec68ff9700 1 -- 172.21.15.146:0/3271872804 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7953+0+0 (secure 0 0 0) 0x7fec54083f50 con 0x7fec6c0ae830 2024-09-13T18:55:24.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.881+0000 7fec6bfff700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fec4c066830 0x7fec4c068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:24.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.881+0000 7fec68ff9700 1 -- 172.21.15.146:0/3271872804 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fec540509d0 con 0x7fec6c0ae830 2024-09-13T18:55:24.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:24.889+0000 7fec6bfff700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fec4c066830 0x7fec4c068cf0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0b1250 tx=0x7fec5c010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:25.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.481+0000 7fec713bc700 1 -- 172.21.15.146:0/3271872804 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec50005b40 con 0x7fec6c0ae830 2024-09-13T18:55:25.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.481+0000 7fec68ff9700 1 -- 172.21.15.146:0/3271872804 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+6297 (secure 0 0 0) 0x7fec54053ff0 con 0x7fec6c0ae830 2024-09-13T18:55:25.498 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:25.498 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":112,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:54:14.801062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":6032,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6032":{"gid":6032,"name":"f","rank":1,"incarnation":107,"state":"up:active","state_seq":85,"addr":"172.21.15.154:6839/2053734262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2053734262},{"type":"v1","addr":"172.21.15.154:6839","nonce":2053734262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:25.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.497+0000 7fec627fc700 1 -- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fec4c066830 msgr2=0x7fec4c068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:25.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.501+0000 7fec627fc700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fec4c066830 0x7fec4c068cf0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0b1250 tx=0x7fec5c010000 comp rx=0 tx=0).stop 2024-09-13T18:55:25.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.501+0000 7fec627fc700 1 -- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec6c0ae830 msgr2=0x7fec6c13fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:25.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.501+0000 7fec627fc700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec6c0ae830 0x7fec6c13fb60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0affa0 tx=0x7fec5400e790 comp rx=0 tx=0).stop 2024-09-13T18:55:25.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.501+0000 7fec627fc700 1 -- 172.21.15.146:0/3271872804 shutdown_connections 2024-09-13T18:55:25.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.501+0000 7fec627fc700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fec4c066830 0x7fec4c068cf0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:25.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.501+0000 7fec627fc700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec6c0ae830 0x7fec6c13fb60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:25.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.501+0000 7fec627fc700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec6c0ae3d0 0x7fec6c13f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:25.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.501+0000 7fec627fc700 1 --2- 172.21.15.146:0/3271872804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec6c0af4e0 0x7fec6c0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:25.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.501+0000 7fec627fc700 1 -- 172.21.15.146:0/3271872804 >> 172.21.15.146:0/3271872804 conn(0x7fec6c066090 msgr2=0x7fec6c0c0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:25.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.501+0000 7fec627fc700 1 -- 172.21.15.146:0/3271872804 shutdown_connections 2024-09-13T18:55:25.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:25.505+0000 7fec627fc700 1 -- 172.21.15.146:0/3271872804 wait complete. 2024-09-13T18:55:25.517 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 112 2024-09-13T18:55:25.534 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-09-13T18:55:27.534 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:27.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.817+0000 7f42c3fff700 1 Processor -- start 2024-09-13T18:55:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.817+0000 7f42c3fff700 1 -- start start 2024-09-13T18:55:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.817+0000 7f42c3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c40cb2b0 0x7f42c40c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.817+0000 7f42c3fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f42c40c8230 0x7f42c415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.817+0000 7f42c3fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c4161060 con 0x7f42c40caed0 2024-09-13T18:55:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.817+0000 7f42c3fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c40c78d0 con 0x7f42c40c8230 2024-09-13T18:55:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.817+0000 7f42c3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c4161300 con 0x7f42c40cb2b0 2024-09-13T18:55:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.817+0000 7f42c2ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f42c40caed0 0x7f42c4072380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50032/0 (socket says 172.21.15.146:50032) 2024-09-13T18:55:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.817+0000 7f42c2ffd700 1 -- 172.21.15.146:0/2357132612 learned_addr learned my addr 172.21.15.146:0/2357132612 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:55:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.817+0000 7f42c37fe700 1 --2- 172.21.15.146:0/2357132612 >> v2:172.21.15.154:3300/0 conn(0x7f42c40c8230 0x7f42c415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.821+0000 7f42c27fc700 1 --2- 172.21.15.146:0/2357132612 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c40cb2b0 0x7f42c40c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.821+0000 7f42c37fe700 1 -- 172.21.15.146:0/2357132612 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c40cb2b0 msgr2=0x7f42c40c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.821+0000 7f42c37fe700 1 --2- 172.21.15.146:0/2357132612 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c40cb2b0 0x7f42c40c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:27.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.821+0000 7f42c1ffb700 1 -- 172.21.15.146:0/2357132612 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890922288 0 0) 0x7f42c4161060 con 0x7f42c40caed0 2024-09-13T18:55:27.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.821+0000 7f42c37fe700 1 -- 172.21.15.146:0/2357132612 >> v1:172.21.15.146:6789/0 conn(0x7f42c40caed0 legacy=0x7f42c4072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:27.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.821+0000 7f42c37fe700 1 -- 172.21.15.146:0/2357132612 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c41615c0 con 0x7f42c40c8230 2024-09-13T18:55:27.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.821+0000 7f42c27fc700 1 --2- 172.21.15.146:0/2357132612 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c40cb2b0 0x7f42c40c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:55:27.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.821+0000 7f42c37fe700 1 --2- 172.21.15.146:0/2357132612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42c40c8230 0x7f42c415e170 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009850 tx=0x7f42b4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26736b582d4f515 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:27.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.821+0000 7f42c1ffb700 1 -- 172.21.15.146:0/2357132612 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4011830 con 0x7f42c40c8230 2024-09-13T18:55:27.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.821+0000 7f42c1ffb700 1 -- 172.21.15.146:0/2357132612 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b4011990 con 0x7f42c40c8230 2024-09-13T18:55:27.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.821+0000 7f42c1ffb700 1 -- 172.21.15.146:0/2357132612 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4011d20 con 0x7f42c40c8230 2024-09-13T18:55:27.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.821+0000 7f42c3fff700 1 -- 172.21.15.146:0/2357132612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42c40c8230 msgr2=0x7f42c415e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:27.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.821+0000 7f42c3fff700 1 --2- 172.21.15.146:0/2357132612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42c40c8230 0x7f42c415e170 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009850 tx=0x7f42b4009b60 comp rx=0 tx=0).stop 2024-09-13T18:55:27.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.829+0000 7f42c3fff700 1 -- 172.21.15.146:0/2357132612 shutdown_connections 2024-09-13T18:55:27.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.829+0000 7f42c3fff700 1 --2- 172.21.15.146:0/2357132612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42c40c8230 0x7f42c415e170 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:27.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.829+0000 7f42c3fff700 1 --2- 172.21.15.146:0/2357132612 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c40cb2b0 0x7f42c40c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:27.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.829+0000 7f42c3fff700 1 -- 172.21.15.146:0/2357132612 >> 172.21.15.146:0/2357132612 conn(0x7f42c4066090 msgr2=0x7f42c4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:27.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.833+0000 7f42c3fff700 1 -- 172.21.15.146:0/2357132612 shutdown_connections 2024-09-13T18:55:27.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.833+0000 7f42c3fff700 1 -- 172.21.15.146:0/2357132612 wait complete. 2024-09-13T18:55:27.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c3fff700 1 Processor -- start 2024-09-13T18:55:27.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c3fff700 1 -- start start 2024-09-13T18:55:27.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42c40c8230 0x7f42c40af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:27.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42c40caed0 0x7f42c413efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:27.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c40cb2b0 0x7f42c413f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:27.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c3fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c4125860 con 0x7f42c40c8230 2024-09-13T18:55:27.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c3fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c40ad960 con 0x7f42c40caed0 2024-09-13T18:55:27.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c413e8a0 con 0x7f42c40cb2b0 2024-09-13T18:55:27.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42c40c8230 0x7f42c40af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:27.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42c40c8230 0x7f42c40af3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:56580/0 (socket says 172.21.15.146:56580) 2024-09-13T18:55:27.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c2ffd700 1 -- 172.21.15.146:0/3129834993 learned_addr learned my addr 172.21.15.146:0/3129834993 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:27.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c37fe700 1 --2- 172.21.15.146:0/3129834993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c40cb2b0 0x7f42c413f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:27.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c2ffd700 1 -- 172.21.15.146:0/3129834993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c40cb2b0 msgr2=0x7f42c413f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:27.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c2ffd700 1 --2- 172.21.15.146:0/3129834993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c40cb2b0 0x7f42c413f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:27.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c2ffd700 1 -- 172.21.15.146:0/3129834993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42c40caed0 msgr2=0x7f42c413efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:27.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c2ffd700 1 --2- 172.21.15.146:0/3129834993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42c40caed0 0x7f42c413efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:27.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c2ffd700 1 -- 172.21.15.146:0/3129834993 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b40094b0 con 0x7f42c40c8230 2024-09-13T18:55:27.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c2ffd700 1 --2- 172.21.15.146:0/3129834993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42c40c8230 0x7f42c40af3a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00cc80 tx=0x7f42bc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:27.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42ab7fe700 1 -- 172.21.15.146:0/3129834993 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc00abb0 con 0x7f42c40c8230 2024-09-13T18:55:27.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42ab7fe700 1 -- 172.21.15.146:0/3129834993 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc00ad10 con 0x7f42c40c8230 2024-09-13T18:55:27.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c3fff700 1 -- 172.21.15.146:0/3129834993 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c413eae0 con 0x7f42c40c8230 2024-09-13T18:55:27.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42ab7fe700 1 -- 172.21.15.146:0/3129834993 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc007310 con 0x7f42c40c8230 2024-09-13T18:55:27.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.837+0000 7f42c3fff700 1 -- 172.21.15.146:0/3129834993 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c40ad3a0 con 0x7f42c40c8230 2024-09-13T18:55:27.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.841+0000 7f42ab7fe700 1 -- 172.21.15.146:0/3129834993 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f42bc007470 con 0x7f42c40c8230 2024-09-13T18:55:27.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.845+0000 7f42ab7fe700 1 --2- 172.21.15.146:0/3129834993 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f42b0066730 0x7f42b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:27.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.845+0000 7f42ab7fe700 1 -- 172.21.15.146:0/3129834993 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f42bc0845e0 con 0x7f42c40c8230 2024-09-13T18:55:27.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.845+0000 7f42c27fc700 1 --2- 172.21.15.146:0/3129834993 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f42b0066730 0x7f42b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:27.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.849+0000 7f42c27fc700 1 --2- 172.21.15.146:0/3129834993 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f42b0066730 0x7f42b0068bf0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f42c4160360 tx=0x7f42b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:27.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.849+0000 7f42c3fff700 1 -- 172.21.15.146:0/3129834993 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c40a9490 con 0x7f42c40c8230 2024-09-13T18:55:27.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:27.861+0000 7f42ab7fe700 1 -- 172.21.15.146:0/3129834993 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f42bc018020 con 0x7f42c40c8230 2024-09-13T18:55:28.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.805+0000 7f42c3fff700 1 -- 172.21.15.146:0/3129834993 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42c413ec70 con 0x7f42c40c8230 2024-09-13T18:55:28.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.805+0000 7f42ab7fe700 1 -- 172.21.15.146:0/3129834993 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+5535 (secure 0 0 0) 0x7f42bc050f90 con 0x7f42c40c8230 2024-09-13T18:55:28.820 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:28.820 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":113,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:55:25.945122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":7224,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7224":{"gid":7224,"name":"c","rank":1,"incarnation":113,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:28.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.813+0000 7f42a97fa700 1 -- 172.21.15.146:0/3129834993 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f42b0066730 msgr2=0x7f42b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:28.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.813+0000 7f42a97fa700 1 --2- 172.21.15.146:0/3129834993 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f42b0066730 0x7f42b0068bf0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f42c4160360 tx=0x7f42b8008040 comp rx=0 tx=0).stop 2024-09-13T18:55:28.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.813+0000 7f42a97fa700 1 -- 172.21.15.146:0/3129834993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42c40c8230 msgr2=0x7f42c40af3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:28.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.813+0000 7f42a97fa700 1 --2- 172.21.15.146:0/3129834993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42c40c8230 0x7f42c40af3a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00cc80 tx=0x7f42bc00cf90 comp rx=0 tx=0).stop 2024-09-13T18:55:28.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.817+0000 7f42a97fa700 1 -- 172.21.15.146:0/3129834993 shutdown_connections 2024-09-13T18:55:28.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.817+0000 7f42a97fa700 1 --2- 172.21.15.146:0/3129834993 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f42b0066730 0x7f42b0068bf0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:28.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.817+0000 7f42a97fa700 1 --2- 172.21.15.146:0/3129834993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c40cb2b0 0x7f42c413f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:28.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.817+0000 7f42a97fa700 1 --2- 172.21.15.146:0/3129834993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42c40caed0 0x7f42c413efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:28.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.817+0000 7f42a97fa700 1 --2- 172.21.15.146:0/3129834993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42c40c8230 0x7f42c40af3a0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:28.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.817+0000 7f42a97fa700 1 -- 172.21.15.146:0/3129834993 >> 172.21.15.146:0/3129834993 conn(0x7f42c4066090 msgr2=0x7f42c406d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:28.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.817+0000 7f42a97fa700 1 -- 172.21.15.146:0/3129834993 shutdown_connections 2024-09-13T18:55:28.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:28.817+0000 7f42a97fa700 1 -- 172.21.15.146:0/3129834993 wait complete. 2024-09-13T18:55:28.847 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 113 2024-09-13T18:55:28.879 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2024-09-13T18:55:28.879 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-09-13T18:55:28.879 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:29.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.369+0000 7fc1c3a12700 1 Processor -- start 2024-09-13T18:55:29.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.373+0000 7fc1c3a12700 1 -- start start 2024-09-13T18:55:29.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1c3a12700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc1bc14f630 0x7fc1bc0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:29.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1c3a12700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc1bc0a9490 0x7fc1bc155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:29.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1c3a12700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1bc06f5c0 con 0x7fc1bc156140 2024-09-13T18:55:29.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1c3a12700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1bc155420 con 0x7fc1bc14f630 2024-09-13T18:55:29.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1c3a12700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1bc0c7b70 con 0x7fc1bc0a9490 2024-09-13T18:55:29.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1c0fad700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc1bc0a9490 0x7fc1bc155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:29.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1c0fad700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc1bc0a9490 0x7fc1bc155c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:55564/0 (socket says 172.21.15.146:55564) 2024-09-13T18:55:29.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1c0fad700 1 -- 172.21.15.146:0/1221562987 learned_addr learned my addr 172.21.15.146:0/1221562987 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:29.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390537286 0 0) 0x7fc1bc06f5c0 con 0x7fc1bc156140 2024-09-13T18:55:29.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1a80035d0 con 0x7fc1bc156140 2024-09-13T18:55:29.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1480679791 0 0) 0x7fc1a80035d0 con 0x7fc1bc156140 2024-09-13T18:55:29.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 >> v2:172.21.15.154:3300/0 conn(0x7fc1bc0a9490 msgr2=0x7fc1bc155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:29.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 --2- 172.21.15.146:0/1221562987 >> v2:172.21.15.154:3300/0 conn(0x7fc1bc0a9490 0x7fc1bc155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:29.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 >> v2:172.21.15.146:3300/0 conn(0x7fc1bc14f630 msgr2=0x7fc1bc0c9500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:55:29.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 --2- 172.21.15.146:0/1221562987 >> v2:172.21.15.146:3300/0 conn(0x7fc1bc14f630 0x7fc1bc0c9500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:29.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc0c7e50 con 0x7fc1bc156140 2024-09-13T18:55:29.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7fc1ac001670 con 0x7fc1bc156140 2024-09-13T18:55:29.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 >> v1:172.21.15.146:6789/0 conn(0x7fc1bc156140 legacy=0x7fc1bc0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:29.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 --2- 172.21.15.146:0/1221562987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a8003ae0 0x7fc1a8003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:29.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 --2- 172.21.15.146:0/1221562987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8004870 0x7fc1a8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:29.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 --2- 172.21.15.146:0/1221562987 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a8005280 0x7fc1a800fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:29.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc155420 con 0x7fc1a8003ae0 2024-09-13T18:55:29.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0c7b70 con 0x7fc1a8004870 2024-09-13T18:55:29.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.377+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a8005a80 con 0x7fc1a8005280 2024-09-13T18:55:29.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.381+0000 7fc1c0fad700 1 --2- 172.21.15.146:0/1221562987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8004870 0x7fc1a8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:29.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.381+0000 7fc1c0fad700 1 -- 172.21.15.146:0/1221562987 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a8005280 msgr2=0x7fc1a800fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:29.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.381+0000 7fc1c0fad700 1 --2- 172.21.15.146:0/1221562987 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a8005280 0x7fc1a800fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:29.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.381+0000 7fc1c0fad700 1 -- 172.21.15.146:0/1221562987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a8003ae0 msgr2=0x7fc1a8003ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:55:29.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.397+0000 7fc1c17ae700 1 --2- 172.21.15.146:0/1221562987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a8003ae0 0x7fc1a8003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:29.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.397+0000 7fc1c0fad700 1 --2- 172.21.15.146:0/1221562987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a8003ae0 0x7fc1a8003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:29.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.397+0000 7fc1c1faf700 1 -- 172.21.15.146:0/1221562987 reap_dead start 2024-09-13T18:55:29.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.397+0000 7fc1c0fad700 1 -- 172.21.15.146:0/1221562987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a80035d0 con 0x7fc1a8004870 2024-09-13T18:55:29.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.397+0000 7fc1c0fad700 1 --2- 172.21.15.146:0/1221562987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8004870 0x7fc1a8004cf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400b310 tx=0x7fc1a80058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e6c09be144ec1d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:29.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.397+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4010070 con 0x7fc1a8004870 2024-09-13T18:55:29.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.397+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b400bc30 con 0x7fc1a8004870 2024-09-13T18:55:29.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.397+0000 7fc1b3fff700 1 -- 172.21.15.146:0/1221562987 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4007d70 con 0x7fc1a8004870 2024-09-13T18:55:29.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.397+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1221562987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8004870 msgr2=0x7fc1a8004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:29.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.397+0000 7fc1c3a12700 1 --2- 172.21.15.146:0/1221562987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8004870 0x7fc1a8004cf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400b310 tx=0x7fc1a80058a0 comp rx=0 tx=0).stop 2024-09-13T18:55:29.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.397+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1221562987 shutdown_connections 2024-09-13T18:55:29.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.397+0000 7fc1c3a12700 1 --2- 172.21.15.146:0/1221562987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8004870 0x7fc1a8004cf0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:29.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.397+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1221562987 >> 172.21.15.146:0/1221562987 conn(0x7fc1bc066090 msgr2=0x7fc1bc066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:29.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.401+0000 7fc1c17ae700 1 --2- 172.21.15.146:0/1221562987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a8003ae0 0x7fc1a8003ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:55:29.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.405+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1221562987 shutdown_connections 2024-09-13T18:55:29.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.405+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1221562987 wait complete. 2024-09-13T18:55:29.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c3a12700 1 Processor -- start 2024-09-13T18:55:29.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c3a12700 1 -- start start 2024-09-13T18:55:29.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c3a12700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8004870 0x7fc1bc1508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:29.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c3a12700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1bc14e600 0x7fc1bc14ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:29.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c3a12700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1bc153290 0x7fc1bc153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:29.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c3a12700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc153c50 con 0x7fc1bc153290 2024-09-13T18:55:29.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c3a12700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc14ef80 con 0x7fc1a8004870 2024-09-13T18:55:29.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c3a12700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc152900 con 0x7fc1bc14e600 2024-09-13T18:55:29.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c1faf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1bc153290 0x7fc1bc153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:29.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c1faf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1bc153290 0x7fc1bc153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:56600/0 (socket says 172.21.15.146:56600) 2024-09-13T18:55:29.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c1faf700 1 -- 172.21.15.146:0/1803951233 learned_addr learned my addr 172.21.15.146:0/1803951233 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:29.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c0fad700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1bc14e600 0x7fc1bc14ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:29.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c17ae700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8004870 0x7fc1bc1508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:29.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c1faf700 1 -- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1bc14e600 msgr2=0x7fc1bc14ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:29.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c1faf700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1bc14e600 0x7fc1bc14ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:29.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c1faf700 1 -- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8004870 msgr2=0x7fc1bc1508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:29.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c1faf700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8004870 0x7fc1bc1508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:29.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c1faf700 1 -- 172.21.15.146:0/1803951233 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc1ff0b0 con 0x7fc1bc153290 2024-09-13T18:55:29.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.409+0000 7fc1c17ae700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8004870 0x7fc1bc1508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:55:29.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.413+0000 7fc1c1faf700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1bc153290 0x7fc1bc153710 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc072510 tx=0x7fc1ac00d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:29.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.445+0000 7fc1b1ffb700 1 -- 172.21.15.146:0/1803951233 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac005c80 con 0x7fc1bc153290 2024-09-13T18:55:29.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.445+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1803951233 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc159530 con 0x7fc1bc153290 2024-09-13T18:55:29.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.445+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1803951233 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc1ff630 con 0x7fc1bc153290 2024-09-13T18:55:29.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.445+0000 7fc1b1ffb700 1 -- 172.21.15.146:0/1803951233 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac005e20 con 0x7fc1bc153290 2024-09-13T18:55:29.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.449+0000 7fc1b1ffb700 1 -- 172.21.15.146:0/1803951233 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac008270 con 0x7fc1bc153290 2024-09-13T18:55:29.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.449+0000 7fc1b1ffb700 1 -- 172.21.15.146:0/1803951233 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc1ac0083d0 con 0x7fc1bc153290 2024-09-13T18:55:29.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.449+0000 7fc1b1ffb700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc1a00668b0 0x7fc1a0068d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:29.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.449+0000 7fc1b1ffb700 1 -- 172.21.15.146:0/1803951233 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fc1ac084220 con 0x7fc1bc153290 2024-09-13T18:55:29.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.449+0000 7fc1c17ae700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc1a00668b0 0x7fc1a0068d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:29.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.449+0000 7fc1c17ae700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc1a00668b0 0x7fc1a0068d70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc075280 tx=0x7fc1b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:29.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.457+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1803951233 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19c005310 con 0x7fc1bc153290 2024-09-13T18:55:29.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:29.481+0000 7fc1b1ffb700 1 -- 172.21.15.146:0/1803951233 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc1ac050ce0 con 0x7fc1bc153290 2024-09-13T18:55:30.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.317+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1803951233 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc19c005b40 con 0x7fc1bc153290 2024-09-13T18:55:30.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.317+0000 7fc1b1ffb700 1 -- 172.21.15.146:0/1803951233 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+5535 (secure 0 0 0) 0x7fc1ac054300 con 0x7fc1bc153290 2024-09-13T18:55:30.331 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:30.331 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":113,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:55:25.945122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":7224,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7224":{"gid":7224,"name":"c","rank":1,"incarnation":113,"state":"up:replay","state_seq":1,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:30.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.321+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc1a00668b0 msgr2=0x7fc1a0068d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:30.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.321+0000 7fc1c3a12700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc1a00668b0 0x7fc1a0068d70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc075280 tx=0x7fc1b8008040 comp rx=0 tx=0).stop 2024-09-13T18:55:30.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.321+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1803951233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1bc153290 msgr2=0x7fc1bc153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:30.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.321+0000 7fc1c3a12700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1bc153290 0x7fc1bc153710 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc072510 tx=0x7fc1ac00d9e0 comp rx=0 tx=0).stop 2024-09-13T18:55:30.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.321+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1803951233 shutdown_connections 2024-09-13T18:55:30.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.321+0000 7fc1c3a12700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc1a00668b0 0x7fc1a0068d70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:30.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.321+0000 7fc1c3a12700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1bc153290 0x7fc1bc153710 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:30.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.321+0000 7fc1c3a12700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1bc14e600 0x7fc1bc14ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:30.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.321+0000 7fc1c3a12700 1 --2- 172.21.15.146:0/1803951233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8004870 0x7fc1bc1508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:30.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.325+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1803951233 >> 172.21.15.146:0/1803951233 conn(0x7fc1bc066090 msgr2=0x7fc1bc1547a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:30.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.329+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1803951233 shutdown_connections 2024-09-13T18:55:30.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:30.329+0000 7fc1c3a12700 1 -- 172.21.15.146:0/1803951233 wait complete. 2024-09-13T18:55:30.346 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 113 2024-09-13T18:55:30.362 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 113, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 113, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6984": { "addr": "172.21.15.146:6835/2934322065", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 2934322065, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 2934322065, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6984, "incarnation": 98, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 28 }, "gid_7098": { "addr": "172.21.15.146:6833/354253522", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 354253522, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 354253522, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7098, "incarnation": 105, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 9 }, "gid_7224": { "addr": "172.21.15.146:6837/3683135798", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3683135798, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3683135798, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7224, "incarnation": 113, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 43, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:55:25.945122+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6984, "mds_1": 7224, "mds_2": 7098 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:55:32.363 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:32.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.737+0000 7f6dc919b700 1 Processor -- start 2024-09-13T18:55:32.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.737+0000 7f6dc919b700 1 -- start start 2024-09-13T18:55:32.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.741+0000 7f6dc919b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6dc40cac60 0x7f6dc40cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:32.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.741+0000 7f6dc919b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6dc408baa0 0x7f6dc406f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:32.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.741+0000 7f6dc919b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6dc40c7530 0x7f6dc40c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:32.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.741+0000 7f6dc919b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc40723f0 con 0x7f6dc40c7530 2024-09-13T18:55:32.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.741+0000 7f6dc919b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc40cb580 con 0x7f6dc40cac60 2024-09-13T18:55:32.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.741+0000 7f6dc919b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc40a9920 con 0x7f6dc408baa0 2024-09-13T18:55:32.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.741+0000 7f6dc2d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6dc40cac60 0x7f6dc40cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:32.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.741+0000 7f6dc2d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6dc40cac60 0x7f6dc40cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:59466/0 (socket says 172.21.15.146:59466) 2024-09-13T18:55:32.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.741+0000 7f6dc2d9d700 1 -- 172.21.15.146:0/2482133711 learned_addr learned my addr 172.21.15.146:0/2482133711 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:32.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.741+0000 7f6dc359e700 1 --2- 172.21.15.146:0/2482133711 >> v2:172.21.15.146:3300/0 conn(0x7f6dc40c7530 0x7f6dc40c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:32.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.741+0000 7f6dc2d9d700 1 -- 172.21.15.146:0/2482133711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6dc408baa0 msgr2=0x7f6dc406f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:32.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.741+0000 7f6dc259c700 1 --2- 172.21.15.146:0/2482133711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6dc408baa0 0x7f6dc406f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:32.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc2d9d700 1 --2- 172.21.15.146:0/2482133711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6dc408baa0 0x7f6dc406f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:32.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc2d9d700 1 -- 172.21.15.146:0/2482133711 >> v2:172.21.15.146:3300/0 conn(0x7f6dc40c7530 msgr2=0x7f6dc40c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:32.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc2d9d700 1 --2- 172.21.15.146:0/2482133711 >> v2:172.21.15.146:3300/0 conn(0x7f6dc40c7530 0x7f6dc40c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:32.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc2d9d700 1 -- 172.21.15.146:0/2482133711 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc40725e0 con 0x7f6dc40cac60 2024-09-13T18:55:32.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc359e700 1 --2- 172.21.15.146:0/2482133711 >> v2:172.21.15.146:3300/0 conn(0x7f6dc40c7530 0x7f6dc40c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:55:32.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc259c700 1 --2- 172.21.15.146:0/2482133711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6dc408baa0 0x7f6dc406f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:55:32.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc2d9d700 1 --2- 172.21.15.146:0/2482133711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dc40cac60 0x7f6dc40cb040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6dc406ee00 tx=0x7f6db400ac40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e0a7ea08dd76b32 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:32.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc1d9b700 1 -- 172.21.15.146:0/2482133711 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4016e70 con 0x7f6dc40cac60 2024-09-13T18:55:32.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc1d9b700 1 -- 172.21.15.146:0/2482133711 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db4004030 con 0x7f6dc40cac60 2024-09-13T18:55:32.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc919b700 1 -- 172.21.15.146:0/2482133711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dc40cac60 msgr2=0x7f6dc40cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:32.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc919b700 1 --2- 172.21.15.146:0/2482133711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dc40cac60 0x7f6dc40cb040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6dc406ee00 tx=0x7f6db400ac40 comp rx=0 tx=0).stop 2024-09-13T18:55:32.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc919b700 1 -- 172.21.15.146:0/2482133711 shutdown_connections 2024-09-13T18:55:32.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc919b700 1 --2- 172.21.15.146:0/2482133711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6dc408baa0 0x7f6dc406f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:32.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc919b700 1 --2- 172.21.15.146:0/2482133711 >> v2:172.21.15.146:3300/0 conn(0x7f6dc40c7530 0x7f6dc40c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:32.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc919b700 1 --2- 172.21.15.146:0/2482133711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dc40cac60 0x7f6dc40cb040 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:32.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc919b700 1 -- 172.21.15.146:0/2482133711 >> 172.21.15.146:0/2482133711 conn(0x7f6dc4066090 msgr2=0x7f6dc4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:32.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc919b700 1 -- 172.21.15.146:0/2482133711 shutdown_connections 2024-09-13T18:55:32.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.745+0000 7f6dc919b700 1 -- 172.21.15.146:0/2482133711 wait complete. 2024-09-13T18:55:32.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc919b700 1 Processor -- start 2024-09-13T18:55:32.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc919b700 1 -- start start 2024-09-13T18:55:32.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc919b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dc408baa0 0x7f6dc41eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:32.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc919b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6dc40c7530 0x7f6dc40afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:32.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc919b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dc40cac60 0x7f6dc41e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:32.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc919b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc406c650 con 0x7f6dc40cac60 2024-09-13T18:55:32.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc919b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc41ff760 con 0x7f6dc408baa0 2024-09-13T18:55:32.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc919b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc40aee30 con 0x7f6dc40c7530 2024-09-13T18:55:32.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6dc40c7530 0x7f6dc40afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:32.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dc40cac60 0x7f6dc41e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:32.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dc40cac60 0x7f6dc41e7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33646/0 (socket says 172.21.15.146:33646) 2024-09-13T18:55:32.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc359e700 1 -- 172.21.15.146:0/1831447034 learned_addr learned my addr 172.21.15.146:0/1831447034 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:32.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6dc40c7530 0x7f6dc40afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:49678/0 (socket says 172.21.15.146:49678) 2024-09-13T18:55:32.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc2d9d700 1 --2- 172.21.15.146:0/1831447034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dc408baa0 0x7f6dc41eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:32.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc359e700 1 -- 172.21.15.146:0/1831447034 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6dc40c7530 msgr2=0x7f6dc40afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:32.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc359e700 1 --2- 172.21.15.146:0/1831447034 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6dc40c7530 0x7f6dc40afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc359e700 1 -- 172.21.15.146:0/1831447034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dc408baa0 msgr2=0x7f6dc41eb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc359e700 1 --2- 172.21.15.146:0/1831447034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dc408baa0 0x7f6dc41eb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.749+0000 7f6dc359e700 1 -- 172.21.15.146:0/1831447034 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db800d950 con 0x7f6dc40cac60 2024-09-13T18:55:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.753+0000 7f6dc359e700 1 --2- 172.21.15.146:0/1831447034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dc40cac60 0x7f6dc41e7630 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f6db800bf00 tx=0x7f6db800f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.753+0000 7f6db37fe700 1 -- 172.21.15.146:0/1831447034 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db800fb20 con 0x7f6dc40cac60 2024-09-13T18:55:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.753+0000 7f6db37fe700 1 -- 172.21.15.146:0/1831447034 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db800fc80 con 0x7f6dc40cac60 2024-09-13T18:55:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.753+0000 7f6db37fe700 1 -- 172.21.15.146:0/1831447034 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db800d280 con 0x7f6dc40cac60 2024-09-13T18:55:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.753+0000 7f6dc919b700 1 -- 172.21.15.146:0/1831447034 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db400a560 con 0x7f6dc40cac60 2024-09-13T18:55:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.753+0000 7f6dc919b700 1 -- 172.21.15.146:0/1831447034 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc40af3f0 con 0x7f6dc40cac60 2024-09-13T18:55:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.757+0000 7f6db37fe700 1 -- 172.21.15.146:0/1831447034 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6db8014070 con 0x7f6dc40cac60 2024-09-13T18:55:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.757+0000 7f6db37fe700 1 --2- 172.21.15.146:0/1831447034 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6da8066830 0x7f6da8068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:32.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.757+0000 7f6db37fe700 1 -- 172.21.15.146:0/1831447034 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f6db80842f0 con 0x7f6dc40cac60 2024-09-13T18:55:32.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.757+0000 7f6dc919b700 1 -- 172.21.15.146:0/1831447034 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc40c9da0 con 0x7f6dc40cac60 2024-09-13T18:55:32.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.761+0000 7f6dc2d9d700 1 --2- 172.21.15.146:0/1831447034 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6da8066830 0x7f6da8068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:32.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.769+0000 7f6db37fe700 1 -- 172.21.15.146:0/1831447034 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6db801b080 con 0x7f6dc40cac60 2024-09-13T18:55:32.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:32.769+0000 7f6dc2d9d700 1 --2- 172.21.15.146:0/1831447034 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6da8066830 0x7f6da8068cf0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6db400a210 tx=0x7f6db401a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:33.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.665+0000 7f6dc919b700 1 -- 172.21.15.146:0/1831447034 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dc4066470 con 0x7f6dc40cac60 2024-09-13T18:55:33.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.665+0000 7f6db37fe700 1 -- 172.21.15.146:0/1831447034 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+5537 (secure 0 0 0) 0x7f6db8050d20 con 0x7f6dc40cac60 2024-09-13T18:55:33.681 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:33.681 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":114,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:55:31.824605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":7224,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7224":{"gid":7224,"name":"c","rank":1,"incarnation":113,"state":"up:resolve","state_seq":19,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:33.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.669+0000 7f6db17fa700 1 -- 172.21.15.146:0/1831447034 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6da8066830 msgr2=0x7f6da8068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:33.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.669+0000 7f6db17fa700 1 --2- 172.21.15.146:0/1831447034 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6da8066830 0x7f6da8068cf0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6db400a210 tx=0x7f6db401a000 comp rx=0 tx=0).stop 2024-09-13T18:55:33.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.669+0000 7f6db17fa700 1 -- 172.21.15.146:0/1831447034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dc40cac60 msgr2=0x7f6dc41e7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:33.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.669+0000 7f6db17fa700 1 --2- 172.21.15.146:0/1831447034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dc40cac60 0x7f6dc41e7630 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f6db800bf00 tx=0x7f6db800f390 comp rx=0 tx=0).stop 2024-09-13T18:55:33.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.673+0000 7f6db17fa700 1 -- 172.21.15.146:0/1831447034 shutdown_connections 2024-09-13T18:55:33.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.673+0000 7f6db17fa700 1 --2- 172.21.15.146:0/1831447034 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6da8066830 0x7f6da8068cf0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:33.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.673+0000 7f6db17fa700 1 --2- 172.21.15.146:0/1831447034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dc40cac60 0x7f6dc41e7630 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:33.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.673+0000 7f6db17fa700 1 --2- 172.21.15.146:0/1831447034 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6dc40c7530 0x7f6dc40afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:33.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.673+0000 7f6db17fa700 1 --2- 172.21.15.146:0/1831447034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dc408baa0 0x7f6dc41eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:33.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.677+0000 7f6db17fa700 1 -- 172.21.15.146:0/1831447034 >> 172.21.15.146:0/1831447034 conn(0x7f6dc4066090 msgr2=0x7f6dc41ee020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:33.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.677+0000 7f6db17fa700 1 -- 172.21.15.146:0/1831447034 shutdown_connections 2024-09-13T18:55:33.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:33.677+0000 7f6db17fa700 1 -- 172.21.15.146:0/1831447034 wait complete. 2024-09-13T18:55:33.695 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 114 2024-09-13T18:55:33.708 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:55:35.709 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:36.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.177+0000 7f7545e81700 1 Processor -- start 2024-09-13T18:55:36.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.177+0000 7f7545e81700 1 -- start start 2024-09-13T18:55:36.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.181+0000 7f7545e81700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540153bc0 0x7f75400c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:36.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.181+0000 7f7545e81700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f75400c6610 0x7f75400c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:36.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.181+0000 7f7545e81700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7540153fa0 con 0x7f75400cac60 2024-09-13T18:55:36.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.181+0000 7f7545e81700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75400aa500 con 0x7f75400c6610 2024-09-13T18:55:36.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.181+0000 7f7545e81700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754015a7e0 con 0x7f7540153bc0 2024-09-13T18:55:36.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.185+0000 7f753ffff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f75400cac60 0x7f75400c7740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34572/0 (socket says 172.21.15.146:34572) 2024-09-13T18:55:36.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.185+0000 7f753ffff700 1 -- 172.21.15.146:0/435996693 learned_addr learned my addr 172.21.15.146:0/435996693 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:55:36.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.185+0000 7f753f7fe700 1 --2- 172.21.15.146:0/435996693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540153bc0 0x7f75400c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:36.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.193+0000 7f753e7fc700 1 -- 172.21.15.146:0/435996693 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3241439539 0 0) 0x7f7540153fa0 con 0x7f75400cac60 2024-09-13T18:55:36.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.193+0000 7f753e7fc700 1 -- 172.21.15.146:0/435996693 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75280035d0 con 0x7f75400cac60 2024-09-13T18:55:36.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.193+0000 7f753f7fe700 1 -- 172.21.15.146:0/435996693 >> v2:172.21.15.154:3300/0 conn(0x7f75400c6610 msgr2=0x7f75400c6a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:55:36.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.193+0000 7f753f7fe700 1 --2- 172.21.15.146:0/435996693 >> v2:172.21.15.154:3300/0 conn(0x7f75400c6610 0x7f75400c6a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:36.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.193+0000 7f753f7fe700 1 -- 172.21.15.146:0/435996693 >> v1:172.21.15.146:6789/0 conn(0x7f75400cac60 legacy=0x7f75400c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:36.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.193+0000 7f753f7fe700 1 -- 172.21.15.146:0/435996693 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75400c9320 con 0x7f7540153bc0 2024-09-13T18:55:36.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.197+0000 7f753f7fe700 1 --2- 172.21.15.146:0/435996693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540153bc0 0x7f75400c8de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7534009850 tx=0x7f7534009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dab31b3edec3c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:36.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.197+0000 7f753e7fc700 1 -- 172.21.15.146:0/435996693 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75380019f0 con 0x7f7540153bc0 2024-09-13T18:55:36.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.197+0000 7f753e7fc700 1 -- 172.21.15.146:0/435996693 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7534010590 con 0x7f7540153bc0 2024-09-13T18:55:36.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.197+0000 7f7545e81700 1 -- 172.21.15.146:0/435996693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540153bc0 msgr2=0x7f75400c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:36.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.197+0000 7f7545e81700 1 --2- 172.21.15.146:0/435996693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540153bc0 0x7f75400c8de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7534009850 tx=0x7f7534009b60 comp rx=0 tx=0).stop 2024-09-13T18:55:36.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.209+0000 7f7545e81700 1 -- 172.21.15.146:0/435996693 shutdown_connections 2024-09-13T18:55:36.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.209+0000 7f7545e81700 1 --2- 172.21.15.146:0/435996693 >> v2:172.21.15.154:3300/0 conn(0x7f75400c6610 0x7f75400c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:36.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.209+0000 7f7545e81700 1 --2- 172.21.15.146:0/435996693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540153bc0 0x7f75400c8de0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:36.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.209+0000 7f7545e81700 1 -- 172.21.15.146:0/435996693 >> 172.21.15.146:0/435996693 conn(0x7f7540066090 msgr2=0x7f7540066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:36.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.217+0000 7f7545e81700 1 -- 172.21.15.146:0/435996693 shutdown_connections 2024-09-13T18:55:36.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.217+0000 7f7545e81700 1 -- 172.21.15.146:0/435996693 wait complete. 2024-09-13T18:55:36.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.217+0000 7f7545e81700 1 Processor -- start 2024-09-13T18:55:36.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.217+0000 7f7545e81700 1 -- start start 2024-09-13T18:55:36.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.217+0000 7f7545e81700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75400c6610 0x7f75400c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:36.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.217+0000 7f7545e81700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75400cac60 0x7f75402011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:36.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.217+0000 7f7545e81700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540153bc0 0x7f75401ff560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:36.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.217+0000 7f7545e81700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75400c49d0 con 0x7f75400cac60 2024-09-13T18:55:36.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.217+0000 7f7545e81700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75400c3b50 con 0x7f75400c6610 2024-09-13T18:55:36.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.217+0000 7f7545e81700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75401ff0d0 con 0x7f7540153bc0 2024-09-13T18:55:36.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.217+0000 7f753f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75400c6610 0x7f75400c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:36.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.217+0000 7f753f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75400c6610 0x7f75400c4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:59484/0 (socket says 172.21.15.146:59484) 2024-09-13T18:55:36.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.217+0000 7f753f7fe700 1 -- 172.21.15.146:0/2912318882 learned_addr learned my addr 172.21.15.146:0/2912318882 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:36.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.221+0000 7f753ffff700 1 --2- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540153bc0 0x7f75401ff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:36.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.221+0000 7f753ffff700 1 -- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75400c6610 msgr2=0x7f75400c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:36.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.221+0000 7f753ffff700 1 --2- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75400c6610 0x7f75400c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:36.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.221+0000 7f753ffff700 1 -- 172.21.15.146:0/2912318882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75400cac60 msgr2=0x7f75402011c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:55:36.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.221+0000 7f753ffff700 1 --2- 172.21.15.146:0/2912318882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75400cac60 0x7f75402011c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:36.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.221+0000 7f753ffff700 1 -- 172.21.15.146:0/2912318882 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75340094b0 con 0x7f7540153bc0 2024-09-13T18:55:36.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.221+0000 7f753ffff700 1 --2- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540153bc0 0x7f75401ff560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f753800cfc0 tx=0x7f753800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:36.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.221+0000 7f752ffff700 1 -- 172.21.15.146:0/2912318882 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753800ada0 con 0x7f7540153bc0 2024-09-13T18:55:36.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.225+0000 7f7545e81700 1 -- 172.21.15.146:0/2912318882 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75400ad4a0 con 0x7f7540153bc0 2024-09-13T18:55:36.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.225+0000 7f7545e81700 1 -- 172.21.15.146:0/2912318882 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75400a9f30 con 0x7f7540153bc0 2024-09-13T18:55:36.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.225+0000 7f752ffff700 1 -- 172.21.15.146:0/2912318882 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75380020b0 con 0x7f7540153bc0 2024-09-13T18:55:36.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.225+0000 7f752ffff700 1 -- 172.21.15.146:0/2912318882 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7538011780 con 0x7f7540153bc0 2024-09-13T18:55:36.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.225+0000 7f7545e81700 1 -- 172.21.15.146:0/2912318882 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751c005310 con 0x7f7540153bc0 2024-09-13T18:55:36.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.229+0000 7f752ffff700 1 -- 172.21.15.146:0/2912318882 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7538017020 con 0x7f7540153bc0 2024-09-13T18:55:36.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.229+0000 7f752ffff700 1 --2- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7524066730 0x7f7524068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:36.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.229+0000 7f753f7fe700 1 --2- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7524066730 0x7f7524068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:36.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.229+0000 7f752ffff700 1 -- 172.21.15.146:0/2912318882 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f7538088de0 con 0x7f7540153bc0 2024-09-13T18:55:36.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.229+0000 7f753f7fe700 1 --2- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7524066730 0x7f7524068bf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7534009ba0 tx=0x7f753401c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:36.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.245+0000 7f752ffff700 1 -- 172.21.15.146:0/2912318882 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7538055820 con 0x7f7540153bc0 2024-09-13T18:55:36.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.981+0000 7f7545e81700 1 -- 172.21.15.146:0/2912318882 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f751c005b40 con 0x7f7540153bc0 2024-09-13T18:55:37.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.993+0000 7f752ffff700 1 -- 172.21.15.146:0/2912318882 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+5539 (secure 0 0 0) 0x7f7538020030 con 0x7f7540153bc0 2024-09-13T18:55:37.006 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:37.007 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":115,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:55:36.873817+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":7224,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7224":{"gid":7224,"name":"c","rank":1,"incarnation":113,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:37.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.997+0000 7f7545e81700 1 -- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7524066730 msgr2=0x7f7524068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:37.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.997+0000 7f7545e81700 1 --2- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7524066730 0x7f7524068bf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7534009ba0 tx=0x7f753401c040 comp rx=0 tx=0).stop 2024-09-13T18:55:37.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.997+0000 7f7545e81700 1 -- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540153bc0 msgr2=0x7f75401ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:37.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.997+0000 7f7545e81700 1 --2- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540153bc0 0x7f75401ff560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f753800cfc0 tx=0x7f753800a340 comp rx=0 tx=0).stop 2024-09-13T18:55:37.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.997+0000 7f7545e81700 1 -- 172.21.15.146:0/2912318882 shutdown_connections 2024-09-13T18:55:37.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.997+0000 7f7545e81700 1 --2- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7524066730 0x7f7524068bf0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:37.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.997+0000 7f7545e81700 1 --2- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540153bc0 0x7f75401ff560 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:37.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.997+0000 7f7545e81700 1 --2- 172.21.15.146:0/2912318882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75400cac60 0x7f75402011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:37.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.997+0000 7f7545e81700 1 --2- 172.21.15.146:0/2912318882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75400c6610 0x7f75400c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:37.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.997+0000 7f7545e81700 1 -- 172.21.15.146:0/2912318882 >> 172.21.15.146:0/2912318882 conn(0x7f7540066090 msgr2=0x7f75400c6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:37.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.997+0000 7f7545e81700 1 -- 172.21.15.146:0/2912318882 shutdown_connections 2024-09-13T18:55:37.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:36.997+0000 7f7545e81700 1 -- 172.21.15.146:0/2912318882 wait complete. 2024-09-13T18:55:37.013 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 115 2024-09-13T18:55:37.060 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:55:39.061 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:39.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.373+0000 7f525ecd3700 1 Processor -- start 2024-09-13T18:55:39.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.373+0000 7f525ecd3700 1 -- start start 2024-09-13T18:55:39.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.373+0000 7f525ecd3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f525808baa0 0x7f52580cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:39.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.373+0000 7f525ecd3700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52580a9920 con 0x7f52580cac60 2024-09-13T18:55:39.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.373+0000 7f525ecd3700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5258070070 con 0x7f525816e0c0 2024-09-13T18:55:39.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.373+0000 7f525ecd3700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52580723f0 con 0x7f525808baa0 2024-09-13T18:55:39.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.389+0000 7f525ca6f700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f52580cac60 0x7f525806f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34582/0 (socket says 172.21.15.146:34582) 2024-09-13T18:55:39.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.389+0000 7f525ca6f700 1 -- 172.21.15.146:0/3121838795 learned_addr learned my addr 172.21.15.146:0/3121838795 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:55:39.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.389+0000 7f5257fff700 1 --2- 172.21.15.146:0/3121838795 >> v2:172.21.15.154:3300/0 conn(0x7f525808baa0 0x7f52580cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:39.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.389+0000 7f5257fff700 1 -- 172.21.15.146:0/3121838795 >> v1:172.21.15.154:6789/0 conn(0x7f525816e0c0 legacy=0x7f52581704b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:55:39.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.389+0000 7f5257fff700 1 -- 172.21.15.146:0/3121838795 >> v1:172.21.15.146:6789/0 conn(0x7f52580cac60 legacy=0x7f525806f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:39.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.389+0000 7f5257fff700 1 -- 172.21.15.146:0/3121838795 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52580725e0 con 0x7f525808baa0 2024-09-13T18:55:39.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.389+0000 7f5257fff700 1 --2- 172.21.15.146:0/3121838795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f525808baa0 0x7f52580cb040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f525000bd20 tx=0x7f5250010730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ce7cded7963750c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:39.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.389+0000 7f52577fe700 1 -- 172.21.15.146:0/3121838795 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250005ea0 con 0x7f525808baa0 2024-09-13T18:55:39.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.389+0000 7f52577fe700 1 -- 172.21.15.146:0/3121838795 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5250003a40 con 0x7f525808baa0 2024-09-13T18:55:39.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.389+0000 7f525ecd3700 1 -- 172.21.15.146:0/3121838795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f525808baa0 msgr2=0x7f52580cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:39.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.389+0000 7f525ecd3700 1 --2- 172.21.15.146:0/3121838795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f525808baa0 0x7f52580cb040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f525000bd20 tx=0x7f5250010730 comp rx=0 tx=0).stop 2024-09-13T18:55:39.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.397+0000 7f525ecd3700 1 -- 172.21.15.146:0/3121838795 shutdown_connections 2024-09-13T18:55:39.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.397+0000 7f525ecd3700 1 --2- 172.21.15.146:0/3121838795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f525808baa0 0x7f52580cb040 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:39.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.397+0000 7f525ecd3700 1 -- 172.21.15.146:0/3121838795 >> 172.21.15.146:0/3121838795 conn(0x7f5258066090 msgr2=0x7f5258066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:39.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.397+0000 7f525ecd3700 1 -- 172.21.15.146:0/3121838795 shutdown_connections 2024-09-13T18:55:39.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.397+0000 7f525ecd3700 1 -- 172.21.15.146:0/3121838795 wait complete. 2024-09-13T18:55:39.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.397+0000 7f525ecd3700 1 Processor -- start 2024-09-13T18:55:39.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.401+0000 7f525ecd3700 1 -- start start 2024-09-13T18:55:39.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525ecd3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f525808baa0 0x7f52580841d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:39.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525ecd3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52580cac60 0x7f5258089fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:39.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525ecd3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f525816e0c0 0x7f52580c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:39.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525ecd3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580c50f0 con 0x7f525816e0c0 2024-09-13T18:55:39.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525ecd3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525808a4e0 con 0x7f52580cac60 2024-09-13T18:55:39.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525ecd3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52580ad780 con 0x7f525808baa0 2024-09-13T18:55:39.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525ca6f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f525808baa0 0x7f52580841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:39.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525ca6f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f525808baa0 0x7f52580841d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:49710/0 (socket says 172.21.15.146:49710) 2024-09-13T18:55:39.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525ca6f700 1 -- 172.21.15.146:0/2058265964 learned_addr learned my addr 172.21.15.146:0/2058265964 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:39.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f5257fff700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52580cac60 0x7f5258089fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:39.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525d270700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f525816e0c0 0x7f52580c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:39.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525d270700 1 -- 172.21.15.146:0/2058265964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f525808baa0 msgr2=0x7f52580841d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:39.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525d270700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f525808baa0 0x7f52580841d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:39.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525d270700 1 -- 172.21.15.146:0/2058265964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52580cac60 msgr2=0x7f5258089fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:39.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525d270700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52580cac60 0x7f5258089fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:39.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525d270700 1 -- 172.21.15.146:0/2058265964 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525000b980 con 0x7f525816e0c0 2024-09-13T18:55:39.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525d270700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f525816e0c0 0x7f52580c3f80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f524c009fd0 tx=0x7f524c00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:39.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f52557fa700 1 -- 172.21.15.146:0/2058265964 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c011620 con 0x7f525816e0c0 2024-09-13T18:55:39.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525ecd3700 1 -- 172.21.15.146:0/2058265964 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52582040b0 con 0x7f525816e0c0 2024-09-13T18:55:39.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.405+0000 7f525ecd3700 1 -- 172.21.15.146:0/2058265964 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5258204370 con 0x7f525816e0c0 2024-09-13T18:55:39.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.413+0000 7f525ecd3700 1 -- 172.21.15.146:0/2058265964 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52580c9da0 con 0x7f525816e0c0 2024-09-13T18:55:39.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.421+0000 7f52557fa700 1 -- 172.21.15.146:0/2058265964 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c0117a0 con 0x7f525816e0c0 2024-09-13T18:55:39.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.429+0000 7f52557fa700 1 -- 172.21.15.146:0/2058265964 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c011b50 con 0x7f525816e0c0 2024-09-13T18:55:39.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.429+0000 7f52557fa700 1 -- 172.21.15.146:0/2058265964 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f524c011cf0 con 0x7f525816e0c0 2024-09-13T18:55:39.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.437+0000 7f52557fa700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5244066880 0x7f5244068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:39.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.437+0000 7f52557fa700 1 -- 172.21.15.146:0/2058265964 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f524c084850 con 0x7f525816e0c0 2024-09-13T18:55:39.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.437+0000 7f52557fa700 1 -- 172.21.15.146:0/2058265964 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f524c086d90 con 0x7f525816e0c0 2024-09-13T18:55:39.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.437+0000 7f525ca6f700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5244066880 0x7f5244068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:39.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:39.445+0000 7f525ca6f700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5244066880 0x7f5244068d40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f52480049a0 tx=0x7f5248009580 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:40.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.229+0000 7f525ecd3700 1 -- 172.21.15.146:0/2058265964 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52580c82c0 con 0x7f525816e0c0 2024-09-13T18:55:40.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.229+0000 7f52557fa700 1 -- 172.21.15.146:0/2058265964 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+5539 (secure 0 0 0) 0x7f524c015090 con 0x7f525816e0c0 2024-09-13T18:55:40.246 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:40.246 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":115,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:55:36.873817+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":7224,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7224":{"gid":7224,"name":"c","rank":1,"incarnation":113,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:40.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.237+0000 7f525ecd3700 1 -- 172.21.15.146:0/2058265964 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5244066880 msgr2=0x7f5244068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:40.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.237+0000 7f525ecd3700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5244066880 0x7f5244068d40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f52480049a0 tx=0x7f5248009580 comp rx=0 tx=0).stop 2024-09-13T18:55:40.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.237+0000 7f525ecd3700 1 -- 172.21.15.146:0/2058265964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f525816e0c0 msgr2=0x7f52580c3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:40.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.241+0000 7f525ecd3700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f525816e0c0 0x7f52580c3f80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f524c009fd0 tx=0x7f524c00b6c0 comp rx=0 tx=0).stop 2024-09-13T18:55:40.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.241+0000 7f525ecd3700 1 -- 172.21.15.146:0/2058265964 shutdown_connections 2024-09-13T18:55:40.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.241+0000 7f525ecd3700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5244066880 0x7f5244068d40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:40.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.241+0000 7f525ecd3700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f525816e0c0 0x7f52580c3f80 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:40.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.241+0000 7f525ecd3700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52580cac60 0x7f5258089fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:40.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.241+0000 7f525ecd3700 1 --2- 172.21.15.146:0/2058265964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f525808baa0 0x7f52580841d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:40.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.241+0000 7f525ecd3700 1 -- 172.21.15.146:0/2058265964 >> 172.21.15.146:0/2058265964 conn(0x7f5258066090 msgr2=0x7f52580afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:40.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.241+0000 7f525ecd3700 1 -- 172.21.15.146:0/2058265964 shutdown_connections 2024-09-13T18:55:40.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:40.241+0000 7f525ecd3700 1 -- 172.21.15.146:0/2058265964 wait complete. 2024-09-13T18:55:40.254 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 115 2024-09-13T18:55:40.270 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:55:42.271 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:42.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.817+0000 7f9307bcb700 1 Processor -- start 2024-09-13T18:55:42.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.817+0000 7f9307bcb700 1 -- start start 2024-09-13T18:55:42.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.825+0000 7f9307bcb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f930014f630 0x7f93000c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:42.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.825+0000 7f9307bcb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f93000cb120 0x7f93000cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:42.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.825+0000 7f9307bcb700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f930006f5c0 con 0x7f93000a9490 2024-09-13T18:55:42.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.825+0000 7f9307bcb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9300155420 con 0x7f93000cb120 2024-09-13T18:55:42.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.825+0000 7f9307bcb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93000c7b70 con 0x7f930014f630 2024-09-13T18:55:42.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.829+0000 7f9305967700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f930014f630 0x7f93000c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:42.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.829+0000 7f9305967700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f930014f630 0x7f93000c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:40720/0 (socket says 172.21.15.146:40720) 2024-09-13T18:55:42.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.829+0000 7f9305967700 1 -- 172.21.15.146:0/4056179635 learned_addr learned my addr 172.21.15.146:0/4056179635 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:42.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.829+0000 7f9305967700 1 -- 172.21.15.146:0/4056179635 >> v2:172.21.15.154:3300/0 conn(0x7f93000cb120 msgr2=0x7f93000cb5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:55:42.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.829+0000 7f9305967700 1 --2- 172.21.15.146:0/4056179635 >> v2:172.21.15.154:3300/0 conn(0x7f93000cb120 0x7f93000cb5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:42.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.829+0000 7f9305967700 1 -- 172.21.15.146:0/4056179635 >> v1:172.21.15.154:6789/0 conn(0x7f93000a9490 legacy=0x7f9300155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:42.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.829+0000 7f9305967700 1 -- 172.21.15.146:0/4056179635 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93000c7e50 con 0x7f930014f630 2024-09-13T18:55:42.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.829+0000 7f9305967700 1 --2- 172.21.15.146:0/4056179635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f930014f630 0x7f93000c9500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f92fc008940 tx=0x7f92fc008970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad82efe228da482 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:42.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.829+0000 7f9304965700 1 -- 172.21.15.146:0/4056179635 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc010070 con 0x7f930014f630 2024-09-13T18:55:42.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.829+0000 7f9304965700 1 -- 172.21.15.146:0/4056179635 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92fc00da30 con 0x7f930014f630 2024-09-13T18:55:42.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.833+0000 7f9307bcb700 1 -- 172.21.15.146:0/4056179635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f930014f630 msgr2=0x7f93000c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:42.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.833+0000 7f9307bcb700 1 --2- 172.21.15.146:0/4056179635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f930014f630 0x7f93000c9500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f92fc008940 tx=0x7f92fc008970 comp rx=0 tx=0).stop 2024-09-13T18:55:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.837+0000 7f9307bcb700 1 -- 172.21.15.146:0/4056179635 shutdown_connections 2024-09-13T18:55:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.837+0000 7f9307bcb700 1 --2- 172.21.15.146:0/4056179635 >> v2:172.21.15.154:3300/0 conn(0x7f93000cb120 0x7f93000cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.837+0000 7f9307bcb700 1 --2- 172.21.15.146:0/4056179635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f930014f630 0x7f93000c9500 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.837+0000 7f9307bcb700 1 -- 172.21.15.146:0/4056179635 >> 172.21.15.146:0/4056179635 conn(0x7f9300066090 msgr2=0x7f9300066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.837+0000 7f9307bcb700 1 -- 172.21.15.146:0/4056179635 shutdown_connections 2024-09-13T18:55:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.837+0000 7f9307bcb700 1 -- 172.21.15.146:0/4056179635 wait complete. 2024-09-13T18:55:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9307bcb700 1 Processor -- start 2024-09-13T18:55:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9307bcb700 1 -- start start 2024-09-13T18:55:42.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9307bcb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93000a9490 0x7f930014e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:42.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9307bcb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93000cb120 0x7f930006f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9307bcb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f930014f630 0x7f9300157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9305967700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93000a9490 0x7f930014e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9305967700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93000a9490 0x7f930014e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:49286/0 (socket says 172.21.15.146:49286) 2024-09-13T18:55:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9305166700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93000cb120 0x7f930006f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9305166700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93000cb120 0x7f930006f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36932/0 (socket says 172.21.15.146:36932) 2024-09-13T18:55:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9305166700 1 -- 172.21.15.146:0/1922904373 learned_addr learned my addr 172.21.15.146:0/1922904373 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9307bcb700 1 -- 172.21.15.146:0/1922904373 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93001574a0 con 0x7f93000cb120 2024-09-13T18:55:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9307bcb700 1 -- 172.21.15.146:0/1922904373 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93000c3540 con 0x7f93000a9490 2024-09-13T18:55:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9306168700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f930014f630 0x7f9300157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9307bcb700 1 -- 172.21.15.146:0/1922904373 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93000c3800 con 0x7f930014f630 2024-09-13T18:55:42.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9305166700 1 -- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f930014f630 msgr2=0x7f9300157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:42.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9305166700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f930014f630 0x7f9300157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:42.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9305166700 1 -- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93000a9490 msgr2=0x7f930014e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:42.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9305166700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93000a9490 0x7f930014e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:42.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9305166700 1 -- 172.21.15.146:0/1922904373 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc0085f0 con 0x7f93000cb120 2024-09-13T18:55:42.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9306168700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f930014f630 0x7f9300157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:55:42.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9305166700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93000cb120 0x7f930006f9c0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f92f000c4d0 tx=0x7f92f000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:42.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.841+0000 7f9305967700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93000a9490 0x7f930014e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:55:42.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.849+0000 7f92f67fc700 1 -- 172.21.15.146:0/1922904373 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f00052d0 con 0x7f93000cb120 2024-09-13T18:55:42.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.849+0000 7f9307bcb700 1 -- 172.21.15.146:0/1922904373 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93000c3ae0 con 0x7f93000cb120 2024-09-13T18:55:42.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.849+0000 7f9307bcb700 1 -- 172.21.15.146:0/1922904373 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93000c40a0 con 0x7f93000cb120 2024-09-13T18:55:42.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.849+0000 7f92f67fc700 1 -- 172.21.15.146:0/1922904373 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f0005430 con 0x7f93000cb120 2024-09-13T18:55:42.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.853+0000 7f92f67fc700 1 -- 172.21.15.146:0/1922904373 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f00055a0 con 0x7f93000cb120 2024-09-13T18:55:42.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.853+0000 7f92f67fc700 1 -- 172.21.15.146:0/1922904373 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f92f00038a0 con 0x7f93000cb120 2024-09-13T18:55:42.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.853+0000 7f92f67fc700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f92e40668b0 0x7f92e4068d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:42.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.853+0000 7f92f67fc700 1 -- 172.21.15.146:0/1922904373 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f92f0083d60 con 0x7f93000cb120 2024-09-13T18:55:42.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.853+0000 7f9305967700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f92e40668b0 0x7f92e4068d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:42.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.853+0000 7f9305967700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f92e40668b0 0x7f92e4068d70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f92fc008910 tx=0x7f92fc012000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:42.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.857+0000 7f92ebfff700 1 -- 172.21.15.146:0/1922904373 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e0005310 con 0x7f93000cb120 2024-09-13T18:55:42.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:42.869+0000 7f92f67fc700 1 -- 172.21.15.146:0/1922904373 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f92f0017080 con 0x7f93000cb120 2024-09-13T18:55:43.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.753+0000 7f92ebfff700 1 -- 172.21.15.146:0/1922904373 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92e0005b40 con 0x7f93000cb120 2024-09-13T18:55:43.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.753+0000 7f92f67fc700 1 -- 172.21.15.146:0/1922904373 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5536 (secure 0 0 0) 0x7f92f0050840 con 0x7f93000cb120 2024-09-13T18:55:43.772 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:43.772 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:55:41.923318+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":7224,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7224":{"gid":7224,"name":"c","rank":1,"incarnation":113,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:43.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.757+0000 7f9307bcb700 1 -- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f92e40668b0 msgr2=0x7f92e4068d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:43.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.757+0000 7f9307bcb700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f92e40668b0 0x7f92e4068d70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f92fc008910 tx=0x7f92fc012000 comp rx=0 tx=0).stop 2024-09-13T18:55:43.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.757+0000 7f9307bcb700 1 -- 172.21.15.146:0/1922904373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93000cb120 msgr2=0x7f930006f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:43.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.757+0000 7f9307bcb700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93000cb120 0x7f930006f9c0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f92f000c4d0 tx=0x7f92f000c7e0 comp rx=0 tx=0).stop 2024-09-13T18:55:43.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.757+0000 7f9307bcb700 1 -- 172.21.15.146:0/1922904373 shutdown_connections 2024-09-13T18:55:43.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.757+0000 7f9307bcb700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f92e40668b0 0x7f92e4068d70 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:43.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.757+0000 7f9307bcb700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f930014f630 0x7f9300157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:43.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.757+0000 7f9307bcb700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93000cb120 0x7f930006f9c0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:43.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.757+0000 7f9307bcb700 1 --2- 172.21.15.146:0/1922904373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93000a9490 0x7f930014e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:43.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.757+0000 7f9307bcb700 1 -- 172.21.15.146:0/1922904373 >> 172.21.15.146:0/1922904373 conn(0x7f9300066090 msgr2=0x7f9300154d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:43.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.757+0000 7f9307bcb700 1 -- 172.21.15.146:0/1922904373 shutdown_connections 2024-09-13T18:55:43.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:43.757+0000 7f9307bcb700 1 -- 172.21.15.146:0/1922904373 wait complete. 2024-09-13T18:55:43.779 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 116 2024-09-13T18:55:43.815 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-09-13T18:55:45.817 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:46.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.670+0000 7fa211f91700 1 Processor -- start 2024-09-13T18:55:46.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.670+0000 7fa211f91700 1 -- start start 2024-09-13T18:55:46.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.670+0000 7fa211f91700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa20c0cac60 0x7fa20c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:46.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.670+0000 7fa211f91700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa20c08baa0 0x7fa20c06f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:46.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.670+0000 7fa211f91700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20c0a9920 con 0x7fa20c0c7530 2024-09-13T18:55:46.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.670+0000 7fa211f91700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20c0723f0 con 0x7fa20c08baa0 2024-09-13T18:55:46.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.670+0000 7fa211f91700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0cb580 con 0x7fa20c0cac60 2024-09-13T18:55:46.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.670+0000 7fa20b7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa20c0cac60 0x7fa20c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:46.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.670+0000 7fa20b7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa20c0cac60 0x7fa20c0cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:40734/0 (socket says 172.21.15.146:40734) 2024-09-13T18:55:46.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.670+0000 7fa20b7fe700 1 -- 172.21.15.146:0/3655675982 learned_addr learned my addr 172.21.15.146:0/3655675982 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:46.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20affd700 1 --2- 172.21.15.146:0/3655675982 >> v2:172.21.15.154:3300/0 conn(0x7fa20c08baa0 0x7fa20c06f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:46.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726236612 0 0) 0x7fa20c0a9920 con 0x7fa20c0c7530 2024-09-13T18:55:46.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1f40035d0 con 0x7fa20c0c7530 2024-09-13T18:55:46.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2677388585 0 0) 0x7fa1f40035d0 con 0x7fa20c0c7530 2024-09-13T18:55:46.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa20c0cac60 msgr2=0x7fa20c0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:46.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 --2- 172.21.15.146:0/3655675982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa20c0cac60 0x7fa20c0cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:46.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 >> v2:172.21.15.154:3300/0 conn(0x7fa20c08baa0 msgr2=0x7fa20c06f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:46.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 --2- 172.21.15.146:0/3655675982 >> v2:172.21.15.154:3300/0 conn(0x7fa20c08baa0 0x7fa20c06f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:46.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c0725e0 con 0x7fa20c0c7530 2024-09-13T18:55:46.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7fa200001670 con 0x7fa20c0c7530 2024-09-13T18:55:46.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 >> v1:172.21.15.154:6789/0 conn(0x7fa20c0c7530 legacy=0x7fa20c0c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:46.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 --2- 172.21.15.146:0/3655675982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1f4003a90 0x7fa1f4003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:46.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 --2- 172.21.15.146:0/3655675982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f4004430 0x7fa1f4004910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:46.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 --2- 172.21.15.146:0/3655675982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1f4005030 0x7fa1f4007a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:46.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0cb580 con 0x7fa1f4005030 2024-09-13T18:55:46.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f4007f50 con 0x7fa1f4004430 2024-09-13T18:55:46.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f4008110 con 0x7fa1f4003a90 2024-09-13T18:55:46.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.674+0000 7fa20b7fe700 1 --2- 172.21.15.146:0/3655675982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1f4003a90 0x7fa1f4003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:46.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa20bfff700 1 --2- 172.21.15.146:0/3655675982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1f4005030 0x7fa1f4007a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:46.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa20b7fe700 1 -- 172.21.15.146:0/3655675982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f4004430 msgr2=0x7fa1f4004910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:55:46.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa20b7fe700 1 --2- 172.21.15.146:0/3655675982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f4004430 0x7fa1f4004910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:46.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa20b7fe700 1 -- 172.21.15.146:0/3655675982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1f4005030 msgr2=0x7fa1f4007a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:46.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa20b7fe700 1 --2- 172.21.15.146:0/3655675982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1f4005030 0x7fa1f4007a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:46.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa20b7fe700 1 -- 172.21.15.146:0/3655675982 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c0725e0 con 0x7fa1f4003a90 2024-09-13T18:55:46.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa20bfff700 1 --2- 172.21.15.146:0/3655675982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1f4005030 0x7fa1f4007a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:55:46.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa20b7fe700 1 --2- 172.21.15.146:0/3655675982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1f4003a90 0x7fa1f4003eb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa20c06ee00 tx=0x7fa1f40058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=126a3ac12ec61354 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:46.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc009d00 con 0x7fa1f4003a90 2024-09-13T18:55:46.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc00f040 con 0x7fa1f4003a90 2024-09-13T18:55:46.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa20a7fc700 1 -- 172.21.15.146:0/3655675982 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00e9a0 con 0x7fa1f4003a90 2024-09-13T18:55:46.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa211f91700 1 -- 172.21.15.146:0/3655675982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1f4003a90 msgr2=0x7fa1f4003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:46.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa211f91700 1 --2- 172.21.15.146:0/3655675982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1f4003a90 0x7fa1f4003eb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa20c06ee00 tx=0x7fa1f40058a0 comp rx=0 tx=0).stop 2024-09-13T18:55:46.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa20bfff700 1 -- 172.21.15.146:0/3655675982 reap_dead start 2024-09-13T18:55:46.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa211f91700 1 -- 172.21.15.146:0/3655675982 shutdown_connections 2024-09-13T18:55:46.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa211f91700 1 -- 172.21.15.146:0/3655675982 >> 172.21.15.146:0/3655675982 conn(0x7fa20c066090 msgr2=0x7fa20c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:46.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa211f91700 1 -- 172.21.15.146:0/3655675982 shutdown_connections 2024-09-13T18:55:46.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.678+0000 7fa211f91700 1 -- 172.21.15.146:0/3655675982 wait complete. 2024-09-13T18:55:46.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.682+0000 7fa211f91700 1 Processor -- start 2024-09-13T18:55:46.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.694+0000 7fa211f91700 1 -- start start 2024-09-13T18:55:46.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.694+0000 7fa211f91700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f4003a90 0x7fa20c204070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:46.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.694+0000 7fa211f91700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa20c1ee090 0x7fa20c2045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:46.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.694+0000 7fa211f91700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa20c1f1560 0x7fa20c1f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:46.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.694+0000 7fa211f91700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0c3ab0 con 0x7fa20c1ee090 2024-09-13T18:55:46.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.694+0000 7fa211f91700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c1f1f00 con 0x7fa1f4003a90 2024-09-13T18:55:46.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.694+0000 7fa211f91700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0c8500 con 0x7fa20c1f1560 2024-09-13T18:55:46.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa20b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f4003a90 0x7fa20c204070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:46.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa20b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f4003a90 0x7fa20c204070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:49320/0 (socket says 172.21.15.146:49320) 2024-09-13T18:55:46.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa20b7fe700 1 -- 172.21.15.146:0/1163981150 learned_addr learned my addr 172.21.15.146:0/1163981150 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:46.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa20b7fe700 1 -- 172.21.15.146:0/1163981150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa20c1f1560 msgr2=0x7fa20c1f19c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:55:46.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa20b7fe700 1 --2- 172.21.15.146:0/1163981150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa20c1f1560 0x7fa20c1f19c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:46.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa20b7fe700 1 -- 172.21.15.146:0/1163981150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa20c1ee090 msgr2=0x7fa20c2045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:46.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa20b7fe700 1 --2- 172.21.15.146:0/1163981150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa20c1ee090 0x7fa20c2045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:46.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa20b7fe700 1 -- 172.21.15.146:0/1163981150 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c1f23c0 con 0x7fa1f4003a90 2024-09-13T18:55:46.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa20b7fe700 1 --2- 172.21.15.146:0/1163981150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f4003a90 0x7fa20c204070 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa20c072110 tx=0x7fa1fc005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:46.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa1f3fff700 1 -- 172.21.15.146:0/1163981150 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc009d00 con 0x7fa1f4003a90 2024-09-13T18:55:46.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa1f3fff700 1 -- 172.21.15.146:0/1163981150 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc00e7a0 con 0x7fa1f4003a90 2024-09-13T18:55:46.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa1f3fff700 1 -- 172.21.15.146:0/1163981150 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00eb50 con 0x7fa1f4003a90 2024-09-13T18:55:46.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa211f91700 1 -- 172.21.15.146:0/1163981150 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20c06cc40 con 0x7fa1f4003a90 2024-09-13T18:55:46.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.702+0000 7fa211f91700 1 -- 172.21.15.146:0/1163981150 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20c1f2100 con 0x7fa1f4003a90 2024-09-13T18:55:46.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.706+0000 7fa1f3fff700 1 -- 172.21.15.146:0/1163981150 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa1fc00ecb0 con 0x7fa1f4003a90 2024-09-13T18:55:46.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.706+0000 7fa1f3fff700 1 --2- 172.21.15.146:0/1163981150 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa1f8066730 0x7fa1f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:46.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.706+0000 7fa20affd700 1 --2- 172.21.15.146:0/1163981150 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa1f8066730 0x7fa1f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:46.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.714+0000 7fa20affd700 1 --2- 172.21.15.146:0/1163981150 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa1f8066730 0x7fa1f8068bf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa204009320 tx=0x7fa204009270 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:46.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.714+0000 7fa1f3fff700 1 -- 172.21.15.146:0/1163981150 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fa1fc086260 con 0x7fa1f4003a90 2024-09-13T18:55:46.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.718+0000 7fa211f91700 1 -- 172.21.15.146:0/1163981150 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20c0aa2b0 con 0x7fa1f4003a90 2024-09-13T18:55:46.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:46.726+0000 7fa1f3fff700 1 -- 172.21.15.146:0/1163981150 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa1fc017020 con 0x7fa1f4003a90 2024-09-13T18:55:47.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.610+0000 7fa211f91700 1 -- 172.21.15.146:0/1163981150 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa20c0ca590 con 0x7fa1f4003a90 2024-09-13T18:55:47.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.610+0000 7fa1f3fff700 1 -- 172.21.15.146:0/1163981150 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5536 (secure 0 0 0) 0x7fa1fc056730 con 0x7fa1f4003a90 2024-09-13T18:55:47.628 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:47.628 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:55:41.923318+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":7224,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7224":{"gid":7224,"name":"c","rank":1,"incarnation":113,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:47.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.638+0000 7fa211f91700 1 -- 172.21.15.146:0/1163981150 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa1f8066730 msgr2=0x7fa1f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:47.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.638+0000 7fa211f91700 1 --2- 172.21.15.146:0/1163981150 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa1f8066730 0x7fa1f8068bf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa204009320 tx=0x7fa204009270 comp rx=0 tx=0).stop 2024-09-13T18:55:47.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.638+0000 7fa211f91700 1 -- 172.21.15.146:0/1163981150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f4003a90 msgr2=0x7fa20c204070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:47.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.638+0000 7fa211f91700 1 --2- 172.21.15.146:0/1163981150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f4003a90 0x7fa20c204070 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa20c072110 tx=0x7fa1fc005a40 comp rx=0 tx=0).stop 2024-09-13T18:55:47.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.638+0000 7fa211f91700 1 -- 172.21.15.146:0/1163981150 shutdown_connections 2024-09-13T18:55:47.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.638+0000 7fa211f91700 1 --2- 172.21.15.146:0/1163981150 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa1f8066730 0x7fa1f8068bf0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:47.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.638+0000 7fa211f91700 1 --2- 172.21.15.146:0/1163981150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa20c1f1560 0x7fa20c1f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:47.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.638+0000 7fa211f91700 1 --2- 172.21.15.146:0/1163981150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa20c1ee090 0x7fa20c2045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:47.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.638+0000 7fa211f91700 1 --2- 172.21.15.146:0/1163981150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f4003a90 0x7fa20c204070 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:47.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.638+0000 7fa211f91700 1 -- 172.21.15.146:0/1163981150 >> 172.21.15.146:0/1163981150 conn(0x7fa20c066090 msgr2=0x7fa20c0c11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:47.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.638+0000 7fa211f91700 1 -- 172.21.15.146:0/1163981150 shutdown_connections 2024-09-13T18:55:47.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:47.638+0000 7fa211f91700 1 -- 172.21.15.146:0/1163981150 wait complete. 2024-09-13T18:55:47.659 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 116 2024-09-13T18:55:47.705 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 116, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T18:26:32.323712+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 116, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6984": { "addr": "172.21.15.146:6835/2934322065", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6834", "nonce": 2934322065, "type": "v2" }, { "addr": "172.21.15.146:6835", "nonce": 2934322065, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6984, "incarnation": 98, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 28 }, "gid_7098": { "addr": "172.21.15.146:6833/354253522", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6832", "nonce": 354253522, "type": "v2" }, { "addr": "172.21.15.146:6833", "nonce": 354253522, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7098, "incarnation": 105, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 9 }, "gid_7224": { "addr": "172.21.15.146:6837/3683135798", "addrs": { "addrvec": [ { "addr": "172.21.15.146:6836", "nonce": 3683135798, "type": "v2" }, { "addr": "172.21.15.146:6837", "nonce": 3683135798, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7224, "incarnation": 113, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:rejoin", "state_seq": 23 } }, "last_failure": 0, "last_failure_osd_epoch": 43, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T18:55:41.923318+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6984, "mds_1": 7224, "mds_2": 7098 } } } ], "standbys": [ { "addr": "172.21.15.154:6838/1534137749", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1534137749, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1534137749, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6835/763096821", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 763096821, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 763096821, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4246, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T18:55:48.537 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:55:48.540 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:55:49.706 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:50.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.446+0000 7f28d0172700 1 Processor -- start 2024-09-13T18:55:50.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.446+0000 7f28d0172700 1 -- start start 2024-09-13T18:55:50.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.446+0000 7f28d0172700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80cac60 0x7f28c80cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:50.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.446+0000 7f28d0172700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c80c8c70 con 0x7f28c80721f0 2024-09-13T18:55:50.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.446+0000 7f28d0172700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c80c8f50 con 0x7f28c8154a40 2024-09-13T18:55:50.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.446+0000 7f28d0172700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c8072810 con 0x7f28c80cac60 2024-09-13T18:55:50.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.450+0000 7f28cd70d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f28c80721f0 0x7f28c8157d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:58026/0 (socket says 172.21.15.146:58026) 2024-09-13T18:55:50.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.450+0000 7f28cd70d700 1 -- 172.21.15.146:0/3346391707 learned_addr learned my addr 172.21.15.146:0/3346391707 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:55:50.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.450+0000 7f28ce70f700 1 --2- 172.21.15.146:0/3346391707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80cac60 0x7f28c80cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:50.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.450+0000 7f28ce70f700 1 -- 172.21.15.146:0/3346391707 >> v1:172.21.15.154:6789/0 conn(0x7f28c8154a40 legacy=0x7f28c806f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:50.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.450+0000 7f28ce70f700 1 -- 172.21.15.146:0/3346391707 >> v1:172.21.15.146:6789/0 conn(0x7f28c80721f0 legacy=0x7f28c8157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:50.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.450+0000 7f28ce70f700 1 -- 172.21.15.146:0/3346391707 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c8157530 con 0x7f28c80cac60 2024-09-13T18:55:50.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.450+0000 7f28ce70f700 1 --2- 172.21.15.146:0/3346391707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80cac60 0x7f28c80cb0f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009880 tx=0x7f28b8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a7563f098154e96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:50.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.458+0000 7f28ccf0c700 1 -- 172.21.15.146:0/3346391707 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8011830 con 0x7f28c80cac60 2024-09-13T18:55:50.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.458+0000 7f28ccf0c700 1 -- 172.21.15.146:0/3346391707 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b8011990 con 0x7f28c80cac60 2024-09-13T18:55:50.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.458+0000 7f28ccf0c700 1 -- 172.21.15.146:0/3346391707 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8011d20 con 0x7f28c80cac60 2024-09-13T18:55:50.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.458+0000 7f28d0172700 1 -- 172.21.15.146:0/3346391707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80cac60 msgr2=0x7f28c80cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:50.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.458+0000 7f28d0172700 1 --2- 172.21.15.146:0/3346391707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80cac60 0x7f28c80cb0f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009880 tx=0x7f28b8009b90 comp rx=0 tx=0).stop 2024-09-13T18:55:50.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.458+0000 7f28d0172700 1 -- 172.21.15.146:0/3346391707 shutdown_connections 2024-09-13T18:55:50.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.458+0000 7f28d0172700 1 --2- 172.21.15.146:0/3346391707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80cac60 0x7f28c80cb0f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:50.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.458+0000 7f28d0172700 1 -- 172.21.15.146:0/3346391707 >> 172.21.15.146:0/3346391707 conn(0x7f28c8066090 msgr2=0x7f28c8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:50.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.462+0000 7f28d0172700 1 -- 172.21.15.146:0/3346391707 shutdown_connections 2024-09-13T18:55:50.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.462+0000 7f28d0172700 1 -- 172.21.15.146:0/3346391707 wait complete. 2024-09-13T18:55:50.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.462+0000 7f28d0172700 1 Processor -- start 2024-09-13T18:55:50.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.462+0000 7f28d0172700 1 -- start start 2024-09-13T18:55:50.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.462+0000 7f28d0172700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80721f0 0x7f28c80adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.462+0000 7f28d0172700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28c80cac60 0x7f28c81458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.462+0000 7f28d0172700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28c8154a40 0x7f28c8145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.462+0000 7f28d0172700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c8157910 con 0x7f28c80cac60 2024-09-13T18:55:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28d0172700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80ad820 con 0x7f28c8154a40 2024-09-13T18:55:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28d0172700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c81466d0 con 0x7f28c80721f0 2024-09-13T18:55:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28cd70d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28c80cac60 0x7f28c81458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28cd70d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28c80cac60 0x7f28c81458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36966/0 (socket says 172.21.15.146:36966) 2024-09-13T18:55:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28cd70d700 1 -- 172.21.15.146:0/1805560870 learned_addr learned my addr 172.21.15.146:0/1805560870 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:50.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28cdf0e700 1 --2- 172.21.15.146:0/1805560870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80721f0 0x7f28c80adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:50.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28cd70d700 1 -- 172.21.15.146:0/1805560870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80721f0 msgr2=0x7f28c80adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:50.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28cd70d700 1 --2- 172.21.15.146:0/1805560870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80721f0 0x7f28c80adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:50.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28cd70d700 1 -- 172.21.15.146:0/1805560870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28c8154a40 msgr2=0x7f28c8145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:50.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28cd70d700 1 --2- 172.21.15.146:0/1805560870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28c8154a40 0x7f28c8145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:50.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28cd70d700 1 -- 172.21.15.146:0/1805560870 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b80094b0 con 0x7f28c80cac60 2024-09-13T18:55:50.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28cd70d700 1 --2- 172.21.15.146:0/1805560870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28c80cac60 0x7f28c81458b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c4d0 tx=0x7f28c400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:50.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28be7fc700 1 -- 172.21.15.146:0/1805560870 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4005d90 con 0x7f28c80cac60 2024-09-13T18:55:50.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28be7fc700 1 -- 172.21.15.146:0/1805560870 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c40070d0 con 0x7f28c80cac60 2024-09-13T18:55:50.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28d0172700 1 -- 172.21.15.146:0/1805560870 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c80ad5d0 con 0x7f28c80cac60 2024-09-13T18:55:50.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28d0172700 1 -- 172.21.15.146:0/1805560870 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c8140770 con 0x7f28c80cac60 2024-09-13T18:55:50.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.466+0000 7f28d0172700 1 -- 172.21.15.146:0/1805560870 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c8140900 con 0x7f28c80cac60 2024-09-13T18:55:50.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.470+0000 7f28be7fc700 1 -- 172.21.15.146:0/1805560870 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4007440 con 0x7f28c80cac60 2024-09-13T18:55:50.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.470+0000 7f28be7fc700 1 -- 172.21.15.146:0/1805560870 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f28c40075e0 con 0x7f28c80cac60 2024-09-13T18:55:50.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.470+0000 7f28be7fc700 1 --2- 172.21.15.146:0/1805560870 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f28ac066880 0x7f28ac068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:50.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.470+0000 7f28be7fc700 1 -- 172.21.15.146:0/1805560870 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f28c4083d50 con 0x7f28c80cac60 2024-09-13T18:55:50.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.470+0000 7f28cdf0e700 1 --2- 172.21.15.146:0/1805560870 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f28ac066880 0x7f28ac068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:50.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.470+0000 7f28cdf0e700 1 --2- 172.21.15.146:0/1805560870 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f28ac066880 0x7f28ac068d40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f28c00012e0 tx=0x7f28c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:50.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:50.486+0000 7f28be7fc700 1 -- 172.21.15.146:0/1805560870 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f28c40508c0 con 0x7f28c80cac60 2024-09-13T18:55:51.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.402+0000 7f28d0172700 1 -- 172.21.15.146:0/1805560870 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28c80c73b0 con 0x7f28c80cac60 2024-09-13T18:55:51.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.402+0000 7f28be7fc700 1 -- 172.21.15.146:0/1805560870 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+5536 (secure 0 0 0) 0x7f28c4017020 con 0x7f28c80cac60 2024-09-13T18:55:51.419 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:51.419 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":117,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:55:47.963024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":7224,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7224":{"gid":7224,"name":"c","rank":1,"incarnation":113,"state":"up:active","state_seq":25,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:51.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.406+0000 7f28d0172700 1 -- 172.21.15.146:0/1805560870 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f28ac066880 msgr2=0x7f28ac068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:51.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.406+0000 7f28d0172700 1 --2- 172.21.15.146:0/1805560870 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f28ac066880 0x7f28ac068d40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f28c00012e0 tx=0x7f28c0009040 comp rx=0 tx=0).stop 2024-09-13T18:55:51.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.406+0000 7f28d0172700 1 -- 172.21.15.146:0/1805560870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28c80cac60 msgr2=0x7f28c81458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:51.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.406+0000 7f28d0172700 1 --2- 172.21.15.146:0/1805560870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28c80cac60 0x7f28c81458b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c4d0 tx=0x7f28c400c7e0 comp rx=0 tx=0).stop 2024-09-13T18:55:51.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.410+0000 7f28d0172700 1 -- 172.21.15.146:0/1805560870 shutdown_connections 2024-09-13T18:55:51.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.410+0000 7f28d0172700 1 --2- 172.21.15.146:0/1805560870 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f28ac066880 0x7f28ac068d40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:51.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.410+0000 7f28d0172700 1 --2- 172.21.15.146:0/1805560870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28c8154a40 0x7f28c8145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:51.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.410+0000 7f28d0172700 1 --2- 172.21.15.146:0/1805560870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28c80cac60 0x7f28c81458b0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:51.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.410+0000 7f28d0172700 1 --2- 172.21.15.146:0/1805560870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80721f0 0x7f28c80adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:51.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.410+0000 7f28d0172700 1 -- 172.21.15.146:0/1805560870 >> 172.21.15.146:0/1805560870 conn(0x7f28c8066090 msgr2=0x7f28c80c53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.414+0000 7f28d0172700 1 -- 172.21.15.146:0/1805560870 shutdown_connections 2024-09-13T18:55:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:51.414+0000 7f28d0172700 1 -- 172.21.15.146:0/1805560870 wait complete. 2024-09-13T18:55:51.435 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 117 2024-09-13T18:55:51.465 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=1, replacing gid=6032 2024-09-13T18:55:51.465 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.f 2024-09-13T18:55:52.770 INFO:tasks.ceph.mds.a.smithi146.stderr:2024-09-13T18:55:52.754+0000 7f40aeb2a700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-13T18:55:56.466 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2024-09-13T18:55:56.466 INFO:tasks.ceph.mds.f:Restarting daemon 2024-09-13T18:55:56.467 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-09-13T18:55:56.469 INFO:tasks.ceph.mds.f:Started 2024-09-13T18:55:56.469 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:56.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.846+0000 7f8ef1246700 1 Processor -- start 2024-09-13T18:55:56.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.846+0000 7f8ef1246700 1 -- start start 2024-09-13T18:55:56.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.846+0000 7f8ef1246700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8eec0cac60 0x7f8eec0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:56.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.846+0000 7f8ef1246700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8eec16e0c0 0x7f8eec1704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:56.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.846+0000 7f8ef1246700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eec0a9920 con 0x7f8eec08baa0 2024-09-13T18:55:56.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.846+0000 7f8ef1246700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eec0723f0 con 0x7f8eec0cac60 2024-09-13T18:55:56.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.846+0000 7f8ef1246700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec0cb580 con 0x7f8eec16e0c0 2024-09-13T18:55:56.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.850+0000 7f8eebfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8eec0cac60 0x7f8eec0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:56.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.850+0000 7f8eebfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8eec0cac60 0x7f8eec0cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39562/0 (socket says 172.21.15.146:39562) 2024-09-13T18:55:56.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.850+0000 7f8eebfff700 1 -- 172.21.15.146:0/3503161725 learned_addr learned my addr 172.21.15.146:0/3503161725 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:56.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.850+0000 7f8eebfff700 1 -- 172.21.15.146:0/3503161725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8eec16e0c0 msgr2=0x7f8eec1704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:56.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.850+0000 7f8eebfff700 1 --2- 172.21.15.146:0/3503161725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8eec16e0c0 0x7f8eec1704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:56.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.850+0000 7f8eebfff700 1 -- 172.21.15.146:0/3503161725 >> v1:172.21.15.146:6789/0 conn(0x7f8eec08baa0 legacy=0x7f8eec06f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:56.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.850+0000 7f8eebfff700 1 -- 172.21.15.146:0/3503161725 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eec070070 con 0x7f8eec0cac60 2024-09-13T18:55:56.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.850+0000 7f8eebfff700 1 --2- 172.21.15.146:0/3503161725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8eec0cac60 0x7f8eec0cb040 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8edc008990 tx=0x7f8edc008ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f387f649f858db54 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:56.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.858+0000 7f8eeaffd700 1 -- 172.21.15.146:0/3503161725 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc004230 con 0x7f8eec0cac60 2024-09-13T18:55:56.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.858+0000 7f8eeaffd700 1 -- 172.21.15.146:0/3503161725 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8edc004390 con 0x7f8eec0cac60 2024-09-13T18:55:56.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.858+0000 7f8ef1246700 1 -- 172.21.15.146:0/3503161725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8eec0cac60 msgr2=0x7f8eec0cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:56.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.858+0000 7f8ef1246700 1 --2- 172.21.15.146:0/3503161725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8eec0cac60 0x7f8eec0cb040 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8edc008990 tx=0x7f8edc008ca0 comp rx=0 tx=0).stop 2024-09-13T18:55:56.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.858+0000 7f8ef1246700 1 -- 172.21.15.146:0/3503161725 shutdown_connections 2024-09-13T18:55:56.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.858+0000 7f8ef1246700 1 --2- 172.21.15.146:0/3503161725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8eec16e0c0 0x7f8eec1704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:56.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.858+0000 7f8ef1246700 1 --2- 172.21.15.146:0/3503161725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8eec0cac60 0x7f8eec0cb040 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:56.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.858+0000 7f8ef1246700 1 -- 172.21.15.146:0/3503161725 >> 172.21.15.146:0/3503161725 conn(0x7f8eec066090 msgr2=0x7f8eec066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:56.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.858+0000 7f8ef1246700 1 -- 172.21.15.146:0/3503161725 shutdown_connections 2024-09-13T18:55:56.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.858+0000 7f8ef1246700 1 -- 172.21.15.146:0/3503161725 wait complete. 2024-09-13T18:55:56.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.858+0000 7f8ef1246700 1 Processor -- start 2024-09-13T18:55:56.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.866+0000 7f8ef1246700 1 -- start start 2024-09-13T18:55:56.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.866+0000 7f8ef1246700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8eec08baa0 0x7f8eec1fc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:56.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.866+0000 7f8ef1246700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8eec16e0c0 0x7f8eec0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:56.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.866+0000 7f8ef1246700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8eec0ad6a0 0x7f8eec0ab840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:56.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.866+0000 7f8ef1246700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec1fce80 con 0x7f8eec16e0c0 2024-09-13T18:55:56.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.866+0000 7f8ef1246700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec0b01b0 con 0x7f8eec0ad6a0 2024-09-13T18:55:56.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.866+0000 7f8ef1246700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec0ab3b0 con 0x7f8eec08baa0 2024-09-13T18:55:56.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.870+0000 7f8ef0a45700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8eec0ad6a0 0x7f8eec0ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:56.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.870+0000 7f8eeb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8eec16e0c0 0x7f8eec0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:56.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.870+0000 7f8eeb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8eec16e0c0 0x7f8eec0aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39570/0 (socket says 172.21.15.146:39570) 2024-09-13T18:55:56.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.870+0000 7f8eeb7fe700 1 -- 172.21.15.146:0/1306260287 learned_addr learned my addr 172.21.15.146:0/1306260287 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:55:56.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.870+0000 7f8eeb7fe700 1 -- 172.21.15.146:0/1306260287 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8eec08baa0 msgr2=0x7f8eec1fc490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:55:56.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.870+0000 7f8eeb7fe700 1 --2- 172.21.15.146:0/1306260287 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8eec08baa0 0x7f8eec1fc490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:56.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.870+0000 7f8eeb7fe700 1 -- 172.21.15.146:0/1306260287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8eec0ad6a0 msgr2=0x7f8eec0ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:56.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.870+0000 7f8eeb7fe700 1 --2- 172.21.15.146:0/1306260287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8eec0ad6a0 0x7f8eec0ab840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:56.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.870+0000 7f8eeb7fe700 1 -- 172.21.15.146:0/1306260287 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc0085f0 con 0x7f8eec16e0c0 2024-09-13T18:55:56.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.870+0000 7f8eeb7fe700 1 --2- 172.21.15.146:0/1306260287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8eec16e0c0 0x7f8eec0aef40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400c870 tx=0x7f8ee400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:56.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.882+0000 7f8ee8ff9700 1 -- 172.21.15.146:0/1306260287 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4003e30 con 0x7f8eec16e0c0 2024-09-13T18:55:56.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.886+0000 7f8ee8ff9700 1 -- 172.21.15.146:0/1306260287 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee400ce70 con 0x7f8eec16e0c0 2024-09-13T18:55:56.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.886+0000 7f8ef1246700 1 -- 172.21.15.146:0/1306260287 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eec16c4c0 con 0x7f8eec16e0c0 2024-09-13T18:55:56.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.886+0000 7f8ef1246700 1 -- 172.21.15.146:0/1306260287 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eec16c730 con 0x7f8eec16e0c0 2024-09-13T18:55:56.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.890+0000 7f8ee8ff9700 1 -- 172.21.15.146:0/1306260287 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4015b80 con 0x7f8eec16e0c0 2024-09-13T18:55:56.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.894+0000 7f8ee8ff9700 1 -- 172.21.15.146:0/1306260287 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8ee401b070 con 0x7f8eec16e0c0 2024-09-13T18:55:56.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.894+0000 7f8ee8ff9700 1 --2- 172.21.15.146:0/1306260287 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ed8066830 0x7f8ed8068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:56.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.894+0000 7f8ee8ff9700 1 -- 172.21.15.146:0/1306260287 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f8ee408cab0 con 0x7f8eec16e0c0 2024-09-13T18:55:56.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.898+0000 7f8eebfff700 1 --2- 172.21.15.146:0/1306260287 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ed8066830 0x7f8ed8068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:56.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.898+0000 7f8eebfff700 1 --2- 172.21.15.146:0/1306260287 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ed8066830 0x7f8ed8068cf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8edc008990 tx=0x7f8edc008dc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:56.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.902+0000 7f8ef1246700 1 -- 172.21.15.146:0/1306260287 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ecc005310 con 0x7f8eec16e0c0 2024-09-13T18:55:56.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:56.914+0000 7f8ee8ff9700 1 -- 172.21.15.146:0/1306260287 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8ee40594e0 con 0x7f8eec16e0c0 2024-09-13T18:55:57.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.754+0000 7f8ef1246700 1 -- 172.21.15.146:0/1306260287 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ecc005b40 con 0x7f8eec16e0c0 2024-09-13T18:55:57.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.754+0000 7f8ee8ff9700 1 -- 172.21.15.146:0/1306260287 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+5536 (secure 0 0 0) 0x7f8ee405cb00 con 0x7f8eec16e0c0 2024-09-13T18:55:57.769 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:55:57.770 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":117,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:55:47.963024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":7224,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7224":{"gid":7224,"name":"c","rank":1,"incarnation":113,"state":"up:active","state_seq":25,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:55:57.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.766+0000 7f8ef1246700 1 -- 172.21.15.146:0/1306260287 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ed8066830 msgr2=0x7f8ed8068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:57.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.766+0000 7f8ef1246700 1 --2- 172.21.15.146:0/1306260287 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ed8066830 0x7f8ed8068cf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8edc008990 tx=0x7f8edc008dc0 comp rx=0 tx=0).stop 2024-09-13T18:55:57.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.766+0000 7f8ef1246700 1 -- 172.21.15.146:0/1306260287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8eec16e0c0 msgr2=0x7f8eec0aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:57.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.766+0000 7f8ef1246700 1 --2- 172.21.15.146:0/1306260287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8eec16e0c0 0x7f8eec0aef40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400c870 tx=0x7f8ee400cc30 comp rx=0 tx=0).stop 2024-09-13T18:55:57.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.766+0000 7f8ef1246700 1 -- 172.21.15.146:0/1306260287 shutdown_connections 2024-09-13T18:55:57.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.766+0000 7f8ef1246700 1 --2- 172.21.15.146:0/1306260287 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8ed8066830 0x7f8ed8068cf0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:57.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.766+0000 7f8ef1246700 1 --2- 172.21.15.146:0/1306260287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8eec0ad6a0 0x7f8eec0ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:57.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.766+0000 7f8ef1246700 1 --2- 172.21.15.146:0/1306260287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8eec16e0c0 0x7f8eec0aef40 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:57.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.766+0000 7f8ef1246700 1 --2- 172.21.15.146:0/1306260287 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8eec08baa0 0x7f8eec1fc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:57.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.766+0000 7f8ef1246700 1 -- 172.21.15.146:0/1306260287 >> 172.21.15.146:0/1306260287 conn(0x7f8eec066090 msgr2=0x7f8eec06e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:57.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.774+0000 7f8ef1246700 1 -- 172.21.15.146:0/1306260287 shutdown_connections 2024-09-13T18:55:57.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:55:57.774+0000 7f8ef1246700 1 -- 172.21.15.146:0/1306260287 wait complete. 2024-09-13T18:55:57.795 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 117 2024-09-13T18:55:57.809 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2024-09-13T18:55:58.938 INFO:tasks.ceph.mds.f.smithi154.stdout:starting mds.f at 2024-09-13T18:55:59.810 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:56:00.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.378+0000 7f8493fff700 1 Processor -- start 2024-09-13T18:56:00.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.378+0000 7f8493fff700 1 -- start start 2024-09-13T18:56:00.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.382+0000 7f8493fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f84940cb040 0x7f84941717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:00.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.382+0000 7f8493fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84940a9920 con 0x7f849408baa0 2024-09-13T18:56:00.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.382+0000 7f8493fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8494070070 con 0x7f84940cac60 2024-09-13T18:56:00.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.382+0000 7f8493fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84940723f0 con 0x7f84940cb040 2024-09-13T18:56:00.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.382+0000 7f84937fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f84940cb040 0x7f84941717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:00.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.382+0000 7f84937fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f84940cb040 0x7f84941717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:39580/0 (socket says 172.21.15.146:39580) 2024-09-13T18:56:00.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.382+0000 7f84937fe700 1 -- 172.21.15.146:0/2088256480 learned_addr learned my addr 172.21.15.146:0/2088256480 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:00.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.386+0000 7f8491ffb700 1 -- 172.21.15.146:0/2088256480 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143742339 0 0) 0x7f8494070070 con 0x7f84940cac60 2024-09-13T18:56:00.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.386+0000 7f84937fe700 1 -- 172.21.15.146:0/2088256480 >> v1:172.21.15.154:6789/0 conn(0x7f84940cac60 legacy=0x7f849406f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:00.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.386+0000 7f84937fe700 1 -- 172.21.15.146:0/2088256480 >> v1:172.21.15.146:6789/0 conn(0x7f849408baa0 legacy=0x7f849416e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:00.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.386+0000 7f84937fe700 1 -- 172.21.15.146:0/2088256480 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84940725e0 con 0x7f84940cb040 2024-09-13T18:56:00.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.386+0000 7f84937fe700 1 --2- 172.21.15.146:0/2088256480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84940cb040 0x7f84941717f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f848c009e50 tx=0x7f848c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a2fcf2608e3de0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:00.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.390+0000 7f8491ffb700 1 -- 172.21.15.146:0/2088256480 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c00e070 con 0x7f84940cb040 2024-09-13T18:56:00.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.394+0000 7f8491ffb700 1 -- 172.21.15.146:0/2088256480 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c00b9c0 con 0x7f84940cb040 2024-09-13T18:56:00.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.394+0000 7f8491ffb700 1 -- 172.21.15.146:0/2088256480 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c00bd70 con 0x7f84940cb040 2024-09-13T18:56:00.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.394+0000 7f8493fff700 1 -- 172.21.15.146:0/2088256480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84940cb040 msgr2=0x7f84941717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:00.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.394+0000 7f8493fff700 1 --2- 172.21.15.146:0/2088256480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84940cb040 0x7f84941717f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f848c009e50 tx=0x7f848c00b330 comp rx=0 tx=0).stop 2024-09-13T18:56:00.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.394+0000 7f8493fff700 1 -- 172.21.15.146:0/2088256480 shutdown_connections 2024-09-13T18:56:00.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.398+0000 7f8493fff700 1 --2- 172.21.15.146:0/2088256480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84940cb040 0x7f84941717f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:00.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.398+0000 7f8493fff700 1 -- 172.21.15.146:0/2088256480 >> 172.21.15.146:0/2088256480 conn(0x7f8494066090 msgr2=0x7f8494066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:00.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.406+0000 7f8493fff700 1 -- 172.21.15.146:0/2088256480 shutdown_connections 2024-09-13T18:56:00.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.410+0000 7f8493fff700 1 -- 172.21.15.146:0/2088256480 wait complete. 2024-09-13T18:56:00.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f8493fff700 1 Processor -- start 2024-09-13T18:56:00.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f8493fff700 1 -- start start 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f8493fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f849408baa0 0x7f849406cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f8493fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84940cac60 0x7f849416c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f8493fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84940cb040 0x7f84941fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f8493fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84940aa7d0 con 0x7f849408baa0 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f8493fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84940aad00 con 0x7f84940cac60 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f8493fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8494200f40 con 0x7f84940cb040 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f84937fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84940cb040 0x7f84941fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f84937fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84940cb040 0x7f84941fd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:56860/0 (socket says 172.21.15.146:56860) 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f84937fe700 1 -- 172.21.15.146:0/1900641820 learned_addr learned my addr 172.21.15.146:0/1900641820 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f84937fe700 1 -- 172.21.15.146:0/1900641820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84940cac60 msgr2=0x7f849416c0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f84937fe700 1 --2- 172.21.15.146:0/1900641820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84940cac60 0x7f849416c0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f84937fe700 1 -- 172.21.15.146:0/1900641820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f849408baa0 msgr2=0x7f849406cba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f84937fe700 1 --2- 172.21.15.146:0/1900641820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f849408baa0 0x7f849406cba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:00.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f84937fe700 1 -- 172.21.15.146:0/1900641820 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c009b30 con 0x7f84940cb040 2024-09-13T18:56:00.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.414+0000 7f84937fe700 1 --2- 172.21.15.146:0/1900641820 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84940cb040 0x7f84941fd5c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f848c00b730 tx=0x7f848c004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:00.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.418+0000 7f847b7fe700 1 -- 172.21.15.146:0/1900641820 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c01eb90 con 0x7f84940cb040 2024-09-13T18:56:00.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.418+0000 7f847b7fe700 1 -- 172.21.15.146:0/1900641820 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c022070 con 0x7f84940cb040 2024-09-13T18:56:00.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.418+0000 7f847b7fe700 1 -- 172.21.15.146:0/1900641820 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c004040 con 0x7f84940cb040 2024-09-13T18:56:00.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.418+0000 7f8493fff700 1 -- 172.21.15.146:0/1900641820 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8494201220 con 0x7f84940cb040 2024-09-13T18:56:00.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.422+0000 7f8493fff700 1 -- 172.21.15.146:0/1900641820 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84941fd0a0 con 0x7f84940cb040 2024-09-13T18:56:00.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.422+0000 7f8493fff700 1 -- 172.21.15.146:0/1900641820 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84941fd2c0 con 0x7f84940cb040 2024-09-13T18:56:00.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.426+0000 7f847b7fe700 1 -- 172.21.15.146:0/1900641820 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f848c024030 con 0x7f84940cb040 2024-09-13T18:56:00.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.426+0000 7f847b7fe700 1 --2- 172.21.15.146:0/1900641820 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8480066800 0x7f8480068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:00.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.426+0000 7f847b7fe700 1 -- 172.21.15.146:0/1900641820 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f848c08a490 con 0x7f84940cb040 2024-09-13T18:56:00.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.430+0000 7f847b7fe700 1 -- 172.21.15.146:0/1900641820 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f848c056e30 con 0x7f84940cb040 2024-09-13T18:56:00.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.430+0000 7f8492ffd700 1 --2- 172.21.15.146:0/1900641820 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8480066800 0x7f8480068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:00.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:00.430+0000 7f8492ffd700 1 --2- 172.21.15.146:0/1900641820 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8480066800 0x7f8480068cc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8488000f60 tx=0x7f8488007930 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:01.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.290+0000 7f8493fff700 1 -- 172.21.15.146:0/1900641820 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84940a9c70 con 0x7f84940cb040 2024-09-13T18:56:01.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.290+0000 7f847b7fe700 1 -- 172.21.15.146:0/1900641820 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+6297 (secure 0 0 0) 0x7f848c05a450 con 0x7f84940cb040 2024-09-13T18:56:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:56:01.305 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":118,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6323,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2261933208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2261933208},{"type":"v1","addr":"172.21.15.154:6839","nonce":2261933208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:55:47.963024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":7224,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7224":{"gid":7224,"name":"c","rank":1,"incarnation":113,"state":"up:active","state_seq":25,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:56:01.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.306+0000 7f84797fa700 1 -- 172.21.15.146:0/1900641820 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8480066800 msgr2=0x7f8480068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:01.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.306+0000 7f84797fa700 1 --2- 172.21.15.146:0/1900641820 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8480066800 0x7f8480068cc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8488000f60 tx=0x7f8488007930 comp rx=0 tx=0).stop 2024-09-13T18:56:01.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.306+0000 7f84797fa700 1 -- 172.21.15.146:0/1900641820 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84940cb040 msgr2=0x7f84941fd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:01.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.306+0000 7f84797fa700 1 --2- 172.21.15.146:0/1900641820 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84940cb040 0x7f84941fd5c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f848c00b730 tx=0x7f848c004af0 comp rx=0 tx=0).stop 2024-09-13T18:56:01.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.306+0000 7f84797fa700 1 -- 172.21.15.146:0/1900641820 shutdown_connections 2024-09-13T18:56:01.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.306+0000 7f84797fa700 1 --2- 172.21.15.146:0/1900641820 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8480066800 0x7f8480068cc0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:01.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.306+0000 7f84797fa700 1 --2- 172.21.15.146:0/1900641820 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84940cb040 0x7f84941fd5c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:01.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.306+0000 7f84797fa700 1 --2- 172.21.15.146:0/1900641820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84940cac60 0x7f849416c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:01.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.306+0000 7f84797fa700 1 --2- 172.21.15.146:0/1900641820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f849408baa0 0x7f849406cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:01.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.306+0000 7f84797fa700 1 -- 172.21.15.146:0/1900641820 >> 172.21.15.146:0/1900641820 conn(0x7f8494066090 msgr2=0x7f84940c0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:01.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.306+0000 7f84797fa700 1 -- 172.21.15.146:0/1900641820 shutdown_connections 2024-09-13T18:56:01.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:01.306+0000 7f84797fa700 1 -- 172.21.15.146:0/1900641820 wait complete. 2024-09-13T18:56:01.327 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 118 2024-09-13T18:56:01.346 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2024-09-13T18:56:01.346 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 57 secs before thrashing 2024-09-13T18:56:19.302 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:56:19.305 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:56:43.040 INFO:tasks.workunit.client.0.smithi146.stdout:Syncing()...4 sec 2024-09-13T18:56:43.044 INFO:tasks.workunit.client.0.smithi146.stdout:FFSB benchmark finished at: Fri Sep 13 18:56:43 2024 2024-09-13T18:56:43.044 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:56:43.044 INFO:tasks.workunit.client.0.smithi146.stdout:Results: 2024-09-13T18:56:43.044 INFO:tasks.workunit.client.0.smithi146.stdout:Benchmark took 316.20 sec 2024-09-13T18:56:43.044 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:56:43.044 INFO:tasks.workunit.client.0.smithi146.stdout:Total Results 2024-09-13T18:56:43.045 INFO:tasks.workunit.client.0.smithi146.stdout:=============== 2024-09-13T18:56:43.045 INFO:tasks.workunit.client.0.smithi146.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-09-13T18:56:43.045 INFO:tasks.workunit.client.0.smithi146.stdout: ======= ============ ========= ======= =========== ========== 2024-09-13T18:56:43.045 INFO:tasks.workunit.client.0.smithi146.stdout: write : 805120 2546.21 100.000% 100.000% 9.95MB/sec 2024-09-13T18:56:43.045 INFO:tasks.workunit.client.0.smithi146.stdout:- 2024-09-13T18:56:43.045 INFO:tasks.workunit.client.0.smithi146.stdout:2546.21 Transactions per Second 2024-09-13T18:56:43.045 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:56:43.045 INFO:tasks.workunit.client.0.smithi146.stdout:Throughput Results 2024-09-13T18:56:43.045 INFO:tasks.workunit.client.0.smithi146.stdout:=================== 2024-09-13T18:56:43.046 INFO:tasks.workunit.client.0.smithi146.stdout:Write Throughput: 9.95MB/sec 2024-09-13T18:56:43.047 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:56:43.047 INFO:tasks.workunit.client.0.smithi146.stdout:System Call Latency statistics in millisecs 2024-09-13T18:56:43.047 INFO:tasks.workunit.client.0.smithi146.stdout:===== 2024-09-13T18:56:43.047 INFO:tasks.workunit.client.0.smithi146.stdout: Min Avg Max Total Calls 2024-09-13T18:56:43.047 INFO:tasks.workunit.client.0.smithi146.stdout: ======== ======== ======== ============ 2024-09-13T18:56:43.047 INFO:tasks.workunit.client.0.smithi146.stdout:[ open] 0.467000 1944.787151 43126.593750 629 2024-09-13T18:56:43.047 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T18:56:43.047 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[1] 2000.000000 - 5000.000000 : 7 2024-09-13T18:56:43.047 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[2] 1000.000000 - 2000.000000 : 1 2024-09-13T18:56:43.047 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[3] 500.000000 - 1000.000000 : 1 2024-09-13T18:56:43.047 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[4] 200.000000 - 500.000000 : 4 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[5] 100.000000 - 200.000000 : 15 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[6] 50.000000 - 100.000000 : 30 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[7] 20.000000 - 50.000000 : 129 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[8] 10.000000 - 20.000000 : 164 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[9] 5.000000 - 10.000000 : 120 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[10] 2.000000 - 5.000000 : 88 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[11] 1.000000 - 2.000000 : 30 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[12] 0.500000 - 1.000000 : 5 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[13] 0.200000 - 0.500000 : 3 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-13T18:56:43.048 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-13T18:56:43.049 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:56:43.049 INFO:tasks.workunit.client.0.smithi146.stdout:[ write] 0.088000 10.720838 4213.671875 805120 2024-09-13T18:56:43.049 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T18:56:43.049 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[1] 2000.000000 - 5000.000000 : 7 2024-09-13T18:56:43.049 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[2] 1000.000000 - 2000.000000 : 13 2024-09-13T18:56:43.049 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[3] 500.000000 - 1000.000000 : 23 2024-09-13T18:56:43.049 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[4] 200.000000 - 500.000000 : 146 2024-09-13T18:56:43.049 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[5] 100.000000 - 200.000000 : 4122 2024-09-13T18:56:43.049 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[6] 50.000000 - 100.000000 : 25808 2024-09-13T18:56:43.049 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[7] 20.000000 - 50.000000 : 97614 2024-09-13T18:56:43.049 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[8] 10.000000 - 20.000000 : 127192 2024-09-13T18:56:43.049 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[9] 5.000000 - 10.000000 : 131182 2024-09-13T18:56:43.050 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[10] 2.000000 - 5.000000 : 117444 2024-09-13T18:56:43.050 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[11] 1.000000 - 2.000000 : 48241 2024-09-13T18:56:43.050 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[12] 0.500000 - 1.000000 : 43768 2024-09-13T18:56:43.050 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[13] 0.200000 - 0.500000 : 151066 2024-09-13T18:56:43.050 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[14] 0.100000 - 0.200000 : 58421 2024-09-13T18:56:43.050 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[15] 0.050000 - 0.100000 : 73 2024-09-13T18:56:43.050 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-13T18:56:43.050 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-13T18:56:43.050 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-13T18:56:43.050 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:56:43.050 INFO:tasks.workunit.client.0.smithi146.stdout:[ lseek] 0.000000 0.001172 6.051000 805120 2024-09-13T18:56:43.050 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T18:56:43.050 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-13T18:56:43.051 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-13T18:56:43.051 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-13T18:56:43.051 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-13T18:56:43.051 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-13T18:56:43.051 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-13T18:56:43.051 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-09-13T18:56:43.051 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-09-13T18:56:43.051 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[9] 5.000000 - 10.000000 : 2 2024-09-13T18:56:43.051 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[10] 2.000000 - 5.000000 : 4 2024-09-13T18:56:43.051 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[11] 1.000000 - 2.000000 : 9 2024-09-13T18:56:43.051 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[12] 0.500000 - 1.000000 : 13 2024-09-13T18:56:43.051 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[13] 0.200000 - 0.500000 : 49 2024-09-13T18:56:43.051 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[14] 0.100000 - 0.200000 : 50 2024-09-13T18:56:43.052 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[15] 0.050000 - 0.100000 : 70 2024-09-13T18:56:43.052 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[16] 0.020000 - 0.050000 : 300 2024-09-13T18:56:43.052 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[17] 0.010000 - 0.020000 : 1010 2024-09-13T18:56:43.052 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[18] 0.000000 - 0.010000 : 803613 2024-09-13T18:56:43.052 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:56:43.052 INFO:tasks.workunit.client.0.smithi146.stdout:[ close] 0.024000 3.918967 189.050003 629 2024-09-13T18:56:43.052 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T18:56:43.052 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-13T18:56:43.052 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-13T18:56:43.053 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-13T18:56:43.053 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-13T18:56:43.053 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[5] 100.000000 - 200.000000 : 1 2024-09-13T18:56:43.053 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[6] 50.000000 - 100.000000 : 7 2024-09-13T18:56:43.053 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[7] 20.000000 - 50.000000 : 27 2024-09-13T18:56:43.053 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[8] 10.000000 - 20.000000 : 13 2024-09-13T18:56:43.053 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[9] 5.000000 - 10.000000 : 42 2024-09-13T18:56:43.053 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[10] 2.000000 - 5.000000 : 63 2024-09-13T18:56:43.053 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[11] 1.000000 - 2.000000 : 51 2024-09-13T18:56:43.053 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[12] 0.500000 - 1.000000 : 42 2024-09-13T18:56:43.054 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[13] 0.200000 - 0.500000 : 48 2024-09-13T18:56:43.054 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[14] 0.100000 - 0.200000 : 102 2024-09-13T18:56:43.054 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[15] 0.050000 - 0.100000 : 95 2024-09-13T18:56:43.054 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[16] 0.020000 - 0.050000 : 138 2024-09-13T18:56:43.054 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-13T18:56:43.054 INFO:tasks.workunit.client.0.smithi146.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-13T18:56:43.054 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:56:43.054 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:56:43.054 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:56:43.054 INFO:tasks.workunit.client.0.smithi146.stdout: 2024-09-13T18:56:43.054 INFO:tasks.workunit.client.0.smithi146.stdout:0.0% User Time 2024-09-13T18:56:43.054 INFO:tasks.workunit.client.0.smithi146.stdout:50.8% System Time 2024-09-13T18:56:43.054 INFO:tasks.workunit.client.0.smithi146.stdout:50.8% CPU Utilization 2024-09-13T18:56:49.456 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-13T18:56:49.456 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-13T18:56:49.474 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:56:49.476 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:56:49.506 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-09-13T18:56:49.506 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-13T18:56:49.525 INFO:tasks.ceph.osd.2.smithi146.stderr:2024-09-13T18:56:49.511+0000 7f5e6b5bc700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 59733) UID: 0 2024-09-13T18:56:49.525 INFO:tasks.ceph.osd.3.smithi146.stderr:2024-09-13T18:56:49.511+0000 7f3b03811700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 59733) UID: 0 2024-09-13T18:56:49.526 INFO:tasks.ceph.osd.0.smithi146.stderr:2024-09-13T18:56:49.515+0000 7f483dbd7700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 59733) UID: 0 2024-09-13T18:56:49.526 INFO:tasks.ceph.osd.1.smithi146.stderr:2024-09-13T18:56:49.515+0000 7f0ab17be700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 59733) UID: 0 2024-09-13T18:56:49.880 DEBUG:teuthology.parallel:result is None 2024-09-13T18:56:49.881 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T18:56:49.929 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T18:56:49.930 DEBUG:teuthology.run_tasks:Unwinding manager mds_thrash 2024-09-13T18:56:49.940 INFO:tasks.mds_thrash:joining mds_thrasher 2024-09-13T18:56:49.940 INFO:tasks.mds_thrash.fs.[cephfs]:stat['max_mds'] = 2 2024-09-13T18:56:49.940 INFO:tasks.mds_thrash.fs.[cephfs]:stat['deactivate'] = 0 2024-09-13T18:56:49.941 INFO:tasks.mds_thrash.fs.[cephfs]:stat['kill'] = 16 2024-09-13T18:56:49.941 INFO:tasks.mds_thrash:done joining 2024-09-13T18:56:49.941 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-13T18:56:49.950 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-13T18:56:49.950 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:56:49.950 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:56:49.995 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:56:49.995 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:56:50.010 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:56:50.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.315+0000 7fa4e39d7700 1 Processor -- start 2024-09-13T18:56:50.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.315+0000 7fa4e39d7700 1 -- start start 2024-09-13T18:56:50.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.315+0000 7fa4e39d7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4dc0c9500 0x7fa4dc155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:50.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.315+0000 7fa4e39d7700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4dc06f5c0 con 0x7fa4dc156140 2024-09-13T18:56:50.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.319+0000 7fa4e39d7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4dc155420 con 0x7fa4dc14f630 2024-09-13T18:56:50.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.319+0000 7fa4e39d7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc0c9920 con 0x7fa4dc0c9500 2024-09-13T18:56:50.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.319+0000 7fa4e1f74700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa4dc156140 0x7fa4dc0cb120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:33056/0 (socket says 172.21.15.146:33056) 2024-09-13T18:56:50.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.319+0000 7fa4e1f74700 1 -- 172.21.15.146:0/772170270 learned_addr learned my addr 172.21.15.146:0/772170270 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:56:50.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.319+0000 7fa4e0f72700 1 --2- 172.21.15.146:0/772170270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4dc0c9500 0x7fa4dc155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:50.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.319+0000 7fa4e0f72700 1 -- 172.21.15.146:0/772170270 >> v1:172.21.15.154:6789/0 conn(0x7fa4dc14f630 legacy=0x7fa4dc0a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:50.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.319+0000 7fa4e0f72700 1 -- 172.21.15.146:0/772170270 >> v1:172.21.15.146:6789/0 conn(0x7fa4dc156140 legacy=0x7fa4dc0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:50.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.319+0000 7fa4e0f72700 1 -- 172.21.15.146:0/772170270 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4dc0c7b70 con 0x7fa4dc0c9500 2024-09-13T18:56:50.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.319+0000 7fa4e0f72700 1 --2- 172.21.15.146:0/772170270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4dc0c9500 0x7fa4dc155c00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009cb0 tx=0x7fa4d400c160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e47a81983bbdfa6f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:50.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.343+0000 7fa4d3fff700 1 -- 172.21.15.146:0/772170270 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4003a40 con 0x7fa4dc0c9500 2024-09-13T18:56:50.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.343+0000 7fa4d3fff700 1 -- 172.21.15.146:0/772170270 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d4003ba0 con 0x7fa4dc0c9500 2024-09-13T18:56:50.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.343+0000 7fa4e39d7700 1 -- 172.21.15.146:0/772170270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4dc0c9500 msgr2=0x7fa4dc155c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:50.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.343+0000 7fa4e39d7700 1 --2- 172.21.15.146:0/772170270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4dc0c9500 0x7fa4dc155c00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009cb0 tx=0x7fa4d400c160 comp rx=0 tx=0).stop 2024-09-13T18:56:50.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.343+0000 7fa4e39d7700 1 -- 172.21.15.146:0/772170270 shutdown_connections 2024-09-13T18:56:50.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.343+0000 7fa4e39d7700 1 --2- 172.21.15.146:0/772170270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4dc0c9500 0x7fa4dc155c00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:50.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.343+0000 7fa4e39d7700 1 -- 172.21.15.146:0/772170270 >> 172.21.15.146:0/772170270 conn(0x7fa4dc066090 msgr2=0x7fa4dc066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:50.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e39d7700 1 -- 172.21.15.146:0/772170270 shutdown_connections 2024-09-13T18:56:50.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e39d7700 1 -- 172.21.15.146:0/772170270 wait complete. 2024-09-13T18:56:50.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e39d7700 1 Processor -- start 2024-09-13T18:56:50.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e39d7700 1 -- start start 2024-09-13T18:56:50.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e39d7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4dc0c9500 0x7fa4dc1472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:50.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e39d7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4dc14f630 0x7fa4dc14e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:50.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e39d7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4dc156140 0x7fa4dc0aab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:50.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e39d7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc147830 con 0x7fa4dc156140 2024-09-13T18:56:50.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e39d7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc0a9ed0 con 0x7fa4dc14f630 2024-09-13T18:56:50.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e39d7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc06ed00 con 0x7fa4dc0c9500 2024-09-13T18:56:50.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e0f72700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4dc14f630 0x7fa4dc14e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:50.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e0f72700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4dc14f630 0x7fa4dc14e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:37952/0 (socket says 172.21.15.146:37952) 2024-09-13T18:56:50.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e0f72700 1 -- 172.21.15.146:0/2307044456 learned_addr learned my addr 172.21.15.146:0/2307044456 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:50.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.347+0000 7fa4e1f74700 1 --2- 172.21.15.146:0/2307044456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4dc156140 0x7fa4dc0aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:50.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.351+0000 7fa4e0f72700 1 -- 172.21.15.146:0/2307044456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4dc0c9500 msgr2=0x7fa4dc1472f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:56:50.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.351+0000 7fa4e0f72700 1 --2- 172.21.15.146:0/2307044456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4dc0c9500 0x7fa4dc1472f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:50.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.351+0000 7fa4e0f72700 1 -- 172.21.15.146:0/2307044456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4dc156140 msgr2=0x7fa4dc0aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:50.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.351+0000 7fa4e0f72700 1 --2- 172.21.15.146:0/2307044456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4dc156140 0x7fa4dc0aab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:50.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.351+0000 7fa4e0f72700 1 -- 172.21.15.146:0/2307044456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d400d040 con 0x7fa4dc14f630 2024-09-13T18:56:50.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.351+0000 7fa4e0f72700 1 --2- 172.21.15.146:0/2307044456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4dc14f630 0x7fa4dc14e470 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0c8860 tx=0x7fa4d4004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:50.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.355+0000 7fa4d1ffb700 1 -- 172.21.15.146:0/2307044456 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d400ce80 con 0x7fa4dc14f630 2024-09-13T18:56:50.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.355+0000 7fa4e39d7700 1 -- 172.21.15.146:0/2307044456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4dc0aa100 con 0x7fa4dc14f630 2024-09-13T18:56:50.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.355+0000 7fa4e39d7700 1 -- 172.21.15.146:0/2307044456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4dc06ef50 con 0x7fa4dc14f630 2024-09-13T18:56:50.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.355+0000 7fa4d1ffb700 1 -- 172.21.15.146:0/2307044456 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d4003b10 con 0x7fa4dc14f630 2024-09-13T18:56:50.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.355+0000 7fa4d1ffb700 1 -- 172.21.15.146:0/2307044456 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40048c0 con 0x7fa4dc14f630 2024-09-13T18:56:50.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.359+0000 7fa4d1ffb700 1 -- 172.21.15.146:0/2307044456 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa4d4022070 con 0x7fa4dc14f630 2024-09-13T18:56:50.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.359+0000 7fa4d1ffb700 1 --2- 172.21.15.146:0/2307044456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa4c0066800 0x7fa4c0068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:50.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.359+0000 7fa4d1ffb700 1 -- 172.21.15.146:0/2307044456 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fa4d40864e0 con 0x7fa4dc14f630 2024-09-13T18:56:50.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.359+0000 7fa4e1773700 1 --2- 172.21.15.146:0/2307044456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa4c0066800 0x7fa4c0068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.359+0000 7fa4e1773700 1 --2- 172.21.15.146:0/2307044456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa4c0066800 0x7fa4c0068cc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0016f0 tx=0x7fa4cc007a60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:50.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.363+0000 7fa4e39d7700 1 -- 172.21.15.146:0/2307044456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4bc005310 con 0x7fa4dc14f630 2024-09-13T18:56:50.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.375+0000 7fa4d1ffb700 1 -- 172.21.15.146:0/2307044456 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa4d4052fa0 con 0x7fa4dc14f630 2024-09-13T18:56:50.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.803+0000 7fa4e39d7700 1 -- 172.21.15.146:0/2307044456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4bc0055c0 con 0x7fa4dc14f630 2024-09-13T18:56:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.807+0000 7fa4d1ffb700 1 -- 172.21.15.146:0/2307044456 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+20825 (secure 0 0 0) 0x7fa4d40565c0 con 0x7fa4dc14f630 2024-09-13T18:56:50.824 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:56:50.825 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":43,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:55:25.944150+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.13","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/2053734262":"2024-09-14T18:55:25.944088+0000","172.21.15.154:6839/3274220802":"2024-09-14T18:36:10.677826+0000","172.21.15.146:6835/294014284":"2024-09-14T18:50:50.876658+0000","172.21.15.154:6837/3274220802":"2024-09-14T18:36:10.677826+0000","172.21.15.146:6836/2169954140":"2024-09-14T18:53:50.924269+0000","172.21.15.146:6832/1498985282":"2024-09-14T18:34:25.647701+0000","172.21.15.146:6835/1642500214":"2024-09-14T18:30:30.598692+0000","172.21.15.146:6836/31041557":"2024-09-14T18:32:55.627334+0000","172.21.15.146:6832/3023779168":"2024-09-14T18:52:35.902095+0000","172.21.15.146:6834/1642500214":"2024-09-14T18:30:30.598692+0000","172.21.15.146:6836/3234109877":"2024-09-14T18:45:55.807881+0000","172.21.15.146:6837/2169954140":"2024-09-14T18:53:50.924269+0000","172.21.15.146:6833/1498985282":"2024-09-14T18:34:25.647701+0000","172.21.15.146:6837/31041557":"2024-09-14T18:32:55.627334+0000","172.21.15.154:6839/776638146":"2024-09-14T18:44:35.790651+0000","172.21.15.154:6839/3978828499":"2024-09-14T18:29:25.578693+0000","172.21.15.146:6832/2835746745":"2024-09-14T18:28:15.564530+0000","172.21.15.146:6834/1652676664":"2024-09-14T18:38:00.699180+0000","172.21.15.154:6837/776638146":"2024-09-14T18:44:35.790651+0000","172.21.15.154:6837/925899756":"2024-09-14T18:48:20.844616+0000","172.21.15.146:6833/2835746745":"2024-09-14T18:28:15.564530+0000","172.21.15.146:6835/1652676664":"2024-09-14T18:38:00.699180+0000","172.21.15.154:6839/2053734262":"2024-09-14T18:55:25.944088+0000","172.21.15.154:6839/925899756":"2024-09-14T18:48:20.844616+0000","172.21.15.154:6837/3978828499":"2024-09-14T18:29:25.578693+0000","172.21.15.146:6836/3926931671":"2024-09-14T18:40:45.735995+0000","172.21.15.146:6837/3926931671":"2024-09-14T18:40:45.735995+0000","172.21.15.146:6833/3901685195":"2024-09-14T18:43:15.768366+0000","172.21.15.146:6837/3234109877":"2024-09-14T18:45:55.807881+0000","172.21.15.146:6834/294014284":"2024-09-14T18:50:50.876658+0000","172.21.15.146:6832/3901685195":"2024-09-14T18:43:15.768366+0000","172.21.15.146:6833/3023779168":"2024-09-14T18:52:35.902095+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:56:50.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.811+0000 7fa4c77fe700 1 -- 172.21.15.146:0/2307044456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa4c0066800 msgr2=0x7fa4c0068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:50.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.811+0000 7fa4c77fe700 1 --2- 172.21.15.146:0/2307044456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa4c0066800 0x7fa4c0068cc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0016f0 tx=0x7fa4cc007a60 comp rx=0 tx=0).stop 2024-09-13T18:56:50.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.811+0000 7fa4c77fe700 1 -- 172.21.15.146:0/2307044456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4dc14f630 msgr2=0x7fa4dc14e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:50.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.811+0000 7fa4c77fe700 1 --2- 172.21.15.146:0/2307044456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4dc14f630 0x7fa4dc14e470 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0c8860 tx=0x7fa4d4004af0 comp rx=0 tx=0).stop 2024-09-13T18:56:50.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.811+0000 7fa4c77fe700 1 -- 172.21.15.146:0/2307044456 shutdown_connections 2024-09-13T18:56:50.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.811+0000 7fa4c77fe700 1 --2- 172.21.15.146:0/2307044456 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa4c0066800 0x7fa4c0068cc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:50.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.811+0000 7fa4c77fe700 1 --2- 172.21.15.146:0/2307044456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4dc156140 0x7fa4dc0aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:50.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.811+0000 7fa4c77fe700 1 --2- 172.21.15.146:0/2307044456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4dc14f630 0x7fa4dc14e470 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:50.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.811+0000 7fa4c77fe700 1 --2- 172.21.15.146:0/2307044456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4dc0c9500 0x7fa4dc1472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:50.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.811+0000 7fa4c77fe700 1 -- 172.21.15.146:0/2307044456 >> 172.21.15.146:0/2307044456 conn(0x7fa4dc066090 msgr2=0x7fa4dc06c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:50.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.811+0000 7fa4c77fe700 1 -- 172.21.15.146:0/2307044456 shutdown_connections 2024-09-13T18:56:50.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:50.811+0000 7fa4c77fe700 1 -- 172.21.15.146:0/2307044456 wait complete. 2024-09-13T18:56:50.851 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:26:24.867515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:26:30.782269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:26:31.323277+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:26:34.393386+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:56:50.852 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:56:51.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.111+0000 7f961b59e700 1 Processor -- start 2024-09-13T18:56:51.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.111+0000 7f961b59e700 1 -- start start 2024-09-13T18:56:51.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961b59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f961c153770 0x7f961c15d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:51.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961b59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f961c0c5520 0x7f961c0c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:51.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961ad9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f961c0c6480 0x7f961c06cd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48912/0 (socket says 172.21.15.146:48912) 2024-09-13T18:56:51.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961ad9d700 1 -- 172.21.15.146:0/698722105 learned_addr learned my addr 172.21.15.146:0/698722105 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:56:51.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961b59e700 1 -- 172.21.15.146:0/698722105 --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c15da70 con 0x7f961c0c6480 2024-09-13T18:56:51.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961b59e700 1 -- 172.21.15.146:0/698722105 --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961c06b440 con 0x7f961c153770 2024-09-13T18:56:51.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961b59e700 1 -- 172.21.15.146:0/698722105 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961c0c68d0 con 0x7f961c0c5520 2024-09-13T18:56:51.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961a59c700 1 --2- 172.21.15.146:0/698722105 >> v2:172.21.15.146:3300/0 conn(0x7f961c153770 0x7f961c15d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:51.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961959a700 1 -- 172.21.15.146:0/698722105 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552713804 0 0) 0x7f9614001490 con 0x7f961c0c6480 2024-09-13T18:56:51.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961959a700 1 -- 172.21.15.146:0/698722105 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96040035d0 con 0x7f961c0c6480 2024-09-13T18:56:51.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961a59c700 1 -- 172.21.15.146:0/698722105 >> v2:172.21.15.154:3300/0 conn(0x7f961c0c5520 msgr2=0x7f961c0c5f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:56:51.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961a59c700 1 --2- 172.21.15.146:0/698722105 >> v2:172.21.15.154:3300/0 conn(0x7f961c0c5520 0x7f961c0c5f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961a59c700 1 -- 172.21.15.146:0/698722105 >> v1:172.21.15.146:6789/0 conn(0x7f961c0c6480 legacy=0x7f961c06cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:51.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961a59c700 1 -- 172.21.15.146:0/698722105 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c0665f0 con 0x7f961c153770 2024-09-13T18:56:51.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961a59c700 1 --2- 172.21.15.146:0/698722105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c153770 0x7f961c15d530 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f961c0668f0 tx=0x7f9610009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc75a6654e3fae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:51.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961959a700 1 -- 172.21.15.146:0/698722105 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610011830 con 0x7f961c153770 2024-09-13T18:56:51.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961959a700 1 -- 172.21.15.146:0/698722105 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9610011990 con 0x7f961c153770 2024-09-13T18:56:51.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961959a700 1 -- 172.21.15.146:0/698722105 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610011d20 con 0x7f961c153770 2024-09-13T18:56:51.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961b59e700 1 -- 172.21.15.146:0/698722105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c153770 msgr2=0x7f961c15d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:51.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.123+0000 7f961b59e700 1 --2- 172.21.15.146:0/698722105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c153770 0x7f961c15d530 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f961c0668f0 tx=0x7f9610009b30 comp rx=0 tx=0).stop 2024-09-13T18:56:51.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.127+0000 7f961b59e700 1 -- 172.21.15.146:0/698722105 shutdown_connections 2024-09-13T18:56:51.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.127+0000 7f961b59e700 1 --2- 172.21.15.146:0/698722105 >> v2:172.21.15.154:3300/0 conn(0x7f961c0c5520 0x7f961c0c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.127+0000 7f961b59e700 1 --2- 172.21.15.146:0/698722105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c153770 0x7f961c15d530 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.127+0000 7f961b59e700 1 -- 172.21.15.146:0/698722105 >> 172.21.15.146:0/698722105 conn(0x7f961c066b00 msgr2=0x7f961c06aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:51.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.127+0000 7f961b59e700 1 -- 172.21.15.146:0/698722105 shutdown_connections 2024-09-13T18:56:51.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.127+0000 7f961b59e700 1 -- 172.21.15.146:0/698722105 wait complete. 2024-09-13T18:56:51.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.127+0000 7f961b59e700 1 Processor -- start 2024-09-13T18:56:51.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.127+0000 7f961b59e700 1 -- start start 2024-09-13T18:56:51.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.127+0000 7f961b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f961c0c5520 0x7f961c0c4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:51.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.127+0000 7f961b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c0c6480 0x7f961c150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:51.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.127+0000 7f961b59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f961c153770 0x7f961c0c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:51.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.127+0000 7f961b59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c15c0c0 con 0x7f961c0c6480 2024-09-13T18:56:51.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f9619d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c0c6480 0x7f961c150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:51.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f961b59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c0c3d70 con 0x7f961c153770 2024-09-13T18:56:51.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f961b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961c0c26f0 con 0x7f961c0c5520 2024-09-13T18:56:51.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f961a59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f961c0c5520 0x7f961c0c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:51.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f9619d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c0c6480 0x7f961c150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35394/0 (socket says 172.21.15.146:35394) 2024-09-13T18:56:51.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f9619d9b700 1 -- 172.21.15.146:0/3162961964 learned_addr learned my addr 172.21.15.146:0/3162961964 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:51.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f961ad9d700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f961c153770 0x7f961c0c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:51.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f961a59c700 1 -- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f961c153770 msgr2=0x7f961c0c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:51.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f961a59c700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f961c153770 0x7f961c0c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f961a59c700 1 -- 172.21.15.146:0/3162961964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c0c6480 msgr2=0x7f961c150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f961a59c700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c0c6480 0x7f961c150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f961a59c700 1 -- 172.21.15.146:0/3162961964 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96100094b0 con 0x7f961c0c5520 2024-09-13T18:56:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f9619d9b700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c0c6480 0x7f961c150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:56:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f961a59c700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f961c0c5520 0x7f961c0c4200 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9610016000 tx=0x7f9610004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f960affd700 1 -- 172.21.15.146:0/3162961964 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610004590 con 0x7f961c0c5520 2024-09-13T18:56:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f960affd700 1 -- 172.21.15.146:0/3162961964 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961001e070 con 0x7f961c0c5520 2024-09-13T18:56:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f961b59e700 1 -- 172.21.15.146:0/3162961964 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961c15c2d0 con 0x7f961c0c5520 2024-09-13T18:56:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f960affd700 1 -- 172.21.15.146:0/3162961964 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610003900 con 0x7f961c0c5520 2024-09-13T18:56:51.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f961b59e700 1 -- 172.21.15.146:0/3162961964 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961c153b50 con 0x7f961c0c5520 2024-09-13T18:56:51.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.131+0000 7f960affd700 1 -- 172.21.15.146:0/3162961964 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9610003a60 con 0x7f961c0c5520 2024-09-13T18:56:51.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.135+0000 7f960affd700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95fc066760 0x7f95fc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:51.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.135+0000 7f960affd700 1 -- 172.21.15.146:0/3162961964 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f9610085790 con 0x7f961c0c5520 2024-09-13T18:56:51.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.135+0000 7f9619d9b700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95fc066760 0x7f95fc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:51.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.147+0000 7f961b59e700 1 -- 172.21.15.146:0/3162961964 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961c1595b0 con 0x7f961c0c5520 2024-09-13T18:56:51.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.151+0000 7f9619d9b700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95fc066760 0x7f95fc068c20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f961c1565c0 tx=0x7f960c009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:51.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.159+0000 7f960affd700 1 -- 172.21.15.146:0/3162961964 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f96100521d0 con 0x7f961c0c5520 2024-09-13T18:56:51.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.471+0000 7f961b59e700 1 -- 172.21.15.146:0/3162961964 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f961c0c6260 con 0x7f961c0c5520 2024-09-13T18:56:51.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.471+0000 7f960affd700 1 -- 172.21.15.146:0/3162961964 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96100150a0 con 0x7f961c0c5520 2024-09-13T18:56:51.485 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-09-13T18:56:51.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.475+0000 7f961b59e700 1 -- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95fc066760 msgr2=0x7f95fc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:51.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.475+0000 7f961b59e700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95fc066760 0x7f95fc068c20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f961c1565c0 tx=0x7f960c009240 comp rx=0 tx=0).stop 2024-09-13T18:56:51.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.475+0000 7f961b59e700 1 -- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f961c0c5520 msgr2=0x7f961c0c4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:51.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.475+0000 7f961b59e700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f961c0c5520 0x7f961c0c4200 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9610016000 tx=0x7f9610004770 comp rx=0 tx=0).stop 2024-09-13T18:56:51.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.479+0000 7f961b59e700 1 -- 172.21.15.146:0/3162961964 shutdown_connections 2024-09-13T18:56:51.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.479+0000 7f961b59e700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95fc066760 0x7f95fc068c20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.479+0000 7f961b59e700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f961c153770 0x7f961c0c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.479+0000 7f961b59e700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c0c6480 0x7f961c150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.479+0000 7f961b59e700 1 --2- 172.21.15.146:0/3162961964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f961c0c5520 0x7f961c0c4200 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.479+0000 7f961b59e700 1 -- 172.21.15.146:0/3162961964 >> 172.21.15.146:0/3162961964 conn(0x7f961c066b00 msgr2=0x7f961c150830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:51.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.479+0000 7f961b59e700 1 -- 172.21.15.146:0/3162961964 shutdown_connections 2024-09-13T18:56:51.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.479+0000 7f961b59e700 1 -- 172.21.15.146:0/3162961964 wait complete. 2024-09-13T18:56:51.514 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:56:51.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3b0a78700 1 Processor -- start 2024-09-13T18:56:51.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3b0a78700 1 -- start start 2024-09-13T18:56:51.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3b0a78700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3ac0c9120 0x7fb3ac0c7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:51.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3b0a78700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb3ac15a390 0x7fb3ac0cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:51.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3b0a78700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ac159f00 con 0x7fb3ac0c8d40 2024-09-13T18:56:51.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3b0a78700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3ac1596e0 con 0x7fb3ac15a390 2024-09-13T18:56:51.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3b0a78700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0a9af0 con 0x7fb3ac0c9120 2024-09-13T18:56:51.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3ab7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb3ac0c8d40 0x7fb3ac072460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48928/0 (socket says 172.21.15.146:48928) 2024-09-13T18:56:51.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3ab7fe700 1 -- 172.21.15.146:0/393991751 learned_addr learned my addr 172.21.15.146:0/393991751 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:56:51.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3aaffd700 1 --2- 172.21.15.146:0/393991751 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3ac0c9120 0x7fb3ac0c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:51.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3abfff700 1 --2- 172.21.15.146:0/393991751 >> v2:172.21.15.154:3300/0 conn(0x7fb3ac15a390 0x7fb3ac0cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:51.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3aaffd700 1 -- 172.21.15.146:0/393991751 >> v2:172.21.15.154:3300/0 conn(0x7fb3ac15a390 msgr2=0x7fb3ac0cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:56:51.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3aaffd700 1 --2- 172.21.15.146:0/393991751 >> v2:172.21.15.154:3300/0 conn(0x7fb3ac15a390 0x7fb3ac0cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3aaffd700 1 -- 172.21.15.146:0/393991751 >> v1:172.21.15.146:6789/0 conn(0x7fb3ac0c8d40 legacy=0x7fb3ac072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:51.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3aaffd700 1 -- 172.21.15.146:0/393991751 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac0c7b70 con 0x7fb3ac0c9120 2024-09-13T18:56:51.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3aa7fc700 1 -- 172.21.15.146:0/393991751 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062079117 0 0) 0x7fb3ac159f00 con 0x7fb3ac0c8d40 2024-09-13T18:56:51.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3aaffd700 1 --2- 172.21.15.146:0/393991751 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3ac0c9120 0x7fb3ac0c7630 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009dc0 tx=0x7fb39c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46f5b1b7bcf794ed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:51.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3aa7fc700 1 -- 172.21.15.146:0/393991751 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c00e070 con 0x7fb3ac0c9120 2024-09-13T18:56:51.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3aa7fc700 1 -- 172.21.15.146:0/393991751 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39c00bae0 con 0x7fb3ac0c9120 2024-09-13T18:56:51.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.731+0000 7fb3aa7fc700 1 -- 172.21.15.146:0/393991751 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c00be90 con 0x7fb3ac0c9120 2024-09-13T18:56:51.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.739+0000 7fb3b0a78700 1 -- 172.21.15.146:0/393991751 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3ac0c9120 msgr2=0x7fb3ac0c7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:51.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.739+0000 7fb3b0a78700 1 --2- 172.21.15.146:0/393991751 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3ac0c9120 0x7fb3ac0c7630 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009dc0 tx=0x7fb39c00b330 comp rx=0 tx=0).stop 2024-09-13T18:56:51.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.751+0000 7fb3b0a78700 1 -- 172.21.15.146:0/393991751 shutdown_connections 2024-09-13T18:56:51.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.751+0000 7fb3b0a78700 1 --2- 172.21.15.146:0/393991751 >> v2:172.21.15.154:3300/0 conn(0x7fb3ac15a390 0x7fb3ac0cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.751+0000 7fb3b0a78700 1 --2- 172.21.15.146:0/393991751 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3ac0c9120 0x7fb3ac0c7630 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.751+0000 7fb3b0a78700 1 -- 172.21.15.146:0/393991751 >> 172.21.15.146:0/393991751 conn(0x7fb3ac066090 msgr2=0x7fb3ac066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:51.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.751+0000 7fb3b0a78700 1 -- 172.21.15.146:0/393991751 shutdown_connections 2024-09-13T18:56:51.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.751+0000 7fb3b0a78700 1 -- 172.21.15.146:0/393991751 wait complete. 2024-09-13T18:56:51.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.755+0000 7fb3b0a78700 1 Processor -- start 2024-09-13T18:56:51.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3b0a78700 1 -- start start 2024-09-13T18:56:51.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3b0a78700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0c8d40 0x7fb3ac152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:51.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3b0a78700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3ac0c9120 0x7fb3ac0c0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:51.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3b0a78700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3ac15a390 0x7fb3ac0c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:51.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3b0a78700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac1ef050 con 0x7fb3ac0c9120 2024-09-13T18:56:51.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3b0a78700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac1ee460 con 0x7fb3ac0c8d40 2024-09-13T18:56:51.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3b0a78700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac1ee720 con 0x7fb3ac15a390 2024-09-13T18:56:51.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3aaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3ac0c9120 0x7fb3ac0c0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:51.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3aaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3ac0c9120 0x7fb3ac0c0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35398/0 (socket says 172.21.15.146:35398) 2024-09-13T18:56:51.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3aaffd700 1 -- 172.21.15.146:0/3532356697 learned_addr learned my addr 172.21.15.146:0/3532356697 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:51.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3abfff700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3ac15a390 0x7fb3ac0c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:51.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3ab7fe700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0c8d40 0x7fb3ac152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:51.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3abfff700 1 -- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0c8d40 msgr2=0x7fb3ac152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:51.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3abfff700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0c8d40 0x7fb3ac152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3abfff700 1 -- 172.21.15.146:0/3532356697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3ac0c9120 msgr2=0x7fb3ac0c0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:51.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3abfff700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3ac0c9120 0x7fb3ac0c0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:51.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3abfff700 1 -- 172.21.15.146:0/3532356697 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c009a70 con 0x7fb3ac15a390 2024-09-13T18:56:51.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.759+0000 7fb3abfff700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3ac15a390 0x7fb3ac0c13e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb394003ab0 tx=0x7fb394003dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:51.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.763+0000 7fb3a3fff700 1 -- 172.21.15.146:0/3532356697 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394018040 con 0x7fb3ac15a390 2024-09-13T18:56:51.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.763+0000 7fb3b0a78700 1 -- 172.21.15.146:0/3532356697 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ac1eea00 con 0x7fb3ac15a390 2024-09-13T18:56:51.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.763+0000 7fb3b0a78700 1 -- 172.21.15.146:0/3532356697 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ac06ea00 con 0x7fb3ac15a390 2024-09-13T18:56:51.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.763+0000 7fb3a3fff700 1 -- 172.21.15.146:0/3532356697 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb394009bd0 con 0x7fb3ac15a390 2024-09-13T18:56:51.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.763+0000 7fb3a3fff700 1 -- 172.21.15.146:0/3532356697 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39401c420 con 0x7fb3ac15a390 2024-09-13T18:56:51.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.767+0000 7fb3a3fff700 1 -- 172.21.15.146:0/3532356697 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb39401c5c0 con 0x7fb3ac15a390 2024-09-13T18:56:51.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.767+0000 7fb3a3fff700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb3980668f0 0x7fb398068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:51.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.767+0000 7fb3a3fff700 1 -- 172.21.15.146:0/3532356697 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fb39408c540 con 0x7fb3ac15a390 2024-09-13T18:56:51.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.767+0000 7fb3ab7fe700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb3980668f0 0x7fb398068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:51.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.767+0000 7fb3ab7fe700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb3980668f0 0x7fb398068db0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4000f60 tx=0x7fb3a4007e00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:51.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.775+0000 7fb3a1ffb700 1 -- 172.21.15.146:0/3532356697 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb390005310 con 0x7fb3ac15a390 2024-09-13T18:56:51.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:51.787+0000 7fb3a3fff700 1 -- 172.21.15.146:0/3532356697 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb394059000 con 0x7fb3ac15a390 2024-09-13T18:56:52.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.103+0000 7fb3a1ffb700 1 -- 172.21.15.146:0/3532356697 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb390005b40 con 0x7fb3ac15a390 2024-09-13T18:56:52.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.107+0000 7fb3a3fff700 1 -- 172.21.15.146:0/3532356697 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb39405c620 con 0x7fb3ac15a390 2024-09-13T18:56:52.121 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:56:52.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.119+0000 7fb3b0a78700 1 -- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb3980668f0 msgr2=0x7fb398068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:52.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.119+0000 7fb3b0a78700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb3980668f0 0x7fb398068db0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4000f60 tx=0x7fb3a4007e00 comp rx=0 tx=0).stop 2024-09-13T18:56:52.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.119+0000 7fb3b0a78700 1 -- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3ac15a390 msgr2=0x7fb3ac0c13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:52.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.119+0000 7fb3b0a78700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3ac15a390 0x7fb3ac0c13e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb394003ab0 tx=0x7fb394003dc0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.119+0000 7fb3b0a78700 1 -- 172.21.15.146:0/3532356697 shutdown_connections 2024-09-13T18:56:52.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.119+0000 7fb3b0a78700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fb3980668f0 0x7fb398068db0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.119+0000 7fb3b0a78700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3ac15a390 0x7fb3ac0c13e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.119+0000 7fb3b0a78700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3ac0c9120 0x7fb3ac0c0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.119+0000 7fb3b0a78700 1 --2- 172.21.15.146:0/3532356697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0c8d40 0x7fb3ac152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.119+0000 7fb3b0a78700 1 -- 172.21.15.146:0/3532356697 >> 172.21.15.146:0/3532356697 conn(0x7fb3ac066090 msgr2=0x7fb3ac066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:52.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.119+0000 7fb3b0a78700 1 -- 172.21.15.146:0/3532356697 shutdown_connections 2024-09-13T18:56:52.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.119+0000 7fb3b0a78700 1 -- 172.21.15.146:0/3532356697 wait complete. 2024-09-13T18:56:52.165 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:56:52.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8d4ba5700 1 Processor -- start 2024-09-13T18:56:52.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8d4ba5700 1 -- start start 2024-09-13T18:56:52.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8d4ba5700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd8d00a9490 0x7fd8d006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:52.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8d4ba5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8d006f5c0 0x7fd8d006fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:52.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8d4ba5700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d006ffb0 con 0x7fd8d014f750 2024-09-13T18:56:52.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8d4ba5700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d014fb30 con 0x7fd8d00a9490 2024-09-13T18:56:52.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8d4ba5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d0155580 con 0x7fd8d006f5c0 2024-09-13T18:56:52.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8ceffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd8d00a9490 0x7fd8d006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:52.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8ceffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd8d00a9490 0x7fd8d006eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35410/0 (socket says 172.21.15.146:35410) 2024-09-13T18:56:52.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8ceffd700 1 -- 172.21.15.146:0/1593740646 learned_addr learned my addr 172.21.15.146:0/1593740646 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:52.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8cf7fe700 1 --1- 172.21.15.146:0/1593740646 >> v1:172.21.15.146:6789/0 conn(0x7fd8d014f750 0x7fd8d00cb120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48930/0 (socket says 172.21.15.146:48930) 2024-09-13T18:56:52.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8ceffd700 1 -- 172.21.15.146:0/1593740646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8d006f5c0 msgr2=0x7fd8d006fa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:56:52.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8ceffd700 1 --2- 172.21.15.146:0/1593740646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8d006f5c0 0x7fd8d006fa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8ceffd700 1 -- 172.21.15.146:0/1593740646 >> v1:172.21.15.146:6789/0 conn(0x7fd8d014f750 legacy=0x7fd8d00cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:52.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.371+0000 7fd8ceffd700 1 -- 172.21.15.146:0/1593740646 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d0155860 con 0x7fd8d00a9490 2024-09-13T18:56:52.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.375+0000 7fd8ceffd700 1 --2- 172.21.15.146:0/1593740646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d00a9490 0x7fd8d006eea0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009e50 tx=0x7fd8c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8115990b6057c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:52.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.375+0000 7fd8ce7fc700 1 -- 172.21.15.146:0/1593740646 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8003c40 con 0x7fd8d00a9490 2024-09-13T18:56:52.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.375+0000 7fd8ce7fc700 1 -- 172.21.15.146:0/1593740646 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c8003da0 con 0x7fd8d00a9490 2024-09-13T18:56:52.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.375+0000 7fd8ce7fc700 1 -- 172.21.15.146:0/1593740646 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800bb00 con 0x7fd8d00a9490 2024-09-13T18:56:52.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.375+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1593740646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d00a9490 msgr2=0x7fd8d006eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:52.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.375+0000 7fd8d4ba5700 1 --2- 172.21.15.146:0/1593740646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d00a9490 0x7fd8d006eea0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009e50 tx=0x7fd8c800b330 comp rx=0 tx=0).stop 2024-09-13T18:56:52.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.379+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1593740646 shutdown_connections 2024-09-13T18:56:52.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.379+0000 7fd8d4ba5700 1 --2- 172.21.15.146:0/1593740646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8d006f5c0 0x7fd8d006fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.379+0000 7fd8d4ba5700 1 --2- 172.21.15.146:0/1593740646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d00a9490 0x7fd8d006eea0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.379+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1593740646 >> 172.21.15.146:0/1593740646 conn(0x7fd8d0066090 msgr2=0x7fd8d0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:52.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.387+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1593740646 shutdown_connections 2024-09-13T18:56:52.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.395+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1593740646 wait complete. 2024-09-13T18:56:52.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.395+0000 7fd8d4ba5700 1 Processor -- start 2024-09-13T18:56:52.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.395+0000 7fd8d4ba5700 1 -- start start 2024-09-13T18:56:52.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.395+0000 7fd8d4ba5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8d006f5c0 0x7fd8d00c10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:52.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.395+0000 7fd8d4ba5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8d00a9490 0x7fd8d01567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:52.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.395+0000 7fd8d4ba5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d014f750 0x7fd8d01533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:52.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8d4ba5700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d0159490 con 0x7fd8d014f750 2024-09-13T18:56:52.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8d4ba5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d0159920 con 0x7fd8d00a9490 2024-09-13T18:56:52.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8d4ba5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00c4930 con 0x7fd8d006f5c0 2024-09-13T18:56:52.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8cffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d014f750 0x7fd8d01533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:52.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8cffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d014f750 0x7fd8d01533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35422/0 (socket says 172.21.15.146:35422) 2024-09-13T18:56:52.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8cffff700 1 -- 172.21.15.146:0/1680445612 learned_addr learned my addr 172.21.15.146:0/1680445612 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:52.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8cf7fe700 1 --2- 172.21.15.146:0/1680445612 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8d006f5c0 0x7fd8d00c10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:52.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8cffff700 1 -- 172.21.15.146:0/1680445612 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8d006f5c0 msgr2=0x7fd8d00c10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:52.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8cffff700 1 --2- 172.21.15.146:0/1680445612 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8d006f5c0 0x7fd8d00c10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8cffff700 1 -- 172.21.15.146:0/1680445612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8d00a9490 msgr2=0x7fd8d01567b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:56:52.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8cffff700 1 --2- 172.21.15.146:0/1680445612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8d00a9490 0x7fd8d01567b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8cffff700 1 -- 172.21.15.146:0/1680445612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8009b30 con 0x7fd8d014f750 2024-09-13T18:56:52.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8cffff700 1 --2- 172.21.15.146:0/1680445612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d014f750 0x7fd8d01533f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400b4d0 tx=0x7fd8c400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:52.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8bffff700 1 -- 172.21.15.146:0/1680445612 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4011830 con 0x7fd8d014f750 2024-09-13T18:56:52.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1680445612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d00c43a0 con 0x7fd8d014f750 2024-09-13T18:56:52.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.399+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1680445612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d00c4e90 con 0x7fd8d014f750 2024-09-13T18:56:52.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.407+0000 7fd8bffff700 1 -- 172.21.15.146:0/1680445612 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c4011990 con 0x7fd8d014f750 2024-09-13T18:56:52.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.407+0000 7fd8bffff700 1 -- 172.21.15.146:0/1680445612 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4011d40 con 0x7fd8d014f750 2024-09-13T18:56:52.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.407+0000 7fd8bffff700 1 -- 172.21.15.146:0/1680445612 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd8c401a400 con 0x7fd8d014f750 2024-09-13T18:56:52.420 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.407+0000 7fd8bffff700 1 --2- 172.21.15.146:0/1680445612 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd8b4066880 0x7fd8b4068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:52.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.407+0000 7fd8cf7fe700 1 --2- 172.21.15.146:0/1680445612 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd8b4066880 0x7fd8b4068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:52.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.407+0000 7fd8bffff700 1 -- 172.21.15.146:0/1680445612 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fd8c4086220 con 0x7fd8d014f750 2024-09-13T18:56:52.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.407+0000 7fd8cf7fe700 1 --2- 172.21.15.146:0/1680445612 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd8b4066880 0x7fd8b4068d40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0001300 tx=0x7fd8c0001290 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:52.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.407+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1680445612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ac005310 con 0x7fd8d014f750 2024-09-13T18:56:52.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.419+0000 7fd8bffff700 1 -- 172.21.15.146:0/1680445612 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd8c4015080 con 0x7fd8d014f750 2024-09-13T18:56:52.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.787+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1680445612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8ac005b40 con 0x7fd8d014f750 2024-09-13T18:56:52.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.787+0000 7fd8bffff700 1 -- 172.21.15.146:0/1680445612 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd8c40507c0 con 0x7fd8d014f750 2024-09-13T18:56:52.799 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-09-13T18:56:52.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.791+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1680445612 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd8b4066880 msgr2=0x7fd8b4068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:52.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.791+0000 7fd8d4ba5700 1 --2- 172.21.15.146:0/1680445612 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd8b4066880 0x7fd8b4068d40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0001300 tx=0x7fd8c0001290 comp rx=0 tx=0).stop 2024-09-13T18:56:52.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.791+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1680445612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d014f750 msgr2=0x7fd8d01533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:52.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.791+0000 7fd8d4ba5700 1 --2- 172.21.15.146:0/1680445612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d014f750 0x7fd8d01533f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400b4d0 tx=0x7fd8c400b7e0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.791+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1680445612 shutdown_connections 2024-09-13T18:56:52.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.791+0000 7fd8d4ba5700 1 --2- 172.21.15.146:0/1680445612 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd8b4066880 0x7fd8b4068d40 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.791+0000 7fd8d4ba5700 1 --2- 172.21.15.146:0/1680445612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d014f750 0x7fd8d01533f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.791+0000 7fd8d4ba5700 1 --2- 172.21.15.146:0/1680445612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8d00a9490 0x7fd8d01567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.791+0000 7fd8d4ba5700 1 --2- 172.21.15.146:0/1680445612 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8d006f5c0 0x7fd8d00c10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:52.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.791+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1680445612 >> 172.21.15.146:0/1680445612 conn(0x7fd8d0066090 msgr2=0x7fd8d0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:52.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.791+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1680445612 shutdown_connections 2024-09-13T18:56:52.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:52.791+0000 7fd8d4ba5700 1 -- 172.21.15.146:0/1680445612 wait complete. 2024-09-13T18:56:52.824 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:56:53.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.039+0000 7f5e527be700 1 Processor -- start 2024-09-13T18:56:53.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.043+0000 7f5e527be700 1 -- start start 2024-09-13T18:56:53.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e527be700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e4c0c8f70 0x7f5e4c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e527be700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5e4c0c9350 0x7f5e4c0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e527be700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4c0c8650 con 0x7f5e4c0a9490 2024-09-13T18:56:53.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e527be700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4c0aa1a0 con 0x7f5e4c0c9350 2024-09-13T18:56:53.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e527be700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0aa480 con 0x7f5e4c0c8f70 2024-09-13T18:56:53.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e51fbd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5e4c0a9490 0x7f5e4c0cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:34456/0 (socket says 172.21.15.146:34456) 2024-09-13T18:56:53.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e51fbd700 1 -- 172.21.15.146:0/4162924673 learned_addr learned my addr 172.21.15.146:0/4162924673 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:56:53.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735480732 0 0) 0x7f5e4c0c8650 con 0x7f5e4c0a9490 2024-09-13T18:56:53.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e380035d0 con 0x7f5e4c0a9490 2024-09-13T18:56:53.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4160428591 0 0) 0x7f5e380035d0 con 0x7f5e4c0a9490 2024-09-13T18:56:53.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e4c0c8f70 msgr2=0x7f5e4c15a370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:56:53.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 --2- 172.21.15.146:0/4162924673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e4c0c8f70 0x7f5e4c15a370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 >> v2:172.21.15.146:3300/0 conn(0x7f5e4c0c9350 msgr2=0x7f5e4c0c76c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:56:53.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 --2- 172.21.15.146:0/4162924673 >> v2:172.21.15.146:3300/0 conn(0x7f5e4c0c9350 0x7f5e4c0c76c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c159890 con 0x7f5e4c0a9490 2024-09-13T18:56:53.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f5e440019f0 con 0x7f5e4c0a9490 2024-09-13T18:56:53.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 >> v1:172.21.15.154:6789/0 conn(0x7f5e4c0a9490 legacy=0x7f5e4c0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:53.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 --2- 172.21.15.146:0/4162924673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e38003a90 0x7f5e38003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 --2- 172.21.15.146:0/4162924673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e38004870 0x7f5e38004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 --2- 172.21.15.146:0/4162924673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e38005350 0x7f5e3801ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0aa1a0 con 0x7f5e38004870 2024-09-13T18:56:53.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0aa480 con 0x7f5e38005350 2024-09-13T18:56:53.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3801d290 con 0x7f5e38003a90 2024-09-13T18:56:53.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e51fbd700 1 --2- 172.21.15.146:0/4162924673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e38005350 0x7f5e3801ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:53.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e51fbd700 1 -- 172.21.15.146:0/4162924673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e38003a90 msgr2=0x7f5e38003eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:56:53.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e51fbd700 1 --2- 172.21.15.146:0/4162924673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e38003a90 0x7f5e38003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e51fbd700 1 -- 172.21.15.146:0/4162924673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e38004870 msgr2=0x7f5e38004cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:56:53.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e51fbd700 1 --2- 172.21.15.146:0/4162924673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e38004870 0x7f5e38004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e51fbd700 1 -- 172.21.15.146:0/4162924673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c159890 con 0x7f5e38005350 2024-09-13T18:56:53.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e51fbd700 1 --2- 172.21.15.146:0/4162924673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e38005350 0x7f5e3801ca40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009990 tx=0x7f5e380058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c86fdb512dafef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:53.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44009380 con 0x7f5e38005350 2024-09-13T18:56:53.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e44030720 con 0x7f5e38005350 2024-09-13T18:56:53.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.047+0000 7f5e43fff700 1 -- 172.21.15.146:0/4162924673 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44030ad0 con 0x7f5e38005350 2024-09-13T18:56:53.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.051+0000 7f5e527be700 1 -- 172.21.15.146:0/4162924673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e38005350 msgr2=0x7f5e3801ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:53.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.051+0000 7f5e527be700 1 --2- 172.21.15.146:0/4162924673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e38005350 0x7f5e3801ca40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009990 tx=0x7f5e380058a0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.051+0000 7f5e51fbd700 1 -- 172.21.15.146:0/4162924673 reap_dead start 2024-09-13T18:56:53.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.075+0000 7f5e527be700 1 -- 172.21.15.146:0/4162924673 shutdown_connections 2024-09-13T18:56:53.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.075+0000 7f5e527be700 1 -- 172.21.15.146:0/4162924673 >> 172.21.15.146:0/4162924673 conn(0x7f5e4c066090 msgr2=0x7f5e4c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:53.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.075+0000 7f5e527be700 1 -- 172.21.15.146:0/4162924673 shutdown_connections 2024-09-13T18:56:53.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.075+0000 7f5e527be700 1 -- 172.21.15.146:0/4162924673 wait complete. 2024-09-13T18:56:53.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.075+0000 7f5e527be700 1 Processor -- start 2024-09-13T18:56:53.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.075+0000 7f5e527be700 1 -- start start 2024-09-13T18:56:53.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e527be700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e38005350 0x7f5e4c0af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e527be700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e4c15d640 0x7f5e4c0c2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e527be700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e4c0abcb0 0x7f5e4c15b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e517bc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e38005350 0x7f5e4c0af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:53.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e517bc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e38005350 0x7f5e4c0af800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:57436/0 (socket says 172.21.15.146:57436) 2024-09-13T18:56:53.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e50fbb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e4c15d640 0x7f5e4c0c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:53.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e50fbb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e4c15d640 0x7f5e4c0c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41396/0 (socket says 172.21.15.146:41396) 2024-09-13T18:56:53.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e517bc700 1 -- 172.21.15.146:0/3803141503 learned_addr learned my addr 172.21.15.146:0/3803141503 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:53.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e527be700 1 -- 172.21.15.146:0/3803141503 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0aed60 con 0x7f5e4c0abcb0 2024-09-13T18:56:53.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e527be700 1 -- 172.21.15.146:0/3803141503 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c15b040 con 0x7f5e38005350 2024-09-13T18:56:53.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e527be700 1 -- 172.21.15.146:0/3803141503 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c15aa80 con 0x7f5e4c15d640 2024-09-13T18:56:53.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e51fbd700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e4c0abcb0 0x7f5e4c15b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:53.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e51fbd700 1 -- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e4c15d640 msgr2=0x7f5e4c0c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:53.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e51fbd700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e4c15d640 0x7f5e4c0c2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e51fbd700 1 -- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e38005350 msgr2=0x7f5e4c0af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:53.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e51fbd700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e38005350 0x7f5e4c0af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e51fbd700 1 -- 172.21.15.146:0/3803141503 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c15ad60 con 0x7f5e4c0abcb0 2024-09-13T18:56:53.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e517bc700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e38005350 0x7f5e4c0af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:56:53.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e50fbb700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e4c15d640 0x7f5e4c0c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:56:53.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e51fbd700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e4c0abcb0 0x7f5e4c15b3b0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5e44037000 tx=0x7f5e44001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:53.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e41ffb700 1 -- 172.21.15.146:0/3803141503 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4403f070 con 0x7f5e4c0abcb0 2024-09-13T18:56:53.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e41ffb700 1 -- 172.21.15.146:0/3803141503 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e44040040 con 0x7f5e4c0abcb0 2024-09-13T18:56:53.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.079+0000 7f5e41ffb700 1 -- 172.21.15.146:0/3803141503 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44009380 con 0x7f5e4c0abcb0 2024-09-13T18:56:53.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.083+0000 7f5e527be700 1 -- 172.21.15.146:0/3803141503 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c0aed60 con 0x7f5e4c0abcb0 2024-09-13T18:56:53.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.083+0000 7f5e527be700 1 -- 172.21.15.146:0/3803141503 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c0af2e0 con 0x7f5e4c0abcb0 2024-09-13T18:56:53.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.091+0000 7f5e527be700 1 -- 172.21.15.146:0/3803141503 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4c0aebc0 con 0x7f5e4c0abcb0 2024-09-13T18:56:53.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.091+0000 7f5e41ffb700 1 -- 172.21.15.146:0/3803141503 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5e44006070 con 0x7f5e4c0abcb0 2024-09-13T18:56:53.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.091+0000 7f5e41ffb700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e380680b0 0x7f5e3806a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.091+0000 7f5e41ffb700 1 -- 172.21.15.146:0/3803141503 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f5e440a7c60 con 0x7f5e4c0abcb0 2024-09-13T18:56:53.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.091+0000 7f5e517bc700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e380680b0 0x7f5e3806a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:53.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.091+0000 7f5e517bc700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e380680b0 0x7f5e3806a570 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5e48006fd0 tx=0x7f5e48009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:53.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.095+0000 7f5e41ffb700 1 -- 172.21.15.146:0/3803141503 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5e4c0aebc0 con 0x7f5e4c0abcb0 2024-09-13T18:56:53.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.387+0000 7f5e527be700 1 -- 172.21.15.146:0/3803141503 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5e4c0aebc0 con 0x7f5e4c0abcb0 2024-09-13T18:56:53.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.391+0000 7f5e41ffb700 1 -- 172.21.15.146:0/3803141503 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v43) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5e4c0aebc0 con 0x7f5e4c0abcb0 2024-09-13T18:56:53.403 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:56:53.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.395+0000 7f5e527be700 1 -- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e380680b0 msgr2=0x7f5e3806a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:53.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.395+0000 7f5e527be700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e380680b0 0x7f5e3806a570 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5e48006fd0 tx=0x7f5e48009190 comp rx=0 tx=0).stop 2024-09-13T18:56:53.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.399+0000 7f5e527be700 1 -- 172.21.15.146:0/3803141503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e4c0abcb0 msgr2=0x7f5e4c15b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:53.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.399+0000 7f5e527be700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e4c0abcb0 0x7f5e4c15b3b0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5e44037000 tx=0x7f5e44001cb0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.399+0000 7f5e527be700 1 -- 172.21.15.146:0/3803141503 shutdown_connections 2024-09-13T18:56:53.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.399+0000 7f5e527be700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e380680b0 0x7f5e3806a570 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.399+0000 7f5e527be700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e4c0abcb0 0x7f5e4c15b3b0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.399+0000 7f5e527be700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e4c15d640 0x7f5e4c0c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.399+0000 7f5e527be700 1 --2- 172.21.15.146:0/3803141503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e38005350 0x7f5e4c0af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.399+0000 7f5e527be700 1 -- 172.21.15.146:0/3803141503 >> 172.21.15.146:0/3803141503 conn(0x7f5e4c066090 msgr2=0x7f5e4c0c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:53.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.399+0000 7f5e527be700 1 -- 172.21.15.146:0/3803141503 shutdown_connections 2024-09-13T18:56:53.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.399+0000 7f5e527be700 1 -- 172.21.15.146:0/3803141503 wait complete. 2024-09-13T18:56:53.434 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:56:53.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.695+0000 7fba26d9d700 1 Processor -- start 2024-09-13T18:56:53.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.695+0000 7fba26d9d700 1 -- start start 2024-09-13T18:56:53.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba26d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fba2015b380 0x7fba20157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba26d9d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba20158030 0x7fba201544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba26d9d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba200a97e0 con 0x7fba200cab20 2024-09-13T18:56:53.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba26d9d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba200c99a0 con 0x7fba2015b380 2024-09-13T18:56:53.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba26d9d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba200723d0 con 0x7fba20158030 2024-09-13T18:56:53.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba1ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fba2015b380 0x7fba20157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:53.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba1ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fba2015b380 0x7fba20157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35446/0 (socket says 172.21.15.146:35446) 2024-09-13T18:56:53.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba1ffff700 1 -- 172.21.15.146:0/1312727407 learned_addr learned my addr 172.21.15.146:0/1312727407 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:53.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba1ffff700 1 -- 172.21.15.146:0/1312727407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba20158030 msgr2=0x7fba201544a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:56:53.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba1ffff700 1 --2- 172.21.15.146:0/1312727407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba20158030 0x7fba201544a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba1ffff700 1 -- 172.21.15.146:0/1312727407 >> v1:172.21.15.154:6789/0 conn(0x7fba200cab20 legacy=0x7fba200caf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:56:53.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba1ffff700 1 -- 172.21.15.146:0/1312727407 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba200c9370 con 0x7fba2015b380 2024-09-13T18:56:53.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba1ffff700 1 --2- 172.21.15.146:0/1312727407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba2015b380 0x7fba20157af0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fba1400ffd0 tx=0x7fba14014850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be77c72a59940ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:53.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba1f7fe700 1 -- 172.21.15.146:0/1312727407 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14004030 con 0x7fba2015b380 2024-09-13T18:56:53.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba1f7fe700 1 -- 172.21.15.146:0/1312727407 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba14004190 con 0x7fba2015b380 2024-09-13T18:56:53.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba1f7fe700 1 -- 172.21.15.146:0/1312727407 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14004520 con 0x7fba2015b380 2024-09-13T18:56:53.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba26d9d700 1 -- 172.21.15.146:0/1312727407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba2015b380 msgr2=0x7fba20157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:53.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.699+0000 7fba26d9d700 1 --2- 172.21.15.146:0/1312727407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba2015b380 0x7fba20157af0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fba1400ffd0 tx=0x7fba14014850 comp rx=0 tx=0).stop 2024-09-13T18:56:53.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.707+0000 7fba26d9d700 1 -- 172.21.15.146:0/1312727407 shutdown_connections 2024-09-13T18:56:53.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.707+0000 7fba26d9d700 1 --2- 172.21.15.146:0/1312727407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba20158030 0x7fba201544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.707+0000 7fba26d9d700 1 --2- 172.21.15.146:0/1312727407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba2015b380 0x7fba20157af0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.707+0000 7fba26d9d700 1 -- 172.21.15.146:0/1312727407 >> 172.21.15.146:0/1312727407 conn(0x7fba20066090 msgr2=0x7fba20066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:53.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.707+0000 7fba26d9d700 1 -- 172.21.15.146:0/1312727407 shutdown_connections 2024-09-13T18:56:53.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.711+0000 7fba26d9d700 1 -- 172.21.15.146:0/1312727407 wait complete. 2024-09-13T18:56:53.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.711+0000 7fba26d9d700 1 Processor -- start 2024-09-13T18:56:53.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.711+0000 7fba26d9d700 1 -- start start 2024-09-13T18:56:53.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba26d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba200cab20 0x7fba201527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba26d9d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba20158030 0x7fba201e1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba26d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba2015b380 0x7fba200c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba26d9d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200c14e0 con 0x7fba200cab20 2024-09-13T18:56:53.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba26d9d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2008b810 con 0x7fba2015b380 2024-09-13T18:56:53.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba26d9d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba200bfcf0 con 0x7fba20158030 2024-09-13T18:56:53.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba1ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba20158030 0x7fba201e1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:53.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba1ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba20158030 0x7fba201e1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41408/0 (socket says 172.21.15.146:41408) 2024-09-13T18:56:53.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba1ffff700 1 -- 172.21.15.146:0/2227146309 learned_addr learned my addr 172.21.15.146:0/2227146309 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:53.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba24b39700 1 --2- 172.21.15.146:0/2227146309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba200cab20 0x7fba201527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:53.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba1ffff700 1 -- 172.21.15.146:0/2227146309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba2015b380 msgr2=0x7fba200c04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:53.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba1ffff700 1 --2- 172.21.15.146:0/2227146309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba2015b380 0x7fba200c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba1ffff700 1 -- 172.21.15.146:0/2227146309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba200cab20 msgr2=0x7fba201527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:53.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba1ffff700 1 --2- 172.21.15.146:0/2227146309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba200cab20 0x7fba201527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:53.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba1ffff700 1 -- 172.21.15.146:0/2227146309 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1400fdb0 con 0x7fba20158030 2024-09-13T18:56:53.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba1ffff700 1 --2- 172.21.15.146:0/2227146309 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba20158030 0x7fba201e1310 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fba14000c00 tx=0x7fba14004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:53.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba1d7fa700 1 -- 172.21.15.146:0/2227146309 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1400f750 con 0x7fba20158030 2024-09-13T18:56:53.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba1d7fa700 1 -- 172.21.15.146:0/2227146309 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba14012870 con 0x7fba20158030 2024-09-13T18:56:53.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba1d7fa700 1 -- 172.21.15.146:0/2227146309 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14012c20 con 0x7fba20158030 2024-09-13T18:56:53.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba26d9d700 1 -- 172.21.15.146:0/2227146309 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba200bff70 con 0x7fba20158030 2024-09-13T18:56:53.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.715+0000 7fba26d9d700 1 -- 172.21.15.146:0/2227146309 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2008ba60 con 0x7fba20158030 2024-09-13T18:56:53.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.719+0000 7fba1d7fa700 1 -- 172.21.15.146:0/2227146309 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fba14027430 con 0x7fba20158030 2024-09-13T18:56:53.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.719+0000 7fba1d7fa700 1 --2- 172.21.15.146:0/2227146309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fba0c066760 0x7fba0c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:53.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.719+0000 7fba1d7fa700 1 -- 172.21.15.146:0/2227146309 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fba1408f4a0 con 0x7fba20158030 2024-09-13T18:56:53.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.719+0000 7fba24b39700 1 --2- 172.21.15.146:0/2227146309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fba0c066760 0x7fba0c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:53.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.719+0000 7fba26d9d700 1 -- 172.21.15.146:0/2227146309 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba201559b0 con 0x7fba20158030 2024-09-13T18:56:53.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.723+0000 7fba24b39700 1 --2- 172.21.15.146:0/2227146309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fba0c066760 0x7fba0c068c20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fba18009f50 tx=0x7fba18009540 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:53.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:53.731+0000 7fba1d7fa700 1 -- 172.21.15.146:0/2227146309 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fba14035080 con 0x7fba20158030 2024-09-13T18:56:54.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.059+0000 7fba26d9d700 1 -- 172.21.15.146:0/2227146309 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fba200c8db0 con 0x7fba20158030 2024-09-13T18:56:54.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.059+0000 7fba1d7fa700 1 -- 172.21.15.146:0/2227146309 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 32 entries v43) v1 ==== 81+0+1974 (secure 0 0 0) 0x7fba1405bf80 con 0x7fba20158030 2024-09-13T18:56:54.084 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6837/2053734262 2024-09-14T18:55:25.944088+0000 2024-09-13T18:56:54.084 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6839/3274220802 2024-09-14T18:36:10.677826+0000 2024-09-13T18:56:54.085 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6835/294014284 2024-09-14T18:50:50.876658+0000 2024-09-13T18:56:54.085 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6837/3274220802 2024-09-14T18:36:10.677826+0000 2024-09-13T18:56:54.085 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6836/2169954140 2024-09-14T18:53:50.924269+0000 2024-09-13T18:56:54.085 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6832/1498985282 2024-09-14T18:34:25.647701+0000 2024-09-13T18:56:54.085 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6835/1642500214 2024-09-14T18:30:30.598692+0000 2024-09-13T18:56:54.085 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6836/31041557 2024-09-14T18:32:55.627334+0000 2024-09-13T18:56:54.085 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6832/3023779168 2024-09-14T18:52:35.902095+0000 2024-09-13T18:56:54.085 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6834/1642500214 2024-09-14T18:30:30.598692+0000 2024-09-13T18:56:54.085 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6836/3234109877 2024-09-14T18:45:55.807881+0000 2024-09-13T18:56:54.085 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6837/2169954140 2024-09-14T18:53:50.924269+0000 2024-09-13T18:56:54.085 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6833/1498985282 2024-09-14T18:34:25.647701+0000 2024-09-13T18:56:54.085 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6837/31041557 2024-09-14T18:32:55.627334+0000 2024-09-13T18:56:54.085 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6839/776638146 2024-09-14T18:44:35.790651+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6839/3978828499 2024-09-14T18:29:25.578693+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6832/2835746745 2024-09-14T18:28:15.564530+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6834/1652676664 2024-09-14T18:38:00.699180+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6837/776638146 2024-09-14T18:44:35.790651+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6837/925899756 2024-09-14T18:48:20.844616+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6833/2835746745 2024-09-14T18:28:15.564530+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6835/1652676664 2024-09-14T18:38:00.699180+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6839/2053734262 2024-09-14T18:55:25.944088+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6839/925899756 2024-09-14T18:48:20.844616+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6837/3978828499 2024-09-14T18:29:25.578693+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6836/3926931671 2024-09-14T18:40:45.735995+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6837/3926931671 2024-09-14T18:40:45.735995+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6833/3901685195 2024-09-14T18:43:15.768366+0000 2024-09-13T18:56:54.086 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6837/3234109877 2024-09-14T18:45:55.807881+0000 2024-09-13T18:56:54.087 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6834/294014284 2024-09-14T18:50:50.876658+0000 2024-09-13T18:56:54.087 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6832/3901685195 2024-09-14T18:43:15.768366+0000 2024-09-13T18:56:54.087 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6833/3023779168 2024-09-14T18:52:35.902095+0000 2024-09-13T18:56:54.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.083+0000 7fba26d9d700 1 -- 172.21.15.146:0/2227146309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fba0c066760 msgr2=0x7fba0c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:54.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.083+0000 7fba26d9d700 1 --2- 172.21.15.146:0/2227146309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fba0c066760 0x7fba0c068c20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fba18009f50 tx=0x7fba18009540 comp rx=0 tx=0).stop 2024-09-13T18:56:54.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.083+0000 7fba26d9d700 1 -- 172.21.15.146:0/2227146309 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba20158030 msgr2=0x7fba201e1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:54.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.083+0000 7fba26d9d700 1 --2- 172.21.15.146:0/2227146309 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba20158030 0x7fba201e1310 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fba14000c00 tx=0x7fba14004770 comp rx=0 tx=0).stop 2024-09-13T18:56:54.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.083+0000 7fba26d9d700 1 -- 172.21.15.146:0/2227146309 shutdown_connections 2024-09-13T18:56:54.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.083+0000 7fba26d9d700 1 --2- 172.21.15.146:0/2227146309 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fba0c066760 0x7fba0c068c20 secure :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fba18009f50 tx=0x7fba18009540 comp rx=0 tx=0).stop 2024-09-13T18:56:54.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.083+0000 7fba26d9d700 1 --2- 172.21.15.146:0/2227146309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba2015b380 0x7fba200c04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:54.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.083+0000 7fba26d9d700 1 --2- 172.21.15.146:0/2227146309 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba20158030 0x7fba201e1310 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:54.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.083+0000 7fba26d9d700 1 --2- 172.21.15.146:0/2227146309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba200cab20 0x7fba201527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:54.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.083+0000 7fba26d9d700 1 -- 172.21.15.146:0/2227146309 >> 172.21.15.146:0/2227146309 conn(0x7fba20066090 msgr2=0x7fba20066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:54.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.083+0000 7fba26d9d700 1 -- 172.21.15.146:0/2227146309 shutdown_connections 2024-09-13T18:56:54.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.083+0000 7fba26d9d700 1 -- 172.21.15.146:0/2227146309 wait complete. 2024-09-13T18:56:54.101 INFO:teuthology.orchestra.run.smithi146.stderr:listed 32 entries 2024-09-13T18:56:54.125 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-13T18:56:54.126 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:56:54.145 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:56:54.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ed21c8700 1 Processor -- start 2024-09-13T18:56:54.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ed21c8700 1 -- start start 2024-09-13T18:56:54.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ed21c8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8ecc150ad0 0x7f8ecc0cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:54.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ed21c8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ecc0adca0 0x7f8ecc0c01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:54.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ed21c8700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ecc06ec90 con 0x7f8ecc1506f0 2024-09-13T18:56:54.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ed21c8700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ecc0c0730 con 0x7f8ecc150ad0 2024-09-13T18:56:54.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ed21c8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0c1da0 con 0x7f8ecc0adca0 2024-09-13T18:56:54.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ec3fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8ecc150ad0 0x7f8ecc0cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:54.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ecb7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f8ecc1506f0 0x7f8ecc06e580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48944/0 (socket says 172.21.15.146:48944) 2024-09-13T18:56:54.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ecb7fe700 1 -- 172.21.15.146:0/2804546350 learned_addr learned my addr 172.21.15.146:0/2804546350 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:56:54.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ecbfff700 1 --2- 172.21.15.146:0/2804546350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ecc0adca0 0x7f8ecc0c01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:54.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ec3fff700 1 -- 172.21.15.146:0/2804546350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ecc0adca0 msgr2=0x7f8ecc0c01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:56:54.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ec3fff700 1 --2- 172.21.15.146:0/2804546350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ecc0adca0 0x7f8ecc0c01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:54.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ec3fff700 1 -- 172.21.15.146:0/2804546350 >> v1:172.21.15.146:6789/0 conn(0x7f8ecc1506f0 legacy=0x7f8ecc06e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:54.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ecaffd700 1 -- 172.21.15.146:0/2804546350 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186026728 0 0) 0x7f8ebc001110 con 0x7f8ecc1506f0 2024-09-13T18:56:54.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ec3fff700 1 -- 172.21.15.146:0/2804546350 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc06d040 con 0x7f8ecc150ad0 2024-09-13T18:56:54.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ec3fff700 1 --2- 172.21.15.146:0/2804546350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ecc150ad0 0x7f8ecc0cbcf0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009850 tx=0x7f8eb4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9b6d204dc58be4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:54.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ecaffd700 1 -- 172.21.15.146:0/2804546350 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4011830 con 0x7f8ecc150ad0 2024-09-13T18:56:54.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ecaffd700 1 -- 172.21.15.146:0/2804546350 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb4011990 con 0x7f8ecc150ad0 2024-09-13T18:56:54.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ecaffd700 1 -- 172.21.15.146:0/2804546350 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4011d40 con 0x7f8ecc150ad0 2024-09-13T18:56:54.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2804546350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ecc150ad0 msgr2=0x7f8ecc0cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:54.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.411+0000 7f8ed21c8700 1 --2- 172.21.15.146:0/2804546350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ecc150ad0 0x7f8ecc0cbcf0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009850 tx=0x7f8eb4009b60 comp rx=0 tx=0).stop 2024-09-13T18:56:54.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2804546350 shutdown_connections 2024-09-13T18:56:54.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 --2- 172.21.15.146:0/2804546350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ecc0adca0 0x7f8ecc0c01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:54.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 --2- 172.21.15.146:0/2804546350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ecc150ad0 0x7f8ecc0cbcf0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:54.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2804546350 >> 172.21.15.146:0/2804546350 conn(0x7f8ecc0cb1d0 msgr2=0x7f8ecc0cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:54.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2804546350 shutdown_connections 2024-09-13T18:56:54.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2804546350 wait complete. 2024-09-13T18:56:54.444 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 Processor -- start 2024-09-13T18:56:54.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 -- start start 2024-09-13T18:56:54.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ecc0adca0 0x7f8ecc1edfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:54.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ecc1506f0 0x7f8ecc0c0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:54.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ecc150ad0 0x7f8ecc0c3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:54.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc1421b0 con 0x7f8ecc1506f0 2024-09-13T18:56:54.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc1ee4f0 con 0x7f8ecc0adca0 2024-09-13T18:56:54.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ed21c8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc1eec80 con 0x7f8ecc150ad0 2024-09-13T18:56:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ec3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ecc1506f0 0x7f8ecc0c0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ec3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ecc1506f0 0x7f8ecc0c0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35488/0 (socket says 172.21.15.146:35488) 2024-09-13T18:56:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ec3fff700 1 -- 172.21.15.146:0/2915878750 learned_addr learned my addr 172.21.15.146:0/2915878750 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ecbfff700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ecc150ad0 0x7f8ecc0c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ecb7fe700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ecc0adca0 0x7f8ecc1edfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ec3fff700 1 -- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ecc150ad0 msgr2=0x7f8ecc0c3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ec3fff700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ecc150ad0 0x7f8ecc0c3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ec3fff700 1 -- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ecc0adca0 msgr2=0x7f8ecc1edfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ec3fff700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ecc0adca0 0x7f8ecc1edfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:54.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ec3fff700 1 -- 172.21.15.146:0/2915878750 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb40094b0 con 0x7f8ecc1506f0 2024-09-13T18:56:54.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.415+0000 7f8ec3fff700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ecc1506f0 0x7f8ecc0c0d70 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4000c00 tx=0x7f8eb4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:54.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.419+0000 7f8ec8ff9700 1 -- 172.21.15.146:0/2915878750 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4019980 con 0x7f8ecc1506f0 2024-09-13T18:56:54.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.419+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2915878750 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc142390 con 0x7f8ecc1506f0 2024-09-13T18:56:54.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.419+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2915878750 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc1ed9f0 con 0x7f8ecc1506f0 2024-09-13T18:56:54.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.419+0000 7f8ec8ff9700 1 -- 172.21.15.146:0/2915878750 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb401e070 con 0x7f8ecc1506f0 2024-09-13T18:56:54.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.419+0000 7f8ec8ff9700 1 -- 172.21.15.146:0/2915878750 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4019d20 con 0x7f8ecc1506f0 2024-09-13T18:56:54.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.419+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2915878750 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb0005310 con 0x7f8ecc1506f0 2024-09-13T18:56:54.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.419+0000 7f8ec8ff9700 1 -- 172.21.15.146:0/2915878750 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8eb401a420 con 0x7f8ecc1506f0 2024-09-13T18:56:54.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.419+0000 7f8ec8ff9700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8eac0667b0 0x7f8eac068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:54.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.419+0000 7f8ec8ff9700 1 -- 172.21.15.146:0/2915878750 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f8eb40858c0 con 0x7f8ecc1506f0 2024-09-13T18:56:54.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.419+0000 7f8ecb7fe700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8eac0667b0 0x7f8eac068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:54.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.419+0000 7f8ecb7fe700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8eac0667b0 0x7f8eac068c70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc000f60 tx=0x7f8ebc007c40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:54.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.431+0000 7f8ec8ff9700 1 -- 172.21.15.146:0/2915878750 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8eb40524c0 con 0x7f8ecc1506f0 2024-09-13T18:56:54.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.823+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2915878750 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8eb0005890 con 0x7f8ecc1506f0 2024-09-13T18:56:54.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.827+0000 7f8ec8ff9700 1 -- 172.21.15.146:0/2915878750 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+20825 (secure 0 0 0) 0x7f8eb4015020 con 0x7f8ecc1506f0 2024-09-13T18:56:54.845 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:56:54.845 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":43,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:55:25.944150+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.13","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/2053734262":"2024-09-14T18:55:25.944088+0000","172.21.15.154:6839/3274220802":"2024-09-14T18:36:10.677826+0000","172.21.15.146:6835/294014284":"2024-09-14T18:50:50.876658+0000","172.21.15.154:6837/3274220802":"2024-09-14T18:36:10.677826+0000","172.21.15.146:6836/2169954140":"2024-09-14T18:53:50.924269+0000","172.21.15.146:6832/1498985282":"2024-09-14T18:34:25.647701+0000","172.21.15.146:6835/1642500214":"2024-09-14T18:30:30.598692+0000","172.21.15.146:6836/31041557":"2024-09-14T18:32:55.627334+0000","172.21.15.146:6832/3023779168":"2024-09-14T18:52:35.902095+0000","172.21.15.146:6834/1642500214":"2024-09-14T18:30:30.598692+0000","172.21.15.146:6836/3234109877":"2024-09-14T18:45:55.807881+0000","172.21.15.146:6837/2169954140":"2024-09-14T18:53:50.924269+0000","172.21.15.146:6833/1498985282":"2024-09-14T18:34:25.647701+0000","172.21.15.146:6837/31041557":"2024-09-14T18:32:55.627334+0000","172.21.15.154:6839/776638146":"2024-09-14T18:44:35.790651+0000","172.21.15.154:6839/3978828499":"2024-09-14T18:29:25.578693+0000","172.21.15.146:6832/2835746745":"2024-09-14T18:28:15.564530+0000","172.21.15.146:6834/1652676664":"2024-09-14T18:38:00.699180+0000","172.21.15.154:6837/776638146":"2024-09-14T18:44:35.790651+0000","172.21.15.154:6837/925899756":"2024-09-14T18:48:20.844616+0000","172.21.15.146:6833/2835746745":"2024-09-14T18:28:15.564530+0000","172.21.15.146:6835/1652676664":"2024-09-14T18:38:00.699180+0000","172.21.15.154:6839/2053734262":"2024-09-14T18:55:25.944088+0000","172.21.15.154:6839/925899756":"2024-09-14T18:48:20.844616+0000","172.21.15.154:6837/3978828499":"2024-09-14T18:29:25.578693+0000","172.21.15.146:6836/3926931671":"2024-09-14T18:40:45.735995+0000","172.21.15.146:6837/3926931671":"2024-09-14T18:40:45.735995+0000","172.21.15.146:6833/3901685195":"2024-09-14T18:43:15.768366+0000","172.21.15.146:6837/3234109877":"2024-09-14T18:45:55.807881+0000","172.21.15.146:6834/294014284":"2024-09-14T18:50:50.876658+0000","172.21.15.146:6832/3901685195":"2024-09-14T18:43:15.768366+0000","172.21.15.146:6833/3023779168":"2024-09-14T18:52:35.902095+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:56:54.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.847+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8eac0667b0 msgr2=0x7f8eac068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:54.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.847+0000 7f8ed21c8700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8eac0667b0 0x7f8eac068c70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc000f60 tx=0x7f8ebc007c40 comp rx=0 tx=0).stop 2024-09-13T18:56:54.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.847+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2915878750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ecc1506f0 msgr2=0x7f8ecc0c0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:54.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.847+0000 7f8ed21c8700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ecc1506f0 0x7f8ecc0c0d70 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4000c00 tx=0x7f8eb4004770 comp rx=0 tx=0).stop 2024-09-13T18:56:54.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.847+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2915878750 shutdown_connections 2024-09-13T18:56:54.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.847+0000 7f8ed21c8700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8eac0667b0 0x7f8eac068c70 secure :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc000f60 tx=0x7f8ebc007c40 comp rx=0 tx=0).stop 2024-09-13T18:56:54.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.847+0000 7f8ed21c8700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ecc150ad0 0x7f8ecc0c3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:54.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.847+0000 7f8ed21c8700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ecc1506f0 0x7f8ecc0c0d70 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:54.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.847+0000 7f8ed21c8700 1 --2- 172.21.15.146:0/2915878750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ecc0adca0 0x7f8ecc1edfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:54.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.847+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2915878750 >> 172.21.15.146:0/2915878750 conn(0x7f8ecc0cb1d0 msgr2=0x7f8ecc070460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:54.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.851+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2915878750 shutdown_connections 2024-09-13T18:56:54.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:54.851+0000 7f8ed21c8700 1 -- 172.21.15.146:0/2915878750 wait complete. 2024-09-13T18:56:54.882 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:26:24.867515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:26:30.782269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:26:31.323277+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:26:34.393386+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:56:54.882 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:56:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc592ab4700 1 Processor -- start 2024-09-13T18:56:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc592ab4700 1 -- start start 2024-09-13T18:56:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc592ab4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc58c0caed0 0x7fc58c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc592ab4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc58c06f7d0 0x7fc58c0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc592ab4700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58c0c7d20 con 0x7fc58c0cb2b0 2024-09-13T18:56:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc592ab4700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58c072970 con 0x7fc58c06f7d0 2024-09-13T18:56:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc592ab4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58c0c8ad0 con 0x7fc58c0caed0 2024-09-13T18:56:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc591ab2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc58c0caed0 0x7fc58c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc591ab2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc58c0caed0 0x7fc58c158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:57462/0 (socket says 172.21.15.146:57462) 2024-09-13T18:56:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc591ab2700 1 -- 172.21.15.146:0/1389710159 learned_addr learned my addr 172.21.15.146:0/1389710159 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:55.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc591ab2700 1 -- 172.21.15.146:0/1389710159 >> v2:172.21.15.146:3300/0 conn(0x7fc58c06f7d0 msgr2=0x7fc58c0c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:56:55.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc5922b3700 1 --2- 172.21.15.146:0/1389710159 >> v2:172.21.15.146:3300/0 conn(0x7fc58c06f7d0 0x7fc58c0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc591ab2700 1 --2- 172.21.15.146:0/1389710159 >> v2:172.21.15.146:3300/0 conn(0x7fc58c06f7d0 0x7fc58c0c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc591ab2700 1 -- 172.21.15.146:0/1389710159 >> v1:172.21.15.146:6789/0 conn(0x7fc58c0cb2b0 legacy=0x7fc58c072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:56:55.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc591ab2700 1 -- 172.21.15.146:0/1389710159 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c0c8d90 con 0x7fc58c0caed0 2024-09-13T18:56:55.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc5922b3700 1 --2- 172.21.15.146:0/1389710159 >> v2:172.21.15.146:3300/0 conn(0x7fc58c06f7d0 0x7fc58c0c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:56:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.043+0000 7fc591ab2700 1 --2- 172.21.15.146:0/1389710159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc58c0caed0 0x7fc58c158110 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0c8870 tx=0x7fc580009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16ce824ec46070e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.047+0000 7fc590ab0700 1 -- 172.21.15.146:0/1389710159 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc580004230 con 0x7fc58c0caed0 2024-09-13T18:56:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.047+0000 7fc590ab0700 1 -- 172.21.15.146:0/1389710159 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc580004390 con 0x7fc58c0caed0 2024-09-13T18:56:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.047+0000 7fc590ab0700 1 -- 172.21.15.146:0/1389710159 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5800036c0 con 0x7fc58c0caed0 2024-09-13T18:56:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.047+0000 7fc592ab4700 1 -- 172.21.15.146:0/1389710159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc58c0caed0 msgr2=0x7fc58c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.047+0000 7fc592ab4700 1 --2- 172.21.15.146:0/1389710159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc58c0caed0 0x7fc58c158110 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0c8870 tx=0x7fc580009bf0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.047+0000 7fc592ab4700 1 -- 172.21.15.146:0/1389710159 shutdown_connections 2024-09-13T18:56:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.047+0000 7fc592ab4700 1 --2- 172.21.15.146:0/1389710159 >> v2:172.21.15.146:3300/0 conn(0x7fc58c06f7d0 0x7fc58c0c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.047+0000 7fc592ab4700 1 --2- 172.21.15.146:0/1389710159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc58c0caed0 0x7fc58c158110 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.047+0000 7fc592ab4700 1 -- 172.21.15.146:0/1389710159 >> 172.21.15.146:0/1389710159 conn(0x7fc58c066090 msgr2=0x7fc58c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.047+0000 7fc592ab4700 1 -- 172.21.15.146:0/1389710159 shutdown_connections 2024-09-13T18:56:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.047+0000 7fc592ab4700 1 -- 172.21.15.146:0/1389710159 wait complete. 2024-09-13T18:56:55.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.047+0000 7fc592ab4700 1 Processor -- start 2024-09-13T18:56:55.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.047+0000 7fc592ab4700 1 -- start start 2024-09-13T18:56:55.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc592ab4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc58c06f7d0 0x7fc58c0af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc592ab4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc58c0caed0 0x7fc58c0afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc592ab4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc58c0cb2b0 0x7fc58c0c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc592ab4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c1ed730 con 0x7fc58c0caed0 2024-09-13T18:56:55.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc592ab4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c0ab3c0 con 0x7fc58c0cb2b0 2024-09-13T18:56:55.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc592ab4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c0c6780 con 0x7fc58c06f7d0 2024-09-13T18:56:55.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc591ab2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc58c06f7d0 0x7fc58c0af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc591ab2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc58c06f7d0 0x7fc58c0af850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41428/0 (socket says 172.21.15.146:41428) 2024-09-13T18:56:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc591ab2700 1 -- 172.21.15.146:0/1917386431 learned_addr learned my addr 172.21.15.146:0/1917386431 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc5922b3700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc58c0cb2b0 0x7fc58c0c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc5912b1700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc58c0caed0 0x7fc58c0afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc591ab2700 1 -- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc58c0cb2b0 msgr2=0x7fc58c0c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc591ab2700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc58c0cb2b0 0x7fc58c0c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc591ab2700 1 -- 172.21.15.146:0/1917386431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc58c0caed0 msgr2=0x7fc58c0afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc591ab2700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc58c0caed0 0x7fc58c0afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc591ab2700 1 -- 172.21.15.146:0/1917386431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc580009570 con 0x7fc58c06f7d0 2024-09-13T18:56:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc5912b1700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc58c0caed0 0x7fc58c0afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:56:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc591ab2700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc58c06f7d0 0x7fc58c0af850 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0c8c10 tx=0x7fc580004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc57e7fc700 1 -- 172.21.15.146:0/1917386431 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58000b660 con 0x7fc58c06f7d0 2024-09-13T18:56:55.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc57e7fc700 1 -- 172.21.15.146:0/1917386431 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58000b7e0 con 0x7fc58c06f7d0 2024-09-13T18:56:55.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc592ab4700 1 -- 172.21.15.146:0/1917386431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58c1ed920 con 0x7fc58c06f7d0 2024-09-13T18:56:55.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc57e7fc700 1 -- 172.21.15.146:0/1917386431 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58000bb90 con 0x7fc58c06f7d0 2024-09-13T18:56:55.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc592ab4700 1 -- 172.21.15.146:0/1917386431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c0aad70 con 0x7fc58c06f7d0 2024-09-13T18:56:55.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.051+0000 7fc592ab4700 1 -- 172.21.15.146:0/1917386431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58c06eea0 con 0x7fc58c06f7d0 2024-09-13T18:56:55.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.055+0000 7fc57e7fc700 1 -- 172.21.15.146:0/1917386431 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc580022070 con 0x7fc58c06f7d0 2024-09-13T18:56:55.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.055+0000 7fc57e7fc700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc570066760 0x7fc570068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.055+0000 7fc57e7fc700 1 -- 172.21.15.146:0/1917386431 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fc5800866a0 con 0x7fc58c06f7d0 2024-09-13T18:56:55.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.055+0000 7fc5912b1700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc570066760 0x7fc570068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.055+0000 7fc5912b1700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc570066760 0x7fc570068c20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc58c06c9e0 tx=0x7fc5880090e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:55.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.059+0000 7fc57e7fc700 1 -- 172.21.15.146:0/1917386431 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc580053150 con 0x7fc58c06f7d0 2024-09-13T18:56:55.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.287+0000 7fc592ab4700 1 -- 172.21.15.146:0/1917386431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc58c158110 con 0x7fc58c06f7d0 2024-09-13T18:56:55.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.287+0000 7fc57e7fc700 1 -- 172.21.15.146:0/1917386431 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc580056770 con 0x7fc58c06f7d0 2024-09-13T18:56:55.300 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-09-13T18:56:55.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.291+0000 7fc592ab4700 1 -- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc570066760 msgr2=0x7fc570068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:55.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.291+0000 7fc592ab4700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc570066760 0x7fc570068c20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc58c06c9e0 tx=0x7fc5880090e0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.291+0000 7fc592ab4700 1 -- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc58c06f7d0 msgr2=0x7fc58c0af850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:55.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.291+0000 7fc592ab4700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc58c06f7d0 0x7fc58c0af850 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0c8c10 tx=0x7fc580004770 comp rx=0 tx=0).stop 2024-09-13T18:56:55.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.291+0000 7fc592ab4700 1 -- 172.21.15.146:0/1917386431 shutdown_connections 2024-09-13T18:56:55.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.291+0000 7fc592ab4700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc570066760 0x7fc570068c20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.291+0000 7fc592ab4700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc58c0cb2b0 0x7fc58c0c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.291+0000 7fc592ab4700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc58c0caed0 0x7fc58c0afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.291+0000 7fc592ab4700 1 --2- 172.21.15.146:0/1917386431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc58c06f7d0 0x7fc58c0af850 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.291+0000 7fc592ab4700 1 -- 172.21.15.146:0/1917386431 >> 172.21.15.146:0/1917386431 conn(0x7fc58c066090 msgr2=0x7fc58c1568c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:55.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.291+0000 7fc592ab4700 1 -- 172.21.15.146:0/1917386431 shutdown_connections 2024-09-13T18:56:55.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.291+0000 7fc592ab4700 1 -- 172.21.15.146:0/1917386431 wait complete. 2024-09-13T18:56:55.316 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:56:55.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.423+0000 7fa689825700 1 Processor -- start 2024-09-13T18:56:55.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.423+0000 7fa689825700 1 -- start start 2024-09-13T18:56:55.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa689825700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6840c9350 0x7fa68415a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa689825700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6840a9490 0x7fa6840c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa689825700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6840c8650 con 0x7fa6840c8f70 2024-09-13T18:56:55.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa689825700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6840aa1a0 con 0x7fa6840c9350 2024-09-13T18:56:55.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa689825700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6840aa480 con 0x7fa6840a9490 2024-09-13T18:56:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa689024700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6840a9490 0x7fa6840c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa689024700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6840a9490 0x7fa6840c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41430/0 (socket says 172.21.15.146:41430) 2024-09-13T18:56:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa689024700 1 -- 172.21.15.146:0/1426133648 learned_addr learned my addr 172.21.15.146:0/1426133648 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa683fff700 1 --2- 172.21.15.146:0/1426133648 >> v2:172.21.15.154:3300/0 conn(0x7fa6840c9350 0x7fa68415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa683fff700 1 -- 172.21.15.146:0/1426133648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6840a9490 msgr2=0x7fa6840c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:56:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa683fff700 1 --2- 172.21.15.146:0/1426133648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6840a9490 0x7fa6840c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa683fff700 1 -- 172.21.15.146:0/1426133648 >> v1:172.21.15.146:6789/0 conn(0x7fa6840c8f70 legacy=0x7fa6840cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:56:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa683fff700 1 -- 172.21.15.146:0/1426133648 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa684159890 con 0x7fa6840c9350 2024-09-13T18:56:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa689024700 1 --2- 172.21.15.146:0/1426133648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6840a9490 0x7fa6840c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:56:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa683fff700 1 --2- 172.21.15.146:0/1426133648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6840c9350 0x7fa68415a370 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa674009850 tx=0x7fa674009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94b5f5e3196ea0b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa6837fe700 1 -- 172.21.15.146:0/1426133648 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674004230 con 0x7fa6840c9350 2024-09-13T18:56:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa6837fe700 1 -- 172.21.15.146:0/1426133648 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa674004390 con 0x7fa6840c9350 2024-09-13T18:56:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa6837fe700 1 -- 172.21.15.146:0/1426133648 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6740036c0 con 0x7fa6840c9350 2024-09-13T18:56:55.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa689825700 1 -- 172.21.15.146:0/1426133648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6840c9350 msgr2=0x7fa68415a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:55.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.427+0000 7fa689825700 1 --2- 172.21.15.146:0/1426133648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6840c9350 0x7fa68415a370 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa674009850 tx=0x7fa674009b60 comp rx=0 tx=0).stop 2024-09-13T18:56:55.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.439+0000 7fa689825700 1 -- 172.21.15.146:0/1426133648 shutdown_connections 2024-09-13T18:56:55.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.439+0000 7fa689825700 1 --2- 172.21.15.146:0/1426133648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6840a9490 0x7fa6840c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.439+0000 7fa689825700 1 --2- 172.21.15.146:0/1426133648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6840c9350 0x7fa68415a370 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.439+0000 7fa689825700 1 -- 172.21.15.146:0/1426133648 >> 172.21.15.146:0/1426133648 conn(0x7fa684066090 msgr2=0x7fa684066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:55.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.439+0000 7fa689825700 1 -- 172.21.15.146:0/1426133648 shutdown_connections 2024-09-13T18:56:55.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.439+0000 7fa689825700 1 -- 172.21.15.146:0/1426133648 wait complete. 2024-09-13T18:56:55.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.439+0000 7fa689825700 1 Processor -- start 2024-09-13T18:56:55.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.439+0000 7fa689825700 1 -- start start 2024-09-13T18:56:55.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689825700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6840a9490 0x7fa6840c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689825700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6840c8f70 0x7fa6841f0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689825700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6840c9350 0x7fa684158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689825700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68415ae10 con 0x7fa6840c9350 2024-09-13T18:56:55.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689825700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68415d400 con 0x7fa6840c8f70 2024-09-13T18:56:55.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689825700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68415b020 con 0x7fa6840a9490 2024-09-13T18:56:55.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689024700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6840c9350 0x7fa684158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689024700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6840c9350 0x7fa684158220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35522/0 (socket says 172.21.15.146:35522) 2024-09-13T18:56:55.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689024700 1 -- 172.21.15.146:0/73368534 learned_addr learned my addr 172.21.15.146:0/73368534 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:55.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa683fff700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6840c8f70 0x7fa6841f0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa688823700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6840a9490 0x7fa6840c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689024700 1 -- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6840a9490 msgr2=0x7fa6840c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:55.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689024700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6840a9490 0x7fa6840c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689024700 1 -- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6840c8f70 msgr2=0x7fa6841f0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:55.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689024700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6840c8f70 0x7fa6841f0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689024700 1 -- 172.21.15.146:0/73368534 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c009b30 con 0x7fa6840c9350 2024-09-13T18:56:55.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689024700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6840c9350 0x7fa684158220 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00efc0 tx=0x7fa67c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:55.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa6817fa700 1 -- 172.21.15.146:0/73368534 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c018070 con 0x7fa6840c9350 2024-09-13T18:56:55.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa6817fa700 1 -- 172.21.15.146:0/73368534 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67c00f040 con 0x7fa6840c9350 2024-09-13T18:56:55.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa6817fa700 1 -- 172.21.15.146:0/73368534 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c003c40 con 0x7fa6840c9350 2024-09-13T18:56:55.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689825700 1 -- 172.21.15.146:0/73368534 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6740094b0 con 0x7fa6840c9350 2024-09-13T18:56:55.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689825700 1 -- 172.21.15.146:0/73368534 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68415b5e0 con 0x7fa6840c9350 2024-09-13T18:56:55.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa689825700 1 -- 172.21.15.146:0/73368534 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6840c9da0 con 0x7fa6840c9350 2024-09-13T18:56:55.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa6817fa700 1 -- 172.21.15.146:0/73368534 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa67c00c880 con 0x7fa6840c9350 2024-09-13T18:56:55.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.443+0000 7fa6817fa700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa668066730 0x7fa668068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.447+0000 7fa6817fa700 1 -- 172.21.15.146:0/73368534 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fa67c084b70 con 0x7fa6840c9350 2024-09-13T18:56:55.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.447+0000 7fa688823700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa668066730 0x7fa668068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.447+0000 7fa688823700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa668066730 0x7fa668068bf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa678007fd0 tx=0x7fa678009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:55.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.451+0000 7fa6817fa700 1 -- 172.21.15.146:0/73368534 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa67c051650 con 0x7fa6840c9350 2024-09-13T18:56:55.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.671+0000 7fa689825700 1 -- 172.21.15.146:0/73368534 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6840a9d60 con 0x7fa6840c9350 2024-09-13T18:56:55.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.671+0000 7fa6817fa700 1 -- 172.21.15.146:0/73368534 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa67c054c70 con 0x7fa6840c9350 2024-09-13T18:56:55.684 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:56:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.675+0000 7fa689825700 1 -- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa668066730 msgr2=0x7fa668068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.675+0000 7fa689825700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa668066730 0x7fa668068bf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa678007fd0 tx=0x7fa678009040 comp rx=0 tx=0).stop 2024-09-13T18:56:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.675+0000 7fa689825700 1 -- 172.21.15.146:0/73368534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6840c9350 msgr2=0x7fa684158220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.675+0000 7fa689825700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6840c9350 0x7fa684158220 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00efc0 tx=0x7fa67c00c3a0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.675+0000 7fa689825700 1 -- 172.21.15.146:0/73368534 shutdown_connections 2024-09-13T18:56:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.675+0000 7fa689825700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa668066730 0x7fa668068bf0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.675+0000 7fa689825700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6840c9350 0x7fa684158220 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.675+0000 7fa689825700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6840c8f70 0x7fa6841f0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.675+0000 7fa689825700 1 --2- 172.21.15.146:0/73368534 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6840a9490 0x7fa6840c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.675+0000 7fa689825700 1 -- 172.21.15.146:0/73368534 >> 172.21.15.146:0/73368534 conn(0x7fa684066090 msgr2=0x7fa68406c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.675+0000 7fa689825700 1 -- 172.21.15.146:0/73368534 shutdown_connections 2024-09-13T18:56:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.675+0000 7fa689825700 1 -- 172.21.15.146:0/73368534 wait complete. 2024-09-13T18:56:55.703 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:56:55.820 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff02bc1700 1 Processor -- start 2024-09-13T18:56:55.820 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff02bc1700 1 -- start start 2024-09-13T18:56:55.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff02bc1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efefc0c11f0 0x7efefc0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff02bc1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efefc0c15d0 0x7efefc159a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff02bc1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efefc0c2e40 0x7efefc159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff02bc1700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efefc159630 con 0x7efefc0c15d0 2024-09-13T18:56:55.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff02bc1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efefc06e480 con 0x7efefc0c2e40 2024-09-13T18:56:55.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff02bc1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efefc06e720 con 0x7efefc0c11f0 2024-09-13T18:56:55.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff01bbf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efefc0c11f0 0x7efefc0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff01bbf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efefc0c11f0 0x7efefc0c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:41450/0 (socket says 172.21.15.146:41450) 2024-09-13T18:56:55.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff01bbf700 1 -- 172.21.15.146:0/1267939906 learned_addr learned my addr 172.21.15.146:0/1267939906 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:55.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff023c0700 1 --2- 172.21.15.146:0/1267939906 >> v2:172.21.15.154:3300/0 conn(0x7efefc0c2e40 0x7efefc159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff01bbf700 1 -- 172.21.15.146:0/1267939906 >> v2:172.21.15.154:3300/0 conn(0x7efefc0c2e40 msgr2=0x7efefc159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:56:55.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff01bbf700 1 --2- 172.21.15.146:0/1267939906 >> v2:172.21.15.154:3300/0 conn(0x7efefc0c2e40 0x7efefc159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff01bbf700 1 -- 172.21.15.146:0/1267939906 >> v2:172.21.15.146:3300/0 conn(0x7efefc0c15d0 msgr2=0x7efefc159a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:56:55.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff01bbf700 1 --2- 172.21.15.146:0/1267939906 >> v2:172.21.15.146:3300/0 conn(0x7efefc0c15d0 0x7efefc159a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff01bbf700 1 -- 172.21.15.146:0/1267939906 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc06ea00 con 0x7efefc0c11f0 2024-09-13T18:56:55.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.807+0000 7eff023c0700 1 --2- 172.21.15.146:0/1267939906 >> v2:172.21.15.154:3300/0 conn(0x7efefc0c2e40 0x7efefc159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:56:55.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.811+0000 7eff01bbf700 1 --2- 172.21.15.146:0/1267939906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efefc0c11f0 0x7efefc0c6960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efefc071ca0 tx=0x7efeec009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4db6af7af29d8ad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:55.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.815+0000 7eff00bbd700 1 -- 172.21.15.146:0/1267939906 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec011830 con 0x7efefc0c11f0 2024-09-13T18:56:55.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.815+0000 7eff00bbd700 1 -- 172.21.15.146:0/1267939906 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec011990 con 0x7efefc0c11f0 2024-09-13T18:56:55.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.815+0000 7eff00bbd700 1 -- 172.21.15.146:0/1267939906 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec011d20 con 0x7efefc0c11f0 2024-09-13T18:56:55.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.815+0000 7eff02bc1700 1 -- 172.21.15.146:0/1267939906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efefc0c11f0 msgr2=0x7efefc0c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:55.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.815+0000 7eff02bc1700 1 --2- 172.21.15.146:0/1267939906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efefc0c11f0 0x7efefc0c6960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efefc071ca0 tx=0x7efeec009bf0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.815+0000 7eff02bc1700 1 -- 172.21.15.146:0/1267939906 shutdown_connections 2024-09-13T18:56:55.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.815+0000 7eff02bc1700 1 --2- 172.21.15.146:0/1267939906 >> v2:172.21.15.154:3300/0 conn(0x7efefc0c2e40 0x7efefc159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.815+0000 7eff02bc1700 1 --2- 172.21.15.146:0/1267939906 >> v2:172.21.15.146:3300/0 conn(0x7efefc0c15d0 0x7efefc159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.815+0000 7eff02bc1700 1 --2- 172.21.15.146:0/1267939906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efefc0c11f0 0x7efefc0c6960 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.815+0000 7eff02bc1700 1 -- 172.21.15.146:0/1267939906 >> 172.21.15.146:0/1267939906 conn(0x7efefc071ed0 msgr2=0x7efefc0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:55.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff02bc1700 1 -- 172.21.15.146:0/1267939906 shutdown_connections 2024-09-13T18:56:55.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff02bc1700 1 -- 172.21.15.146:0/1267939906 wait complete. 2024-09-13T18:56:55.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff02bc1700 1 Processor -- start 2024-09-13T18:56:55.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff02bc1700 1 -- start start 2024-09-13T18:56:55.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff02bc1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efefc0c11f0 0x7efefc06fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff02bc1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efefc0c15d0 0x7efefc0c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff02bc1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efefc0c2e40 0x7efefc1f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff02bc1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc071640 con 0x7efefc0c11f0 2024-09-13T18:56:55.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff02bc1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc0702c0 con 0x7efefc0c15d0 2024-09-13T18:56:55.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff02bc1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efefc0705a0 con 0x7efefc0c2e40 2024-09-13T18:56:55.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff01bbf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efefc0c11f0 0x7efefc06fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff01bbf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efefc0c11f0 0x7efefc06fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35528/0 (socket says 172.21.15.146:35528) 2024-09-13T18:56:55.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff01bbf700 1 -- 172.21.15.146:0/3524057020 learned_addr learned my addr 172.21.15.146:0/3524057020 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:55.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff023c0700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efefc0c2e40 0x7efefc1f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff013be700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efefc0c15d0 0x7efefc0c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff01bbf700 1 -- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efefc0c2e40 msgr2=0x7efefc1f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:55.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff01bbf700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efefc0c2e40 0x7efefc1f2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff01bbf700 1 -- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efefc0c15d0 msgr2=0x7efefc0c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:55.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff01bbf700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efefc0c15d0 0x7efefc0c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:55.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff01bbf700 1 -- 172.21.15.146:0/3524057020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef4009a70 con 0x7efefc0c11f0 2024-09-13T18:56:55.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.819+0000 7eff01bbf700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efefc0c11f0 0x7efefc06fc30 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7efeec000c00 tx=0x7efeec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:55.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.823+0000 7efef27fc700 1 -- 172.21.15.146:0/3524057020 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec004450 con 0x7efefc0c11f0 2024-09-13T18:56:55.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.823+0000 7eff02bc1700 1 -- 172.21.15.146:0/3524057020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeec009570 con 0x7efefc0c11f0 2024-09-13T18:56:55.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.823+0000 7efef27fc700 1 -- 172.21.15.146:0/3524057020 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec0045d0 con 0x7efefc0c11f0 2024-09-13T18:56:55.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.823+0000 7efef27fc700 1 -- 172.21.15.146:0/3524057020 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec003900 con 0x7efefc0c11f0 2024-09-13T18:56:55.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.823+0000 7eff02bc1700 1 -- 172.21.15.146:0/3524057020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc070b60 con 0x7efefc0c11f0 2024-09-13T18:56:55.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.823+0000 7efef27fc700 1 -- 172.21.15.146:0/3524057020 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efeec003a60 con 0x7efefc0c11f0 2024-09-13T18:56:55.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.823+0000 7efef27fc700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efee0066730 0x7efee0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:55.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.823+0000 7efef27fc700 1 -- 172.21.15.146:0/3524057020 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7efeec085d50 con 0x7efefc0c11f0 2024-09-13T18:56:55.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.823+0000 7eff013be700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efee0066730 0x7efee0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:55.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.823+0000 7eff013be700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efee0066730 0x7efee0068bf0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efefc152990 tx=0x7efef8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:55.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.827+0000 7eff02bc1700 1 -- 172.21.15.146:0/3524057020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee4005310 con 0x7efefc0c11f0 2024-09-13T18:56:55.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:55.831+0000 7efef27fc700 1 -- 172.21.15.146:0/3524057020 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efeec0527a0 con 0x7efefc0c11f0 2024-09-13T18:56:56.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.055+0000 7eff02bc1700 1 -- 172.21.15.146:0/3524057020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efee4005b40 con 0x7efefc0c11f0 2024-09-13T18:56:56.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.055+0000 7efef27fc700 1 -- 172.21.15.146:0/3524057020 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7efeec015030 con 0x7efefc0c11f0 2024-09-13T18:56:56.070 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-09-13T18:56:56.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.063+0000 7eff02bc1700 1 -- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efee0066730 msgr2=0x7efee0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:56.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.063+0000 7eff02bc1700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efee0066730 0x7efee0068bf0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efefc152990 tx=0x7efef8009190 comp rx=0 tx=0).stop 2024-09-13T18:56:56.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.063+0000 7eff02bc1700 1 -- 172.21.15.146:0/3524057020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efefc0c11f0 msgr2=0x7efefc06fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:56.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.063+0000 7eff02bc1700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efefc0c11f0 0x7efefc06fc30 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7efeec000c00 tx=0x7efeec004770 comp rx=0 tx=0).stop 2024-09-13T18:56:56.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.063+0000 7eff02bc1700 1 -- 172.21.15.146:0/3524057020 shutdown_connections 2024-09-13T18:56:56.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.063+0000 7eff02bc1700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7efee0066730 0x7efee0068bf0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.063+0000 7eff02bc1700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efefc0c2e40 0x7efefc1f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.063+0000 7eff02bc1700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efefc0c15d0 0x7efefc0c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.063+0000 7eff02bc1700 1 --2- 172.21.15.146:0/3524057020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efefc0c11f0 0x7efefc06fc30 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.063+0000 7eff02bc1700 1 -- 172.21.15.146:0/3524057020 >> 172.21.15.146:0/3524057020 conn(0x7efefc071ed0 msgr2=0x7efefc158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:56.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.063+0000 7eff02bc1700 1 -- 172.21.15.146:0/3524057020 shutdown_connections 2024-09-13T18:56:56.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.063+0000 7eff02bc1700 1 -- 172.21.15.146:0/3524057020 wait complete. 2024-09-13T18:56:56.090 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:56:56.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.247+0000 7f98b9436700 1 Processor -- start 2024-09-13T18:56:56.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.247+0000 7f98b9436700 1 -- start start 2024-09-13T18:56:56.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.251+0000 7f98b9436700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f98b40caed0 0x7f98b4072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:56.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.251+0000 7f98b9436700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f98b406e6b0 0x7f98b4156990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:56.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.251+0000 7f98b9436700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b40ca390 con 0x7f98b40cb2b0 2024-09-13T18:56:56.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.251+0000 7f98b9436700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b4159980 con 0x7f98b40caed0 2024-09-13T18:56:56.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.251+0000 7f98b9436700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b40728c0 con 0x7f98b406e6b0 2024-09-13T18:56:56.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.251+0000 7f98b37fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f98b40cb2b0 0x7f98b4159e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:34466/0 (socket says 172.21.15.146:34466) 2024-09-13T18:56:56.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.251+0000 7f98b37fe700 1 -- 172.21.15.146:0/3809421460 learned_addr learned my addr 172.21.15.146:0/3809421460 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:56:56.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.251+0000 7f98b3fff700 1 --2- 172.21.15.146:0/3809421460 >> v2:172.21.15.146:3300/0 conn(0x7f98b40caed0 0x7f98b4072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:56.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.251+0000 7f98b8c35700 1 --2- 172.21.15.146:0/3809421460 >> v2:172.21.15.154:3300/0 conn(0x7f98b406e6b0 0x7f98b4156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:56.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.251+0000 7f98b8c35700 1 -- 172.21.15.146:0/3809421460 >> v2:172.21.15.146:3300/0 conn(0x7f98b40caed0 msgr2=0x7f98b4072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:56:56.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.259+0000 7f98b8c35700 1 --2- 172.21.15.146:0/3809421460 >> v2:172.21.15.146:3300/0 conn(0x7f98b40caed0 0x7f98b4072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.259+0000 7f98b8c35700 1 -- 172.21.15.146:0/3809421460 >> v1:172.21.15.154:6789/0 conn(0x7f98b40cb2b0 legacy=0x7f98b4159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:56.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.259+0000 7f98b8c35700 1 -- 172.21.15.146:0/3809421460 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b40c7480 con 0x7f98b406e6b0 2024-09-13T18:56:56.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.259+0000 7f98b8c35700 1 --2- 172.21.15.146:0/3809421460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98b406e6b0 0x7f98b4156990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009850 tx=0x7f98a8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14a28f99a67965b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:56.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.259+0000 7f98b3fff700 1 --2- 172.21.15.146:0/3809421460 >> v2:172.21.15.146:3300/0 conn(0x7f98b40caed0 0x7f98b4072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:56:56.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.267+0000 7f98b2ffd700 1 -- 172.21.15.146:0/3809421460 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8004230 con 0x7f98b406e6b0 2024-09-13T18:56:56.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.267+0000 7f98b2ffd700 1 -- 172.21.15.146:0/3809421460 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a8004390 con 0x7f98b406e6b0 2024-09-13T18:56:56.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.267+0000 7f98b2ffd700 1 -- 172.21.15.146:0/3809421460 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a80036c0 con 0x7f98b406e6b0 2024-09-13T18:56:56.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.267+0000 7f98b9436700 1 -- 172.21.15.146:0/3809421460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98b406e6b0 msgr2=0x7f98b4156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:56.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.267+0000 7f98b9436700 1 --2- 172.21.15.146:0/3809421460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98b406e6b0 0x7f98b4156990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009850 tx=0x7f98a8009b60 comp rx=0 tx=0).stop 2024-09-13T18:56:56.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.267+0000 7f98b9436700 1 -- 172.21.15.146:0/3809421460 shutdown_connections 2024-09-13T18:56:56.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.267+0000 7f98b9436700 1 --2- 172.21.15.146:0/3809421460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98b406e6b0 0x7f98b4156990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.267+0000 7f98b9436700 1 --2- 172.21.15.146:0/3809421460 >> v2:172.21.15.146:3300/0 conn(0x7f98b40caed0 0x7f98b4072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.267+0000 7f98b9436700 1 -- 172.21.15.146:0/3809421460 >> 172.21.15.146:0/3809421460 conn(0x7f98b4066090 msgr2=0x7f98b4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:56.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.283+0000 7f98b9436700 1 -- 172.21.15.146:0/3809421460 shutdown_connections 2024-09-13T18:56:56.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.283+0000 7f98b9436700 1 -- 172.21.15.146:0/3809421460 wait complete. 2024-09-13T18:56:56.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.283+0000 7f98b9436700 1 Processor -- start 2024-09-13T18:56:56.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.283+0000 7f98b9436700 1 -- start start 2024-09-13T18:56:56.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b9436700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98b406e6b0 0x7f98b41f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:56.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b9436700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98b40caed0 0x7f98b406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:56.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b9436700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98b40cb2b0 0x7f98b40ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:56.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b9436700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b41f4340 con 0x7f98b40caed0 2024-09-13T18:56:56.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b9436700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b4123e50 con 0x7f98b406e6b0 2024-09-13T18:56:56.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b9436700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b40abd10 con 0x7f98b40cb2b0 2024-09-13T18:56:56.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98b40caed0 0x7f98b406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:56.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98b40caed0 0x7f98b406d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35540/0 (socket says 172.21.15.146:35540) 2024-09-13T18:56:56.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b37fe700 1 -- 172.21.15.146:0/2895485145 learned_addr learned my addr 172.21.15.146:0/2895485145 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:56.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b37fe700 1 -- 172.21.15.146:0/2895485145 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98b40cb2b0 msgr2=0x7f98b40ac1a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:56:56.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b37fe700 1 --2- 172.21.15.146:0/2895485145 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98b40cb2b0 0x7f98b40ac1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b37fe700 1 -- 172.21.15.146:0/2895485145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98b406e6b0 msgr2=0x7f98b41f3e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:56:56.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b37fe700 1 --2- 172.21.15.146:0/2895485145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98b406e6b0 0x7f98b41f3e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.287+0000 7f98b37fe700 1 -- 172.21.15.146:0/2895485145 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac009a70 con 0x7f98b40caed0 2024-09-13T18:56:56.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.291+0000 7f98b37fe700 1 --2- 172.21.15.146:0/2895485145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98b40caed0 0x7f98b406d8f0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f98a400c8c0 tx=0x7f98a400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:56.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.299+0000 7f98b0ff9700 1 -- 172.21.15.146:0/2895485145 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4004a90 con 0x7f98b40caed0 2024-09-13T18:56:56.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.299+0000 7f98b0ff9700 1 -- 172.21.15.146:0/2895485145 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a4004c10 con 0x7f98b40caed0 2024-09-13T18:56:56.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.299+0000 7f98b0ff9700 1 -- 172.21.15.146:0/2895485145 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a400ce60 con 0x7f98b40caed0 2024-09-13T18:56:56.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.299+0000 7f98b9436700 1 -- 172.21.15.146:0/2895485145 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a80094b0 con 0x7f98b40caed0 2024-09-13T18:56:56.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.299+0000 7f98b9436700 1 -- 172.21.15.146:0/2895485145 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b4125660 con 0x7f98b40caed0 2024-09-13T18:56:56.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.303+0000 7f98b9436700 1 -- 172.21.15.146:0/2895485145 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b40aa4c0 con 0x7f98b40caed0 2024-09-13T18:56:56.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.307+0000 7f98b0ff9700 1 -- 172.21.15.146:0/2895485145 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f98a40070d0 con 0x7f98b40caed0 2024-09-13T18:56:56.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.311+0000 7f98b0ff9700 1 --2- 172.21.15.146:0/2895485145 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f98a00667b0 0x7f98a0068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:56.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.311+0000 7f98b0ff9700 1 -- 172.21.15.146:0/2895485145 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f98a40843c0 con 0x7f98b40caed0 2024-09-13T18:56:56.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.311+0000 7f98b3fff700 1 --2- 172.21.15.146:0/2895485145 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f98a00667b0 0x7f98a0068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:56.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.311+0000 7f98b0ff9700 1 -- 172.21.15.146:0/2895485145 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f98a4050c90 con 0x7f98b40caed0 2024-09-13T18:56:56.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.311+0000 7f98b3fff700 1 --2- 172.21.15.146:0/2895485145 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f98a00667b0 0x7f98a0068c70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00e2b0 tx=0x7f98ac00d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:56.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.567+0000 7f98b9436700 1 -- 172.21.15.146:0/2895485145 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f98b40a9c30 con 0x7f98b40caed0 2024-09-13T18:56:56.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.567+0000 7f98b0ff9700 1 -- 172.21.15.146:0/2895485145 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v43) v1 ==== 103+0+11 (secure 0 0 0) 0x7f98a4018020 con 0x7f98b40caed0 2024-09-13T18:56:56.580 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-09-13T18:56:56.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.571+0000 7f98b9436700 1 -- 172.21.15.146:0/2895485145 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f98a00667b0 msgr2=0x7f98a0068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:56.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.571+0000 7f98b9436700 1 --2- 172.21.15.146:0/2895485145 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f98a00667b0 0x7f98a0068c70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00e2b0 tx=0x7f98ac00d040 comp rx=0 tx=0).stop 2024-09-13T18:56:56.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.571+0000 7f98b9436700 1 -- 172.21.15.146:0/2895485145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98b40caed0 msgr2=0x7f98b406d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:56.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.571+0000 7f98b9436700 1 --2- 172.21.15.146:0/2895485145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98b40caed0 0x7f98b406d8f0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f98a400c8c0 tx=0x7f98a400cc80 comp rx=0 tx=0).stop 2024-09-13T18:56:56.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.571+0000 7f98b9436700 1 -- 172.21.15.146:0/2895485145 shutdown_connections 2024-09-13T18:56:56.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.571+0000 7f98b9436700 1 --2- 172.21.15.146:0/2895485145 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f98a00667b0 0x7f98a0068c70 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.571+0000 7f98b9436700 1 --2- 172.21.15.146:0/2895485145 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98b40cb2b0 0x7f98b40ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.571+0000 7f98b9436700 1 --2- 172.21.15.146:0/2895485145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98b40caed0 0x7f98b406d8f0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.571+0000 7f98b9436700 1 --2- 172.21.15.146:0/2895485145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98b406e6b0 0x7f98b41f3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.571+0000 7f98b9436700 1 -- 172.21.15.146:0/2895485145 >> 172.21.15.146:0/2895485145 conn(0x7f98b4066090 msgr2=0x7f98b4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:56.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.571+0000 7f98b9436700 1 -- 172.21.15.146:0/2895485145 shutdown_connections 2024-09-13T18:56:56.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.571+0000 7f98b9436700 1 -- 172.21.15.146:0/2895485145 wait complete. 2024-09-13T18:56:56.602 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:56:56.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.791+0000 7f03dc5eb700 1 Processor -- start 2024-09-13T18:56:56.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.791+0000 7f03dc5eb700 1 -- start start 2024-09-13T18:56:56.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.791+0000 7f03dc5eb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03d4070e00 0x7f03d40c6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:56.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.791+0000 7f03dc5eb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f03d40711e0 0x7f03d40c8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:56.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.791+0000 7f03dc5eb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f03d4071600 0x7f03d40c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:56.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.791+0000 7f03dc5eb700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d40c0a70 con 0x7f03d40711e0 2024-09-13T18:56:56.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.791+0000 7f03dc5eb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d40c0d50 con 0x7f03d4071600 2024-09-13T18:56:56.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.791+0000 7f03dc5eb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4167320 con 0x7f03d4070e00 2024-09-13T18:56:56.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.791+0000 7f03d9b86700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f03d40711e0 0x7f03d40c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:56.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03d9b86700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f03d40711e0 0x7f03d40c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35556/0 (socket says 172.21.15.146:35556) 2024-09-13T18:56:56.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03d9b86700 1 -- 172.21.15.146:0/1294639210 learned_addr learned my addr 172.21.15.146:0/1294639210 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:56.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03da387700 1 --2- 172.21.15.146:0/1294639210 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03d4070e00 0x7f03d40c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:56.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dab88700 1 --2- 172.21.15.146:0/1294639210 >> v2:172.21.15.154:3300/0 conn(0x7f03d4071600 0x7f03d40c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:56.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03d9b86700 1 -- 172.21.15.146:0/1294639210 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03d4070e00 msgr2=0x7f03d40c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:56:56.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03d9b86700 1 --2- 172.21.15.146:0/1294639210 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03d4070e00 0x7f03d40c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03d9b86700 1 -- 172.21.15.146:0/1294639210 >> v2:172.21.15.154:3300/0 conn(0x7f03d4071600 msgr2=0x7f03d40c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:56:56.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03d9b86700 1 --2- 172.21.15.146:0/1294639210 >> v2:172.21.15.154:3300/0 conn(0x7f03d4071600 0x7f03d40c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03d9b86700 1 -- 172.21.15.146:0/1294639210 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d40c5ee0 con 0x7f03d40711e0 2024-09-13T18:56:56.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03d9b86700 1 --2- 172.21.15.146:0/1294639210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d40711e0 0x7f03d40c8050 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f03d408bc90 tx=0x7f03d000cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f8116a589fe1f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:56.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03d9385700 1 -- 172.21.15.146:0/1294639210 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0015bf0 con 0x7f03d40711e0 2024-09-13T18:56:56.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03d9385700 1 -- 172.21.15.146:0/1294639210 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d0019070 con 0x7f03d40711e0 2024-09-13T18:56:56.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03d9385700 1 -- 172.21.15.146:0/1294639210 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d001e430 con 0x7f03d40711e0 2024-09-13T18:56:56.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1294639210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d40711e0 msgr2=0x7f03d40c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:56.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 --2- 172.21.15.146:0/1294639210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d40711e0 0x7f03d40c8050 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f03d408bc90 tx=0x7f03d000cb50 comp rx=0 tx=0).stop 2024-09-13T18:56:56.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1294639210 shutdown_connections 2024-09-13T18:56:56.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 --2- 172.21.15.146:0/1294639210 >> v2:172.21.15.154:3300/0 conn(0x7f03d4071600 0x7f03d40c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 --2- 172.21.15.146:0/1294639210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d40711e0 0x7f03d40c8050 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 --2- 172.21.15.146:0/1294639210 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03d4070e00 0x7f03d40c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1294639210 >> 172.21.15.146:0/1294639210 conn(0x7f03d406f5c0 msgr2=0x7f03d406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:56.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1294639210 shutdown_connections 2024-09-13T18:56:56.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1294639210 wait complete. 2024-09-13T18:56:56.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 Processor -- start 2024-09-13T18:56:56.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 -- start start 2024-09-13T18:56:56.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03d4070e00 0x7f03d40c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:56.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03d40711e0 0x7f03d41fee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:56.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d4071600 0x7f03d40c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:56.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40c42a0 con 0x7f03d4071600 2024-09-13T18:56:56.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40c34f0 con 0x7f03d4070e00 2024-09-13T18:56:56.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dc5eb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41ff740 con 0x7f03d40711e0 2024-09-13T18:56:56.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dab88700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d4071600 0x7f03d40c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:56.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dab88700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d4071600 0x7f03d40c46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:35558/0 (socket says 172.21.15.146:35558) 2024-09-13T18:56:56.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dab88700 1 -- 172.21.15.146:0/1760719208 learned_addr learned my addr 172.21.15.146:0/1760719208 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:56:56.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03d9b86700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03d40711e0 0x7f03d41fee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:56.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03da387700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03d4070e00 0x7f03d40c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dab88700 1 -- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03d40711e0 msgr2=0x7f03d41fee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dab88700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03d40711e0 0x7f03d41fee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dab88700 1 -- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03d4070e00 msgr2=0x7f03d40c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dab88700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03d4070e00 0x7f03d40c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dab88700 1 -- 172.21.15.146:0/1760719208 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d000c4d0 con 0x7f03d4071600 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.795+0000 7f03dab88700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d4071600 0x7f03d40c46c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00efc0 tx=0x7f03cc00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.799+0000 7f03caffd700 1 -- 172.21.15.146:0/1760719208 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00cad0 con 0x7f03d4071600 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.799+0000 7f03caffd700 1 -- 172.21.15.146:0/1760719208 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03cc00f040 con 0x7f03d4071600 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.799+0000 7f03caffd700 1 -- 172.21.15.146:0/1760719208 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00ce70 con 0x7f03d4071600 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.799+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1760719208 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d41ff9c0 con 0x7f03d4071600 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.799+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1760719208 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d40c30e0 con 0x7f03d4071600 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.799+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1760719208 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d40c43e0 con 0x7f03d4071600 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.799+0000 7f03caffd700 1 -- 172.21.15.146:0/1760719208 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f03cc014480 con 0x7f03d4071600 2024-09-13T18:56:56.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.799+0000 7f03caffd700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f03b8066730 0x7f03b8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:56:56.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.799+0000 7f03caffd700 1 -- 172.21.15.146:0/1760719208 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f03cc085190 con 0x7f03d4071600 2024-09-13T18:56:56.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.799+0000 7f03da387700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f03b8066730 0x7f03b8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:56:56.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.799+0000 7f03da387700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f03b8066730 0x7f03b8068bf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f03c400d440 tx=0x7f03c400f040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:56:56.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:56.803+0000 7f03caffd700 1 -- 172.21.15.146:0/1760719208 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f03d40c43e0 con 0x7f03d4071600 2024-09-13T18:56:57.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.099+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1760719208 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f03d40c43e0 con 0x7f03d4071600 2024-09-13T18:56:57.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.103+0000 7f03caffd700 1 -- 172.21.15.146:0/1760719208 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 32 entries v43) v1 ==== 81+0+1974 (secure 0 0 0) 0x7f03d40c43e0 con 0x7f03d4071600 2024-09-13T18:56:57.115 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6837/2053734262 2024-09-14T18:55:25.944088+0000 2024-09-13T18:56:57.116 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6839/3274220802 2024-09-14T18:36:10.677826+0000 2024-09-13T18:56:57.116 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6835/294014284 2024-09-14T18:50:50.876658+0000 2024-09-13T18:56:57.116 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6837/3274220802 2024-09-14T18:36:10.677826+0000 2024-09-13T18:56:57.116 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6836/2169954140 2024-09-14T18:53:50.924269+0000 2024-09-13T18:56:57.116 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6832/1498985282 2024-09-14T18:34:25.647701+0000 2024-09-13T18:56:57.116 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6835/1642500214 2024-09-14T18:30:30.598692+0000 2024-09-13T18:56:57.117 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6836/31041557 2024-09-14T18:32:55.627334+0000 2024-09-13T18:56:57.117 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6832/3023779168 2024-09-14T18:52:35.902095+0000 2024-09-13T18:56:57.117 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6834/1642500214 2024-09-14T18:30:30.598692+0000 2024-09-13T18:56:57.117 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6836/3234109877 2024-09-14T18:45:55.807881+0000 2024-09-13T18:56:57.117 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6837/2169954140 2024-09-14T18:53:50.924269+0000 2024-09-13T18:56:57.117 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6833/1498985282 2024-09-14T18:34:25.647701+0000 2024-09-13T18:56:57.117 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6837/31041557 2024-09-14T18:32:55.627334+0000 2024-09-13T18:56:57.117 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6839/776638146 2024-09-14T18:44:35.790651+0000 2024-09-13T18:56:57.118 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6839/3978828499 2024-09-14T18:29:25.578693+0000 2024-09-13T18:56:57.118 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6832/2835746745 2024-09-14T18:28:15.564530+0000 2024-09-13T18:56:57.118 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6834/1652676664 2024-09-14T18:38:00.699180+0000 2024-09-13T18:56:57.118 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6837/776638146 2024-09-14T18:44:35.790651+0000 2024-09-13T18:56:57.118 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6837/925899756 2024-09-14T18:48:20.844616+0000 2024-09-13T18:56:57.118 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6833/2835746745 2024-09-14T18:28:15.564530+0000 2024-09-13T18:56:57.119 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6835/1652676664 2024-09-14T18:38:00.699180+0000 2024-09-13T18:56:57.119 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6839/2053734262 2024-09-14T18:55:25.944088+0000 2024-09-13T18:56:57.119 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6839/925899756 2024-09-14T18:48:20.844616+0000 2024-09-13T18:56:57.119 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.154:6837/3978828499 2024-09-14T18:29:25.578693+0000 2024-09-13T18:56:57.120 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6836/3926931671 2024-09-14T18:40:45.735995+0000 2024-09-13T18:56:57.120 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6837/3926931671 2024-09-14T18:40:45.735995+0000 2024-09-13T18:56:57.120 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6833/3901685195 2024-09-14T18:43:15.768366+0000 2024-09-13T18:56:57.120 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6837/3234109877 2024-09-14T18:45:55.807881+0000 2024-09-13T18:56:57.120 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6834/294014284 2024-09-14T18:50:50.876658+0000 2024-09-13T18:56:57.120 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6832/3901685195 2024-09-14T18:43:15.768366+0000 2024-09-13T18:56:57.120 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146:6833/3023779168 2024-09-14T18:52:35.902095+0000 2024-09-13T18:56:57.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.103+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f03b8066730 msgr2=0x7f03b8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:57.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.103+0000 7f03dc5eb700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f03b8066730 0x7f03b8068bf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f03c400d440 tx=0x7f03c400f040 comp rx=0 tx=0).stop 2024-09-13T18:56:57.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.103+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1760719208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d4071600 msgr2=0x7f03d40c46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:56:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.103+0000 7f03dc5eb700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d4071600 0x7f03d40c46c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00efc0 tx=0x7f03cc00c0c0 comp rx=0 tx=0).stop 2024-09-13T18:56:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.103+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1760719208 shutdown_connections 2024-09-13T18:56:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.103+0000 7f03dc5eb700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f03b8066730 0x7f03b8068bf0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.103+0000 7f03dc5eb700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d4071600 0x7f03d40c46c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.103+0000 7f03dc5eb700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03d40711e0 0x7f03d41fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.103+0000 7f03dc5eb700 1 --2- 172.21.15.146:0/1760719208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03d4070e00 0x7f03d40c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:56:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.103+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1760719208 >> 172.21.15.146:0/1760719208 conn(0x7f03d406f5c0 msgr2=0x7f03d40ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:56:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.103+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1760719208 shutdown_connections 2024-09-13T18:56:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:56:57.103+0000 7f03dc5eb700 1 -- 172.21.15.146:0/1760719208 wait complete. 2024-09-13T18:56:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:listed 32 entries 2024-09-13T18:56:57.129 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-13T18:56:57.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:56:57.129 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T18:56:57.149 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi146.stderr:ceph-fuse[33473]: fuse finished with error 0 and tester_r 0 2024-09-13T18:56:57.150 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:57:03.252 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-13T18:57:03.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:57:03.253 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T18:57:03.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:57:03.268 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-13T18:57:03.791 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T18:57:03.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:57:03.792 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T18:57:03.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:57:03.841 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-09-13T18:57:03.991 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T18:57:04.000 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T18:57:04.001 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:57:04.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.127+0000 7fa7f8f81700 1 Processor -- start 2024-09-13T18:57:04.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f8f81700 1 -- start start 2024-09-13T18:57:04.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f8f81700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa7f4157e10 0x7fa7f40c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:04.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f8f81700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa7f406fe60 0x7fa7f4070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:04.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f8f81700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f40c5bd0 0x7fa7f40c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:04.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f8f81700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f415e690 con 0x7fa7f406fe60 2024-09-13T18:57:04.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f8f81700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f40adfe0 con 0x7fa7f4157e10 2024-09-13T18:57:04.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f8f81700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40c7570 con 0x7fa7f40c5bd0 2024-09-13T18:57:04.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f1d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa7f406fe60 0x7fa7f4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:04.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f1d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa7f406fe60 0x7fa7f4070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57570/0 (socket says 172.21.15.146:57570) 2024-09-13T18:57:04.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f1d9b700 1 -- 172.21.15.146:0/2619686452 learned_addr learned my addr 172.21.15.146:0/2619686452 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:04.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f1d9b700 1 -- 172.21.15.146:0/2619686452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f40c5bd0 msgr2=0x7fa7f40c6060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:57:04.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f1d9b700 1 --2- 172.21.15.146:0/2619686452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f40c5bd0 0x7fa7f40c6060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f1d9b700 1 -- 172.21.15.146:0/2619686452 >> v2:172.21.15.154:3300/0 conn(0x7fa7f4157e10 msgr2=0x7fa7f40c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:04.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f1d9b700 1 --2- 172.21.15.146:0/2619686452 >> v2:172.21.15.154:3300/0 conn(0x7fa7f4157e10 0x7fa7f40c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f1d9b700 1 -- 172.21.15.146:0/2619686452 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f415eb80 con 0x7fa7f406fe60 2024-09-13T18:57:04.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.131+0000 7fa7f1d9b700 1 --2- 172.21.15.146:0/2619686452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f406fe60 0x7fa7f4070260 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40728e0 tx=0x7fa7ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3b387a2d066717 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:04.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.139+0000 7fa7f159a700 1 -- 172.21.15.146:0/2619686452 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00e070 con 0x7fa7f406fe60 2024-09-13T18:57:04.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.139+0000 7fa7f159a700 1 -- 172.21.15.146:0/2619686452 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec00b9c0 con 0x7fa7f406fe60 2024-09-13T18:57:04.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.139+0000 7fa7f159a700 1 -- 172.21.15.146:0/2619686452 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00bd50 con 0x7fa7f406fe60 2024-09-13T18:57:04.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.143+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2619686452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f406fe60 msgr2=0x7fa7f4070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:04.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.143+0000 7fa7f8f81700 1 --2- 172.21.15.146:0/2619686452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f406fe60 0x7fa7f4070260 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40728e0 tx=0x7fa7ec00b330 comp rx=0 tx=0).stop 2024-09-13T18:57:04.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.143+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2619686452 shutdown_connections 2024-09-13T18:57:04.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.143+0000 7fa7f8f81700 1 --2- 172.21.15.146:0/2619686452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f40c5bd0 0x7fa7f40c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.143+0000 7fa7f8f81700 1 --2- 172.21.15.146:0/2619686452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f406fe60 0x7fa7f4070260 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.143+0000 7fa7f8f81700 1 --2- 172.21.15.146:0/2619686452 >> v2:172.21.15.154:3300/0 conn(0x7fa7f4157e10 0x7fa7f40c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.143+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2619686452 >> 172.21.15.146:0/2619686452 conn(0x7fa7f40cb1d0 msgr2=0x7fa7f40cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:04.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.143+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2619686452 shutdown_connections 2024-09-13T18:57:04.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.143+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2619686452 wait complete. 2024-09-13T18:57:04.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.147+0000 7fa7f8f81700 1 Processor -- start 2024-09-13T18:57:04.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.147+0000 7fa7f8f81700 1 -- start start 2024-09-13T18:57:04.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.147+0000 7fa7f8f81700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f406fe60 0x7fa7f414f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:04.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.147+0000 7fa7f8f81700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f40c5bd0 0x7fa7f41506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:04.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.147+0000 7fa7f8f81700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f4157e10 0x7fa7f415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:04.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.147+0000 7fa7f8f81700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f41f3dc0 con 0x7fa7f40c5bd0 2024-09-13T18:57:04.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.147+0000 7fa7f8f81700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f414ef10 con 0x7fa7f4157e10 2024-09-13T18:57:04.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.147+0000 7fa7f8f81700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f41f3fb0 con 0x7fa7f406fe60 2024-09-13T18:57:04.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.147+0000 7fa7f1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f40c5bd0 0x7fa7f41506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:04.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.147+0000 7fa7f1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f40c5bd0 0x7fa7f41506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57578/0 (socket says 172.21.15.146:57578) 2024-09-13T18:57:04.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.147+0000 7fa7f1d9b700 1 -- 172.21.15.146:0/2843668198 learned_addr learned my addr 172.21.15.146:0/2843668198 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:04.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.147+0000 7fa7f1d9b700 1 -- 172.21.15.146:0/2843668198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f406fe60 msgr2=0x7fa7f414f480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:57:04.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.155+0000 7fa7f1d9b700 1 --2- 172.21.15.146:0/2843668198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f406fe60 0x7fa7f414f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.155+0000 7fa7f1d9b700 1 -- 172.21.15.146:0/2843668198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f4157e10 msgr2=0x7fa7f415bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:04.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.155+0000 7fa7f1d9b700 1 --2- 172.21.15.146:0/2843668198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f4157e10 0x7fa7f415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.155+0000 7fa7f1d9b700 1 -- 172.21.15.146:0/2843668198 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec009b30 con 0x7fa7f40c5bd0 2024-09-13T18:57:04.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.155+0000 7fa7f1d9b700 1 --2- 172.21.15.146:0/2843668198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f40c5bd0 0x7fa7f41506a0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec000f80 tx=0x7fa7ec004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:04.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.155+0000 7fa7e2ffd700 1 -- 172.21.15.146:0/2843668198 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec004410 con 0x7fa7f40c5bd0 2024-09-13T18:57:04.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.155+0000 7fa7e2ffd700 1 -- 172.21.15.146:0/2843668198 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec004570 con 0x7fa7f40c5bd0 2024-09-13T18:57:04.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.155+0000 7fa7e2ffd700 1 -- 172.21.15.146:0/2843668198 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec004920 con 0x7fa7f40c5bd0 2024-09-13T18:57:04.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.155+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2843668198 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f41f4290 con 0x7fa7f40c5bd0 2024-09-13T18:57:04.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.155+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2843668198 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f41f4850 con 0x7fa7f40c5bd0 2024-09-13T18:57:04.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.155+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2843668198 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d8005310 con 0x7fa7f40c5bd0 2024-09-13T18:57:04.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.155+0000 7fa7e2ffd700 1 -- 172.21.15.146:0/2843668198 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa7ec01b430 con 0x7fa7f40c5bd0 2024-09-13T18:57:04.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.159+0000 7fa7e2ffd700 1 --2- 172.21.15.146:0/2843668198 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa7d4066780 0x7fa7d4068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:04.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.159+0000 7fa7e2ffd700 1 -- 172.21.15.146:0/2843668198 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fa7ec085cc0 con 0x7fa7f40c5bd0 2024-09-13T18:57:04.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.159+0000 7fa7f259c700 1 --2- 172.21.15.146:0/2843668198 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa7d4066780 0x7fa7d4068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:04.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.159+0000 7fa7f259c700 1 --2- 172.21.15.146:0/2843668198 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa7d4066780 0x7fa7d4068c40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009750 tx=0x7fa7e8006cd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:04.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.167+0000 7fa7e2ffd700 1 -- 172.21.15.146:0/2843668198 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa7ec056230 con 0x7fa7f40c5bd0 2024-09-13T18:57:04.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.483+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2843668198 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7d8000bc0 con 0x7fa7d4066780 2024-09-13T18:57:04.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.499+0000 7fa7e2ffd700 1 -- 172.21.15.146:0/2843668198 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370788 (secure 0 0 0) 0x7fa7d8000bc0 con 0x7fa7d4066780 2024-09-13T18:57:04.521 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:57:04.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.507+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2843668198 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa7d4066780 msgr2=0x7fa7d4068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:04.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.507+0000 7fa7f8f81700 1 --2- 172.21.15.146:0/2843668198 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa7d4066780 0x7fa7d4068c40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009750 tx=0x7fa7e8006cd0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.507+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2843668198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f40c5bd0 msgr2=0x7fa7f41506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:04.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.507+0000 7fa7f8f81700 1 --2- 172.21.15.146:0/2843668198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f40c5bd0 0x7fa7f41506a0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec000f80 tx=0x7fa7ec004af0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.507+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2843668198 shutdown_connections 2024-09-13T18:57:04.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.507+0000 7fa7f8f81700 1 --2- 172.21.15.146:0/2843668198 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa7d4066780 0x7fa7d4068c40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.507+0000 7fa7f8f81700 1 --2- 172.21.15.146:0/2843668198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f4157e10 0x7fa7f415bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.507+0000 7fa7f8f81700 1 --2- 172.21.15.146:0/2843668198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f40c5bd0 0x7fa7f41506a0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.507+0000 7fa7f8f81700 1 --2- 172.21.15.146:0/2843668198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f406fe60 0x7fa7f414f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.507+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2843668198 >> 172.21.15.146:0/2843668198 conn(0x7fa7f40cb1d0 msgr2=0x7fa7f40ca610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:04.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.507+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2843668198 shutdown_connections 2024-09-13T18:57:04.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.507+0000 7fa7f8f81700 1 -- 172.21.15.146:0/2843668198 wait complete. 2024-09-13T18:57:04.526 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:57:04.533 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":955,"stamp":"2024-09-13T18:57:03.595662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":11315141164,"num_objects":2956,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2956,"num_whiteouts":0,"num_read":2275,"num_read_kb":52180,"num_write":804602,"num_write_kb":16339528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":176496,"ondisk_log_size":176496,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":17086924,"kb_used_data":9699040,"kb_used_omap":12,"kb_used_meta":7384371,"kb_avail":737887796,"statfs":{"total":773094113280,"available":755597103104,"internally_reserved":0,"allocated":9931816960,"data_stored":22902620864,"data_compressed":1090594739,"data_compressed_allocated":1529430016,"data_compressed_original":14502305792,"omap_allocated":12741,"internal_metadata":7561596475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1832536528,"num_objects":-438,"num_object_clones":0,"num_object_copies":-1752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-438,"num_whiteouts":0,"num_read":7,"num_read_kb":21,"num_write":7455,"num_write_kb":28720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004697"},"pg_stats":[{"pgid":"2.1f","version":"43'464","reported_seq":542,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:57.832117+0000","last_change":"2024-09-13T18:26:31.972797+0000","last_active":"2024-09-13T18:56:57.832117+0000","last_peered":"2024-09-13T18:56:57.832117+0000","last_clean":"2024-09-13T18:56:57.832117+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:56:57.832117+0000","last_undegraded":"2024-09-13T18:56:57.832117+0000","last_fullsized":"2024-09-13T18:56:57.832117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:50.511747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3814072,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":428,"num_write":455,"num_write_kb":3981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"29'4","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.078641+0000","last_change":"2024-09-13T18:26:32.349901+0000","last_active":"2024-09-13T18:55:26.078641+0000","last_peered":"2024-09-13T18:55:26.078641+0000","last_clean":"2024-09-13T18:55:26.078641+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:55:26.078641+0000","last_undegraded":"2024-09-13T18:55:26.078641+0000","last_fullsized":"2024-09-13T18:55:26.078641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:24.978813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"43'5677","reported_seq":5755,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.295010+0000","last_change":"2024-09-13T18:26:36.407579+0000","last_active":"2024-09-13T18:57:00.295010+0000","last_peered":"2024-09-13T18:57:00.295010+0000","last_clean":"2024-09-13T18:57:00.295010+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:57:00.295010+0000","last_undegraded":"2024-09-13T18:57:00.295010+0000","last_fullsized":"2024-09-13T18:57:00.295010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5677,"log_dups_size":0,"ondisk_log_size":5677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:53:56.370013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348712512,"num_objects":93,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":26674,"num_write_kb":537654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"41'12","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.079133+0000","last_change":"2024-09-13T18:26:31.980716+0000","last_active":"2024-09-13T18:55:26.079133+0000","last_peered":"2024-09-13T18:55:26.079133+0000","last_clean":"2024-09-13T18:55:26.079133+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:55:26.079133+0000","last_undegraded":"2024-09-13T18:55:26.079133+0000","last_fullsized":"2024-09-13T18:55:26.079133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:20.020903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"43'5","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:40.890683+0000","last_change":"2024-09-13T18:26:32.342690+0000","last_active":"2024-09-13T18:55:40.890683+0000","last_peered":"2024-09-13T18:55:40.890683+0000","last_clean":"2024-09-13T18:55:40.890683+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:55:40.890683+0000","last_undegraded":"2024-09-13T18:55:40.890683+0000","last_fullsized":"2024-09-13T18:55:40.890683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.212930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"43'5724","reported_seq":5795,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.404837+0000","last_change":"2024-09-13T18:26:36.396545+0000","last_active":"2024-09-13T18:57:00.404837+0000","last_peered":"2024-09-13T18:57:00.404837+0000","last_clean":"2024-09-13T18:57:00.404837+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:57:00.404837+0000","last_undegraded":"2024-09-13T18:57:00.404837+0000","last_fullsized":"2024-09-13T18:57:00.404837+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5724,"log_dups_size":0,"ondisk_log_size":5724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:38.760215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394351840,"num_objects":99,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":26684,"num_write_kb":545346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"43'23","reported_seq":83,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:35.970133+0000","last_change":"2024-09-13T18:26:31.977823+0000","last_active":"2024-09-13T18:56:35.970133+0000","last_peered":"2024-09-13T18:56:35.970133+0000","last_clean":"2024-09-13T18:56:35.970133+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:56:35.970133+0000","last_undegraded":"2024-09-13T18:56:35.970133+0000","last_fullsized":"2024-09-13T18:56:35.970133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:46.956527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":65,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"29'8","reported_seq":45,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T18:54:56.048253+0000","last_change":"2024-09-13T18:27:22.609917+0000","last_active":"2024-09-13T18:54:56.048253+0000","last_peered":"2024-09-13T18:54:56.048253+0000","last_clean":"2024-09-13T18:54:56.048253+0000","last_became_active":"2024-09-13T18:27:22.609625+0000","last_became_peered":"2024-09-13T18:27:22.609625+0000","last_unstale":"2024-09-13T18:54:56.048253+0000","last_undegraded":"2024-09-13T18:54:56.048253+0000","last_fullsized":"2024-09-13T18:54:56.048253+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:53.184214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"43'4801","reported_seq":4863,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.146983+0000","last_change":"2024-09-13T18:26:36.406675+0000","last_active":"2024-09-13T18:57:00.146983+0000","last_peered":"2024-09-13T18:57:00.146983+0000","last_clean":"2024-09-13T18:57:00.146983+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:57:00.146983+0000","last_undegraded":"2024-09-13T18:57:00.146983+0000","last_fullsized":"2024-09-13T18:57:00.146983+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4801,"log_dups_size":0,"ondisk_log_size":4801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:36.896027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327189222,"num_objects":86,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21538,"num_write_kb":467229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"38'5","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983854+0000","last_change":"2024-09-13T18:26:31.979860+0000","last_active":"2024-09-13T18:55:25.983854+0000","last_peered":"2024-09-13T18:55:25.983854+0000","last_clean":"2024-09-13T18:55:25.983854+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:55:25.983854+0000","last_undegraded":"2024-09-13T18:55:25.983854+0000","last_fullsized":"2024-09-13T18:55:25.983854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:45.323631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":9,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"43'7","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:47.612235+0000","last_change":"2024-09-13T18:26:32.351494+0000","last_active":"2024-09-13T18:56:47.612235+0000","last_peered":"2024-09-13T18:56:47.612235+0000","last_clean":"2024-09-13T18:56:47.612235+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:56:47.612235+0000","last_undegraded":"2024-09-13T18:56:47.612235+0000","last_fullsized":"2024-09-13T18:56:47.612235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:52.804526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"43'4723","reported_seq":4785,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.301135+0000","last_change":"2024-09-13T18:26:36.398763+0000","last_active":"2024-09-13T18:57:00.301135+0000","last_peered":"2024-09-13T18:57:00.301135+0000","last_clean":"2024-09-13T18:57:00.301135+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:57:00.301135+0000","last_undegraded":"2024-09-13T18:57:00.301135+0000","last_fullsized":"2024-09-13T18:57:00.301135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4723,"log_dups_size":0,"ondisk_log_size":4723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:35.417438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302004026,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21549,"num_write_kb":434281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"29'7","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T18:54:52.195299+0000","last_change":"2024-09-13T18:26:32.360976+0000","last_active":"2024-09-13T18:54:52.195299+0000","last_peered":"2024-09-13T18:54:52.195299+0000","last_clean":"2024-09-13T18:54:52.195299+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:54:52.195299+0000","last_undegraded":"2024-09-13T18:54:52.195299+0000","last_fullsized":"2024-09-13T18:54:52.195299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:53:52.373515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"43'1217","reported_seq":1289,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.163136+0000","last_change":"2024-09-13T18:26:31.977891+0000","last_active":"2024-09-13T18:55:53.163136+0000","last_peered":"2024-09-13T18:55:53.163136+0000","last_clean":"2024-09-13T18:55:53.163136+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:55:53.163136+0000","last_undegraded":"2024-09-13T18:55:53.163136+0000","last_fullsized":"2024-09-13T18:55:53.163136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:01.453094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":10235,"num_write":1211,"num_write_kb":4423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"43'5284","reported_seq":5350,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.312723+0000","last_change":"2024-09-13T18:26:36.397484+0000","last_active":"2024-09-13T18:57:00.312723+0000","last_peered":"2024-09-13T18:57:00.312723+0000","last_clean":"2024-09-13T18:57:00.312723+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:57:00.312723+0000","last_undegraded":"2024-09-13T18:57:00.312723+0000","last_fullsized":"2024-09-13T18:57:00.312723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5284,"log_dups_size":0,"ondisk_log_size":5284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:44.686244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356551933,"num_objects":93,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":24035,"num_write_kb":505876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"29'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.000833+0000","last_change":"2024-09-13T18:26:32.345564+0000","last_active":"2024-09-13T18:55:26.000833+0000","last_peered":"2024-09-13T18:55:26.000833+0000","last_clean":"2024-09-13T18:55:26.000833+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:55:26.000833+0000","last_undegraded":"2024-09-13T18:55:26.000833+0000","last_fullsized":"2024-09-13T18:55:26.000833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:54.331806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"43'10","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:33.983335+0000","last_change":"2024-09-13T18:26:31.974107+0000","last_active":"2024-09-13T18:55:33.983335+0000","last_peered":"2024-09-13T18:55:33.983335+0000","last_clean":"2024-09-13T18:55:33.983335+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:55:33.983335+0000","last_undegraded":"2024-09-13T18:55:33.983335+0000","last_fullsized":"2024-09-13T18:55:33.983335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:37.547480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"43'5259","reported_seq":5328,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.452338+0000","last_change":"2024-09-13T18:26:36.405986+0000","last_active":"2024-09-13T18:57:00.452338+0000","last_peered":"2024-09-13T18:57:00.452338+0000","last_clean":"2024-09-13T18:57:00.452338+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:57:00.452338+0000","last_undegraded":"2024-09-13T18:57:00.452338+0000","last_fullsized":"2024-09-13T18:57:00.452338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5259,"log_dups_size":0,"ondisk_log_size":5259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:30.052430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348141733,"num_objects":90,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":24322,"num_write_kb":498911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"43'9","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:36.128739+0000","last_change":"2024-09-13T18:27:22.609984+0000","last_active":"2024-09-13T18:55:36.128739+0000","last_peered":"2024-09-13T18:55:36.128739+0000","last_clean":"2024-09-13T18:55:36.128739+0000","last_became_active":"2024-09-13T18:27:22.609642+0000","last_became_peered":"2024-09-13T18:27:22.609642+0000","last_unstale":"2024-09-13T18:55:36.128739+0000","last_undegraded":"2024-09-13T18:55:36.128739+0000","last_fullsized":"2024-09-13T18:55:36.128739+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:10.097398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.161825+0000","last_change":"2024-09-13T18:26:31.968895+0000","last_active":"2024-09-13T18:55:53.161825+0000","last_peered":"2024-09-13T18:55:53.161825+0000","last_clean":"2024-09-13T18:55:53.161825+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:55:53.161825+0000","last_undegraded":"2024-09-13T18:55:53.161825+0000","last_fullsized":"2024-09-13T18:55:53.161825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:15.525097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"43'6820","reported_seq":6889,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.075354+0000","last_change":"2024-09-13T18:27:22.610932+0000","last_active":"2024-09-13T18:57:01.075354+0000","last_peered":"2024-09-13T18:57:01.075354+0000","last_clean":"2024-09-13T18:57:01.075354+0000","last_became_active":"2024-09-13T18:27:22.610527+0000","last_became_peered":"2024-09-13T18:27:22.610527+0000","last_unstale":"2024-09-13T18:57:01.075354+0000","last_undegraded":"2024-09-13T18:57:01.075354+0000","last_fullsized":"2024-09-13T18:57:01.075354+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6820,"log_dups_size":0,"ondisk_log_size":6820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:49:12.851146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432038324,"num_objects":108,"num_object_clones":0,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":31702,"num_write_kb":622755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"29'4","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.987621+0000","last_change":"2024-09-13T18:26:32.360406+0000","last_active":"2024-09-13T18:55:25.987621+0000","last_peered":"2024-09-13T18:55:25.987621+0000","last_clean":"2024-09-13T18:55:25.987621+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:55:25.987621+0000","last_undegraded":"2024-09-13T18:55:25.987621+0000","last_fullsized":"2024-09-13T18:55:25.987621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:37.417707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"28'4","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.171564+0000","last_change":"2024-09-13T18:26:31.976872+0000","last_active":"2024-09-13T18:55:53.171564+0000","last_peered":"2024-09-13T18:55:53.171564+0000","last_clean":"2024-09-13T18:55:53.171564+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:55:53.171564+0000","last_undegraded":"2024-09-13T18:55:53.171564+0000","last_fullsized":"2024-09-13T18:55:53.171564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:26.163029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"43'5235","reported_seq":5312,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.409347+0000","last_change":"2024-09-13T18:27:22.648736+0000","last_active":"2024-09-13T18:57:00.409347+0000","last_peered":"2024-09-13T18:57:00.409347+0000","last_clean":"2024-09-13T18:57:00.409347+0000","last_became_active":"2024-09-13T18:27:22.648054+0000","last_became_peered":"2024-09-13T18:27:22.648054+0000","last_unstale":"2024-09-13T18:57:00.409347+0000","last_undegraded":"2024-09-13T18:57:00.409347+0000","last_fullsized":"2024-09-13T18:57:00.409347+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5235,"log_dups_size":0,"ondisk_log_size":5235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:11.855024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348247684,"num_objects":91,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":23534,"num_write_kb":471297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"29'14","reported_seq":59,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.025486+0000","last_change":"2024-09-13T18:26:32.343915+0000","last_active":"2024-09-13T18:55:26.025486+0000","last_peered":"2024-09-13T18:55:26.025486+0000","last_clean":"2024-09-13T18:55:26.025486+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:55:26.025486+0000","last_undegraded":"2024-09-13T18:55:26.025486+0000","last_fullsized":"2024-09-13T18:55:26.025486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:19.357243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"28'5","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.162612+0000","last_change":"2024-09-13T18:26:31.979944+0000","last_active":"2024-09-13T18:55:53.162612+0000","last_peered":"2024-09-13T18:55:53.162612+0000","last_clean":"2024-09-13T18:55:53.162612+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:55:53.162612+0000","last_undegraded":"2024-09-13T18:55:53.162612+0000","last_fullsized":"2024-09-13T18:55:53.162612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:21.721921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"43'5232","reported_seq":5303,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.336494+0000","last_change":"2024-09-13T18:26:36.410028+0000","last_active":"2024-09-13T18:57:00.336494+0000","last_peered":"2024-09-13T18:57:00.336494+0000","last_clean":"2024-09-13T18:57:00.336494+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:57:00.336494+0000","last_undegraded":"2024-09-13T18:57:00.336494+0000","last_fullsized":"2024-09-13T18:57:00.336494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5232,"log_dups_size":0,"ondisk_log_size":5232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:32.264341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352362513,"num_objects":88,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":24364,"num_write_kb":507343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"29'9","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.010706+0000","last_change":"2024-09-13T18:26:32.358980+0000","last_active":"2024-09-13T18:55:26.010706+0000","last_peered":"2024-09-13T18:55:26.010706+0000","last_clean":"2024-09-13T18:55:26.010706+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:55:26.010706+0000","last_undegraded":"2024-09-13T18:55:26.010706+0000","last_fullsized":"2024-09-13T18:55:26.010706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:08.748139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"43'283","reported_seq":345,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:52.834136+0000","last_change":"2024-09-13T18:26:31.968972+0000","last_active":"2024-09-13T18:56:52.834136+0000","last_peered":"2024-09-13T18:56:52.834136+0000","last_clean":"2024-09-13T18:56:52.834136+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:56:52.834136+0000","last_undegraded":"2024-09-13T18:56:52.834136+0000","last_fullsized":"2024-09-13T18:56:52.834136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:56.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":52,"num_write":622,"num_write_kb":212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"43'5922","reported_seq":6003,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.069738+0000","last_change":"2024-09-13T18:27:22.645275+0000","last_active":"2024-09-13T18:57:00.069738+0000","last_peered":"2024-09-13T18:57:00.069738+0000","last_clean":"2024-09-13T18:57:00.069738+0000","last_became_active":"2024-09-13T18:27:22.644681+0000","last_became_peered":"2024-09-13T18:27:22.644681+0000","last_unstale":"2024-09-13T18:57:00.069738+0000","last_undegraded":"2024-09-13T18:57:00.069738+0000","last_fullsized":"2024-09-13T18:57:00.069738+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5922,"log_dups_size":0,"ondisk_log_size":5922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:18:38.581700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394336124,"num_objects":101,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":27801,"num_write_kb":553947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"43'6","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:50.152562+0000","last_change":"2024-09-13T18:26:32.342823+0000","last_active":"2024-09-13T18:56:50.152562+0000","last_peered":"2024-09-13T18:56:50.152562+0000","last_clean":"2024-09-13T18:56:50.152562+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:56:50.152562+0000","last_undegraded":"2024-09-13T18:56:50.152562+0000","last_fullsized":"2024-09-13T18:56:50.152562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:28.051252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:30.981249+0000","last_change":"2024-09-13T18:26:31.978976+0000","last_active":"2024-09-13T18:55:30.981249+0000","last_peered":"2024-09-13T18:55:30.981249+0000","last_clean":"2024-09-13T18:55:30.981249+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:55:30.981249+0000","last_undegraded":"2024-09-13T18:55:30.981249+0000","last_fullsized":"2024-09-13T18:55:30.981249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:11.279557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"43'4621","reported_seq":4684,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.841241+0000","last_change":"2024-09-13T18:26:36.408844+0000","last_active":"2024-09-13T18:56:58.841241+0000","last_peered":"2024-09-13T18:56:58.841241+0000","last_clean":"2024-09-13T18:56:58.841241+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:56:58.841241+0000","last_undegraded":"2024-09-13T18:56:58.841241+0000","last_fullsized":"2024-09-13T18:56:58.841241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4621,"log_dups_size":0,"ondisk_log_size":4621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:06.969342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306187993,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":22360,"num_write_kb":450115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"28'7","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.969209+0000","last_change":"2024-09-13T18:26:32.355038+0000","last_active":"2024-09-13T18:55:25.969209+0000","last_peered":"2024-09-13T18:55:25.969209+0000","last_clean":"2024-09-13T18:55:25.969209+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:55:25.969209+0000","last_undegraded":"2024-09-13T18:55:25.969209+0000","last_fullsized":"2024-09-13T18:55:25.969209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.159231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.007762+0000","last_change":"2024-09-13T18:26:31.972931+0000","last_active":"2024-09-13T18:55:26.007762+0000","last_peered":"2024-09-13T18:55:26.007762+0000","last_clean":"2024-09-13T18:55:26.007762+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:55:26.007762+0000","last_undegraded":"2024-09-13T18:55:26.007762+0000","last_fullsized":"2024-09-13T18:55:26.007762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:05.646924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"43'6011","reported_seq":6080,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.360889+0000","last_change":"2024-09-13T18:26:36.398876+0000","last_active":"2024-09-13T18:57:01.360889+0000","last_peered":"2024-09-13T18:57:01.360889+0000","last_clean":"2024-09-13T18:57:01.360889+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:57:01.360889+0000","last_undegraded":"2024-09-13T18:57:01.360889+0000","last_fullsized":"2024-09-13T18:57:01.360889+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6011,"log_dups_size":0,"ondisk_log_size":6011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:56.918239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369124519,"num_objects":90,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":27977,"num_write_kb":562911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"28'6","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.982753+0000","last_change":"2024-09-13T18:26:32.353111+0000","last_active":"2024-09-13T18:55:25.982753+0000","last_peered":"2024-09-13T18:55:25.982753+0000","last_clean":"2024-09-13T18:55:25.982753+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:55:25.982753+0000","last_undegraded":"2024-09-13T18:55:25.982753+0000","last_fullsized":"2024-09-13T18:55:25.982753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:43.927936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"41'6","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.977227+0000","last_change":"2024-09-13T18:26:31.976075+0000","last_active":"2024-09-13T18:55:25.977227+0000","last_peered":"2024-09-13T18:55:25.977227+0000","last_clean":"2024-09-13T18:55:25.977227+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:55:25.977227+0000","last_undegraded":"2024-09-13T18:55:25.977227+0000","last_fullsized":"2024-09-13T18:55:25.977227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:03.693518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"43'5256","reported_seq":5328,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.125917+0000","last_change":"2024-09-13T18:26:36.401777+0000","last_active":"2024-09-13T18:57:01.125917+0000","last_peered":"2024-09-13T18:57:01.125917+0000","last_clean":"2024-09-13T18:57:01.125917+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:57:01.125917+0000","last_undegraded":"2024-09-13T18:57:01.125917+0000","last_fullsized":"2024-09-13T18:57:01.125917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5256,"log_dups_size":0,"ondisk_log_size":5256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:43.440284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343954983,"num_objects":87,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":24458,"num_write_kb":507684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"29'10","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978450+0000","last_change":"2024-09-13T18:26:32.343791+0000","last_active":"2024-09-13T18:55:25.978450+0000","last_peered":"2024-09-13T18:55:25.978450+0000","last_clean":"2024-09-13T18:55:25.978450+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:55:25.978450+0000","last_undegraded":"2024-09-13T18:55:25.978450+0000","last_fullsized":"2024-09-13T18:55:25.978450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:36.296571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"43'128","reported_seq":302,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.464900+0000","last_change":"2024-09-13T18:26:31.970786+0000","last_active":"2024-09-13T18:56:58.464900+0000","last_peered":"2024-09-13T18:56:58.464900+0000","last_clean":"2024-09-13T18:56:58.464900+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:56:58.464900+0000","last_undegraded":"2024-09-13T18:56:58.464900+0000","last_fullsized":"2024-09-13T18:56:58.464900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:54.584009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93931,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":203,"num_write":119,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"43'6142","reported_seq":6212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.392774+0000","last_change":"2024-09-13T18:26:36.401529+0000","last_active":"2024-09-13T18:57:01.392774+0000","last_peered":"2024-09-13T18:57:01.392774+0000","last_clean":"2024-09-13T18:57:01.392774+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:57:01.392774+0000","last_undegraded":"2024-09-13T18:57:01.392774+0000","last_fullsized":"2024-09-13T18:57:01.392774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6142,"log_dups_size":0,"ondisk_log_size":6142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:47.318347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377531496,"num_objects":99,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":28204,"num_write_kb":563741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"28'4","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.987962+0000","last_change":"2024-09-13T18:26:32.350470+0000","last_active":"2024-09-13T18:55:25.987962+0000","last_peered":"2024-09-13T18:55:25.987962+0000","last_clean":"2024-09-13T18:55:25.987962+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:55:25.987962+0000","last_undegraded":"2024-09-13T18:55:25.987962+0000","last_fullsized":"2024-09-13T18:55:25.987962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:55.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"43'14","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:31.126491+0000","last_change":"2024-09-13T18:26:31.972587+0000","last_active":"2024-09-13T18:55:31.126491+0000","last_peered":"2024-09-13T18:55:31.126491+0000","last_clean":"2024-09-13T18:55:31.126491+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:55:31.126491+0000","last_undegraded":"2024-09-13T18:55:31.126491+0000","last_fullsized":"2024-09-13T18:55:31.126491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:27.547763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"43'4295","reported_seq":4365,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.861128+0000","last_change":"2024-09-13T18:26:36.405086+0000","last_active":"2024-09-13T18:56:58.861128+0000","last_peered":"2024-09-13T18:56:58.861128+0000","last_clean":"2024-09-13T18:56:58.861128+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:56:58.861128+0000","last_undegraded":"2024-09-13T18:56:58.861128+0000","last_fullsized":"2024-09-13T18:56:58.861128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4295,"log_dups_size":0,"ondisk_log_size":4295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:02:51.694634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335811241,"num_objects":85,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":19777,"num_write_kb":435588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"29'4","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.994918+0000","last_change":"2024-09-13T18:26:32.350510+0000","last_active":"2024-09-13T18:55:25.994918+0000","last_peered":"2024-09-13T18:55:25.994918+0000","last_clean":"2024-09-13T18:55:25.994918+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:55:25.994918+0000","last_undegraded":"2024-09-13T18:55:25.994918+0000","last_fullsized":"2024-09-13T18:55:25.994918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:40.850255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"38'6","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:45.169259+0000","last_change":"2024-09-13T18:26:31.978355+0000","last_active":"2024-09-13T18:55:45.169259+0000","last_peered":"2024-09-13T18:55:45.169259+0000","last_clean":"2024-09-13T18:55:45.169259+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:55:45.169259+0000","last_undegraded":"2024-09-13T18:55:45.169259+0000","last_fullsized":"2024-09-13T18:55:45.169259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:12.946518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":40,"num_read_kb":68,"num_write":8,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983663+0000","last_change":"2024-09-13T18:26:26.931403+0000","last_active":"2024-09-13T18:55:25.983663+0000","last_peered":"2024-09-13T18:55:25.983663+0000","last_clean":"2024-09-13T18:55:25.983663+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:55:25.983663+0000","last_undegraded":"2024-09-13T18:55:25.983663+0000","last_fullsized":"2024-09-13T18:55:25.983663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_clean_scrub_stamp":"2024-09-13T18:26:24.880028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:30.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"43'5845","reported_seq":5924,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.292017+0000","last_change":"2024-09-13T18:26:36.409827+0000","last_active":"2024-09-13T18:57:00.292017+0000","last_peered":"2024-09-13T18:57:00.292017+0000","last_clean":"2024-09-13T18:57:00.292017+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:57:00.292017+0000","last_undegraded":"2024-09-13T18:57:00.292017+0000","last_fullsized":"2024-09-13T18:57:00.292017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5845,"log_dups_size":0,"ondisk_log_size":5845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:50.482400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377529592,"num_objects":94,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":27327,"num_write_kb":519330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"28'6","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984035+0000","last_change":"2024-09-13T18:26:32.356850+0000","last_active":"2024-09-13T18:55:25.984035+0000","last_peered":"2024-09-13T18:55:25.984035+0000","last_clean":"2024-09-13T18:55:25.984035+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:55:25.984035+0000","last_undegraded":"2024-09-13T18:55:25.984035+0000","last_fullsized":"2024-09-13T18:55:25.984035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:38.710828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"36'4","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.973775+0000","last_change":"2024-09-13T18:26:31.975942+0000","last_active":"2024-09-13T18:55:25.973775+0000","last_peered":"2024-09-13T18:55:25.973775+0000","last_clean":"2024-09-13T18:55:25.973775+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:55:25.973775+0000","last_undegraded":"2024-09-13T18:55:25.973775+0000","last_fullsized":"2024-09-13T18:55:25.973775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:05.840582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":35,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"43'5261","reported_seq":5324,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.262169+0000","last_change":"2024-09-13T18:26:36.409802+0000","last_active":"2024-09-13T18:57:01.262169+0000","last_peered":"2024-09-13T18:57:01.262169+0000","last_clean":"2024-09-13T18:57:01.262169+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:57:01.262169+0000","last_undegraded":"2024-09-13T18:57:01.262169+0000","last_fullsized":"2024-09-13T18:57:01.262169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5261,"log_dups_size":0,"ondisk_log_size":5261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:38.178151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348128716,"num_objects":85,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":23982,"num_write_kb":513904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"29'8","reported_seq":47,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.973293+0000","last_change":"2024-09-13T18:27:22.610607+0000","last_active":"2024-09-13T18:55:25.973293+0000","last_peered":"2024-09-13T18:55:25.973293+0000","last_clean":"2024-09-13T18:55:25.973293+0000","last_became_active":"2024-09-13T18:27:22.610326+0000","last_became_peered":"2024-09-13T18:27:22.610326+0000","last_unstale":"2024-09-13T18:55:25.973293+0000","last_undegraded":"2024-09-13T18:55:25.973293+0000","last_fullsized":"2024-09-13T18:55:25.973293+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:38:04.117608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"38'2","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978777+0000","last_change":"2024-09-13T18:26:31.966618+0000","last_active":"2024-09-13T18:55:25.978777+0000","last_peered":"2024-09-13T18:55:25.978777+0000","last_clean":"2024-09-13T18:55:25.978777+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:55:25.978777+0000","last_undegraded":"2024-09-13T18:55:25.978777+0000","last_fullsized":"2024-09-13T18:55:25.978777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:54:26.422611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"43'4977","reported_seq":5044,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.799194+0000","last_change":"2024-09-13T18:26:36.394344+0000","last_active":"2024-09-13T18:56:58.799194+0000","last_peered":"2024-09-13T18:56:58.799194+0000","last_clean":"2024-09-13T18:56:58.799194+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:56:58.799194+0000","last_undegraded":"2024-09-13T18:56:58.799194+0000","last_fullsized":"2024-09-13T18:56:58.799194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4977,"log_dups_size":0,"ondisk_log_size":4977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:18.425816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360783295,"num_objects":94,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":22736,"num_write_kb":472029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"29'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984452+0000","last_change":"2024-09-13T18:26:32.365655+0000","last_active":"2024-09-13T18:55:25.984452+0000","last_peered":"2024-09-13T18:55:25.984452+0000","last_clean":"2024-09-13T18:55:25.984452+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:55:25.984452+0000","last_undegraded":"2024-09-13T18:55:25.984452+0000","last_fullsized":"2024-09-13T18:55:25.984452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:40.288128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'4","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.973070+0000","last_change":"2024-09-13T18:26:31.970652+0000","last_active":"2024-09-13T18:55:25.973070+0000","last_peered":"2024-09-13T18:55:25.973070+0000","last_clean":"2024-09-13T18:55:25.973070+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:55:25.973070+0000","last_undegraded":"2024-09-13T18:55:25.973070+0000","last_fullsized":"2024-09-13T18:55:25.973070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:11.169661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"43'4832","reported_seq":4893,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.526006+0000","last_change":"2024-09-13T18:26:36.410280+0000","last_active":"2024-09-13T18:56:59.526006+0000","last_peered":"2024-09-13T18:56:59.526006+0000","last_clean":"2024-09-13T18:56:59.526006+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:56:59.526006+0000","last_undegraded":"2024-09-13T18:56:59.526006+0000","last_fullsized":"2024-09-13T18:56:59.526006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4832,"log_dups_size":0,"ondisk_log_size":4832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:01.395290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327189351,"num_objects":82,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":17,"num_read_kb":0,"num_write":22693,"num_write_kb":463670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"28'3","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984106+0000","last_change":"2024-09-13T18:26:32.361575+0000","last_active":"2024-09-13T18:55:25.984106+0000","last_peered":"2024-09-13T18:55:25.984106+0000","last_clean":"2024-09-13T18:55:25.984106+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:55:25.984106+0000","last_undegraded":"2024-09-13T18:55:25.984106+0000","last_fullsized":"2024-09-13T18:55:25.984106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:08.818877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"41'9","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983121+0000","last_change":"2024-09-13T18:26:31.977029+0000","last_active":"2024-09-13T18:55:25.983121+0000","last_peered":"2024-09-13T18:55:25.983121+0000","last_clean":"2024-09-13T18:55:25.983121+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:55:25.983121+0000","last_undegraded":"2024-09-13T18:55:25.983121+0000","last_fullsized":"2024-09-13T18:55:25.983121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:13.701689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"43'4972","reported_seq":5037,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.287176+0000","last_change":"2024-09-13T18:26:36.403479+0000","last_active":"2024-09-13T18:57:00.287176+0000","last_peered":"2024-09-13T18:57:00.287176+0000","last_clean":"2024-09-13T18:57:00.287176+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:57:00.287176+0000","last_undegraded":"2024-09-13T18:57:00.287176+0000","last_fullsized":"2024-09-13T18:57:00.287176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:51.095852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327213309,"num_objects":84,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":22631,"num_write_kb":463382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"29'9","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984952+0000","last_change":"2024-09-13T18:26:32.365143+0000","last_active":"2024-09-13T18:55:25.984952+0000","last_peered":"2024-09-13T18:55:25.984952+0000","last_clean":"2024-09-13T18:55:25.984952+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:55:25.984952+0000","last_undegraded":"2024-09-13T18:55:25.984952+0000","last_fullsized":"2024-09-13T18:55:25.984952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:46.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"41'21","reported_seq":126,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.152848+0000","last_change":"2024-09-13T18:26:31.965802+0000","last_active":"2024-09-13T18:55:53.152848+0000","last_peered":"2024-09-13T18:55:53.152848+0000","last_clean":"2024-09-13T18:55:53.152848+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:55:53.152848+0000","last_undegraded":"2024-09-13T18:55:53.152848+0000","last_fullsized":"2024-09-13T18:55:53.152848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:12.759181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":158,"num_read_kb":157,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"43'5873","reported_seq":5946,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.072818+0000","last_change":"2024-09-13T18:26:36.399235+0000","last_active":"2024-09-13T18:56:59.072818+0000","last_peered":"2024-09-13T18:56:59.072818+0000","last_clean":"2024-09-13T18:56:59.072818+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:56:59.072818+0000","last_undegraded":"2024-09-13T18:56:59.072818+0000","last_fullsized":"2024-09-13T18:56:59.072818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5873,"log_dups_size":0,"ondisk_log_size":5873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:13:54.711083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381775740,"num_objects":100,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":27640,"num_write_kb":536907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"43'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:56.844127+0000","last_change":"2024-09-13T18:26:32.357403+0000","last_active":"2024-09-13T18:56:56.844127+0000","last_peered":"2024-09-13T18:56:56.844127+0000","last_clean":"2024-09-13T18:56:56.844127+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:56:56.844127+0000","last_undegraded":"2024-09-13T18:56:56.844127+0000","last_fullsized":"2024-09-13T18:56:56.844127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:12.600397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"41'12","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.982231+0000","last_change":"2024-09-13T18:26:31.976148+0000","last_active":"2024-09-13T18:55:25.982231+0000","last_peered":"2024-09-13T18:55:25.982231+0000","last_clean":"2024-09-13T18:55:25.982231+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:55:25.982231+0000","last_undegraded":"2024-09-13T18:55:25.982231+0000","last_fullsized":"2024-09-13T18:55:25.982231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:38.023636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1620,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"43'5978","reported_seq":6044,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.401591+0000","last_change":"2024-09-13T18:26:36.406177+0000","last_active":"2024-09-13T18:57:00.401591+0000","last_peered":"2024-09-13T18:57:00.401591+0000","last_clean":"2024-09-13T18:57:00.401591+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:57:00.401591+0000","last_undegraded":"2024-09-13T18:57:00.401591+0000","last_fullsized":"2024-09-13T18:57:00.401591+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5978,"log_dups_size":0,"ondisk_log_size":5978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:00.309995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369133062,"num_objects":92,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":28176,"num_write_kb":551294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"43'12","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978424+0000","last_change":"2024-09-13T18:27:22.609309+0000","last_active":"2024-09-13T18:55:25.978424+0000","last_peered":"2024-09-13T18:55:25.978424+0000","last_clean":"2024-09-13T18:55:25.978424+0000","last_became_active":"2024-09-13T18:27:22.609101+0000","last_became_peered":"2024-09-13T18:27:22.609101+0000","last_unstale":"2024-09-13T18:55:25.978424+0000","last_undegraded":"2024-09-13T18:55:25.978424+0000","last_fullsized":"2024-09-13T18:55:25.978424+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:15.692612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"43'10","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.165844+0000","last_change":"2024-09-13T18:26:31.982114+0000","last_active":"2024-09-13T18:55:53.165844+0000","last_peered":"2024-09-13T18:55:53.165844+0000","last_clean":"2024-09-13T18:55:53.165844+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:55:53.165844+0000","last_undegraded":"2024-09-13T18:55:53.165844+0000","last_fullsized":"2024-09-13T18:55:53.165844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.266528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"43'5767","reported_seq":5836,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.376830+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:57:00.376830+0000","last_peered":"2024-09-13T18:57:00.376830+0000","last_clean":"2024-09-13T18:57:00.376830+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:57:00.376830+0000","last_undegraded":"2024-09-13T18:57:00.376830+0000","last_fullsized":"2024-09-13T18:57:00.376830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5767,"log_dups_size":0,"ondisk_log_size":5767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:58.853971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377595894,"num_objects":98,"num_object_clones":0,"num_object_copies":392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27027,"num_write_kb":550847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"28'8","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984791+0000","last_change":"2024-09-13T18:26:32.351145+0000","last_active":"2024-09-13T18:55:25.984791+0000","last_peered":"2024-09-13T18:55:25.984791+0000","last_clean":"2024-09-13T18:55:25.984791+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:55:25.984791+0000","last_undegraded":"2024-09-13T18:55:25.984791+0000","last_fullsized":"2024-09-13T18:55:25.984791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:57.002435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"38'9","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.158612+0000","last_change":"2024-09-13T18:26:31.961202+0000","last_active":"2024-09-13T18:55:53.158612+0000","last_peered":"2024-09-13T18:55:53.158612+0000","last_clean":"2024-09-13T18:55:53.158612+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:55:53.158612+0000","last_undegraded":"2024-09-13T18:55:53.158612+0000","last_fullsized":"2024-09-13T18:55:53.158612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:27.340127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":93,"num_read_kb":112,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"43'5137","reported_seq":5202,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.582316+0000","last_change":"2024-09-13T18:26:36.410014+0000","last_active":"2024-09-13T18:56:59.582316+0000","last_peered":"2024-09-13T18:56:59.582316+0000","last_clean":"2024-09-13T18:56:59.582316+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:56:59.582316+0000","last_undegraded":"2024-09-13T18:56:59.582316+0000","last_fullsized":"2024-09-13T18:56:59.582316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5137,"log_dups_size":0,"ondisk_log_size":5137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:24.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767127,"num_objects":77,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":23163,"num_write_kb":490059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"29'9","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983909+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:55:25.983909+0000","last_peered":"2024-09-13T18:55:25.983909+0000","last_clean":"2024-09-13T18:55:25.983909+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:55:25.983909+0000","last_undegraded":"2024-09-13T18:55:25.983909+0000","last_fullsized":"2024-09-13T18:55:25.983909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:40.102815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"43'381","reported_seq":468,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:47.848809+0000","last_change":"2024-09-13T18:26:31.974110+0000","last_active":"2024-09-13T18:56:47.848809+0000","last_peered":"2024-09-13T18:56:47.848809+0000","last_clean":"2024-09-13T18:56:47.848809+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:56:47.848809+0000","last_undegraded":"2024-09-13T18:56:47.848809+0000","last_fullsized":"2024-09-13T18:56:47.848809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:56.058561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":93,"num_read_kb":20568,"num_write":381,"num_write_kb":4317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"43'5306","reported_seq":5372,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.367559+0000","last_change":"2024-09-13T18:26:36.394080+0000","last_active":"2024-09-13T18:57:00.367559+0000","last_peered":"2024-09-13T18:57:00.367559+0000","last_clean":"2024-09-13T18:57:00.367559+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:57:00.367559+0000","last_undegraded":"2024-09-13T18:57:00.367559+0000","last_fullsized":"2024-09-13T18:57:00.367559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5306,"log_dups_size":0,"ondisk_log_size":5306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:47.121814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335673691,"num_objects":86,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":24948,"num_write_kb":485267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"43'4","reported_seq":47,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.971630+0000","last_change":"2024-09-13T18:26:32.362873+0000","last_active":"2024-09-13T18:55:25.971630+0000","last_peered":"2024-09-13T18:55:25.971630+0000","last_clean":"2024-09-13T18:55:25.971630+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:55:25.971630+0000","last_undegraded":"2024-09-13T18:55:25.971630+0000","last_fullsized":"2024-09-13T18:55:25.971630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:37.598189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"43'61","reported_seq":123,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:46.092289+0000","last_change":"2024-09-13T18:26:31.976401+0000","last_active":"2024-09-13T18:56:46.092289+0000","last_peered":"2024-09-13T18:56:46.092289+0000","last_clean":"2024-09-13T18:56:46.092289+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:56:46.092289+0000","last_undegraded":"2024-09-13T18:56:46.092289+0000","last_fullsized":"2024-09-13T18:56:46.092289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:26.434292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":62,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"43'5783","reported_seq":5853,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.356403+0000","last_change":"2024-09-13T18:26:36.406228+0000","last_active":"2024-09-13T18:57:00.356403+0000","last_peered":"2024-09-13T18:57:00.356403+0000","last_clean":"2024-09-13T18:57:00.356403+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:57:00.356403+0000","last_undegraded":"2024-09-13T18:57:00.356403+0000","last_fullsized":"2024-09-13T18:57:00.356403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5783,"log_dups_size":0,"ondisk_log_size":5783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:49.198397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352479214,"num_objects":93,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27380,"num_write_kb":527755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"29'9","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.987447+0000","last_change":"2024-09-13T18:26:32.350841+0000","last_active":"2024-09-13T18:55:25.987447+0000","last_peered":"2024-09-13T18:55:25.987447+0000","last_clean":"2024-09-13T18:55:25.987447+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:55:25.987447+0000","last_undegraded":"2024-09-13T18:55:25.987447+0000","last_fullsized":"2024-09-13T18:55:25.987447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:05.137882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'3","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:52.993936+0000","last_change":"2024-09-13T18:26:31.978214+0000","last_active":"2024-09-13T18:55:52.993936+0000","last_peered":"2024-09-13T18:55:52.993936+0000","last_clean":"2024-09-13T18:55:52.993936+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:55:52.993936+0000","last_undegraded":"2024-09-13T18:55:52.993936+0000","last_fullsized":"2024-09-13T18:55:52.993936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:17.381375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"43'5336","reported_seq":5397,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.340297+0000","last_change":"2024-09-13T18:26:36.406090+0000","last_active":"2024-09-13T18:57:00.340297+0000","last_peered":"2024-09-13T18:57:00.340297+0000","last_clean":"2024-09-13T18:57:00.340297+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:57:00.340297+0000","last_undegraded":"2024-09-13T18:57:00.340297+0000","last_fullsized":"2024-09-13T18:57:00.340297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:09.865147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327181663,"num_objects":85,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":24960,"num_write_kb":489215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"29'5","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.969146+0000","last_change":"2024-09-13T18:26:32.357701+0000","last_active":"2024-09-13T18:55:25.969146+0000","last_peered":"2024-09-13T18:55:25.969146+0000","last_clean":"2024-09-13T18:55:25.969146+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:55:25.969146+0000","last_undegraded":"2024-09-13T18:55:25.969146+0000","last_fullsized":"2024-09-13T18:55:25.969146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:47.587683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"43'15","reported_seq":81,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:46.038602+0000","last_change":"2024-09-13T18:26:31.980133+0000","last_active":"2024-09-13T18:56:46.038602+0000","last_peered":"2024-09-13T18:56:46.038602+0000","last_clean":"2024-09-13T18:56:46.038602+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:56:46.038602+0000","last_undegraded":"2024-09-13T18:56:46.038602+0000","last_fullsized":"2024-09-13T18:56:46.038602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:05.354335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":16,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"43'4996","reported_seq":5064,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.435081+0000","last_change":"2024-09-13T18:26:36.406553+0000","last_active":"2024-09-13T18:57:00.435081+0000","last_peered":"2024-09-13T18:57:00.435081+0000","last_clean":"2024-09-13T18:57:00.435081+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:57:00.435081+0000","last_undegraded":"2024-09-13T18:57:00.435081+0000","last_fullsized":"2024-09-13T18:57:00.435081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4996,"log_dups_size":0,"ondisk_log_size":4996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:02.980826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":365112283,"num_objects":94,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":23246,"num_write_kb":507027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"43'5990","reported_seq":6060,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.955468+0000","last_change":"2024-09-13T18:26:36.406986+0000","last_active":"2024-09-13T18:56:58.955468+0000","last_peered":"2024-09-13T18:56:58.955468+0000","last_clean":"2024-09-13T18:56:58.955468+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:56:58.955468+0000","last_undegraded":"2024-09-13T18:56:58.955468+0000","last_fullsized":"2024-09-13T18:56:58.955468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5990,"log_dups_size":0,"ondisk_log_size":5990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:53.984231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419441008,"num_objects":103,"num_object_clones":0,"num_object_copies":412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":28460,"num_write_kb":581141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"29'4","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978161+0000","last_change":"2024-09-13T18:26:32.349081+0000","last_active":"2024-09-13T18:55:25.978161+0000","last_peered":"2024-09-13T18:55:25.978161+0000","last_clean":"2024-09-13T18:55:25.978161+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:55:25.978161+0000","last_undegraded":"2024-09-13T18:55:25.978161+0000","last_fullsized":"2024-09-13T18:55:25.978161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:47.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"43'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:31.137406+0000","last_change":"2024-09-13T18:26:31.968944+0000","last_active":"2024-09-13T18:55:31.137406+0000","last_peered":"2024-09-13T18:55:31.137406+0000","last_clean":"2024-09-13T18:55:31.137406+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:55:31.137406+0000","last_undegraded":"2024-09-13T18:55:31.137406+0000","last_fullsized":"2024-09-13T18:55:31.137406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:29.363312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"43'5472","reported_seq":5544,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.453436+0000","last_change":"2024-09-13T18:26:36.396418+0000","last_active":"2024-09-13T18:57:00.453436+0000","last_peered":"2024-09-13T18:57:00.453436+0000","last_clean":"2024-09-13T18:57:00.453436+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:57:00.453436+0000","last_undegraded":"2024-09-13T18:57:00.453436+0000","last_fullsized":"2024-09-13T18:57:00.453436+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5472,"log_dups_size":0,"ondisk_log_size":5472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:03.252773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":340075764,"num_objects":93,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":25514,"num_write_kb":520259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"29'14","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.111521+0000","last_change":"2024-09-13T18:26:32.351369+0000","last_active":"2024-09-13T18:55:26.111521+0000","last_peered":"2024-09-13T18:55:26.111521+0000","last_clean":"2024-09-13T18:55:26.111521+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:55:26.111521+0000","last_undegraded":"2024-09-13T18:55:26.111521+0000","last_fullsized":"2024-09-13T18:55:26.111521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:10.892347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"36'4","reported_seq":59,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.001369+0000","last_change":"2024-09-13T18:26:31.972048+0000","last_active":"2024-09-13T18:55:26.001369+0000","last_peered":"2024-09-13T18:55:26.001369+0000","last_clean":"2024-09-13T18:55:26.001369+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:55:26.001369+0000","last_undegraded":"2024-09-13T18:55:26.001369+0000","last_fullsized":"2024-09-13T18:55:26.001369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:28.430119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"41'9","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981371+0000","last_change":"2024-09-13T18:26:31.975310+0000","last_active":"2024-09-13T18:55:25.981371+0000","last_peered":"2024-09-13T18:55:25.981371+0000","last_clean":"2024-09-13T18:55:25.981371+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:55:25.981371+0000","last_undegraded":"2024-09-13T18:55:25.981371+0000","last_fullsized":"2024-09-13T18:55:25.981371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:52.915310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"29'7","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.999839+0000","last_change":"2024-09-13T18:26:32.345825+0000","last_active":"2024-09-13T18:55:25.999839+0000","last_peered":"2024-09-13T18:55:25.999839+0000","last_clean":"2024-09-13T18:55:25.999839+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:55:25.999839+0000","last_undegraded":"2024-09-13T18:55:25.999839+0000","last_fullsized":"2024-09-13T18:55:25.999839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:25.200489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"43'4065","reported_seq":4127,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.144665+0000","last_change":"2024-09-13T18:26:36.391520+0000","last_active":"2024-09-13T18:57:01.144665+0000","last_peered":"2024-09-13T18:57:01.144665+0000","last_clean":"2024-09-13T18:57:01.144665+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:57:01.144665+0000","last_undegraded":"2024-09-13T18:57:01.144665+0000","last_fullsized":"2024-09-13T18:57:01.144665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4065,"log_dups_size":0,"ondisk_log_size":4065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:42.638713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255876134,"num_objects":63,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":19156,"num_write_kb":424944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"43'754","reported_seq":826,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:35.286998+0000","last_change":"2024-09-13T18:26:31.976603+0000","last_active":"2024-09-13T18:56:35.286998+0000","last_peered":"2024-09-13T18:56:35.286998+0000","last_clean":"2024-09-13T18:56:35.286998+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:56:35.286998+0000","last_undegraded":"2024-09-13T18:56:35.286998+0000","last_fullsized":"2024-09-13T18:56:35.286998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:03.965376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196273,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":19627,"num_write":735,"num_write_kb":4345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"29'9","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.979121+0000","last_change":"2024-09-13T18:26:32.352982+0000","last_active":"2024-09-13T18:55:25.979121+0000","last_peered":"2024-09-13T18:55:25.979121+0000","last_clean":"2024-09-13T18:55:25.979121+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:55:25.979121+0000","last_undegraded":"2024-09-13T18:55:25.979121+0000","last_fullsized":"2024-09-13T18:55:25.979121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:55.670360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"43'5828","reported_seq":5896,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.118020+0000","last_change":"2024-09-13T18:26:36.403817+0000","last_active":"2024-09-13T18:57:00.118020+0000","last_peered":"2024-09-13T18:57:00.118020+0000","last_clean":"2024-09-13T18:57:00.118020+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:57:00.118020+0000","last_undegraded":"2024-09-13T18:57:00.118020+0000","last_fullsized":"2024-09-13T18:57:00.118020+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5828,"log_dups_size":0,"ondisk_log_size":5828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:34.892901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377551906,"num_objects":98,"num_object_clones":0,"num_object_copies":392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":26701,"num_write_kb":557751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"43'8","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:47.227020+0000","last_change":"2024-09-13T18:26:32.357856+0000","last_active":"2024-09-13T18:55:47.227020+0000","last_peered":"2024-09-13T18:55:47.227020+0000","last_clean":"2024-09-13T18:55:47.227020+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:55:47.227020+0000","last_undegraded":"2024-09-13T18:55:47.227020+0000","last_fullsized":"2024-09-13T18:55:47.227020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:26.316214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"29'10","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.972396+0000","last_change":"2024-09-13T18:26:32.350966+0000","last_active":"2024-09-13T18:55:25.972396+0000","last_peered":"2024-09-13T18:55:25.972396+0000","last_clean":"2024-09-13T18:55:25.972396+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:55:25.972396+0000","last_undegraded":"2024-09-13T18:55:25.972396+0000","last_fullsized":"2024-09-13T18:55:25.972396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:52.632112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"29'11","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.977018+0000","last_change":"2024-09-13T18:26:32.355806+0000","last_active":"2024-09-13T18:55:25.977018+0000","last_peered":"2024-09-13T18:55:25.977018+0000","last_clean":"2024-09-13T18:55:25.977018+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:55:25.977018+0000","last_undegraded":"2024-09-13T18:55:25.977018+0000","last_fullsized":"2024-09-13T18:55:25.977018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.988231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"29'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.999728+0000","last_change":"2024-09-13T18:26:32.340965+0000","last_active":"2024-09-13T18:55:25.999728+0000","last_peered":"2024-09-13T18:55:25.999728+0000","last_clean":"2024-09-13T18:55:25.999728+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:55:25.999728+0000","last_undegraded":"2024-09-13T18:55:25.999728+0000","last_fullsized":"2024-09-13T18:55:25.999728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:01:04.716919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"29'7","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.046037+0000","last_change":"2024-09-13T18:26:32.349625+0000","last_active":"2024-09-13T18:55:26.046037+0000","last_peered":"2024-09-13T18:55:26.046037+0000","last_clean":"2024-09-13T18:55:26.046037+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:55:26.046037+0000","last_undegraded":"2024-09-13T18:55:26.046037+0000","last_fullsized":"2024-09-13T18:55:26.046037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.519754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"28'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978502+0000","last_change":"2024-09-13T18:26:32.349170+0000","last_active":"2024-09-13T18:55:25.978502+0000","last_peered":"2024-09-13T18:55:25.978502+0000","last_clean":"2024-09-13T18:55:25.978502+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:55:25.978502+0000","last_undegraded":"2024-09-13T18:55:25.978502+0000","last_fullsized":"2024-09-13T18:55:25.978502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:43.942732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"29'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981727+0000","last_change":"2024-09-13T18:26:32.364786+0000","last_active":"2024-09-13T18:55:25.981727+0000","last_peered":"2024-09-13T18:55:25.981727+0000","last_clean":"2024-09-13T18:55:25.981727+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:55:25.981727+0000","last_undegraded":"2024-09-13T18:55:25.981727+0000","last_fullsized":"2024-09-13T18:55:25.981727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:22.949875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"28'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.010595+0000","last_change":"2024-09-13T18:26:32.347541+0000","last_active":"2024-09-13T18:55:26.010595+0000","last_peered":"2024-09-13T18:55:26.010595+0000","last_clean":"2024-09-13T18:55:26.010595+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:55:26.010595+0000","last_undegraded":"2024-09-13T18:55:26.010595+0000","last_fullsized":"2024-09-13T18:55:26.010595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:58.144128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"29'5","reported_seq":50,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T18:54:50.138427+0000","last_change":"2024-09-13T18:26:32.353359+0000","last_active":"2024-09-13T18:54:50.138427+0000","last_peered":"2024-09-13T18:54:50.138427+0000","last_clean":"2024-09-13T18:54:50.138427+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:54:50.138427+0000","last_undegraded":"2024-09-13T18:54:50.138427+0000","last_fullsized":"2024-09-13T18:54:50.138427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:09.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"29'10","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.001188+0000","last_change":"2024-09-13T18:26:32.349566+0000","last_active":"2024-09-13T18:55:26.001188+0000","last_peered":"2024-09-13T18:55:26.001188+0000","last_clean":"2024-09-13T18:55:26.001188+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:55:26.001188+0000","last_undegraded":"2024-09-13T18:55:26.001188+0000","last_fullsized":"2024-09-13T18:55:26.001188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:22.341899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"43'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:50.192401+0000","last_change":"2024-09-13T18:26:32.358640+0000","last_active":"2024-09-13T18:55:50.192401+0000","last_peered":"2024-09-13T18:55:50.192401+0000","last_clean":"2024-09-13T18:55:50.192401+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:55:50.192401+0000","last_undegraded":"2024-09-13T18:55:50.192401+0000","last_fullsized":"2024-09-13T18:55:50.192401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:40.174754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"43'11","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:50.939549+0000","last_change":"2024-09-13T18:26:32.353675+0000","last_active":"2024-09-13T18:55:50.939549+0000","last_peered":"2024-09-13T18:55:50.939549+0000","last_clean":"2024-09-13T18:55:50.939549+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:55:50.939549+0000","last_undegraded":"2024-09-13T18:55:50.939549+0000","last_fullsized":"2024-09-13T18:55:50.939549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:58.501843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"28'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981840+0000","last_change":"2024-09-13T18:26:32.353223+0000","last_active":"2024-09-13T18:55:25.981840+0000","last_peered":"2024-09-13T18:55:25.981840+0000","last_clean":"2024-09-13T18:55:25.981840+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:55:25.981840+0000","last_undegraded":"2024-09-13T18:55:25.981840+0000","last_fullsized":"2024-09-13T18:55:25.981840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:45.898213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"43'10","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:51.407384+0000","last_change":"2024-09-13T18:26:32.357354+0000","last_active":"2024-09-13T18:55:51.407384+0000","last_peered":"2024-09-13T18:55:51.407384+0000","last_clean":"2024-09-13T18:55:51.407384+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:55:51.407384+0000","last_undegraded":"2024-09-13T18:55:51.407384+0000","last_fullsized":"2024-09-13T18:55:51.407384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:19.185008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"43'11","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:56.238386+0000","last_change":"2024-09-13T18:26:32.348832+0000","last_active":"2024-09-13T18:56:56.238386+0000","last_peered":"2024-09-13T18:56:56.238386+0000","last_clean":"2024-09-13T18:56:56.238386+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:56:56.238386+0000","last_undegraded":"2024-09-13T18:56:56.238386+0000","last_fullsized":"2024-09-13T18:56:56.238386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:11:59.874789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"28'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.972098+0000","last_change":"2024-09-13T18:26:32.356902+0000","last_active":"2024-09-13T18:55:25.972098+0000","last_peered":"2024-09-13T18:55:25.972098+0000","last_clean":"2024-09-13T18:55:25.972098+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:55:25.972098+0000","last_undegraded":"2024-09-13T18:55:25.972098+0000","last_fullsized":"2024-09-13T18:55:25.972098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:18.362772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"28'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.968811+0000","last_change":"2024-09-13T18:26:32.347047+0000","last_active":"2024-09-13T18:55:25.968811+0000","last_peered":"2024-09-13T18:55:25.968811+0000","last_clean":"2024-09-13T18:55:25.968811+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:55:25.968811+0000","last_undegraded":"2024-09-13T18:55:25.968811+0000","last_fullsized":"2024-09-13T18:55:25.968811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:40.882789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"43'10","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:52.028230+0000","last_change":"2024-09-13T18:26:32.349651+0000","last_active":"2024-09-13T18:56:52.028230+0000","last_peered":"2024-09-13T18:56:52.028230+0000","last_clean":"2024-09-13T18:56:52.028230+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:56:52.028230+0000","last_undegraded":"2024-09-13T18:56:52.028230+0000","last_fullsized":"2024-09-13T18:56:52.028230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:33.356563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"29'10","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T18:54:40.392762+0000","last_change":"2024-09-13T18:26:32.344607+0000","last_active":"2024-09-13T18:54:40.392762+0000","last_peered":"2024-09-13T18:54:40.392762+0000","last_clean":"2024-09-13T18:54:40.392762+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:54:40.392762+0000","last_undegraded":"2024-09-13T18:54:40.392762+0000","last_fullsized":"2024-09-13T18:54:40.392762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:31.658172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"29'6","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.000730+0000","last_change":"2024-09-13T18:26:32.347717+0000","last_active":"2024-09-13T18:55:26.000730+0000","last_peered":"2024-09-13T18:55:26.000730+0000","last_clean":"2024-09-13T18:55:26.000730+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:55:26.000730+0000","last_undegraded":"2024-09-13T18:55:26.000730+0000","last_fullsized":"2024-09-13T18:55:26.000730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:47.187837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"29'6","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.971507+0000","last_change":"2024-09-13T18:26:32.359506+0000","last_active":"2024-09-13T18:55:25.971507+0000","last_peered":"2024-09-13T18:55:25.971507+0000","last_clean":"2024-09-13T18:55:25.971507+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:55:25.971507+0000","last_undegraded":"2024-09-13T18:55:25.971507+0000","last_fullsized":"2024-09-13T18:55:25.971507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:39:57.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"43'12","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:37.221615+0000","last_change":"2024-09-13T18:26:32.357729+0000","last_active":"2024-09-13T18:55:37.221615+0000","last_peered":"2024-09-13T18:55:37.221615+0000","last_clean":"2024-09-13T18:55:37.221615+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:55:37.221615+0000","last_undegraded":"2024-09-13T18:55:37.221615+0000","last_fullsized":"2024-09-13T18:55:37.221615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:05.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"29'9","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981655+0000","last_change":"2024-09-13T18:26:32.357980+0000","last_active":"2024-09-13T18:55:25.981655+0000","last_peered":"2024-09-13T18:55:25.981655+0000","last_clean":"2024-09-13T18:55:25.981655+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:55:25.981655+0000","last_undegraded":"2024-09-13T18:55:25.981655+0000","last_fullsized":"2024-09-13T18:55:25.981655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:00.858702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"43'13","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.154866+0000","last_change":"2024-09-13T18:26:32.365638+0000","last_active":"2024-09-13T18:56:59.154866+0000","last_peered":"2024-09-13T18:56:59.154866+0000","last_clean":"2024-09-13T18:56:59.154866+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:56:59.154866+0000","last_undegraded":"2024-09-13T18:56:59.154866+0000","last_fullsized":"2024-09-13T18:56:59.154866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.065853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"29'3","reported_seq":47,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.109132+0000","last_change":"2024-09-13T18:26:32.345496+0000","last_active":"2024-09-13T18:55:26.109132+0000","last_peered":"2024-09-13T18:55:26.109132+0000","last_clean":"2024-09-13T18:55:26.109132+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:55:26.109132+0000","last_undegraded":"2024-09-13T18:55:26.109132+0000","last_fullsized":"2024-09-13T18:55:26.109132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:22.447315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"43'4","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:35.970044+0000","last_change":"2024-09-13T18:26:32.356901+0000","last_active":"2024-09-13T18:56:35.970044+0000","last_peered":"2024-09-13T18:56:35.970044+0000","last_clean":"2024-09-13T18:56:35.970044+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:56:35.970044+0000","last_undegraded":"2024-09-13T18:56:35.970044+0000","last_fullsized":"2024-09-13T18:56:35.970044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:03.311603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"28'8","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.976794+0000","last_change":"2024-09-13T18:26:32.355203+0000","last_active":"2024-09-13T18:55:25.976794+0000","last_peered":"2024-09-13T18:55:25.976794+0000","last_clean":"2024-09-13T18:55:25.976794+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:55:25.976794+0000","last_undegraded":"2024-09-13T18:55:25.976794+0000","last_fullsized":"2024-09-13T18:55:25.976794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:52.769708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"29'6","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983681+0000","last_change":"2024-09-13T18:26:32.358190+0000","last_active":"2024-09-13T18:55:25.983681+0000","last_peered":"2024-09-13T18:55:25.983681+0000","last_clean":"2024-09-13T18:55:25.983681+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:55:25.983681+0000","last_undegraded":"2024-09-13T18:55:25.983681+0000","last_fullsized":"2024-09-13T18:55:25.983681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:35.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"28'4","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.968851+0000","last_change":"2024-09-13T18:26:32.355121+0000","last_active":"2024-09-13T18:55:25.968851+0000","last_peered":"2024-09-13T18:55:25.968851+0000","last_clean":"2024-09-13T18:55:25.968851+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:55:25.968851+0000","last_undegraded":"2024-09-13T18:55:25.968851+0000","last_fullsized":"2024-09-13T18:55:25.968851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:48.427332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"29'6","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.987776+0000","last_change":"2024-09-13T18:26:32.341497+0000","last_active":"2024-09-13T18:55:25.987776+0000","last_peered":"2024-09-13T18:55:25.987776+0000","last_clean":"2024-09-13T18:55:25.987776+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:55:25.987776+0000","last_undegraded":"2024-09-13T18:55:25.987776+0000","last_fullsized":"2024-09-13T18:55:25.987776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:15.510048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"28'2","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978877+0000","last_change":"2024-09-13T18:26:32.350106+0000","last_active":"2024-09-13T18:55:25.978877+0000","last_peered":"2024-09-13T18:55:25.978877+0000","last_clean":"2024-09-13T18:55:25.978877+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:55:25.978877+0000","last_undegraded":"2024-09-13T18:55:25.978877+0000","last_fullsized":"2024-09-13T18:55:25.978877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:12.387451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"29'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984165+0000","last_change":"2024-09-13T18:26:32.362196+0000","last_active":"2024-09-13T18:55:25.984165+0000","last_peered":"2024-09-13T18:55:25.984165+0000","last_clean":"2024-09-13T18:55:25.984165+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:55:25.984165+0000","last_undegraded":"2024-09-13T18:55:25.984165+0000","last_fullsized":"2024-09-13T18:55:25.984165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:14.946904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":11298053892,"num_objects":2881,"num_object_clones":0,"num_object_copies":11524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2881,"num_whiteouts":0,"num_read":797,"num_read_kb":0,"num_write":800719,"num_write_kb":16319459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9769914368,"data_stored":22543564800,"data_compressed":1072758487,"data_compressed_allocated":1504456704,"data_compressed_original":14278107136,"omap_allocated":0,"internal_metadata":0},"log_size":172420,"ondisk_log_size":172420,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":472,"ondisk_log_size":472,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":16496904,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":1362,"num_read_kb":52080,"num_write":3724,"num_write_kb":17653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22700032,"data_stored":32993808,"data_compressed":2441304,"data_compressed_allocated":3506176,"data_compressed_original":13950976,"omap_allocated":0,"internal_metadata":0},"log_size":3487,"ondisk_log_size":3487,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509810,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2270664,"kb_used_data":1249600,"kb_used_omap":1,"kb_used_meta":1021054,"kb_avail":92101176,"statfs":{"total":96636764160,"available":94311604224,"internally_reserved":0,"allocated":1279590400,"data_stored":2930967260,"data_compressed":139752701,"data_compressed_allocated":196038656,"data_compressed_original":1847668736,"omap_allocated":1593,"internal_metadata":1045559751},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":1.3979999999999999,"15min":0.96599999999999997},"min":{"1min":0.224,"5min":0.223,"15min":0.223},"max":{"1min":9.0050000000000008,"5min":10.43,"15min":10.43},"last":0.25600000000000001},{"interface":"front","average":{"1min":2.5230000000000001,"5min":1.8320000000000001,"15min":1.141},"min":{"1min":0.25900000000000001,"5min":0.245,"15min":0.245},"max":{"1min":9.9190000000000005,"5min":16.600999999999999,"15min":16.600999999999999},"last":0.27200000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.9630000000000001,"5min":3.831,"15min":1.825},"min":{"1min":0.26000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":7.7409999999999997,"5min":52.290999999999997,"15min":52.290999999999997},"last":0.31},{"interface":"front","average":{"1min":1.6719999999999999,"5min":4.0910000000000002,"15min":1.9239999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":7.8410000000000002,"5min":52.149000000000001,"15min":52.149000000000001},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.518,"15min":1.0580000000000001},"min":{"1min":0.33500000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.8360000000000003,"5min":14.933,"15min":14.933},"last":1.419},{"interface":"front","average":{"1min":1.7509999999999999,"5min":1.4239999999999999,"15min":1.048},"min":{"1min":0.29099999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":6.7160000000000002,"5min":17.792000000000002,"15min":17.792000000000002},"last":0.43099999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":2.4580000000000002,"15min":1.3899999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":8.1809999999999992,"5min":15.622,"15min":15.622},"last":2.6400000000000001},{"interface":"front","average":{"1min":3.2429999999999999,"5min":3.2829999999999999,"15min":1.6799999999999999},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":14.993,"5min":28.248000000000001,"15min":28.248000000000001},"last":1.8979999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.304,"15min":1.0820000000000001},"min":{"1min":0.31,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.3570000000000002,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.232,"15min":1.048},"min":{"1min":0.38700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":4.4569999999999999,"5min":7.9409999999999998,"15min":7.9409999999999998},"last":0.48699999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.95699999999999996,"15min":0.97699999999999998},"min":{"1min":0.376,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.7410000000000001,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.099,"5min":0.96999999999999997,"15min":1.004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.2480000000000002,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":0.34599999999999997}]},{"osd":7,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.0429999999999999,"15min":1.0720000000000001},"min":{"1min":0.39300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.8319999999999999,"5min":12.021000000000001,"15min":12.021000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.91300000000000003,"15min":0.98599999999999999},"min":{"1min":0.32500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":3.9430000000000001,"5min":9.0440000000000005,"15min":9.0440000000000005},"last":0.40200000000000002}]}]},{"osd":7,"up_from":15,"seq":64424509809,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1929960,"kb_used_data":1066208,"kb_used_omap":1,"kb_used_meta":863742,"kb_avail":92441880,"statfs":{"total":96636764160,"available":94660485120,"internally_reserved":0,"allocated":1091796992,"data_stored":2515818119,"data_compressed":119837066,"data_compressed_allocated":168169472,"data_compressed_original":1592455168,"omap_allocated":1589,"internal_metadata":884472267},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":2.121,"15min":1.206},"min":{"1min":0.23000000000000001,"5min":0.217,"15min":0.217},"max":{"1min":14.778,"5min":24.414000000000001,"15min":24.414000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":2.1680000000000001,"5min":2.254,"15min":1.3460000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":14.814,"5min":24.469000000000001,"15min":24.469000000000001},"last":0.79000000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":2.4889999999999999,"5min":3.355,"15min":1.617},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":6.0490000000000004,"5min":46.161999999999999,"15min":46.161999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":5.1929999999999996,"5min":4.3570000000000002,"15min":2.0129999999999999},"min":{"1min":0.29099999999999998,"5min":0.252,"15min":0.252},"max":{"1min":39.853999999999999,"5min":45.713000000000001,"15min":45.713000000000001},"last":0.95799999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.228,"15min":0.98699999999999999},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":5.3230000000000004,"5min":7.4740000000000002,"15min":7.4740000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.329,"15min":0.97899999999999998},"min":{"1min":0.315,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.8559999999999999,"5min":9.1679999999999993,"15min":9.1679999999999993},"last":1.1000000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":3.4129999999999998,"5min":3.6960000000000002,"15min":1.8029999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":17.721,"5min":35.494999999999997,"15min":35.494999999999997},"last":1.052},{"interface":"front","average":{"1min":2.278,"5min":2.976,"15min":1.6850000000000001},"min":{"1min":0.33800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":11.353999999999999,"5min":19.946000000000002,"15min":19.946000000000002},"last":1.0089999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.92600000000000005,"15min":0.91700000000000004},"min":{"1min":0.25800000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.2749999999999999,"5min":7.2880000000000003,"15min":7.2880000000000003},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.0820000000000001,"15min":1.0589999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":7.8810000000000002,"5min":7.8810000000000002,"15min":7.8810000000000002},"last":1.1950000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.80700000000000005,"15min":0.877},"min":{"1min":0.29399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":3.6200000000000001,"5min":6.7309999999999999,"15min":6.7309999999999999},"last":0.753},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90100000000000002,"15min":0.96999999999999997},"min":{"1min":0.32000000000000001,"5min":0.318,"15min":0.318},"max":{"1min":3.5819999999999999,"5min":6.6699999999999999,"15min":6.6699999999999999},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:56:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.2889999999999999,"15min":1.137},"min":{"1min":0.35299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.4260000000000002,"5min":32.491,"15min":32.491},"last":1.143},{"interface":"front","average":{"1min":0.54100000000000004,"5min":1.137,"15min":1.091},"min":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.82899999999999996,"5min":27.291,"15min":27.291},"last":0.90400000000000003}]}]},{"osd":4,"up_from":15,"seq":64424509809,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2176860,"kb_used_data":1236180,"kb_used_omap":1,"kb_used_meta":940670,"kb_avail":92194980,"statfs":{"total":96636764160,"available":94407659520,"internally_reserved":0,"allocated":1265848320,"data_stored":2899198048,"data_compressed":137665483,"data_compressed_allocated":193257472,"data_compressed_original":1826861056,"omap_allocated":1594,"internal_metadata":963246534},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":2.2599999999999998,"5min":2.1259999999999999,"15min":1.214},"min":{"1min":0.26300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":8.8740000000000006,"5min":28.593,"15min":28.593},"last":0.441},{"interface":"front","average":{"1min":1.9850000000000001,"5min":1.837,"15min":1.1930000000000001},"min":{"1min":0.26400000000000001,"5min":0.255,"15min":0.255},"max":{"1min":10.711,"5min":28.559000000000001,"15min":28.559000000000001},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":2.6299999999999999,"5min":2.907,"15min":1.49},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":10.044,"5min":37.100999999999999,"15min":37.100999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":2.0089999999999999,"5min":3.4969999999999999,"15min":1.6910000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":8.3810000000000002,"5min":44.798000000000002,"15min":44.798000000000002},"last":0.28799999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":2.6789999999999998,"5min":1.845,"15min":1.1859999999999999},"min":{"1min":0.30199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":16.617000000000001,"5min":27.634,"15min":27.634},"last":0.42399999999999999},{"interface":"front","average":{"1min":2.8279999999999998,"5min":1.7170000000000001,"15min":1.2050000000000001},"min":{"1min":0.317,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":16.396999999999998,"5min":27.606000000000002,"15min":27.606000000000002},"last":0.34200000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":3.496,"5min":3.339,"15min":1.6499999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":18.503,"5min":40.676000000000002,"15min":40.676000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":5.9610000000000003,"5min":3.8050000000000002,"15min":1.887},"min":{"1min":0.47999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":39.649000000000001,"5min":40.881999999999998,"15min":40.881999999999998},"last":0.36699999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:56:26 2024","interfaces":[{"interface":"back","average":{"1min":1.7769999999999999,"5min":0.876,"15min":0.88300000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":8.5649999999999995,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":0.91600000000000004,"15min":0.98499999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.0810000000000004,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.167,"15min":1.0900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":5.984,"5min":9.4570000000000007,"15min":9.4570000000000007},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.1559999999999999,"5min":0.996,"15min":1.0109999999999999},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":5.1059999999999999,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.38600000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8779999999999999,"5min":1.4279999999999999,"15min":1.177},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":10.101000000000001,"5min":10.101000000000001,"15min":10.101000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.9239999999999999,"5min":1.492,"15min":1.274},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":10.127000000000001,"5min":10.127000000000001,"15min":10.127000000000001},"last":0.50600000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542513,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2824836,"kb_used_data":1633712,"kb_used_omap":1,"kb_used_meta":1190142,"kb_avail":91547004,"statfs":{"total":96636764160,"available":93744132096,"internally_reserved":0,"allocated":1672921088,"data_stored":3841409738,"data_compressed":183202141,"data_compressed_allocated":257212416,"data_compressed_original":2425958400,"omap_allocated":1591,"internal_metadata":1218705865},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:32 2024","interfaces":[{"interface":"back","average":{"1min":3.0569999999999999,"5min":3.3959999999999999,"15min":1.5109999999999999},"min":{"1min":0.20300000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":22.670000000000002,"5min":22.670000000000002,"15min":22.670000000000002},"last":0.29499999999999998},{"interface":"front","average":{"1min":3.1179999999999999,"5min":3.5009999999999999,"15min":1.677},"min":{"1min":0.246,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":22.728000000000002,"5min":22.728000000000002,"15min":22.728000000000002},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":7.9240000000000004,"5min":6.282,"15min":2.4940000000000002},"min":{"1min":0.38500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":42.046999999999997,"5min":42.046999999999997,"15min":42.046999999999997},"last":0.316},{"interface":"front","average":{"1min":7.6500000000000004,"5min":6.1680000000000001,"15min":2.5630000000000002},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":42.024999999999999,"5min":42.024999999999999,"15min":42.024999999999999},"last":0.503}]},{"osd":2,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":6.4550000000000001,"5min":4.2300000000000004,"15min":1.9259999999999999},"min":{"1min":0.27700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":41.909999999999997,"5min":41.909999999999997,"15min":41.909999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":5.1609999999999996,"5min":3.1819999999999999,"15min":1.5229999999999999},"min":{"1min":0.317,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":27.25,"5min":27.25,"15min":27.25},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:56:32 2024","interfaces":[{"interface":"back","average":{"1min":4.6550000000000002,"5min":3.3100000000000001,"15min":1.7430000000000001},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":42.003,"5min":42.003,"15min":42.003},"last":0.73299999999999998},{"interface":"front","average":{"1min":5.4329999999999998,"5min":3.9399999999999999,"15min":1.956},"min":{"1min":0.39300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":41.951000000000001,"5min":41.951000000000001,"15min":41.951000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":4.79,"5min":2.46,"15min":1.508},"min":{"1min":0.38100000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":22.988,"5min":22.988,"15min":22.988},"last":0.68700000000000006},{"interface":"front","average":{"1min":3.2829999999999999,"5min":2.2799999999999998,"15min":1.462},"min":{"1min":0.35199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":23.030000000000001,"5min":23.030000000000001,"15min":23.030000000000001},"last":0.80300000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":4.0529999999999999,"5min":2.9689999999999999,"15min":1.6719999999999999},"min":{"1min":0.35799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":23.087,"5min":23.087,"15min":23.087},"last":0.64200000000000002},{"interface":"front","average":{"1min":5.4210000000000003,"5min":3.4540000000000002,"15min":1.843},"min":{"1min":0.52700000000000002,"5min":0.317,"15min":0.317},"max":{"1min":23.425000000000001,"5min":23.425000000000001,"15min":23.425000000000001},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":4.9379999999999997,"5min":3.1600000000000001,"15min":1.714},"min":{"1min":0.437,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":41.975999999999999,"5min":41.975999999999999,"15min":41.975999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":4.9550000000000001,"5min":2.956,"15min":1.6619999999999999},"min":{"1min":0.42699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":23.457999999999998,"5min":23.457999999999998,"15min":23.457999999999998},"last":0.75600000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542517,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1698048,"kb_used_data":966456,"kb_used_omap":1,"kb_used_meta":731582,"kb_avail":92673792,"statfs":{"total":96636764160,"available":94897963008,"internally_reserved":0,"allocated":989650944,"data_stored":2283014993,"data_compressed":108755460,"data_compressed_allocated":152350720,"data_compressed_original":1445974016,"omap_allocated":1593,"internal_metadata":749140423},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":2.5489999999999999,"15min":1.2270000000000001},"min":{"1min":0.17799999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":5.8200000000000003,"5min":37.600000000000001,"15min":37.600000000000001},"last":2.6019999999999999},{"interface":"front","average":{"1min":1.056,"5min":2.4569999999999999,"15min":1.266},"min":{"1min":0.218,"5min":0.19,"15min":0.19},"max":{"1min":6.0839999999999996,"5min":37.651000000000003,"15min":37.651000000000003},"last":2.6800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":2.5800000000000001,"5min":3.0449999999999999,"15min":1.5209999999999999},"min":{"1min":0.26500000000000001,"5min":0.255,"15min":0.255},"max":{"1min":12.244,"5min":18.055,"15min":18.055},"last":9.1189999999999998},{"interface":"front","average":{"1min":2.851,"5min":2.8900000000000001,"15min":1.431},"min":{"1min":0.28899999999999998,"5min":0.188,"15min":0.188},"max":{"1min":11.916,"5min":18.175999999999998,"15min":18.175999999999998},"last":10.196}]},{"osd":3,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":2.6179999999999999,"5min":3.5470000000000002,"15min":1.7010000000000001},"min":{"1min":0.313,"5min":0.249,"15min":0.249},"max":{"1min":12.888999999999999,"5min":25.832999999999998,"15min":25.832999999999998},"last":3.5350000000000001},{"interface":"front","average":{"1min":2.621,"5min":3.5230000000000001,"15min":1.77},"min":{"1min":0.375,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":12.702999999999999,"5min":17.515000000000001,"15min":17.515000000000001},"last":3.508}]},{"osd":4,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.2190000000000001,"15min":0.996},"min":{"1min":0.27700000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.5590000000000002,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":3.036},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.242,"15min":1.0820000000000001},"min":{"1min":0.33700000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":5.0899999999999999,"5min":8.0999999999999996,"15min":8.0999999999999996},"last":1.7949999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.27,"15min":1.032},"min":{"1min":0.29799999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":8.4800000000000004,"5min":8.4800000000000004,"15min":8.4800000000000004},"last":2.6360000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.4319999999999999,"15min":1.1220000000000001},"min":{"1min":0.36499999999999999,"5min":0.309,"15min":0.309},"max":{"1min":3.8079999999999998,"5min":8.907,"15min":8.907},"last":2.3460000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":1.321,"15min":1.133},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.032,"5min":9.6720000000000006,"15min":9.6720000000000006},"last":1.875},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.2070000000000001,"15min":1.069},"min":{"1min":0.35799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":5.3289999999999997,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":2.375}]},{"osd":7,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3560000000000001,"15min":1.129},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":4.7939999999999996,"5min":8.9250000000000007,"15min":8.9250000000000007},"last":2.661},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.585,"15min":1.236},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":6.335,"5min":9.2330000000000005,"15min":9.2330000000000005},"last":1.835}]}]},{"osd":5,"up_from":13,"seq":55834575218,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1570116,"kb_used_data":872956,"kb_used_omap":1,"kb_used_meta":697150,"kb_avail":92801724,"statfs":{"total":96636764160,"available":95028965376,"internally_reserved":0,"allocated":893906944,"data_stored":2074980080,"data_compressed":98665143,"data_compressed_allocated":138342400,"data_compressed_original":1319673856,"omap_allocated":1593,"internal_metadata":713882055},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":1.8879999999999999,"5min":1.8919999999999999,"15min":1.109},"min":{"1min":0.23799999999999999,"5min":0.224,"15min":0.224},"max":{"1min":8.2240000000000002,"5min":9.952,"15min":9.952},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.95,"5min":2.0139999999999998,"15min":1.2350000000000001},"min":{"1min":0.29899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.2579999999999991,"5min":14.317,"15min":14.317},"last":0.86799999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":1.9550000000000001,"5min":2.29,"15min":1.3129999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":10.058,"5min":10.85,"15min":10.85},"last":0.66700000000000004},{"interface":"front","average":{"1min":2.1629999999999998,"5min":2.2719999999999998,"15min":1.3140000000000001},"min":{"1min":0.29299999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":10.462999999999999,"5min":10.868,"15min":10.868},"last":0.82299999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":3.8519999999999999,"5min":1.915,"15min":1.1759999999999999},"min":{"1min":0.27300000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":38.484000000000002,"5min":38.484000000000002,"15min":38.484000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":4.0890000000000004,"5min":2.2309999999999999,"15min":1.286},"min":{"1min":0.27100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":40.686,"5min":40.686,"15min":40.686},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":3.9580000000000002,"5min":4.2320000000000002,"15min":1.9750000000000001},"min":{"1min":0.38100000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":19.093,"5min":72.534999999999997,"15min":72.534999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":3.5720000000000001,"5min":4.0199999999999996,"15min":1.9710000000000001},"min":{"1min":0.43099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":19.071000000000002,"5min":72.486999999999995,"15min":72.486999999999995},"last":0.75600000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":1.02,"15min":0.91900000000000004},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.2290000000000001,"5min":7.8710000000000004,"15min":7.8710000000000004},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.131,"15min":1.0760000000000001},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":8.7829999999999995,"5min":8.7829999999999995,"15min":8.7829999999999995},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":0.96399999999999997,"15min":0.997},"min":{"1min":0.28699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.04,"15min":1.0489999999999999},"min":{"1min":0.35599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.516,"5min":4.516,"15min":4.516},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.02,"15min":1.0209999999999999},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":4.4249999999999998,"5min":7.6020000000000003,"15min":7.6020000000000003},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.0169999999999999,"15min":1.101},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.1900000000000004,"5min":7.9109999999999996,"15min":7.9109999999999996},"last":1.9810000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575216,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2673912,"kb_used_data":1523088,"kb_used_omap":1,"kb_used_meta":1148350,"kb_avail":91697928,"statfs":{"total":96636764160,"available":93898678272,"internally_reserved":0,"allocated":1559642112,"data_stored":3627326782,"data_compressed":172575409,"data_compressed_allocated":241844224,"data_compressed_original":2309791744,"omap_allocated":1594,"internal_metadata":1175910854},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":3.2570000000000001,"15min":1.4910000000000001},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":7.9580000000000002,"5min":28.297000000000001,"15min":28.297000000000001},"last":10.279},{"interface":"front","average":{"1min":3.1040000000000001,"5min":2.7749999999999999,"15min":1.341},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":19.785,"5min":19.785,"15min":19.785},"last":10.162000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":2.8340000000000001,"15min":1.385},"min":{"1min":0.252,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":8.0630000000000006,"5min":25.609999999999999,"15min":25.609999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":4.0890000000000004,"5min":3.097,"15min":1.536},"min":{"1min":0.38500000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":20.173999999999999,"5min":30.469999999999999,"15min":30.469999999999999},"last":10.465}]},{"osd":3,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":5.524,"5min":5.7249999999999996,"15min":2.383},"min":{"1min":0.58199999999999996,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":20.001000000000001,"5min":36.643999999999998,"15min":36.643999999999998},"last":10.406000000000001},{"interface":"front","average":{"1min":5.5519999999999996,"5min":5.9119999999999999,"15min":2.5110000000000001},"min":{"1min":0.59899999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":20.030999999999999,"5min":36.670000000000002,"15min":36.670000000000002},"last":10.429}]},{"osd":4,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":1.7549999999999999,"15min":1.1759999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.2249999999999996,"5min":7.6799999999999997,"15min":7.6799999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.26,"5min":2.1299999999999999,"15min":1.331},"min":{"1min":0.32800000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":6.5540000000000003,"5min":14.906000000000001,"15min":14.906000000000001},"last":0.47999999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":3.1880000000000002,"5min":1.9350000000000001,"15min":1.3149999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":9.0519999999999996,"5min":9.0519999999999996,"15min":9.0519999999999996},"last":10.259},{"interface":"front","average":{"1min":2.7149999999999999,"5min":1.9410000000000001,"15min":1.3300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.1050000000000004,"5min":12.27,"15min":12.27},"last":10.231999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":2.577,"5min":2.4300000000000002,"15min":1.468},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":9.077,"5min":28.981999999999999,"15min":28.981999999999999},"last":10.538},{"interface":"front","average":{"1min":2.1819999999999999,"5min":2.1379999999999999,"15min":1.3799999999999999},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":15.727,"5min":15.727,"15min":15.727},"last":10.342000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":2.379,"15min":1.4359999999999999},"min":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":7.9930000000000003,"5min":28.100000000000001,"15min":28.100000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":1.728,"5min":2.4860000000000002,"15min":1.518},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.0220000000000002,"5min":28.896999999999998,"15min":28.896999999999998},"last":0.41099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575217,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1942528,"kb_used_data":1150840,"kb_used_omap":1,"kb_used_meta":791678,"kb_avail":92429312,"statfs":{"total":96636764160,"available":94647615488,"internally_reserved":0,"allocated":1178460160,"data_stored":2729905844,"data_compressed":130141336,"data_compressed_allocated":182214656,"data_compressed_original":1733922816,"omap_allocated":1594,"internal_metadata":810678726},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":5.1710000000000003,"5min":4.0839999999999996,"15min":1.732},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":29.712,"5min":29.712,"15min":29.712},"last":0.34100000000000003},{"interface":"front","average":{"1min":5.4039999999999999,"5min":3.0979999999999999,"15min":1.4810000000000001},"min":{"1min":0.495,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":29.748000000000001,"5min":29.748000000000001,"15min":29.748000000000001},"last":0.36499999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":1.7849999999999999,"15min":1.014},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":17.611000000000001,"5min":17.611000000000001,"15min":17.611000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":2.847,"5min":1.9570000000000001,"15min":1.1180000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":17.501999999999999,"5min":17.501999999999999,"15min":17.501999999999999},"last":0.375}]},{"osd":3,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":2.9580000000000002,"5min":3.4689999999999999,"15min":1.615},"min":{"1min":0.26200000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":17.260000000000002,"5min":17.260000000000002,"15min":17.260000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":2.5459999999999998,"5min":3.0129999999999999,"15min":1.512},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":17.088000000000001,"5min":17.088000000000001,"15min":17.088000000000001},"last":0.439}]},{"osd":4,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":5.3200000000000003,"5min":2.3799999999999999,"15min":1.407},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":40.101999999999997,"5min":40.101999999999997,"15min":40.101999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":5.2830000000000004,"5min":2.089,"15min":1.2989999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":40.981000000000002,"5min":40.981000000000002,"15min":40.981000000000002},"last":0.69599999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":4.5419999999999998,"5min":1.857,"15min":1.252},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":40.701999999999998,"5min":40.701999999999998,"15min":40.701999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":4.8120000000000003,"5min":2.0230000000000001,"15min":1.3440000000000001},"min":{"1min":0.36899999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":40.130000000000003,"5min":40.130000000000003,"15min":40.130000000000003},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":5.7699999999999996,"5min":2.1499999999999999,"15min":1.3979999999999999},"min":{"1min":0.41399999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":51.719000000000001,"5min":51.719000000000001,"15min":51.719000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":5.8150000000000004,"5min":2.1989999999999998,"15min":1.448},"min":{"1min":0.49099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":51.75,"5min":51.75,"15min":51.75},"last":0.79300000000000004}]},{"osd":7,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":5.5179999999999998,"5min":2.3180000000000001,"15min":1.4770000000000001},"min":{"1min":0.36899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":58.810000000000002,"5min":58.810000000000002,"15min":58.810000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":4.609,"5min":2.125,"15min":1.4370000000000001},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":39.622,"5min":39.622,"15min":39.622},"last":0.74199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2064384,"data_stored":3814150,"data_compressed":409575,"data_compressed_allocated":573440,"data_compressed_original":2342912,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":94659,"data_compressed":1568,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5124096,"data_stored":8485180,"data_compressed":790472,"data_compressed_allocated":1138688,"data_compressed_original":4517888,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4136960,"data_stored":4196050,"data_compressed":20605,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1777664,"data_stored":4196686,"data_compressed":519875,"data_compressed_allocated":692224,"data_compressed_original":3125248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9441280,"data_stored":12204761,"data_compressed":699209,"data_compressed_allocated":1056768,"data_compressed_original":3842048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1138999296,"data_stored":2622500864,"data_compressed":124826741,"data_compressed_allocated":174882816,"data_compressed_original":1658384384,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1543139328,"data_stored":3583045632,"data_compressed":170484417,"data_compressed_allocated":238940160,"data_compressed_original":2278846464,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":973479936,"data_stored":2257514496,"data_compressed":107562294,"data_compressed_allocated":150663168,"data_compressed_original":1434697728,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1628086272,"data_stored":3742507008,"data_compressed":178128102,"data_compressed_allocated":250073088,"data_compressed_original":2364493824,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1234100224,"data_stored":2836041728,"data_compressed":134852831,"data_compressed_allocated":189272064,"data_compressed_original":1791213568,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":892964864,"data_stored":2072641536,"data_compressed":98566295,"data_compressed_allocated":138211328,"data_compressed_original":1317888000,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1277333504,"data_stored":2926530560,"data_compressed":139232826,"data_compressed_allocated":195346432,"data_compressed_original":1844543488,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1081810944,"data_stored":2502782976,"data_compressed":119104981,"data_compressed_allocated":167067648,"data_compressed_original":1588039680,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:57:04.542 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:57:04.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.723+0000 7fa59677d700 1 Processor -- start 2024-09-13T18:57:04.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.723+0000 7fa59677d700 1 -- start start 2024-09-13T18:57:04.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.723+0000 7fa59677d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa5900cb2b0 0x7fa5900c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:04.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.723+0000 7fa59677d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900c8230 0x7fa59015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:04.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.723+0000 7fa59677d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa590161060 con 0x7fa5900caed0 2024-09-13T18:57:04.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.723+0000 7fa59677d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5900c78d0 con 0x7fa5900cb2b0 2024-09-13T18:57:04.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.723+0000 7fa59677d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa590161300 con 0x7fa5900c8230 2024-09-13T18:57:04.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.723+0000 7fa59577b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa5900caed0 0x7fa590072380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59120/0 (socket says 172.21.15.146:59120) 2024-09-13T18:57:04.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.723+0000 7fa59577b700 1 -- 172.21.15.146:0/2346139607 learned_addr learned my addr 172.21.15.146:0/2346139607 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:57:04.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.723+0000 7fa594f7a700 1 --2- 172.21.15.146:0/2346139607 >> v2:172.21.15.146:3300/0 conn(0x7fa5900cb2b0 0x7fa5900c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:04.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.723+0000 7fa595f7c700 1 --2- 172.21.15.146:0/2346139607 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900c8230 0x7fa59015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:04.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa595f7c700 1 -- 172.21.15.146:0/2346139607 >> v2:172.21.15.146:3300/0 conn(0x7fa5900cb2b0 msgr2=0x7fa5900c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:04.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa595f7c700 1 --2- 172.21.15.146:0/2346139607 >> v2:172.21.15.146:3300/0 conn(0x7fa5900cb2b0 0x7fa5900c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa595f7c700 1 -- 172.21.15.146:0/2346139607 >> v1:172.21.15.146:6789/0 conn(0x7fa5900caed0 legacy=0x7fa590072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:04.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa595f7c700 1 -- 172.21.15.146:0/2346139607 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5901615c0 con 0x7fa5900c8230 2024-09-13T18:57:04.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa595f7c700 1 --2- 172.21.15.146:0/2346139607 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900c8230 0x7fa59015e170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa588009dc0 tx=0x7fa58800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a710bdf05c0b6fee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:04.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa583fff700 1 -- 172.21.15.146:0/2346139607 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58800e070 con 0x7fa5900c8230 2024-09-13T18:57:04.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa583fff700 1 -- 172.21.15.146:0/2346139607 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa588003a40 con 0x7fa5900c8230 2024-09-13T18:57:04.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa59677d700 1 -- 172.21.15.146:0/2346139607 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900c8230 msgr2=0x7fa59015e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:04.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa59677d700 1 --2- 172.21.15.146:0/2346139607 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900c8230 0x7fa59015e170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa588009dc0 tx=0x7fa58800b330 comp rx=0 tx=0).stop 2024-09-13T18:57:04.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa59677d700 1 -- 172.21.15.146:0/2346139607 shutdown_connections 2024-09-13T18:57:04.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa59677d700 1 --2- 172.21.15.146:0/2346139607 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900c8230 0x7fa59015e170 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa59677d700 1 --2- 172.21.15.146:0/2346139607 >> v2:172.21.15.146:3300/0 conn(0x7fa5900cb2b0 0x7fa5900c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa59677d700 1 -- 172.21.15.146:0/2346139607 >> 172.21.15.146:0/2346139607 conn(0x7fa590066090 msgr2=0x7fa590066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:04.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa59677d700 1 -- 172.21.15.146:0/2346139607 shutdown_connections 2024-09-13T18:57:04.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa59677d700 1 -- 172.21.15.146:0/2346139607 wait complete. 2024-09-13T18:57:04.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa59677d700 1 Processor -- start 2024-09-13T18:57:04.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.727+0000 7fa59677d700 1 -- start start 2024-09-13T18:57:04.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59677d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5900c8230 0x7fa5900c2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:04.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59677d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900caed0 0x7fa5900c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:04.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59677d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5900cb2b0 0x7fa5900c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:04.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59677d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5900abb00 con 0x7fa5900c8230 2024-09-13T18:57:04.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59677d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa590124900 con 0x7fa5900cb2b0 2024-09-13T18:57:04.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59677d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5901259f0 con 0x7fa5900caed0 2024-09-13T18:57:04.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59577b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5900c8230 0x7fa5900c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:04.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59577b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5900c8230 0x7fa5900c2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57600/0 (socket says 172.21.15.146:57600) 2024-09-13T18:57:04.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59577b700 1 -- 172.21.15.146:0/751844954 learned_addr learned my addr 172.21.15.146:0/751844954 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:04.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa594f7a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900caed0 0x7fa5900c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:04.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa595f7c700 1 --2- 172.21.15.146:0/751844954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5900cb2b0 0x7fa5900c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:04.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59577b700 1 -- 172.21.15.146:0/751844954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900caed0 msgr2=0x7fa5900c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:04.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59577b700 1 --2- 172.21.15.146:0/751844954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900caed0 0x7fa5900c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59577b700 1 -- 172.21.15.146:0/751844954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5900cb2b0 msgr2=0x7fa5900c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:04.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59577b700 1 --2- 172.21.15.146:0/751844954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5900cb2b0 0x7fa5900c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:04.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59577b700 1 -- 172.21.15.146:0/751844954 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa588009a70 con 0x7fa5900c8230 2024-09-13T18:57:04.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59577b700 1 --2- 172.21.15.146:0/751844954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5900c8230 0x7fa5900c2290 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa58400c7e0 tx=0x7fa58400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:04.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa581ffb700 1 -- 172.21.15.146:0/751844954 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584004ca0 con 0x7fa5900c8230 2024-09-13T18:57:04.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa581ffb700 1 -- 172.21.15.146:0/751844954 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa584004e00 con 0x7fa5900c8230 2024-09-13T18:57:04.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59677d700 1 -- 172.21.15.146:0/751844954 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa590124b70 con 0x7fa5900c8230 2024-09-13T18:57:04.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa581ffb700 1 -- 172.21.15.146:0/751844954 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584007310 con 0x7fa5900c8230 2024-09-13T18:57:04.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59677d700 1 -- 172.21.15.146:0/751844954 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa590125710 con 0x7fa5900c8230 2024-09-13T18:57:04.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.731+0000 7fa59677d700 1 -- 172.21.15.146:0/751844954 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59006d430 con 0x7fa5900c8230 2024-09-13T18:57:04.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.739+0000 7fa581ffb700 1 -- 172.21.15.146:0/751844954 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa584007470 con 0x7fa5900c8230 2024-09-13T18:57:04.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.739+0000 7fa581ffb700 1 --2- 172.21.15.146:0/751844954 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa57c0669a0 0x7fa57c068e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:04.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.739+0000 7fa581ffb700 1 -- 172.21.15.146:0/751844954 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fa584084df0 con 0x7fa5900c8230 2024-09-13T18:57:04.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.739+0000 7fa581ffb700 1 -- 172.21.15.146:0/751844954 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa5840518b0 con 0x7fa5900c8230 2024-09-13T18:57:04.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.743+0000 7fa594f7a700 1 --2- 172.21.15.146:0/751844954 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa57c0669a0 0x7fa57c068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:04.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.743+0000 7fa594f7a700 1 --2- 172.21.15.146:0/751844954 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa57c0669a0 0x7fa57c068e60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa59015ddb0 tx=0x7fa58c009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:05.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:04.995+0000 7fa59677d700 1 -- 172.21.15.146:0/751844954 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa590160e80 con 0x7fa57c0669a0 2024-09-13T18:57:05.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.011+0000 7fa581ffb700 1 -- 172.21.15.146:0/751844954 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370788 (secure 0 0 0) 0x7fa590160e80 con 0x7fa57c0669a0 2024-09-13T18:57:05.028 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:57:05.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.015+0000 7fa59677d700 1 -- 172.21.15.146:0/751844954 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa57c0669a0 msgr2=0x7fa57c068e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:05.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.015+0000 7fa59677d700 1 --2- 172.21.15.146:0/751844954 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa57c0669a0 0x7fa57c068e60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa59015ddb0 tx=0x7fa58c009240 comp rx=0 tx=0).stop 2024-09-13T18:57:05.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.015+0000 7fa59677d700 1 -- 172.21.15.146:0/751844954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5900c8230 msgr2=0x7fa5900c2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:05.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.015+0000 7fa59677d700 1 --2- 172.21.15.146:0/751844954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5900c8230 0x7fa5900c2290 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa58400c7e0 tx=0x7fa58400caf0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.015+0000 7fa59677d700 1 -- 172.21.15.146:0/751844954 shutdown_connections 2024-09-13T18:57:05.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.015+0000 7fa59677d700 1 --2- 172.21.15.146:0/751844954 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa57c0669a0 0x7fa57c068e60 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.015+0000 7fa59677d700 1 --2- 172.21.15.146:0/751844954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5900cb2b0 0x7fa5900c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.015+0000 7fa59677d700 1 --2- 172.21.15.146:0/751844954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5900caed0 0x7fa5900c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.015+0000 7fa59677d700 1 --2- 172.21.15.146:0/751844954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5900c8230 0x7fa5900c2290 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.015+0000 7fa59677d700 1 -- 172.21.15.146:0/751844954 >> 172.21.15.146:0/751844954 conn(0x7fa590066090 msgr2=0x7fa59015fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:05.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.015+0000 7fa59677d700 1 -- 172.21.15.146:0/751844954 shutdown_connections 2024-09-13T18:57:05.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.015+0000 7fa59677d700 1 -- 172.21.15.146:0/751844954 wait complete. 2024-09-13T18:57:05.032 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:57:05.041 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":955,"stamp":"2024-09-13T18:57:03.595662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":11315141164,"num_objects":2956,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2956,"num_whiteouts":0,"num_read":2275,"num_read_kb":52180,"num_write":804602,"num_write_kb":16339528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":176496,"ondisk_log_size":176496,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":17086924,"kb_used_data":9699040,"kb_used_omap":12,"kb_used_meta":7384371,"kb_avail":737887796,"statfs":{"total":773094113280,"available":755597103104,"internally_reserved":0,"allocated":9931816960,"data_stored":22902620864,"data_compressed":1090594739,"data_compressed_allocated":1529430016,"data_compressed_original":14502305792,"omap_allocated":12741,"internal_metadata":7561596475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1832536528,"num_objects":-438,"num_object_clones":0,"num_object_copies":-1752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-438,"num_whiteouts":0,"num_read":7,"num_read_kb":21,"num_write":7455,"num_write_kb":28720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004697"},"pg_stats":[{"pgid":"2.1f","version":"43'464","reported_seq":542,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:57.832117+0000","last_change":"2024-09-13T18:26:31.972797+0000","last_active":"2024-09-13T18:56:57.832117+0000","last_peered":"2024-09-13T18:56:57.832117+0000","last_clean":"2024-09-13T18:56:57.832117+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:56:57.832117+0000","last_undegraded":"2024-09-13T18:56:57.832117+0000","last_fullsized":"2024-09-13T18:56:57.832117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:50.511747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3814072,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":428,"num_write":455,"num_write_kb":3981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"29'4","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.078641+0000","last_change":"2024-09-13T18:26:32.349901+0000","last_active":"2024-09-13T18:55:26.078641+0000","last_peered":"2024-09-13T18:55:26.078641+0000","last_clean":"2024-09-13T18:55:26.078641+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:55:26.078641+0000","last_undegraded":"2024-09-13T18:55:26.078641+0000","last_fullsized":"2024-09-13T18:55:26.078641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:24.978813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"43'5677","reported_seq":5755,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.295010+0000","last_change":"2024-09-13T18:26:36.407579+0000","last_active":"2024-09-13T18:57:00.295010+0000","last_peered":"2024-09-13T18:57:00.295010+0000","last_clean":"2024-09-13T18:57:00.295010+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:57:00.295010+0000","last_undegraded":"2024-09-13T18:57:00.295010+0000","last_fullsized":"2024-09-13T18:57:00.295010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5677,"log_dups_size":0,"ondisk_log_size":5677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:53:56.370013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348712512,"num_objects":93,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":26674,"num_write_kb":537654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"41'12","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.079133+0000","last_change":"2024-09-13T18:26:31.980716+0000","last_active":"2024-09-13T18:55:26.079133+0000","last_peered":"2024-09-13T18:55:26.079133+0000","last_clean":"2024-09-13T18:55:26.079133+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:55:26.079133+0000","last_undegraded":"2024-09-13T18:55:26.079133+0000","last_fullsized":"2024-09-13T18:55:26.079133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:20.020903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"43'5","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:40.890683+0000","last_change":"2024-09-13T18:26:32.342690+0000","last_active":"2024-09-13T18:55:40.890683+0000","last_peered":"2024-09-13T18:55:40.890683+0000","last_clean":"2024-09-13T18:55:40.890683+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:55:40.890683+0000","last_undegraded":"2024-09-13T18:55:40.890683+0000","last_fullsized":"2024-09-13T18:55:40.890683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.212930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"43'5724","reported_seq":5795,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.404837+0000","last_change":"2024-09-13T18:26:36.396545+0000","last_active":"2024-09-13T18:57:00.404837+0000","last_peered":"2024-09-13T18:57:00.404837+0000","last_clean":"2024-09-13T18:57:00.404837+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:57:00.404837+0000","last_undegraded":"2024-09-13T18:57:00.404837+0000","last_fullsized":"2024-09-13T18:57:00.404837+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5724,"log_dups_size":0,"ondisk_log_size":5724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:38.760215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394351840,"num_objects":99,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":26684,"num_write_kb":545346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"43'23","reported_seq":83,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:35.970133+0000","last_change":"2024-09-13T18:26:31.977823+0000","last_active":"2024-09-13T18:56:35.970133+0000","last_peered":"2024-09-13T18:56:35.970133+0000","last_clean":"2024-09-13T18:56:35.970133+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:56:35.970133+0000","last_undegraded":"2024-09-13T18:56:35.970133+0000","last_fullsized":"2024-09-13T18:56:35.970133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:46.956527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":65,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"29'8","reported_seq":45,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T18:54:56.048253+0000","last_change":"2024-09-13T18:27:22.609917+0000","last_active":"2024-09-13T18:54:56.048253+0000","last_peered":"2024-09-13T18:54:56.048253+0000","last_clean":"2024-09-13T18:54:56.048253+0000","last_became_active":"2024-09-13T18:27:22.609625+0000","last_became_peered":"2024-09-13T18:27:22.609625+0000","last_unstale":"2024-09-13T18:54:56.048253+0000","last_undegraded":"2024-09-13T18:54:56.048253+0000","last_fullsized":"2024-09-13T18:54:56.048253+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:53.184214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"43'4801","reported_seq":4863,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.146983+0000","last_change":"2024-09-13T18:26:36.406675+0000","last_active":"2024-09-13T18:57:00.146983+0000","last_peered":"2024-09-13T18:57:00.146983+0000","last_clean":"2024-09-13T18:57:00.146983+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:57:00.146983+0000","last_undegraded":"2024-09-13T18:57:00.146983+0000","last_fullsized":"2024-09-13T18:57:00.146983+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4801,"log_dups_size":0,"ondisk_log_size":4801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:36.896027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327189222,"num_objects":86,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21538,"num_write_kb":467229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"38'5","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983854+0000","last_change":"2024-09-13T18:26:31.979860+0000","last_active":"2024-09-13T18:55:25.983854+0000","last_peered":"2024-09-13T18:55:25.983854+0000","last_clean":"2024-09-13T18:55:25.983854+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:55:25.983854+0000","last_undegraded":"2024-09-13T18:55:25.983854+0000","last_fullsized":"2024-09-13T18:55:25.983854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:45.323631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":9,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"43'7","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:47.612235+0000","last_change":"2024-09-13T18:26:32.351494+0000","last_active":"2024-09-13T18:56:47.612235+0000","last_peered":"2024-09-13T18:56:47.612235+0000","last_clean":"2024-09-13T18:56:47.612235+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:56:47.612235+0000","last_undegraded":"2024-09-13T18:56:47.612235+0000","last_fullsized":"2024-09-13T18:56:47.612235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:52.804526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"43'4723","reported_seq":4785,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.301135+0000","last_change":"2024-09-13T18:26:36.398763+0000","last_active":"2024-09-13T18:57:00.301135+0000","last_peered":"2024-09-13T18:57:00.301135+0000","last_clean":"2024-09-13T18:57:00.301135+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:57:00.301135+0000","last_undegraded":"2024-09-13T18:57:00.301135+0000","last_fullsized":"2024-09-13T18:57:00.301135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4723,"log_dups_size":0,"ondisk_log_size":4723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:35.417438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302004026,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21549,"num_write_kb":434281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"29'7","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T18:54:52.195299+0000","last_change":"2024-09-13T18:26:32.360976+0000","last_active":"2024-09-13T18:54:52.195299+0000","last_peered":"2024-09-13T18:54:52.195299+0000","last_clean":"2024-09-13T18:54:52.195299+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:54:52.195299+0000","last_undegraded":"2024-09-13T18:54:52.195299+0000","last_fullsized":"2024-09-13T18:54:52.195299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:53:52.373515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"43'1217","reported_seq":1289,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.163136+0000","last_change":"2024-09-13T18:26:31.977891+0000","last_active":"2024-09-13T18:55:53.163136+0000","last_peered":"2024-09-13T18:55:53.163136+0000","last_clean":"2024-09-13T18:55:53.163136+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:55:53.163136+0000","last_undegraded":"2024-09-13T18:55:53.163136+0000","last_fullsized":"2024-09-13T18:55:53.163136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:01.453094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":10235,"num_write":1211,"num_write_kb":4423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"43'5284","reported_seq":5350,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.312723+0000","last_change":"2024-09-13T18:26:36.397484+0000","last_active":"2024-09-13T18:57:00.312723+0000","last_peered":"2024-09-13T18:57:00.312723+0000","last_clean":"2024-09-13T18:57:00.312723+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:57:00.312723+0000","last_undegraded":"2024-09-13T18:57:00.312723+0000","last_fullsized":"2024-09-13T18:57:00.312723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5284,"log_dups_size":0,"ondisk_log_size":5284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:44.686244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356551933,"num_objects":93,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":24035,"num_write_kb":505876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"29'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.000833+0000","last_change":"2024-09-13T18:26:32.345564+0000","last_active":"2024-09-13T18:55:26.000833+0000","last_peered":"2024-09-13T18:55:26.000833+0000","last_clean":"2024-09-13T18:55:26.000833+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:55:26.000833+0000","last_undegraded":"2024-09-13T18:55:26.000833+0000","last_fullsized":"2024-09-13T18:55:26.000833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:54.331806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"43'10","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:33.983335+0000","last_change":"2024-09-13T18:26:31.974107+0000","last_active":"2024-09-13T18:55:33.983335+0000","last_peered":"2024-09-13T18:55:33.983335+0000","last_clean":"2024-09-13T18:55:33.983335+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:55:33.983335+0000","last_undegraded":"2024-09-13T18:55:33.983335+0000","last_fullsized":"2024-09-13T18:55:33.983335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:37.547480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"43'5259","reported_seq":5328,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.452338+0000","last_change":"2024-09-13T18:26:36.405986+0000","last_active":"2024-09-13T18:57:00.452338+0000","last_peered":"2024-09-13T18:57:00.452338+0000","last_clean":"2024-09-13T18:57:00.452338+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:57:00.452338+0000","last_undegraded":"2024-09-13T18:57:00.452338+0000","last_fullsized":"2024-09-13T18:57:00.452338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5259,"log_dups_size":0,"ondisk_log_size":5259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:30.052430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348141733,"num_objects":90,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":24322,"num_write_kb":498911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"43'9","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:36.128739+0000","last_change":"2024-09-13T18:27:22.609984+0000","last_active":"2024-09-13T18:55:36.128739+0000","last_peered":"2024-09-13T18:55:36.128739+0000","last_clean":"2024-09-13T18:55:36.128739+0000","last_became_active":"2024-09-13T18:27:22.609642+0000","last_became_peered":"2024-09-13T18:27:22.609642+0000","last_unstale":"2024-09-13T18:55:36.128739+0000","last_undegraded":"2024-09-13T18:55:36.128739+0000","last_fullsized":"2024-09-13T18:55:36.128739+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:10.097398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.161825+0000","last_change":"2024-09-13T18:26:31.968895+0000","last_active":"2024-09-13T18:55:53.161825+0000","last_peered":"2024-09-13T18:55:53.161825+0000","last_clean":"2024-09-13T18:55:53.161825+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:55:53.161825+0000","last_undegraded":"2024-09-13T18:55:53.161825+0000","last_fullsized":"2024-09-13T18:55:53.161825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:15.525097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"43'6820","reported_seq":6889,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.075354+0000","last_change":"2024-09-13T18:27:22.610932+0000","last_active":"2024-09-13T18:57:01.075354+0000","last_peered":"2024-09-13T18:57:01.075354+0000","last_clean":"2024-09-13T18:57:01.075354+0000","last_became_active":"2024-09-13T18:27:22.610527+0000","last_became_peered":"2024-09-13T18:27:22.610527+0000","last_unstale":"2024-09-13T18:57:01.075354+0000","last_undegraded":"2024-09-13T18:57:01.075354+0000","last_fullsized":"2024-09-13T18:57:01.075354+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6820,"log_dups_size":0,"ondisk_log_size":6820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:49:12.851146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432038324,"num_objects":108,"num_object_clones":0,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":31702,"num_write_kb":622755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"29'4","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.987621+0000","last_change":"2024-09-13T18:26:32.360406+0000","last_active":"2024-09-13T18:55:25.987621+0000","last_peered":"2024-09-13T18:55:25.987621+0000","last_clean":"2024-09-13T18:55:25.987621+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:55:25.987621+0000","last_undegraded":"2024-09-13T18:55:25.987621+0000","last_fullsized":"2024-09-13T18:55:25.987621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:37.417707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"28'4","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.171564+0000","last_change":"2024-09-13T18:26:31.976872+0000","last_active":"2024-09-13T18:55:53.171564+0000","last_peered":"2024-09-13T18:55:53.171564+0000","last_clean":"2024-09-13T18:55:53.171564+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:55:53.171564+0000","last_undegraded":"2024-09-13T18:55:53.171564+0000","last_fullsized":"2024-09-13T18:55:53.171564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:26.163029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"43'5235","reported_seq":5312,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.409347+0000","last_change":"2024-09-13T18:27:22.648736+0000","last_active":"2024-09-13T18:57:00.409347+0000","last_peered":"2024-09-13T18:57:00.409347+0000","last_clean":"2024-09-13T18:57:00.409347+0000","last_became_active":"2024-09-13T18:27:22.648054+0000","last_became_peered":"2024-09-13T18:27:22.648054+0000","last_unstale":"2024-09-13T18:57:00.409347+0000","last_undegraded":"2024-09-13T18:57:00.409347+0000","last_fullsized":"2024-09-13T18:57:00.409347+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5235,"log_dups_size":0,"ondisk_log_size":5235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:11.855024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348247684,"num_objects":91,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":23534,"num_write_kb":471297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"29'14","reported_seq":59,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.025486+0000","last_change":"2024-09-13T18:26:32.343915+0000","last_active":"2024-09-13T18:55:26.025486+0000","last_peered":"2024-09-13T18:55:26.025486+0000","last_clean":"2024-09-13T18:55:26.025486+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:55:26.025486+0000","last_undegraded":"2024-09-13T18:55:26.025486+0000","last_fullsized":"2024-09-13T18:55:26.025486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:19.357243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"28'5","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.162612+0000","last_change":"2024-09-13T18:26:31.979944+0000","last_active":"2024-09-13T18:55:53.162612+0000","last_peered":"2024-09-13T18:55:53.162612+0000","last_clean":"2024-09-13T18:55:53.162612+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:55:53.162612+0000","last_undegraded":"2024-09-13T18:55:53.162612+0000","last_fullsized":"2024-09-13T18:55:53.162612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:21.721921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"43'5232","reported_seq":5303,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.336494+0000","last_change":"2024-09-13T18:26:36.410028+0000","last_active":"2024-09-13T18:57:00.336494+0000","last_peered":"2024-09-13T18:57:00.336494+0000","last_clean":"2024-09-13T18:57:00.336494+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:57:00.336494+0000","last_undegraded":"2024-09-13T18:57:00.336494+0000","last_fullsized":"2024-09-13T18:57:00.336494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5232,"log_dups_size":0,"ondisk_log_size":5232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:32.264341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352362513,"num_objects":88,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":24364,"num_write_kb":507343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"29'9","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.010706+0000","last_change":"2024-09-13T18:26:32.358980+0000","last_active":"2024-09-13T18:55:26.010706+0000","last_peered":"2024-09-13T18:55:26.010706+0000","last_clean":"2024-09-13T18:55:26.010706+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:55:26.010706+0000","last_undegraded":"2024-09-13T18:55:26.010706+0000","last_fullsized":"2024-09-13T18:55:26.010706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:08.748139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"43'283","reported_seq":345,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:52.834136+0000","last_change":"2024-09-13T18:26:31.968972+0000","last_active":"2024-09-13T18:56:52.834136+0000","last_peered":"2024-09-13T18:56:52.834136+0000","last_clean":"2024-09-13T18:56:52.834136+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:56:52.834136+0000","last_undegraded":"2024-09-13T18:56:52.834136+0000","last_fullsized":"2024-09-13T18:56:52.834136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:56.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":52,"num_write":622,"num_write_kb":212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"43'5922","reported_seq":6003,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.069738+0000","last_change":"2024-09-13T18:27:22.645275+0000","last_active":"2024-09-13T18:57:00.069738+0000","last_peered":"2024-09-13T18:57:00.069738+0000","last_clean":"2024-09-13T18:57:00.069738+0000","last_became_active":"2024-09-13T18:27:22.644681+0000","last_became_peered":"2024-09-13T18:27:22.644681+0000","last_unstale":"2024-09-13T18:57:00.069738+0000","last_undegraded":"2024-09-13T18:57:00.069738+0000","last_fullsized":"2024-09-13T18:57:00.069738+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5922,"log_dups_size":0,"ondisk_log_size":5922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:18:38.581700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394336124,"num_objects":101,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":27801,"num_write_kb":553947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"43'6","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:50.152562+0000","last_change":"2024-09-13T18:26:32.342823+0000","last_active":"2024-09-13T18:56:50.152562+0000","last_peered":"2024-09-13T18:56:50.152562+0000","last_clean":"2024-09-13T18:56:50.152562+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:56:50.152562+0000","last_undegraded":"2024-09-13T18:56:50.152562+0000","last_fullsized":"2024-09-13T18:56:50.152562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:28.051252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:30.981249+0000","last_change":"2024-09-13T18:26:31.978976+0000","last_active":"2024-09-13T18:55:30.981249+0000","last_peered":"2024-09-13T18:55:30.981249+0000","last_clean":"2024-09-13T18:55:30.981249+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:55:30.981249+0000","last_undegraded":"2024-09-13T18:55:30.981249+0000","last_fullsized":"2024-09-13T18:55:30.981249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:11.279557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"43'4621","reported_seq":4684,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.841241+0000","last_change":"2024-09-13T18:26:36.408844+0000","last_active":"2024-09-13T18:56:58.841241+0000","last_peered":"2024-09-13T18:56:58.841241+0000","last_clean":"2024-09-13T18:56:58.841241+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:56:58.841241+0000","last_undegraded":"2024-09-13T18:56:58.841241+0000","last_fullsized":"2024-09-13T18:56:58.841241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4621,"log_dups_size":0,"ondisk_log_size":4621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:06.969342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306187993,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":22360,"num_write_kb":450115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"28'7","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.969209+0000","last_change":"2024-09-13T18:26:32.355038+0000","last_active":"2024-09-13T18:55:25.969209+0000","last_peered":"2024-09-13T18:55:25.969209+0000","last_clean":"2024-09-13T18:55:25.969209+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:55:25.969209+0000","last_undegraded":"2024-09-13T18:55:25.969209+0000","last_fullsized":"2024-09-13T18:55:25.969209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.159231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.007762+0000","last_change":"2024-09-13T18:26:31.972931+0000","last_active":"2024-09-13T18:55:26.007762+0000","last_peered":"2024-09-13T18:55:26.007762+0000","last_clean":"2024-09-13T18:55:26.007762+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:55:26.007762+0000","last_undegraded":"2024-09-13T18:55:26.007762+0000","last_fullsized":"2024-09-13T18:55:26.007762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:05.646924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"43'6011","reported_seq":6080,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.360889+0000","last_change":"2024-09-13T18:26:36.398876+0000","last_active":"2024-09-13T18:57:01.360889+0000","last_peered":"2024-09-13T18:57:01.360889+0000","last_clean":"2024-09-13T18:57:01.360889+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:57:01.360889+0000","last_undegraded":"2024-09-13T18:57:01.360889+0000","last_fullsized":"2024-09-13T18:57:01.360889+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6011,"log_dups_size":0,"ondisk_log_size":6011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:56.918239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369124519,"num_objects":90,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":27977,"num_write_kb":562911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"28'6","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.982753+0000","last_change":"2024-09-13T18:26:32.353111+0000","last_active":"2024-09-13T18:55:25.982753+0000","last_peered":"2024-09-13T18:55:25.982753+0000","last_clean":"2024-09-13T18:55:25.982753+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:55:25.982753+0000","last_undegraded":"2024-09-13T18:55:25.982753+0000","last_fullsized":"2024-09-13T18:55:25.982753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:43.927936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"41'6","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.977227+0000","last_change":"2024-09-13T18:26:31.976075+0000","last_active":"2024-09-13T18:55:25.977227+0000","last_peered":"2024-09-13T18:55:25.977227+0000","last_clean":"2024-09-13T18:55:25.977227+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:55:25.977227+0000","last_undegraded":"2024-09-13T18:55:25.977227+0000","last_fullsized":"2024-09-13T18:55:25.977227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:03.693518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"43'5256","reported_seq":5328,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.125917+0000","last_change":"2024-09-13T18:26:36.401777+0000","last_active":"2024-09-13T18:57:01.125917+0000","last_peered":"2024-09-13T18:57:01.125917+0000","last_clean":"2024-09-13T18:57:01.125917+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:57:01.125917+0000","last_undegraded":"2024-09-13T18:57:01.125917+0000","last_fullsized":"2024-09-13T18:57:01.125917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5256,"log_dups_size":0,"ondisk_log_size":5256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:43.440284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343954983,"num_objects":87,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":24458,"num_write_kb":507684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"29'10","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978450+0000","last_change":"2024-09-13T18:26:32.343791+0000","last_active":"2024-09-13T18:55:25.978450+0000","last_peered":"2024-09-13T18:55:25.978450+0000","last_clean":"2024-09-13T18:55:25.978450+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:55:25.978450+0000","last_undegraded":"2024-09-13T18:55:25.978450+0000","last_fullsized":"2024-09-13T18:55:25.978450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:36.296571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"43'128","reported_seq":302,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.464900+0000","last_change":"2024-09-13T18:26:31.970786+0000","last_active":"2024-09-13T18:56:58.464900+0000","last_peered":"2024-09-13T18:56:58.464900+0000","last_clean":"2024-09-13T18:56:58.464900+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:56:58.464900+0000","last_undegraded":"2024-09-13T18:56:58.464900+0000","last_fullsized":"2024-09-13T18:56:58.464900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:54.584009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93931,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":203,"num_write":119,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"43'6142","reported_seq":6212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.392774+0000","last_change":"2024-09-13T18:26:36.401529+0000","last_active":"2024-09-13T18:57:01.392774+0000","last_peered":"2024-09-13T18:57:01.392774+0000","last_clean":"2024-09-13T18:57:01.392774+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:57:01.392774+0000","last_undegraded":"2024-09-13T18:57:01.392774+0000","last_fullsized":"2024-09-13T18:57:01.392774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6142,"log_dups_size":0,"ondisk_log_size":6142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:47.318347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377531496,"num_objects":99,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":28204,"num_write_kb":563741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"28'4","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.987962+0000","last_change":"2024-09-13T18:26:32.350470+0000","last_active":"2024-09-13T18:55:25.987962+0000","last_peered":"2024-09-13T18:55:25.987962+0000","last_clean":"2024-09-13T18:55:25.987962+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:55:25.987962+0000","last_undegraded":"2024-09-13T18:55:25.987962+0000","last_fullsized":"2024-09-13T18:55:25.987962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:55.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"43'14","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:31.126491+0000","last_change":"2024-09-13T18:26:31.972587+0000","last_active":"2024-09-13T18:55:31.126491+0000","last_peered":"2024-09-13T18:55:31.126491+0000","last_clean":"2024-09-13T18:55:31.126491+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:55:31.126491+0000","last_undegraded":"2024-09-13T18:55:31.126491+0000","last_fullsized":"2024-09-13T18:55:31.126491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:27.547763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"43'4295","reported_seq":4365,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.861128+0000","last_change":"2024-09-13T18:26:36.405086+0000","last_active":"2024-09-13T18:56:58.861128+0000","last_peered":"2024-09-13T18:56:58.861128+0000","last_clean":"2024-09-13T18:56:58.861128+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:56:58.861128+0000","last_undegraded":"2024-09-13T18:56:58.861128+0000","last_fullsized":"2024-09-13T18:56:58.861128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4295,"log_dups_size":0,"ondisk_log_size":4295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:02:51.694634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335811241,"num_objects":85,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":19777,"num_write_kb":435588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"29'4","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.994918+0000","last_change":"2024-09-13T18:26:32.350510+0000","last_active":"2024-09-13T18:55:25.994918+0000","last_peered":"2024-09-13T18:55:25.994918+0000","last_clean":"2024-09-13T18:55:25.994918+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:55:25.994918+0000","last_undegraded":"2024-09-13T18:55:25.994918+0000","last_fullsized":"2024-09-13T18:55:25.994918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:40.850255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"38'6","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:45.169259+0000","last_change":"2024-09-13T18:26:31.978355+0000","last_active":"2024-09-13T18:55:45.169259+0000","last_peered":"2024-09-13T18:55:45.169259+0000","last_clean":"2024-09-13T18:55:45.169259+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:55:45.169259+0000","last_undegraded":"2024-09-13T18:55:45.169259+0000","last_fullsized":"2024-09-13T18:55:45.169259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:12.946518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":40,"num_read_kb":68,"num_write":8,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983663+0000","last_change":"2024-09-13T18:26:26.931403+0000","last_active":"2024-09-13T18:55:25.983663+0000","last_peered":"2024-09-13T18:55:25.983663+0000","last_clean":"2024-09-13T18:55:25.983663+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:55:25.983663+0000","last_undegraded":"2024-09-13T18:55:25.983663+0000","last_fullsized":"2024-09-13T18:55:25.983663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_clean_scrub_stamp":"2024-09-13T18:26:24.880028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:30.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"43'5845","reported_seq":5924,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.292017+0000","last_change":"2024-09-13T18:26:36.409827+0000","last_active":"2024-09-13T18:57:00.292017+0000","last_peered":"2024-09-13T18:57:00.292017+0000","last_clean":"2024-09-13T18:57:00.292017+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:57:00.292017+0000","last_undegraded":"2024-09-13T18:57:00.292017+0000","last_fullsized":"2024-09-13T18:57:00.292017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5845,"log_dups_size":0,"ondisk_log_size":5845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:50.482400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377529592,"num_objects":94,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":27327,"num_write_kb":519330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"28'6","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984035+0000","last_change":"2024-09-13T18:26:32.356850+0000","last_active":"2024-09-13T18:55:25.984035+0000","last_peered":"2024-09-13T18:55:25.984035+0000","last_clean":"2024-09-13T18:55:25.984035+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:55:25.984035+0000","last_undegraded":"2024-09-13T18:55:25.984035+0000","last_fullsized":"2024-09-13T18:55:25.984035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:38.710828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"36'4","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.973775+0000","last_change":"2024-09-13T18:26:31.975942+0000","last_active":"2024-09-13T18:55:25.973775+0000","last_peered":"2024-09-13T18:55:25.973775+0000","last_clean":"2024-09-13T18:55:25.973775+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:55:25.973775+0000","last_undegraded":"2024-09-13T18:55:25.973775+0000","last_fullsized":"2024-09-13T18:55:25.973775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:05.840582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":35,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"43'5261","reported_seq":5324,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.262169+0000","last_change":"2024-09-13T18:26:36.409802+0000","last_active":"2024-09-13T18:57:01.262169+0000","last_peered":"2024-09-13T18:57:01.262169+0000","last_clean":"2024-09-13T18:57:01.262169+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:57:01.262169+0000","last_undegraded":"2024-09-13T18:57:01.262169+0000","last_fullsized":"2024-09-13T18:57:01.262169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5261,"log_dups_size":0,"ondisk_log_size":5261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:38.178151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348128716,"num_objects":85,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":23982,"num_write_kb":513904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"29'8","reported_seq":47,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.973293+0000","last_change":"2024-09-13T18:27:22.610607+0000","last_active":"2024-09-13T18:55:25.973293+0000","last_peered":"2024-09-13T18:55:25.973293+0000","last_clean":"2024-09-13T18:55:25.973293+0000","last_became_active":"2024-09-13T18:27:22.610326+0000","last_became_peered":"2024-09-13T18:27:22.610326+0000","last_unstale":"2024-09-13T18:55:25.973293+0000","last_undegraded":"2024-09-13T18:55:25.973293+0000","last_fullsized":"2024-09-13T18:55:25.973293+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:38:04.117608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"38'2","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978777+0000","last_change":"2024-09-13T18:26:31.966618+0000","last_active":"2024-09-13T18:55:25.978777+0000","last_peered":"2024-09-13T18:55:25.978777+0000","last_clean":"2024-09-13T18:55:25.978777+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:55:25.978777+0000","last_undegraded":"2024-09-13T18:55:25.978777+0000","last_fullsized":"2024-09-13T18:55:25.978777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:54:26.422611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"43'4977","reported_seq":5044,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.799194+0000","last_change":"2024-09-13T18:26:36.394344+0000","last_active":"2024-09-13T18:56:58.799194+0000","last_peered":"2024-09-13T18:56:58.799194+0000","last_clean":"2024-09-13T18:56:58.799194+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:56:58.799194+0000","last_undegraded":"2024-09-13T18:56:58.799194+0000","last_fullsized":"2024-09-13T18:56:58.799194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4977,"log_dups_size":0,"ondisk_log_size":4977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:18.425816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360783295,"num_objects":94,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":22736,"num_write_kb":472029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"29'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984452+0000","last_change":"2024-09-13T18:26:32.365655+0000","last_active":"2024-09-13T18:55:25.984452+0000","last_peered":"2024-09-13T18:55:25.984452+0000","last_clean":"2024-09-13T18:55:25.984452+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:55:25.984452+0000","last_undegraded":"2024-09-13T18:55:25.984452+0000","last_fullsized":"2024-09-13T18:55:25.984452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:40.288128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'4","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.973070+0000","last_change":"2024-09-13T18:26:31.970652+0000","last_active":"2024-09-13T18:55:25.973070+0000","last_peered":"2024-09-13T18:55:25.973070+0000","last_clean":"2024-09-13T18:55:25.973070+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:55:25.973070+0000","last_undegraded":"2024-09-13T18:55:25.973070+0000","last_fullsized":"2024-09-13T18:55:25.973070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:11.169661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"43'4832","reported_seq":4893,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.526006+0000","last_change":"2024-09-13T18:26:36.410280+0000","last_active":"2024-09-13T18:56:59.526006+0000","last_peered":"2024-09-13T18:56:59.526006+0000","last_clean":"2024-09-13T18:56:59.526006+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:56:59.526006+0000","last_undegraded":"2024-09-13T18:56:59.526006+0000","last_fullsized":"2024-09-13T18:56:59.526006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4832,"log_dups_size":0,"ondisk_log_size":4832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:01.395290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327189351,"num_objects":82,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":17,"num_read_kb":0,"num_write":22693,"num_write_kb":463670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"28'3","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984106+0000","last_change":"2024-09-13T18:26:32.361575+0000","last_active":"2024-09-13T18:55:25.984106+0000","last_peered":"2024-09-13T18:55:25.984106+0000","last_clean":"2024-09-13T18:55:25.984106+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:55:25.984106+0000","last_undegraded":"2024-09-13T18:55:25.984106+0000","last_fullsized":"2024-09-13T18:55:25.984106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:08.818877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"41'9","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983121+0000","last_change":"2024-09-13T18:26:31.977029+0000","last_active":"2024-09-13T18:55:25.983121+0000","last_peered":"2024-09-13T18:55:25.983121+0000","last_clean":"2024-09-13T18:55:25.983121+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:55:25.983121+0000","last_undegraded":"2024-09-13T18:55:25.983121+0000","last_fullsized":"2024-09-13T18:55:25.983121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:13.701689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"43'4972","reported_seq":5037,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.287176+0000","last_change":"2024-09-13T18:26:36.403479+0000","last_active":"2024-09-13T18:57:00.287176+0000","last_peered":"2024-09-13T18:57:00.287176+0000","last_clean":"2024-09-13T18:57:00.287176+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:57:00.287176+0000","last_undegraded":"2024-09-13T18:57:00.287176+0000","last_fullsized":"2024-09-13T18:57:00.287176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:51.095852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327213309,"num_objects":84,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":22631,"num_write_kb":463382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"29'9","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984952+0000","last_change":"2024-09-13T18:26:32.365143+0000","last_active":"2024-09-13T18:55:25.984952+0000","last_peered":"2024-09-13T18:55:25.984952+0000","last_clean":"2024-09-13T18:55:25.984952+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:55:25.984952+0000","last_undegraded":"2024-09-13T18:55:25.984952+0000","last_fullsized":"2024-09-13T18:55:25.984952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:46.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"41'21","reported_seq":126,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.152848+0000","last_change":"2024-09-13T18:26:31.965802+0000","last_active":"2024-09-13T18:55:53.152848+0000","last_peered":"2024-09-13T18:55:53.152848+0000","last_clean":"2024-09-13T18:55:53.152848+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:55:53.152848+0000","last_undegraded":"2024-09-13T18:55:53.152848+0000","last_fullsized":"2024-09-13T18:55:53.152848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:12.759181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":158,"num_read_kb":157,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"43'5873","reported_seq":5946,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.072818+0000","last_change":"2024-09-13T18:26:36.399235+0000","last_active":"2024-09-13T18:56:59.072818+0000","last_peered":"2024-09-13T18:56:59.072818+0000","last_clean":"2024-09-13T18:56:59.072818+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:56:59.072818+0000","last_undegraded":"2024-09-13T18:56:59.072818+0000","last_fullsized":"2024-09-13T18:56:59.072818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5873,"log_dups_size":0,"ondisk_log_size":5873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:13:54.711083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381775740,"num_objects":100,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":27640,"num_write_kb":536907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"43'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:56.844127+0000","last_change":"2024-09-13T18:26:32.357403+0000","last_active":"2024-09-13T18:56:56.844127+0000","last_peered":"2024-09-13T18:56:56.844127+0000","last_clean":"2024-09-13T18:56:56.844127+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:56:56.844127+0000","last_undegraded":"2024-09-13T18:56:56.844127+0000","last_fullsized":"2024-09-13T18:56:56.844127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:12.600397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"41'12","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.982231+0000","last_change":"2024-09-13T18:26:31.976148+0000","last_active":"2024-09-13T18:55:25.982231+0000","last_peered":"2024-09-13T18:55:25.982231+0000","last_clean":"2024-09-13T18:55:25.982231+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:55:25.982231+0000","last_undegraded":"2024-09-13T18:55:25.982231+0000","last_fullsized":"2024-09-13T18:55:25.982231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:38.023636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1620,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"43'5978","reported_seq":6044,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.401591+0000","last_change":"2024-09-13T18:26:36.406177+0000","last_active":"2024-09-13T18:57:00.401591+0000","last_peered":"2024-09-13T18:57:00.401591+0000","last_clean":"2024-09-13T18:57:00.401591+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:57:00.401591+0000","last_undegraded":"2024-09-13T18:57:00.401591+0000","last_fullsized":"2024-09-13T18:57:00.401591+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5978,"log_dups_size":0,"ondisk_log_size":5978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:00.309995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369133062,"num_objects":92,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":28176,"num_write_kb":551294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"43'12","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978424+0000","last_change":"2024-09-13T18:27:22.609309+0000","last_active":"2024-09-13T18:55:25.978424+0000","last_peered":"2024-09-13T18:55:25.978424+0000","last_clean":"2024-09-13T18:55:25.978424+0000","last_became_active":"2024-09-13T18:27:22.609101+0000","last_became_peered":"2024-09-13T18:27:22.609101+0000","last_unstale":"2024-09-13T18:55:25.978424+0000","last_undegraded":"2024-09-13T18:55:25.978424+0000","last_fullsized":"2024-09-13T18:55:25.978424+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:15.692612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"43'10","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.165844+0000","last_change":"2024-09-13T18:26:31.982114+0000","last_active":"2024-09-13T18:55:53.165844+0000","last_peered":"2024-09-13T18:55:53.165844+0000","last_clean":"2024-09-13T18:55:53.165844+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:55:53.165844+0000","last_undegraded":"2024-09-13T18:55:53.165844+0000","last_fullsized":"2024-09-13T18:55:53.165844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.266528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"43'5767","reported_seq":5836,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.376830+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:57:00.376830+0000","last_peered":"2024-09-13T18:57:00.376830+0000","last_clean":"2024-09-13T18:57:00.376830+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:57:00.376830+0000","last_undegraded":"2024-09-13T18:57:00.376830+0000","last_fullsized":"2024-09-13T18:57:00.376830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5767,"log_dups_size":0,"ondisk_log_size":5767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:58.853971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377595894,"num_objects":98,"num_object_clones":0,"num_object_copies":392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27027,"num_write_kb":550847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"28'8","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984791+0000","last_change":"2024-09-13T18:26:32.351145+0000","last_active":"2024-09-13T18:55:25.984791+0000","last_peered":"2024-09-13T18:55:25.984791+0000","last_clean":"2024-09-13T18:55:25.984791+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:55:25.984791+0000","last_undegraded":"2024-09-13T18:55:25.984791+0000","last_fullsized":"2024-09-13T18:55:25.984791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:57.002435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"38'9","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.158612+0000","last_change":"2024-09-13T18:26:31.961202+0000","last_active":"2024-09-13T18:55:53.158612+0000","last_peered":"2024-09-13T18:55:53.158612+0000","last_clean":"2024-09-13T18:55:53.158612+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:55:53.158612+0000","last_undegraded":"2024-09-13T18:55:53.158612+0000","last_fullsized":"2024-09-13T18:55:53.158612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:27.340127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":93,"num_read_kb":112,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"43'5137","reported_seq":5202,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.582316+0000","last_change":"2024-09-13T18:26:36.410014+0000","last_active":"2024-09-13T18:56:59.582316+0000","last_peered":"2024-09-13T18:56:59.582316+0000","last_clean":"2024-09-13T18:56:59.582316+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:56:59.582316+0000","last_undegraded":"2024-09-13T18:56:59.582316+0000","last_fullsized":"2024-09-13T18:56:59.582316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5137,"log_dups_size":0,"ondisk_log_size":5137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:24.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767127,"num_objects":77,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":23163,"num_write_kb":490059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"29'9","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983909+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:55:25.983909+0000","last_peered":"2024-09-13T18:55:25.983909+0000","last_clean":"2024-09-13T18:55:25.983909+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:55:25.983909+0000","last_undegraded":"2024-09-13T18:55:25.983909+0000","last_fullsized":"2024-09-13T18:55:25.983909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:40.102815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"43'381","reported_seq":468,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:47.848809+0000","last_change":"2024-09-13T18:26:31.974110+0000","last_active":"2024-09-13T18:56:47.848809+0000","last_peered":"2024-09-13T18:56:47.848809+0000","last_clean":"2024-09-13T18:56:47.848809+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:56:47.848809+0000","last_undegraded":"2024-09-13T18:56:47.848809+0000","last_fullsized":"2024-09-13T18:56:47.848809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:56.058561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":93,"num_read_kb":20568,"num_write":381,"num_write_kb":4317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"43'5306","reported_seq":5372,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.367559+0000","last_change":"2024-09-13T18:26:36.394080+0000","last_active":"2024-09-13T18:57:00.367559+0000","last_peered":"2024-09-13T18:57:00.367559+0000","last_clean":"2024-09-13T18:57:00.367559+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:57:00.367559+0000","last_undegraded":"2024-09-13T18:57:00.367559+0000","last_fullsized":"2024-09-13T18:57:00.367559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5306,"log_dups_size":0,"ondisk_log_size":5306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:47.121814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335673691,"num_objects":86,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":24948,"num_write_kb":485267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"43'4","reported_seq":47,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.971630+0000","last_change":"2024-09-13T18:26:32.362873+0000","last_active":"2024-09-13T18:55:25.971630+0000","last_peered":"2024-09-13T18:55:25.971630+0000","last_clean":"2024-09-13T18:55:25.971630+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:55:25.971630+0000","last_undegraded":"2024-09-13T18:55:25.971630+0000","last_fullsized":"2024-09-13T18:55:25.971630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:37.598189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"43'61","reported_seq":123,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:46.092289+0000","last_change":"2024-09-13T18:26:31.976401+0000","last_active":"2024-09-13T18:56:46.092289+0000","last_peered":"2024-09-13T18:56:46.092289+0000","last_clean":"2024-09-13T18:56:46.092289+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:56:46.092289+0000","last_undegraded":"2024-09-13T18:56:46.092289+0000","last_fullsized":"2024-09-13T18:56:46.092289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:26.434292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":62,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"43'5783","reported_seq":5853,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.356403+0000","last_change":"2024-09-13T18:26:36.406228+0000","last_active":"2024-09-13T18:57:00.356403+0000","last_peered":"2024-09-13T18:57:00.356403+0000","last_clean":"2024-09-13T18:57:00.356403+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:57:00.356403+0000","last_undegraded":"2024-09-13T18:57:00.356403+0000","last_fullsized":"2024-09-13T18:57:00.356403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5783,"log_dups_size":0,"ondisk_log_size":5783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:49.198397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352479214,"num_objects":93,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27380,"num_write_kb":527755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"29'9","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.987447+0000","last_change":"2024-09-13T18:26:32.350841+0000","last_active":"2024-09-13T18:55:25.987447+0000","last_peered":"2024-09-13T18:55:25.987447+0000","last_clean":"2024-09-13T18:55:25.987447+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:55:25.987447+0000","last_undegraded":"2024-09-13T18:55:25.987447+0000","last_fullsized":"2024-09-13T18:55:25.987447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:05.137882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'3","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:52.993936+0000","last_change":"2024-09-13T18:26:31.978214+0000","last_active":"2024-09-13T18:55:52.993936+0000","last_peered":"2024-09-13T18:55:52.993936+0000","last_clean":"2024-09-13T18:55:52.993936+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:55:52.993936+0000","last_undegraded":"2024-09-13T18:55:52.993936+0000","last_fullsized":"2024-09-13T18:55:52.993936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:17.381375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"43'5336","reported_seq":5397,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.340297+0000","last_change":"2024-09-13T18:26:36.406090+0000","last_active":"2024-09-13T18:57:00.340297+0000","last_peered":"2024-09-13T18:57:00.340297+0000","last_clean":"2024-09-13T18:57:00.340297+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:57:00.340297+0000","last_undegraded":"2024-09-13T18:57:00.340297+0000","last_fullsized":"2024-09-13T18:57:00.340297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:09.865147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327181663,"num_objects":85,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":24960,"num_write_kb":489215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"29'5","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.969146+0000","last_change":"2024-09-13T18:26:32.357701+0000","last_active":"2024-09-13T18:55:25.969146+0000","last_peered":"2024-09-13T18:55:25.969146+0000","last_clean":"2024-09-13T18:55:25.969146+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:55:25.969146+0000","last_undegraded":"2024-09-13T18:55:25.969146+0000","last_fullsized":"2024-09-13T18:55:25.969146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:47.587683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"43'15","reported_seq":81,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:46.038602+0000","last_change":"2024-09-13T18:26:31.980133+0000","last_active":"2024-09-13T18:56:46.038602+0000","last_peered":"2024-09-13T18:56:46.038602+0000","last_clean":"2024-09-13T18:56:46.038602+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:56:46.038602+0000","last_undegraded":"2024-09-13T18:56:46.038602+0000","last_fullsized":"2024-09-13T18:56:46.038602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:05.354335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":16,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"43'4996","reported_seq":5064,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.435081+0000","last_change":"2024-09-13T18:26:36.406553+0000","last_active":"2024-09-13T18:57:00.435081+0000","last_peered":"2024-09-13T18:57:00.435081+0000","last_clean":"2024-09-13T18:57:00.435081+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:57:00.435081+0000","last_undegraded":"2024-09-13T18:57:00.435081+0000","last_fullsized":"2024-09-13T18:57:00.435081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4996,"log_dups_size":0,"ondisk_log_size":4996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:02.980826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":365112283,"num_objects":94,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":23246,"num_write_kb":507027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"43'5990","reported_seq":6060,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.955468+0000","last_change":"2024-09-13T18:26:36.406986+0000","last_active":"2024-09-13T18:56:58.955468+0000","last_peered":"2024-09-13T18:56:58.955468+0000","last_clean":"2024-09-13T18:56:58.955468+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:56:58.955468+0000","last_undegraded":"2024-09-13T18:56:58.955468+0000","last_fullsized":"2024-09-13T18:56:58.955468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5990,"log_dups_size":0,"ondisk_log_size":5990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:53.984231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419441008,"num_objects":103,"num_object_clones":0,"num_object_copies":412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":28460,"num_write_kb":581141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"29'4","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978161+0000","last_change":"2024-09-13T18:26:32.349081+0000","last_active":"2024-09-13T18:55:25.978161+0000","last_peered":"2024-09-13T18:55:25.978161+0000","last_clean":"2024-09-13T18:55:25.978161+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:55:25.978161+0000","last_undegraded":"2024-09-13T18:55:25.978161+0000","last_fullsized":"2024-09-13T18:55:25.978161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:47.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"43'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:31.137406+0000","last_change":"2024-09-13T18:26:31.968944+0000","last_active":"2024-09-13T18:55:31.137406+0000","last_peered":"2024-09-13T18:55:31.137406+0000","last_clean":"2024-09-13T18:55:31.137406+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:55:31.137406+0000","last_undegraded":"2024-09-13T18:55:31.137406+0000","last_fullsized":"2024-09-13T18:55:31.137406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:29.363312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"43'5472","reported_seq":5544,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.453436+0000","last_change":"2024-09-13T18:26:36.396418+0000","last_active":"2024-09-13T18:57:00.453436+0000","last_peered":"2024-09-13T18:57:00.453436+0000","last_clean":"2024-09-13T18:57:00.453436+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:57:00.453436+0000","last_undegraded":"2024-09-13T18:57:00.453436+0000","last_fullsized":"2024-09-13T18:57:00.453436+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5472,"log_dups_size":0,"ondisk_log_size":5472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:03.252773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":340075764,"num_objects":93,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":25514,"num_write_kb":520259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"29'14","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.111521+0000","last_change":"2024-09-13T18:26:32.351369+0000","last_active":"2024-09-13T18:55:26.111521+0000","last_peered":"2024-09-13T18:55:26.111521+0000","last_clean":"2024-09-13T18:55:26.111521+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:55:26.111521+0000","last_undegraded":"2024-09-13T18:55:26.111521+0000","last_fullsized":"2024-09-13T18:55:26.111521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:10.892347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"36'4","reported_seq":59,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.001369+0000","last_change":"2024-09-13T18:26:31.972048+0000","last_active":"2024-09-13T18:55:26.001369+0000","last_peered":"2024-09-13T18:55:26.001369+0000","last_clean":"2024-09-13T18:55:26.001369+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:55:26.001369+0000","last_undegraded":"2024-09-13T18:55:26.001369+0000","last_fullsized":"2024-09-13T18:55:26.001369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:28.430119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"41'9","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981371+0000","last_change":"2024-09-13T18:26:31.975310+0000","last_active":"2024-09-13T18:55:25.981371+0000","last_peered":"2024-09-13T18:55:25.981371+0000","last_clean":"2024-09-13T18:55:25.981371+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:55:25.981371+0000","last_undegraded":"2024-09-13T18:55:25.981371+0000","last_fullsized":"2024-09-13T18:55:25.981371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:52.915310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"29'7","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.999839+0000","last_change":"2024-09-13T18:26:32.345825+0000","last_active":"2024-09-13T18:55:25.999839+0000","last_peered":"2024-09-13T18:55:25.999839+0000","last_clean":"2024-09-13T18:55:25.999839+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:55:25.999839+0000","last_undegraded":"2024-09-13T18:55:25.999839+0000","last_fullsized":"2024-09-13T18:55:25.999839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:25.200489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"43'4065","reported_seq":4127,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.144665+0000","last_change":"2024-09-13T18:26:36.391520+0000","last_active":"2024-09-13T18:57:01.144665+0000","last_peered":"2024-09-13T18:57:01.144665+0000","last_clean":"2024-09-13T18:57:01.144665+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:57:01.144665+0000","last_undegraded":"2024-09-13T18:57:01.144665+0000","last_fullsized":"2024-09-13T18:57:01.144665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4065,"log_dups_size":0,"ondisk_log_size":4065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:42.638713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255876134,"num_objects":63,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":19156,"num_write_kb":424944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"43'754","reported_seq":826,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:35.286998+0000","last_change":"2024-09-13T18:26:31.976603+0000","last_active":"2024-09-13T18:56:35.286998+0000","last_peered":"2024-09-13T18:56:35.286998+0000","last_clean":"2024-09-13T18:56:35.286998+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:56:35.286998+0000","last_undegraded":"2024-09-13T18:56:35.286998+0000","last_fullsized":"2024-09-13T18:56:35.286998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:03.965376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196273,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":19627,"num_write":735,"num_write_kb":4345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"29'9","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.979121+0000","last_change":"2024-09-13T18:26:32.352982+0000","last_active":"2024-09-13T18:55:25.979121+0000","last_peered":"2024-09-13T18:55:25.979121+0000","last_clean":"2024-09-13T18:55:25.979121+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:55:25.979121+0000","last_undegraded":"2024-09-13T18:55:25.979121+0000","last_fullsized":"2024-09-13T18:55:25.979121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:55.670360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"43'5828","reported_seq":5896,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.118020+0000","last_change":"2024-09-13T18:26:36.403817+0000","last_active":"2024-09-13T18:57:00.118020+0000","last_peered":"2024-09-13T18:57:00.118020+0000","last_clean":"2024-09-13T18:57:00.118020+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:57:00.118020+0000","last_undegraded":"2024-09-13T18:57:00.118020+0000","last_fullsized":"2024-09-13T18:57:00.118020+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5828,"log_dups_size":0,"ondisk_log_size":5828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:34.892901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377551906,"num_objects":98,"num_object_clones":0,"num_object_copies":392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":26701,"num_write_kb":557751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"43'8","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:47.227020+0000","last_change":"2024-09-13T18:26:32.357856+0000","last_active":"2024-09-13T18:55:47.227020+0000","last_peered":"2024-09-13T18:55:47.227020+0000","last_clean":"2024-09-13T18:55:47.227020+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:55:47.227020+0000","last_undegraded":"2024-09-13T18:55:47.227020+0000","last_fullsized":"2024-09-13T18:55:47.227020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:26.316214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"29'10","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.972396+0000","last_change":"2024-09-13T18:26:32.350966+0000","last_active":"2024-09-13T18:55:25.972396+0000","last_peered":"2024-09-13T18:55:25.972396+0000","last_clean":"2024-09-13T18:55:25.972396+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:55:25.972396+0000","last_undegraded":"2024-09-13T18:55:25.972396+0000","last_fullsized":"2024-09-13T18:55:25.972396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:52.632112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"29'11","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.977018+0000","last_change":"2024-09-13T18:26:32.355806+0000","last_active":"2024-09-13T18:55:25.977018+0000","last_peered":"2024-09-13T18:55:25.977018+0000","last_clean":"2024-09-13T18:55:25.977018+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:55:25.977018+0000","last_undegraded":"2024-09-13T18:55:25.977018+0000","last_fullsized":"2024-09-13T18:55:25.977018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.988231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"29'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.999728+0000","last_change":"2024-09-13T18:26:32.340965+0000","last_active":"2024-09-13T18:55:25.999728+0000","last_peered":"2024-09-13T18:55:25.999728+0000","last_clean":"2024-09-13T18:55:25.999728+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:55:25.999728+0000","last_undegraded":"2024-09-13T18:55:25.999728+0000","last_fullsized":"2024-09-13T18:55:25.999728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:01:04.716919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"29'7","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.046037+0000","last_change":"2024-09-13T18:26:32.349625+0000","last_active":"2024-09-13T18:55:26.046037+0000","last_peered":"2024-09-13T18:55:26.046037+0000","last_clean":"2024-09-13T18:55:26.046037+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:55:26.046037+0000","last_undegraded":"2024-09-13T18:55:26.046037+0000","last_fullsized":"2024-09-13T18:55:26.046037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.519754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"28'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978502+0000","last_change":"2024-09-13T18:26:32.349170+0000","last_active":"2024-09-13T18:55:25.978502+0000","last_peered":"2024-09-13T18:55:25.978502+0000","last_clean":"2024-09-13T18:55:25.978502+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:55:25.978502+0000","last_undegraded":"2024-09-13T18:55:25.978502+0000","last_fullsized":"2024-09-13T18:55:25.978502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:43.942732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"29'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981727+0000","last_change":"2024-09-13T18:26:32.364786+0000","last_active":"2024-09-13T18:55:25.981727+0000","last_peered":"2024-09-13T18:55:25.981727+0000","last_clean":"2024-09-13T18:55:25.981727+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:55:25.981727+0000","last_undegraded":"2024-09-13T18:55:25.981727+0000","last_fullsized":"2024-09-13T18:55:25.981727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:22.949875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"28'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.010595+0000","last_change":"2024-09-13T18:26:32.347541+0000","last_active":"2024-09-13T18:55:26.010595+0000","last_peered":"2024-09-13T18:55:26.010595+0000","last_clean":"2024-09-13T18:55:26.010595+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:55:26.010595+0000","last_undegraded":"2024-09-13T18:55:26.010595+0000","last_fullsized":"2024-09-13T18:55:26.010595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:58.144128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"29'5","reported_seq":50,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T18:54:50.138427+0000","last_change":"2024-09-13T18:26:32.353359+0000","last_active":"2024-09-13T18:54:50.138427+0000","last_peered":"2024-09-13T18:54:50.138427+0000","last_clean":"2024-09-13T18:54:50.138427+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:54:50.138427+0000","last_undegraded":"2024-09-13T18:54:50.138427+0000","last_fullsized":"2024-09-13T18:54:50.138427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:09.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"29'10","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.001188+0000","last_change":"2024-09-13T18:26:32.349566+0000","last_active":"2024-09-13T18:55:26.001188+0000","last_peered":"2024-09-13T18:55:26.001188+0000","last_clean":"2024-09-13T18:55:26.001188+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:55:26.001188+0000","last_undegraded":"2024-09-13T18:55:26.001188+0000","last_fullsized":"2024-09-13T18:55:26.001188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:22.341899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"43'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:50.192401+0000","last_change":"2024-09-13T18:26:32.358640+0000","last_active":"2024-09-13T18:55:50.192401+0000","last_peered":"2024-09-13T18:55:50.192401+0000","last_clean":"2024-09-13T18:55:50.192401+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:55:50.192401+0000","last_undegraded":"2024-09-13T18:55:50.192401+0000","last_fullsized":"2024-09-13T18:55:50.192401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:40.174754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"43'11","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:50.939549+0000","last_change":"2024-09-13T18:26:32.353675+0000","last_active":"2024-09-13T18:55:50.939549+0000","last_peered":"2024-09-13T18:55:50.939549+0000","last_clean":"2024-09-13T18:55:50.939549+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:55:50.939549+0000","last_undegraded":"2024-09-13T18:55:50.939549+0000","last_fullsized":"2024-09-13T18:55:50.939549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:58.501843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"28'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981840+0000","last_change":"2024-09-13T18:26:32.353223+0000","last_active":"2024-09-13T18:55:25.981840+0000","last_peered":"2024-09-13T18:55:25.981840+0000","last_clean":"2024-09-13T18:55:25.981840+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:55:25.981840+0000","last_undegraded":"2024-09-13T18:55:25.981840+0000","last_fullsized":"2024-09-13T18:55:25.981840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:45.898213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"43'10","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:51.407384+0000","last_change":"2024-09-13T18:26:32.357354+0000","last_active":"2024-09-13T18:55:51.407384+0000","last_peered":"2024-09-13T18:55:51.407384+0000","last_clean":"2024-09-13T18:55:51.407384+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:55:51.407384+0000","last_undegraded":"2024-09-13T18:55:51.407384+0000","last_fullsized":"2024-09-13T18:55:51.407384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:19.185008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"43'11","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:56.238386+0000","last_change":"2024-09-13T18:26:32.348832+0000","last_active":"2024-09-13T18:56:56.238386+0000","last_peered":"2024-09-13T18:56:56.238386+0000","last_clean":"2024-09-13T18:56:56.238386+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:56:56.238386+0000","last_undegraded":"2024-09-13T18:56:56.238386+0000","last_fullsized":"2024-09-13T18:56:56.238386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:11:59.874789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"28'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.972098+0000","last_change":"2024-09-13T18:26:32.356902+0000","last_active":"2024-09-13T18:55:25.972098+0000","last_peered":"2024-09-13T18:55:25.972098+0000","last_clean":"2024-09-13T18:55:25.972098+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:55:25.972098+0000","last_undegraded":"2024-09-13T18:55:25.972098+0000","last_fullsized":"2024-09-13T18:55:25.972098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:18.362772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"28'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.968811+0000","last_change":"2024-09-13T18:26:32.347047+0000","last_active":"2024-09-13T18:55:25.968811+0000","last_peered":"2024-09-13T18:55:25.968811+0000","last_clean":"2024-09-13T18:55:25.968811+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:55:25.968811+0000","last_undegraded":"2024-09-13T18:55:25.968811+0000","last_fullsized":"2024-09-13T18:55:25.968811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:40.882789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"43'10","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:52.028230+0000","last_change":"2024-09-13T18:26:32.349651+0000","last_active":"2024-09-13T18:56:52.028230+0000","last_peered":"2024-09-13T18:56:52.028230+0000","last_clean":"2024-09-13T18:56:52.028230+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:56:52.028230+0000","last_undegraded":"2024-09-13T18:56:52.028230+0000","last_fullsized":"2024-09-13T18:56:52.028230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:33.356563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"29'10","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T18:54:40.392762+0000","last_change":"2024-09-13T18:26:32.344607+0000","last_active":"2024-09-13T18:54:40.392762+0000","last_peered":"2024-09-13T18:54:40.392762+0000","last_clean":"2024-09-13T18:54:40.392762+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:54:40.392762+0000","last_undegraded":"2024-09-13T18:54:40.392762+0000","last_fullsized":"2024-09-13T18:54:40.392762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:31.658172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"29'6","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.000730+0000","last_change":"2024-09-13T18:26:32.347717+0000","last_active":"2024-09-13T18:55:26.000730+0000","last_peered":"2024-09-13T18:55:26.000730+0000","last_clean":"2024-09-13T18:55:26.000730+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:55:26.000730+0000","last_undegraded":"2024-09-13T18:55:26.000730+0000","last_fullsized":"2024-09-13T18:55:26.000730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:47.187837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"29'6","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.971507+0000","last_change":"2024-09-13T18:26:32.359506+0000","last_active":"2024-09-13T18:55:25.971507+0000","last_peered":"2024-09-13T18:55:25.971507+0000","last_clean":"2024-09-13T18:55:25.971507+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:55:25.971507+0000","last_undegraded":"2024-09-13T18:55:25.971507+0000","last_fullsized":"2024-09-13T18:55:25.971507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:39:57.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"43'12","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:37.221615+0000","last_change":"2024-09-13T18:26:32.357729+0000","last_active":"2024-09-13T18:55:37.221615+0000","last_peered":"2024-09-13T18:55:37.221615+0000","last_clean":"2024-09-13T18:55:37.221615+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:55:37.221615+0000","last_undegraded":"2024-09-13T18:55:37.221615+0000","last_fullsized":"2024-09-13T18:55:37.221615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:05.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"29'9","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981655+0000","last_change":"2024-09-13T18:26:32.357980+0000","last_active":"2024-09-13T18:55:25.981655+0000","last_peered":"2024-09-13T18:55:25.981655+0000","last_clean":"2024-09-13T18:55:25.981655+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:55:25.981655+0000","last_undegraded":"2024-09-13T18:55:25.981655+0000","last_fullsized":"2024-09-13T18:55:25.981655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:00.858702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"43'13","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.154866+0000","last_change":"2024-09-13T18:26:32.365638+0000","last_active":"2024-09-13T18:56:59.154866+0000","last_peered":"2024-09-13T18:56:59.154866+0000","last_clean":"2024-09-13T18:56:59.154866+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:56:59.154866+0000","last_undegraded":"2024-09-13T18:56:59.154866+0000","last_fullsized":"2024-09-13T18:56:59.154866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.065853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"29'3","reported_seq":47,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.109132+0000","last_change":"2024-09-13T18:26:32.345496+0000","last_active":"2024-09-13T18:55:26.109132+0000","last_peered":"2024-09-13T18:55:26.109132+0000","last_clean":"2024-09-13T18:55:26.109132+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:55:26.109132+0000","last_undegraded":"2024-09-13T18:55:26.109132+0000","last_fullsized":"2024-09-13T18:55:26.109132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:22.447315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"43'4","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:35.970044+0000","last_change":"2024-09-13T18:26:32.356901+0000","last_active":"2024-09-13T18:56:35.970044+0000","last_peered":"2024-09-13T18:56:35.970044+0000","last_clean":"2024-09-13T18:56:35.970044+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:56:35.970044+0000","last_undegraded":"2024-09-13T18:56:35.970044+0000","last_fullsized":"2024-09-13T18:56:35.970044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:03.311603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"28'8","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.976794+0000","last_change":"2024-09-13T18:26:32.355203+0000","last_active":"2024-09-13T18:55:25.976794+0000","last_peered":"2024-09-13T18:55:25.976794+0000","last_clean":"2024-09-13T18:55:25.976794+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:55:25.976794+0000","last_undegraded":"2024-09-13T18:55:25.976794+0000","last_fullsized":"2024-09-13T18:55:25.976794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:52.769708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"29'6","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983681+0000","last_change":"2024-09-13T18:26:32.358190+0000","last_active":"2024-09-13T18:55:25.983681+0000","last_peered":"2024-09-13T18:55:25.983681+0000","last_clean":"2024-09-13T18:55:25.983681+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:55:25.983681+0000","last_undegraded":"2024-09-13T18:55:25.983681+0000","last_fullsized":"2024-09-13T18:55:25.983681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:35.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"28'4","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.968851+0000","last_change":"2024-09-13T18:26:32.355121+0000","last_active":"2024-09-13T18:55:25.968851+0000","last_peered":"2024-09-13T18:55:25.968851+0000","last_clean":"2024-09-13T18:55:25.968851+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:55:25.968851+0000","last_undegraded":"2024-09-13T18:55:25.968851+0000","last_fullsized":"2024-09-13T18:55:25.968851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:48.427332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"29'6","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.987776+0000","last_change":"2024-09-13T18:26:32.341497+0000","last_active":"2024-09-13T18:55:25.987776+0000","last_peered":"2024-09-13T18:55:25.987776+0000","last_clean":"2024-09-13T18:55:25.987776+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:55:25.987776+0000","last_undegraded":"2024-09-13T18:55:25.987776+0000","last_fullsized":"2024-09-13T18:55:25.987776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:15.510048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"28'2","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978877+0000","last_change":"2024-09-13T18:26:32.350106+0000","last_active":"2024-09-13T18:55:25.978877+0000","last_peered":"2024-09-13T18:55:25.978877+0000","last_clean":"2024-09-13T18:55:25.978877+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:55:25.978877+0000","last_undegraded":"2024-09-13T18:55:25.978877+0000","last_fullsized":"2024-09-13T18:55:25.978877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:12.387451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"29'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984165+0000","last_change":"2024-09-13T18:26:32.362196+0000","last_active":"2024-09-13T18:55:25.984165+0000","last_peered":"2024-09-13T18:55:25.984165+0000","last_clean":"2024-09-13T18:55:25.984165+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:55:25.984165+0000","last_undegraded":"2024-09-13T18:55:25.984165+0000","last_fullsized":"2024-09-13T18:55:25.984165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:14.946904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":11298053892,"num_objects":2881,"num_object_clones":0,"num_object_copies":11524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2881,"num_whiteouts":0,"num_read":797,"num_read_kb":0,"num_write":800719,"num_write_kb":16319459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9769914368,"data_stored":22543564800,"data_compressed":1072758487,"data_compressed_allocated":1504456704,"data_compressed_original":14278107136,"omap_allocated":0,"internal_metadata":0},"log_size":172420,"ondisk_log_size":172420,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":472,"ondisk_log_size":472,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":16496904,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":1362,"num_read_kb":52080,"num_write":3724,"num_write_kb":17653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22700032,"data_stored":32993808,"data_compressed":2441304,"data_compressed_allocated":3506176,"data_compressed_original":13950976,"omap_allocated":0,"internal_metadata":0},"log_size":3487,"ondisk_log_size":3487,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509810,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2270664,"kb_used_data":1249600,"kb_used_omap":1,"kb_used_meta":1021054,"kb_avail":92101176,"statfs":{"total":96636764160,"available":94311604224,"internally_reserved":0,"allocated":1279590400,"data_stored":2930967260,"data_compressed":139752701,"data_compressed_allocated":196038656,"data_compressed_original":1847668736,"omap_allocated":1593,"internal_metadata":1045559751},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":1.3979999999999999,"15min":0.96599999999999997},"min":{"1min":0.224,"5min":0.223,"15min":0.223},"max":{"1min":9.0050000000000008,"5min":10.43,"15min":10.43},"last":0.25600000000000001},{"interface":"front","average":{"1min":2.5230000000000001,"5min":1.8320000000000001,"15min":1.141},"min":{"1min":0.25900000000000001,"5min":0.245,"15min":0.245},"max":{"1min":9.9190000000000005,"5min":16.600999999999999,"15min":16.600999999999999},"last":0.27200000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.9630000000000001,"5min":3.831,"15min":1.825},"min":{"1min":0.26000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":7.7409999999999997,"5min":52.290999999999997,"15min":52.290999999999997},"last":0.31},{"interface":"front","average":{"1min":1.6719999999999999,"5min":4.0910000000000002,"15min":1.9239999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":7.8410000000000002,"5min":52.149000000000001,"15min":52.149000000000001},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.518,"15min":1.0580000000000001},"min":{"1min":0.33500000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.8360000000000003,"5min":14.933,"15min":14.933},"last":1.419},{"interface":"front","average":{"1min":1.7509999999999999,"5min":1.4239999999999999,"15min":1.048},"min":{"1min":0.29099999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":6.7160000000000002,"5min":17.792000000000002,"15min":17.792000000000002},"last":0.43099999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":2.4580000000000002,"15min":1.3899999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":8.1809999999999992,"5min":15.622,"15min":15.622},"last":2.6400000000000001},{"interface":"front","average":{"1min":3.2429999999999999,"5min":3.2829999999999999,"15min":1.6799999999999999},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":14.993,"5min":28.248000000000001,"15min":28.248000000000001},"last":1.8979999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.304,"15min":1.0820000000000001},"min":{"1min":0.31,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.3570000000000002,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.232,"15min":1.048},"min":{"1min":0.38700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":4.4569999999999999,"5min":7.9409999999999998,"15min":7.9409999999999998},"last":0.48699999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.95699999999999996,"15min":0.97699999999999998},"min":{"1min":0.376,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.7410000000000001,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.099,"5min":0.96999999999999997,"15min":1.004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.2480000000000002,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":0.34599999999999997}]},{"osd":7,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.0429999999999999,"15min":1.0720000000000001},"min":{"1min":0.39300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.8319999999999999,"5min":12.021000000000001,"15min":12.021000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.91300000000000003,"15min":0.98599999999999999},"min":{"1min":0.32500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":3.9430000000000001,"5min":9.0440000000000005,"15min":9.0440000000000005},"last":0.40200000000000002}]}]},{"osd":7,"up_from":15,"seq":64424509809,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1929960,"kb_used_data":1066208,"kb_used_omap":1,"kb_used_meta":863742,"kb_avail":92441880,"statfs":{"total":96636764160,"available":94660485120,"internally_reserved":0,"allocated":1091796992,"data_stored":2515818119,"data_compressed":119837066,"data_compressed_allocated":168169472,"data_compressed_original":1592455168,"omap_allocated":1589,"internal_metadata":884472267},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":2.121,"15min":1.206},"min":{"1min":0.23000000000000001,"5min":0.217,"15min":0.217},"max":{"1min":14.778,"5min":24.414000000000001,"15min":24.414000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":2.1680000000000001,"5min":2.254,"15min":1.3460000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":14.814,"5min":24.469000000000001,"15min":24.469000000000001},"last":0.79000000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":2.4889999999999999,"5min":3.355,"15min":1.617},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":6.0490000000000004,"5min":46.161999999999999,"15min":46.161999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":5.1929999999999996,"5min":4.3570000000000002,"15min":2.0129999999999999},"min":{"1min":0.29099999999999998,"5min":0.252,"15min":0.252},"max":{"1min":39.853999999999999,"5min":45.713000000000001,"15min":45.713000000000001},"last":0.95799999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.228,"15min":0.98699999999999999},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":5.3230000000000004,"5min":7.4740000000000002,"15min":7.4740000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.329,"15min":0.97899999999999998},"min":{"1min":0.315,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.8559999999999999,"5min":9.1679999999999993,"15min":9.1679999999999993},"last":1.1000000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":3.4129999999999998,"5min":3.6960000000000002,"15min":1.8029999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":17.721,"5min":35.494999999999997,"15min":35.494999999999997},"last":1.052},{"interface":"front","average":{"1min":2.278,"5min":2.976,"15min":1.6850000000000001},"min":{"1min":0.33800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":11.353999999999999,"5min":19.946000000000002,"15min":19.946000000000002},"last":1.0089999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.92600000000000005,"15min":0.91700000000000004},"min":{"1min":0.25800000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.2749999999999999,"5min":7.2880000000000003,"15min":7.2880000000000003},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.0820000000000001,"15min":1.0589999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":7.8810000000000002,"5min":7.8810000000000002,"15min":7.8810000000000002},"last":1.1950000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.80700000000000005,"15min":0.877},"min":{"1min":0.29399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":3.6200000000000001,"5min":6.7309999999999999,"15min":6.7309999999999999},"last":0.753},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90100000000000002,"15min":0.96999999999999997},"min":{"1min":0.32000000000000001,"5min":0.318,"15min":0.318},"max":{"1min":3.5819999999999999,"5min":6.6699999999999999,"15min":6.6699999999999999},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:56:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.2889999999999999,"15min":1.137},"min":{"1min":0.35299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.4260000000000002,"5min":32.491,"15min":32.491},"last":1.143},{"interface":"front","average":{"1min":0.54100000000000004,"5min":1.137,"15min":1.091},"min":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.82899999999999996,"5min":27.291,"15min":27.291},"last":0.90400000000000003}]}]},{"osd":4,"up_from":15,"seq":64424509809,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2176860,"kb_used_data":1236180,"kb_used_omap":1,"kb_used_meta":940670,"kb_avail":92194980,"statfs":{"total":96636764160,"available":94407659520,"internally_reserved":0,"allocated":1265848320,"data_stored":2899198048,"data_compressed":137665483,"data_compressed_allocated":193257472,"data_compressed_original":1826861056,"omap_allocated":1594,"internal_metadata":963246534},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":2.2599999999999998,"5min":2.1259999999999999,"15min":1.214},"min":{"1min":0.26300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":8.8740000000000006,"5min":28.593,"15min":28.593},"last":0.441},{"interface":"front","average":{"1min":1.9850000000000001,"5min":1.837,"15min":1.1930000000000001},"min":{"1min":0.26400000000000001,"5min":0.255,"15min":0.255},"max":{"1min":10.711,"5min":28.559000000000001,"15min":28.559000000000001},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":2.6299999999999999,"5min":2.907,"15min":1.49},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":10.044,"5min":37.100999999999999,"15min":37.100999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":2.0089999999999999,"5min":3.4969999999999999,"15min":1.6910000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":8.3810000000000002,"5min":44.798000000000002,"15min":44.798000000000002},"last":0.28799999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":2.6789999999999998,"5min":1.845,"15min":1.1859999999999999},"min":{"1min":0.30199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":16.617000000000001,"5min":27.634,"15min":27.634},"last":0.42399999999999999},{"interface":"front","average":{"1min":2.8279999999999998,"5min":1.7170000000000001,"15min":1.2050000000000001},"min":{"1min":0.317,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":16.396999999999998,"5min":27.606000000000002,"15min":27.606000000000002},"last":0.34200000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":3.496,"5min":3.339,"15min":1.6499999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":18.503,"5min":40.676000000000002,"15min":40.676000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":5.9610000000000003,"5min":3.8050000000000002,"15min":1.887},"min":{"1min":0.47999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":39.649000000000001,"5min":40.881999999999998,"15min":40.881999999999998},"last":0.36699999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:56:26 2024","interfaces":[{"interface":"back","average":{"1min":1.7769999999999999,"5min":0.876,"15min":0.88300000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":8.5649999999999995,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":0.91600000000000004,"15min":0.98499999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.0810000000000004,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.167,"15min":1.0900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":5.984,"5min":9.4570000000000007,"15min":9.4570000000000007},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.1559999999999999,"5min":0.996,"15min":1.0109999999999999},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":5.1059999999999999,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.38600000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8779999999999999,"5min":1.4279999999999999,"15min":1.177},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":10.101000000000001,"5min":10.101000000000001,"15min":10.101000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.9239999999999999,"5min":1.492,"15min":1.274},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":10.127000000000001,"5min":10.127000000000001,"15min":10.127000000000001},"last":0.50600000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542513,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2824836,"kb_used_data":1633712,"kb_used_omap":1,"kb_used_meta":1190142,"kb_avail":91547004,"statfs":{"total":96636764160,"available":93744132096,"internally_reserved":0,"allocated":1672921088,"data_stored":3841409738,"data_compressed":183202141,"data_compressed_allocated":257212416,"data_compressed_original":2425958400,"omap_allocated":1591,"internal_metadata":1218705865},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:32 2024","interfaces":[{"interface":"back","average":{"1min":3.0569999999999999,"5min":3.3959999999999999,"15min":1.5109999999999999},"min":{"1min":0.20300000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":22.670000000000002,"5min":22.670000000000002,"15min":22.670000000000002},"last":0.29499999999999998},{"interface":"front","average":{"1min":3.1179999999999999,"5min":3.5009999999999999,"15min":1.677},"min":{"1min":0.246,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":22.728000000000002,"5min":22.728000000000002,"15min":22.728000000000002},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":7.9240000000000004,"5min":6.282,"15min":2.4940000000000002},"min":{"1min":0.38500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":42.046999999999997,"5min":42.046999999999997,"15min":42.046999999999997},"last":0.316},{"interface":"front","average":{"1min":7.6500000000000004,"5min":6.1680000000000001,"15min":2.5630000000000002},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":42.024999999999999,"5min":42.024999999999999,"15min":42.024999999999999},"last":0.503}]},{"osd":2,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":6.4550000000000001,"5min":4.2300000000000004,"15min":1.9259999999999999},"min":{"1min":0.27700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":41.909999999999997,"5min":41.909999999999997,"15min":41.909999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":5.1609999999999996,"5min":3.1819999999999999,"15min":1.5229999999999999},"min":{"1min":0.317,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":27.25,"5min":27.25,"15min":27.25},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:56:32 2024","interfaces":[{"interface":"back","average":{"1min":4.6550000000000002,"5min":3.3100000000000001,"15min":1.7430000000000001},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":42.003,"5min":42.003,"15min":42.003},"last":0.73299999999999998},{"interface":"front","average":{"1min":5.4329999999999998,"5min":3.9399999999999999,"15min":1.956},"min":{"1min":0.39300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":41.951000000000001,"5min":41.951000000000001,"15min":41.951000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":4.79,"5min":2.46,"15min":1.508},"min":{"1min":0.38100000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":22.988,"5min":22.988,"15min":22.988},"last":0.68700000000000006},{"interface":"front","average":{"1min":3.2829999999999999,"5min":2.2799999999999998,"15min":1.462},"min":{"1min":0.35199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":23.030000000000001,"5min":23.030000000000001,"15min":23.030000000000001},"last":0.80300000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":4.0529999999999999,"5min":2.9689999999999999,"15min":1.6719999999999999},"min":{"1min":0.35799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":23.087,"5min":23.087,"15min":23.087},"last":0.64200000000000002},{"interface":"front","average":{"1min":5.4210000000000003,"5min":3.4540000000000002,"15min":1.843},"min":{"1min":0.52700000000000002,"5min":0.317,"15min":0.317},"max":{"1min":23.425000000000001,"5min":23.425000000000001,"15min":23.425000000000001},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":4.9379999999999997,"5min":3.1600000000000001,"15min":1.714},"min":{"1min":0.437,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":41.975999999999999,"5min":41.975999999999999,"15min":41.975999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":4.9550000000000001,"5min":2.956,"15min":1.6619999999999999},"min":{"1min":0.42699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":23.457999999999998,"5min":23.457999999999998,"15min":23.457999999999998},"last":0.75600000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542517,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1698048,"kb_used_data":966456,"kb_used_omap":1,"kb_used_meta":731582,"kb_avail":92673792,"statfs":{"total":96636764160,"available":94897963008,"internally_reserved":0,"allocated":989650944,"data_stored":2283014993,"data_compressed":108755460,"data_compressed_allocated":152350720,"data_compressed_original":1445974016,"omap_allocated":1593,"internal_metadata":749140423},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":2.5489999999999999,"15min":1.2270000000000001},"min":{"1min":0.17799999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":5.8200000000000003,"5min":37.600000000000001,"15min":37.600000000000001},"last":2.6019999999999999},{"interface":"front","average":{"1min":1.056,"5min":2.4569999999999999,"15min":1.266},"min":{"1min":0.218,"5min":0.19,"15min":0.19},"max":{"1min":6.0839999999999996,"5min":37.651000000000003,"15min":37.651000000000003},"last":2.6800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":2.5800000000000001,"5min":3.0449999999999999,"15min":1.5209999999999999},"min":{"1min":0.26500000000000001,"5min":0.255,"15min":0.255},"max":{"1min":12.244,"5min":18.055,"15min":18.055},"last":9.1189999999999998},{"interface":"front","average":{"1min":2.851,"5min":2.8900000000000001,"15min":1.431},"min":{"1min":0.28899999999999998,"5min":0.188,"15min":0.188},"max":{"1min":11.916,"5min":18.175999999999998,"15min":18.175999999999998},"last":10.196}]},{"osd":3,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":2.6179999999999999,"5min":3.5470000000000002,"15min":1.7010000000000001},"min":{"1min":0.313,"5min":0.249,"15min":0.249},"max":{"1min":12.888999999999999,"5min":25.832999999999998,"15min":25.832999999999998},"last":3.5350000000000001},{"interface":"front","average":{"1min":2.621,"5min":3.5230000000000001,"15min":1.77},"min":{"1min":0.375,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":12.702999999999999,"5min":17.515000000000001,"15min":17.515000000000001},"last":3.508}]},{"osd":4,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.2190000000000001,"15min":0.996},"min":{"1min":0.27700000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.5590000000000002,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":3.036},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.242,"15min":1.0820000000000001},"min":{"1min":0.33700000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":5.0899999999999999,"5min":8.0999999999999996,"15min":8.0999999999999996},"last":1.7949999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.27,"15min":1.032},"min":{"1min":0.29799999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":8.4800000000000004,"5min":8.4800000000000004,"15min":8.4800000000000004},"last":2.6360000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.4319999999999999,"15min":1.1220000000000001},"min":{"1min":0.36499999999999999,"5min":0.309,"15min":0.309},"max":{"1min":3.8079999999999998,"5min":8.907,"15min":8.907},"last":2.3460000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":1.321,"15min":1.133},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.032,"5min":9.6720000000000006,"15min":9.6720000000000006},"last":1.875},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.2070000000000001,"15min":1.069},"min":{"1min":0.35799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":5.3289999999999997,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":2.375}]},{"osd":7,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3560000000000001,"15min":1.129},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":4.7939999999999996,"5min":8.9250000000000007,"15min":8.9250000000000007},"last":2.661},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.585,"15min":1.236},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":6.335,"5min":9.2330000000000005,"15min":9.2330000000000005},"last":1.835}]}]},{"osd":5,"up_from":13,"seq":55834575218,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1570116,"kb_used_data":872956,"kb_used_omap":1,"kb_used_meta":697150,"kb_avail":92801724,"statfs":{"total":96636764160,"available":95028965376,"internally_reserved":0,"allocated":893906944,"data_stored":2074980080,"data_compressed":98665143,"data_compressed_allocated":138342400,"data_compressed_original":1319673856,"omap_allocated":1593,"internal_metadata":713882055},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":1.8879999999999999,"5min":1.8919999999999999,"15min":1.109},"min":{"1min":0.23799999999999999,"5min":0.224,"15min":0.224},"max":{"1min":8.2240000000000002,"5min":9.952,"15min":9.952},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.95,"5min":2.0139999999999998,"15min":1.2350000000000001},"min":{"1min":0.29899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.2579999999999991,"5min":14.317,"15min":14.317},"last":0.86799999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":1.9550000000000001,"5min":2.29,"15min":1.3129999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":10.058,"5min":10.85,"15min":10.85},"last":0.66700000000000004},{"interface":"front","average":{"1min":2.1629999999999998,"5min":2.2719999999999998,"15min":1.3140000000000001},"min":{"1min":0.29299999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":10.462999999999999,"5min":10.868,"15min":10.868},"last":0.82299999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":3.8519999999999999,"5min":1.915,"15min":1.1759999999999999},"min":{"1min":0.27300000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":38.484000000000002,"5min":38.484000000000002,"15min":38.484000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":4.0890000000000004,"5min":2.2309999999999999,"15min":1.286},"min":{"1min":0.27100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":40.686,"5min":40.686,"15min":40.686},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":3.9580000000000002,"5min":4.2320000000000002,"15min":1.9750000000000001},"min":{"1min":0.38100000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":19.093,"5min":72.534999999999997,"15min":72.534999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":3.5720000000000001,"5min":4.0199999999999996,"15min":1.9710000000000001},"min":{"1min":0.43099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":19.071000000000002,"5min":72.486999999999995,"15min":72.486999999999995},"last":0.75600000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":1.02,"15min":0.91900000000000004},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.2290000000000001,"5min":7.8710000000000004,"15min":7.8710000000000004},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.131,"15min":1.0760000000000001},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":8.7829999999999995,"5min":8.7829999999999995,"15min":8.7829999999999995},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":0.96399999999999997,"15min":0.997},"min":{"1min":0.28699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.04,"15min":1.0489999999999999},"min":{"1min":0.35599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.516,"5min":4.516,"15min":4.516},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.02,"15min":1.0209999999999999},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":4.4249999999999998,"5min":7.6020000000000003,"15min":7.6020000000000003},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.0169999999999999,"15min":1.101},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.1900000000000004,"5min":7.9109999999999996,"15min":7.9109999999999996},"last":1.9810000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575216,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2673912,"kb_used_data":1523088,"kb_used_omap":1,"kb_used_meta":1148350,"kb_avail":91697928,"statfs":{"total":96636764160,"available":93898678272,"internally_reserved":0,"allocated":1559642112,"data_stored":3627326782,"data_compressed":172575409,"data_compressed_allocated":241844224,"data_compressed_original":2309791744,"omap_allocated":1594,"internal_metadata":1175910854},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":3.2570000000000001,"15min":1.4910000000000001},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":7.9580000000000002,"5min":28.297000000000001,"15min":28.297000000000001},"last":10.279},{"interface":"front","average":{"1min":3.1040000000000001,"5min":2.7749999999999999,"15min":1.341},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":19.785,"5min":19.785,"15min":19.785},"last":10.162000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":2.8340000000000001,"15min":1.385},"min":{"1min":0.252,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":8.0630000000000006,"5min":25.609999999999999,"15min":25.609999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":4.0890000000000004,"5min":3.097,"15min":1.536},"min":{"1min":0.38500000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":20.173999999999999,"5min":30.469999999999999,"15min":30.469999999999999},"last":10.465}]},{"osd":3,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":5.524,"5min":5.7249999999999996,"15min":2.383},"min":{"1min":0.58199999999999996,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":20.001000000000001,"5min":36.643999999999998,"15min":36.643999999999998},"last":10.406000000000001},{"interface":"front","average":{"1min":5.5519999999999996,"5min":5.9119999999999999,"15min":2.5110000000000001},"min":{"1min":0.59899999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":20.030999999999999,"5min":36.670000000000002,"15min":36.670000000000002},"last":10.429}]},{"osd":4,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":1.7549999999999999,"15min":1.1759999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.2249999999999996,"5min":7.6799999999999997,"15min":7.6799999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.26,"5min":2.1299999999999999,"15min":1.331},"min":{"1min":0.32800000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":6.5540000000000003,"5min":14.906000000000001,"15min":14.906000000000001},"last":0.47999999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":3.1880000000000002,"5min":1.9350000000000001,"15min":1.3149999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":9.0519999999999996,"5min":9.0519999999999996,"15min":9.0519999999999996},"last":10.259},{"interface":"front","average":{"1min":2.7149999999999999,"5min":1.9410000000000001,"15min":1.3300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.1050000000000004,"5min":12.27,"15min":12.27},"last":10.231999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":2.577,"5min":2.4300000000000002,"15min":1.468},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":9.077,"5min":28.981999999999999,"15min":28.981999999999999},"last":10.538},{"interface":"front","average":{"1min":2.1819999999999999,"5min":2.1379999999999999,"15min":1.3799999999999999},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":15.727,"5min":15.727,"15min":15.727},"last":10.342000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":2.379,"15min":1.4359999999999999},"min":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":7.9930000000000003,"5min":28.100000000000001,"15min":28.100000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":1.728,"5min":2.4860000000000002,"15min":1.518},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.0220000000000002,"5min":28.896999999999998,"15min":28.896999999999998},"last":0.41099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575217,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1942528,"kb_used_data":1150840,"kb_used_omap":1,"kb_used_meta":791678,"kb_avail":92429312,"statfs":{"total":96636764160,"available":94647615488,"internally_reserved":0,"allocated":1178460160,"data_stored":2729905844,"data_compressed":130141336,"data_compressed_allocated":182214656,"data_compressed_original":1733922816,"omap_allocated":1594,"internal_metadata":810678726},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":5.1710000000000003,"5min":4.0839999999999996,"15min":1.732},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":29.712,"5min":29.712,"15min":29.712},"last":0.34100000000000003},{"interface":"front","average":{"1min":5.4039999999999999,"5min":3.0979999999999999,"15min":1.4810000000000001},"min":{"1min":0.495,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":29.748000000000001,"5min":29.748000000000001,"15min":29.748000000000001},"last":0.36499999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":1.7849999999999999,"15min":1.014},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":17.611000000000001,"5min":17.611000000000001,"15min":17.611000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":2.847,"5min":1.9570000000000001,"15min":1.1180000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":17.501999999999999,"5min":17.501999999999999,"15min":17.501999999999999},"last":0.375}]},{"osd":3,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":2.9580000000000002,"5min":3.4689999999999999,"15min":1.615},"min":{"1min":0.26200000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":17.260000000000002,"5min":17.260000000000002,"15min":17.260000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":2.5459999999999998,"5min":3.0129999999999999,"15min":1.512},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":17.088000000000001,"5min":17.088000000000001,"15min":17.088000000000001},"last":0.439}]},{"osd":4,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":5.3200000000000003,"5min":2.3799999999999999,"15min":1.407},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":40.101999999999997,"5min":40.101999999999997,"15min":40.101999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":5.2830000000000004,"5min":2.089,"15min":1.2989999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":40.981000000000002,"5min":40.981000000000002,"15min":40.981000000000002},"last":0.69599999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":4.5419999999999998,"5min":1.857,"15min":1.252},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":40.701999999999998,"5min":40.701999999999998,"15min":40.701999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":4.8120000000000003,"5min":2.0230000000000001,"15min":1.3440000000000001},"min":{"1min":0.36899999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":40.130000000000003,"5min":40.130000000000003,"15min":40.130000000000003},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":5.7699999999999996,"5min":2.1499999999999999,"15min":1.3979999999999999},"min":{"1min":0.41399999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":51.719000000000001,"5min":51.719000000000001,"15min":51.719000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":5.8150000000000004,"5min":2.1989999999999998,"15min":1.448},"min":{"1min":0.49099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":51.75,"5min":51.75,"15min":51.75},"last":0.79300000000000004}]},{"osd":7,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":5.5179999999999998,"5min":2.3180000000000001,"15min":1.4770000000000001},"min":{"1min":0.36899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":58.810000000000002,"5min":58.810000000000002,"15min":58.810000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":4.609,"5min":2.125,"15min":1.4370000000000001},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":39.622,"5min":39.622,"15min":39.622},"last":0.74199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2064384,"data_stored":3814150,"data_compressed":409575,"data_compressed_allocated":573440,"data_compressed_original":2342912,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":94659,"data_compressed":1568,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5124096,"data_stored":8485180,"data_compressed":790472,"data_compressed_allocated":1138688,"data_compressed_original":4517888,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4136960,"data_stored":4196050,"data_compressed":20605,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1777664,"data_stored":4196686,"data_compressed":519875,"data_compressed_allocated":692224,"data_compressed_original":3125248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9441280,"data_stored":12204761,"data_compressed":699209,"data_compressed_allocated":1056768,"data_compressed_original":3842048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1138999296,"data_stored":2622500864,"data_compressed":124826741,"data_compressed_allocated":174882816,"data_compressed_original":1658384384,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1543139328,"data_stored":3583045632,"data_compressed":170484417,"data_compressed_allocated":238940160,"data_compressed_original":2278846464,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":973479936,"data_stored":2257514496,"data_compressed":107562294,"data_compressed_allocated":150663168,"data_compressed_original":1434697728,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1628086272,"data_stored":3742507008,"data_compressed":178128102,"data_compressed_allocated":250073088,"data_compressed_original":2364493824,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1234100224,"data_stored":2836041728,"data_compressed":134852831,"data_compressed_allocated":189272064,"data_compressed_original":1791213568,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":892964864,"data_stored":2072641536,"data_compressed":98566295,"data_compressed_allocated":138211328,"data_compressed_original":1317888000,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1277333504,"data_stored":2926530560,"data_compressed":139232826,"data_compressed_allocated":195346432,"data_compressed_original":1844543488,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1081810944,"data_stored":2502782976,"data_compressed":119104981,"data_compressed_allocated":167067648,"data_compressed_original":1588039680,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:57:05.047 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T18:57:05.047 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:57:05.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.139+0000 7f5e57fff700 1 Processor -- start 2024-09-13T18:57:05.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.139+0000 7f5e57fff700 1 -- start start 2024-09-13T18:57:05.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.139+0000 7f5e57fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e580cbd50 0x7f5e580b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:05.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.139+0000 7f5e57fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5815da20 con 0x7f5e58071390 2024-09-13T18:57:05.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.139+0000 7f5e57fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e580ca750 con 0x7f5e580ca210 2024-09-13T18:57:05.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.139+0000 7f5e57fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580caa30 con 0x7f5e580cbd50 2024-09-13T18:57:05.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.143+0000 7f5e577fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5e58071390 0x7f5e58163e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59128/0 (socket says 172.21.15.146:59128) 2024-09-13T18:57:05.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.143+0000 7f5e577fe700 1 -- 172.21.15.146:0/596595234 learned_addr learned my addr 172.21.15.146:0/596595234 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:57:05.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.143+0000 7f5e56ffd700 1 --2- 172.21.15.146:0/596595234 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e580cbd50 0x7f5e580b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:05.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.143+0000 7f5e56ffd700 1 -- 172.21.15.146:0/596595234 >> v1:172.21.15.154:6789/0 conn(0x7f5e580ca210 legacy=0x7f5e580701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:05.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.143+0000 7f5e56ffd700 1 -- 172.21.15.146:0/596595234 >> v1:172.21.15.146:6789/0 conn(0x7f5e58071390 legacy=0x7f5e58163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:05.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.143+0000 7f5e56ffd700 1 -- 172.21.15.146:0/596595234 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5815cc60 con 0x7f5e580cbd50 2024-09-13T18:57:05.165 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.143+0000 7f5e56ffd700 1 --2- 172.21.15.146:0/596595234 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e580cbd50 0x7f5e580b1c40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009880 tx=0x7f5e48009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9af143da3fa0d85 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:05.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.155+0000 7f5e55ffb700 1 -- 172.21.15.146:0/596595234 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e48011830 con 0x7f5e580cbd50 2024-09-13T18:57:05.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.155+0000 7f5e55ffb700 1 -- 172.21.15.146:0/596595234 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e48011990 con 0x7f5e580cbd50 2024-09-13T18:57:05.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.155+0000 7f5e55ffb700 1 -- 172.21.15.146:0/596595234 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e48011d00 con 0x7f5e580cbd50 2024-09-13T18:57:05.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.155+0000 7f5e57fff700 1 -- 172.21.15.146:0/596595234 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e580cbd50 msgr2=0x7f5e580b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:05.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.155+0000 7f5e57fff700 1 --2- 172.21.15.146:0/596595234 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e580cbd50 0x7f5e580b1c40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009880 tx=0x7f5e48009b90 comp rx=0 tx=0).stop 2024-09-13T18:57:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.155+0000 7f5e57fff700 1 -- 172.21.15.146:0/596595234 shutdown_connections 2024-09-13T18:57:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.155+0000 7f5e57fff700 1 --2- 172.21.15.146:0/596595234 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e580cbd50 0x7f5e580b1c40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.155+0000 7f5e57fff700 1 -- 172.21.15.146:0/596595234 >> 172.21.15.146:0/596595234 conn(0x7f5e58066270 msgr2=0x7f5e58066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.155+0000 7f5e57fff700 1 -- 172.21.15.146:0/596595234 shutdown_connections 2024-09-13T18:57:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.155+0000 7f5e57fff700 1 -- 172.21.15.146:0/596595234 wait complete. 2024-09-13T18:57:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.155+0000 7f5e57fff700 1 Processor -- start 2024-09-13T18:57:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.155+0000 7f5e57fff700 1 -- start start 2024-09-13T18:57:05.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e57fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e58071390 0x7f5e580cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:05.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e57fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e580ca210 0x7f5e580ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:05.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e57fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e580cbd50 0x7f5e58143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:05.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e57fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580cd9e0 con 0x7f5e580ca210 2024-09-13T18:57:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e57fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580cdca0 con 0x7f5e58071390 2024-09-13T18:57:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e57fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580ccf10 con 0x7f5e580cbd50 2024-09-13T18:57:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e56ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e58071390 0x7f5e580cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e56ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e58071390 0x7f5e580cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:35000/0 (socket says 172.21.15.146:35000) 2024-09-13T18:57:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e56ffd700 1 -- 172.21.15.146:0/2590173106 learned_addr learned my addr 172.21.15.146:0/2590173106 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e567fc700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e580ca210 0x7f5e580ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e577fe700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e580cbd50 0x7f5e58143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e577fe700 1 -- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e58071390 msgr2=0x7f5e580cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e577fe700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e58071390 0x7f5e580cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e577fe700 1 -- 172.21.15.146:0/2590173106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e580ca210 msgr2=0x7f5e580ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e577fe700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e580ca210 0x7f5e580ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e577fe700 1 -- 172.21.15.146:0/2590173106 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e480094b0 con 0x7f5e580cbd50 2024-09-13T18:57:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e577fe700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e580cbd50 0x7f5e58143d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5e5001db00 tx=0x7f5e5001db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e3f7fe700 1 -- 172.21.15.146:0/2590173106 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50007280 con 0x7f5e580cbd50 2024-09-13T18:57:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e57fff700 1 -- 172.21.15.146:0/2590173106 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e580cd1f0 con 0x7f5e580cbd50 2024-09-13T18:57:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e57fff700 1 -- 172.21.15.146:0/2590173106 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e58150d70 con 0x7f5e580cbd50 2024-09-13T18:57:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e3f7fe700 1 -- 172.21.15.146:0/2590173106 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e500073e0 con 0x7f5e580cbd50 2024-09-13T18:57:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.159+0000 7f5e3f7fe700 1 -- 172.21.15.146:0/2590173106 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50007770 con 0x7f5e580cbd50 2024-09-13T18:57:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.163+0000 7f5e3f7fe700 1 -- 172.21.15.146:0/2590173106 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5e500078d0 con 0x7f5e580cbd50 2024-09-13T18:57:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.163+0000 7f5e57fff700 1 -- 172.21.15.146:0/2590173106 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e38005310 con 0x7f5e580cbd50 2024-09-13T18:57:05.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.167+0000 7f5e3f7fe700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e44066730 0x7f5e44068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:05.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.167+0000 7f5e3f7fe700 1 -- 172.21.15.146:0/2590173106 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f5e5000c020 con 0x7f5e580cbd50 2024-09-13T18:57:05.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.167+0000 7f5e56ffd700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e44066730 0x7f5e44068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:05.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.167+0000 7f5e56ffd700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e44066730 0x7f5e44068bf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009bd0 tx=0x7f5e4801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:05.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.171+0000 7f5e3f7fe700 1 -- 172.21.15.146:0/2590173106 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5e500511a0 con 0x7f5e580cbd50 2024-09-13T18:57:05.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.455+0000 7f5e57fff700 1 -- 172.21.15.146:0/2590173106 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e38000bc0 con 0x7f5e44066730 2024-09-13T18:57:05.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.467+0000 7f5e3f7fe700 1 -- 172.21.15.146:0/2590173106 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370788 (secure 0 0 0) 0x7f5e38000bc0 con 0x7f5e44066730 2024-09-13T18:57:05.485 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:57:05.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.475+0000 7f5e57fff700 1 -- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e44066730 msgr2=0x7f5e44068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:05.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.475+0000 7f5e57fff700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e44066730 0x7f5e44068bf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009bd0 tx=0x7f5e4801c040 comp rx=0 tx=0).stop 2024-09-13T18:57:05.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.475+0000 7f5e57fff700 1 -- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e580cbd50 msgr2=0x7f5e58143d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:05.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.475+0000 7f5e57fff700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e580cbd50 0x7f5e58143d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5e5001db00 tx=0x7f5e5001db30 comp rx=0 tx=0).stop 2024-09-13T18:57:05.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.475+0000 7f5e57fff700 1 -- 172.21.15.146:0/2590173106 shutdown_connections 2024-09-13T18:57:05.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.475+0000 7f5e57fff700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5e44066730 0x7f5e44068bf0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.475+0000 7f5e57fff700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e580cbd50 0x7f5e58143d10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.475+0000 7f5e57fff700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e580ca210 0x7f5e580ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.475+0000 7f5e57fff700 1 --2- 172.21.15.146:0/2590173106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e58071390 0x7f5e580cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.475+0000 7f5e57fff700 1 -- 172.21.15.146:0/2590173106 >> 172.21.15.146:0/2590173106 conn(0x7f5e58066270 msgr2=0x7f5e58152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:05.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.479+0000 7f5e57fff700 1 -- 172.21.15.146:0/2590173106 shutdown_connections 2024-09-13T18:57:05.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.479+0000 7f5e57fff700 1 -- 172.21.15.146:0/2590173106 wait complete. 2024-09-13T18:57:05.504 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:57:05.518 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":955,"stamp":"2024-09-13T18:57:03.595662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":11315141164,"num_objects":2956,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2956,"num_whiteouts":0,"num_read":2275,"num_read_kb":52180,"num_write":804602,"num_write_kb":16339528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":176496,"ondisk_log_size":176496,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":17086924,"kb_used_data":9699040,"kb_used_omap":12,"kb_used_meta":7384371,"kb_avail":737887796,"statfs":{"total":773094113280,"available":755597103104,"internally_reserved":0,"allocated":9931816960,"data_stored":22902620864,"data_compressed":1090594739,"data_compressed_allocated":1529430016,"data_compressed_original":14502305792,"omap_allocated":12741,"internal_metadata":7561596475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1832536528,"num_objects":-438,"num_object_clones":0,"num_object_copies":-1752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-438,"num_whiteouts":0,"num_read":7,"num_read_kb":21,"num_write":7455,"num_write_kb":28720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004697"},"pg_stats":[{"pgid":"2.1f","version":"43'464","reported_seq":542,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:57.832117+0000","last_change":"2024-09-13T18:26:31.972797+0000","last_active":"2024-09-13T18:56:57.832117+0000","last_peered":"2024-09-13T18:56:57.832117+0000","last_clean":"2024-09-13T18:56:57.832117+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:56:57.832117+0000","last_undegraded":"2024-09-13T18:56:57.832117+0000","last_fullsized":"2024-09-13T18:56:57.832117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:50.511747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3814072,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":428,"num_write":455,"num_write_kb":3981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"29'4","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.078641+0000","last_change":"2024-09-13T18:26:32.349901+0000","last_active":"2024-09-13T18:55:26.078641+0000","last_peered":"2024-09-13T18:55:26.078641+0000","last_clean":"2024-09-13T18:55:26.078641+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:55:26.078641+0000","last_undegraded":"2024-09-13T18:55:26.078641+0000","last_fullsized":"2024-09-13T18:55:26.078641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:24.978813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"43'5677","reported_seq":5755,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.295010+0000","last_change":"2024-09-13T18:26:36.407579+0000","last_active":"2024-09-13T18:57:00.295010+0000","last_peered":"2024-09-13T18:57:00.295010+0000","last_clean":"2024-09-13T18:57:00.295010+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:57:00.295010+0000","last_undegraded":"2024-09-13T18:57:00.295010+0000","last_fullsized":"2024-09-13T18:57:00.295010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5677,"log_dups_size":0,"ondisk_log_size":5677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:53:56.370013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348712512,"num_objects":93,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":26674,"num_write_kb":537654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"41'12","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.079133+0000","last_change":"2024-09-13T18:26:31.980716+0000","last_active":"2024-09-13T18:55:26.079133+0000","last_peered":"2024-09-13T18:55:26.079133+0000","last_clean":"2024-09-13T18:55:26.079133+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:55:26.079133+0000","last_undegraded":"2024-09-13T18:55:26.079133+0000","last_fullsized":"2024-09-13T18:55:26.079133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:20.020903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"43'5","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:40.890683+0000","last_change":"2024-09-13T18:26:32.342690+0000","last_active":"2024-09-13T18:55:40.890683+0000","last_peered":"2024-09-13T18:55:40.890683+0000","last_clean":"2024-09-13T18:55:40.890683+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:55:40.890683+0000","last_undegraded":"2024-09-13T18:55:40.890683+0000","last_fullsized":"2024-09-13T18:55:40.890683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.212930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"43'5724","reported_seq":5795,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.404837+0000","last_change":"2024-09-13T18:26:36.396545+0000","last_active":"2024-09-13T18:57:00.404837+0000","last_peered":"2024-09-13T18:57:00.404837+0000","last_clean":"2024-09-13T18:57:00.404837+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:57:00.404837+0000","last_undegraded":"2024-09-13T18:57:00.404837+0000","last_fullsized":"2024-09-13T18:57:00.404837+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5724,"log_dups_size":0,"ondisk_log_size":5724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:38.760215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394351840,"num_objects":99,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":26684,"num_write_kb":545346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"43'23","reported_seq":83,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:35.970133+0000","last_change":"2024-09-13T18:26:31.977823+0000","last_active":"2024-09-13T18:56:35.970133+0000","last_peered":"2024-09-13T18:56:35.970133+0000","last_clean":"2024-09-13T18:56:35.970133+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:56:35.970133+0000","last_undegraded":"2024-09-13T18:56:35.970133+0000","last_fullsized":"2024-09-13T18:56:35.970133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:46.956527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":65,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"29'8","reported_seq":45,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T18:54:56.048253+0000","last_change":"2024-09-13T18:27:22.609917+0000","last_active":"2024-09-13T18:54:56.048253+0000","last_peered":"2024-09-13T18:54:56.048253+0000","last_clean":"2024-09-13T18:54:56.048253+0000","last_became_active":"2024-09-13T18:27:22.609625+0000","last_became_peered":"2024-09-13T18:27:22.609625+0000","last_unstale":"2024-09-13T18:54:56.048253+0000","last_undegraded":"2024-09-13T18:54:56.048253+0000","last_fullsized":"2024-09-13T18:54:56.048253+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:53.184214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"43'4801","reported_seq":4863,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.146983+0000","last_change":"2024-09-13T18:26:36.406675+0000","last_active":"2024-09-13T18:57:00.146983+0000","last_peered":"2024-09-13T18:57:00.146983+0000","last_clean":"2024-09-13T18:57:00.146983+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:57:00.146983+0000","last_undegraded":"2024-09-13T18:57:00.146983+0000","last_fullsized":"2024-09-13T18:57:00.146983+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4801,"log_dups_size":0,"ondisk_log_size":4801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:36.896027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327189222,"num_objects":86,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21538,"num_write_kb":467229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"38'5","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983854+0000","last_change":"2024-09-13T18:26:31.979860+0000","last_active":"2024-09-13T18:55:25.983854+0000","last_peered":"2024-09-13T18:55:25.983854+0000","last_clean":"2024-09-13T18:55:25.983854+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:55:25.983854+0000","last_undegraded":"2024-09-13T18:55:25.983854+0000","last_fullsized":"2024-09-13T18:55:25.983854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:45.323631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":9,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"43'7","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:47.612235+0000","last_change":"2024-09-13T18:26:32.351494+0000","last_active":"2024-09-13T18:56:47.612235+0000","last_peered":"2024-09-13T18:56:47.612235+0000","last_clean":"2024-09-13T18:56:47.612235+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:56:47.612235+0000","last_undegraded":"2024-09-13T18:56:47.612235+0000","last_fullsized":"2024-09-13T18:56:47.612235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:52.804526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"43'4723","reported_seq":4785,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.301135+0000","last_change":"2024-09-13T18:26:36.398763+0000","last_active":"2024-09-13T18:57:00.301135+0000","last_peered":"2024-09-13T18:57:00.301135+0000","last_clean":"2024-09-13T18:57:00.301135+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:57:00.301135+0000","last_undegraded":"2024-09-13T18:57:00.301135+0000","last_fullsized":"2024-09-13T18:57:00.301135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4723,"log_dups_size":0,"ondisk_log_size":4723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:35.417438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302004026,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21549,"num_write_kb":434281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"29'7","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T18:54:52.195299+0000","last_change":"2024-09-13T18:26:32.360976+0000","last_active":"2024-09-13T18:54:52.195299+0000","last_peered":"2024-09-13T18:54:52.195299+0000","last_clean":"2024-09-13T18:54:52.195299+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:54:52.195299+0000","last_undegraded":"2024-09-13T18:54:52.195299+0000","last_fullsized":"2024-09-13T18:54:52.195299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:53:52.373515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"43'1217","reported_seq":1289,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.163136+0000","last_change":"2024-09-13T18:26:31.977891+0000","last_active":"2024-09-13T18:55:53.163136+0000","last_peered":"2024-09-13T18:55:53.163136+0000","last_clean":"2024-09-13T18:55:53.163136+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:55:53.163136+0000","last_undegraded":"2024-09-13T18:55:53.163136+0000","last_fullsized":"2024-09-13T18:55:53.163136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:01.453094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":10235,"num_write":1211,"num_write_kb":4423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"43'5284","reported_seq":5350,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.312723+0000","last_change":"2024-09-13T18:26:36.397484+0000","last_active":"2024-09-13T18:57:00.312723+0000","last_peered":"2024-09-13T18:57:00.312723+0000","last_clean":"2024-09-13T18:57:00.312723+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:57:00.312723+0000","last_undegraded":"2024-09-13T18:57:00.312723+0000","last_fullsized":"2024-09-13T18:57:00.312723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5284,"log_dups_size":0,"ondisk_log_size":5284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:44.686244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356551933,"num_objects":93,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":24035,"num_write_kb":505876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"29'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.000833+0000","last_change":"2024-09-13T18:26:32.345564+0000","last_active":"2024-09-13T18:55:26.000833+0000","last_peered":"2024-09-13T18:55:26.000833+0000","last_clean":"2024-09-13T18:55:26.000833+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:55:26.000833+0000","last_undegraded":"2024-09-13T18:55:26.000833+0000","last_fullsized":"2024-09-13T18:55:26.000833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:54.331806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"43'10","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:33.983335+0000","last_change":"2024-09-13T18:26:31.974107+0000","last_active":"2024-09-13T18:55:33.983335+0000","last_peered":"2024-09-13T18:55:33.983335+0000","last_clean":"2024-09-13T18:55:33.983335+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:55:33.983335+0000","last_undegraded":"2024-09-13T18:55:33.983335+0000","last_fullsized":"2024-09-13T18:55:33.983335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:37.547480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"43'5259","reported_seq":5328,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.452338+0000","last_change":"2024-09-13T18:26:36.405986+0000","last_active":"2024-09-13T18:57:00.452338+0000","last_peered":"2024-09-13T18:57:00.452338+0000","last_clean":"2024-09-13T18:57:00.452338+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:57:00.452338+0000","last_undegraded":"2024-09-13T18:57:00.452338+0000","last_fullsized":"2024-09-13T18:57:00.452338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5259,"log_dups_size":0,"ondisk_log_size":5259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:30.052430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348141733,"num_objects":90,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":24322,"num_write_kb":498911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"43'9","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:36.128739+0000","last_change":"2024-09-13T18:27:22.609984+0000","last_active":"2024-09-13T18:55:36.128739+0000","last_peered":"2024-09-13T18:55:36.128739+0000","last_clean":"2024-09-13T18:55:36.128739+0000","last_became_active":"2024-09-13T18:27:22.609642+0000","last_became_peered":"2024-09-13T18:27:22.609642+0000","last_unstale":"2024-09-13T18:55:36.128739+0000","last_undegraded":"2024-09-13T18:55:36.128739+0000","last_fullsized":"2024-09-13T18:55:36.128739+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:10.097398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.161825+0000","last_change":"2024-09-13T18:26:31.968895+0000","last_active":"2024-09-13T18:55:53.161825+0000","last_peered":"2024-09-13T18:55:53.161825+0000","last_clean":"2024-09-13T18:55:53.161825+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:55:53.161825+0000","last_undegraded":"2024-09-13T18:55:53.161825+0000","last_fullsized":"2024-09-13T18:55:53.161825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:15.525097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"43'6820","reported_seq":6889,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.075354+0000","last_change":"2024-09-13T18:27:22.610932+0000","last_active":"2024-09-13T18:57:01.075354+0000","last_peered":"2024-09-13T18:57:01.075354+0000","last_clean":"2024-09-13T18:57:01.075354+0000","last_became_active":"2024-09-13T18:27:22.610527+0000","last_became_peered":"2024-09-13T18:27:22.610527+0000","last_unstale":"2024-09-13T18:57:01.075354+0000","last_undegraded":"2024-09-13T18:57:01.075354+0000","last_fullsized":"2024-09-13T18:57:01.075354+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6820,"log_dups_size":0,"ondisk_log_size":6820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:49:12.851146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432038324,"num_objects":108,"num_object_clones":0,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":31702,"num_write_kb":622755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"29'4","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.987621+0000","last_change":"2024-09-13T18:26:32.360406+0000","last_active":"2024-09-13T18:55:25.987621+0000","last_peered":"2024-09-13T18:55:25.987621+0000","last_clean":"2024-09-13T18:55:25.987621+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:55:25.987621+0000","last_undegraded":"2024-09-13T18:55:25.987621+0000","last_fullsized":"2024-09-13T18:55:25.987621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:37.417707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"28'4","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.171564+0000","last_change":"2024-09-13T18:26:31.976872+0000","last_active":"2024-09-13T18:55:53.171564+0000","last_peered":"2024-09-13T18:55:53.171564+0000","last_clean":"2024-09-13T18:55:53.171564+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:55:53.171564+0000","last_undegraded":"2024-09-13T18:55:53.171564+0000","last_fullsized":"2024-09-13T18:55:53.171564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:26.163029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"43'5235","reported_seq":5312,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.409347+0000","last_change":"2024-09-13T18:27:22.648736+0000","last_active":"2024-09-13T18:57:00.409347+0000","last_peered":"2024-09-13T18:57:00.409347+0000","last_clean":"2024-09-13T18:57:00.409347+0000","last_became_active":"2024-09-13T18:27:22.648054+0000","last_became_peered":"2024-09-13T18:27:22.648054+0000","last_unstale":"2024-09-13T18:57:00.409347+0000","last_undegraded":"2024-09-13T18:57:00.409347+0000","last_fullsized":"2024-09-13T18:57:00.409347+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5235,"log_dups_size":0,"ondisk_log_size":5235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:11.855024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348247684,"num_objects":91,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":23534,"num_write_kb":471297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"29'14","reported_seq":59,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.025486+0000","last_change":"2024-09-13T18:26:32.343915+0000","last_active":"2024-09-13T18:55:26.025486+0000","last_peered":"2024-09-13T18:55:26.025486+0000","last_clean":"2024-09-13T18:55:26.025486+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:55:26.025486+0000","last_undegraded":"2024-09-13T18:55:26.025486+0000","last_fullsized":"2024-09-13T18:55:26.025486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:19.357243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"28'5","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.162612+0000","last_change":"2024-09-13T18:26:31.979944+0000","last_active":"2024-09-13T18:55:53.162612+0000","last_peered":"2024-09-13T18:55:53.162612+0000","last_clean":"2024-09-13T18:55:53.162612+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:55:53.162612+0000","last_undegraded":"2024-09-13T18:55:53.162612+0000","last_fullsized":"2024-09-13T18:55:53.162612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:21.721921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"43'5232","reported_seq":5303,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.336494+0000","last_change":"2024-09-13T18:26:36.410028+0000","last_active":"2024-09-13T18:57:00.336494+0000","last_peered":"2024-09-13T18:57:00.336494+0000","last_clean":"2024-09-13T18:57:00.336494+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:57:00.336494+0000","last_undegraded":"2024-09-13T18:57:00.336494+0000","last_fullsized":"2024-09-13T18:57:00.336494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5232,"log_dups_size":0,"ondisk_log_size":5232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:32.264341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352362513,"num_objects":88,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":24364,"num_write_kb":507343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"29'9","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.010706+0000","last_change":"2024-09-13T18:26:32.358980+0000","last_active":"2024-09-13T18:55:26.010706+0000","last_peered":"2024-09-13T18:55:26.010706+0000","last_clean":"2024-09-13T18:55:26.010706+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:55:26.010706+0000","last_undegraded":"2024-09-13T18:55:26.010706+0000","last_fullsized":"2024-09-13T18:55:26.010706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:08.748139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"43'283","reported_seq":345,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:52.834136+0000","last_change":"2024-09-13T18:26:31.968972+0000","last_active":"2024-09-13T18:56:52.834136+0000","last_peered":"2024-09-13T18:56:52.834136+0000","last_clean":"2024-09-13T18:56:52.834136+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:56:52.834136+0000","last_undegraded":"2024-09-13T18:56:52.834136+0000","last_fullsized":"2024-09-13T18:56:52.834136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:56.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":52,"num_write":622,"num_write_kb":212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"43'5922","reported_seq":6003,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.069738+0000","last_change":"2024-09-13T18:27:22.645275+0000","last_active":"2024-09-13T18:57:00.069738+0000","last_peered":"2024-09-13T18:57:00.069738+0000","last_clean":"2024-09-13T18:57:00.069738+0000","last_became_active":"2024-09-13T18:27:22.644681+0000","last_became_peered":"2024-09-13T18:27:22.644681+0000","last_unstale":"2024-09-13T18:57:00.069738+0000","last_undegraded":"2024-09-13T18:57:00.069738+0000","last_fullsized":"2024-09-13T18:57:00.069738+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5922,"log_dups_size":0,"ondisk_log_size":5922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:18:38.581700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394336124,"num_objects":101,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":27801,"num_write_kb":553947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"43'6","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:50.152562+0000","last_change":"2024-09-13T18:26:32.342823+0000","last_active":"2024-09-13T18:56:50.152562+0000","last_peered":"2024-09-13T18:56:50.152562+0000","last_clean":"2024-09-13T18:56:50.152562+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:56:50.152562+0000","last_undegraded":"2024-09-13T18:56:50.152562+0000","last_fullsized":"2024-09-13T18:56:50.152562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:28.051252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:30.981249+0000","last_change":"2024-09-13T18:26:31.978976+0000","last_active":"2024-09-13T18:55:30.981249+0000","last_peered":"2024-09-13T18:55:30.981249+0000","last_clean":"2024-09-13T18:55:30.981249+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:55:30.981249+0000","last_undegraded":"2024-09-13T18:55:30.981249+0000","last_fullsized":"2024-09-13T18:55:30.981249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:11.279557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"43'4621","reported_seq":4684,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.841241+0000","last_change":"2024-09-13T18:26:36.408844+0000","last_active":"2024-09-13T18:56:58.841241+0000","last_peered":"2024-09-13T18:56:58.841241+0000","last_clean":"2024-09-13T18:56:58.841241+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:56:58.841241+0000","last_undegraded":"2024-09-13T18:56:58.841241+0000","last_fullsized":"2024-09-13T18:56:58.841241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4621,"log_dups_size":0,"ondisk_log_size":4621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:06.969342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306187993,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":22360,"num_write_kb":450115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"28'7","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.969209+0000","last_change":"2024-09-13T18:26:32.355038+0000","last_active":"2024-09-13T18:55:25.969209+0000","last_peered":"2024-09-13T18:55:25.969209+0000","last_clean":"2024-09-13T18:55:25.969209+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:55:25.969209+0000","last_undegraded":"2024-09-13T18:55:25.969209+0000","last_fullsized":"2024-09-13T18:55:25.969209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.159231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.007762+0000","last_change":"2024-09-13T18:26:31.972931+0000","last_active":"2024-09-13T18:55:26.007762+0000","last_peered":"2024-09-13T18:55:26.007762+0000","last_clean":"2024-09-13T18:55:26.007762+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:55:26.007762+0000","last_undegraded":"2024-09-13T18:55:26.007762+0000","last_fullsized":"2024-09-13T18:55:26.007762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:05.646924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"43'6011","reported_seq":6080,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.360889+0000","last_change":"2024-09-13T18:26:36.398876+0000","last_active":"2024-09-13T18:57:01.360889+0000","last_peered":"2024-09-13T18:57:01.360889+0000","last_clean":"2024-09-13T18:57:01.360889+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:57:01.360889+0000","last_undegraded":"2024-09-13T18:57:01.360889+0000","last_fullsized":"2024-09-13T18:57:01.360889+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6011,"log_dups_size":0,"ondisk_log_size":6011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:56.918239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369124519,"num_objects":90,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":27977,"num_write_kb":562911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"28'6","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.982753+0000","last_change":"2024-09-13T18:26:32.353111+0000","last_active":"2024-09-13T18:55:25.982753+0000","last_peered":"2024-09-13T18:55:25.982753+0000","last_clean":"2024-09-13T18:55:25.982753+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:55:25.982753+0000","last_undegraded":"2024-09-13T18:55:25.982753+0000","last_fullsized":"2024-09-13T18:55:25.982753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:43.927936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"41'6","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.977227+0000","last_change":"2024-09-13T18:26:31.976075+0000","last_active":"2024-09-13T18:55:25.977227+0000","last_peered":"2024-09-13T18:55:25.977227+0000","last_clean":"2024-09-13T18:55:25.977227+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:55:25.977227+0000","last_undegraded":"2024-09-13T18:55:25.977227+0000","last_fullsized":"2024-09-13T18:55:25.977227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:03.693518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"43'5256","reported_seq":5328,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.125917+0000","last_change":"2024-09-13T18:26:36.401777+0000","last_active":"2024-09-13T18:57:01.125917+0000","last_peered":"2024-09-13T18:57:01.125917+0000","last_clean":"2024-09-13T18:57:01.125917+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:57:01.125917+0000","last_undegraded":"2024-09-13T18:57:01.125917+0000","last_fullsized":"2024-09-13T18:57:01.125917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5256,"log_dups_size":0,"ondisk_log_size":5256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:43.440284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343954983,"num_objects":87,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":24458,"num_write_kb":507684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"29'10","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978450+0000","last_change":"2024-09-13T18:26:32.343791+0000","last_active":"2024-09-13T18:55:25.978450+0000","last_peered":"2024-09-13T18:55:25.978450+0000","last_clean":"2024-09-13T18:55:25.978450+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:55:25.978450+0000","last_undegraded":"2024-09-13T18:55:25.978450+0000","last_fullsized":"2024-09-13T18:55:25.978450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:36.296571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"43'128","reported_seq":302,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.464900+0000","last_change":"2024-09-13T18:26:31.970786+0000","last_active":"2024-09-13T18:56:58.464900+0000","last_peered":"2024-09-13T18:56:58.464900+0000","last_clean":"2024-09-13T18:56:58.464900+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:56:58.464900+0000","last_undegraded":"2024-09-13T18:56:58.464900+0000","last_fullsized":"2024-09-13T18:56:58.464900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:54.584009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93931,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":203,"num_write":119,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"43'6142","reported_seq":6212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.392774+0000","last_change":"2024-09-13T18:26:36.401529+0000","last_active":"2024-09-13T18:57:01.392774+0000","last_peered":"2024-09-13T18:57:01.392774+0000","last_clean":"2024-09-13T18:57:01.392774+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:57:01.392774+0000","last_undegraded":"2024-09-13T18:57:01.392774+0000","last_fullsized":"2024-09-13T18:57:01.392774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6142,"log_dups_size":0,"ondisk_log_size":6142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:47.318347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377531496,"num_objects":99,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":28204,"num_write_kb":563741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"28'4","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.987962+0000","last_change":"2024-09-13T18:26:32.350470+0000","last_active":"2024-09-13T18:55:25.987962+0000","last_peered":"2024-09-13T18:55:25.987962+0000","last_clean":"2024-09-13T18:55:25.987962+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:55:25.987962+0000","last_undegraded":"2024-09-13T18:55:25.987962+0000","last_fullsized":"2024-09-13T18:55:25.987962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:55.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"43'14","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:31.126491+0000","last_change":"2024-09-13T18:26:31.972587+0000","last_active":"2024-09-13T18:55:31.126491+0000","last_peered":"2024-09-13T18:55:31.126491+0000","last_clean":"2024-09-13T18:55:31.126491+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:55:31.126491+0000","last_undegraded":"2024-09-13T18:55:31.126491+0000","last_fullsized":"2024-09-13T18:55:31.126491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:27.547763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"43'4295","reported_seq":4365,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.861128+0000","last_change":"2024-09-13T18:26:36.405086+0000","last_active":"2024-09-13T18:56:58.861128+0000","last_peered":"2024-09-13T18:56:58.861128+0000","last_clean":"2024-09-13T18:56:58.861128+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:56:58.861128+0000","last_undegraded":"2024-09-13T18:56:58.861128+0000","last_fullsized":"2024-09-13T18:56:58.861128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4295,"log_dups_size":0,"ondisk_log_size":4295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:02:51.694634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335811241,"num_objects":85,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":19777,"num_write_kb":435588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"29'4","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.994918+0000","last_change":"2024-09-13T18:26:32.350510+0000","last_active":"2024-09-13T18:55:25.994918+0000","last_peered":"2024-09-13T18:55:25.994918+0000","last_clean":"2024-09-13T18:55:25.994918+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:55:25.994918+0000","last_undegraded":"2024-09-13T18:55:25.994918+0000","last_fullsized":"2024-09-13T18:55:25.994918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:40.850255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"38'6","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:45.169259+0000","last_change":"2024-09-13T18:26:31.978355+0000","last_active":"2024-09-13T18:55:45.169259+0000","last_peered":"2024-09-13T18:55:45.169259+0000","last_clean":"2024-09-13T18:55:45.169259+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:55:45.169259+0000","last_undegraded":"2024-09-13T18:55:45.169259+0000","last_fullsized":"2024-09-13T18:55:45.169259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:12.946518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":40,"num_read_kb":68,"num_write":8,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983663+0000","last_change":"2024-09-13T18:26:26.931403+0000","last_active":"2024-09-13T18:55:25.983663+0000","last_peered":"2024-09-13T18:55:25.983663+0000","last_clean":"2024-09-13T18:55:25.983663+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:55:25.983663+0000","last_undegraded":"2024-09-13T18:55:25.983663+0000","last_fullsized":"2024-09-13T18:55:25.983663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_clean_scrub_stamp":"2024-09-13T18:26:24.880028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:30.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"43'5845","reported_seq":5924,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.292017+0000","last_change":"2024-09-13T18:26:36.409827+0000","last_active":"2024-09-13T18:57:00.292017+0000","last_peered":"2024-09-13T18:57:00.292017+0000","last_clean":"2024-09-13T18:57:00.292017+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:57:00.292017+0000","last_undegraded":"2024-09-13T18:57:00.292017+0000","last_fullsized":"2024-09-13T18:57:00.292017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5845,"log_dups_size":0,"ondisk_log_size":5845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:50.482400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377529592,"num_objects":94,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":27327,"num_write_kb":519330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"28'6","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984035+0000","last_change":"2024-09-13T18:26:32.356850+0000","last_active":"2024-09-13T18:55:25.984035+0000","last_peered":"2024-09-13T18:55:25.984035+0000","last_clean":"2024-09-13T18:55:25.984035+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:55:25.984035+0000","last_undegraded":"2024-09-13T18:55:25.984035+0000","last_fullsized":"2024-09-13T18:55:25.984035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:38.710828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"36'4","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.973775+0000","last_change":"2024-09-13T18:26:31.975942+0000","last_active":"2024-09-13T18:55:25.973775+0000","last_peered":"2024-09-13T18:55:25.973775+0000","last_clean":"2024-09-13T18:55:25.973775+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:55:25.973775+0000","last_undegraded":"2024-09-13T18:55:25.973775+0000","last_fullsized":"2024-09-13T18:55:25.973775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:05.840582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":35,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"43'5261","reported_seq":5324,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.262169+0000","last_change":"2024-09-13T18:26:36.409802+0000","last_active":"2024-09-13T18:57:01.262169+0000","last_peered":"2024-09-13T18:57:01.262169+0000","last_clean":"2024-09-13T18:57:01.262169+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:57:01.262169+0000","last_undegraded":"2024-09-13T18:57:01.262169+0000","last_fullsized":"2024-09-13T18:57:01.262169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5261,"log_dups_size":0,"ondisk_log_size":5261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:38.178151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348128716,"num_objects":85,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":23982,"num_write_kb":513904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"29'8","reported_seq":47,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.973293+0000","last_change":"2024-09-13T18:27:22.610607+0000","last_active":"2024-09-13T18:55:25.973293+0000","last_peered":"2024-09-13T18:55:25.973293+0000","last_clean":"2024-09-13T18:55:25.973293+0000","last_became_active":"2024-09-13T18:27:22.610326+0000","last_became_peered":"2024-09-13T18:27:22.610326+0000","last_unstale":"2024-09-13T18:55:25.973293+0000","last_undegraded":"2024-09-13T18:55:25.973293+0000","last_fullsized":"2024-09-13T18:55:25.973293+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:38:04.117608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"38'2","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978777+0000","last_change":"2024-09-13T18:26:31.966618+0000","last_active":"2024-09-13T18:55:25.978777+0000","last_peered":"2024-09-13T18:55:25.978777+0000","last_clean":"2024-09-13T18:55:25.978777+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:55:25.978777+0000","last_undegraded":"2024-09-13T18:55:25.978777+0000","last_fullsized":"2024-09-13T18:55:25.978777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:54:26.422611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"43'4977","reported_seq":5044,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.799194+0000","last_change":"2024-09-13T18:26:36.394344+0000","last_active":"2024-09-13T18:56:58.799194+0000","last_peered":"2024-09-13T18:56:58.799194+0000","last_clean":"2024-09-13T18:56:58.799194+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:56:58.799194+0000","last_undegraded":"2024-09-13T18:56:58.799194+0000","last_fullsized":"2024-09-13T18:56:58.799194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4977,"log_dups_size":0,"ondisk_log_size":4977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:18.425816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360783295,"num_objects":94,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":22736,"num_write_kb":472029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"29'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984452+0000","last_change":"2024-09-13T18:26:32.365655+0000","last_active":"2024-09-13T18:55:25.984452+0000","last_peered":"2024-09-13T18:55:25.984452+0000","last_clean":"2024-09-13T18:55:25.984452+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:55:25.984452+0000","last_undegraded":"2024-09-13T18:55:25.984452+0000","last_fullsized":"2024-09-13T18:55:25.984452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:40.288128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'4","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.973070+0000","last_change":"2024-09-13T18:26:31.970652+0000","last_active":"2024-09-13T18:55:25.973070+0000","last_peered":"2024-09-13T18:55:25.973070+0000","last_clean":"2024-09-13T18:55:25.973070+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:55:25.973070+0000","last_undegraded":"2024-09-13T18:55:25.973070+0000","last_fullsized":"2024-09-13T18:55:25.973070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:11.169661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"43'4832","reported_seq":4893,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.526006+0000","last_change":"2024-09-13T18:26:36.410280+0000","last_active":"2024-09-13T18:56:59.526006+0000","last_peered":"2024-09-13T18:56:59.526006+0000","last_clean":"2024-09-13T18:56:59.526006+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:56:59.526006+0000","last_undegraded":"2024-09-13T18:56:59.526006+0000","last_fullsized":"2024-09-13T18:56:59.526006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4832,"log_dups_size":0,"ondisk_log_size":4832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:01.395290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327189351,"num_objects":82,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":17,"num_read_kb":0,"num_write":22693,"num_write_kb":463670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"28'3","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984106+0000","last_change":"2024-09-13T18:26:32.361575+0000","last_active":"2024-09-13T18:55:25.984106+0000","last_peered":"2024-09-13T18:55:25.984106+0000","last_clean":"2024-09-13T18:55:25.984106+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:55:25.984106+0000","last_undegraded":"2024-09-13T18:55:25.984106+0000","last_fullsized":"2024-09-13T18:55:25.984106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:08.818877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"41'9","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983121+0000","last_change":"2024-09-13T18:26:31.977029+0000","last_active":"2024-09-13T18:55:25.983121+0000","last_peered":"2024-09-13T18:55:25.983121+0000","last_clean":"2024-09-13T18:55:25.983121+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:55:25.983121+0000","last_undegraded":"2024-09-13T18:55:25.983121+0000","last_fullsized":"2024-09-13T18:55:25.983121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:13.701689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"43'4972","reported_seq":5037,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.287176+0000","last_change":"2024-09-13T18:26:36.403479+0000","last_active":"2024-09-13T18:57:00.287176+0000","last_peered":"2024-09-13T18:57:00.287176+0000","last_clean":"2024-09-13T18:57:00.287176+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:57:00.287176+0000","last_undegraded":"2024-09-13T18:57:00.287176+0000","last_fullsized":"2024-09-13T18:57:00.287176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:51.095852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327213309,"num_objects":84,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":22631,"num_write_kb":463382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"29'9","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984952+0000","last_change":"2024-09-13T18:26:32.365143+0000","last_active":"2024-09-13T18:55:25.984952+0000","last_peered":"2024-09-13T18:55:25.984952+0000","last_clean":"2024-09-13T18:55:25.984952+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:55:25.984952+0000","last_undegraded":"2024-09-13T18:55:25.984952+0000","last_fullsized":"2024-09-13T18:55:25.984952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:46.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"41'21","reported_seq":126,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.152848+0000","last_change":"2024-09-13T18:26:31.965802+0000","last_active":"2024-09-13T18:55:53.152848+0000","last_peered":"2024-09-13T18:55:53.152848+0000","last_clean":"2024-09-13T18:55:53.152848+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:55:53.152848+0000","last_undegraded":"2024-09-13T18:55:53.152848+0000","last_fullsized":"2024-09-13T18:55:53.152848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:12.759181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":158,"num_read_kb":157,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"43'5873","reported_seq":5946,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.072818+0000","last_change":"2024-09-13T18:26:36.399235+0000","last_active":"2024-09-13T18:56:59.072818+0000","last_peered":"2024-09-13T18:56:59.072818+0000","last_clean":"2024-09-13T18:56:59.072818+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:56:59.072818+0000","last_undegraded":"2024-09-13T18:56:59.072818+0000","last_fullsized":"2024-09-13T18:56:59.072818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5873,"log_dups_size":0,"ondisk_log_size":5873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:13:54.711083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381775740,"num_objects":100,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":27640,"num_write_kb":536907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"43'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:56.844127+0000","last_change":"2024-09-13T18:26:32.357403+0000","last_active":"2024-09-13T18:56:56.844127+0000","last_peered":"2024-09-13T18:56:56.844127+0000","last_clean":"2024-09-13T18:56:56.844127+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:56:56.844127+0000","last_undegraded":"2024-09-13T18:56:56.844127+0000","last_fullsized":"2024-09-13T18:56:56.844127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:12.600397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"41'12","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.982231+0000","last_change":"2024-09-13T18:26:31.976148+0000","last_active":"2024-09-13T18:55:25.982231+0000","last_peered":"2024-09-13T18:55:25.982231+0000","last_clean":"2024-09-13T18:55:25.982231+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:55:25.982231+0000","last_undegraded":"2024-09-13T18:55:25.982231+0000","last_fullsized":"2024-09-13T18:55:25.982231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:38.023636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1620,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"43'5978","reported_seq":6044,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.401591+0000","last_change":"2024-09-13T18:26:36.406177+0000","last_active":"2024-09-13T18:57:00.401591+0000","last_peered":"2024-09-13T18:57:00.401591+0000","last_clean":"2024-09-13T18:57:00.401591+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:57:00.401591+0000","last_undegraded":"2024-09-13T18:57:00.401591+0000","last_fullsized":"2024-09-13T18:57:00.401591+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5978,"log_dups_size":0,"ondisk_log_size":5978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:00.309995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369133062,"num_objects":92,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":28176,"num_write_kb":551294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"43'12","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978424+0000","last_change":"2024-09-13T18:27:22.609309+0000","last_active":"2024-09-13T18:55:25.978424+0000","last_peered":"2024-09-13T18:55:25.978424+0000","last_clean":"2024-09-13T18:55:25.978424+0000","last_became_active":"2024-09-13T18:27:22.609101+0000","last_became_peered":"2024-09-13T18:27:22.609101+0000","last_unstale":"2024-09-13T18:55:25.978424+0000","last_undegraded":"2024-09-13T18:55:25.978424+0000","last_fullsized":"2024-09-13T18:55:25.978424+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:15.692612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"43'10","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.165844+0000","last_change":"2024-09-13T18:26:31.982114+0000","last_active":"2024-09-13T18:55:53.165844+0000","last_peered":"2024-09-13T18:55:53.165844+0000","last_clean":"2024-09-13T18:55:53.165844+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:55:53.165844+0000","last_undegraded":"2024-09-13T18:55:53.165844+0000","last_fullsized":"2024-09-13T18:55:53.165844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.266528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"43'5767","reported_seq":5836,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.376830+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:57:00.376830+0000","last_peered":"2024-09-13T18:57:00.376830+0000","last_clean":"2024-09-13T18:57:00.376830+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:57:00.376830+0000","last_undegraded":"2024-09-13T18:57:00.376830+0000","last_fullsized":"2024-09-13T18:57:00.376830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5767,"log_dups_size":0,"ondisk_log_size":5767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:58.853971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377595894,"num_objects":98,"num_object_clones":0,"num_object_copies":392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27027,"num_write_kb":550847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"28'8","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984791+0000","last_change":"2024-09-13T18:26:32.351145+0000","last_active":"2024-09-13T18:55:25.984791+0000","last_peered":"2024-09-13T18:55:25.984791+0000","last_clean":"2024-09-13T18:55:25.984791+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:55:25.984791+0000","last_undegraded":"2024-09-13T18:55:25.984791+0000","last_fullsized":"2024-09-13T18:55:25.984791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:57.002435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"38'9","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.158612+0000","last_change":"2024-09-13T18:26:31.961202+0000","last_active":"2024-09-13T18:55:53.158612+0000","last_peered":"2024-09-13T18:55:53.158612+0000","last_clean":"2024-09-13T18:55:53.158612+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:55:53.158612+0000","last_undegraded":"2024-09-13T18:55:53.158612+0000","last_fullsized":"2024-09-13T18:55:53.158612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:27.340127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":93,"num_read_kb":112,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"43'5137","reported_seq":5202,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.582316+0000","last_change":"2024-09-13T18:26:36.410014+0000","last_active":"2024-09-13T18:56:59.582316+0000","last_peered":"2024-09-13T18:56:59.582316+0000","last_clean":"2024-09-13T18:56:59.582316+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:56:59.582316+0000","last_undegraded":"2024-09-13T18:56:59.582316+0000","last_fullsized":"2024-09-13T18:56:59.582316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5137,"log_dups_size":0,"ondisk_log_size":5137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:24.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767127,"num_objects":77,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":23163,"num_write_kb":490059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"29'9","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983909+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:55:25.983909+0000","last_peered":"2024-09-13T18:55:25.983909+0000","last_clean":"2024-09-13T18:55:25.983909+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:55:25.983909+0000","last_undegraded":"2024-09-13T18:55:25.983909+0000","last_fullsized":"2024-09-13T18:55:25.983909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:40.102815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"43'381","reported_seq":468,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:47.848809+0000","last_change":"2024-09-13T18:26:31.974110+0000","last_active":"2024-09-13T18:56:47.848809+0000","last_peered":"2024-09-13T18:56:47.848809+0000","last_clean":"2024-09-13T18:56:47.848809+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:56:47.848809+0000","last_undegraded":"2024-09-13T18:56:47.848809+0000","last_fullsized":"2024-09-13T18:56:47.848809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:56.058561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":93,"num_read_kb":20568,"num_write":381,"num_write_kb":4317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"43'5306","reported_seq":5372,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.367559+0000","last_change":"2024-09-13T18:26:36.394080+0000","last_active":"2024-09-13T18:57:00.367559+0000","last_peered":"2024-09-13T18:57:00.367559+0000","last_clean":"2024-09-13T18:57:00.367559+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:57:00.367559+0000","last_undegraded":"2024-09-13T18:57:00.367559+0000","last_fullsized":"2024-09-13T18:57:00.367559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5306,"log_dups_size":0,"ondisk_log_size":5306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:47.121814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335673691,"num_objects":86,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":24948,"num_write_kb":485267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"43'4","reported_seq":47,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.971630+0000","last_change":"2024-09-13T18:26:32.362873+0000","last_active":"2024-09-13T18:55:25.971630+0000","last_peered":"2024-09-13T18:55:25.971630+0000","last_clean":"2024-09-13T18:55:25.971630+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:55:25.971630+0000","last_undegraded":"2024-09-13T18:55:25.971630+0000","last_fullsized":"2024-09-13T18:55:25.971630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:37.598189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"43'61","reported_seq":123,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:46.092289+0000","last_change":"2024-09-13T18:26:31.976401+0000","last_active":"2024-09-13T18:56:46.092289+0000","last_peered":"2024-09-13T18:56:46.092289+0000","last_clean":"2024-09-13T18:56:46.092289+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:56:46.092289+0000","last_undegraded":"2024-09-13T18:56:46.092289+0000","last_fullsized":"2024-09-13T18:56:46.092289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:26.434292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":62,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"43'5783","reported_seq":5853,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.356403+0000","last_change":"2024-09-13T18:26:36.406228+0000","last_active":"2024-09-13T18:57:00.356403+0000","last_peered":"2024-09-13T18:57:00.356403+0000","last_clean":"2024-09-13T18:57:00.356403+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:57:00.356403+0000","last_undegraded":"2024-09-13T18:57:00.356403+0000","last_fullsized":"2024-09-13T18:57:00.356403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5783,"log_dups_size":0,"ondisk_log_size":5783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:49.198397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352479214,"num_objects":93,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27380,"num_write_kb":527755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"29'9","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.987447+0000","last_change":"2024-09-13T18:26:32.350841+0000","last_active":"2024-09-13T18:55:25.987447+0000","last_peered":"2024-09-13T18:55:25.987447+0000","last_clean":"2024-09-13T18:55:25.987447+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:55:25.987447+0000","last_undegraded":"2024-09-13T18:55:25.987447+0000","last_fullsized":"2024-09-13T18:55:25.987447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:05.137882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'3","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:52.993936+0000","last_change":"2024-09-13T18:26:31.978214+0000","last_active":"2024-09-13T18:55:52.993936+0000","last_peered":"2024-09-13T18:55:52.993936+0000","last_clean":"2024-09-13T18:55:52.993936+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:55:52.993936+0000","last_undegraded":"2024-09-13T18:55:52.993936+0000","last_fullsized":"2024-09-13T18:55:52.993936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:17.381375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"43'5336","reported_seq":5397,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.340297+0000","last_change":"2024-09-13T18:26:36.406090+0000","last_active":"2024-09-13T18:57:00.340297+0000","last_peered":"2024-09-13T18:57:00.340297+0000","last_clean":"2024-09-13T18:57:00.340297+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:57:00.340297+0000","last_undegraded":"2024-09-13T18:57:00.340297+0000","last_fullsized":"2024-09-13T18:57:00.340297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:09.865147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327181663,"num_objects":85,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":24960,"num_write_kb":489215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"29'5","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.969146+0000","last_change":"2024-09-13T18:26:32.357701+0000","last_active":"2024-09-13T18:55:25.969146+0000","last_peered":"2024-09-13T18:55:25.969146+0000","last_clean":"2024-09-13T18:55:25.969146+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:55:25.969146+0000","last_undegraded":"2024-09-13T18:55:25.969146+0000","last_fullsized":"2024-09-13T18:55:25.969146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:47.587683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"43'15","reported_seq":81,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:46.038602+0000","last_change":"2024-09-13T18:26:31.980133+0000","last_active":"2024-09-13T18:56:46.038602+0000","last_peered":"2024-09-13T18:56:46.038602+0000","last_clean":"2024-09-13T18:56:46.038602+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:56:46.038602+0000","last_undegraded":"2024-09-13T18:56:46.038602+0000","last_fullsized":"2024-09-13T18:56:46.038602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:05.354335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":16,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"43'4996","reported_seq":5064,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.435081+0000","last_change":"2024-09-13T18:26:36.406553+0000","last_active":"2024-09-13T18:57:00.435081+0000","last_peered":"2024-09-13T18:57:00.435081+0000","last_clean":"2024-09-13T18:57:00.435081+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:57:00.435081+0000","last_undegraded":"2024-09-13T18:57:00.435081+0000","last_fullsized":"2024-09-13T18:57:00.435081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4996,"log_dups_size":0,"ondisk_log_size":4996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:02.980826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":365112283,"num_objects":94,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":23246,"num_write_kb":507027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"43'5990","reported_seq":6060,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.955468+0000","last_change":"2024-09-13T18:26:36.406986+0000","last_active":"2024-09-13T18:56:58.955468+0000","last_peered":"2024-09-13T18:56:58.955468+0000","last_clean":"2024-09-13T18:56:58.955468+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:56:58.955468+0000","last_undegraded":"2024-09-13T18:56:58.955468+0000","last_fullsized":"2024-09-13T18:56:58.955468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5990,"log_dups_size":0,"ondisk_log_size":5990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:53.984231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419441008,"num_objects":103,"num_object_clones":0,"num_object_copies":412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":28460,"num_write_kb":581141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"29'4","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978161+0000","last_change":"2024-09-13T18:26:32.349081+0000","last_active":"2024-09-13T18:55:25.978161+0000","last_peered":"2024-09-13T18:55:25.978161+0000","last_clean":"2024-09-13T18:55:25.978161+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:55:25.978161+0000","last_undegraded":"2024-09-13T18:55:25.978161+0000","last_fullsized":"2024-09-13T18:55:25.978161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:47.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"43'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:31.137406+0000","last_change":"2024-09-13T18:26:31.968944+0000","last_active":"2024-09-13T18:55:31.137406+0000","last_peered":"2024-09-13T18:55:31.137406+0000","last_clean":"2024-09-13T18:55:31.137406+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:55:31.137406+0000","last_undegraded":"2024-09-13T18:55:31.137406+0000","last_fullsized":"2024-09-13T18:55:31.137406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:29.363312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"43'5472","reported_seq":5544,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.453436+0000","last_change":"2024-09-13T18:26:36.396418+0000","last_active":"2024-09-13T18:57:00.453436+0000","last_peered":"2024-09-13T18:57:00.453436+0000","last_clean":"2024-09-13T18:57:00.453436+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:57:00.453436+0000","last_undegraded":"2024-09-13T18:57:00.453436+0000","last_fullsized":"2024-09-13T18:57:00.453436+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5472,"log_dups_size":0,"ondisk_log_size":5472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:03.252773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":340075764,"num_objects":93,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":25514,"num_write_kb":520259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"29'14","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.111521+0000","last_change":"2024-09-13T18:26:32.351369+0000","last_active":"2024-09-13T18:55:26.111521+0000","last_peered":"2024-09-13T18:55:26.111521+0000","last_clean":"2024-09-13T18:55:26.111521+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:55:26.111521+0000","last_undegraded":"2024-09-13T18:55:26.111521+0000","last_fullsized":"2024-09-13T18:55:26.111521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:10.892347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"36'4","reported_seq":59,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.001369+0000","last_change":"2024-09-13T18:26:31.972048+0000","last_active":"2024-09-13T18:55:26.001369+0000","last_peered":"2024-09-13T18:55:26.001369+0000","last_clean":"2024-09-13T18:55:26.001369+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:55:26.001369+0000","last_undegraded":"2024-09-13T18:55:26.001369+0000","last_fullsized":"2024-09-13T18:55:26.001369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:28.430119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"41'9","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981371+0000","last_change":"2024-09-13T18:26:31.975310+0000","last_active":"2024-09-13T18:55:25.981371+0000","last_peered":"2024-09-13T18:55:25.981371+0000","last_clean":"2024-09-13T18:55:25.981371+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:55:25.981371+0000","last_undegraded":"2024-09-13T18:55:25.981371+0000","last_fullsized":"2024-09-13T18:55:25.981371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:52.915310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"29'7","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.999839+0000","last_change":"2024-09-13T18:26:32.345825+0000","last_active":"2024-09-13T18:55:25.999839+0000","last_peered":"2024-09-13T18:55:25.999839+0000","last_clean":"2024-09-13T18:55:25.999839+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:55:25.999839+0000","last_undegraded":"2024-09-13T18:55:25.999839+0000","last_fullsized":"2024-09-13T18:55:25.999839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:25.200489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"43'4065","reported_seq":4127,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.144665+0000","last_change":"2024-09-13T18:26:36.391520+0000","last_active":"2024-09-13T18:57:01.144665+0000","last_peered":"2024-09-13T18:57:01.144665+0000","last_clean":"2024-09-13T18:57:01.144665+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:57:01.144665+0000","last_undegraded":"2024-09-13T18:57:01.144665+0000","last_fullsized":"2024-09-13T18:57:01.144665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4065,"log_dups_size":0,"ondisk_log_size":4065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:42.638713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255876134,"num_objects":63,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":19156,"num_write_kb":424944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"43'754","reported_seq":826,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:35.286998+0000","last_change":"2024-09-13T18:26:31.976603+0000","last_active":"2024-09-13T18:56:35.286998+0000","last_peered":"2024-09-13T18:56:35.286998+0000","last_clean":"2024-09-13T18:56:35.286998+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:56:35.286998+0000","last_undegraded":"2024-09-13T18:56:35.286998+0000","last_fullsized":"2024-09-13T18:56:35.286998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:03.965376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196273,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":19627,"num_write":735,"num_write_kb":4345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"29'9","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.979121+0000","last_change":"2024-09-13T18:26:32.352982+0000","last_active":"2024-09-13T18:55:25.979121+0000","last_peered":"2024-09-13T18:55:25.979121+0000","last_clean":"2024-09-13T18:55:25.979121+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:55:25.979121+0000","last_undegraded":"2024-09-13T18:55:25.979121+0000","last_fullsized":"2024-09-13T18:55:25.979121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:55.670360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"43'5828","reported_seq":5896,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:00.118020+0000","last_change":"2024-09-13T18:26:36.403817+0000","last_active":"2024-09-13T18:57:00.118020+0000","last_peered":"2024-09-13T18:57:00.118020+0000","last_clean":"2024-09-13T18:57:00.118020+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:57:00.118020+0000","last_undegraded":"2024-09-13T18:57:00.118020+0000","last_fullsized":"2024-09-13T18:57:00.118020+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5828,"log_dups_size":0,"ondisk_log_size":5828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:34.892901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377551906,"num_objects":98,"num_object_clones":0,"num_object_copies":392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":26701,"num_write_kb":557751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"43'8","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:47.227020+0000","last_change":"2024-09-13T18:26:32.357856+0000","last_active":"2024-09-13T18:55:47.227020+0000","last_peered":"2024-09-13T18:55:47.227020+0000","last_clean":"2024-09-13T18:55:47.227020+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:55:47.227020+0000","last_undegraded":"2024-09-13T18:55:47.227020+0000","last_fullsized":"2024-09-13T18:55:47.227020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:26.316214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"29'10","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.972396+0000","last_change":"2024-09-13T18:26:32.350966+0000","last_active":"2024-09-13T18:55:25.972396+0000","last_peered":"2024-09-13T18:55:25.972396+0000","last_clean":"2024-09-13T18:55:25.972396+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:55:25.972396+0000","last_undegraded":"2024-09-13T18:55:25.972396+0000","last_fullsized":"2024-09-13T18:55:25.972396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:52.632112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"29'11","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.977018+0000","last_change":"2024-09-13T18:26:32.355806+0000","last_active":"2024-09-13T18:55:25.977018+0000","last_peered":"2024-09-13T18:55:25.977018+0000","last_clean":"2024-09-13T18:55:25.977018+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:55:25.977018+0000","last_undegraded":"2024-09-13T18:55:25.977018+0000","last_fullsized":"2024-09-13T18:55:25.977018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.988231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"29'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.999728+0000","last_change":"2024-09-13T18:26:32.340965+0000","last_active":"2024-09-13T18:55:25.999728+0000","last_peered":"2024-09-13T18:55:25.999728+0000","last_clean":"2024-09-13T18:55:25.999728+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:55:25.999728+0000","last_undegraded":"2024-09-13T18:55:25.999728+0000","last_fullsized":"2024-09-13T18:55:25.999728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:01:04.716919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"29'7","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.046037+0000","last_change":"2024-09-13T18:26:32.349625+0000","last_active":"2024-09-13T18:55:26.046037+0000","last_peered":"2024-09-13T18:55:26.046037+0000","last_clean":"2024-09-13T18:55:26.046037+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:55:26.046037+0000","last_undegraded":"2024-09-13T18:55:26.046037+0000","last_fullsized":"2024-09-13T18:55:26.046037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:19.519754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"28'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978502+0000","last_change":"2024-09-13T18:26:32.349170+0000","last_active":"2024-09-13T18:55:25.978502+0000","last_peered":"2024-09-13T18:55:25.978502+0000","last_clean":"2024-09-13T18:55:25.978502+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:55:25.978502+0000","last_undegraded":"2024-09-13T18:55:25.978502+0000","last_fullsized":"2024-09-13T18:55:25.978502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:43.942732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"29'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981727+0000","last_change":"2024-09-13T18:26:32.364786+0000","last_active":"2024-09-13T18:55:25.981727+0000","last_peered":"2024-09-13T18:55:25.981727+0000","last_clean":"2024-09-13T18:55:25.981727+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:55:25.981727+0000","last_undegraded":"2024-09-13T18:55:25.981727+0000","last_fullsized":"2024-09-13T18:55:25.981727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:22.949875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"28'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.010595+0000","last_change":"2024-09-13T18:26:32.347541+0000","last_active":"2024-09-13T18:55:26.010595+0000","last_peered":"2024-09-13T18:55:26.010595+0000","last_clean":"2024-09-13T18:55:26.010595+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:55:26.010595+0000","last_undegraded":"2024-09-13T18:55:26.010595+0000","last_fullsized":"2024-09-13T18:55:26.010595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:58.144128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"29'5","reported_seq":50,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T18:54:50.138427+0000","last_change":"2024-09-13T18:26:32.353359+0000","last_active":"2024-09-13T18:54:50.138427+0000","last_peered":"2024-09-13T18:54:50.138427+0000","last_clean":"2024-09-13T18:54:50.138427+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:54:50.138427+0000","last_undegraded":"2024-09-13T18:54:50.138427+0000","last_fullsized":"2024-09-13T18:54:50.138427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:09.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"29'10","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.001188+0000","last_change":"2024-09-13T18:26:32.349566+0000","last_active":"2024-09-13T18:55:26.001188+0000","last_peered":"2024-09-13T18:55:26.001188+0000","last_clean":"2024-09-13T18:55:26.001188+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:55:26.001188+0000","last_undegraded":"2024-09-13T18:55:26.001188+0000","last_fullsized":"2024-09-13T18:55:26.001188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:22.341899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"43'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:50.192401+0000","last_change":"2024-09-13T18:26:32.358640+0000","last_active":"2024-09-13T18:55:50.192401+0000","last_peered":"2024-09-13T18:55:50.192401+0000","last_clean":"2024-09-13T18:55:50.192401+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:55:50.192401+0000","last_undegraded":"2024-09-13T18:55:50.192401+0000","last_fullsized":"2024-09-13T18:55:50.192401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:40.174754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"43'11","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:50.939549+0000","last_change":"2024-09-13T18:26:32.353675+0000","last_active":"2024-09-13T18:55:50.939549+0000","last_peered":"2024-09-13T18:55:50.939549+0000","last_clean":"2024-09-13T18:55:50.939549+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:55:50.939549+0000","last_undegraded":"2024-09-13T18:55:50.939549+0000","last_fullsized":"2024-09-13T18:55:50.939549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:58.501843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"28'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981840+0000","last_change":"2024-09-13T18:26:32.353223+0000","last_active":"2024-09-13T18:55:25.981840+0000","last_peered":"2024-09-13T18:55:25.981840+0000","last_clean":"2024-09-13T18:55:25.981840+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:55:25.981840+0000","last_undegraded":"2024-09-13T18:55:25.981840+0000","last_fullsized":"2024-09-13T18:55:25.981840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:45.898213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"43'10","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:51.407384+0000","last_change":"2024-09-13T18:26:32.357354+0000","last_active":"2024-09-13T18:55:51.407384+0000","last_peered":"2024-09-13T18:55:51.407384+0000","last_clean":"2024-09-13T18:55:51.407384+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:55:51.407384+0000","last_undegraded":"2024-09-13T18:55:51.407384+0000","last_fullsized":"2024-09-13T18:55:51.407384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:19.185008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"43'11","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:56.238386+0000","last_change":"2024-09-13T18:26:32.348832+0000","last_active":"2024-09-13T18:56:56.238386+0000","last_peered":"2024-09-13T18:56:56.238386+0000","last_clean":"2024-09-13T18:56:56.238386+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:56:56.238386+0000","last_undegraded":"2024-09-13T18:56:56.238386+0000","last_fullsized":"2024-09-13T18:56:56.238386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:11:59.874789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"28'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.972098+0000","last_change":"2024-09-13T18:26:32.356902+0000","last_active":"2024-09-13T18:55:25.972098+0000","last_peered":"2024-09-13T18:55:25.972098+0000","last_clean":"2024-09-13T18:55:25.972098+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:55:25.972098+0000","last_undegraded":"2024-09-13T18:55:25.972098+0000","last_fullsized":"2024-09-13T18:55:25.972098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:18.362772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"28'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.968811+0000","last_change":"2024-09-13T18:26:32.347047+0000","last_active":"2024-09-13T18:55:25.968811+0000","last_peered":"2024-09-13T18:55:25.968811+0000","last_clean":"2024-09-13T18:55:25.968811+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:55:25.968811+0000","last_undegraded":"2024-09-13T18:55:25.968811+0000","last_fullsized":"2024-09-13T18:55:25.968811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:40.882789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"43'10","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:52.028230+0000","last_change":"2024-09-13T18:26:32.349651+0000","last_active":"2024-09-13T18:56:52.028230+0000","last_peered":"2024-09-13T18:56:52.028230+0000","last_clean":"2024-09-13T18:56:52.028230+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:56:52.028230+0000","last_undegraded":"2024-09-13T18:56:52.028230+0000","last_fullsized":"2024-09-13T18:56:52.028230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:33.356563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"29'10","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T18:54:40.392762+0000","last_change":"2024-09-13T18:26:32.344607+0000","last_active":"2024-09-13T18:54:40.392762+0000","last_peered":"2024-09-13T18:54:40.392762+0000","last_clean":"2024-09-13T18:54:40.392762+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:54:40.392762+0000","last_undegraded":"2024-09-13T18:54:40.392762+0000","last_fullsized":"2024-09-13T18:54:40.392762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:31.658172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"29'6","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.000730+0000","last_change":"2024-09-13T18:26:32.347717+0000","last_active":"2024-09-13T18:55:26.000730+0000","last_peered":"2024-09-13T18:55:26.000730+0000","last_clean":"2024-09-13T18:55:26.000730+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:55:26.000730+0000","last_undegraded":"2024-09-13T18:55:26.000730+0000","last_fullsized":"2024-09-13T18:55:26.000730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:47.187837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"29'6","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.971507+0000","last_change":"2024-09-13T18:26:32.359506+0000","last_active":"2024-09-13T18:55:25.971507+0000","last_peered":"2024-09-13T18:55:25.971507+0000","last_clean":"2024-09-13T18:55:25.971507+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:55:25.971507+0000","last_undegraded":"2024-09-13T18:55:25.971507+0000","last_fullsized":"2024-09-13T18:55:25.971507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:39:57.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"43'12","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:37.221615+0000","last_change":"2024-09-13T18:26:32.357729+0000","last_active":"2024-09-13T18:55:37.221615+0000","last_peered":"2024-09-13T18:55:37.221615+0000","last_clean":"2024-09-13T18:55:37.221615+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:55:37.221615+0000","last_undegraded":"2024-09-13T18:55:37.221615+0000","last_fullsized":"2024-09-13T18:55:37.221615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:05.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"29'9","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981655+0000","last_change":"2024-09-13T18:26:32.357980+0000","last_active":"2024-09-13T18:55:25.981655+0000","last_peered":"2024-09-13T18:55:25.981655+0000","last_clean":"2024-09-13T18:55:25.981655+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:55:25.981655+0000","last_undegraded":"2024-09-13T18:55:25.981655+0000","last_fullsized":"2024-09-13T18:55:25.981655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:00.858702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"43'13","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.154866+0000","last_change":"2024-09-13T18:26:32.365638+0000","last_active":"2024-09-13T18:56:59.154866+0000","last_peered":"2024-09-13T18:56:59.154866+0000","last_clean":"2024-09-13T18:56:59.154866+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:56:59.154866+0000","last_undegraded":"2024-09-13T18:56:59.154866+0000","last_fullsized":"2024-09-13T18:56:59.154866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.065853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"29'3","reported_seq":47,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.109132+0000","last_change":"2024-09-13T18:26:32.345496+0000","last_active":"2024-09-13T18:55:26.109132+0000","last_peered":"2024-09-13T18:55:26.109132+0000","last_clean":"2024-09-13T18:55:26.109132+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:55:26.109132+0000","last_undegraded":"2024-09-13T18:55:26.109132+0000","last_fullsized":"2024-09-13T18:55:26.109132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:22.447315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"43'4","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:35.970044+0000","last_change":"2024-09-13T18:26:32.356901+0000","last_active":"2024-09-13T18:56:35.970044+0000","last_peered":"2024-09-13T18:56:35.970044+0000","last_clean":"2024-09-13T18:56:35.970044+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:56:35.970044+0000","last_undegraded":"2024-09-13T18:56:35.970044+0000","last_fullsized":"2024-09-13T18:56:35.970044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:03.311603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"28'8","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.976794+0000","last_change":"2024-09-13T18:26:32.355203+0000","last_active":"2024-09-13T18:55:25.976794+0000","last_peered":"2024-09-13T18:55:25.976794+0000","last_clean":"2024-09-13T18:55:25.976794+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:55:25.976794+0000","last_undegraded":"2024-09-13T18:55:25.976794+0000","last_fullsized":"2024-09-13T18:55:25.976794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:52.769708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"29'6","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983681+0000","last_change":"2024-09-13T18:26:32.358190+0000","last_active":"2024-09-13T18:55:25.983681+0000","last_peered":"2024-09-13T18:55:25.983681+0000","last_clean":"2024-09-13T18:55:25.983681+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:55:25.983681+0000","last_undegraded":"2024-09-13T18:55:25.983681+0000","last_fullsized":"2024-09-13T18:55:25.983681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:35.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"28'4","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.968851+0000","last_change":"2024-09-13T18:26:32.355121+0000","last_active":"2024-09-13T18:55:25.968851+0000","last_peered":"2024-09-13T18:55:25.968851+0000","last_clean":"2024-09-13T18:55:25.968851+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:55:25.968851+0000","last_undegraded":"2024-09-13T18:55:25.968851+0000","last_fullsized":"2024-09-13T18:55:25.968851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:48.427332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"29'6","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.987776+0000","last_change":"2024-09-13T18:26:32.341497+0000","last_active":"2024-09-13T18:55:25.987776+0000","last_peered":"2024-09-13T18:55:25.987776+0000","last_clean":"2024-09-13T18:55:25.987776+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:55:25.987776+0000","last_undegraded":"2024-09-13T18:55:25.987776+0000","last_fullsized":"2024-09-13T18:55:25.987776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:15.510048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"28'2","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978877+0000","last_change":"2024-09-13T18:26:32.350106+0000","last_active":"2024-09-13T18:55:25.978877+0000","last_peered":"2024-09-13T18:55:25.978877+0000","last_clean":"2024-09-13T18:55:25.978877+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:55:25.978877+0000","last_undegraded":"2024-09-13T18:55:25.978877+0000","last_fullsized":"2024-09-13T18:55:25.978877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:12.387451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"29'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984165+0000","last_change":"2024-09-13T18:26:32.362196+0000","last_active":"2024-09-13T18:55:25.984165+0000","last_peered":"2024-09-13T18:55:25.984165+0000","last_clean":"2024-09-13T18:55:25.984165+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:55:25.984165+0000","last_undegraded":"2024-09-13T18:55:25.984165+0000","last_fullsized":"2024-09-13T18:55:25.984165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:14.946904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":11298053892,"num_objects":2881,"num_object_clones":0,"num_object_copies":11524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2881,"num_whiteouts":0,"num_read":797,"num_read_kb":0,"num_write":800719,"num_write_kb":16319459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9769914368,"data_stored":22543564800,"data_compressed":1072758487,"data_compressed_allocated":1504456704,"data_compressed_original":14278107136,"omap_allocated":0,"internal_metadata":0},"log_size":172420,"ondisk_log_size":172420,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":472,"ondisk_log_size":472,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":16496904,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":1362,"num_read_kb":52080,"num_write":3724,"num_write_kb":17653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22700032,"data_stored":32993808,"data_compressed":2441304,"data_compressed_allocated":3506176,"data_compressed_original":13950976,"omap_allocated":0,"internal_metadata":0},"log_size":3487,"ondisk_log_size":3487,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509810,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2270664,"kb_used_data":1249600,"kb_used_omap":1,"kb_used_meta":1021054,"kb_avail":92101176,"statfs":{"total":96636764160,"available":94311604224,"internally_reserved":0,"allocated":1279590400,"data_stored":2930967260,"data_compressed":139752701,"data_compressed_allocated":196038656,"data_compressed_original":1847668736,"omap_allocated":1593,"internal_metadata":1045559751},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":1.3979999999999999,"15min":0.96599999999999997},"min":{"1min":0.224,"5min":0.223,"15min":0.223},"max":{"1min":9.0050000000000008,"5min":10.43,"15min":10.43},"last":0.25600000000000001},{"interface":"front","average":{"1min":2.5230000000000001,"5min":1.8320000000000001,"15min":1.141},"min":{"1min":0.25900000000000001,"5min":0.245,"15min":0.245},"max":{"1min":9.9190000000000005,"5min":16.600999999999999,"15min":16.600999999999999},"last":0.27200000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.9630000000000001,"5min":3.831,"15min":1.825},"min":{"1min":0.26000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":7.7409999999999997,"5min":52.290999999999997,"15min":52.290999999999997},"last":0.31},{"interface":"front","average":{"1min":1.6719999999999999,"5min":4.0910000000000002,"15min":1.9239999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":7.8410000000000002,"5min":52.149000000000001,"15min":52.149000000000001},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.518,"15min":1.0580000000000001},"min":{"1min":0.33500000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.8360000000000003,"5min":14.933,"15min":14.933},"last":1.419},{"interface":"front","average":{"1min":1.7509999999999999,"5min":1.4239999999999999,"15min":1.048},"min":{"1min":0.29099999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":6.7160000000000002,"5min":17.792000000000002,"15min":17.792000000000002},"last":0.43099999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":2.4580000000000002,"15min":1.3899999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":8.1809999999999992,"5min":15.622,"15min":15.622},"last":2.6400000000000001},{"interface":"front","average":{"1min":3.2429999999999999,"5min":3.2829999999999999,"15min":1.6799999999999999},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":14.993,"5min":28.248000000000001,"15min":28.248000000000001},"last":1.8979999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.304,"15min":1.0820000000000001},"min":{"1min":0.31,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.3570000000000002,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.232,"15min":1.048},"min":{"1min":0.38700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":4.4569999999999999,"5min":7.9409999999999998,"15min":7.9409999999999998},"last":0.48699999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.95699999999999996,"15min":0.97699999999999998},"min":{"1min":0.376,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.7410000000000001,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.099,"5min":0.96999999999999997,"15min":1.004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.2480000000000002,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":0.34599999999999997}]},{"osd":7,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.0429999999999999,"15min":1.0720000000000001},"min":{"1min":0.39300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.8319999999999999,"5min":12.021000000000001,"15min":12.021000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.91300000000000003,"15min":0.98599999999999999},"min":{"1min":0.32500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":3.9430000000000001,"5min":9.0440000000000005,"15min":9.0440000000000005},"last":0.40200000000000002}]}]},{"osd":7,"up_from":15,"seq":64424509809,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1929960,"kb_used_data":1066208,"kb_used_omap":1,"kb_used_meta":863742,"kb_avail":92441880,"statfs":{"total":96636764160,"available":94660485120,"internally_reserved":0,"allocated":1091796992,"data_stored":2515818119,"data_compressed":119837066,"data_compressed_allocated":168169472,"data_compressed_original":1592455168,"omap_allocated":1589,"internal_metadata":884472267},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":2.121,"15min":1.206},"min":{"1min":0.23000000000000001,"5min":0.217,"15min":0.217},"max":{"1min":14.778,"5min":24.414000000000001,"15min":24.414000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":2.1680000000000001,"5min":2.254,"15min":1.3460000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":14.814,"5min":24.469000000000001,"15min":24.469000000000001},"last":0.79000000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":2.4889999999999999,"5min":3.355,"15min":1.617},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":6.0490000000000004,"5min":46.161999999999999,"15min":46.161999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":5.1929999999999996,"5min":4.3570000000000002,"15min":2.0129999999999999},"min":{"1min":0.29099999999999998,"5min":0.252,"15min":0.252},"max":{"1min":39.853999999999999,"5min":45.713000000000001,"15min":45.713000000000001},"last":0.95799999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.228,"15min":0.98699999999999999},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":5.3230000000000004,"5min":7.4740000000000002,"15min":7.4740000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.329,"15min":0.97899999999999998},"min":{"1min":0.315,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.8559999999999999,"5min":9.1679999999999993,"15min":9.1679999999999993},"last":1.1000000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":3.4129999999999998,"5min":3.6960000000000002,"15min":1.8029999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":17.721,"5min":35.494999999999997,"15min":35.494999999999997},"last":1.052},{"interface":"front","average":{"1min":2.278,"5min":2.976,"15min":1.6850000000000001},"min":{"1min":0.33800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":11.353999999999999,"5min":19.946000000000002,"15min":19.946000000000002},"last":1.0089999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.92600000000000005,"15min":0.91700000000000004},"min":{"1min":0.25800000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.2749999999999999,"5min":7.2880000000000003,"15min":7.2880000000000003},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.0820000000000001,"15min":1.0589999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":7.8810000000000002,"5min":7.8810000000000002,"15min":7.8810000000000002},"last":1.1950000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.80700000000000005,"15min":0.877},"min":{"1min":0.29399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":3.6200000000000001,"5min":6.7309999999999999,"15min":6.7309999999999999},"last":0.753},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90100000000000002,"15min":0.96999999999999997},"min":{"1min":0.32000000000000001,"5min":0.318,"15min":0.318},"max":{"1min":3.5819999999999999,"5min":6.6699999999999999,"15min":6.6699999999999999},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:56:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.2889999999999999,"15min":1.137},"min":{"1min":0.35299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.4260000000000002,"5min":32.491,"15min":32.491},"last":1.143},{"interface":"front","average":{"1min":0.54100000000000004,"5min":1.137,"15min":1.091},"min":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.82899999999999996,"5min":27.291,"15min":27.291},"last":0.90400000000000003}]}]},{"osd":4,"up_from":15,"seq":64424509809,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2176860,"kb_used_data":1236180,"kb_used_omap":1,"kb_used_meta":940670,"kb_avail":92194980,"statfs":{"total":96636764160,"available":94407659520,"internally_reserved":0,"allocated":1265848320,"data_stored":2899198048,"data_compressed":137665483,"data_compressed_allocated":193257472,"data_compressed_original":1826861056,"omap_allocated":1594,"internal_metadata":963246534},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":2.2599999999999998,"5min":2.1259999999999999,"15min":1.214},"min":{"1min":0.26300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":8.8740000000000006,"5min":28.593,"15min":28.593},"last":0.441},{"interface":"front","average":{"1min":1.9850000000000001,"5min":1.837,"15min":1.1930000000000001},"min":{"1min":0.26400000000000001,"5min":0.255,"15min":0.255},"max":{"1min":10.711,"5min":28.559000000000001,"15min":28.559000000000001},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":2.6299999999999999,"5min":2.907,"15min":1.49},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":10.044,"5min":37.100999999999999,"15min":37.100999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":2.0089999999999999,"5min":3.4969999999999999,"15min":1.6910000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":8.3810000000000002,"5min":44.798000000000002,"15min":44.798000000000002},"last":0.28799999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":2.6789999999999998,"5min":1.845,"15min":1.1859999999999999},"min":{"1min":0.30199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":16.617000000000001,"5min":27.634,"15min":27.634},"last":0.42399999999999999},{"interface":"front","average":{"1min":2.8279999999999998,"5min":1.7170000000000001,"15min":1.2050000000000001},"min":{"1min":0.317,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":16.396999999999998,"5min":27.606000000000002,"15min":27.606000000000002},"last":0.34200000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":3.496,"5min":3.339,"15min":1.6499999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":18.503,"5min":40.676000000000002,"15min":40.676000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":5.9610000000000003,"5min":3.8050000000000002,"15min":1.887},"min":{"1min":0.47999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":39.649000000000001,"5min":40.881999999999998,"15min":40.881999999999998},"last":0.36699999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:56:26 2024","interfaces":[{"interface":"back","average":{"1min":1.7769999999999999,"5min":0.876,"15min":0.88300000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":8.5649999999999995,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":0.91600000000000004,"15min":0.98499999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.0810000000000004,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.167,"15min":1.0900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":5.984,"5min":9.4570000000000007,"15min":9.4570000000000007},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.1559999999999999,"5min":0.996,"15min":1.0109999999999999},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":5.1059999999999999,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.38600000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8779999999999999,"5min":1.4279999999999999,"15min":1.177},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":10.101000000000001,"5min":10.101000000000001,"15min":10.101000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.9239999999999999,"5min":1.492,"15min":1.274},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":10.127000000000001,"5min":10.127000000000001,"15min":10.127000000000001},"last":0.50600000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542513,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2824836,"kb_used_data":1633712,"kb_used_omap":1,"kb_used_meta":1190142,"kb_avail":91547004,"statfs":{"total":96636764160,"available":93744132096,"internally_reserved":0,"allocated":1672921088,"data_stored":3841409738,"data_compressed":183202141,"data_compressed_allocated":257212416,"data_compressed_original":2425958400,"omap_allocated":1591,"internal_metadata":1218705865},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:32 2024","interfaces":[{"interface":"back","average":{"1min":3.0569999999999999,"5min":3.3959999999999999,"15min":1.5109999999999999},"min":{"1min":0.20300000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":22.670000000000002,"5min":22.670000000000002,"15min":22.670000000000002},"last":0.29499999999999998},{"interface":"front","average":{"1min":3.1179999999999999,"5min":3.5009999999999999,"15min":1.677},"min":{"1min":0.246,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":22.728000000000002,"5min":22.728000000000002,"15min":22.728000000000002},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":7.9240000000000004,"5min":6.282,"15min":2.4940000000000002},"min":{"1min":0.38500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":42.046999999999997,"5min":42.046999999999997,"15min":42.046999999999997},"last":0.316},{"interface":"front","average":{"1min":7.6500000000000004,"5min":6.1680000000000001,"15min":2.5630000000000002},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":42.024999999999999,"5min":42.024999999999999,"15min":42.024999999999999},"last":0.503}]},{"osd":2,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":6.4550000000000001,"5min":4.2300000000000004,"15min":1.9259999999999999},"min":{"1min":0.27700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":41.909999999999997,"5min":41.909999999999997,"15min":41.909999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":5.1609999999999996,"5min":3.1819999999999999,"15min":1.5229999999999999},"min":{"1min":0.317,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":27.25,"5min":27.25,"15min":27.25},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:56:32 2024","interfaces":[{"interface":"back","average":{"1min":4.6550000000000002,"5min":3.3100000000000001,"15min":1.7430000000000001},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":42.003,"5min":42.003,"15min":42.003},"last":0.73299999999999998},{"interface":"front","average":{"1min":5.4329999999999998,"5min":3.9399999999999999,"15min":1.956},"min":{"1min":0.39300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":41.951000000000001,"5min":41.951000000000001,"15min":41.951000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":4.79,"5min":2.46,"15min":1.508},"min":{"1min":0.38100000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":22.988,"5min":22.988,"15min":22.988},"last":0.68700000000000006},{"interface":"front","average":{"1min":3.2829999999999999,"5min":2.2799999999999998,"15min":1.462},"min":{"1min":0.35199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":23.030000000000001,"5min":23.030000000000001,"15min":23.030000000000001},"last":0.80300000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":4.0529999999999999,"5min":2.9689999999999999,"15min":1.6719999999999999},"min":{"1min":0.35799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":23.087,"5min":23.087,"15min":23.087},"last":0.64200000000000002},{"interface":"front","average":{"1min":5.4210000000000003,"5min":3.4540000000000002,"15min":1.843},"min":{"1min":0.52700000000000002,"5min":0.317,"15min":0.317},"max":{"1min":23.425000000000001,"5min":23.425000000000001,"15min":23.425000000000001},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":4.9379999999999997,"5min":3.1600000000000001,"15min":1.714},"min":{"1min":0.437,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":41.975999999999999,"5min":41.975999999999999,"15min":41.975999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":4.9550000000000001,"5min":2.956,"15min":1.6619999999999999},"min":{"1min":0.42699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":23.457999999999998,"5min":23.457999999999998,"15min":23.457999999999998},"last":0.75600000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542517,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1698048,"kb_used_data":966456,"kb_used_omap":1,"kb_used_meta":731582,"kb_avail":92673792,"statfs":{"total":96636764160,"available":94897963008,"internally_reserved":0,"allocated":989650944,"data_stored":2283014993,"data_compressed":108755460,"data_compressed_allocated":152350720,"data_compressed_original":1445974016,"omap_allocated":1593,"internal_metadata":749140423},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":2.5489999999999999,"15min":1.2270000000000001},"min":{"1min":0.17799999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":5.8200000000000003,"5min":37.600000000000001,"15min":37.600000000000001},"last":2.6019999999999999},{"interface":"front","average":{"1min":1.056,"5min":2.4569999999999999,"15min":1.266},"min":{"1min":0.218,"5min":0.19,"15min":0.19},"max":{"1min":6.0839999999999996,"5min":37.651000000000003,"15min":37.651000000000003},"last":2.6800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":2.5800000000000001,"5min":3.0449999999999999,"15min":1.5209999999999999},"min":{"1min":0.26500000000000001,"5min":0.255,"15min":0.255},"max":{"1min":12.244,"5min":18.055,"15min":18.055},"last":9.1189999999999998},{"interface":"front","average":{"1min":2.851,"5min":2.8900000000000001,"15min":1.431},"min":{"1min":0.28899999999999998,"5min":0.188,"15min":0.188},"max":{"1min":11.916,"5min":18.175999999999998,"15min":18.175999999999998},"last":10.196}]},{"osd":3,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":2.6179999999999999,"5min":3.5470000000000002,"15min":1.7010000000000001},"min":{"1min":0.313,"5min":0.249,"15min":0.249},"max":{"1min":12.888999999999999,"5min":25.832999999999998,"15min":25.832999999999998},"last":3.5350000000000001},{"interface":"front","average":{"1min":2.621,"5min":3.5230000000000001,"15min":1.77},"min":{"1min":0.375,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":12.702999999999999,"5min":17.515000000000001,"15min":17.515000000000001},"last":3.508}]},{"osd":4,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.2190000000000001,"15min":0.996},"min":{"1min":0.27700000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.5590000000000002,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":3.036},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.242,"15min":1.0820000000000001},"min":{"1min":0.33700000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":5.0899999999999999,"5min":8.0999999999999996,"15min":8.0999999999999996},"last":1.7949999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.27,"15min":1.032},"min":{"1min":0.29799999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":8.4800000000000004,"5min":8.4800000000000004,"15min":8.4800000000000004},"last":2.6360000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.4319999999999999,"15min":1.1220000000000001},"min":{"1min":0.36499999999999999,"5min":0.309,"15min":0.309},"max":{"1min":3.8079999999999998,"5min":8.907,"15min":8.907},"last":2.3460000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":1.321,"15min":1.133},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.032,"5min":9.6720000000000006,"15min":9.6720000000000006},"last":1.875},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.2070000000000001,"15min":1.069},"min":{"1min":0.35799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":5.3289999999999997,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":2.375}]},{"osd":7,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3560000000000001,"15min":1.129},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":4.7939999999999996,"5min":8.9250000000000007,"15min":8.9250000000000007},"last":2.661},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.585,"15min":1.236},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":6.335,"5min":9.2330000000000005,"15min":9.2330000000000005},"last":1.835}]}]},{"osd":5,"up_from":13,"seq":55834575218,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1570116,"kb_used_data":872956,"kb_used_omap":1,"kb_used_meta":697150,"kb_avail":92801724,"statfs":{"total":96636764160,"available":95028965376,"internally_reserved":0,"allocated":893906944,"data_stored":2074980080,"data_compressed":98665143,"data_compressed_allocated":138342400,"data_compressed_original":1319673856,"omap_allocated":1593,"internal_metadata":713882055},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":1.8879999999999999,"5min":1.8919999999999999,"15min":1.109},"min":{"1min":0.23799999999999999,"5min":0.224,"15min":0.224},"max":{"1min":8.2240000000000002,"5min":9.952,"15min":9.952},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.95,"5min":2.0139999999999998,"15min":1.2350000000000001},"min":{"1min":0.29899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.2579999999999991,"5min":14.317,"15min":14.317},"last":0.86799999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":1.9550000000000001,"5min":2.29,"15min":1.3129999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":10.058,"5min":10.85,"15min":10.85},"last":0.66700000000000004},{"interface":"front","average":{"1min":2.1629999999999998,"5min":2.2719999999999998,"15min":1.3140000000000001},"min":{"1min":0.29299999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":10.462999999999999,"5min":10.868,"15min":10.868},"last":0.82299999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":3.8519999999999999,"5min":1.915,"15min":1.1759999999999999},"min":{"1min":0.27300000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":38.484000000000002,"5min":38.484000000000002,"15min":38.484000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":4.0890000000000004,"5min":2.2309999999999999,"15min":1.286},"min":{"1min":0.27100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":40.686,"5min":40.686,"15min":40.686},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":3.9580000000000002,"5min":4.2320000000000002,"15min":1.9750000000000001},"min":{"1min":0.38100000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":19.093,"5min":72.534999999999997,"15min":72.534999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":3.5720000000000001,"5min":4.0199999999999996,"15min":1.9710000000000001},"min":{"1min":0.43099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":19.071000000000002,"5min":72.486999999999995,"15min":72.486999999999995},"last":0.75600000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":1.02,"15min":0.91900000000000004},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.2290000000000001,"5min":7.8710000000000004,"15min":7.8710000000000004},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.131,"15min":1.0760000000000001},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":8.7829999999999995,"5min":8.7829999999999995,"15min":8.7829999999999995},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":0.96399999999999997,"15min":0.997},"min":{"1min":0.28699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.04,"15min":1.0489999999999999},"min":{"1min":0.35599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.516,"5min":4.516,"15min":4.516},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.02,"15min":1.0209999999999999},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":4.4249999999999998,"5min":7.6020000000000003,"15min":7.6020000000000003},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.0169999999999999,"15min":1.101},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.1900000000000004,"5min":7.9109999999999996,"15min":7.9109999999999996},"last":1.9810000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575216,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2673912,"kb_used_data":1523088,"kb_used_omap":1,"kb_used_meta":1148350,"kb_avail":91697928,"statfs":{"total":96636764160,"available":93898678272,"internally_reserved":0,"allocated":1559642112,"data_stored":3627326782,"data_compressed":172575409,"data_compressed_allocated":241844224,"data_compressed_original":2309791744,"omap_allocated":1594,"internal_metadata":1175910854},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":3.2570000000000001,"15min":1.4910000000000001},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":7.9580000000000002,"5min":28.297000000000001,"15min":28.297000000000001},"last":10.279},{"interface":"front","average":{"1min":3.1040000000000001,"5min":2.7749999999999999,"15min":1.341},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":19.785,"5min":19.785,"15min":19.785},"last":10.162000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":2.8340000000000001,"15min":1.385},"min":{"1min":0.252,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":8.0630000000000006,"5min":25.609999999999999,"15min":25.609999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":4.0890000000000004,"5min":3.097,"15min":1.536},"min":{"1min":0.38500000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":20.173999999999999,"5min":30.469999999999999,"15min":30.469999999999999},"last":10.465}]},{"osd":3,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":5.524,"5min":5.7249999999999996,"15min":2.383},"min":{"1min":0.58199999999999996,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":20.001000000000001,"5min":36.643999999999998,"15min":36.643999999999998},"last":10.406000000000001},{"interface":"front","average":{"1min":5.5519999999999996,"5min":5.9119999999999999,"15min":2.5110000000000001},"min":{"1min":0.59899999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":20.030999999999999,"5min":36.670000000000002,"15min":36.670000000000002},"last":10.429}]},{"osd":4,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":1.7549999999999999,"15min":1.1759999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.2249999999999996,"5min":7.6799999999999997,"15min":7.6799999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.26,"5min":2.1299999999999999,"15min":1.331},"min":{"1min":0.32800000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":6.5540000000000003,"5min":14.906000000000001,"15min":14.906000000000001},"last":0.47999999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":3.1880000000000002,"5min":1.9350000000000001,"15min":1.3149999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":9.0519999999999996,"5min":9.0519999999999996,"15min":9.0519999999999996},"last":10.259},{"interface":"front","average":{"1min":2.7149999999999999,"5min":1.9410000000000001,"15min":1.3300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.1050000000000004,"5min":12.27,"15min":12.27},"last":10.231999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":2.577,"5min":2.4300000000000002,"15min":1.468},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":9.077,"5min":28.981999999999999,"15min":28.981999999999999},"last":10.538},{"interface":"front","average":{"1min":2.1819999999999999,"5min":2.1379999999999999,"15min":1.3799999999999999},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":15.727,"5min":15.727,"15min":15.727},"last":10.342000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":2.379,"15min":1.4359999999999999},"min":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":7.9930000000000003,"5min":28.100000000000001,"15min":28.100000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":1.728,"5min":2.4860000000000002,"15min":1.518},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.0220000000000002,"5min":28.896999999999998,"15min":28.896999999999998},"last":0.41099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575217,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1942528,"kb_used_data":1150840,"kb_used_omap":1,"kb_used_meta":791678,"kb_avail":92429312,"statfs":{"total":96636764160,"available":94647615488,"internally_reserved":0,"allocated":1178460160,"data_stored":2729905844,"data_compressed":130141336,"data_compressed_allocated":182214656,"data_compressed_original":1733922816,"omap_allocated":1594,"internal_metadata":810678726},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":5.1710000000000003,"5min":4.0839999999999996,"15min":1.732},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":29.712,"5min":29.712,"15min":29.712},"last":0.34100000000000003},{"interface":"front","average":{"1min":5.4039999999999999,"5min":3.0979999999999999,"15min":1.4810000000000001},"min":{"1min":0.495,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":29.748000000000001,"5min":29.748000000000001,"15min":29.748000000000001},"last":0.36499999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":1.7849999999999999,"15min":1.014},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":17.611000000000001,"5min":17.611000000000001,"15min":17.611000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":2.847,"5min":1.9570000000000001,"15min":1.1180000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":17.501999999999999,"5min":17.501999999999999,"15min":17.501999999999999},"last":0.375}]},{"osd":3,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":2.9580000000000002,"5min":3.4689999999999999,"15min":1.615},"min":{"1min":0.26200000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":17.260000000000002,"5min":17.260000000000002,"15min":17.260000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":2.5459999999999998,"5min":3.0129999999999999,"15min":1.512},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":17.088000000000001,"5min":17.088000000000001,"15min":17.088000000000001},"last":0.439}]},{"osd":4,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":5.3200000000000003,"5min":2.3799999999999999,"15min":1.407},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":40.101999999999997,"5min":40.101999999999997,"15min":40.101999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":5.2830000000000004,"5min":2.089,"15min":1.2989999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":40.981000000000002,"5min":40.981000000000002,"15min":40.981000000000002},"last":0.69599999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":4.5419999999999998,"5min":1.857,"15min":1.252},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":40.701999999999998,"5min":40.701999999999998,"15min":40.701999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":4.8120000000000003,"5min":2.0230000000000001,"15min":1.3440000000000001},"min":{"1min":0.36899999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":40.130000000000003,"5min":40.130000000000003,"15min":40.130000000000003},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":5.7699999999999996,"5min":2.1499999999999999,"15min":1.3979999999999999},"min":{"1min":0.41399999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":51.719000000000001,"5min":51.719000000000001,"15min":51.719000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":5.8150000000000004,"5min":2.1989999999999998,"15min":1.448},"min":{"1min":0.49099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":51.75,"5min":51.75,"15min":51.75},"last":0.79300000000000004}]},{"osd":7,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":5.5179999999999998,"5min":2.3180000000000001,"15min":1.4770000000000001},"min":{"1min":0.36899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":58.810000000000002,"5min":58.810000000000002,"15min":58.810000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":4.609,"5min":2.125,"15min":1.4370000000000001},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":39.622,"5min":39.622,"15min":39.622},"last":0.74199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2064384,"data_stored":3814150,"data_compressed":409575,"data_compressed_allocated":573440,"data_compressed_original":2342912,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":94659,"data_compressed":1568,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5124096,"data_stored":8485180,"data_compressed":790472,"data_compressed_allocated":1138688,"data_compressed_original":4517888,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4136960,"data_stored":4196050,"data_compressed":20605,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1777664,"data_stored":4196686,"data_compressed":519875,"data_compressed_allocated":692224,"data_compressed_original":3125248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9441280,"data_stored":12204761,"data_compressed":699209,"data_compressed_allocated":1056768,"data_compressed_original":3842048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1138999296,"data_stored":2622500864,"data_compressed":124826741,"data_compressed_allocated":174882816,"data_compressed_original":1658384384,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1543139328,"data_stored":3583045632,"data_compressed":170484417,"data_compressed_allocated":238940160,"data_compressed_original":2278846464,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":973479936,"data_stored":2257514496,"data_compressed":107562294,"data_compressed_allocated":150663168,"data_compressed_original":1434697728,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1628086272,"data_stored":3742507008,"data_compressed":178128102,"data_compressed_allocated":250073088,"data_compressed_original":2364493824,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1234100224,"data_stored":2836041728,"data_compressed":134852831,"data_compressed_allocated":189272064,"data_compressed_original":1791213568,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":892964864,"data_stored":2072641536,"data_compressed":98566295,"data_compressed_allocated":138211328,"data_compressed_original":1317888000,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1277333504,"data_stored":2926530560,"data_compressed":139232826,"data_compressed_allocated":195346432,"data_compressed_original":1844543488,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1081810944,"data_stored":2502782976,"data_compressed":119104981,"data_compressed_allocated":167067648,"data_compressed_original":1588039680,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:57:05.594 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:57:05.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 Processor -- start 2024-09-13T18:57:05.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 -- start start 2024-09-13T18:57:05.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f56b40c6610 0x7f56b40cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:05.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b40c6a80 0x7f56b40cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:05.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b4153fa0 con 0x7f56b4153bc0 2024-09-13T18:57:05.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b40aa500 con 0x7f56b40c6610 2024-09-13T18:57:05.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b415a7e0 con 0x7f56b40c6a80 2024-09-13T18:57:05.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56ba847700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b40c6a80 0x7f56b40cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:05.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56ba046700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f56b4153bc0 0x7f56b40c8de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:40258/0 (socket says 172.21.15.146:40258) 2024-09-13T18:57:05.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56ba046700 1 -- 172.21.15.146:0/2308326328 learned_addr learned my addr 172.21.15.146:0/2308326328 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:57:05.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56b9845700 1 --2- 172.21.15.146:0/2308326328 >> v2:172.21.15.154:3300/0 conn(0x7f56b40c6610 0x7f56b40cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:05.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56ba847700 1 -- 172.21.15.146:0/2308326328 >> v2:172.21.15.154:3300/0 conn(0x7f56b40c6610 msgr2=0x7f56b40cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:05.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56ba847700 1 --2- 172.21.15.146:0/2308326328 >> v2:172.21.15.154:3300/0 conn(0x7f56b40c6610 0x7f56b40cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56ba847700 1 -- 172.21.15.146:0/2308326328 >> v1:172.21.15.154:6789/0 conn(0x7f56b4153bc0 legacy=0x7f56b40c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:05.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56ba847700 1 -- 172.21.15.146:0/2308326328 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b40aa220 con 0x7f56b40c6a80 2024-09-13T18:57:05.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56ba847700 1 --2- 172.21.15.146:0/2308326328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b40c6a80 0x7f56b40cb1a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009d90 tx=0x7f56ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69ad9f86e4d3e6ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:05.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56b9044700 1 -- 172.21.15.146:0/2308326328 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00e070 con 0x7f56b40c6a80 2024-09-13T18:57:05.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56b9044700 1 -- 172.21.15.146:0/2308326328 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac00bb20 con 0x7f56b40c6a80 2024-09-13T18:57:05.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56b9044700 1 -- 172.21.15.146:0/2308326328 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac012430 con 0x7f56b40c6a80 2024-09-13T18:57:05.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 -- 172.21.15.146:0/2308326328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b40c6a80 msgr2=0x7f56b40cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:05.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 --2- 172.21.15.146:0/2308326328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b40c6a80 0x7f56b40cb1a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009d90 tx=0x7f56ac00b330 comp rx=0 tx=0).stop 2024-09-13T18:57:05.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 -- 172.21.15.146:0/2308326328 shutdown_connections 2024-09-13T18:57:05.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 --2- 172.21.15.146:0/2308326328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b40c6a80 0x7f56b40cb1a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 --2- 172.21.15.146:0/2308326328 >> v2:172.21.15.154:3300/0 conn(0x7f56b40c6610 0x7f56b40cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 -- 172.21.15.146:0/2308326328 >> 172.21.15.146:0/2308326328 conn(0x7f56b4066090 msgr2=0x7f56b4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:05.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 -- 172.21.15.146:0/2308326328 shutdown_connections 2024-09-13T18:57:05.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.767+0000 7f56bb048700 1 -- 172.21.15.146:0/2308326328 wait complete. 2024-09-13T18:57:05.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.771+0000 7f56bb048700 1 Processor -- start 2024-09-13T18:57:05.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56bb048700 1 -- start start 2024-09-13T18:57:05.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56bb048700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56b40c6610 0x7f56b40c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:05.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56bb048700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b40c6a80 0x7f56b40c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:05.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56bb048700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b4153bc0 0x7f56b414cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:05.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56bb048700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40ad2f0 con 0x7f56b40c6610 2024-09-13T18:57:05.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56bb048700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40c2750 con 0x7f56b4153bc0 2024-09-13T18:57:05.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56bb048700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40c22b0 con 0x7f56b40c6a80 2024-09-13T18:57:05.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56ba046700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56b40c6610 0x7f56b40c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:05.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56ba046700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56b40c6610 0x7f56b40c3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57616/0 (socket says 172.21.15.146:57616) 2024-09-13T18:57:05.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56ba046700 1 -- 172.21.15.146:0/2718899026 learned_addr learned my addr 172.21.15.146:0/2718899026 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:05.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56ba046700 1 -- 172.21.15.146:0/2718899026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b40c6a80 msgr2=0x7f56b40c3f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:57:05.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56ba046700 1 --2- 172.21.15.146:0/2718899026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b40c6a80 0x7f56b40c3f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56ba046700 1 -- 172.21.15.146:0/2718899026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b4153bc0 msgr2=0x7f56b414cfc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:57:05.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56ba046700 1 --2- 172.21.15.146:0/2718899026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b4153bc0 0x7f56b414cfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:05.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56ba046700 1 -- 172.21.15.146:0/2718899026 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac009a70 con 0x7f56b40c6610 2024-09-13T18:57:05.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.775+0000 7f56ba046700 1 --2- 172.21.15.146:0/2718899026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56b40c6610 0x7f56b40c3a00 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f56a800c4d0 tx=0x7f56a800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:05.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.783+0000 7f56b2ffd700 1 -- 172.21.15.146:0/2718899026 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8005d90 con 0x7f56b40c6610 2024-09-13T18:57:05.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.787+0000 7f56b2ffd700 1 -- 172.21.15.146:0/2718899026 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a80070d0 con 0x7f56b40c6610 2024-09-13T18:57:05.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.791+0000 7f56bb048700 1 -- 172.21.15.146:0/2718899026 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b40c2980 con 0x7f56b40c6610 2024-09-13T18:57:05.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.791+0000 7f56bb048700 1 -- 172.21.15.146:0/2718899026 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b40c2f20 con 0x7f56b40c6610 2024-09-13T18:57:05.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.791+0000 7f56b2ffd700 1 -- 172.21.15.146:0/2718899026 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8007440 con 0x7f56b40c6610 2024-09-13T18:57:05.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.791+0000 7f56b2ffd700 1 -- 172.21.15.146:0/2718899026 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f56a8007620 con 0x7f56b40c6610 2024-09-13T18:57:05.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.791+0000 7f56b2ffd700 1 --2- 172.21.15.146:0/2718899026 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56a00668b0 0x7f56a0068d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:05.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.791+0000 7f56b2ffd700 1 -- 172.21.15.146:0/2718899026 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f56a8084010 con 0x7f56b40c6610 2024-09-13T18:57:05.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.791+0000 7f56b9845700 1 --2- 172.21.15.146:0/2718899026 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56a00668b0 0x7f56a0068d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:05.812 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.795+0000 7f56b9845700 1 --2- 172.21.15.146:0/2718899026 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56a00668b0 0x7f56a0068d70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f56b40c5920 tx=0x7f56a4009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:05.812 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.799+0000 7f56bb048700 1 -- 172.21.15.146:0/2718899026 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b40a9de0 con 0x7f56b40c6610 2024-09-13T18:57:05.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:05.819+0000 7f56b2ffd700 1 -- 172.21.15.146:0/2718899026 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f56a8017080 con 0x7f56b40c6610 2024-09-13T18:57:06.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.191+0000 7f56bb048700 1 -- 172.21.15.146:0/2718899026 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56b406f690 con 0x7f56b40c6610 2024-09-13T18:57:06.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.195+0000 7f56b2ffd700 1 -- 172.21.15.146:0/2718899026 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+20825 (secure 0 0 0) 0x7f56a8050a60 con 0x7f56b40c6610 2024-09-13T18:57:06.210 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:57:06.210 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":43,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:55:25.944150+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.13","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/2053734262":"2024-09-14T18:55:25.944088+0000","172.21.15.154:6839/3274220802":"2024-09-14T18:36:10.677826+0000","172.21.15.146:6835/294014284":"2024-09-14T18:50:50.876658+0000","172.21.15.154:6837/3274220802":"2024-09-14T18:36:10.677826+0000","172.21.15.146:6836/2169954140":"2024-09-14T18:53:50.924269+0000","172.21.15.146:6832/1498985282":"2024-09-14T18:34:25.647701+0000","172.21.15.146:6835/1642500214":"2024-09-14T18:30:30.598692+0000","172.21.15.146:6836/31041557":"2024-09-14T18:32:55.627334+0000","172.21.15.146:6832/3023779168":"2024-09-14T18:52:35.902095+0000","172.21.15.146:6834/1642500214":"2024-09-14T18:30:30.598692+0000","172.21.15.146:6836/3234109877":"2024-09-14T18:45:55.807881+0000","172.21.15.146:6837/2169954140":"2024-09-14T18:53:50.924269+0000","172.21.15.146:6833/1498985282":"2024-09-14T18:34:25.647701+0000","172.21.15.146:6837/31041557":"2024-09-14T18:32:55.627334+0000","172.21.15.154:6839/776638146":"2024-09-14T18:44:35.790651+0000","172.21.15.154:6839/3978828499":"2024-09-14T18:29:25.578693+0000","172.21.15.146:6832/2835746745":"2024-09-14T18:28:15.564530+0000","172.21.15.146:6834/1652676664":"2024-09-14T18:38:00.699180+0000","172.21.15.154:6837/776638146":"2024-09-14T18:44:35.790651+0000","172.21.15.154:6837/925899756":"2024-09-14T18:48:20.844616+0000","172.21.15.146:6833/2835746745":"2024-09-14T18:28:15.564530+0000","172.21.15.146:6835/1652676664":"2024-09-14T18:38:00.699180+0000","172.21.15.154:6839/2053734262":"2024-09-14T18:55:25.944088+0000","172.21.15.154:6839/925899756":"2024-09-14T18:48:20.844616+0000","172.21.15.154:6837/3978828499":"2024-09-14T18:29:25.578693+0000","172.21.15.146:6836/3926931671":"2024-09-14T18:40:45.735995+0000","172.21.15.146:6837/3926931671":"2024-09-14T18:40:45.735995+0000","172.21.15.146:6833/3901685195":"2024-09-14T18:43:15.768366+0000","172.21.15.146:6837/3234109877":"2024-09-14T18:45:55.807881+0000","172.21.15.146:6834/294014284":"2024-09-14T18:50:50.876658+0000","172.21.15.146:6832/3901685195":"2024-09-14T18:43:15.768366+0000","172.21.15.146:6833/3023779168":"2024-09-14T18:52:35.902095+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:57:06.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.199+0000 7f56bb048700 1 -- 172.21.15.146:0/2718899026 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56a00668b0 msgr2=0x7f56a0068d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:06.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.199+0000 7f56bb048700 1 --2- 172.21.15.146:0/2718899026 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56a00668b0 0x7f56a0068d70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f56b40c5920 tx=0x7f56a4009240 comp rx=0 tx=0).stop 2024-09-13T18:57:06.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.199+0000 7f56bb048700 1 -- 172.21.15.146:0/2718899026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56b40c6610 msgr2=0x7f56b40c3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:06.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.199+0000 7f56bb048700 1 --2- 172.21.15.146:0/2718899026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56b40c6610 0x7f56b40c3a00 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f56a800c4d0 tx=0x7f56a800c890 comp rx=0 tx=0).stop 2024-09-13T18:57:06.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.199+0000 7f56bb048700 1 -- 172.21.15.146:0/2718899026 shutdown_connections 2024-09-13T18:57:06.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.199+0000 7f56bb048700 1 --2- 172.21.15.146:0/2718899026 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f56a00668b0 0x7f56a0068d70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:06.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.199+0000 7f56bb048700 1 --2- 172.21.15.146:0/2718899026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b4153bc0 0x7f56b414cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:06.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.199+0000 7f56bb048700 1 --2- 172.21.15.146:0/2718899026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b40c6a80 0x7f56b40c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:06.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.199+0000 7f56bb048700 1 --2- 172.21.15.146:0/2718899026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56b40c6610 0x7f56b40c3a00 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:06.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.199+0000 7f56bb048700 1 -- 172.21.15.146:0/2718899026 >> 172.21.15.146:0/2718899026 conn(0x7f56b4066090 msgr2=0x7f56b40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:06.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.199+0000 7f56bb048700 1 -- 172.21.15.146:0/2718899026 shutdown_connections 2024-09-13T18:57:06.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:06.199+0000 7f56bb048700 1 -- 172.21.15.146:0/2718899026 wait complete. 2024-09-13T18:57:07.226 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T18:57:07.226 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:57:07.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f49118bf700 1 Processor -- start 2024-09-13T18:57:07.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f49118bf700 1 -- start start 2024-09-13T18:57:07.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f49118bf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f490c070e00 0x7f490c0c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:07.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f49118bf700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f490c15a450 0x7f490c1536e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:07.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f49118bf700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c06d010 con 0x7f490c06cbd0 2024-09-13T18:57:07.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f49118bf700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c0adad0 con 0x7f490c15a450 2024-09-13T18:57:07.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f49118bf700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0add70 con 0x7f490c070e00 2024-09-13T18:57:07.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f490b7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f490c15a450 0x7f490c1536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:07.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f490b7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f490c15a450 0x7f490c1536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57620/0 (socket says 172.21.15.146:57620) 2024-09-13T18:57:07.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f490b7fe700 1 -- 172.21.15.146:0/330008715 learned_addr learned my addr 172.21.15.146:0/330008715 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:07.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f490affd700 1 --2- 172.21.15.146:0/330008715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f490c070e00 0x7f490c0c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:07.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f490b7fe700 1 -- 172.21.15.146:0/330008715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f490c070e00 msgr2=0x7f490c0c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:07.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f490b7fe700 1 --2- 172.21.15.146:0/330008715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f490c070e00 0x7f490c0c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:07.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f490b7fe700 1 -- 172.21.15.146:0/330008715 >> v1:172.21.15.154:6789/0 conn(0x7f490c06cbd0 legacy=0x7f490c0711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:07.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f490b7fe700 1 -- 172.21.15.146:0/330008715 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c0c0370 con 0x7f490c15a450 2024-09-13T18:57:07.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f490b7fe700 1 --2- 172.21.15.146:0/330008715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c15a450 0x7f490c1536e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4904009e50 tx=0x7f490400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55217246b926bb72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:07.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f4909ffb700 1 -- 172.21.15.146:0/330008715 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490400e070 con 0x7f490c15a450 2024-09-13T18:57:07.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f4909ffb700 1 -- 172.21.15.146:0/330008715 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490400b910 con 0x7f490c15a450 2024-09-13T18:57:07.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.367+0000 7f4909ffb700 1 -- 172.21.15.146:0/330008715 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490400bcc0 con 0x7f490c15a450 2024-09-13T18:57:07.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 -- 172.21.15.146:0/330008715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c15a450 msgr2=0x7f490c1536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:07.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 --2- 172.21.15.146:0/330008715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c15a450 0x7f490c1536e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4904009e50 tx=0x7f490400b330 comp rx=0 tx=0).stop 2024-09-13T18:57:07.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 -- 172.21.15.146:0/330008715 shutdown_connections 2024-09-13T18:57:07.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 --2- 172.21.15.146:0/330008715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c15a450 0x7f490c1536e0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:07.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 --2- 172.21.15.146:0/330008715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f490c070e00 0x7f490c0c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:07.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 -- 172.21.15.146:0/330008715 >> 172.21.15.146:0/330008715 conn(0x7f490c06f5c0 msgr2=0x7f490c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:07.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 -- 172.21.15.146:0/330008715 shutdown_connections 2024-09-13T18:57:07.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 -- 172.21.15.146:0/330008715 wait complete. 2024-09-13T18:57:07.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 Processor -- start 2024-09-13T18:57:07.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 -- start start 2024-09-13T18:57:07.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f490c06cbd0 0x7f490c14ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:07.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c070e00 0x7f490c1f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:07.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f490c15a450 0x7f490c157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:07.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c14b3a0 con 0x7f490c070e00 2024-09-13T18:57:07.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0af6d0 con 0x7f490c06cbd0 2024-09-13T18:57:07.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c159840 con 0x7f490c15a450 2024-09-13T18:57:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f490a7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c070e00 0x7f490c1f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f490a7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c070e00 0x7f490c1f23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57626/0 (socket says 172.21.15.146:57626) 2024-09-13T18:57:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f490a7fc700 1 -- 172.21.15.146:0/194699592 learned_addr learned my addr 172.21.15.146:0/194699592 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f490affd700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f490c06cbd0 0x7f490c14ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f490b7fe700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f490c15a450 0x7f490c157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f490a7fc700 1 -- 172.21.15.146:0/194699592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f490c15a450 msgr2=0x7f490c157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f490a7fc700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f490c15a450 0x7f490c157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f490a7fc700 1 -- 172.21.15.146:0/194699592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f490c06cbd0 msgr2=0x7f490c14ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f490a7fc700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f490c06cbd0 0x7f490c14ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f490a7fc700 1 -- 172.21.15.146:0/194699592 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4904009b30 con 0x7f490c070e00 2024-09-13T18:57:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f490a7fc700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c070e00 0x7f490c1f23e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f490000c4d0 tx=0x7f490000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f48f3fff700 1 -- 172.21.15.146:0/194699592 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49000052d0 con 0x7f490c070e00 2024-09-13T18:57:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f48f3fff700 1 -- 172.21.15.146:0/194699592 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4900005430 con 0x7f490c070e00 2024-09-13T18:57:07.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.371+0000 7f49118bf700 1 -- 172.21.15.146:0/194699592 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c159b00 con 0x7f490c070e00 2024-09-13T18:57:07.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.375+0000 7f49118bf700 1 -- 172.21.15.146:0/194699592 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c1f2f80 con 0x7f490c070e00 2024-09-13T18:57:07.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.375+0000 7f48f3fff700 1 -- 172.21.15.146:0/194699592 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49000055a0 con 0x7f490c070e00 2024-09-13T18:57:07.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.375+0000 7f49118bf700 1 -- 172.21.15.146:0/194699592 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f490c0c1b30 con 0x7f490c070e00 2024-09-13T18:57:07.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.375+0000 7f48f3fff700 1 -- 172.21.15.146:0/194699592 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f49000038c0 con 0x7f490c070e00 2024-09-13T18:57:07.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.375+0000 7f48f3fff700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48f8066730 0x7f48f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:07.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.375+0000 7f48f3fff700 1 -- 172.21.15.146:0/194699592 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f49000845b0 con 0x7f490c070e00 2024-09-13T18:57:07.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.375+0000 7f48f3fff700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] conn(0x7f48f8070cd0 0x7f48f8073170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:07.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.375+0000 7f48f3fff700 1 -- 172.21.15.146:0/194699592 --> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48f8073840 con 0x7f48f8070cd0 2024-09-13T18:57:07.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.375+0000 7f48f3fff700 1 -- 172.21.15.146:0/194699592 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4900084a00 con 0x7f490c070e00 2024-09-13T18:57:07.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.375+0000 7f490b7fe700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] conn(0x7f48f8070cd0 0x7f48f8073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:07.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.375+0000 7f490b7fe700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] conn(0x7f48f8070cd0 0x7f48f8073170 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:07.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.379+0000 7f48f3fff700 1 -- 172.21.15.146:0/194699592 <== osd.0 v2:172.21.15.146:6816/2811789208 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f48f8073840 con 0x7f48f8070cd0 2024-09-13T18:57:07.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.379+0000 7f490affd700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48f8066730 0x7f48f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:07.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.387+0000 7f490affd700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48f8066730 0x7f48f8068bf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009bf0 tx=0x7f48fc009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:07.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.399+0000 7f49118bf700 1 -- 172.21.15.146:0/194699592 --> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f490c0ae290 con 0x7f48f8070cd0 2024-09-13T18:57:07.411 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.399+0000 7f48f3fff700 1 -- 172.21.15.146:0/194699592 <== osd.0 v2:172.21.15.146:6816/2811789208 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f490400bc00 con 0x7f48f8070cd0 2024-09-13T18:57:07.412 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:57:07.412 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:57:07.412 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:57:07.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.407+0000 7f49118bf700 1 -- 172.21.15.146:0/194699592 >> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] conn(0x7f48f8070cd0 msgr2=0x7f48f8073170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:07.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.407+0000 7f49118bf700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.146:6816/2811789208,v1:172.21.15.146:6817/2811789208] conn(0x7f48f8070cd0 0x7f48f8073170 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:07.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.407+0000 7f49118bf700 1 -- 172.21.15.146:0/194699592 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48f8066730 msgr2=0x7f48f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:07.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.407+0000 7f49118bf700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f48f8066730 0x7f48f8068bf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009bf0 tx=0x7f48fc009240 comp rx=0 tx=0).stop 2024-09-13T18:57:07.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.407+0000 7f49118bf700 1 -- 172.21.15.146:0/194699592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c070e00 msgr2=0x7f490c1f23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:07.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.407+0000 7f49118bf700 1 --2- 172.21.15.146:0/194699592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c070e00 0x7f490c1f23e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f490000c4d0 tx=0x7f490000c7e0 comp rx=0 tx=0).stop 2024-09-13T18:57:07.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.407+0000 7f490b7fe700 1 -- 172.21.15.146:0/194699592 reap_dead start 2024-09-13T18:57:07.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.407+0000 7f49118bf700 1 -- 172.21.15.146:0/194699592 shutdown_connections 2024-09-13T18:57:07.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.407+0000 7f49118bf700 1 -- 172.21.15.146:0/194699592 >> 172.21.15.146:0/194699592 conn(0x7f490c06f5c0 msgr2=0x7f490c14dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:07.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.411+0000 7f49118bf700 1 -- 172.21.15.146:0/194699592 shutdown_connections 2024-09-13T18:57:07.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.411+0000 7f49118bf700 1 -- 172.21.15.146:0/194699592 wait complete. 2024-09-13T18:57:07.441 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T18:57:07.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.647+0000 7fc4b359e700 1 Processor -- start 2024-09-13T18:57:07.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.647+0000 7fc4b359e700 1 -- start start 2024-09-13T18:57:07.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.647+0000 7fc4b359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4b40c9120 0x7fc4b40a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:07.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.647+0000 7fc4b359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc4b4072460 0x7fc4b40aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:07.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.647+0000 7fc4b359e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b40a9490 con 0x7fc4b40c8d40 2024-09-13T18:57:07.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.647+0000 7fc4b359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b40ca400 con 0x7fc4b4072460 2024-09-13T18:57:07.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.647+0000 7fc4b359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b40c7ec0 con 0x7fc4b40c9120 2024-09-13T18:57:07.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.647+0000 7fc4b259c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc4b40c8d40 0x7fc4b40c76a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:40270/0 (socket says 172.21.15.146:40270) 2024-09-13T18:57:07.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.647+0000 7fc4b259c700 1 -- 172.21.15.146:0/2568785449 learned_addr learned my addr 172.21.15.146:0/2568785449 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:57:07.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.651+0000 7fc4b2d9d700 1 --2- 172.21.15.146:0/2568785449 >> v2:172.21.15.146:3300/0 conn(0x7fc4b4072460 0x7fc4b40aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:07.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.651+0000 7fc4b1d9b700 1 --2- 172.21.15.146:0/2568785449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4b40c9120 0x7fc4b40a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:07.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.651+0000 7fc4b1d9b700 1 -- 172.21.15.146:0/2568785449 >> v2:172.21.15.146:3300/0 conn(0x7fc4b4072460 msgr2=0x7fc4b40aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:07.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.651+0000 7fc4b1d9b700 1 --2- 172.21.15.146:0/2568785449 >> v2:172.21.15.146:3300/0 conn(0x7fc4b4072460 0x7fc4b40aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:07.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.651+0000 7fc4b1d9b700 1 -- 172.21.15.146:0/2568785449 >> v1:172.21.15.154:6789/0 conn(0x7fc4b40c8d40 legacy=0x7fc4b40c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:07.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.651+0000 7fc4b1d9b700 1 -- 172.21.15.146:0/2568785449 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b40c81a0 con 0x7fc4b40c9120 2024-09-13T18:57:07.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.651+0000 7fc4b1d9b700 1 --2- 172.21.15.146:0/2568785449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4b40c9120 0x7fc4b40a9ee0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009d90 tx=0x7fc4ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba945e32db1a2051 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:07.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.655+0000 7fc4b159a700 1 -- 172.21.15.146:0/2568785449 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac00e070 con 0x7fc4b40c9120 2024-09-13T18:57:07.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.655+0000 7fc4b159a700 1 -- 172.21.15.146:0/2568785449 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ac003a40 con 0x7fc4b40c9120 2024-09-13T18:57:07.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.655+0000 7fc4b159a700 1 -- 172.21.15.146:0/2568785449 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac003df0 con 0x7fc4b40c9120 2024-09-13T18:57:07.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.659+0000 7fc4b359e700 1 -- 172.21.15.146:0/2568785449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4b40c9120 msgr2=0x7fc4b40a9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:07.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.659+0000 7fc4b359e700 1 --2- 172.21.15.146:0/2568785449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4b40c9120 0x7fc4b40a9ee0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009d90 tx=0x7fc4ac00b330 comp rx=0 tx=0).stop 2024-09-13T18:57:07.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.659+0000 7fc4b359e700 1 -- 172.21.15.146:0/2568785449 shutdown_connections 2024-09-13T18:57:07.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.659+0000 7fc4b359e700 1 --2- 172.21.15.146:0/2568785449 >> v2:172.21.15.146:3300/0 conn(0x7fc4b4072460 0x7fc4b40aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:07.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.659+0000 7fc4b359e700 1 --2- 172.21.15.146:0/2568785449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4b40c9120 0x7fc4b40a9ee0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:07.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.659+0000 7fc4b359e700 1 -- 172.21.15.146:0/2568785449 >> 172.21.15.146:0/2568785449 conn(0x7fc4b4066090 msgr2=0x7fc4b4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:07.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b359e700 1 -- 172.21.15.146:0/2568785449 shutdown_connections 2024-09-13T18:57:07.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b359e700 1 -- 172.21.15.146:0/2568785449 wait complete. 2024-09-13T18:57:07.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b359e700 1 Processor -- start 2024-09-13T18:57:07.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b359e700 1 -- start start 2024-09-13T18:57:07.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b4072460 0x7fc4b41602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:07.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4b40c8d40 0x7fc4b4070000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:07.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4b40c9120 0x7fc4b4159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:07.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b4159270 con 0x7fc4b40c8d40 2024-09-13T18:57:07.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b40acb50 con 0x7fc4b4072460 2024-09-13T18:57:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b40ace30 con 0x7fc4b40c9120 2024-09-13T18:57:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4b40c8d40 0x7fc4b4070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4b40c8d40 0x7fc4b4070000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57644/0 (socket says 172.21.15.146:57644) 2024-09-13T18:57:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b1d9b700 1 -- 172.21.15.146:0/3333583619 learned_addr learned my addr 172.21.15.146:0/3333583619 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b2d9d700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4b40c9120 0x7fc4b4159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.663+0000 7fc4b259c700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b4072460 0x7fc4b41602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4b1d9b700 1 -- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4b40c9120 msgr2=0x7fc4b4159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4b1d9b700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4b40c9120 0x7fc4b4159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4b1d9b700 1 -- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b4072460 msgr2=0x7fc4b41602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4b1d9b700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b4072460 0x7fc4b41602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4b1d9b700 1 -- 172.21.15.146:0/3333583619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac009a70 con 0x7fc4b40c8d40 2024-09-13T18:57:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4b259c700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b4072460 0x7fc4b41602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:57:07.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4b1d9b700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4b40c8d40 0x7fc4b4070000 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40c8000 tx=0x7fc4ac004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:07.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4a2ffd700 1 -- 172.21.15.146:0/3333583619 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac004570 con 0x7fc4b40c8d40 2024-09-13T18:57:07.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4a2ffd700 1 -- 172.21.15.146:0/3333583619 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ac0046d0 con 0x7fc4b40c8d40 2024-09-13T18:57:07.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4b359e700 1 -- 172.21.15.146:0/3333583619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b40ab390 con 0x7fc4b40c8d40 2024-09-13T18:57:07.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4a2ffd700 1 -- 172.21.15.146:0/3333583619 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac012430 con 0x7fc4b40c8d40 2024-09-13T18:57:07.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4b359e700 1 -- 172.21.15.146:0/3333583619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b41f62d0 con 0x7fc4b40c8d40 2024-09-13T18:57:07.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4b359e700 1 -- 172.21.15.146:0/3333583619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b41593e0 con 0x7fc4b40c8d40 2024-09-13T18:57:07.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4a2ffd700 1 -- 172.21.15.146:0/3333583619 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc4ac012590 con 0x7fc4b40c8d40 2024-09-13T18:57:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4a2ffd700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc494066730 0x7fc494068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4a2ffd700 1 -- 172.21.15.146:0/3333583619 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fc4ac086060 con 0x7fc4b40c8d40 2024-09-13T18:57:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4b259c700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc494066730 0x7fc494068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:07.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.667+0000 7fc4b259c700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc494066730 0x7fc494068bf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80070a0 tx=0x7fc4a8007e30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:07.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:07.683+0000 7fc4a2ffd700 1 -- 172.21.15.146:0/3333583619 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc4ac0565d0 con 0x7fc4b40c8d40 2024-09-13T18:57:08.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.019+0000 7fc4b359e700 1 -- 172.21.15.146:0/3333583619 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4b40a99c0 con 0x7fc494066730 2024-09-13T18:57:08.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.023+0000 7fc4a2ffd700 1 -- 172.21.15.146:0/3333583619 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc4a8005cd0 con 0x7fc494066730 2024-09-13T18:57:08.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.031+0000 7fc4b359e700 1 -- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc494066730 msgr2=0x7fc494068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.031+0000 7fc4b359e700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc494066730 0x7fc494068bf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80070a0 tx=0x7fc4a8007e30 comp rx=0 tx=0).stop 2024-09-13T18:57:08.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.031+0000 7fc4b359e700 1 -- 172.21.15.146:0/3333583619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4b40c8d40 msgr2=0x7fc4b4070000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.031+0000 7fc4b359e700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4b40c8d40 0x7fc4b4070000 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40c8000 tx=0x7fc4ac004af0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.031+0000 7fc4b359e700 1 -- 172.21.15.146:0/3333583619 shutdown_connections 2024-09-13T18:57:08.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.031+0000 7fc4b359e700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fc494066730 0x7fc494068bf0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.031+0000 7fc4b359e700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4b40c9120 0x7fc4b4159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.031+0000 7fc4b359e700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4b40c8d40 0x7fc4b4070000 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.031+0000 7fc4b359e700 1 --2- 172.21.15.146:0/3333583619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b4072460 0x7fc4b41602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.031+0000 7fc4b359e700 1 -- 172.21.15.146:0/3333583619 >> 172.21.15.146:0/3333583619 conn(0x7fc4b4066090 msgr2=0x7fc4b40aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:08.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.031+0000 7fc4b359e700 1 -- 172.21.15.146:0/3333583619 shutdown_connections 2024-09-13T18:57:08.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.031+0000 7fc4b359e700 1 -- 172.21.15.146:0/3333583619 wait complete. 2024-09-13T18:57:08.050 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T18:57:08.065 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T18:57:08.066 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:57:08.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.267+0000 7fba49724700 1 Processor -- start 2024-09-13T18:57:08.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.267+0000 7fba49724700 1 -- start start 2024-09-13T18:57:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.267+0000 7fba49724700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fba440c8f70 0x7fba4415a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.267+0000 7fba49724700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fba440c9350 0x7fba440c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.267+0000 7fba49724700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba440c8650 con 0x7fba440a9490 2024-09-13T18:57:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.267+0000 7fba49724700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba440aa1a0 con 0x7fba440c8f70 2024-09-13T18:57:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.267+0000 7fba49724700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba440aa480 con 0x7fba440c9350 2024-09-13T18:57:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.267+0000 7fba43fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fba440c8f70 0x7fba4415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.271+0000 7fba43fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fba440c8f70 0x7fba4415a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57652/0 (socket says 172.21.15.146:57652) 2024-09-13T18:57:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.271+0000 7fba43fff700 1 -- 172.21.15.146:0/2718053641 learned_addr learned my addr 172.21.15.146:0/2718053641 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.271+0000 7fba437fe700 1 --2- 172.21.15.146:0/2718053641 >> v2:172.21.15.154:3300/0 conn(0x7fba440c9350 0x7fba440c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.271+0000 7fba43fff700 1 -- 172.21.15.146:0/2718053641 >> v2:172.21.15.154:3300/0 conn(0x7fba440c9350 msgr2=0x7fba440c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.271+0000 7fba43fff700 1 --2- 172.21.15.146:0/2718053641 >> v2:172.21.15.154:3300/0 conn(0x7fba440c9350 0x7fba440c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.271+0000 7fba43fff700 1 -- 172.21.15.146:0/2718053641 >> v1:172.21.15.146:6789/0 conn(0x7fba440a9490 legacy=0x7fba440cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.271+0000 7fba43fff700 1 -- 172.21.15.146:0/2718053641 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba44159890 con 0x7fba440c8f70 2024-09-13T18:57:08.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.275+0000 7fba43fff700 1 --2- 172.21.15.146:0/2718053641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba440c8f70 0x7fba4415a370 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fba440c7300 tx=0x7fba3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc21ea4bbcc5d5b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:08.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.275+0000 7fba42ffd700 1 -- 172.21.15.146:0/2718053641 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c00e070 con 0x7fba440c8f70 2024-09-13T18:57:08.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.275+0000 7fba42ffd700 1 -- 172.21.15.146:0/2718053641 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3c00b910 con 0x7fba440c8f70 2024-09-13T18:57:08.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.275+0000 7fba42ffd700 1 -- 172.21.15.146:0/2718053641 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c00e070 con 0x7fba440c8f70 2024-09-13T18:57:08.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.275+0000 7fba49724700 1 -- 172.21.15.146:0/2718053641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba440c8f70 msgr2=0x7fba4415a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.275+0000 7fba49724700 1 --2- 172.21.15.146:0/2718053641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba440c8f70 0x7fba4415a370 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fba440c7300 tx=0x7fba3c00b330 comp rx=0 tx=0).stop 2024-09-13T18:57:08.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.275+0000 7fba49724700 1 -- 172.21.15.146:0/2718053641 shutdown_connections 2024-09-13T18:57:08.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.275+0000 7fba49724700 1 --2- 172.21.15.146:0/2718053641 >> v2:172.21.15.154:3300/0 conn(0x7fba440c9350 0x7fba440c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.275+0000 7fba49724700 1 --2- 172.21.15.146:0/2718053641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba440c8f70 0x7fba4415a370 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.275+0000 7fba49724700 1 -- 172.21.15.146:0/2718053641 >> 172.21.15.146:0/2718053641 conn(0x7fba44066090 msgr2=0x7fba44066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:08.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.275+0000 7fba49724700 1 -- 172.21.15.146:0/2718053641 shutdown_connections 2024-09-13T18:57:08.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.275+0000 7fba49724700 1 -- 172.21.15.146:0/2718053641 wait complete. 2024-09-13T18:57:08.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.279+0000 7fba49724700 1 Processor -- start 2024-09-13T18:57:08.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.279+0000 7fba49724700 1 -- start start 2024-09-13T18:57:08.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba49724700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba440a9490 0x7fba44151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba49724700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba440c8f70 0x7fba440af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba49724700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba440c9350 0x7fba440ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba49724700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba440ad870 con 0x7fba440c9350 2024-09-13T18:57:08.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba49724700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba44151560 con 0x7fba440c8f70 2024-09-13T18:57:08.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba49724700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba44153640 con 0x7fba440a9490 2024-09-13T18:57:08.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba48f23700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba440c9350 0x7fba440ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba48f23700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba440c9350 0x7fba440ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57662/0 (socket says 172.21.15.146:57662) 2024-09-13T18:57:08.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba48f23700 1 -- 172.21.15.146:0/2558769869 learned_addr learned my addr 172.21.15.146:0/2558769869 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:08.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba437fe700 1 --2- 172.21.15.146:0/2558769869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba440c8f70 0x7fba440af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba48f23700 1 -- 172.21.15.146:0/2558769869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba440a9490 msgr2=0x7fba44151940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:57:08.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba48f23700 1 --2- 172.21.15.146:0/2558769869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba440a9490 0x7fba44151940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba48f23700 1 -- 172.21.15.146:0/2558769869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba440c8f70 msgr2=0x7fba440af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba48f23700 1 --2- 172.21.15.146:0/2558769869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba440c8f70 0x7fba440af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba48f23700 1 -- 172.21.15.146:0/2558769869 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c009b30 con 0x7fba440c9350 2024-09-13T18:57:08.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba48f23700 1 --2- 172.21.15.146:0/2558769869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba440c9350 0x7fba440ace40 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c390 tx=0x7fba3400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:08.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba40ff9700 1 -- 172.21.15.146:0/2558769869 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba340052d0 con 0x7fba440c9350 2024-09-13T18:57:08.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba40ff9700 1 -- 172.21.15.146:0/2558769869 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba34005430 con 0x7fba440c9350 2024-09-13T18:57:08.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba40ff9700 1 -- 172.21.15.146:0/2558769869 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba340055a0 con 0x7fba440c9350 2024-09-13T18:57:08.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba49724700 1 -- 172.21.15.146:0/2558769869 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba44153920 con 0x7fba440c9350 2024-09-13T18:57:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba49724700 1 -- 172.21.15.146:0/2558769869 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba44153ee0 con 0x7fba440c9350 2024-09-13T18:57:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.283+0000 7fba49724700 1 -- 172.21.15.146:0/2558769869 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fba44066a10 con 0x7fba440c9350 2024-09-13T18:57:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.287+0000 7fba40ff9700 1 -- 172.21.15.146:0/2558769869 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fba340036a0 con 0x7fba440c9350 2024-09-13T18:57:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.287+0000 7fba40ff9700 1 --2- 172.21.15.146:0/2558769869 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fba300667b0 0x7fba30068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.287+0000 7fba40ff9700 1 -- 172.21.15.146:0/2558769869 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fba34083a10 con 0x7fba440c9350 2024-09-13T18:57:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.287+0000 7fba40ff9700 1 --2- 172.21.15.146:0/2558769869 >> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] conn(0x7fba30070e50 0x7fba300732f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.287+0000 7fba40ff9700 1 -- 172.21.15.146:0/2558769869 --> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba300739c0 con 0x7fba30070e50 2024-09-13T18:57:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.287+0000 7fba40ff9700 1 -- 172.21.15.146:0/2558769869 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba340882c0 con 0x7fba440c9350 2024-09-13T18:57:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.287+0000 7fba43fff700 1 --2- 172.21.15.146:0/2558769869 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fba300667b0 0x7fba30068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.287+0000 7fba43fff700 1 --2- 172.21.15.146:0/2558769869 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fba300667b0 0x7fba30068c70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00b370 tx=0x7fba3c009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:08.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.291+0000 7fba437fe700 1 --2- 172.21.15.146:0/2558769869 >> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] conn(0x7fba30070e50 0x7fba300732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.299+0000 7fba437fe700 1 --2- 172.21.15.146:0/2558769869 >> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] conn(0x7fba30070e50 0x7fba300732f0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:08.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.303+0000 7fba40ff9700 1 -- 172.21.15.146:0/2558769869 <== osd.1 v2:172.21.15.146:6804/3907005797 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fba300739c0 con 0x7fba30070e50 2024-09-13T18:57:08.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.327+0000 7fba49724700 1 -- 172.21.15.146:0/2558769869 --> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fba440c6670 con 0x7fba30070e50 2024-09-13T18:57:08.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.327+0000 7fba40ff9700 1 -- 172.21.15.146:0/2558769869 <== osd.1 v2:172.21.15.146:6804/3907005797 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fba440c6670 con 0x7fba30070e50 2024-09-13T18:57:08.340 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:57:08.341 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:57:08.341 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:57:08.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.331+0000 7fba49724700 1 -- 172.21.15.146:0/2558769869 >> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] conn(0x7fba30070e50 msgr2=0x7fba300732f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.331+0000 7fba49724700 1 --2- 172.21.15.146:0/2558769869 >> [v2:172.21.15.146:6804/3907005797,v1:172.21.15.146:6806/3907005797] conn(0x7fba30070e50 0x7fba300732f0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.331+0000 7fba49724700 1 -- 172.21.15.146:0/2558769869 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fba300667b0 msgr2=0x7fba30068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.331+0000 7fba49724700 1 --2- 172.21.15.146:0/2558769869 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fba300667b0 0x7fba30068c70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00b370 tx=0x7fba3c009000 comp rx=0 tx=0).stop 2024-09-13T18:57:08.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.331+0000 7fba49724700 1 -- 172.21.15.146:0/2558769869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba440c9350 msgr2=0x7fba440ace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.331+0000 7fba49724700 1 --2- 172.21.15.146:0/2558769869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba440c9350 0x7fba440ace40 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c390 tx=0x7fba3400c6a0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.331+0000 7fba48f23700 1 -- 172.21.15.146:0/2558769869 reap_dead start 2024-09-13T18:57:08.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.331+0000 7fba49724700 1 -- 172.21.15.146:0/2558769869 shutdown_connections 2024-09-13T18:57:08.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.331+0000 7fba49724700 1 -- 172.21.15.146:0/2558769869 >> 172.21.15.146:0/2558769869 conn(0x7fba44066090 msgr2=0x7fba44157ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:08.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.335+0000 7fba49724700 1 -- 172.21.15.146:0/2558769869 shutdown_connections 2024-09-13T18:57:08.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.339+0000 7fba49724700 1 -- 172.21.15.146:0/2558769869 wait complete. 2024-09-13T18:57:08.367 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T18:57:08.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.555+0000 7fbe02b3f700 1 Processor -- start 2024-09-13T18:57:08.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.555+0000 7fbe02b3f700 1 -- start start 2024-09-13T18:57:08.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe02b3f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdfc06eea0 0x7fbdfc06f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe02b3f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbdfc06fb00 0x7fbdfc0c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe02b3f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdfc06ffb0 con 0x7fbdfc0a9490 2024-09-13T18:57:08.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe02b3f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdfc14fbf0 con 0x7fbdfc06fb00 2024-09-13T18:57:08.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe02b3f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc155860 con 0x7fbdfc06eea0 2024-09-13T18:57:08.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe010dc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbdfc06fb00 0x7fbdfc0c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe010dc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbdfc06fb00 0x7fbdfc0c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:35056/0 (socket says 172.21.15.146:35056) 2024-09-13T18:57:08.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe010dc700 1 -- 172.21.15.146:0/3416224070 learned_addr learned my addr 172.21.15.146:0/3416224070 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:08.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe010dc700 1 -- 172.21.15.146:0/3416224070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdfc06eea0 msgr2=0x7fbdfc06f5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:57:08.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe010dc700 1 --2- 172.21.15.146:0/3416224070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdfc06eea0 0x7fbdfc06f5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe010dc700 1 -- 172.21.15.146:0/3416224070 >> v1:172.21.15.154:6789/0 conn(0x7fbdfc0a9490 legacy=0x7fbdfc14f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe010dc700 1 -- 172.21.15.146:0/3416224070 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdfc155b40 con 0x7fbdfc06fb00 2024-09-13T18:57:08.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe010dc700 1 --2- 172.21.15.146:0/3416224070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdfc06fb00 0x7fbdfc0c7960 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009910 tx=0x7fbdec009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc99223c60ccb03a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:08.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbdfb7fe700 1 -- 172.21.15.146:0/3416224070 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec009e60 con 0x7fbdfc06fb00 2024-09-13T18:57:08.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbdfb7fe700 1 -- 172.21.15.146:0/3416224070 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdec00b860 con 0x7fbdfc06fb00 2024-09-13T18:57:08.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3416224070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdfc06fb00 msgr2=0x7fbdfc0c7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.559+0000 7fbe02b3f700 1 --2- 172.21.15.146:0/3416224070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdfc06fb00 0x7fbdfc0c7960 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009910 tx=0x7fbdec009c20 comp rx=0 tx=0).stop 2024-09-13T18:57:08.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.571+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3416224070 shutdown_connections 2024-09-13T18:57:08.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.571+0000 7fbe02b3f700 1 --2- 172.21.15.146:0/3416224070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdfc06fb00 0x7fbdfc0c7960 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.571+0000 7fbe02b3f700 1 --2- 172.21.15.146:0/3416224070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdfc06eea0 0x7fbdfc06f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.571+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3416224070 >> 172.21.15.146:0/3416224070 conn(0x7fbdfc066090 msgr2=0x7fbdfc066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:08.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.575+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3416224070 shutdown_connections 2024-09-13T18:57:08.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.575+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3416224070 wait complete. 2024-09-13T18:57:08.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.579+0000 7fbe02b3f700 1 Processor -- start 2024-09-13T18:57:08.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.583+0000 7fbe02b3f700 1 -- start start 2024-09-13T18:57:08.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.583+0000 7fbe02b3f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdfc06eea0 0x7fbdfc0afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.583+0000 7fbe02b3f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdfc06fb00 0x7fbdfc1ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.583+0000 7fbe02b3f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdfc0a9490 0x7fbdfc1ee720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.583+0000 7fbe02b3f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc1eddc0 con 0x7fbdfc06eea0 2024-09-13T18:57:08.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.583+0000 7fbe02b3f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc1565f0 con 0x7fbdfc06fb00 2024-09-13T18:57:08.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.583+0000 7fbe02b3f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc0ad8a0 con 0x7fbdfc0a9490 2024-09-13T18:57:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe008db700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdfc06eea0 0x7fbdfc0afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe008db700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdfc06eea0 0x7fbdfc0afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57676/0 (socket says 172.21.15.146:57676) 2024-09-13T18:57:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe008db700 1 -- 172.21.15.146:0/3894548560 learned_addr learned my addr 172.21.15.146:0/3894548560 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe010dc700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdfc0a9490 0x7fbdfc1ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe008db700 1 -- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdfc0a9490 msgr2=0x7fbdfc1ee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe008db700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdfc0a9490 0x7fbdfc1ee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe008db700 1 -- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdfc06fb00 msgr2=0x7fbdfc1ee1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:57:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe008db700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdfc06fb00 0x7fbdfc1ee1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe008db700 1 -- 172.21.15.146:0/3894548560 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec009570 con 0x7fbdfc06eea0 2024-09-13T18:57:08.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe010dc700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdfc0a9490 0x7fbdfc1ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:57:08.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe008db700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdfc06eea0 0x7fbdfc0afa70 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4001ed0 tx=0x7fbdf400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:08.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbdf97fa700 1 -- 172.21.15.146:0/3894548560 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400ade0 con 0x7fbdfc06eea0 2024-09-13T18:57:08.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbdf97fa700 1 -- 172.21.15.146:0/3894548560 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf40020d0 con 0x7fbdfc06eea0 2024-09-13T18:57:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbdf97fa700 1 -- 172.21.15.146:0/3894548560 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4003c40 con 0x7fbdfc06eea0 2024-09-13T18:57:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3894548560 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdfc1568b0 con 0x7fbdfc06eea0 2024-09-13T18:57:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3894548560 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdfc156e70 con 0x7fbdfc06eea0 2024-09-13T18:57:08.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.587+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3894548560 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdfc06f5c0 con 0x7fbdfc06eea0 2024-09-13T18:57:08.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.591+0000 7fbdf97fa700 1 -- 172.21.15.146:0/3894548560 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbdf40050d0 con 0x7fbdfc06eea0 2024-09-13T18:57:08.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.591+0000 7fbdf97fa700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbde0066730 0x7fbde0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.591+0000 7fbdf97fa700 1 -- 172.21.15.146:0/3894548560 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fbdf40551e0 con 0x7fbdfc06eea0 2024-09-13T18:57:08.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.591+0000 7fbdfbfff700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbde0066730 0x7fbde0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.595+0000 7fbdfbfff700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbde0066730 0x7fbde0068bf0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc152fb0 tx=0x7fbdf0006bf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:08.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.599+0000 7fbdf97fa700 1 -- 172.21.15.146:0/3894548560 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbdf4089060 con 0x7fbdfc06eea0 2024-09-13T18:57:08.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.803+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3894548560 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdfc0aa6c0 con 0x7fbde0066730 2024-09-13T18:57:08.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.803+0000 7fbdf97fa700 1 -- 172.21.15.146:0/3894548560 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbdf0009330 con 0x7fbde0066730 2024-09-13T18:57:08.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.807+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbde0066730 msgr2=0x7fbde0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.807+0000 7fbe02b3f700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbde0066730 0x7fbde0068bf0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc152fb0 tx=0x7fbdf0006bf0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.807+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3894548560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdfc06eea0 msgr2=0x7fbdfc0afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.807+0000 7fbe02b3f700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdfc06eea0 0x7fbdfc0afa70 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4001ed0 tx=0x7fbdf400a340 comp rx=0 tx=0).stop 2024-09-13T18:57:08.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.807+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3894548560 shutdown_connections 2024-09-13T18:57:08.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.807+0000 7fbe02b3f700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbde0066730 0x7fbde0068bf0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.807+0000 7fbe02b3f700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdfc0a9490 0x7fbdfc1ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.807+0000 7fbe02b3f700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdfc06fb00 0x7fbdfc1ee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.807+0000 7fbe02b3f700 1 --2- 172.21.15.146:0/3894548560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdfc06eea0 0x7fbdfc0afa70 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.807+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3894548560 >> 172.21.15.146:0/3894548560 conn(0x7fbdfc066090 msgr2=0x7fbdfc0adb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:08.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.807+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3894548560 shutdown_connections 2024-09-13T18:57:08.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.807+0000 7fbe02b3f700 1 -- 172.21.15.146:0/3894548560 wait complete. 2024-09-13T18:57:08.822 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T18:57:08.834 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T18:57:08.834 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:57:08.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.923+0000 7f15fe7cb700 1 Processor -- start 2024-09-13T18:57:08.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.923+0000 7f15fe7cb700 1 -- start start 2024-09-13T18:57:08.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fe7cb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f15f00073f0 0x7f15f004eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fe7cb700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f00505d0 con 0x7f15f00d8c60 2024-09-13T18:57:08.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fe7cb700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f00508b0 con 0x7f15f004f920 2024-09-13T18:57:08.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fe7cb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f0049640 con 0x7f15f00073f0 2024-09-13T18:57:08.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fd7c9700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f15f00d8c60 0x7f15f000e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59148/0 (socket says 172.21.15.146:59148) 2024-09-13T18:57:08.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fd7c9700 1 -- 172.21.15.146:0/3572449429 learned_addr learned my addr 172.21.15.146:0/3572449429 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:57:08.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fcfc8700 1 --2- 172.21.15.146:0/3572449429 >> v2:172.21.15.154:3300/0 conn(0x7f15f00073f0 0x7f15f004eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533313591 0 0) 0x7f15f00505d0 con 0x7f15f00d8c60 2024-09-13T18:57:08.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15e40035d0 con 0x7f15f00d8c60 2024-09-13T18:57:08.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950969288 0 0) 0x7f15f00508b0 con 0x7f15f004f920 2024-09-13T18:57:08.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15f00505d0 con 0x7f15f004f920 2024-09-13T18:57:08.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2770547927 0 0) 0x7f15e40035d0 con 0x7f15f00d8c60 2024-09-13T18:57:08.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 >> v2:172.21.15.154:3300/0 conn(0x7f15f00073f0 msgr2=0x7f15f004eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:08.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 --2- 172.21.15.146:0/3572449429 >> v2:172.21.15.154:3300/0 conn(0x7f15f00073f0 0x7f15f004eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 >> v1:172.21.15.154:6789/0 conn(0x7f15f004f920 legacy=0x7f15f00e9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f004f120 con 0x7f15f00d8c60 2024-09-13T18:57:08.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f15f4001670 con 0x7f15f00d8c60 2024-09-13T18:57:08.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 >> v1:172.21.15.146:6789/0 conn(0x7f15f00d8c60 legacy=0x7f15f000e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 --2- 172.21.15.146:0/3572449429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15e4003bb0 0x7f15e4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 --2- 172.21.15.146:0/3572449429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15e4004870 0x7f15e4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 --2- 172.21.15.146:0/3572449429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15e4005230 0x7f15e400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f0049640 con 0x7f15e4004870 2024-09-13T18:57:08.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e400c020 con 0x7f15e4005230 2024-09-13T18:57:08.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e400c1e0 con 0x7f15e4003bb0 2024-09-13T18:57:08.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fcfc8700 1 --2- 172.21.15.146:0/3572449429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15e4004870 0x7f15e4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fdfca700 1 --2- 172.21.15.146:0/3572449429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15e4005230 0x7f15e400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fd7c9700 1 --2- 172.21.15.146:0/3572449429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15e4003bb0 0x7f15e4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fcfc8700 1 -- 172.21.15.146:0/3572449429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15e4003bb0 msgr2=0x7f15e4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:08.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fcfc8700 1 --2- 172.21.15.146:0/3572449429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15e4003bb0 0x7f15e4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fcfc8700 1 -- 172.21.15.146:0/3572449429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15e4005230 msgr2=0x7f15e400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:08.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fcfc8700 1 --2- 172.21.15.146:0/3572449429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15e4005230 0x7f15e400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fcfc8700 1 -- 172.21.15.146:0/3572449429 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e40035d0 con 0x7f15e4004870 2024-09-13T18:57:08.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fd7c9700 1 --2- 172.21.15.146:0/3572449429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15e4003bb0 0x7f15e4003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:57:08.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fdfca700 1 --2- 172.21.15.146:0/3572449429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15e4005230 0x7f15e400ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:57:08.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fdfca700 1 -- 172.21.15.146:0/3572449429 reap_dead start 2024-09-13T18:57:08.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fcfc8700 1 --2- 172.21.15.146:0/3572449429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15e4004870 0x7f15e4004cf0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009b40 tx=0x7f15e40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243a96acf40abd78 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:08.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e800e6c0 con 0x7f15e4004870 2024-09-13T18:57:08.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e800e820 con 0x7f15e4004870 2024-09-13T18:57:08.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15effff700 1 -- 172.21.15.146:0/3572449429 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e800eb90 con 0x7f15e4004870 2024-09-13T18:57:08.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fe7cb700 1 -- 172.21.15.146:0/3572449429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15e4004870 msgr2=0x7f15e4004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fe7cb700 1 --2- 172.21.15.146:0/3572449429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15e4004870 0x7f15e4004cf0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009b40 tx=0x7f15e40058a0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fe7cb700 1 -- 172.21.15.146:0/3572449429 shutdown_connections 2024-09-13T18:57:08.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fe7cb700 1 --2- 172.21.15.146:0/3572449429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15e4004870 0x7f15e4004cf0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fe7cb700 1 -- 172.21.15.146:0/3572449429 >> 172.21.15.146:0/3572449429 conn(0x7f15f0054520 msgr2=0x7f15f0054930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:08.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fe7cb700 1 -- 172.21.15.146:0/3572449429 shutdown_connections 2024-09-13T18:57:08.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.927+0000 7f15fe7cb700 1 -- 172.21.15.146:0/3572449429 wait complete. 2024-09-13T18:57:08.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fe7cb700 1 Processor -- start 2024-09-13T18:57:08.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fe7cb700 1 -- start start 2024-09-13T18:57:08.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fe7cb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15e4004870 0x7f15f00d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fe7cb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f00cf0e0 0x7f15f01750e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fe7cb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15f00ceb90 0x7f15f00d5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fe7cb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f00d3b90 con 0x7f15f00ceb90 2024-09-13T18:57:08.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fe7cb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f00d38d0 con 0x7f15f00cf0e0 2024-09-13T18:57:08.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fe7cb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f0174620 con 0x7f15e4004870 2024-09-13T18:57:08.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fdfca700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15f00ceb90 0x7f15f00d5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fdfca700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15f00ceb90 0x7f15f00d5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57700/0 (socket says 172.21.15.146:57700) 2024-09-13T18:57:08.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fdfca700 1 -- 172.21.15.146:0/673873572 learned_addr learned my addr 172.21.15.146:0/673873572 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:08.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fcfc8700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f00cf0e0 0x7f15f01750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fd7c9700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15e4004870 0x7f15f00d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fdfca700 1 -- 172.21.15.146:0/673873572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15e4004870 msgr2=0x7f15f00d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fdfca700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15e4004870 0x7f15f00d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fdfca700 1 -- 172.21.15.146:0/673873572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f00cf0e0 msgr2=0x7f15f01750e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fdfca700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f00cf0e0 0x7f15f01750e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fdfca700 1 -- 172.21.15.146:0/673873572 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f00d9a30 con 0x7f15f00ceb90 2024-09-13T18:57:08.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fdfca700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15f00ceb90 0x7f15f00d5340 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f15f8068650 tx=0x7f15f80739e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:08.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15edffb700 1 -- 172.21.15.146:0/673873572 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f8067950 con 0x7f15f00ceb90 2024-09-13T18:57:08.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15edffb700 1 -- 172.21.15.146:0/673873572 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f8075df0 con 0x7f15f00ceb90 2024-09-13T18:57:08.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15edffb700 1 -- 172.21.15.146:0/673873572 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f8067e90 con 0x7f15f00ceb90 2024-09-13T18:57:08.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fe7cb700 1 -- 172.21.15.146:0/673873572 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f01748e0 con 0x7f15f00ceb90 2024-09-13T18:57:08.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.931+0000 7f15fe7cb700 1 -- 172.21.15.146:0/673873572 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f00d4940 con 0x7f15f00ceb90 2024-09-13T18:57:08.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.935+0000 7f15fe7cb700 1 -- 172.21.15.146:0/673873572 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f15f00d6b50 con 0x7f15f00ceb90 2024-09-13T18:57:08.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.935+0000 7f15edffb700 1 -- 172.21.15.146:0/673873572 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f15f8070740 con 0x7f15f00ceb90 2024-09-13T18:57:08.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.935+0000 7f15edffb700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f15dc066730 0x7f15dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.935+0000 7f15edffb700 1 -- 172.21.15.146:0/673873572 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f15f80ff380 con 0x7f15f00ceb90 2024-09-13T18:57:08.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.935+0000 7f15edffb700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] conn(0x7f15dc070d50 0x7f15dc0731f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:08.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.935+0000 7f15fd7c9700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f15dc066730 0x7f15dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.935+0000 7f15edffb700 1 -- 172.21.15.146:0/673873572 --> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15dc0738c0 con 0x7f15dc070d50 2024-09-13T18:57:08.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.935+0000 7f15edffb700 1 -- 172.21.15.146:0/673873572 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15f8101a40 con 0x7f15f00ceb90 2024-09-13T18:57:08.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.935+0000 7f15fcfc8700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] conn(0x7f15dc070d50 0x7f15dc0731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:08.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.935+0000 7f15fcfc8700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] conn(0x7f15dc070d50 0x7f15dc0731f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:08.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.935+0000 7f15fd7c9700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f15dc066730 0x7f15dc068bf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f15f4005c60 tx=0x7f15f4007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:08.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.939+0000 7f15edffb700 1 -- 172.21.15.146:0/673873572 <== osd.2 v2:172.21.15.146:6800/3571336781 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f15dc0738c0 con 0x7f15dc070d50 2024-09-13T18:57:08.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.951+0000 7f15fe7cb700 1 -- 172.21.15.146:0/673873572 --> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f15f00d6f00 con 0x7f15dc070d50 2024-09-13T18:57:08.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.951+0000 7f15edffb700 1 -- 172.21.15.146:0/673873572 <== osd.2 v2:172.21.15.146:6800/3571336781 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f15f00d6f00 con 0x7f15dc070d50 2024-09-13T18:57:08.964 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:57:08.964 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:57:08.964 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:57:08.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.951+0000 7f15fe7cb700 1 -- 172.21.15.146:0/673873572 >> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] conn(0x7f15dc070d50 msgr2=0x7f15dc0731f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.951+0000 7f15fe7cb700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.146:6800/3571336781,v1:172.21.15.146:6801/3571336781] conn(0x7f15dc070d50 0x7f15dc0731f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.951+0000 7f15fe7cb700 1 -- 172.21.15.146:0/673873572 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f15dc066730 msgr2=0x7f15dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.951+0000 7f15fe7cb700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f15dc066730 0x7f15dc068bf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f15f4005c60 tx=0x7f15f4007040 comp rx=0 tx=0).stop 2024-09-13T18:57:08.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.951+0000 7f15fe7cb700 1 -- 172.21.15.146:0/673873572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15f00ceb90 msgr2=0x7f15f00d5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:08.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.951+0000 7f15fe7cb700 1 --2- 172.21.15.146:0/673873572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15f00ceb90 0x7f15f00d5340 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f15f8068650 tx=0x7f15f80739e0 comp rx=0 tx=0).stop 2024-09-13T18:57:08.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.951+0000 7f15fdfca700 1 -- 172.21.15.146:0/673873572 reap_dead start 2024-09-13T18:57:08.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.951+0000 7f15fe7cb700 1 -- 172.21.15.146:0/673873572 shutdown_connections 2024-09-13T18:57:08.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.951+0000 7f15fe7cb700 1 -- 172.21.15.146:0/673873572 >> 172.21.15.146:0/673873572 conn(0x7f15f0054520 msgr2=0x7f15f0050af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:08.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.951+0000 7f15fe7cb700 1 -- 172.21.15.146:0/673873572 shutdown_connections 2024-09-13T18:57:08.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:08.951+0000 7f15fe7cb700 1 -- 172.21.15.146:0/673873572 wait complete. 2024-09-13T18:57:08.980 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T18:57:09.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7adfd2700 1 Processor -- start 2024-09-13T18:57:09.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7adfd2700 1 -- start start 2024-09-13T18:57:09.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7adfd2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa7a815d960 0x7fa7a80c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7adfd2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a80c5e50 con 0x7fa7a80ca500 2024-09-13T18:57:09.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7adfd2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a80cb480 con 0x7fa7a81539e0 2024-09-13T18:57:09.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7adfd2700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a80cc3a0 con 0x7fa7a815d960 2024-09-13T18:57:09.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7ad7d1700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa7a80ca500 0x7fa7a80cf580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59164/0 (socket says 172.21.15.146:59164) 2024-09-13T18:57:09.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7ad7d1700 1 -- 172.21.15.146:0/3094134606 learned_addr learned my addr 172.21.15.146:0/3094134606 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:57:09.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7a7fff700 1 --2- 172.21.15.146:0/3094134606 >> v2:172.21.15.154:3300/0 conn(0x7fa7a815d960 0x7fa7a80c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7a77fe700 1 -- 172.21.15.146:0/3094134606 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183751826 0 0) 0x7fa7a0001110 con 0x7fa7a80ca500 2024-09-13T18:57:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7a77fe700 1 -- 172.21.15.146:0/3094134606 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7940035d0 con 0x7fa7a80ca500 2024-09-13T18:57:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7a7fff700 1 -- 172.21.15.146:0/3094134606 >> v1:172.21.15.154:6789/0 conn(0x7fa7a81539e0 legacy=0x7fa7a80b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7a7fff700 1 -- 172.21.15.146:0/3094134606 >> v1:172.21.15.146:6789/0 conn(0x7fa7a80ca500 legacy=0x7fa7a80cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7a7fff700 1 -- 172.21.15.146:0/3094134606 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a80cb720 con 0x7fa7a815d960 2024-09-13T18:57:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7a7fff700 1 --2- 172.21.15.146:0/3094134606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a815d960 0x7fa7a80c9fc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa798009880 tx=0x7fa798009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=144d3b3fe3a536fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7a77fe700 1 -- 172.21.15.146:0/3094134606 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798011830 con 0x7fa7a815d960 2024-09-13T18:57:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7a77fe700 1 -- 172.21.15.146:0/3094134606 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa798011990 con 0x7fa7a815d960 2024-09-13T18:57:09.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7a77fe700 1 -- 172.21.15.146:0/3094134606 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798011d20 con 0x7fa7a815d960 2024-09-13T18:57:09.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7adfd2700 1 -- 172.21.15.146:0/3094134606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a815d960 msgr2=0x7fa7a80c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7adfd2700 1 --2- 172.21.15.146:0/3094134606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a815d960 0x7fa7a80c9fc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa798009880 tx=0x7fa798009b90 comp rx=0 tx=0).stop 2024-09-13T18:57:09.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7adfd2700 1 -- 172.21.15.146:0/3094134606 shutdown_connections 2024-09-13T18:57:09.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7adfd2700 1 --2- 172.21.15.146:0/3094134606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a815d960 0x7fa7a80c9fc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7adfd2700 1 -- 172.21.15.146:0/3094134606 >> 172.21.15.146:0/3094134606 conn(0x7fa7a8071ec0 msgr2=0x7fa7a80722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:09.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7adfd2700 1 -- 172.21.15.146:0/3094134606 shutdown_connections 2024-09-13T18:57:09.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.075+0000 7fa7adfd2700 1 -- 172.21.15.146:0/3094134606 wait complete. 2024-09-13T18:57:09.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7adfd2700 1 Processor -- start 2024-09-13T18:57:09.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7adfd2700 1 -- start start 2024-09-13T18:57:09.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7adfd2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7a80ca500 0x7fa7a80c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7adfd2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7a81539e0 0x7fa7a814b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7adfd2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a815d960 0x7fa7a80c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7adfd2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a81f0080 con 0x7fa7a80ca500 2024-09-13T18:57:09.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7adfd2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a81ef500 con 0x7fa7a815d960 2024-09-13T18:57:09.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7adfd2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a81ef7e0 con 0x7fa7a81539e0 2024-09-13T18:57:09.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7acfd0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7a80ca500 0x7fa7a80c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7acfd0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7a80ca500 0x7fa7a80c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57702/0 (socket says 172.21.15.146:57702) 2024-09-13T18:57:09.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7acfd0700 1 -- 172.21.15.146:0/1319878302 learned_addr learned my addr 172.21.15.146:0/1319878302 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:09.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7ad7d1700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a815d960 0x7fa7a80c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7a7fff700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7a81539e0 0x7fa7a814b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7acfd0700 1 -- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7a81539e0 msgr2=0x7fa7a814b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7acfd0700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7a81539e0 0x7fa7a814b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7acfd0700 1 -- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a815d960 msgr2=0x7fa7a80c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7acfd0700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a815d960 0x7fa7a80c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7acfd0700 1 -- 172.21.15.146:0/1319878302 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7980094b0 con 0x7fa7a80ca500 2024-09-13T18:57:09.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7a7fff700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7a81539e0 0x7fa7a814b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:57:09.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7ad7d1700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a815d960 0x7fa7a80c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:57:09.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7acfd0700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7a80ca500 0x7fa7a80c9020 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa790001ed0 tx=0x7fa79000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7a57fa700 1 -- 172.21.15.146:0/1319878302 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79000ade0 con 0x7fa7a80ca500 2024-09-13T18:57:09.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7a57fa700 1 -- 172.21.15.146:0/1319878302 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7900020d0 con 0x7fa7a80ca500 2024-09-13T18:57:09.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7adfd2700 1 -- 172.21.15.146:0/1319878302 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a81efac0 con 0x7fa7a80ca500 2024-09-13T18:57:09.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7a57fa700 1 -- 172.21.15.146:0/1319878302 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790003c40 con 0x7fa7a80ca500 2024-09-13T18:57:09.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.079+0000 7fa7adfd2700 1 -- 172.21.15.146:0/1319878302 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a80c7530 con 0x7fa7a80ca500 2024-09-13T18:57:09.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.083+0000 7fa7adfd2700 1 -- 172.21.15.146:0/1319878302 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a80c7720 con 0x7fa7a80ca500 2024-09-13T18:57:09.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.083+0000 7fa7a57fa700 1 -- 172.21.15.146:0/1319878302 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa7900050d0 con 0x7fa7a80ca500 2024-09-13T18:57:09.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.083+0000 7fa7a57fa700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa788066760 0x7fa788068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.083+0000 7fa7a57fa700 1 -- 172.21.15.146:0/1319878302 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fa790084b50 con 0x7fa7a80ca500 2024-09-13T18:57:09.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.083+0000 7fa7a7fff700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa788066760 0x7fa788068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.087+0000 7fa7a7fff700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa788066760 0x7fa788068c20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa7980090b0 tx=0x7fa798008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.091+0000 7fa7a57fa700 1 -- 172.21.15.146:0/1319878302 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa790051630 con 0x7fa7a80ca500 2024-09-13T18:57:09.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.323+0000 7fa7adfd2700 1 -- 172.21.15.146:0/1319878302 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7a80cbfc0 con 0x7fa788066760 2024-09-13T18:57:09.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.327+0000 7fa7a57fa700 1 -- 172.21.15.146:0/1319878302 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa798011750 con 0x7fa788066760 2024-09-13T18:57:09.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.331+0000 7fa7adfd2700 1 -- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa788066760 msgr2=0x7fa788068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.331+0000 7fa7adfd2700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa788066760 0x7fa788068c20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa7980090b0 tx=0x7fa798008030 comp rx=0 tx=0).stop 2024-09-13T18:57:09.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.331+0000 7fa7adfd2700 1 -- 172.21.15.146:0/1319878302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7a80ca500 msgr2=0x7fa7a80c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.331+0000 7fa7adfd2700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7a80ca500 0x7fa7a80c9020 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa790001ed0 tx=0x7fa79000a340 comp rx=0 tx=0).stop 2024-09-13T18:57:09.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.331+0000 7fa7adfd2700 1 -- 172.21.15.146:0/1319878302 shutdown_connections 2024-09-13T18:57:09.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.331+0000 7fa7adfd2700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fa788066760 0x7fa788068c20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.331+0000 7fa7adfd2700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a815d960 0x7fa7a80c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.331+0000 7fa7adfd2700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7a81539e0 0x7fa7a814b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.331+0000 7fa7adfd2700 1 --2- 172.21.15.146:0/1319878302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7a80ca500 0x7fa7a80c9020 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.331+0000 7fa7adfd2700 1 -- 172.21.15.146:0/1319878302 >> 172.21.15.146:0/1319878302 conn(0x7fa7a8071ec0 msgr2=0x7fa7a80b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:09.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.331+0000 7fa7adfd2700 1 -- 172.21.15.146:0/1319878302 shutdown_connections 2024-09-13T18:57:09.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.331+0000 7fa7adfd2700 1 -- 172.21.15.146:0/1319878302 wait complete. 2024-09-13T18:57:09.346 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T18:57:09.370 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T18:57:09.370 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:57:09.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.515+0000 7f8fe9184700 1 Processor -- start 2024-09-13T18:57:09.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.515+0000 7f8fe9184700 1 -- start start 2024-09-13T18:57:09.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe9184700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8fe4153b50 0x7f8fe4159f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe9184700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8fe4066090 0x7f8fe41640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe9184700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe406e230 con 0x7f8fe406d350 2024-09-13T18:57:09.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe9184700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe406ea40 con 0x7f8fe4153b50 2024-09-13T18:57:09.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe9184700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe40c6090 con 0x7f8fe4066090 2024-09-13T18:57:09.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe2d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8fe4153b50 0x7f8fe4159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe2d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8fe4153b50 0x7f8fe4159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57712/0 (socket says 172.21.15.146:57712) 2024-09-13T18:57:09.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe2d9d700 1 -- 172.21.15.146:0/3686235397 learned_addr learned my addr 172.21.15.146:0/3686235397 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:09.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe359e700 1 --2- 172.21.15.146:0/3686235397 >> v2:172.21.15.154:3300/0 conn(0x7f8fe4066090 0x7f8fe41640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe2d9d700 1 -- 172.21.15.146:0/3686235397 >> v2:172.21.15.154:3300/0 conn(0x7f8fe4066090 msgr2=0x7f8fe41640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:09.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe2d9d700 1 --2- 172.21.15.146:0/3686235397 >> v2:172.21.15.154:3300/0 conn(0x7f8fe4066090 0x7f8fe41640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe2d9d700 1 -- 172.21.15.146:0/3686235397 >> v1:172.21.15.146:6789/0 conn(0x7f8fe406d350 legacy=0x7f8fe40c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe2d9d700 1 -- 172.21.15.146:0/3686235397 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe415db30 con 0x7f8fe4153b50 2024-09-13T18:57:09.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe359e700 1 --2- 172.21.15.146:0/3686235397 >> v2:172.21.15.154:3300/0 conn(0x7f8fe4066090 0x7f8fe41640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:57:09.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe2d9d700 1 --2- 172.21.15.146:0/3686235397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe4153b50 0x7f8fe4159f30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4066700 tx=0x7f8fd4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3063a130f8496019 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe1d9b700 1 -- 172.21.15.146:0/3686235397 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd400b8a0 con 0x7f8fe4153b50 2024-09-13T18:57:09.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe1d9b700 1 -- 172.21.15.146:0/3686235397 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd400ba00 con 0x7f8fe4153b50 2024-09-13T18:57:09.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe1d9b700 1 -- 172.21.15.146:0/3686235397 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd400bd90 con 0x7f8fe4153b50 2024-09-13T18:57:09.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe9184700 1 -- 172.21.15.146:0/3686235397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe4153b50 msgr2=0x7f8fe4159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe9184700 1 --2- 172.21.15.146:0/3686235397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe4153b50 0x7f8fe4159f30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4066700 tx=0x7f8fd4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe9184700 1 -- 172.21.15.146:0/3686235397 shutdown_connections 2024-09-13T18:57:09.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe9184700 1 --2- 172.21.15.146:0/3686235397 >> v2:172.21.15.154:3300/0 conn(0x7f8fe4066090 0x7f8fe41640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe9184700 1 --2- 172.21.15.146:0/3686235397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe4153b50 0x7f8fe4159f30 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe9184700 1 -- 172.21.15.146:0/3686235397 >> 172.21.15.146:0/3686235397 conn(0x7f8fe4066920 msgr2=0x7f8fe406acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:09.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe9184700 1 -- 172.21.15.146:0/3686235397 shutdown_connections 2024-09-13T18:57:09.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.519+0000 7f8fe9184700 1 -- 172.21.15.146:0/3686235397 wait complete. 2024-09-13T18:57:09.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe9184700 1 Processor -- start 2024-09-13T18:57:09.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe9184700 1 -- start start 2024-09-13T18:57:09.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe9184700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fe4066090 0x7f8fe40c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe9184700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe406d350 0x7f8fe415c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe9184700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fe4153b50 0x7f8fe415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe9184700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe40c3750 con 0x7f8fe406d350 2024-09-13T18:57:09.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe9184700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe415c190 con 0x7f8fe4066090 2024-09-13T18:57:09.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe9184700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe415b160 con 0x7f8fe4153b50 2024-09-13T18:57:09.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe406d350 0x7f8fe415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe406d350 0x7f8fe415c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57720/0 (socket says 172.21.15.146:57720) 2024-09-13T18:57:09.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe259c700 1 -- 172.21.15.146:0/2818226861 learned_addr learned my addr 172.21.15.146:0/2818226861 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:09.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe2d9d700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fe4066090 0x7f8fe40c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe359e700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fe4153b50 0x7f8fe415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe259c700 1 -- 172.21.15.146:0/2818226861 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fe4153b50 msgr2=0x7f8fe415b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe259c700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fe4153b50 0x7f8fe415b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe259c700 1 -- 172.21.15.146:0/2818226861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fe4066090 msgr2=0x7f8fe40c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe259c700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fe4066090 0x7f8fe40c38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe259c700 1 -- 172.21.15.146:0/2818226861 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd4009570 con 0x7f8fe406d350 2024-09-13T18:57:09.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe359e700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fe4153b50 0x7f8fe415b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:57:09.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe259c700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe406d350 0x7f8fe415c470 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800c530 tx=0x7f8fd800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fcb7fe700 1 -- 172.21.15.146:0/2818226861 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80052d0 con 0x7f8fe406d350 2024-09-13T18:57:09.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fcb7fe700 1 -- 172.21.15.146:0/2818226861 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd8005430 con 0x7f8fe406d350 2024-09-13T18:57:09.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe9184700 1 -- 172.21.15.146:0/2818226861 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe41e59e0 con 0x7f8fe406d350 2024-09-13T18:57:09.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fcb7fe700 1 -- 172.21.15.146:0/2818226861 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80055a0 con 0x7f8fe406d350 2024-09-13T18:57:09.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe9184700 1 -- 172.21.15.146:0/2818226861 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe415ab50 con 0x7f8fe406d350 2024-09-13T18:57:09.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.523+0000 7f8fe9184700 1 -- 172.21.15.146:0/2818226861 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8fe406d890 con 0x7f8fe406d350 2024-09-13T18:57:09.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.527+0000 7f8fcb7fe700 1 -- 172.21.15.146:0/2818226861 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8fd80036a0 con 0x7f8fe406d350 2024-09-13T18:57:09.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.527+0000 7f8fcb7fe700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8fd0066730 0x7f8fd0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.527+0000 7f8fcb7fe700 1 -- 172.21.15.146:0/2818226861 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f8fd8084540 con 0x7f8fe406d350 2024-09-13T18:57:09.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.527+0000 7f8fcb7fe700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] conn(0x7f8fd0070cd0 0x7f8fd0073170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.527+0000 7f8fcb7fe700 1 -- 172.21.15.146:0/2818226861 --> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fd0073840 con 0x7f8fd0070cd0 2024-09-13T18:57:09.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.527+0000 7f8fcb7fe700 1 -- 172.21.15.146:0/2818226861 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd8084990 con 0x7f8fe406d350 2024-09-13T18:57:09.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.527+0000 7f8fe2d9d700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8fd0066730 0x7f8fd0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.527+0000 7f8fe359e700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] conn(0x7f8fd0070cd0 0x7f8fd0073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.527+0000 7f8fe359e700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] conn(0x7f8fd0070cd0 0x7f8fd0073170 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.527+0000 7f8fe2d9d700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8fd0066730 0x7f8fd0068bf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4014040 tx=0x7f8fd4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.527+0000 7f8fcb7fe700 1 -- 172.21.15.146:0/2818226861 <== osd.3 v2:172.21.15.146:6824/3480660705 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f8fd0073840 con 0x7f8fd0070cd0 2024-09-13T18:57:09.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.543+0000 7f8fe9184700 1 -- 172.21.15.146:0/2818226861 --> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8fe406e760 con 0x7f8fd0070cd0 2024-09-13T18:57:09.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.543+0000 7f8fcb7fe700 1 -- 172.21.15.146:0/2818226861 <== osd.3 v2:172.21.15.146:6824/3480660705 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8fdc00e5a0 con 0x7f8fd0070cd0 2024-09-13T18:57:09.557 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:57:09.557 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:57:09.557 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:57:09.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.543+0000 7f8fe9184700 1 -- 172.21.15.146:0/2818226861 >> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] conn(0x7f8fd0070cd0 msgr2=0x7f8fd0073170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.543+0000 7f8fe9184700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.146:6824/3480660705,v1:172.21.15.146:6825/3480660705] conn(0x7f8fd0070cd0 0x7f8fd0073170 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.543+0000 7f8fe9184700 1 -- 172.21.15.146:0/2818226861 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8fd0066730 msgr2=0x7f8fd0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.543+0000 7f8fe9184700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8fd0066730 0x7f8fd0068bf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4014040 tx=0x7f8fd4008030 comp rx=0 tx=0).stop 2024-09-13T18:57:09.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.543+0000 7f8fe9184700 1 -- 172.21.15.146:0/2818226861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe406d350 msgr2=0x7f8fe415c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.543+0000 7f8fe9184700 1 --2- 172.21.15.146:0/2818226861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe406d350 0x7f8fe415c470 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800c530 tx=0x7f8fd800c840 comp rx=0 tx=0).stop 2024-09-13T18:57:09.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.543+0000 7f8fe359e700 1 -- 172.21.15.146:0/2818226861 reap_dead start 2024-09-13T18:57:09.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.543+0000 7f8fe9184700 1 -- 172.21.15.146:0/2818226861 shutdown_connections 2024-09-13T18:57:09.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.543+0000 7f8fe9184700 1 -- 172.21.15.146:0/2818226861 >> 172.21.15.146:0/2818226861 conn(0x7f8fe4066920 msgr2=0x7f8fe41519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:09.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.543+0000 7f8fe9184700 1 -- 172.21.15.146:0/2818226861 shutdown_connections 2024-09-13T18:57:09.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.543+0000 7f8fe9184700 1 -- 172.21.15.146:0/2818226861 wait complete. 2024-09-13T18:57:09.568 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T18:57:09.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.659+0000 7f65bb59e700 1 Processor -- start 2024-09-13T18:57:09.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.659+0000 7f65bb59e700 1 -- start start 2024-09-13T18:57:09.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65bb59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f65bc0c6980 0x7f65bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65bb59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f65bc06ef40 0x7f65bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65bb59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65bc0c7920 con 0x7f65bc1538d0 2024-09-13T18:57:09.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65bb59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc0c1720 con 0x7f65bc06ef40 2024-09-13T18:57:09.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65bb59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc0c8570 con 0x7f65bc0c6980 2024-09-13T18:57:09.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65ba59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f65bc1538d0 0x7f65bc0c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59190/0 (socket says 172.21.15.146:59190) 2024-09-13T18:57:09.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65ba59c700 1 -- 172.21.15.146:0/2486124459 learned_addr learned my addr 172.21.15.146:0/2486124459 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:57:09.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65bad9d700 1 --2- 172.21.15.146:0/2486124459 >> v2:172.21.15.146:3300/0 conn(0x7f65bc06ef40 0x7f65bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b9d9b700 1 --2- 172.21.15.146:0/2486124459 >> v2:172.21.15.154:3300/0 conn(0x7f65bc0c6980 0x7f65bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662060095 0 0) 0x7f65bc0c7920 con 0x7f65bc1538d0 2024-09-13T18:57:09.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65a40035d0 con 0x7f65bc1538d0 2024-09-13T18:57:09.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2258894810 0 0) 0x7f65a40035d0 con 0x7f65bc1538d0 2024-09-13T18:57:09.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 >> v2:172.21.15.154:3300/0 conn(0x7f65bc0c6980 msgr2=0x7f65bc06d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:09.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 --2- 172.21.15.146:0/2486124459 >> v2:172.21.15.154:3300/0 conn(0x7f65bc0c6980 0x7f65bc06d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 >> v2:172.21.15.146:3300/0 conn(0x7f65bc06ef40 msgr2=0x7f65bc06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:09.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 --2- 172.21.15.146:0/2486124459 >> v2:172.21.15.146:3300/0 conn(0x7f65bc06ef40 0x7f65bc06d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc0c7030 con 0x7f65bc1538d0 2024-09-13T18:57:09.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f65ac001670 con 0x7f65bc1538d0 2024-09-13T18:57:09.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 >> v1:172.21.15.146:6789/0 conn(0x7f65bc1538d0 legacy=0x7f65bc0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 --2- 172.21.15.146:0/2486124459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65a4003ac0 0x7f65a4003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 --2- 172.21.15.146:0/2486124459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65a4004870 0x7f65a4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 --2- 172.21.15.146:0/2486124459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a4005230 0x7f65a400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc0c8570 con 0x7f65a4003ac0 2024-09-13T18:57:09.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a400bfd0 con 0x7f65a4004870 2024-09-13T18:57:09.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a400c190 con 0x7f65a4005230 2024-09-13T18:57:09.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65bad9d700 1 --2- 172.21.15.146:0/2486124459 >> v2:172.21.15.146:3300/0 conn(0x7f65bc06ef40 0x7f65bc06d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:57:09.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65ba59c700 1 --2- 172.21.15.146:0/2486124459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65a4003ac0 0x7f65a4003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b9d9b700 1 --2- 172.21.15.146:0/2486124459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65a4004870 0x7f65a4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65ba59c700 1 -- 172.21.15.146:0/2486124459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a4005230 msgr2=0x7f65a400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:09.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65ba59c700 1 --2- 172.21.15.146:0/2486124459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a4005230 0x7f65a400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65ba59c700 1 -- 172.21.15.146:0/2486124459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65a4004870 msgr2=0x7f65a4004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:09.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65ba59c700 1 --2- 172.21.15.146:0/2486124459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65a4004870 0x7f65a4004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65bad9d700 1 -- 172.21.15.146:0/2486124459 reap_dead start 2024-09-13T18:57:09.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65ba59c700 1 -- 172.21.15.146:0/2486124459 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a40035d0 con 0x7f65a4003ac0 2024-09-13T18:57:09.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b9d9b700 1 --2- 172.21.15.146:0/2486124459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65a4004870 0x7f65a4004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:57:09.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65ba59c700 1 --2- 172.21.15.146:0/2486124459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65a4003ac0 0x7f65a4003ee0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00c2a0 tx=0x7f65a40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51da39c1d4a452a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac007c30 con 0x7f65a4003ac0 2024-09-13T18:57:09.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac00cdc0 con 0x7f65a4003ac0 2024-09-13T18:57:09.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.663+0000 7f65b959a700 1 -- 172.21.15.146:0/2486124459 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac005bc0 con 0x7f65a4003ac0 2024-09-13T18:57:09.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.667+0000 7f65bb59e700 1 -- 172.21.15.146:0/2486124459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65a4003ac0 msgr2=0x7f65a4003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.667+0000 7f65bb59e700 1 --2- 172.21.15.146:0/2486124459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65a4003ac0 0x7f65a4003ee0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00c2a0 tx=0x7f65a40058a0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.667+0000 7f65bb59e700 1 -- 172.21.15.146:0/2486124459 shutdown_connections 2024-09-13T18:57:09.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.667+0000 7f65bb59e700 1 --2- 172.21.15.146:0/2486124459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65a4003ac0 0x7f65a4003ee0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.667+0000 7f65bb59e700 1 -- 172.21.15.146:0/2486124459 >> 172.21.15.146:0/2486124459 conn(0x7f65bc06b220 msgr2=0x7f65bc06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:09.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.667+0000 7f65bb59e700 1 -- 172.21.15.146:0/2486124459 shutdown_connections 2024-09-13T18:57:09.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.667+0000 7f65bb59e700 1 -- 172.21.15.146:0/2486124459 wait complete. 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.667+0000 7f65bb59e700 1 Processor -- start 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.667+0000 7f65bb59e700 1 -- start start 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bb59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a4003ac0 0x7f65bc0c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bb59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65bc0c9a30 0x7f65bc1525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bb59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65bc1eecb0 0x7f65bc1ef130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bb59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc0c5db0 con 0x7f65bc1eecb0 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bb59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc0c43d0 con 0x7f65bc0c9a30 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bb59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc1ef670 con 0x7f65a4003ac0 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65bc1eecb0 0x7f65bc1ef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65bc1eecb0 0x7f65bc1ef130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57742/0 (socket says 172.21.15.146:57742) 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bad9d700 1 -- 172.21.15.146:0/3324281346 learned_addr learned my addr 172.21.15.146:0/3324281346 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65b9d9b700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65bc0c9a30 0x7f65bc1525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65ba59c700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a4003ac0 0x7f65bc0c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bad9d700 1 -- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a4003ac0 msgr2=0x7f65bc0c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bad9d700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a4003ac0 0x7f65bc0c3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bad9d700 1 -- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65bc0c9a30 msgr2=0x7f65bc1525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bad9d700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65bc0c9a30 0x7f65bc1525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bad9d700 1 -- 172.21.15.146:0/3324281346 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc0c48b0 con 0x7f65bc1eecb0 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bad9d700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65bc1eecb0 0x7f65bc1ef130 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0c9790 tx=0x7f65b400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65aaffd700 1 -- 172.21.15.146:0/3324281346 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b4010040 con 0x7f65bc1eecb0 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65aaffd700 1 -- 172.21.15.146:0/3324281346 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b400bd00 con 0x7f65bc1eecb0 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bb59e700 1 -- 172.21.15.146:0/3324281346 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc0c45d0 con 0x7f65bc1eecb0 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65aaffd700 1 -- 172.21.15.146:0/3324281346 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b4003c60 con 0x7f65bc1eecb0 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.671+0000 7f65bb59e700 1 -- 172.21.15.146:0/3324281346 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc0c56f0 con 0x7f65bc1eecb0 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.675+0000 7f65bb59e700 1 -- 172.21.15.146:0/3324281346 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65bc0c5ef0 con 0x7f65bc1eecb0 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.675+0000 7f65aaffd700 1 -- 172.21.15.146:0/3324281346 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f65b4004030 con 0x7f65bc1eecb0 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.675+0000 7f65aaffd700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f659c066760 0x7f659c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.675+0000 7f65aaffd700 1 -- 172.21.15.146:0/3324281346 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f65b4084580 con 0x7f65bc1eecb0 2024-09-13T18:57:09.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.675+0000 7f65ba59c700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f659c066760 0x7f659c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.675+0000 7f65ba59c700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f659c066760 0x7f659c068c20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f65ac000c00 tx=0x7f65ac007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.679+0000 7f65aaffd700 1 -- 172.21.15.146:0/3324281346 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f65bc0c5ef0 con 0x7f65bc1eecb0 2024-09-13T18:57:09.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.839+0000 7f65bb59e700 1 -- 172.21.15.146:0/3324281346 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f65bc0c75b0 con 0x7f659c066760 2024-09-13T18:57:09.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.839+0000 7f65aaffd700 1 -- 172.21.15.146:0/3324281346 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f65ac001430 con 0x7f659c066760 2024-09-13T18:57:09.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.843+0000 7f65bb59e700 1 -- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f659c066760 msgr2=0x7f659c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.843+0000 7f65bb59e700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f659c066760 0x7f659c068c20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f65ac000c00 tx=0x7f65ac007000 comp rx=0 tx=0).stop 2024-09-13T18:57:09.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.843+0000 7f65bb59e700 1 -- 172.21.15.146:0/3324281346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65bc1eecb0 msgr2=0x7f65bc1ef130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.843+0000 7f65bb59e700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65bc1eecb0 0x7f65bc1ef130 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0c9790 tx=0x7f65b400b390 comp rx=0 tx=0).stop 2024-09-13T18:57:09.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.843+0000 7f65bb59e700 1 -- 172.21.15.146:0/3324281346 shutdown_connections 2024-09-13T18:57:09.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.843+0000 7f65bb59e700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f659c066760 0x7f659c068c20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.843+0000 7f65bb59e700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65bc1eecb0 0x7f65bc1ef130 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.843+0000 7f65bb59e700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65bc0c9a30 0x7f65bc1525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.843+0000 7f65bb59e700 1 --2- 172.21.15.146:0/3324281346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a4003ac0 0x7f65bc0c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.843+0000 7f65bb59e700 1 -- 172.21.15.146:0/3324281346 >> 172.21.15.146:0/3324281346 conn(0x7f65bc06b220 msgr2=0x7f65bc06d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:09.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.843+0000 7f65bb59e700 1 -- 172.21.15.146:0/3324281346 shutdown_connections 2024-09-13T18:57:09.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.843+0000 7f65bb59e700 1 -- 172.21.15.146:0/3324281346 wait complete. 2024-09-13T18:57:09.857 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T18:57:09.867 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T18:57:09.867 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:57:09.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.963+0000 7f6b16422700 1 Processor -- start 2024-09-13T18:57:09.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.963+0000 7f6b16422700 1 -- start start 2024-09-13T18:57:09.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b16422700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b100cf580 0x7f6b100cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b16422700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b100c5e50 con 0x7f6b101539e0 2024-09-13T18:57:09.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b16422700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b100cff30 con 0x7f6b1015d960 2024-09-13T18:57:09.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b16422700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b100cc3a0 con 0x7f6b100cf580 2024-09-13T18:57:09.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b15420700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6b101539e0 0x7f6b100b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59192/0 (socket says 172.21.15.146:59192) 2024-09-13T18:57:09.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b15420700 1 -- 172.21.15.146:0/3516717963 learned_addr learned my addr 172.21.15.146:0/3516717963 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:57:09.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b15c21700 1 --2- 172.21.15.146:0/3516717963 >> v2:172.21.15.154:3300/0 conn(0x7f6b100cf580 0x7f6b100cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470211078 0 0) 0x7f6b0c001110 con 0x7f6b101539e0 2024-09-13T18:57:09.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6afc0035d0 con 0x7f6b101539e0 2024-09-13T18:57:09.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 466405466 0 0) 0x7f6afc0035d0 con 0x7f6b101539e0 2024-09-13T18:57:09.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 >> v2:172.21.15.154:3300/0 conn(0x7f6b100cf580 msgr2=0x7f6b100cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:09.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 --2- 172.21.15.146:0/3516717963 >> v2:172.21.15.154:3300/0 conn(0x7f6b100cf580 0x7f6b100cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 >> v1:172.21.15.154:6789/0 conn(0x7f6b1015d960 legacy=0x7f6b100c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b100cb480 con 0x7f6b101539e0 2024-09-13T18:57:09.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f6b0c001800 con 0x7f6b101539e0 2024-09-13T18:57:09.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 >> v1:172.21.15.146:6789/0 conn(0x7f6b101539e0 legacy=0x7f6b100b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 --2- 172.21.15.146:0/3516717963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6afc003a90 0x7f6afc003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 --2- 172.21.15.146:0/3516717963 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6afc004870 0x7f6afc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 --2- 172.21.15.146:0/3516717963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6afc005230 0x7f6afc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100cc3a0 con 0x7f6afc005230 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc00c160 con 0x7f6afc003a90 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc00c320 con 0x7f6afc004870 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b15c21700 1 --2- 172.21.15.146:0/3516717963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6afc005230 0x7f6afc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b15420700 1 --2- 172.21.15.146:0/3516717963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6afc003a90 0x7f6afc003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b15c21700 1 -- 172.21.15.146:0/3516717963 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6afc004870 msgr2=0x7f6afc004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b15c21700 1 --2- 172.21.15.146:0/3516717963 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6afc004870 0x7f6afc004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b15c21700 1 -- 172.21.15.146:0/3516717963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6afc003a90 msgr2=0x7f6afc003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b15c21700 1 --2- 172.21.15.146:0/3516717963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6afc003a90 0x7f6afc003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b15c21700 1 -- 172.21.15.146:0/3516717963 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc0035d0 con 0x7f6afc005230 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b15c21700 1 -- 172.21.15.146:0/3516717963 reap_dead start 2024-09-13T18:57:09.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b15c21700 1 --2- 172.21.15.146:0/3516717963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6afc005230 0x7f6afc00ba30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009fc0 tx=0x7f6afc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e5a99bae30a4eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08003c40 con 0x7f6afc005230 2024-09-13T18:57:09.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0800f040 con 0x7f6afc005230 2024-09-13T18:57:09.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b07fff700 1 -- 172.21.15.146:0/3516717963 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800ea30 con 0x7f6afc005230 2024-09-13T18:57:09.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b16422700 1 -- 172.21.15.146:0/3516717963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6afc005230 msgr2=0x7f6afc00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b16422700 1 --2- 172.21.15.146:0/3516717963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6afc005230 0x7f6afc00ba30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009fc0 tx=0x7f6afc0058a0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b16422700 1 -- 172.21.15.146:0/3516717963 shutdown_connections 2024-09-13T18:57:09.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b16422700 1 --2- 172.21.15.146:0/3516717963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6afc005230 0x7f6afc00ba30 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b16422700 1 -- 172.21.15.146:0/3516717963 >> 172.21.15.146:0/3516717963 conn(0x7f6b10071ec0 msgr2=0x7f6b100722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:09.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b16422700 1 -- 172.21.15.146:0/3516717963 shutdown_connections 2024-09-13T18:57:09.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.967+0000 7f6b16422700 1 -- 172.21.15.146:0/3516717963 wait complete. 2024-09-13T18:57:09.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b16422700 1 Processor -- start 2024-09-13T18:57:09.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b16422700 1 -- start start 2024-09-13T18:57:09.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b16422700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6afc005230 0x7f6b101460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b16422700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b101f0330 0x7f6b1014d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b16422700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b101f2430 0x7f6b101f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b16422700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1014f550 con 0x7f6b101f2430 2024-09-13T18:57:09.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b16422700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100b1d10 con 0x7f6afc005230 2024-09-13T18:57:09.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b16422700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1014ea20 con 0x7f6b101f0330 2024-09-13T18:57:09.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b15c21700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b101f2430 0x7f6b101f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b15c21700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b101f2430 0x7f6b101f28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57750/0 (socket says 172.21.15.146:57750) 2024-09-13T18:57:09.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b15c21700 1 -- 172.21.15.146:0/4274491687 learned_addr learned my addr 172.21.15.146:0/4274491687 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:09.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b15420700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6afc005230 0x7f6b101460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b14c1f700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b101f0330 0x7f6b1014d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b15c21700 1 -- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b101f0330 msgr2=0x7f6b1014d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b15c21700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b101f0330 0x7f6b1014d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b15c21700 1 -- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6afc005230 msgr2=0x7f6b101460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:09.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b15c21700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6afc005230 0x7f6b101460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:09.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b15c21700 1 -- 172.21.15.146:0/4274491687 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1014efe0 con 0x7f6b101f2430 2024-09-13T18:57:09.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b14c1f700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b101f0330 0x7f6b1014d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:57:09.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b15c21700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b101f2430 0x7f6b101f28b0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e700 tx=0x7f6b08009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b05ffb700 1 -- 172.21.15.146:0/4274491687 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08004430 con 0x7f6b101f2430 2024-09-13T18:57:09.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b16422700 1 -- 172.21.15.146:0/4274491687 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b10146620 con 0x7f6b101f2430 2024-09-13T18:57:09.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b05ffb700 1 -- 172.21.15.146:0/4274491687 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08004590 con 0x7f6b101f2430 2024-09-13T18:57:09.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b05ffb700 1 -- 172.21.15.146:0/4274491687 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08004940 con 0x7f6b101f2430 2024-09-13T18:57:09.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b16422700 1 -- 172.21.15.146:0/4274491687 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b101f31a0 con 0x7f6b101f2430 2024-09-13T18:57:09.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.971+0000 7f6b16422700 1 -- 172.21.15.146:0/4274491687 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6b100cbce0 con 0x7f6b101f2430 2024-09-13T18:57:09.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.975+0000 7f6b05ffb700 1 -- 172.21.15.146:0/4274491687 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b08014480 con 0x7f6b101f2430 2024-09-13T18:57:09.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.975+0000 7f6b05ffb700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6af4066730 0x7f6af4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.975+0000 7f6b05ffb700 1 -- 172.21.15.146:0/4274491687 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f6b08086630 con 0x7f6b101f2430 2024-09-13T18:57:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.975+0000 7f6b05ffb700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] conn(0x7f6af4070cd0 0x7f6af4073170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.975+0000 7f6b05ffb700 1 -- 172.21.15.146:0/4274491687 --> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6af4073840 con 0x7f6af4070cd0 2024-09-13T18:57:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.975+0000 7f6b05ffb700 1 -- 172.21.15.146:0/4274491687 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b08088cf0 con 0x7f6b101f2430 2024-09-13T18:57:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.975+0000 7f6b15420700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6af4066730 0x7f6af4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.975+0000 7f6b14c1f700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] conn(0x7f6af4070cd0 0x7f6af4073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.975+0000 7f6b14c1f700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] conn(0x7f6af4070cd0 0x7f6af4073170 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.975+0000 7f6b15420700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6af4066730 0x7f6af4068bf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c001050 tx=0x7f6b0c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.979+0000 7f6b05ffb700 1 -- 172.21.15.146:0/4274491687 <== osd.4 v2:172.21.15.154:6808/2872450676 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f6af4073840 con 0x7f6af4070cd0 2024-09-13T18:57:10.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.995+0000 7f6b16422700 1 -- 172.21.15.146:0/4274491687 --> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6b100cba00 con 0x7f6af4070cd0 2024-09-13T18:57:10.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.995+0000 7f6b05ffb700 1 -- 172.21.15.146:0/4274491687 <== osd.4 v2:172.21.15.154:6808/2872450676 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6b00001870 con 0x7f6af4070cd0 2024-09-13T18:57:10.009 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:57:10.009 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:57:10.009 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:57:10.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.995+0000 7f6b16422700 1 -- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] conn(0x7f6af4070cd0 msgr2=0x7f6af4073170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.995+0000 7f6b16422700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:6808/2872450676,v1:172.21.15.154:6809/2872450676] conn(0x7f6af4070cd0 0x7f6af4073170 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.995+0000 7f6b16422700 1 -- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6af4066730 msgr2=0x7f6af4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.995+0000 7f6b16422700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6af4066730 0x7f6af4068bf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c001050 tx=0x7f6b0c009040 comp rx=0 tx=0).stop 2024-09-13T18:57:10.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.995+0000 7f6b16422700 1 -- 172.21.15.146:0/4274491687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b101f2430 msgr2=0x7f6b101f28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.995+0000 7f6b16422700 1 --2- 172.21.15.146:0/4274491687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b101f2430 0x7f6b101f28b0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e700 tx=0x7f6b08009750 comp rx=0 tx=0).stop 2024-09-13T18:57:10.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.995+0000 7f6b15c21700 1 -- 172.21.15.146:0/4274491687 reap_dead start 2024-09-13T18:57:10.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.995+0000 7f6b16422700 1 -- 172.21.15.146:0/4274491687 shutdown_connections 2024-09-13T18:57:10.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.995+0000 7f6b16422700 1 -- 172.21.15.146:0/4274491687 >> 172.21.15.146:0/4274491687 conn(0x7f6b10071ec0 msgr2=0x7f6b100b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:10.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.995+0000 7f6b16422700 1 -- 172.21.15.146:0/4274491687 shutdown_connections 2024-09-13T18:57:10.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:09.995+0000 7f6b16422700 1 -- 172.21.15.146:0/4274491687 wait complete. 2024-09-13T18:57:10.021 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T18:57:10.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.187+0000 7f396af5f700 1 Processor -- start 2024-09-13T18:57:10.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.187+0000 7f396af5f700 1 -- start start 2024-09-13T18:57:10.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.191+0000 7f396af5f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f396406e0d0 0x7f39640c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.191+0000 7f396af5f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3964156960 con 0x7f39640c86d0 2024-09-13T18:57:10.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.191+0000 7f396af5f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39640c0aa0 con 0x7f396406e4f0 2024-09-13T18:57:10.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.191+0000 7f396af5f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3964155e20 con 0x7f396406e0d0 2024-09-13T18:57:10.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.191+0000 7f3968cfb700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f39640c86d0 0x7f39640c01f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59202/0 (socket says 172.21.15.146:59202) 2024-09-13T18:57:10.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.191+0000 7f3968cfb700 1 -- 172.21.15.146:0/3905918473 learned_addr learned my addr 172.21.15.146:0/3905918473 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:57:10.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.191+0000 7f3963fff700 1 --2- 172.21.15.146:0/3905918473 >> v2:172.21.15.146:3300/0 conn(0x7f396406e0d0 0x7f39640c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.191+0000 7f3963fff700 1 -- 172.21.15.146:0/3905918473 >> v1:172.21.15.154:6789/0 conn(0x7f396406e4f0 legacy=0x7f39640c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.191+0000 7f3963fff700 1 -- 172.21.15.146:0/3905918473 >> v1:172.21.15.146:6789/0 conn(0x7f39640c86d0 legacy=0x7f39640c01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.191+0000 7f3963fff700 1 -- 172.21.15.146:0/3905918473 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3964156030 con 0x7f396406e0d0 2024-09-13T18:57:10.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.191+0000 7f3963fff700 1 --2- 172.21.15.146:0/3905918473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f396406e0d0 0x7f39640c7b80 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f3958009ab0 tx=0x7f3958009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa868bb52464bd6d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:10.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.195+0000 7f39637fe700 1 -- 172.21.15.146:0/3905918473 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395800b750 con 0x7f396406e0d0 2024-09-13T18:57:10.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.195+0000 7f39637fe700 1 -- 172.21.15.146:0/3905918473 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395800b8b0 con 0x7f396406e0d0 2024-09-13T18:57:10.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.195+0000 7f39637fe700 1 -- 172.21.15.146:0/3905918473 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395800bc60 con 0x7f396406e0d0 2024-09-13T18:57:10.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.195+0000 7f396af5f700 1 -- 172.21.15.146:0/3905918473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f396406e0d0 msgr2=0x7f39640c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.195+0000 7f396af5f700 1 --2- 172.21.15.146:0/3905918473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f396406e0d0 0x7f39640c7b80 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f3958009ab0 tx=0x7f3958009dc0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.195+0000 7f396af5f700 1 -- 172.21.15.146:0/3905918473 shutdown_connections 2024-09-13T18:57:10.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.195+0000 7f396af5f700 1 --2- 172.21.15.146:0/3905918473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f396406e0d0 0x7f39640c7b80 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.195+0000 7f396af5f700 1 -- 172.21.15.146:0/3905918473 >> 172.21.15.146:0/3905918473 conn(0x7f396406f5c0 msgr2=0x7f396406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:10.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.195+0000 7f396af5f700 1 -- 172.21.15.146:0/3905918473 shutdown_connections 2024-09-13T18:57:10.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.195+0000 7f396af5f700 1 -- 172.21.15.146:0/3905918473 wait complete. 2024-09-13T18:57:10.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.195+0000 7f396af5f700 1 Processor -- start 2024-09-13T18:57:10.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.195+0000 7f396af5f700 1 -- start start 2024-09-13T18:57:10.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f396af5f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f396406e0d0 0x7f39640afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f396af5f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f396406e4f0 0x7f39640b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f396af5f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39640c86d0 0x7f39640c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f396af5f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39640ae7d0 con 0x7f396406e0d0 2024-09-13T18:57:10.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f396af5f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396414fe80 con 0x7f396406e4f0 2024-09-13T18:57:10.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f396af5f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39640aff90 con 0x7f39640c86d0 2024-09-13T18:57:10.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f3968cfb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f396406e0d0 0x7f39640afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f3968cfb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f396406e0d0 0x7f39640afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57772/0 (socket says 172.21.15.146:57772) 2024-09-13T18:57:10.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f3968cfb700 1 -- 172.21.15.146:0/4133084317 learned_addr learned my addr 172.21.15.146:0/4133084317 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:10.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f39694fc700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39640c86d0 0x7f39640c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f3968cfb700 1 -- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39640c86d0 msgr2=0x7f39640c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f3968cfb700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39640c86d0 0x7f39640c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f3968cfb700 1 -- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f396406e4f0 msgr2=0x7f39640b0590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:57:10.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f3968cfb700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f396406e4f0 0x7f39640b0590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f3968cfb700 1 -- 172.21.15.146:0/4133084317 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39580094b0 con 0x7f396406e0d0 2024-09-13T18:57:10.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f39694fc700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39640c86d0 0x7f39640c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:57:10.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f3968cfb700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f396406e0d0 0x7f39640afa50 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f395c0020b0 tx=0x7f395c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:10.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f39617fa700 1 -- 172.21.15.146:0/4133084317 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c00adc0 con 0x7f396406e0d0 2024-09-13T18:57:10.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f39617fa700 1 -- 172.21.15.146:0/4133084317 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c0070d0 con 0x7f396406e0d0 2024-09-13T18:57:10.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f396af5f700 1 -- 172.21.15.146:0/4133084317 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3964150070 con 0x7f396406e0d0 2024-09-13T18:57:10.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f39617fa700 1 -- 172.21.15.146:0/4133084317 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c007480 con 0x7f396406e0d0 2024-09-13T18:57:10.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f396af5f700 1 -- 172.21.15.146:0/4133084317 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3964150370 con 0x7f396406e0d0 2024-09-13T18:57:10.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f396af5f700 1 -- 172.21.15.146:0/4133084317 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396406cdc0 con 0x7f396406e0d0 2024-09-13T18:57:10.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f39617fa700 1 -- 172.21.15.146:0/4133084317 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f395c0075e0 con 0x7f396406e0d0 2024-09-13T18:57:10.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f39617fa700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3948066730 0x7f3948068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f39617fa700 1 -- 172.21.15.146:0/4133084317 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f395c084970 con 0x7f396406e0d0 2024-09-13T18:57:10.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.199+0000 7f3963fff700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3948066730 0x7f3948068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.203+0000 7f3963fff700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3948066730 0x7f3948068bf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f39640c8680 tx=0x7f3958008000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:10.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.207+0000 7f39617fa700 1 -- 172.21.15.146:0/4133084317 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f395c051450 con 0x7f396406e0d0 2024-09-13T18:57:10.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.595+0000 7f396af5f700 1 -- 172.21.15.146:0/4133084317 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f39640715a0 con 0x7f3948066730 2024-09-13T18:57:10.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.603+0000 7f39617fa700 1 -- 172.21.15.146:0/4133084317 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f395801f070 con 0x7f3948066730 2024-09-13T18:57:10.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.611+0000 7f396af5f700 1 -- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3948066730 msgr2=0x7f3948068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.611+0000 7f396af5f700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3948066730 0x7f3948068bf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f39640c8680 tx=0x7f3958008000 comp rx=0 tx=0).stop 2024-09-13T18:57:10.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.611+0000 7f396af5f700 1 -- 172.21.15.146:0/4133084317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f396406e0d0 msgr2=0x7f39640afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.611+0000 7f396af5f700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f396406e0d0 0x7f39640afa50 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f395c0020b0 tx=0x7f395c00a340 comp rx=0 tx=0).stop 2024-09-13T18:57:10.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.611+0000 7f396af5f700 1 -- 172.21.15.146:0/4133084317 shutdown_connections 2024-09-13T18:57:10.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.611+0000 7f396af5f700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f3948066730 0x7f3948068bf0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.611+0000 7f396af5f700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39640c86d0 0x7f39640c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.611+0000 7f396af5f700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f396406e4f0 0x7f39640b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.611+0000 7f396af5f700 1 --2- 172.21.15.146:0/4133084317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f396406e0d0 0x7f39640afa50 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.611+0000 7f396af5f700 1 -- 172.21.15.146:0/4133084317 >> 172.21.15.146:0/4133084317 conn(0x7f396406f5c0 msgr2=0x7f3964154280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:10.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.611+0000 7f396af5f700 1 -- 172.21.15.146:0/4133084317 shutdown_connections 2024-09-13T18:57:10.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.611+0000 7f396af5f700 1 -- 172.21.15.146:0/4133084317 wait complete. 2024-09-13T18:57:10.626 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T18:57:10.655 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T18:57:10.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:57:10.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.767+0000 7f70a70f6700 1 Processor -- start 2024-09-13T18:57:10.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.767+0000 7f70a70f6700 1 -- start start 2024-09-13T18:57:10.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a70f6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70a00cc490 0x7f70a00c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a70f6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f70a00c79d0 0x7f70a00c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.785 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a70f6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a006f8c0 con 0x7f70a00cc870 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a70f6700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a00c5860 con 0x7f70a00c79d0 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a70f6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a006ea40 con 0x7f70a00cc490 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a68f5700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f70a00c79d0 0x7f70a00c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a68f5700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f70a00c79d0 0x7f70a00c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57776/0 (socket says 172.21.15.146:57776) 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a68f5700 1 -- 172.21.15.146:0/261400343 learned_addr learned my addr 172.21.15.146:0/261400343 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a60f4700 1 --2- 172.21.15.146:0/261400343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70a00cc490 0x7f70a00c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a68f5700 1 -- 172.21.15.146:0/261400343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70a00cc490 msgr2=0x7f70a00c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a68f5700 1 --2- 172.21.15.146:0/261400343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70a00cc490 0x7f70a00c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a68f5700 1 -- 172.21.15.146:0/261400343 >> v1:172.21.15.154:6789/0 conn(0x7f70a00cc870 legacy=0x7f70a006e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a68f5700 1 -- 172.21.15.146:0/261400343 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a006fb80 con 0x7f70a00c79d0 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a68f5700 1 --2- 172.21.15.146:0/261400343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00c79d0 0x7f70a00c5cb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7098009e50 tx=0x7f709800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9913f98578cf8e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a50f2700 1 -- 172.21.15.146:0/261400343 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709800e070 con 0x7f70a00c79d0 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a50f2700 1 -- 172.21.15.146:0/261400343 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709800b9c0 con 0x7f70a00c79d0 2024-09-13T18:57:10.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a50f2700 1 -- 172.21.15.146:0/261400343 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709800bd70 con 0x7f70a00c79d0 2024-09-13T18:57:10.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a70f6700 1 -- 172.21.15.146:0/261400343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00c79d0 msgr2=0x7f70a00c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a70f6700 1 --2- 172.21.15.146:0/261400343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00c79d0 0x7f70a00c5cb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7098009e50 tx=0x7f709800b330 comp rx=0 tx=0).stop 2024-09-13T18:57:10.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a70f6700 1 -- 172.21.15.146:0/261400343 shutdown_connections 2024-09-13T18:57:10.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a70f6700 1 --2- 172.21.15.146:0/261400343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00c79d0 0x7f70a00c5cb0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a70f6700 1 --2- 172.21.15.146:0/261400343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70a00cc490 0x7f70a00c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a70f6700 1 -- 172.21.15.146:0/261400343 >> 172.21.15.146:0/261400343 conn(0x7f70a0071ed0 msgr2=0x7f70a00722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:10.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a70f6700 1 -- 172.21.15.146:0/261400343 shutdown_connections 2024-09-13T18:57:10.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.771+0000 7f70a70f6700 1 -- 172.21.15.146:0/261400343 wait complete. 2024-09-13T18:57:10.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a70f6700 1 Processor -- start 2024-09-13T18:57:10.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a70f6700 1 -- start start 2024-09-13T18:57:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a70f6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70a00c79d0 0x7f70a01ed0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a70f6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00cc490 0x7f70a00c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a70f6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70a00cc870 0x7f70a01dd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a70f6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01dd590 con 0x7f70a00cc490 2024-09-13T18:57:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a70f6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a014a9a0 con 0x7f70a00c79d0 2024-09-13T18:57:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a70f6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01de590 con 0x7f70a00cc870 2024-09-13T18:57:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a58f3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00cc490 0x7f70a00c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a68f5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70a00cc870 0x7f70a01dd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a60f4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70a00c79d0 0x7f70a01ed0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a58f3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00cc490 0x7f70a00c4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57784/0 (socket says 172.21.15.146:57784) 2024-09-13T18:57:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a58f3700 1 -- 172.21.15.146:0/1896650163 learned_addr learned my addr 172.21.15.146:0/1896650163 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:10.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a58f3700 1 -- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70a00cc870 msgr2=0x7f70a01dd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a58f3700 1 --2- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70a00cc870 0x7f70a01dd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a58f3700 1 -- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70a00c79d0 msgr2=0x7f70a01ed0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a58f3700 1 --2- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70a00c79d0 0x7f70a01ed0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a58f3700 1 -- 172.21.15.146:0/1896650163 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7098009b30 con 0x7f70a00cc490 2024-09-13T18:57:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a58f3700 1 --2- 172.21.15.146:0/1896650163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00cc490 0x7f70a00c4a90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f709000c4d0 tx=0x7f709000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f7096ffd700 1 -- 172.21.15.146:0/1896650163 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70900052d0 con 0x7f70a00cc490 2024-09-13T18:57:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f7096ffd700 1 -- 172.21.15.146:0/1896650163 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7090005430 con 0x7f70a00cc490 2024-09-13T18:57:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f70a70f6700 1 -- 172.21.15.146:0/1896650163 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a01de850 con 0x7f70a00cc490 2024-09-13T18:57:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.775+0000 7f7096ffd700 1 -- 172.21.15.146:0/1896650163 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70900055a0 con 0x7f70a00cc490 2024-09-13T18:57:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.779+0000 7f70a70f6700 1 -- 172.21.15.146:0/1896650163 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a01df0f0 con 0x7f70a00cc490 2024-09-13T18:57:10.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.779+0000 7f70a70f6700 1 -- 172.21.15.146:0/1896650163 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f70a006f5c0 con 0x7f70a00cc490 2024-09-13T18:57:10.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.779+0000 7f7096ffd700 1 -- 172.21.15.146:0/1896650163 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f70900038c0 con 0x7f70a00cc490 2024-09-13T18:57:10.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.779+0000 7f7096ffd700 1 --2- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7084066730 0x7f7084068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.779+0000 7f7096ffd700 1 -- 172.21.15.146:0/1896650163 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f7090083dc0 con 0x7f70a00cc490 2024-09-13T18:57:10.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.779+0000 7f7096ffd700 1 --2- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] conn(0x7f7084070d50 0x7f70840731f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.779+0000 7f70a60f4700 1 --2- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7084066730 0x7f7084068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.779+0000 7f7096ffd700 1 -- 172.21.15.146:0/1896650163 --> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70840738c0 con 0x7f7084070d50 2024-09-13T18:57:10.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.779+0000 7f7096ffd700 1 -- 172.21.15.146:0/1896650163 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70900863a0 con 0x7f70a00cc490 2024-09-13T18:57:10.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.779+0000 7f70a68f5700 1 --2- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] conn(0x7f7084070d50 0x7f70840731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.783+0000 7f70a60f4700 1 --2- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7084066730 0x7f7084068bf0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f709c009a00 tx=0x7f709c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:10.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.783+0000 7f70a68f5700 1 --2- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] conn(0x7f7084070d50 0x7f70840731f0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:10.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.783+0000 7f7096ffd700 1 -- 172.21.15.146:0/1896650163 <== osd.5 v2:172.21.15.154:6800/363322763 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f70840738c0 con 0x7f7084070d50 2024-09-13T18:57:10.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.799+0000 7f70a70f6700 1 -- 172.21.15.146:0/1896650163 --> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f70a006efe0 con 0x7f7084070d50 2024-09-13T18:57:10.812 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.799+0000 7f7096ffd700 1 -- 172.21.15.146:0/1896650163 <== osd.5 v2:172.21.15.154:6800/363322763 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f70980099b0 con 0x7f7084070d50 2024-09-13T18:57:10.812 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:57:10.812 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:57:10.812 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:57:10.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.799+0000 7f70a70f6700 1 -- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] conn(0x7f7084070d50 msgr2=0x7f70840731f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.799+0000 7f70a70f6700 1 --2- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:6800/363322763,v1:172.21.15.154:6801/363322763] conn(0x7f7084070d50 0x7f70840731f0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.799+0000 7f70a70f6700 1 -- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7084066730 msgr2=0x7f7084068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.799+0000 7f70a70f6700 1 --2- 172.21.15.146:0/1896650163 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7084066730 0x7f7084068bf0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f709c009a00 tx=0x7f709c008040 comp rx=0 tx=0).stop 2024-09-13T18:57:10.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.799+0000 7f70a70f6700 1 -- 172.21.15.146:0/1896650163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00cc490 msgr2=0x7f70a00c4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.799+0000 7f70a70f6700 1 --2- 172.21.15.146:0/1896650163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00cc490 0x7f70a00c4a90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f709000c4d0 tx=0x7f709000c7e0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.799+0000 7f70a68f5700 1 -- 172.21.15.146:0/1896650163 reap_dead start 2024-09-13T18:57:10.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.799+0000 7f70a70f6700 1 -- 172.21.15.146:0/1896650163 shutdown_connections 2024-09-13T18:57:10.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.799+0000 7f70a70f6700 1 -- 172.21.15.146:0/1896650163 >> 172.21.15.146:0/1896650163 conn(0x7f70a0071ed0 msgr2=0x7f70a014c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:10.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.799+0000 7f70a70f6700 1 -- 172.21.15.146:0/1896650163 shutdown_connections 2024-09-13T18:57:10.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.799+0000 7f70a70f6700 1 -- 172.21.15.146:0/1896650163 wait complete. 2024-09-13T18:57:10.825 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T18:57:10.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 Processor -- start 2024-09-13T18:57:10.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 -- start start 2024-09-13T18:57:10.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96000d5640 0x7f9600005ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9600045080 0x7f96000462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f96000467e0 0x7f96000db700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9600033710 con 0x7f9600045080 2024-09-13T18:57:10.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96000db2b0 con 0x7f96000467e0 2024-09-13T18:57:10.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96000d5a20 con 0x7f96000d5640 2024-09-13T18:57:10.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f9605d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9600045080 0x7f96000462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f9605d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9600045080 0x7f96000462a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57786/0 (socket says 172.21.15.146:57786) 2024-09-13T18:57:10.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f9605d9b700 1 -- 172.21.15.146:0/2196893819 learned_addr learned my addr 172.21.15.146:0/2196893819 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:10.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960659c700 1 --2- 172.21.15.146:0/2196893819 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96000d5640 0x7f9600005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f9606d9d700 1 --2- 172.21.15.146:0/2196893819 >> v2:172.21.15.154:3300/0 conn(0x7f96000467e0 0x7f96000db700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f9605d9b700 1 -- 172.21.15.146:0/2196893819 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96000d5640 msgr2=0x7f9600005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:10.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f9605d9b700 1 --2- 172.21.15.146:0/2196893819 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96000d5640 0x7f9600005ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f9605d9b700 1 -- 172.21.15.146:0/2196893819 >> v2:172.21.15.154:3300/0 conn(0x7f96000467e0 msgr2=0x7f96000db700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:10.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f9605d9b700 1 --2- 172.21.15.146:0/2196893819 >> v2:172.21.15.154:3300/0 conn(0x7f96000467e0 0x7f96000db700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f9605d9b700 1 -- 172.21.15.146:0/2196893819 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9600035870 con 0x7f9600045080 2024-09-13T18:57:10.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960659c700 1 --2- 172.21.15.146:0/2196893819 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96000d5640 0x7f9600005ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:57:10.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f9606d9d700 1 --2- 172.21.15.146:0/2196893819 >> v2:172.21.15.154:3300/0 conn(0x7f96000467e0 0x7f96000db700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:57:10.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f9605d9b700 1 --2- 172.21.15.146:0/2196893819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9600045080 0x7f96000462a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f960004fc90 tx=0x7f95f8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e52d8ee48f679da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:10.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960559a700 1 -- 172.21.15.146:0/2196893819 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f800ba20 con 0x7f9600045080 2024-09-13T18:57:10.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960559a700 1 -- 172.21.15.146:0/2196893819 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f800bb80 con 0x7f9600045080 2024-09-13T18:57:10.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960559a700 1 -- 172.21.15.146:0/2196893819 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8009e60 con 0x7f9600045080 2024-09-13T18:57:10.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 -- 172.21.15.146:0/2196893819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9600045080 msgr2=0x7f96000462a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 --2- 172.21.15.146:0/2196893819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9600045080 0x7f96000462a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f960004fc90 tx=0x7f95f8009c20 comp rx=0 tx=0).stop 2024-09-13T18:57:10.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 -- 172.21.15.146:0/2196893819 shutdown_connections 2024-09-13T18:57:10.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 --2- 172.21.15.146:0/2196893819 >> v2:172.21.15.154:3300/0 conn(0x7f96000467e0 0x7f96000db700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 --2- 172.21.15.146:0/2196893819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9600045080 0x7f96000462a0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 --2- 172.21.15.146:0/2196893819 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96000d5640 0x7f9600005ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 -- 172.21.15.146:0/2196893819 >> 172.21.15.146:0/2196893819 conn(0x7f960004fd00 msgr2=0x7f9600050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:10.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 -- 172.21.15.146:0/2196893819 shutdown_connections 2024-09-13T18:57:10.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.915+0000 7f960759e700 1 -- 172.21.15.146:0/2196893819 wait complete. 2024-09-13T18:57:10.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f960759e700 1 Processor -- start 2024-09-13T18:57:10.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f960759e700 1 -- start start 2024-09-13T18:57:10.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f960759e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9600045080 0x7f96000c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f960759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96000467e0 0x7f96000c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f960759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96000d5640 0x7f960004f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f960759e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96000cf810 con 0x7f96000467e0 2024-09-13T18:57:10.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f960759e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96000cfaf0 con 0x7f9600045080 2024-09-13T18:57:10.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f960759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96000ce0a0 con 0x7f96000d5640 2024-09-13T18:57:10.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f9605d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96000467e0 0x7f96000c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f9605d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96000467e0 0x7f96000c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57796/0 (socket says 172.21.15.146:57796) 2024-09-13T18:57:10.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f9605d9b700 1 -- 172.21.15.146:0/3521651331 learned_addr learned my addr 172.21.15.146:0/3521651331 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:10.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f9606d9d700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96000d5640 0x7f960004f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f960659c700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9600045080 0x7f96000c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f9605d9b700 1 -- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96000d5640 msgr2=0x7f960004f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f9605d9b700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96000d5640 0x7f960004f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f9605d9b700 1 -- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9600045080 msgr2=0x7f96000c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:10.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f9605d9b700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9600045080 0x7f96000c4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:10.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f9605d9b700 1 -- 172.21.15.146:0/3521651331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f8009570 con 0x7f96000467e0 2024-09-13T18:57:10.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f9606d9d700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96000d5640 0x7f960004f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:57:10.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f9605d9b700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96000467e0 0x7f96000c58b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9600050e00 tx=0x7f95f8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:10.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f95feffd700 1 -- 172.21.15.146:0/3521651331 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8004530 con 0x7f96000467e0 2024-09-13T18:57:10.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f95feffd700 1 -- 172.21.15.146:0/3521651331 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f801e070 con 0x7f96000467e0 2024-09-13T18:57:10.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f95feffd700 1 -- 172.21.15.146:0/3521651331 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f80038e0 con 0x7f96000467e0 2024-09-13T18:57:10.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f960759e700 1 -- 172.21.15.146:0/3521651331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96000cff80 con 0x7f96000467e0 2024-09-13T18:57:10.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.919+0000 7f960759e700 1 -- 172.21.15.146:0/3521651331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96000df2c0 con 0x7f96000467e0 2024-09-13T18:57:10.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.923+0000 7f960759e700 1 -- 172.21.15.146:0/3521651331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96000df120 con 0x7f96000467e0 2024-09-13T18:57:10.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.923+0000 7f95feffd700 1 -- 172.21.15.146:0/3521651331 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f95f8020030 con 0x7f96000467e0 2024-09-13T18:57:10.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.923+0000 7f95feffd700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95e8066760 0x7f95e8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:10.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.923+0000 7f95feffd700 1 -- 172.21.15.146:0/3521651331 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f95f8085ba0 con 0x7f96000467e0 2024-09-13T18:57:10.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.923+0000 7f960659c700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95e8066760 0x7f95e8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:10.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.923+0000 7f960659c700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95e8066760 0x7f95e8068c20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f95f000a9a0 tx=0x7f95f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:10.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:10.935+0000 7f95feffd700 1 -- 172.21.15.146:0/3521651331 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f96000df120 con 0x7f96000467e0 2024-09-13T18:57:11.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.151+0000 7f960759e700 1 -- 172.21.15.146:0/3521651331 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f9600034fd0 con 0x7f95e8066760 2024-09-13T18:57:11.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.155+0000 7f95feffd700 1 -- 172.21.15.146:0/3521651331 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f95f000a670 con 0x7f95e8066760 2024-09-13T18:57:11.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.159+0000 7f960759e700 1 -- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95e8066760 msgr2=0x7f95e8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.159+0000 7f960759e700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95e8066760 0x7f95e8068c20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f95f000a9a0 tx=0x7f95f0008040 comp rx=0 tx=0).stop 2024-09-13T18:57:11.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.159+0000 7f960759e700 1 -- 172.21.15.146:0/3521651331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96000467e0 msgr2=0x7f96000c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.159+0000 7f960759e700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96000467e0 0x7f96000c58b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9600050e00 tx=0x7f95f8004770 comp rx=0 tx=0).stop 2024-09-13T18:57:11.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.159+0000 7f960759e700 1 -- 172.21.15.146:0/3521651331 shutdown_connections 2024-09-13T18:57:11.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.159+0000 7f960759e700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f95e8066760 0x7f95e8068c20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.159+0000 7f960759e700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96000d5640 0x7f960004f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.159+0000 7f960759e700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96000467e0 0x7f96000c58b0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.159+0000 7f960759e700 1 --2- 172.21.15.146:0/3521651331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9600045080 0x7f96000c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.159+0000 7f960759e700 1 -- 172.21.15.146:0/3521651331 >> 172.21.15.146:0/3521651331 conn(0x7f960004fd00 msgr2=0x7f96000da460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:11.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.159+0000 7f960759e700 1 -- 172.21.15.146:0/3521651331 shutdown_connections 2024-09-13T18:57:11.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.159+0000 7f960759e700 1 -- 172.21.15.146:0/3521651331 wait complete. 2024-09-13T18:57:11.179 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T18:57:11.197 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T18:57:11.197 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:57:11.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe953fff700 1 Processor -- start 2024-09-13T18:57:11.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe953fff700 1 -- start start 2024-09-13T18:57:11.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe953fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540c11f0 0x7fe9540c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe953fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe9540c15d0 0x7fe954159a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe953fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95406e480 con 0x7fe9540c2e40 2024-09-13T18:57:11.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe953fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe954159630 con 0x7fe9540c15d0 2024-09-13T18:57:11.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe953fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95406e720 con 0x7fe9540c11f0 2024-09-13T18:57:11.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe9527fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe9540c15d0 0x7fe954159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe9527fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe9540c15d0 0x7fe954159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49204/0 (socket says 172.21.15.146:49204) 2024-09-13T18:57:11.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe952ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540c11f0 0x7fe9540c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe9527fc700 1 -- 172.21.15.146:0/3651326852 learned_addr learned my addr 172.21.15.146:0/3651326852 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:11.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe9527fc700 1 -- 172.21.15.146:0/3651326852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540c11f0 msgr2=0x7fe9540c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:11.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe9527fc700 1 --2- 172.21.15.146:0/3651326852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540c11f0 0x7fe9540c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe9527fc700 1 -- 172.21.15.146:0/3651326852 >> v1:172.21.15.154:6789/0 conn(0x7fe9540c2e40 legacy=0x7fe9541567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe951ffb700 1 -- 172.21.15.146:0/3651326852 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749236392 0 0) 0x7fe95406e480 con 0x7fe9540c2e40 2024-09-13T18:57:11.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe9527fc700 1 -- 172.21.15.146:0/3651326852 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95406ea00 con 0x7fe9540c15d0 2024-09-13T18:57:11.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe9527fc700 1 --2- 172.21.15.146:0/3651326852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9540c15d0 0x7fe954159a80 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fe948009910 tx=0x7fe948009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8060cd12a52678c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:11.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe951ffb700 1 -- 172.21.15.146:0/3651326852 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948011830 con 0x7fe9540c15d0 2024-09-13T18:57:11.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe951ffb700 1 -- 172.21.15.146:0/3651326852 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe948011990 con 0x7fe9540c15d0 2024-09-13T18:57:11.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.323+0000 7fe951ffb700 1 -- 172.21.15.146:0/3651326852 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948011d40 con 0x7fe9540c15d0 2024-09-13T18:57:11.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 -- 172.21.15.146:0/3651326852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9540c15d0 msgr2=0x7fe954159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 --2- 172.21.15.146:0/3651326852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9540c15d0 0x7fe954159a80 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fe948009910 tx=0x7fe948009c20 comp rx=0 tx=0).stop 2024-09-13T18:57:11.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 -- 172.21.15.146:0/3651326852 shutdown_connections 2024-09-13T18:57:11.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 --2- 172.21.15.146:0/3651326852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9540c15d0 0x7fe954159a80 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 --2- 172.21.15.146:0/3651326852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540c11f0 0x7fe9540c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 -- 172.21.15.146:0/3651326852 >> 172.21.15.146:0/3651326852 conn(0x7fe954071ed0 msgr2=0x7fe9540722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:11.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 -- 172.21.15.146:0/3651326852 shutdown_connections 2024-09-13T18:57:11.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 -- 172.21.15.146:0/3651326852 wait complete. 2024-09-13T18:57:11.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 Processor -- start 2024-09-13T18:57:11.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 -- start start 2024-09-13T18:57:11.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540c11f0 0x7fe9540aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9540c15d0 0x7fe9540afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9540c2e40 0x7fe9541f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9540c5e30 con 0x7fe9540c2e40 2024-09-13T18:57:11.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9540c52f0 con 0x7fe9540c15d0 2024-09-13T18:57:11.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe953fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9540c5590 con 0x7fe9540c11f0 2024-09-13T18:57:11.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe9537fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9540c2e40 0x7fe9541f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe9537fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9540c2e40 0x7fe9541f2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49214/0 (socket says 172.21.15.146:49214) 2024-09-13T18:57:11.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe952ffd700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540c11f0 0x7fe9540aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe9537fe700 1 -- 172.21.15.146:0/1860497999 learned_addr learned my addr 172.21.15.146:0/1860497999 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:11.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe9527fc700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9540c15d0 0x7fe9540afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe9537fe700 1 -- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540c11f0 msgr2=0x7fe9540aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe9537fe700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540c11f0 0x7fe9540aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe9537fe700 1 -- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9540c15d0 msgr2=0x7fe9540afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe9537fe700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9540c15d0 0x7fe9540afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe9537fe700 1 -- 172.21.15.146:0/1860497999 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe948009570 con 0x7fe9540c2e40 2024-09-13T18:57:11.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe9537fe700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9540c2e40 0x7fe9541f2d30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00cb60 tx=0x7fe94c00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:11.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.327+0000 7fe93b7fe700 1 -- 172.21.15.146:0/1860497999 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c003e50 con 0x7fe9540c2e40 2024-09-13T18:57:11.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.331+0000 7fe93b7fe700 1 -- 172.21.15.146:0/1860497999 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c005070 con 0x7fe9540c2e40 2024-09-13T18:57:11.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.331+0000 7fe93b7fe700 1 -- 172.21.15.146:0/1860497999 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c005420 con 0x7fe9540c2e40 2024-09-13T18:57:11.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.331+0000 7fe953fff700 1 -- 172.21.15.146:0/1860497999 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9540c5870 con 0x7fe9540c2e40 2024-09-13T18:57:11.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.331+0000 7fe953fff700 1 -- 172.21.15.146:0/1860497999 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9541f2810 con 0x7fe9540c2e40 2024-09-13T18:57:11.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.331+0000 7fe953fff700 1 -- 172.21.15.146:0/1860497999 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe95406efc0 con 0x7fe9540c2e40 2024-09-13T18:57:11.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.331+0000 7fe93b7fe700 1 -- 172.21.15.146:0/1860497999 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe94c005580 con 0x7fe9540c2e40 2024-09-13T18:57:11.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.335+0000 7fe93b7fe700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe940066760 0x7fe940068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.335+0000 7fe93b7fe700 1 -- 172.21.15.146:0/1860497999 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fe94c0843e0 con 0x7fe9540c2e40 2024-09-13T18:57:11.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.335+0000 7fe93b7fe700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] conn(0x7fe940070d00 0x7fe9400731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.335+0000 7fe93b7fe700 1 -- 172.21.15.146:0/1860497999 --> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe940073870 con 0x7fe940070d00 2024-09-13T18:57:11.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.335+0000 7fe93b7fe700 1 -- 172.21.15.146:0/1860497999 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe94c086aa0 con 0x7fe9540c2e40 2024-09-13T18:57:11.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.335+0000 7fe952ffd700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe940066760 0x7fe940068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.335+0000 7fe9527fc700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] conn(0x7fe940070d00 0x7fe9400731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.335+0000 7fe9527fc700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] conn(0x7fe940070d00 0x7fe9400731a0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.335+0000 7fe952ffd700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe940066760 0x7fe940068c20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe944009bd0 tx=0x7fe944009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.335+0000 7fe93b7fe700 1 -- 172.21.15.146:0/1860497999 <== osd.6 v2:172.21.15.154:6816/2272517392 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe940073870 con 0x7fe940070d00 2024-09-13T18:57:11.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.355+0000 7fe953fff700 1 -- 172.21.15.146:0/1860497999 --> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe9541f2a00 con 0x7fe940070d00 2024-09-13T18:57:11.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.355+0000 7fe93b7fe700 1 -- 172.21.15.146:0/1860497999 <== osd.6 v2:172.21.15.154:6816/2272517392 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe9541f2a00 con 0x7fe940070d00 2024-09-13T18:57:11.368 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:57:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:57:11.368 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:57:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.355+0000 7fe953fff700 1 -- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] conn(0x7fe940070d00 msgr2=0x7fe9400731a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.355+0000 7fe953fff700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:6816/2272517392,v1:172.21.15.154:6818/2272517392] conn(0x7fe940070d00 0x7fe9400731a0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.355+0000 7fe953fff700 1 -- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe940066760 msgr2=0x7fe940068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.355+0000 7fe953fff700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe940066760 0x7fe940068c20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe944009bd0 tx=0x7fe944009240 comp rx=0 tx=0).stop 2024-09-13T18:57:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.355+0000 7fe953fff700 1 -- 172.21.15.146:0/1860497999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9540c2e40 msgr2=0x7fe9541f2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.355+0000 7fe953fff700 1 --2- 172.21.15.146:0/1860497999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9540c2e40 0x7fe9541f2d30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00cb60 tx=0x7fe94c00ce70 comp rx=0 tx=0).stop 2024-09-13T18:57:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.355+0000 7fe9537fe700 1 -- 172.21.15.146:0/1860497999 reap_dead start 2024-09-13T18:57:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.355+0000 7fe953fff700 1 -- 172.21.15.146:0/1860497999 shutdown_connections 2024-09-13T18:57:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.355+0000 7fe953fff700 1 -- 172.21.15.146:0/1860497999 >> 172.21.15.146:0/1860497999 conn(0x7fe954071ed0 msgr2=0x7fe954159840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:11.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.355+0000 7fe953fff700 1 -- 172.21.15.146:0/1860497999 shutdown_connections 2024-09-13T18:57:11.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.355+0000 7fe953fff700 1 -- 172.21.15.146:0/1860497999 wait complete. 2024-09-13T18:57:11.384 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T18:57:11.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.483+0000 7f7fe1e13700 1 Processor -- start 2024-09-13T18:57:11.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.483+0000 7f7fe1e13700 1 -- start start 2024-09-13T18:57:11.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1e13700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fdc1538c0 0x7f7fdc0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1e13700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7fdc15a080 0x7f7fdc0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1e13700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fdc0cb340 con 0x7f7fdc0cc680 2024-09-13T18:57:11.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1e13700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fdc15a530 con 0x7f7fdc15a080 2024-09-13T18:57:11.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1e13700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0cb5e0 con 0x7f7fdc1538c0 2024-09-13T18:57:11.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe0e11700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fdc1538c0 0x7f7fdc0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fdbfff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7fdc0cc680 0x7f7fdc0c7580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:41510/0 (socket says 172.21.15.146:41510) 2024-09-13T18:57:11.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1612700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7fdc15a080 0x7f7fdc0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fdbfff700 1 -- 172.21.15.146:0/663914560 learned_addr learned my addr 172.21.15.146:0/663914560 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:57:11.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fdb7fe700 1 -- 172.21.15.146:0/663914560 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134986489 0 0) 0x7f7fdc0cb340 con 0x7f7fdc0cc680 2024-09-13T18:57:11.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fdb7fe700 1 -- 172.21.15.146:0/663914560 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fc40035d0 con 0x7f7fdc0cc680 2024-09-13T18:57:11.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1612700 1 -- 172.21.15.146:0/663914560 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fdc1538c0 msgr2=0x7f7fdc0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:11.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1612700 1 --2- 172.21.15.146:0/663914560 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fdc1538c0 0x7f7fdc0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1612700 1 -- 172.21.15.146:0/663914560 >> v1:172.21.15.154:6789/0 conn(0x7f7fdc0cc680 legacy=0x7f7fdc0c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1612700 1 -- 172.21.15.146:0/663914560 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc0cb8c0 con 0x7f7fdc15a080 2024-09-13T18:57:11.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1612700 1 --2- 172.21.15.146:0/663914560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fdc15a080 0x7f7fdc0cf390 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009850 tx=0x7f7fcc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=202c0af8d658f9a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:11.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fdb7fe700 1 -- 172.21.15.146:0/663914560 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc011830 con 0x7f7fdc15a080 2024-09-13T18:57:11.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fdb7fe700 1 -- 172.21.15.146:0/663914560 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc011990 con 0x7f7fdc15a080 2024-09-13T18:57:11.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fdb7fe700 1 -- 172.21.15.146:0/663914560 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc011d40 con 0x7f7fdc15a080 2024-09-13T18:57:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1e13700 1 -- 172.21.15.146:0/663914560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fdc15a080 msgr2=0x7f7fdc0cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1e13700 1 --2- 172.21.15.146:0/663914560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fdc15a080 0x7f7fdc0cf390 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009850 tx=0x7f7fcc009b60 comp rx=0 tx=0).stop 2024-09-13T18:57:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1e13700 1 -- 172.21.15.146:0/663914560 shutdown_connections 2024-09-13T18:57:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1e13700 1 --2- 172.21.15.146:0/663914560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fdc15a080 0x7f7fdc0cf390 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1e13700 1 --2- 172.21.15.146:0/663914560 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fdc1538c0 0x7f7fdc0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1e13700 1 -- 172.21.15.146:0/663914560 >> 172.21.15.146:0/663914560 conn(0x7f7fdc066270 msgr2=0x7f7fdc066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1e13700 1 -- 172.21.15.146:0/663914560 shutdown_connections 2024-09-13T18:57:11.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.487+0000 7f7fe1e13700 1 -- 172.21.15.146:0/663914560 wait complete. 2024-09-13T18:57:11.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe1e13700 1 Processor -- start 2024-09-13T18:57:11.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe1e13700 1 -- start start 2024-09-13T18:57:11.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe1e13700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fdc0cc680 0x7f7fdc1ef770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe1e13700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fdc1538c0 0x7f7fdc1f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe1e13700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fdc15a080 0x7f7fdc0c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe1e13700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc14d760 con 0x7f7fdc1538c0 2024-09-13T18:57:11.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe1e13700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0b5360 con 0x7f7fdc15a080 2024-09-13T18:57:11.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe1e13700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc1efcb0 con 0x7f7fdc0cc680 2024-09-13T18:57:11.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe0e11700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fdc0cc680 0x7f7fdc1ef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fdbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fdc1538c0 0x7f7fdc1f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe0e11700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fdc0cc680 0x7f7fdc1ef770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:46534/0 (socket says 172.21.15.146:46534) 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe0e11700 1 -- 172.21.15.146:0/420701782 learned_addr learned my addr 172.21.15.146:0/420701782 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe1612700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fdc15a080 0x7f7fdc0c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fdbfff700 1 -- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fdc0cc680 msgr2=0x7f7fdc1ef770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fdbfff700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fdc0cc680 0x7f7fdc1ef770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fdbfff700 1 -- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fdc15a080 msgr2=0x7f7fdc0c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fdbfff700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fdc15a080 0x7f7fdc0c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fdbfff700 1 -- 172.21.15.146:0/420701782 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc0094b0 con 0x7f7fdc1538c0 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe1612700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fdc15a080 0x7f7fdc0c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fdbfff700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fdc1538c0 0x7f7fdc1f2bd0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000c4d0 tx=0x7f7fd000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fd97fa700 1 -- 172.21.15.146:0/420701782 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0005d90 con 0x7f7fdc1538c0 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fd97fa700 1 -- 172.21.15.146:0/420701782 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd00070d0 con 0x7f7fdc1538c0 2024-09-13T18:57:11.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe1e13700 1 -- 172.21.15.146:0/420701782 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fdc0b4ae0 con 0x7f7fdc1538c0 2024-09-13T18:57:11.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fd97fa700 1 -- 172.21.15.146:0/420701782 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0007440 con 0x7f7fdc1538c0 2024-09-13T18:57:11.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.491+0000 7f7fe1e13700 1 -- 172.21.15.146:0/420701782 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fdc0b5080 con 0x7f7fdc1538c0 2024-09-13T18:57:11.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.495+0000 7f7fe1e13700 1 -- 172.21.15.146:0/420701782 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fdc0c99c0 con 0x7f7fdc1538c0 2024-09-13T18:57:11.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.495+0000 7f7fd97fa700 1 -- 172.21.15.146:0/420701782 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7fd00075a0 con 0x7f7fdc1538c0 2024-09-13T18:57:11.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.495+0000 7f7fd97fa700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7fc8066730 0x7f7fc8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.495+0000 7f7fd97fa700 1 -- 172.21.15.146:0/420701782 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f7fd0083d70 con 0x7f7fdc1538c0 2024-09-13T18:57:11.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.495+0000 7f7fe0e11700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7fc8066730 0x7f7fc8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.495+0000 7f7fe0e11700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7fc8066730 0x7f7fc8068bf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009a40 tx=0x7f7fd400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:11.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.503+0000 7f7fd97fa700 1 -- 172.21.15.146:0/420701782 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7fd0050880 con 0x7f7fdc1538c0 2024-09-13T18:57:11.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.663+0000 7f7fe1e13700 1 -- 172.21.15.146:0/420701782 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fdc0cc160 con 0x7f7fc8066730 2024-09-13T18:57:11.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.663+0000 7f7fd97fa700 1 -- 172.21.15.146:0/420701782 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7fd400f040 con 0x7f7fc8066730 2024-09-13T18:57:11.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.663+0000 7f7fe1e13700 1 -- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7fc8066730 msgr2=0x7f7fc8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.663+0000 7f7fe1e13700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7fc8066730 0x7f7fc8068bf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009a40 tx=0x7f7fd400b040 comp rx=0 tx=0).stop 2024-09-13T18:57:11.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.663+0000 7f7fe1e13700 1 -- 172.21.15.146:0/420701782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fdc1538c0 msgr2=0x7f7fdc1f2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.663+0000 7f7fe1e13700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fdc1538c0 0x7f7fdc1f2bd0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000c4d0 tx=0x7f7fd000c7e0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.663+0000 7f7fe1e13700 1 -- 172.21.15.146:0/420701782 shutdown_connections 2024-09-13T18:57:11.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.663+0000 7f7fe1e13700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7fc8066730 0x7f7fc8068bf0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.663+0000 7f7fe1e13700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fdc15a080 0x7f7fdc0c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.663+0000 7f7fe1e13700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fdc1538c0 0x7f7fdc1f2bd0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.663+0000 7f7fe1e13700 1 --2- 172.21.15.146:0/420701782 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fdc0cc680 0x7f7fdc1ef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.663+0000 7f7fe1e13700 1 -- 172.21.15.146:0/420701782 >> 172.21.15.146:0/420701782 conn(0x7f7fdc066270 msgr2=0x7f7fdc0cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:11.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.667+0000 7f7fe1e13700 1 -- 172.21.15.146:0/420701782 shutdown_connections 2024-09-13T18:57:11.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.667+0000 7f7fe1e13700 1 -- 172.21.15.146:0/420701782 wait complete. 2024-09-13T18:57:11.680 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T18:57:11.694 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T18:57:11.695 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:57:11.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.815+0000 7f2da3fff700 1 Processor -- start 2024-09-13T18:57:11.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.815+0000 7f2da3fff700 1 -- start start 2024-09-13T18:57:11.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.815+0000 7f2da3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2da40a9490 0x7f2da406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.815+0000 7f2da3fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da414fb30 con 0x7f2da406f5c0 2024-09-13T18:57:11.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.815+0000 7f2da3fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da4155580 con 0x7f2da414f750 2024-09-13T18:57:11.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.815+0000 7f2da3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da4155860 con 0x7f2da40a9490 2024-09-13T18:57:11.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.815+0000 7f2da37fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f2da406f5c0 0x7f2da406fa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:54424/0 (socket says 172.21.15.146:54424) 2024-09-13T18:57:11.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.815+0000 7f2da37fe700 1 -- 172.21.15.146:0/239412348 learned_addr learned my addr 172.21.15.146:0/239412348 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:57:11.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 -- 172.21.15.146:0/239412348 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221232216 0 0) 0x7f2da414fb30 con 0x7f2da406f5c0 2024-09-13T18:57:11.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 -- 172.21.15.146:0/239412348 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d8c0035d0 con 0x7f2da406f5c0 2024-09-13T18:57:11.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 -- 172.21.15.146:0/239412348 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 949073859 0 0) 0x7f2d8c0035d0 con 0x7f2da406f5c0 2024-09-13T18:57:11.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 -- 172.21.15.146:0/239412348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2da40a9490 msgr2=0x7f2da406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:11.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 --2- 172.21.15.146:0/239412348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2da40a9490 0x7f2da406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 -- 172.21.15.146:0/239412348 >> v1:172.21.15.154:6789/0 conn(0x7f2da414f750 legacy=0x7f2da40cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:11.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 -- 172.21.15.146:0/239412348 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da4155b40 con 0x7f2da406f5c0 2024-09-13T18:57:11.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 -- 172.21.15.146:0/239412348 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f2d9c0019f0 con 0x7f2da406f5c0 2024-09-13T18:57:11.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 -- 172.21.15.146:0/239412348 >> v1:172.21.15.146:6789/0 conn(0x7f2da406f5c0 legacy=0x7f2da406fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 --2- 172.21.15.146:0/239412348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d8c004350 0x7f2d8c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 --2- 172.21.15.146:0/239412348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d8c004fb0 0x7f2d8c009a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 --2- 172.21.15.146:0/239412348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d8c009f60 0x7f2d8c00c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 -- 172.21.15.146:0/239412348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da4155860 con 0x7f2d8c004fb0 2024-09-13T18:57:11.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 -- 172.21.15.146:0/239412348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c00cae0 con 0x7f2d8c009f60 2024-09-13T18:57:11.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da1ffb700 1 -- 172.21.15.146:0/239412348 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c00cca0 con 0x7f2d8c004350 2024-09-13T18:57:11.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da2ffd700 1 --2- 172.21.15.146:0/239412348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d8c004350 0x7f2d8c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da37fe700 1 --2- 172.21.15.146:0/239412348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d8c009f60 0x7f2d8c00c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da2ffd700 1 -- 172.21.15.146:0/239412348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d8c009f60 msgr2=0x7f2d8c00c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:11.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da2ffd700 1 --2- 172.21.15.146:0/239412348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d8c009f60 0x7f2d8c00c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da2ffd700 1 -- 172.21.15.146:0/239412348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d8c004fb0 msgr2=0x7f2d8c009a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:57:11.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da2ffd700 1 --2- 172.21.15.146:0/239412348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d8c004fb0 0x7f2d8c009a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da2ffd700 1 -- 172.21.15.146:0/239412348 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c0035d0 con 0x7f2d8c004350 2024-09-13T18:57:11.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.819+0000 7f2da2ffd700 1 --2- 172.21.15.146:0/239412348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d8c004350 0x7f2d8c004870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d940015f0 tx=0x7f2d8c0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee30fb116aa64e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:11.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.823+0000 7f2da1ffb700 1 -- 172.21.15.146:0/239412348 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94001200 con 0x7f2d8c004350 2024-09-13T18:57:11.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.823+0000 7f2da1ffb700 1 -- 172.21.15.146:0/239412348 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d94005310 con 0x7f2d8c004350 2024-09-13T18:57:11.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.823+0000 7f2da3fff700 1 -- 172.21.15.146:0/239412348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d8c004350 msgr2=0x7f2d8c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.823+0000 7f2da3fff700 1 --2- 172.21.15.146:0/239412348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d8c004350 0x7f2d8c004870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d940015f0 tx=0x7f2d8c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.827+0000 7f2da37fe700 1 -- 172.21.15.146:0/239412348 reap_dead start 2024-09-13T18:57:11.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.827+0000 7f2da3fff700 1 -- 172.21.15.146:0/239412348 shutdown_connections 2024-09-13T18:57:11.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.827+0000 7f2da3fff700 1 -- 172.21.15.146:0/239412348 >> 172.21.15.146:0/239412348 conn(0x7f2da4066090 msgr2=0x7f2da4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:11.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.839+0000 7f2da3fff700 1 -- 172.21.15.146:0/239412348 shutdown_connections 2024-09-13T18:57:11.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.839+0000 7f2da3fff700 1 -- 172.21.15.146:0/239412348 wait complete. 2024-09-13T18:57:11.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da3fff700 1 Processor -- start 2024-09-13T18:57:11.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da3fff700 1 -- start start 2024-09-13T18:57:11.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2da40ace70 0x7f2da41507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2da40ad250 0x7f2da4156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2da4159630 0x7f2da414ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da3fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da4156b70 con 0x7f2da4159630 2024-09-13T18:57:11.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da3fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da41513c0 con 0x7f2da40ace70 2024-09-13T18:57:11.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da41588e0 con 0x7f2da40ad250 2024-09-13T18:57:11.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2da4159630 0x7f2da414ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2da4159630 0x7f2da414ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49230/0 (socket says 172.21.15.146:49230) 2024-09-13T18:57:11.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da37fe700 1 -- 172.21.15.146:0/106767418 learned_addr learned my addr 172.21.15.146:0/106767418 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:11.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da27fc700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2da40ad250 0x7f2da4156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da2ffd700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2da40ace70 0x7f2da41507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da37fe700 1 -- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2da40ad250 msgr2=0x7f2da4156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da37fe700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2da40ad250 0x7f2da4156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da37fe700 1 -- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2da40ace70 msgr2=0x7f2da41507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da37fe700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2da40ace70 0x7f2da41507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da37fe700 1 -- 172.21.15.146:0/106767418 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da4150e60 con 0x7f2da4159630 2024-09-13T18:57:11.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da2ffd700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2da40ace70 0x7f2da41507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:57:11.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da37fe700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2da4159630 0x7f2da414ded0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2da4071a40 tx=0x7f2d9c0019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:11.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2d8b7fe700 1 -- 172.21.15.146:0/106767418 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c00a8d0 con 0x7f2da4159630 2024-09-13T18:57:11.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2d8b7fe700 1 -- 172.21.15.146:0/106767418 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c00aa30 con 0x7f2da4159630 2024-09-13T18:57:11.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2d8b7fe700 1 -- 172.21.15.146:0/106767418 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c00ade0 con 0x7f2da4159630 2024-09-13T18:57:11.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da3fff700 1 -- 172.21.15.146:0/106767418 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da4151100 con 0x7f2da4159630 2024-09-13T18:57:11.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.843+0000 7f2da3fff700 1 -- 172.21.15.146:0/106767418 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da41f0680 con 0x7f2da4159630 2024-09-13T18:57:11.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.847+0000 7f2da3fff700 1 -- 172.21.15.146:0/106767418 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2da40c9be0 con 0x7f2da4159630 2024-09-13T18:57:11.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.847+0000 7f2d8b7fe700 1 -- 172.21.15.146:0/106767418 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2d9c01d460 con 0x7f2da4159630 2024-09-13T18:57:11.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.847+0000 7f2d8b7fe700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2d90066730 0x7f2d90068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.847+0000 7f2d8b7fe700 1 -- 172.21.15.146:0/106767418 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f2d9c085030 con 0x7f2da4159630 2024-09-13T18:57:11.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.847+0000 7f2d8b7fe700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] conn(0x7f2d90070cd0 0x7f2d90073170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:11.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.847+0000 7f2da2ffd700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2d90066730 0x7f2d90068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.847+0000 7f2d8b7fe700 1 -- 172.21.15.146:0/106767418 --> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d90073840 con 0x7f2d90070cd0 2024-09-13T18:57:11.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.847+0000 7f2d8b7fe700 1 -- 172.21.15.146:0/106767418 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d9c051960 con 0x7f2da4159630 2024-09-13T18:57:11.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.851+0000 7f2da27fc700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] conn(0x7f2d90070cd0 0x7f2d90073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:11.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.855+0000 7f2da2ffd700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2d90066730 0x7f2d90068bf0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2d940015f0 tx=0x7f2d9400a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:11.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.855+0000 7f2da27fc700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] conn(0x7f2d90070cd0 0x7f2d90073170 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:11.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.855+0000 7f2d8b7fe700 1 -- 172.21.15.146:0/106767418 <== osd.7 v2:172.21.15.154:6817/3339245911 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2d90073840 con 0x7f2d90070cd0 2024-09-13T18:57:11.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.875+0000 7f2da3fff700 1 -- 172.21.15.146:0/106767418 --> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2da40aa6c0 con 0x7f2d90070cd0 2024-09-13T18:57:11.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.883+0000 7f2d8b7fe700 1 -- 172.21.15.146:0/106767418 <== osd.7 v2:172.21.15.154:6817/3339245911 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2d98005a70 con 0x7f2d90070cd0 2024-09-13T18:57:11.897 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-13T18:57:11.897 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:57:11.898 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-13T18:57:11.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.883+0000 7f2da3fff700 1 -- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] conn(0x7f2d90070cd0 msgr2=0x7f2d90073170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.883+0000 7f2da3fff700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:6817/3339245911,v1:172.21.15.154:6819/3339245911] conn(0x7f2d90070cd0 0x7f2d90073170 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.883+0000 7f2da3fff700 1 -- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2d90066730 msgr2=0x7f2d90068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.883+0000 7f2da3fff700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f2d90066730 0x7f2d90068bf0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2d940015f0 tx=0x7f2d9400a000 comp rx=0 tx=0).stop 2024-09-13T18:57:11.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.883+0000 7f2da3fff700 1 -- 172.21.15.146:0/106767418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2da4159630 msgr2=0x7f2da414ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:11.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.883+0000 7f2da3fff700 1 --2- 172.21.15.146:0/106767418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2da4159630 0x7f2da414ded0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2da4071a40 tx=0x7f2d9c0019e0 comp rx=0 tx=0).stop 2024-09-13T18:57:11.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.887+0000 7f2da37fe700 1 -- 172.21.15.146:0/106767418 reap_dead start 2024-09-13T18:57:11.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.887+0000 7f2da3fff700 1 -- 172.21.15.146:0/106767418 shutdown_connections 2024-09-13T18:57:11.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.887+0000 7f2da3fff700 1 -- 172.21.15.146:0/106767418 >> 172.21.15.146:0/106767418 conn(0x7f2da4066090 msgr2=0x7f2da40adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:11.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.887+0000 7f2da3fff700 1 -- 172.21.15.146:0/106767418 shutdown_connections 2024-09-13T18:57:11.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:11.887+0000 7f2da3fff700 1 -- 172.21.15.146:0/106767418 wait complete. 2024-09-13T18:57:11.918 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T18:57:12.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.079+0000 7fac1e3a9700 1 Processor -- start 2024-09-13T18:57:12.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.079+0000 7fac1e3a9700 1 -- start start 2024-09-13T18:57:12.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac1e3a9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac1814f750 0x7fac1806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:12.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac1e3a9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac1806f5c0 0x7fac1806fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:12.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac1e3a9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac1806ff60 con 0x7fac180a9490 2024-09-13T18:57:12.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac1e3a9700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1814fb30 con 0x7fac1814f750 2024-09-13T18:57:12.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac1e3a9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac18155580 con 0x7fac1806f5c0 2024-09-13T18:57:12.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac17fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac1814f750 0x7fac1806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:12.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac17fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac1814f750 0x7fac1806eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:35890/0 (socket says 172.21.15.146:35890) 2024-09-13T18:57:12.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac17fff700 1 -- 172.21.15.146:0/2160764669 learned_addr learned my addr 172.21.15.146:0/2160764669 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:12.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac177fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fac180a9490 0x7fac180cb120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:41522/0 (socket says 172.21.15.146:41522) 2024-09-13T18:57:12.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac17fff700 1 -- 172.21.15.146:0/2160764669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac1806f5c0 msgr2=0x7fac1806fa20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:57:12.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac17fff700 1 --2- 172.21.15.146:0/2160764669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac1806f5c0 0x7fac1806fa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac17fff700 1 -- 172.21.15.146:0/2160764669 >> v1:172.21.15.154:6789/0 conn(0x7fac180a9490 legacy=0x7fac180cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:12.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac17fff700 1 -- 172.21.15.146:0/2160764669 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac18155860 con 0x7fac1814f750 2024-09-13T18:57:12.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.083+0000 7fac17fff700 1 --2- 172.21.15.146:0/2160764669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac1814f750 0x7fac1806eea0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fac1000d370 tx=0x7fac1000d680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81ad302b9bb87fa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:12.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.099+0000 7fac16ffd700 1 -- 172.21.15.146:0/2160764669 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10010070 con 0x7fac1814f750 2024-09-13T18:57:12.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.099+0000 7fac16ffd700 1 -- 172.21.15.146:0/2160764669 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1000dcd0 con 0x7fac1814f750 2024-09-13T18:57:12.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.099+0000 7fac1e3a9700 1 -- 172.21.15.146:0/2160764669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac1814f750 msgr2=0x7fac1806eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:12.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.099+0000 7fac1e3a9700 1 --2- 172.21.15.146:0/2160764669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac1814f750 0x7fac1806eea0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fac1000d370 tx=0x7fac1000d680 comp rx=0 tx=0).stop 2024-09-13T18:57:12.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.107+0000 7fac1e3a9700 1 -- 172.21.15.146:0/2160764669 shutdown_connections 2024-09-13T18:57:12.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.107+0000 7fac1e3a9700 1 --2- 172.21.15.146:0/2160764669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac1806f5c0 0x7fac1806fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.107+0000 7fac1e3a9700 1 --2- 172.21.15.146:0/2160764669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac1814f750 0x7fac1806eea0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.107+0000 7fac1e3a9700 1 -- 172.21.15.146:0/2160764669 >> 172.21.15.146:0/2160764669 conn(0x7fac18066090 msgr2=0x7fac18066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:12.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.107+0000 7fac1e3a9700 1 -- 172.21.15.146:0/2160764669 shutdown_connections 2024-09-13T18:57:12.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.107+0000 7fac1e3a9700 1 -- 172.21.15.146:0/2160764669 wait complete. 2024-09-13T18:57:12.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1e3a9700 1 Processor -- start 2024-09-13T18:57:12.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1e3a9700 1 -- start start 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1e3a9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac1806f5c0 0x7fac18200060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1e3a9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac180a9490 0x7fac181fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1e3a9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac1814f750 0x7fac180acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1e3a9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac18158260 con 0x7fac180a9490 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1e3a9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac18157ed0 con 0x7fac1814f750 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1e3a9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac181f9c00 con 0x7fac1806f5c0 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac17fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac1806f5c0 0x7fac18200060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1c946700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac1814f750 0x7fac180acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac17fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac1806f5c0 0x7fac18200060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:46562/0 (socket says 172.21.15.146:46562) 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1c946700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac1814f750 0x7fac180acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.146:35902/0 (socket says 172.21.15.146:35902) 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac17fff700 1 -- 172.21.15.146:0/850787243 learned_addr learned my addr 172.21.15.146:0/850787243 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1c946700 1 -- 172.21.15.146:0/850787243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac1806f5c0 msgr2=0x7fac18200060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1c946700 1 --2- 172.21.15.146:0/850787243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac1806f5c0 0x7fac18200060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1c946700 1 -- 172.21.15.146:0/850787243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac180a9490 msgr2=0x7fac181fcab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:57:12.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1c946700 1 --2- 172.21.15.146:0/850787243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac180a9490 0x7fac181fcab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1c946700 1 -- 172.21.15.146:0/850787243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1000d050 con 0x7fac1814f750 2024-09-13T18:57:12.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.111+0000 7fac1c946700 1 --2- 172.21.15.146:0/850787243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac1814f750 0x7fac180acbc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00b410 tx=0x7fac0c00b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:12.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.115+0000 7fac14ff9700 1 -- 172.21.15.146:0/850787243 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c011830 con 0x7fac1814f750 2024-09-13T18:57:12.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.115+0000 7fac14ff9700 1 -- 172.21.15.146:0/850787243 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0c00be90 con 0x7fac1814f750 2024-09-13T18:57:12.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.115+0000 7fac1e3a9700 1 -- 172.21.15.146:0/850787243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac181593b0 con 0x7fac1814f750 2024-09-13T18:57:12.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.115+0000 7fac1e3a9700 1 -- 172.21.15.146:0/850787243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac182005f0 con 0x7fac1814f750 2024-09-13T18:57:12.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.115+0000 7fac14ff9700 1 -- 172.21.15.146:0/850787243 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c011bf0 con 0x7fac1814f750 2024-09-13T18:57:12.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.115+0000 7fac1e3a9700 1 -- 172.21.15.146:0/850787243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf8005310 con 0x7fac1814f750 2024-09-13T18:57:12.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.115+0000 7fac14ff9700 1 -- 172.21.15.146:0/850787243 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fac0c011d50 con 0x7fac1814f750 2024-09-13T18:57:12.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.119+0000 7fac14ff9700 1 --2- 172.21.15.146:0/850787243 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fac04066780 0x7fac04068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:12.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.119+0000 7fac14ff9700 1 -- 172.21.15.146:0/850787243 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fac0c0847c0 con 0x7fac1814f750 2024-09-13T18:57:12.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.119+0000 7fac17fff700 1 --2- 172.21.15.146:0/850787243 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fac04066780 0x7fac04068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:12.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.123+0000 7fac17fff700 1 --2- 172.21.15.146:0/850787243 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fac04066780 0x7fac04068c40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fac1000d6c0 tx=0x7fac10012000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:12.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.127+0000 7fac14ff9700 1 -- 172.21.15.146:0/850787243 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fac0c051380 con 0x7fac1814f750 2024-09-13T18:57:12.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.407+0000 7fac1e3a9700 1 -- 172.21.15.146:0/850787243 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fabf8000bc0 con 0x7fac04066780 2024-09-13T18:57:12.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.411+0000 7fac14ff9700 1 -- 172.21.15.146:0/850787243 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fabf8000bc0 con 0x7fac04066780 2024-09-13T18:57:12.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.411+0000 7fac1e3a9700 1 -- 172.21.15.146:0/850787243 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fac04066780 msgr2=0x7fac04068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:12.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.411+0000 7fac1e3a9700 1 --2- 172.21.15.146:0/850787243 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fac04066780 0x7fac04068c40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fac1000d6c0 tx=0x7fac10012000 comp rx=0 tx=0).stop 2024-09-13T18:57:12.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.411+0000 7fac1e3a9700 1 -- 172.21.15.146:0/850787243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac1814f750 msgr2=0x7fac180acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:12.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.411+0000 7fac1e3a9700 1 --2- 172.21.15.146:0/850787243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac1814f750 0x7fac180acbc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00b410 tx=0x7fac0c00b720 comp rx=0 tx=0).stop 2024-09-13T18:57:12.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.411+0000 7fac1e3a9700 1 -- 172.21.15.146:0/850787243 shutdown_connections 2024-09-13T18:57:12.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.411+0000 7fac1e3a9700 1 --2- 172.21.15.146:0/850787243 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fac04066780 0x7fac04068c40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.411+0000 7fac1e3a9700 1 --2- 172.21.15.146:0/850787243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac1814f750 0x7fac180acbc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.411+0000 7fac1e3a9700 1 --2- 172.21.15.146:0/850787243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac180a9490 0x7fac181fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.411+0000 7fac1e3a9700 1 --2- 172.21.15.146:0/850787243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac1806f5c0 0x7fac18200060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.411+0000 7fac1e3a9700 1 -- 172.21.15.146:0/850787243 >> 172.21.15.146:0/850787243 conn(0x7fac18066090 msgr2=0x7fac180ade80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:12.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.411+0000 7fac1e3a9700 1 -- 172.21.15.146:0/850787243 shutdown_connections 2024-09-13T18:57:12.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.411+0000 7fac1e3a9700 1 -- 172.21.15.146:0/850787243 wait complete. 2024-09-13T18:57:12.426 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T18:57:12.436 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:57:12.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.523+0000 7f8e63044700 1 Processor -- start 2024-09-13T18:57:12.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.523+0000 7f8e63044700 1 -- start start 2024-09-13T18:57:12.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.523+0000 7f8e63044700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8e5c0cb2b0 0x7f8e5c0723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:12.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.523+0000 7f8e63044700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8e5c0a9840 0x7f8e5c15e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:12.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.523+0000 7f8e63044700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e5c161550 con 0x7f8e5c0caed0 2024-09-13T18:57:12.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.523+0000 7f8e63044700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5c072a30 con 0x7f8e5c0cb2b0 2024-09-13T18:57:12.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.523+0000 7f8e63044700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5c0aa800 con 0x7f8e5c0a9840 2024-09-13T18:57:12.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.523+0000 7f8e61841700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8e5c0cb2b0 0x7f8e5c0723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:12.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.523+0000 7f8e61841700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8e5c0cb2b0 0x7f8e5c0723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49232/0 (socket says 172.21.15.146:49232) 2024-09-13T18:57:12.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.523+0000 7f8e61841700 1 -- 172.21.15.146:0/1550371110 learned_addr learned my addr 172.21.15.146:0/1550371110 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:12.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.523+0000 7f8e62843700 1 --2- 172.21.15.146:0/1550371110 >> v2:172.21.15.154:3300/0 conn(0x7f8e5c0a9840 0x7f8e5c15e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:12.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e61841700 1 -- 172.21.15.146:0/1550371110 >> v2:172.21.15.154:3300/0 conn(0x7f8e5c0a9840 msgr2=0x7f8e5c15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:12.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e61841700 1 --2- 172.21.15.146:0/1550371110 >> v2:172.21.15.154:3300/0 conn(0x7f8e5c0a9840 0x7f8e5c15e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e61841700 1 -- 172.21.15.146:0/1550371110 >> v1:172.21.15.154:6789/0 conn(0x7f8e5c0caed0 legacy=0x7f8e5c1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:12.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e61841700 1 -- 172.21.15.146:0/1550371110 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c161220 con 0x7f8e5c0cb2b0 2024-09-13T18:57:12.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e61841700 1 --2- 172.21.15.146:0/1550371110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5c0cb2b0 0x7f8e5c0723f0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0b0180 tx=0x7f8e4c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94346bfe2a6c94fc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:12.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e61040700 1 -- 172.21.15.146:0/1550371110 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c011830 con 0x7f8e5c0cb2b0 2024-09-13T18:57:12.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e61040700 1 -- 172.21.15.146:0/1550371110 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4c011990 con 0x7f8e5c0cb2b0 2024-09-13T18:57:12.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e61040700 1 -- 172.21.15.146:0/1550371110 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c011d20 con 0x7f8e5c0cb2b0 2024-09-13T18:57:12.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e63044700 1 -- 172.21.15.146:0/1550371110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5c0cb2b0 msgr2=0x7f8e5c0723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:12.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e63044700 1 --2- 172.21.15.146:0/1550371110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5c0cb2b0 0x7f8e5c0723f0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0b0180 tx=0x7f8e4c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e63044700 1 -- 172.21.15.146:0/1550371110 shutdown_connections 2024-09-13T18:57:12.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e63044700 1 --2- 172.21.15.146:0/1550371110 >> v2:172.21.15.154:3300/0 conn(0x7f8e5c0a9840 0x7f8e5c15e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e63044700 1 --2- 172.21.15.146:0/1550371110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5c0cb2b0 0x7f8e5c0723f0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e63044700 1 -- 172.21.15.146:0/1550371110 >> 172.21.15.146:0/1550371110 conn(0x7f8e5c066090 msgr2=0x7f8e5c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:12.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e63044700 1 -- 172.21.15.146:0/1550371110 shutdown_connections 2024-09-13T18:57:12.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.527+0000 7f8e63044700 1 -- 172.21.15.146:0/1550371110 wait complete. 2024-09-13T18:57:12.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e63044700 1 Processor -- start 2024-09-13T18:57:12.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e63044700 1 -- start start 2024-09-13T18:57:12.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e63044700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e5c0a9840 0x7f8e5c0adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:12.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e63044700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5c0caed0 0x7f8e5c0ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:12.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e63044700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e5c0cb2b0 0x7f8e5c0af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:12.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e63044700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0ab4d0 con 0x7f8e5c0caed0 2024-09-13T18:57:12.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e63044700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0afec0 con 0x7f8e5c0a9840 2024-09-13T18:57:12.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e63044700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0af900 con 0x7f8e5c0cb2b0 2024-09-13T18:57:12.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e61841700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5c0caed0 0x7f8e5c0ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:12.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e61841700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5c0caed0 0x7f8e5c0ab8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49244/0 (socket says 172.21.15.146:49244) 2024-09-13T18:57:12.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e61841700 1 -- 172.21.15.146:0/1775121448 learned_addr learned my addr 172.21.15.146:0/1775121448 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:12.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e62042700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e5c0a9840 0x7f8e5c0adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:12.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e62843700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e5c0cb2b0 0x7f8e5c0af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:12.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.531+0000 7f8e61841700 1 -- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e5c0cb2b0 msgr2=0x7f8e5c0af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:12.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e61841700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e5c0cb2b0 0x7f8e5c0af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e61841700 1 -- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e5c0a9840 msgr2=0x7f8e5c0adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:12.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e61841700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e5c0a9840 0x7f8e5c0adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e61841700 1 -- 172.21.15.146:0/1775121448 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c009570 con 0x7f8e5c0caed0 2024-09-13T18:57:12.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e62042700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e5c0a9840 0x7f8e5c0adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:57:12.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e62843700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e5c0cb2b0 0x7f8e5c0af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:57:12.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e61841700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5c0caed0 0x7f8e5c0ab8f0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c013000 tx=0x7f8e4c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:12.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e52ffd700 1 -- 172.21.15.146:0/1775121448 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c004550 con 0x7f8e5c0caed0 2024-09-13T18:57:12.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e52ffd700 1 -- 172.21.15.146:0/1775121448 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4c01f070 con 0x7f8e5c0caed0 2024-09-13T18:57:12.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e52ffd700 1 -- 172.21.15.146:0/1775121448 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c003900 con 0x7f8e5c0caed0 2024-09-13T18:57:12.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e63044700 1 -- 172.21.15.146:0/1775121448 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c0afbe0 con 0x7f8e5c0caed0 2024-09-13T18:57:12.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e63044700 1 -- 172.21.15.146:0/1775121448 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c0ab0f0 con 0x7f8e5c0caed0 2024-09-13T18:57:12.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e63044700 1 -- 172.21.15.146:0/1775121448 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c160e00 con 0x7f8e5c0caed0 2024-09-13T18:57:12.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e52ffd700 1 -- 172.21.15.146:0/1775121448 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8e4c003a60 con 0x7f8e5c0caed0 2024-09-13T18:57:12.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e52ffd700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e40066730 0x7f8e40068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:12.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.535+0000 7f8e52ffd700 1 -- 172.21.15.146:0/1775121448 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f8e4c021070 con 0x7f8e5c0caed0 2024-09-13T18:57:12.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.539+0000 7f8e62042700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e40066730 0x7f8e40068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:12.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.539+0000 7f8e62042700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e40066730 0x7f8e40068bf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8e58000f60 tx=0x7f8e58009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:12.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.543+0000 7f8e52ffd700 1 -- 172.21.15.146:0/1775121448 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8e4c0529e0 con 0x7f8e5c0caed0 2024-09-13T18:57:12.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.803+0000 7f8e63044700 1 -- 172.21.15.146:0/1775121448 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e5c071c50 con 0x7f8e40066730 2024-09-13T18:57:12.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.819+0000 7f8e52ffd700 1 -- 172.21.15.146:0/1775121448 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+368374 (secure 0 0 0) 0x7f8e5c071c50 con 0x7f8e40066730 2024-09-13T18:57:12.837 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:57:12.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.823+0000 7f8e63044700 1 -- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e40066730 msgr2=0x7f8e40068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:12.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.823+0000 7f8e63044700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e40066730 0x7f8e40068bf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8e58000f60 tx=0x7f8e58009040 comp rx=0 tx=0).stop 2024-09-13T18:57:12.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.823+0000 7f8e63044700 1 -- 172.21.15.146:0/1775121448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5c0caed0 msgr2=0x7f8e5c0ab8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:12.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.823+0000 7f8e63044700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5c0caed0 0x7f8e5c0ab8f0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c013000 tx=0x7f8e4c004770 comp rx=0 tx=0).stop 2024-09-13T18:57:12.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.823+0000 7f8e63044700 1 -- 172.21.15.146:0/1775121448 shutdown_connections 2024-09-13T18:57:12.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.823+0000 7f8e63044700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f8e40066730 0x7f8e40068bf0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.823+0000 7f8e63044700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e5c0cb2b0 0x7f8e5c0af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.823+0000 7f8e63044700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5c0caed0 0x7f8e5c0ab8f0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.823+0000 7f8e63044700 1 --2- 172.21.15.146:0/1775121448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e5c0a9840 0x7f8e5c0adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:12.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.823+0000 7f8e63044700 1 -- 172.21.15.146:0/1775121448 >> 172.21.15.146:0/1775121448 conn(0x7f8e5c066090 msgr2=0x7f8e5c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:12.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.823+0000 7f8e63044700 1 -- 172.21.15.146:0/1775121448 shutdown_connections 2024-09-13T18:57:12.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:12.823+0000 7f8e63044700 1 -- 172.21.15.146:0/1775121448 wait complete. 2024-09-13T18:57:12.842 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:57:12.852 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":959,"stamp":"2024-09-13T18:57:11.598039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7657605255,"num_objects":2063,"num_object_clones":0,"num_object_copies":8120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2063,"num_whiteouts":0,"num_read":2275,"num_read_kb":52180,"num_write":805499,"num_write_kb":16339539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178331,"ondisk_log_size":178331,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":14237184,"kb_used_data":6801336,"kb_used_omap":12,"kb_used_meta":7434419,"kb_avail":740737536,"statfs":{"total":773094113280,"available":758515236864,"internally_reserved":0,"allocated":6964568064,"data_stored":16019518030,"data_compressed":763696415,"data_compressed_allocated":1072291840,"data_compressed_original":10129309696,"omap_allocated":12741,"internal_metadata":7612845627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,5,8,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4974554057,"num_objects":-1208,"num_object_clones":0,"num_object_copies":-4814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1208,"num_whiteouts":0,"num_read":2,"num_read_kb":3,"num_write":2096,"num_write_kb":3594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004150"},"pg_stats":[{"pgid":"2.1f","version":"43'465","reported_seq":544,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:02.861215+0000","last_change":"2024-09-13T18:26:31.972797+0000","last_active":"2024-09-13T18:57:02.861215+0000","last_peered":"2024-09-13T18:57:02.861215+0000","last_clean":"2024-09-13T18:57:02.861215+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:57:02.861215+0000","last_undegraded":"2024-09-13T18:57:02.861215+0000","last_fullsized":"2024-09-13T18:57:02.861215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3824255,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":428,"num_write":456,"num_write_kb":3991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"43'5","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:09.844882+0000","last_change":"2024-09-13T18:26:32.349901+0000","last_active":"2024-09-13T18:57:09.844882+0000","last_peered":"2024-09-13T18:57:09.844882+0000","last_clean":"2024-09-13T18:57:09.844882+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:57:09.844882+0000","last_undegraded":"2024-09-13T18:57:09.844882+0000","last_fullsized":"2024-09-13T18:57:09.844882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"43'5750","reported_seq":5829,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.412151+0000","last_change":"2024-09-13T18:26:36.407579+0000","last_active":"2024-09-13T18:57:10.412151+0000","last_peered":"2024-09-13T18:57:10.412151+0000","last_clean":"2024-09-13T18:57:10.412151+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:57:10.412151+0000","last_undegraded":"2024-09-13T18:57:10.412151+0000","last_fullsized":"2024-09-13T18:57:10.412151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5750,"log_dups_size":0,"ondisk_log_size":5750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201885121,"num_objects":56,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":26711,"num_write_kb":537654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"41'12","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:50.507126+0000","last_change":"2024-09-13T18:26:31.980716+0000","last_active":"2024-09-13T18:56:50.507126+0000","last_peered":"2024-09-13T18:56:50.507126+0000","last_clean":"2024-09-13T18:56:50.507126+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:56:50.507126+0000","last_undegraded":"2024-09-13T18:56:50.507126+0000","last_fullsized":"2024-09-13T18:56:50.507126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"43'6","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:07.081139+0000","last_change":"2024-09-13T18:26:32.342690+0000","last_active":"2024-09-13T18:57:07.081139+0000","last_peered":"2024-09-13T18:57:07.081139+0000","last_clean":"2024-09-13T18:57:07.081139+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:57:07.081139+0000","last_undegraded":"2024-09-13T18:57:07.081139+0000","last_fullsized":"2024-09-13T18:57:07.081139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"43'5794","reported_seq":5866,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.600074+0000","last_change":"2024-09-13T18:26:36.396545+0000","last_active":"2024-09-13T18:57:10.600074+0000","last_peered":"2024-09-13T18:57:10.600074+0000","last_clean":"2024-09-13T18:57:10.600074+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:57:10.600074+0000","last_undegraded":"2024-09-13T18:57:10.600074+0000","last_fullsized":"2024-09-13T18:57:10.600074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5794,"log_dups_size":0,"ondisk_log_size":5794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243353372,"num_objects":62,"num_object_clones":0,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":26721,"num_write_kb":545346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"43'23","reported_seq":83,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:35.970133+0000","last_change":"2024-09-13T18:26:31.977823+0000","last_active":"2024-09-13T18:56:35.970133+0000","last_peered":"2024-09-13T18:56:35.970133+0000","last_clean":"2024-09-13T18:56:35.970133+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:56:35.970133+0000","last_undegraded":"2024-09-13T18:56:35.970133+0000","last_fullsized":"2024-09-13T18:56:35.970133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:46.956527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":65,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"29'8","reported_seq":46,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.973176+0000","last_change":"2024-09-13T18:27:22.609917+0000","last_active":"2024-09-13T18:55:25.973176+0000","last_peered":"2024-09-13T18:55:25.973176+0000","last_clean":"2024-09-13T18:55:25.973176+0000","last_became_active":"2024-09-13T18:27:22.609625+0000","last_became_peered":"2024-09-13T18:27:22.609625+0000","last_unstale":"2024-09-13T18:55:25.973176+0000","last_undegraded":"2024-09-13T18:55:25.973176+0000","last_fullsized":"2024-09-13T18:55:25.973176+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"43'4872","reported_seq":4935,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.480959+0000","last_change":"2024-09-13T18:26:36.406675+0000","last_active":"2024-09-13T18:57:10.480959+0000","last_peered":"2024-09-13T18:57:10.480959+0000","last_clean":"2024-09-13T18:57:10.480959+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:57:10.480959+0000","last_undegraded":"2024-09-13T18:57:10.480959+0000","last_fullsized":"2024-09-13T18:57:10.480959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4872,"log_dups_size":0,"ondisk_log_size":4872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205554406,"num_objects":57,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21567,"num_write_kb":467229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"43'8","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:05.494087+0000","last_change":"2024-09-13T18:26:31.979860+0000","last_active":"2024-09-13T18:57:05.494087+0000","last_peered":"2024-09-13T18:57:05.494087+0000","last_clean":"2024-09-13T18:57:05.494087+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:57:05.494087+0000","last_undegraded":"2024-09-13T18:57:05.494087+0000","last_fullsized":"2024-09-13T18:57:05.494087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:45.323631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":12,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"43'11","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:05.942835+0000","last_change":"2024-09-13T18:26:32.351494+0000","last_active":"2024-09-13T18:57:05.942835+0000","last_peered":"2024-09-13T18:57:05.942835+0000","last_clean":"2024-09-13T18:57:05.942835+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:57:05.942835+0000","last_undegraded":"2024-09-13T18:57:05.942835+0000","last_fullsized":"2024-09-13T18:57:05.942835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:52.804526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"43'4786","reported_seq":4849,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.614534+0000","last_change":"2024-09-13T18:26:36.398763+0000","last_active":"2024-09-13T18:57:10.614534+0000","last_peered":"2024-09-13T18:57:10.614534+0000","last_clean":"2024-09-13T18:57:10.614534+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:57:10.614534+0000","last_undegraded":"2024-09-13T18:57:10.614534+0000","last_fullsized":"2024-09-13T18:57:10.614534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4786,"log_dups_size":0,"ondisk_log_size":4786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192952122,"num_objects":49,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21575,"num_write_kb":434281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"43'8","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:05.665964+0000","last_change":"2024-09-13T18:26:32.360976+0000","last_active":"2024-09-13T18:57:05.665964+0000","last_peered":"2024-09-13T18:57:05.665964+0000","last_clean":"2024-09-13T18:57:05.665964+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:57:05.665964+0000","last_undegraded":"2024-09-13T18:57:05.665964+0000","last_fullsized":"2024-09-13T18:57:05.665964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:53:52.373515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"43'1217","reported_seq":1289,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.163136+0000","last_change":"2024-09-13T18:26:31.977891+0000","last_active":"2024-09-13T18:55:53.163136+0000","last_peered":"2024-09-13T18:55:53.163136+0000","last_clean":"2024-09-13T18:55:53.163136+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:55:53.163136+0000","last_undegraded":"2024-09-13T18:55:53.163136+0000","last_fullsized":"2024-09-13T18:55:53.163136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:01.453094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":10235,"num_write":1211,"num_write_kb":4423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"43'5336","reported_seq":5403,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.609301+0000","last_change":"2024-09-13T18:26:36.397484+0000","last_active":"2024-09-13T18:57:10.609301+0000","last_peered":"2024-09-13T18:57:10.609301+0000","last_clean":"2024-09-13T18:57:10.609301+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:57:10.609301+0000","last_undegraded":"2024-09-13T18:57:10.609301+0000","last_fullsized":"2024-09-13T18:57:10.609301+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234910210,"num_objects":63,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":24065,"num_write_kb":505876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"29'8","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:47.795795+0000","last_change":"2024-09-13T18:26:32.345564+0000","last_active":"2024-09-13T18:55:47.795795+0000","last_peered":"2024-09-13T18:55:47.795795+0000","last_clean":"2024-09-13T18:55:47.795795+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:55:47.795795+0000","last_undegraded":"2024-09-13T18:55:47.795795+0000","last_fullsized":"2024-09-13T18:55:47.795795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"43'10","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.158591+0000","last_change":"2024-09-13T18:26:31.974107+0000","last_active":"2024-09-13T18:55:53.158591+0000","last_peered":"2024-09-13T18:55:53.158591+0000","last_clean":"2024-09-13T18:55:53.158591+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:55:53.158591+0000","last_undegraded":"2024-09-13T18:55:53.158591+0000","last_fullsized":"2024-09-13T18:55:53.158591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:37.547480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"43'5320","reported_seq":5390,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.662908+0000","last_change":"2024-09-13T18:26:36.405986+0000","last_active":"2024-09-13T18:57:10.662908+0000","last_peered":"2024-09-13T18:57:10.662908+0000","last_clean":"2024-09-13T18:57:10.662908+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:57:10.662908+0000","last_undegraded":"2024-09-13T18:57:10.662908+0000","last_fullsized":"2024-09-13T18:57:10.662908+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5320,"log_dups_size":0,"ondisk_log_size":5320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226506917,"num_objects":61,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":24351,"num_write_kb":498911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"43'9","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:57.445541+0000","last_change":"2024-09-13T18:27:22.609984+0000","last_active":"2024-09-13T18:56:57.445541+0000","last_peered":"2024-09-13T18:56:57.445541+0000","last_clean":"2024-09-13T18:56:57.445541+0000","last_became_active":"2024-09-13T18:27:22.609642+0000","last_became_peered":"2024-09-13T18:27:22.609642+0000","last_unstale":"2024-09-13T18:56:57.445541+0000","last_undegraded":"2024-09-13T18:56:57.445541+0000","last_fullsized":"2024-09-13T18:56:57.445541+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.161825+0000","last_change":"2024-09-13T18:26:31.968895+0000","last_active":"2024-09-13T18:55:53.161825+0000","last_peered":"2024-09-13T18:55:53.161825+0000","last_clean":"2024-09-13T18:55:53.161825+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:55:53.161825+0000","last_undegraded":"2024-09-13T18:55:53.161825+0000","last_fullsized":"2024-09-13T18:55:53.161825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:15.525097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"43'6873","reported_seq":6943,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.232382+0000","last_change":"2024-09-13T18:27:22.610932+0000","last_active":"2024-09-13T18:57:10.232382+0000","last_peered":"2024-09-13T18:57:10.232382+0000","last_clean":"2024-09-13T18:57:10.232382+0000","last_became_active":"2024-09-13T18:27:22.610527+0000","last_became_peered":"2024-09-13T18:27:22.610527+0000","last_unstale":"2024-09-13T18:57:10.232382+0000","last_undegraded":"2024-09-13T18:57:10.232382+0000","last_fullsized":"2024-09-13T18:57:10.232382+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6873,"log_dups_size":0,"ondisk_log_size":6873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289422424,"num_objects":73,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":31737,"num_write_kb":622755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"43'6","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:07.455127+0000","last_change":"2024-09-13T18:26:32.360406+0000","last_active":"2024-09-13T18:57:07.455127+0000","last_peered":"2024-09-13T18:57:07.455127+0000","last_clean":"2024-09-13T18:57:07.455127+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:57:07.455127+0000","last_undegraded":"2024-09-13T18:57:07.455127+0000","last_fullsized":"2024-09-13T18:57:07.455127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"28'4","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.171564+0000","last_change":"2024-09-13T18:26:31.976872+0000","last_active":"2024-09-13T18:55:53.171564+0000","last_peered":"2024-09-13T18:55:53.171564+0000","last_clean":"2024-09-13T18:55:53.171564+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:55:53.171564+0000","last_undegraded":"2024-09-13T18:55:53.171564+0000","last_fullsized":"2024-09-13T18:55:53.171564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:26.163029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"43'5301","reported_seq":5379,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.673001+0000","last_change":"2024-09-13T18:27:22.648736+0000","last_active":"2024-09-13T18:57:10.673001+0000","last_peered":"2024-09-13T18:57:10.673001+0000","last_clean":"2024-09-13T18:57:10.673001+0000","last_became_active":"2024-09-13T18:27:22.648054+0000","last_became_peered":"2024-09-13T18:27:22.648054+0000","last_unstale":"2024-09-13T18:57:10.673001+0000","last_undegraded":"2024-09-13T18:57:10.673001+0000","last_fullsized":"2024-09-13T18:57:10.673001+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5301,"log_dups_size":0,"ondisk_log_size":5301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222418564,"num_objects":61,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":23564,"num_write_kb":471297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"43'15","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:06.456677+0000","last_change":"2024-09-13T18:26:32.343915+0000","last_active":"2024-09-13T18:57:06.456677+0000","last_peered":"2024-09-13T18:57:06.456677+0000","last_clean":"2024-09-13T18:57:06.456677+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:57:06.456677+0000","last_undegraded":"2024-09-13T18:57:06.456677+0000","last_fullsized":"2024-09-13T18:57:06.456677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"28'5","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.162612+0000","last_change":"2024-09-13T18:26:31.979944+0000","last_active":"2024-09-13T18:55:53.162612+0000","last_peered":"2024-09-13T18:55:53.162612+0000","last_clean":"2024-09-13T18:55:53.162612+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:55:53.162612+0000","last_undegraded":"2024-09-13T18:55:53.162612+0000","last_fullsized":"2024-09-13T18:55:53.162612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:21.721921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"43'5293","reported_seq":5365,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.619900+0000","last_change":"2024-09-13T18:26:36.410028+0000","last_active":"2024-09-13T18:57:10.619900+0000","last_peered":"2024-09-13T18:57:10.619900+0000","last_clean":"2024-09-13T18:57:10.619900+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:57:10.619900+0000","last_undegraded":"2024-09-13T18:57:10.619900+0000","last_fullsized":"2024-09-13T18:57:10.619900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5293,"log_dups_size":0,"ondisk_log_size":5293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230727697,"num_objects":59,"num_object_clones":0,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":24393,"num_write_kb":507343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"43'10","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:08.224577+0000","last_change":"2024-09-13T18:26:32.358980+0000","last_active":"2024-09-13T18:57:08.224577+0000","last_peered":"2024-09-13T18:57:08.224577+0000","last_clean":"2024-09-13T18:57:08.224577+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:57:08.224577+0000","last_undegraded":"2024-09-13T18:57:08.224577+0000","last_fullsized":"2024-09-13T18:57:08.224577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"43'284","reported_seq":346,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:57.831331+0000","last_change":"2024-09-13T18:26:31.968972+0000","last_active":"2024-09-13T18:56:57.831331+0000","last_peered":"2024-09-13T18:56:57.831331+0000","last_clean":"2024-09-13T18:56:57.831331+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:56:57.831331+0000","last_undegraded":"2024-09-13T18:56:57.831331+0000","last_fullsized":"2024-09-13T18:56:57.831331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:56.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":52,"num_write":624,"num_write_kb":212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"43'5996","reported_seq":6078,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.493422+0000","last_change":"2024-09-13T18:27:22.645275+0000","last_active":"2024-09-13T18:57:10.493422+0000","last_peered":"2024-09-13T18:57:10.493422+0000","last_clean":"2024-09-13T18:57:10.493422+0000","last_became_active":"2024-09-13T18:27:22.644681+0000","last_became_peered":"2024-09-13T18:27:22.644681+0000","last_unstale":"2024-09-13T18:57:10.493422+0000","last_undegraded":"2024-09-13T18:57:10.493422+0000","last_fullsized":"2024-09-13T18:57:10.493422+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5996,"log_dups_size":0,"ondisk_log_size":5996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239146876,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":27838,"num_write_kb":553947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"43'6","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:50.152562+0000","last_change":"2024-09-13T18:26:32.342823+0000","last_active":"2024-09-13T18:56:50.152562+0000","last_peered":"2024-09-13T18:56:50.152562+0000","last_clean":"2024-09-13T18:56:50.152562+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:56:50.152562+0000","last_undegraded":"2024-09-13T18:56:50.152562+0000","last_fullsized":"2024-09-13T18:56:50.152562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:28.051252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"43'2","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:46.406661+0000","last_change":"2024-09-13T18:26:31.978976+0000","last_active":"2024-09-13T18:55:46.406661+0000","last_peered":"2024-09-13T18:55:46.406661+0000","last_clean":"2024-09-13T18:55:46.406661+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:55:46.406661+0000","last_undegraded":"2024-09-13T18:55:46.406661+0000","last_fullsized":"2024-09-13T18:55:46.406661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:11.279557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"43'4673","reported_seq":4737,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:08.026588+0000","last_change":"2024-09-13T18:26:36.408844+0000","last_active":"2024-09-13T18:57:08.026588+0000","last_peered":"2024-09-13T18:57:08.026588+0000","last_clean":"2024-09-13T18:57:08.026588+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:57:08.026588+0000","last_undegraded":"2024-09-13T18:57:08.026588+0000","last_fullsized":"2024-09-13T18:57:08.026588+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4673,"log_dups_size":0,"ondisk_log_size":4673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226496217,"num_objects":56,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":22379,"num_write_kb":450115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"28'7","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.969209+0000","last_change":"2024-09-13T18:26:32.355038+0000","last_active":"2024-09-13T18:55:25.969209+0000","last_peered":"2024-09-13T18:55:25.969209+0000","last_clean":"2024-09-13T18:55:25.969209+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:55:25.969209+0000","last_undegraded":"2024-09-13T18:55:25.969209+0000","last_fullsized":"2024-09-13T18:55:25.969209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.159231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:26.007762+0000","last_change":"2024-09-13T18:26:31.972931+0000","last_active":"2024-09-13T18:55:26.007762+0000","last_peered":"2024-09-13T18:55:26.007762+0000","last_clean":"2024-09-13T18:55:26.007762+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:55:26.007762+0000","last_undegraded":"2024-09-13T18:55:26.007762+0000","last_fullsized":"2024-09-13T18:55:26.007762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:49:05.646924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"43'6040","reported_seq":6109,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:06.368049+0000","last_change":"2024-09-13T18:26:36.398876+0000","last_active":"2024-09-13T18:57:06.368049+0000","last_peered":"2024-09-13T18:57:06.368049+0000","last_clean":"2024-09-13T18:57:06.368049+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:57:06.368049+0000","last_undegraded":"2024-09-13T18:57:06.368049+0000","last_fullsized":"2024-09-13T18:57:06.368049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6040,"log_dups_size":0,"ondisk_log_size":6040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:56.918239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281044135,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":27998,"num_write_kb":562911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"28'6","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:46.631594+0000","last_change":"2024-09-13T18:26:32.353111+0000","last_active":"2024-09-13T18:55:46.631594+0000","last_peered":"2024-09-13T18:55:46.631594+0000","last_clean":"2024-09-13T18:55:46.631594+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:55:46.631594+0000","last_undegraded":"2024-09-13T18:55:46.631594+0000","last_fullsized":"2024-09-13T18:55:46.631594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"41'6","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.977227+0000","last_change":"2024-09-13T18:26:31.976075+0000","last_active":"2024-09-13T18:55:25.977227+0000","last_peered":"2024-09-13T18:55:25.977227+0000","last_clean":"2024-09-13T18:55:25.977227+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:55:25.977227+0000","last_undegraded":"2024-09-13T18:55:25.977227+0000","last_fullsized":"2024-09-13T18:55:25.977227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:03.693518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"43'5282","reported_seq":5354,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:06.257621+0000","last_change":"2024-09-13T18:26:36.401777+0000","last_active":"2024-09-13T18:57:06.257621+0000","last_peered":"2024-09-13T18:57:06.257621+0000","last_clean":"2024-09-13T18:57:06.257621+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:57:06.257621+0000","last_undegraded":"2024-09-13T18:57:06.257621+0000","last_fullsized":"2024-09-13T18:57:06.257621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5282,"log_dups_size":0,"ondisk_log_size":5282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:43.440284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293623335,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":24470,"num_write_kb":507684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"29'10","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978450+0000","last_change":"2024-09-13T18:26:32.343791+0000","last_active":"2024-09-13T18:55:25.978450+0000","last_peered":"2024-09-13T18:55:25.978450+0000","last_clean":"2024-09-13T18:55:25.978450+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:55:25.978450+0000","last_undegraded":"2024-09-13T18:55:25.978450+0000","last_fullsized":"2024-09-13T18:55:25.978450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:36.296571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"43'128","reported_seq":303,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.469054+0000","last_change":"2024-09-13T18:26:31.970786+0000","last_active":"2024-09-13T18:56:58.469054+0000","last_peered":"2024-09-13T18:56:58.469054+0000","last_clean":"2024-09-13T18:56:58.469054+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:56:58.469054+0000","last_undegraded":"2024-09-13T18:56:58.469054+0000","last_fullsized":"2024-09-13T18:56:58.469054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93931,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":203,"num_write":119,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"43'6167","reported_seq":6237,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:06.103918+0000","last_change":"2024-09-13T18:26:36.401529+0000","last_active":"2024-09-13T18:57:06.103918+0000","last_peered":"2024-09-13T18:57:06.103918+0000","last_clean":"2024-09-13T18:57:06.103918+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:57:06.103918+0000","last_undegraded":"2024-09-13T18:57:06.103918+0000","last_fullsized":"2024-09-13T18:57:06.103918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6167,"log_dups_size":0,"ondisk_log_size":6167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:47.318347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314605270,"num_objects":83,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":28220,"num_write_kb":563741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"28'4","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:40.890524+0000","last_change":"2024-09-13T18:26:32.350470+0000","last_active":"2024-09-13T18:55:40.890524+0000","last_peered":"2024-09-13T18:55:40.890524+0000","last_clean":"2024-09-13T18:55:40.890524+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:55:40.890524+0000","last_undegraded":"2024-09-13T18:55:40.890524+0000","last_fullsized":"2024-09-13T18:55:40.890524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"43'14","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:31.126491+0000","last_change":"2024-09-13T18:26:31.972587+0000","last_active":"2024-09-13T18:55:31.126491+0000","last_peered":"2024-09-13T18:55:31.126491+0000","last_clean":"2024-09-13T18:55:31.126491+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:55:31.126491+0000","last_undegraded":"2024-09-13T18:55:31.126491+0000","last_fullsized":"2024-09-13T18:55:31.126491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:27.547763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"43'4353","reported_seq":4424,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:08.400580+0000","last_change":"2024-09-13T18:26:36.405086+0000","last_active":"2024-09-13T18:57:08.400580+0000","last_peered":"2024-09-13T18:57:08.400580+0000","last_clean":"2024-09-13T18:57:08.400580+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:57:08.400580+0000","last_undegraded":"2024-09-13T18:57:08.400580+0000","last_fullsized":"2024-09-13T18:57:08.400580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4353,"log_dups_size":0,"ondisk_log_size":4353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218370729,"num_objects":57,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":19805,"num_write_kb":435588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"43'5","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:09.844867+0000","last_change":"2024-09-13T18:26:32.350510+0000","last_active":"2024-09-13T18:57:09.844867+0000","last_peered":"2024-09-13T18:57:09.844867+0000","last_clean":"2024-09-13T18:57:09.844867+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:57:09.844867+0000","last_undegraded":"2024-09-13T18:57:09.844867+0000","last_fullsized":"2024-09-13T18:57:09.844867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"43'7","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:52.985612+0000","last_change":"2024-09-13T18:26:31.978355+0000","last_active":"2024-09-13T18:55:52.985612+0000","last_peered":"2024-09-13T18:55:52.985612+0000","last_clean":"2024-09-13T18:55:52.985612+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:55:52.985612+0000","last_undegraded":"2024-09-13T18:55:52.985612+0000","last_fullsized":"2024-09-13T18:55:52.985612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:12.946518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":68,"num_write":9,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983663+0000","last_change":"2024-09-13T18:26:26.931403+0000","last_active":"2024-09-13T18:55:25.983663+0000","last_peered":"2024-09-13T18:55:25.983663+0000","last_clean":"2024-09-13T18:55:25.983663+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:55:25.983663+0000","last_undegraded":"2024-09-13T18:55:25.983663+0000","last_fullsized":"2024-09-13T18:55:25.983663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:24.880028+0000","last_clean_scrub_stamp":"2024-09-13T18:26:24.880028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:30.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"43'5874","reported_seq":5953,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:06.313000+0000","last_change":"2024-09-13T18:26:36.409827+0000","last_active":"2024-09-13T18:57:06.313000+0000","last_peered":"2024-09-13T18:57:06.313000+0000","last_clean":"2024-09-13T18:57:06.313000+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:57:06.313000+0000","last_undegraded":"2024-09-13T18:57:06.313000+0000","last_fullsized":"2024-09-13T18:57:06.313000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5874,"log_dups_size":0,"ondisk_log_size":5874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:50.482400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297837816,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":27346,"num_write_kb":519330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"43'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:51.407345+0000","last_change":"2024-09-13T18:26:32.356850+0000","last_active":"2024-09-13T18:55:51.407345+0000","last_peered":"2024-09-13T18:55:51.407345+0000","last_clean":"2024-09-13T18:55:51.407345+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:55:51.407345+0000","last_undegraded":"2024-09-13T18:55:51.407345+0000","last_fullsized":"2024-09-13T18:55:51.407345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:38.710828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"36'4","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:51.176416+0000","last_change":"2024-09-13T18:26:31.975942+0000","last_active":"2024-09-13T18:55:51.176416+0000","last_peered":"2024-09-13T18:55:51.176416+0000","last_clean":"2024-09-13T18:55:51.176416+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:55:51.176416+0000","last_undegraded":"2024-09-13T18:55:51.176416+0000","last_fullsized":"2024-09-13T18:55:51.176416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":35,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"43'5296","reported_seq":5360,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:06.476770+0000","last_change":"2024-09-13T18:26:36.409802+0000","last_active":"2024-09-13T18:57:06.476770+0000","last_peered":"2024-09-13T18:57:06.476770+0000","last_clean":"2024-09-13T18:57:06.476770+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:57:06.476770+0000","last_undegraded":"2024-09-13T18:57:06.476770+0000","last_fullsized":"2024-09-13T18:57:06.476770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5296,"log_dups_size":0,"ondisk_log_size":5296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:38.178151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289408460,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":23996,"num_write_kb":513904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"29'8","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:41.231777+0000","last_change":"2024-09-13T18:27:22.610607+0000","last_active":"2024-09-13T18:56:41.231777+0000","last_peered":"2024-09-13T18:56:41.231777+0000","last_clean":"2024-09-13T18:56:41.231777+0000","last_became_active":"2024-09-13T18:27:22.610326+0000","last_became_peered":"2024-09-13T18:27:22.610326+0000","last_unstale":"2024-09-13T18:56:41.231777+0000","last_undegraded":"2024-09-13T18:56:41.231777+0000","last_fullsized":"2024-09-13T18:56:41.231777+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'3","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:50.383171+0000","last_change":"2024-09-13T18:26:31.966618+0000","last_active":"2024-09-13T18:55:50.383171+0000","last_peered":"2024-09-13T18:55:50.383171+0000","last_clean":"2024-09-13T18:55:50.383171+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:55:50.383171+0000","last_undegraded":"2024-09-13T18:55:50.383171+0000","last_fullsized":"2024-09-13T18:55:50.383171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:54:26.422611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"43'5045","reported_seq":5113,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:08.593401+0000","last_change":"2024-09-13T18:26:36.394344+0000","last_active":"2024-09-13T18:57:08.593401+0000","last_peered":"2024-09-13T18:57:08.593401+0000","last_clean":"2024-09-13T18:57:08.593401+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:57:08.593401+0000","last_undegraded":"2024-09-13T18:57:08.593401+0000","last_fullsized":"2024-09-13T18:57:08.593401+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5045,"log_dups_size":0,"ondisk_log_size":5045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213982655,"num_objects":59,"num_object_clones":0,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":22771,"num_write_kb":472029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"29'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984452+0000","last_change":"2024-09-13T18:26:32.365655+0000","last_active":"2024-09-13T18:55:25.984452+0000","last_peered":"2024-09-13T18:55:25.984452+0000","last_clean":"2024-09-13T18:55:25.984452+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:55:25.984452+0000","last_undegraded":"2024-09-13T18:55:25.984452+0000","last_fullsized":"2024-09-13T18:55:25.984452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:40.288128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"43'5","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:05.933054+0000","last_change":"2024-09-13T18:26:31.970652+0000","last_active":"2024-09-13T18:57:05.933054+0000","last_peered":"2024-09-13T18:57:05.933054+0000","last_clean":"2024-09-13T18:57:05.933054+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:57:05.933054+0000","last_undegraded":"2024-09-13T18:57:05.933054+0000","last_fullsized":"2024-09-13T18:57:05.933054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"43'4886","reported_seq":4947,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:09.471646+0000","last_change":"2024-09-13T18:26:36.410280+0000","last_active":"2024-09-13T18:57:09.471646+0000","last_peered":"2024-09-13T18:57:09.471646+0000","last_clean":"2024-09-13T18:57:09.471646+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:57:09.471646+0000","last_undegraded":"2024-09-13T18:57:09.471646+0000","last_fullsized":"2024-09-13T18:57:09.471646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4886,"log_dups_size":0,"ondisk_log_size":4886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:01.395290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205554535,"num_objects":53,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":17,"num_read_kb":0,"num_write":22722,"num_write_kb":463670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"28'3","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984106+0000","last_change":"2024-09-13T18:26:32.361575+0000","last_active":"2024-09-13T18:55:25.984106+0000","last_peered":"2024-09-13T18:55:25.984106+0000","last_clean":"2024-09-13T18:55:25.984106+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:55:25.984106+0000","last_undegraded":"2024-09-13T18:55:25.984106+0000","last_fullsized":"2024-09-13T18:55:25.984106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:07:08.818877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"41'9","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:56.637866+0000","last_change":"2024-09-13T18:26:31.977029+0000","last_active":"2024-09-13T18:55:56.637866+0000","last_peered":"2024-09-13T18:55:56.637866+0000","last_clean":"2024-09-13T18:55:56.637866+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:55:56.637866+0000","last_undegraded":"2024-09-13T18:55:56.637866+0000","last_fullsized":"2024-09-13T18:55:56.637866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"43'5038","reported_seq":5104,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.688418+0000","last_change":"2024-09-13T18:26:36.403479+0000","last_active":"2024-09-13T18:57:10.688418+0000","last_peered":"2024-09-13T18:57:10.688418+0000","last_clean":"2024-09-13T18:57:10.688418+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:57:10.688418+0000","last_undegraded":"2024-09-13T18:57:10.688418+0000","last_fullsized":"2024-09-13T18:57:10.688418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5038,"log_dups_size":0,"ondisk_log_size":5038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209772797,"num_objects":56,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":22659,"num_write_kb":463382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"29'9","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984952+0000","last_change":"2024-09-13T18:26:32.365143+0000","last_active":"2024-09-13T18:55:25.984952+0000","last_peered":"2024-09-13T18:55:25.984952+0000","last_clean":"2024-09-13T18:55:25.984952+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:55:25.984952+0000","last_undegraded":"2024-09-13T18:55:25.984952+0000","last_fullsized":"2024-09-13T18:55:25.984952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:46.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"41'21","reported_seq":127,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:56.238319+0000","last_change":"2024-09-13T18:26:31.965802+0000","last_active":"2024-09-13T18:56:56.238319+0000","last_peered":"2024-09-13T18:56:56.238319+0000","last_clean":"2024-09-13T18:56:56.238319+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:56:56.238319+0000","last_undegraded":"2024-09-13T18:56:56.238319+0000","last_fullsized":"2024-09-13T18:56:56.238319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":158,"num_read_kb":157,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"43'5943","reported_seq":6017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:08.495305+0000","last_change":"2024-09-13T18:26:36.399235+0000","last_active":"2024-09-13T18:57:08.495305+0000","last_peered":"2024-09-13T18:57:08.495305+0000","last_clean":"2024-09-13T18:57:08.495305+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:57:08.495305+0000","last_undegraded":"2024-09-13T18:57:08.495305+0000","last_fullsized":"2024-09-13T18:57:08.495305+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5943,"log_dups_size":0,"ondisk_log_size":5943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243354183,"num_objects":66,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":27674,"num_write_kb":536907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"43'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:56.934698+0000","last_change":"2024-09-13T18:26:32.357403+0000","last_active":"2024-09-13T18:56:56.934698+0000","last_peered":"2024-09-13T18:56:56.934698+0000","last_clean":"2024-09-13T18:56:56.934698+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:56:56.934698+0000","last_undegraded":"2024-09-13T18:56:56.934698+0000","last_fullsized":"2024-09-13T18:56:56.934698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:12.600397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"41'12","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.982231+0000","last_change":"2024-09-13T18:26:31.976148+0000","last_active":"2024-09-13T18:55:25.982231+0000","last_peered":"2024-09-13T18:55:25.982231+0000","last_clean":"2024-09-13T18:55:25.982231+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:55:25.982231+0000","last_undegraded":"2024-09-13T18:55:25.982231+0000","last_fullsized":"2024-09-13T18:55:25.982231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:38.023636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1620,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"43'6040","reported_seq":6107,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.210665+0000","last_change":"2024-09-13T18:26:36.406177+0000","last_active":"2024-09-13T18:57:10.210665+0000","last_peered":"2024-09-13T18:57:10.210665+0000","last_clean":"2024-09-13T18:57:10.210665+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:57:10.210665+0000","last_undegraded":"2024-09-13T18:57:10.210665+0000","last_fullsized":"2024-09-13T18:57:10.210665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6040,"log_dups_size":0,"ondisk_log_size":6040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234915334,"num_objects":60,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":28208,"num_write_kb":551294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"43'12","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978424+0000","last_change":"2024-09-13T18:27:22.609309+0000","last_active":"2024-09-13T18:55:25.978424+0000","last_peered":"2024-09-13T18:55:25.978424+0000","last_clean":"2024-09-13T18:55:25.978424+0000","last_became_active":"2024-09-13T18:27:22.609101+0000","last_became_peered":"2024-09-13T18:27:22.609101+0000","last_unstale":"2024-09-13T18:55:25.978424+0000","last_undegraded":"2024-09-13T18:55:25.978424+0000","last_fullsized":"2024-09-13T18:55:25.978424+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:15.692612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"43'10","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:53.165844+0000","last_change":"2024-09-13T18:26:31.982114+0000","last_active":"2024-09-13T18:55:53.165844+0000","last_peered":"2024-09-13T18:55:53.165844+0000","last_clean":"2024-09-13T18:55:53.165844+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:55:53.165844+0000","last_undegraded":"2024-09-13T18:55:53.165844+0000","last_fullsized":"2024-09-13T18:55:53.165844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.266528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"43'5831","reported_seq":5901,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.533008+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:57:10.533008+0000","last_peered":"2024-09-13T18:57:10.533008+0000","last_clean":"2024-09-13T18:57:10.533008+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:57:10.533008+0000","last_undegraded":"2024-09-13T18:57:10.533008+0000","last_fullsized":"2024-09-13T18:57:10.533008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5831,"log_dups_size":0,"ondisk_log_size":5831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239179452,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27061,"num_write_kb":550847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"28'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:54.008494+0000","last_change":"2024-09-13T18:26:32.351145+0000","last_active":"2024-09-13T18:56:54.008494+0000","last_peered":"2024-09-13T18:56:54.008494+0000","last_clean":"2024-09-13T18:56:54.008494+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:56:54.008494+0000","last_undegraded":"2024-09-13T18:56:54.008494+0000","last_fullsized":"2024-09-13T18:56:54.008494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"43'10","reported_seq":90,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:05.487518+0000","last_change":"2024-09-13T18:26:31.961202+0000","last_active":"2024-09-13T18:57:05.487518+0000","last_peered":"2024-09-13T18:57:05.487518+0000","last_clean":"2024-09-13T18:57:05.487518+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:57:05.487518+0000","last_undegraded":"2024-09-13T18:57:05.487518+0000","last_fullsized":"2024-09-13T18:57:05.487518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":93,"num_read_kb":112,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"43'5189","reported_seq":5254,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:09.347397+0000","last_change":"2024-09-13T18:26:36.410014+0000","last_active":"2024-09-13T18:57:09.347397+0000","last_peered":"2024-09-13T18:57:09.347397+0000","last_clean":"2024-09-13T18:57:09.347397+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:57:09.347397+0000","last_undegraded":"2024-09-13T18:57:09.347397+0000","last_fullsized":"2024-09-13T18:57:09.347397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5189,"log_dups_size":0,"ondisk_log_size":5189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:24.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909527,"num_objects":52,"num_object_clones":0,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":23188,"num_write_kb":490059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"29'9","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983909+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:55:25.983909+0000","last_peered":"2024-09-13T18:55:25.983909+0000","last_clean":"2024-09-13T18:55:25.983909+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:55:25.983909+0000","last_undegraded":"2024-09-13T18:55:25.983909+0000","last_fullsized":"2024-09-13T18:55:25.983909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:40.102815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"43'383","reported_seq":470,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:02.838401+0000","last_change":"2024-09-13T18:26:31.974110+0000","last_active":"2024-09-13T18:57:02.838401+0000","last_peered":"2024-09-13T18:57:02.838401+0000","last_clean":"2024-09-13T18:57:02.838401+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:57:02.838401+0000","last_undegraded":"2024-09-13T18:57:02.838401+0000","last_fullsized":"2024-09-13T18:57:02.838401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:56.058561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":93,"num_read_kb":20568,"num_write":383,"num_write_kb":4318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"43'5341","reported_seq":5407,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:06.397038+0000","last_change":"2024-09-13T18:26:36.394080+0000","last_active":"2024-09-13T18:57:06.397038+0000","last_peered":"2024-09-13T18:57:06.397038+0000","last_clean":"2024-09-13T18:57:06.397038+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:57:06.397038+0000","last_undegraded":"2024-09-13T18:57:06.397038+0000","last_fullsized":"2024-09-13T18:57:06.397038+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5341,"log_dups_size":0,"ondisk_log_size":5341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:47.121814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260176219,"num_objects":68,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":24966,"num_write_kb":485267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"43'5","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:08.421271+0000","last_change":"2024-09-13T18:26:32.362873+0000","last_active":"2024-09-13T18:57:08.421271+0000","last_peered":"2024-09-13T18:57:08.421271+0000","last_clean":"2024-09-13T18:57:08.421271+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:57:08.421271+0000","last_undegraded":"2024-09-13T18:57:08.421271+0000","last_fullsized":"2024-09-13T18:57:08.421271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"43'61","reported_seq":123,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:46.092289+0000","last_change":"2024-09-13T18:26:31.976401+0000","last_active":"2024-09-13T18:56:46.092289+0000","last_peered":"2024-09-13T18:56:46.092289+0000","last_clean":"2024-09-13T18:56:46.092289+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:56:46.092289+0000","last_undegraded":"2024-09-13T18:56:46.092289+0000","last_fullsized":"2024-09-13T18:56:46.092289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:26.434292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":62,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"43'5848","reported_seq":5919,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.614486+0000","last_change":"2024-09-13T18:26:36.406228+0000","last_active":"2024-09-13T18:57:10.614486+0000","last_peered":"2024-09-13T18:57:10.614486+0000","last_clean":"2024-09-13T18:57:10.614486+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:57:10.614486+0000","last_undegraded":"2024-09-13T18:57:10.614486+0000","last_fullsized":"2024-09-13T18:57:10.614486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5848,"log_dups_size":0,"ondisk_log_size":5848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239221109,"num_objects":65,"num_object_clones":0,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27408,"num_write_kb":527755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"43'11","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:03.787320+0000","last_change":"2024-09-13T18:26:32.350841+0000","last_active":"2024-09-13T18:57:03.787320+0000","last_peered":"2024-09-13T18:57:03.787320+0000","last_clean":"2024-09-13T18:57:03.787320+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:57:03.787320+0000","last_undegraded":"2024-09-13T18:57:03.787320+0000","last_fullsized":"2024-09-13T18:57:03.787320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'3","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:52.993936+0000","last_change":"2024-09-13T18:26:31.978214+0000","last_active":"2024-09-13T18:55:52.993936+0000","last_peered":"2024-09-13T18:55:52.993936+0000","last_clean":"2024-09-13T18:55:52.993936+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:55:52.993936+0000","last_undegraded":"2024-09-13T18:55:52.993936+0000","last_fullsized":"2024-09-13T18:55:52.993936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:17.381375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"43'5393","reported_seq":5455,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.632106+0000","last_change":"2024-09-13T18:26:36.406090+0000","last_active":"2024-09-13T18:57:10.632106+0000","last_peered":"2024-09-13T18:57:10.632106+0000","last_clean":"2024-09-13T18:57:10.632106+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:57:10.632106+0000","last_undegraded":"2024-09-13T18:57:10.632106+0000","last_fullsized":"2024-09-13T18:57:10.632106+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5393,"log_dups_size":0,"ondisk_log_size":5393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213935455,"num_objects":58,"num_object_clones":0,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":24987,"num_write_kb":489215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"43'7","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:02.577128+0000","last_change":"2024-09-13T18:26:32.357701+0000","last_active":"2024-09-13T18:57:02.577128+0000","last_peered":"2024-09-13T18:57:02.577128+0000","last_clean":"2024-09-13T18:57:02.577128+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:57:02.577128+0000","last_undegraded":"2024-09-13T18:57:02.577128+0000","last_fullsized":"2024-09-13T18:57:02.577128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:47.587683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"43'17","reported_seq":83,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:05.495411+0000","last_change":"2024-09-13T18:26:31.980133+0000","last_active":"2024-09-13T18:57:05.495411+0000","last_peered":"2024-09-13T18:57:05.495411+0000","last_clean":"2024-09-13T18:57:05.495411+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:57:05.495411+0000","last_undegraded":"2024-09-13T18:57:05.495411+0000","last_fullsized":"2024-09-13T18:57:05.495411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:05.354335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":18,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"43'5068","reported_seq":5137,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.531090+0000","last_change":"2024-09-13T18:26:36.406553+0000","last_active":"2024-09-13T18:57:10.531090+0000","last_peered":"2024-09-13T18:57:10.531090+0000","last_clean":"2024-09-13T18:57:10.531090+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:57:10.531090+0000","last_undegraded":"2024-09-13T18:57:10.531090+0000","last_fullsized":"2024-09-13T18:57:10.531090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5068,"log_dups_size":0,"ondisk_log_size":5068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197340123,"num_objects":54,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":23286,"num_write_kb":507027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"43'6049","reported_seq":6120,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:08.513009+0000","last_change":"2024-09-13T18:26:36.406986+0000","last_active":"2024-09-13T18:57:08.513009+0000","last_peered":"2024-09-13T18:57:08.513009+0000","last_clean":"2024-09-13T18:57:08.513009+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:57:08.513009+0000","last_undegraded":"2024-09-13T18:57:08.513009+0000","last_fullsized":"2024-09-13T18:57:08.513009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6049,"log_dups_size":0,"ondisk_log_size":6049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289417584,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":28491,"num_write_kb":581141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"29'4","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978161+0000","last_change":"2024-09-13T18:26:32.349081+0000","last_active":"2024-09-13T18:55:25.978161+0000","last_peered":"2024-09-13T18:55:25.978161+0000","last_clean":"2024-09-13T18:55:25.978161+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:55:25.978161+0000","last_undegraded":"2024-09-13T18:55:25.978161+0000","last_fullsized":"2024-09-13T18:55:25.978161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:47.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"43'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:31.137406+0000","last_change":"2024-09-13T18:26:31.968944+0000","last_active":"2024-09-13T18:55:31.137406+0000","last_peered":"2024-09-13T18:55:31.137406+0000","last_clean":"2024-09-13T18:55:31.137406+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:55:31.137406+0000","last_undegraded":"2024-09-13T18:55:31.137406+0000","last_fullsized":"2024-09-13T18:55:31.137406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:29.363312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"43'5504","reported_seq":5576,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:06.576376+0000","last_change":"2024-09-13T18:26:36.396418+0000","last_active":"2024-09-13T18:57:06.576376+0000","last_peered":"2024-09-13T18:57:06.576376+0000","last_clean":"2024-09-13T18:57:06.576376+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:57:06.576376+0000","last_undegraded":"2024-09-13T18:57:06.576376+0000","last_fullsized":"2024-09-13T18:57:06.576376+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5504,"log_dups_size":0,"ondisk_log_size":5504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:03.252773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251995380,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":25535,"num_write_kb":520259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"43'17","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:06.904296+0000","last_change":"2024-09-13T18:26:32.351369+0000","last_active":"2024-09-13T18:57:06.904296+0000","last_peered":"2024-09-13T18:57:06.904296+0000","last_clean":"2024-09-13T18:57:06.904296+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:57:06.904296+0000","last_undegraded":"2024-09-13T18:57:06.904296+0000","last_fullsized":"2024-09-13T18:57:06.904296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"36'4","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:47.795759+0000","last_change":"2024-09-13T18:26:31.972048+0000","last_active":"2024-09-13T18:55:47.795759+0000","last_peered":"2024-09-13T18:55:47.795759+0000","last_clean":"2024-09-13T18:55:47.795759+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:55:47.795759+0000","last_undegraded":"2024-09-13T18:55:47.795759+0000","last_fullsized":"2024-09-13T18:55:47.795759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"41'9","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981371+0000","last_change":"2024-09-13T18:26:31.975310+0000","last_active":"2024-09-13T18:55:25.981371+0000","last_peered":"2024-09-13T18:55:25.981371+0000","last_clean":"2024-09-13T18:55:25.981371+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:55:25.981371+0000","last_undegraded":"2024-09-13T18:55:25.981371+0000","last_fullsized":"2024-09-13T18:55:25.981371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:52.915310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"43'8","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.144935+0000","last_change":"2024-09-13T18:26:32.345825+0000","last_active":"2024-09-13T18:56:59.144935+0000","last_peered":"2024-09-13T18:56:59.144935+0000","last_clean":"2024-09-13T18:56:59.144935+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:56:59.144935+0000","last_undegraded":"2024-09-13T18:56:59.144935+0000","last_fullsized":"2024-09-13T18:56:59.144935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"43'4113","reported_seq":4176,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.559405+0000","last_change":"2024-09-13T18:26:36.391520+0000","last_active":"2024-09-13T18:57:10.559405+0000","last_peered":"2024-09-13T18:57:10.559405+0000","last_clean":"2024-09-13T18:57:10.559405+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:57:10.559405+0000","last_undegraded":"2024-09-13T18:57:10.559405+0000","last_fullsized":"2024-09-13T18:57:10.559405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4113,"log_dups_size":0,"ondisk_log_size":4113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180378662,"num_objects":45,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":19174,"num_write_kb":424944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"43'755","reported_seq":828,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:05.630661+0000","last_change":"2024-09-13T18:26:31.976603+0000","last_active":"2024-09-13T18:57:05.630661+0000","last_peered":"2024-09-13T18:57:05.630661+0000","last_clean":"2024-09-13T18:57:05.630661+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:57:05.630661+0000","last_undegraded":"2024-09-13T18:57:05.630661+0000","last_fullsized":"2024-09-13T18:57:05.630661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196273,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":19627,"num_write":735,"num_write_kb":4345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"29'9","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:36.000924+0000","last_change":"2024-09-13T18:26:32.352982+0000","last_active":"2024-09-13T18:56:36.000924+0000","last_peered":"2024-09-13T18:56:36.000924+0000","last_clean":"2024-09-13T18:56:36.000924+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:56:36.000924+0000","last_undegraded":"2024-09-13T18:56:36.000924+0000","last_fullsized":"2024-09-13T18:56:36.000924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"43'5901","reported_seq":5970,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.522008+0000","last_change":"2024-09-13T18:26:36.403817+0000","last_active":"2024-09-13T18:57:10.522008+0000","last_peered":"2024-09-13T18:57:10.522008+0000","last_clean":"2024-09-13T18:57:10.522008+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:57:10.522008+0000","last_undegraded":"2024-09-13T18:57:10.522008+0000","last_fullsized":"2024-09-13T18:57:10.522008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5901,"log_dups_size":0,"ondisk_log_size":5901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239111114,"num_objects":62,"num_object_clones":0,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":26737,"num_write_kb":557751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"43'8","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:47.227020+0000","last_change":"2024-09-13T18:26:32.357856+0000","last_active":"2024-09-13T18:55:47.227020+0000","last_peered":"2024-09-13T18:55:47.227020+0000","last_clean":"2024-09-13T18:55:47.227020+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:55:47.227020+0000","last_undegraded":"2024-09-13T18:55:47.227020+0000","last_fullsized":"2024-09-13T18:55:47.227020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:26.316214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"29'10","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:46.174873+0000","last_change":"2024-09-13T18:26:32.350966+0000","last_active":"2024-09-13T18:55:46.174873+0000","last_peered":"2024-09-13T18:55:46.174873+0000","last_clean":"2024-09-13T18:55:46.174873+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:55:46.174873+0000","last_undegraded":"2024-09-13T18:55:46.174873+0000","last_fullsized":"2024-09-13T18:55:46.174873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"43'12","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:45.169209+0000","last_change":"2024-09-13T18:26:32.355806+0000","last_active":"2024-09-13T18:55:45.169209+0000","last_peered":"2024-09-13T18:55:45.169209+0000","last_clean":"2024-09-13T18:55:45.169209+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:55:45.169209+0000","last_undegraded":"2024-09-13T18:55:45.169209+0000","last_fullsized":"2024-09-13T18:55:45.169209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.988231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"29'8","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:50.939583+0000","last_change":"2024-09-13T18:26:32.340965+0000","last_active":"2024-09-13T18:55:50.939583+0000","last_peered":"2024-09-13T18:55:50.939583+0000","last_clean":"2024-09-13T18:55:50.939583+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:55:50.939583+0000","last_undegraded":"2024-09-13T18:55:50.939583+0000","last_fullsized":"2024-09-13T18:55:50.939583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"29'7","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:50.507072+0000","last_change":"2024-09-13T18:26:32.349625+0000","last_active":"2024-09-13T18:56:50.507072+0000","last_peered":"2024-09-13T18:56:50.507072+0000","last_clean":"2024-09-13T18:56:50.507072+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:56:50.507072+0000","last_undegraded":"2024-09-13T18:56:50.507072+0000","last_fullsized":"2024-09-13T18:56:50.507072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"28'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978502+0000","last_change":"2024-09-13T18:26:32.349170+0000","last_active":"2024-09-13T18:55:25.978502+0000","last_peered":"2024-09-13T18:55:25.978502+0000","last_clean":"2024-09-13T18:55:25.978502+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:55:25.978502+0000","last_undegraded":"2024-09-13T18:55:25.978502+0000","last_fullsized":"2024-09-13T18:55:25.978502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:43.942732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"29'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981727+0000","last_change":"2024-09-13T18:26:32.364786+0000","last_active":"2024-09-13T18:55:25.981727+0000","last_peered":"2024-09-13T18:55:25.981727+0000","last_clean":"2024-09-13T18:55:25.981727+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:55:25.981727+0000","last_undegraded":"2024-09-13T18:55:25.981727+0000","last_fullsized":"2024-09-13T18:55:25.981727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:22.949875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"43'7","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:06.099871+0000","last_change":"2024-09-13T18:26:32.347541+0000","last_active":"2024-09-13T18:57:06.099871+0000","last_peered":"2024-09-13T18:57:06.099871+0000","last_clean":"2024-09-13T18:57:06.099871+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:57:06.099871+0000","last_undegraded":"2024-09-13T18:57:06.099871+0000","last_fullsized":"2024-09-13T18:57:06.099871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"29'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.968928+0000","last_change":"2024-09-13T18:26:32.353359+0000","last_active":"2024-09-13T18:55:25.968928+0000","last_peered":"2024-09-13T18:55:25.968928+0000","last_clean":"2024-09-13T18:55:25.968928+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:55:25.968928+0000","last_undegraded":"2024-09-13T18:55:25.968928+0000","last_fullsized":"2024-09-13T18:55:25.968928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:09.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"43'11","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:04.433815+0000","last_change":"2024-09-13T18:26:32.349566+0000","last_active":"2024-09-13T18:57:04.433815+0000","last_peered":"2024-09-13T18:57:04.433815+0000","last_clean":"2024-09-13T18:57:04.433815+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:57:04.433815+0000","last_undegraded":"2024-09-13T18:57:04.433815+0000","last_fullsized":"2024-09-13T18:57:04.433815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"43'9","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:56.205756+0000","last_change":"2024-09-13T18:26:32.358640+0000","last_active":"2024-09-13T18:56:56.205756+0000","last_peered":"2024-09-13T18:56:56.205756+0000","last_clean":"2024-09-13T18:56:56.205756+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:56:56.205756+0000","last_undegraded":"2024-09-13T18:56:56.205756+0000","last_fullsized":"2024-09-13T18:56:56.205756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:40.174754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"43'12","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:59.814857+0000","last_change":"2024-09-13T18:26:32.353675+0000","last_active":"2024-09-13T18:56:59.814857+0000","last_peered":"2024-09-13T18:56:59.814857+0000","last_clean":"2024-09-13T18:56:59.814857+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:56:59.814857+0000","last_undegraded":"2024-09-13T18:56:59.814857+0000","last_fullsized":"2024-09-13T18:56:59.814857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"43'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:52.229460+0000","last_change":"2024-09-13T18:26:32.353223+0000","last_active":"2024-09-13T18:55:52.229460+0000","last_peered":"2024-09-13T18:55:52.229460+0000","last_clean":"2024-09-13T18:55:52.229460+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:55:52.229460+0000","last_undegraded":"2024-09-13T18:55:52.229460+0000","last_fullsized":"2024-09-13T18:55:52.229460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:45.898213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"43'11","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:54.434135+0000","last_change":"2024-09-13T18:26:32.357354+0000","last_active":"2024-09-13T18:56:54.434135+0000","last_peered":"2024-09-13T18:56:54.434135+0000","last_clean":"2024-09-13T18:56:54.434135+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:56:54.434135+0000","last_undegraded":"2024-09-13T18:56:54.434135+0000","last_fullsized":"2024-09-13T18:56:54.434135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:19.185008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"43'11","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.466951+0000","last_change":"2024-09-13T18:26:32.348832+0000","last_active":"2024-09-13T18:56:58.466951+0000","last_peered":"2024-09-13T18:56:58.466951+0000","last_clean":"2024-09-13T18:56:58.466951+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:56:58.466951+0000","last_undegraded":"2024-09-13T18:56:58.466951+0000","last_fullsized":"2024-09-13T18:56:58.466951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"43'8","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:54.897254+0000","last_change":"2024-09-13T18:26:32.356902+0000","last_active":"2024-09-13T18:56:54.897254+0000","last_peered":"2024-09-13T18:56:54.897254+0000","last_clean":"2024-09-13T18:56:54.897254+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:56:54.897254+0000","last_undegraded":"2024-09-13T18:56:54.897254+0000","last_fullsized":"2024-09-13T18:56:54.897254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:18.362772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"43'10","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:04.594973+0000","last_change":"2024-09-13T18:26:32.347047+0000","last_active":"2024-09-13T18:57:04.594973+0000","last_peered":"2024-09-13T18:57:04.594973+0000","last_clean":"2024-09-13T18:57:04.594973+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:57:04.594973+0000","last_undegraded":"2024-09-13T18:57:04.594973+0000","last_fullsized":"2024-09-13T18:57:04.594973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:40.882789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"43'11","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:01.507468+0000","last_change":"2024-09-13T18:26:32.349651+0000","last_active":"2024-09-13T18:57:01.507468+0000","last_peered":"2024-09-13T18:57:01.507468+0000","last_clean":"2024-09-13T18:57:01.507468+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:57:01.507468+0000","last_undegraded":"2024-09-13T18:57:01.507468+0000","last_fullsized":"2024-09-13T18:57:01.507468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:33.356563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"43'11","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:04.143992+0000","last_change":"2024-09-13T18:26:32.344607+0000","last_active":"2024-09-13T18:57:04.143992+0000","last_peered":"2024-09-13T18:57:04.143992+0000","last_clean":"2024-09-13T18:57:04.143992+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:57:04.143992+0000","last_undegraded":"2024-09-13T18:57:04.143992+0000","last_fullsized":"2024-09-13T18:57:04.143992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"43'7","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:09.845142+0000","last_change":"2024-09-13T18:26:32.347717+0000","last_active":"2024-09-13T18:57:09.845142+0000","last_peered":"2024-09-13T18:57:09.845142+0000","last_clean":"2024-09-13T18:57:09.845142+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:57:09.845142+0000","last_undegraded":"2024-09-13T18:57:09.845142+0000","last_fullsized":"2024-09-13T18:57:09.845142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"43'7","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:07.426772+0000","last_change":"2024-09-13T18:26:32.359506+0000","last_active":"2024-09-13T18:57:07.426772+0000","last_peered":"2024-09-13T18:57:07.426772+0000","last_clean":"2024-09-13T18:57:07.426772+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:57:07.426772+0000","last_undegraded":"2024-09-13T18:57:07.426772+0000","last_fullsized":"2024-09-13T18:57:07.426772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"43'12","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:37.221615+0000","last_change":"2024-09-13T18:26:32.357729+0000","last_active":"2024-09-13T18:55:37.221615+0000","last_peered":"2024-09-13T18:55:37.221615+0000","last_clean":"2024-09-13T18:55:37.221615+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:55:37.221615+0000","last_undegraded":"2024-09-13T18:55:37.221615+0000","last_fullsized":"2024-09-13T18:55:37.221615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:05.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"29'9","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.981655+0000","last_change":"2024-09-13T18:26:32.357980+0000","last_active":"2024-09-13T18:55:25.981655+0000","last_peered":"2024-09-13T18:55:25.981655+0000","last_clean":"2024-09-13T18:55:25.981655+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:55:25.981655+0000","last_undegraded":"2024-09-13T18:55:25.981655+0000","last_fullsized":"2024-09-13T18:55:25.981655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:00.858702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"43'15","reported_seq":59,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:07.243603+0000","last_change":"2024-09-13T18:26:32.365638+0000","last_active":"2024-09-13T18:57:07.243603+0000","last_peered":"2024-09-13T18:57:07.243603+0000","last_clean":"2024-09-13T18:57:07.243603+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:57:07.243603+0000","last_undegraded":"2024-09-13T18:57:07.243603+0000","last_fullsized":"2024-09-13T18:57:07.243603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.065853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"29'3","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:35.276187+0000","last_change":"2024-09-13T18:26:32.345496+0000","last_active":"2024-09-13T18:56:35.276187+0000","last_peered":"2024-09-13T18:56:35.276187+0000","last_clean":"2024-09-13T18:56:35.276187+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:56:35.276187+0000","last_undegraded":"2024-09-13T18:56:35.276187+0000","last_fullsized":"2024-09-13T18:56:35.276187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"43'5","reported_seq":49,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:56:58.694709+0000","last_change":"2024-09-13T18:26:32.356901+0000","last_active":"2024-09-13T18:56:58.694709+0000","last_peered":"2024-09-13T18:56:58.694709+0000","last_clean":"2024-09-13T18:56:58.694709+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:56:58.694709+0000","last_undegraded":"2024-09-13T18:56:58.694709+0000","last_fullsized":"2024-09-13T18:56:58.694709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:25:03.311603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"43'10","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:02.385954+0000","last_change":"2024-09-13T18:26:32.355203+0000","last_active":"2024-09-13T18:57:02.385954+0000","last_peered":"2024-09-13T18:57:02.385954+0000","last_clean":"2024-09-13T18:57:02.385954+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:57:02.385954+0000","last_undegraded":"2024-09-13T18:57:02.385954+0000","last_fullsized":"2024-09-13T18:57:02.385954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:52.769708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"29'6","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.983681+0000","last_change":"2024-09-13T18:26:32.358190+0000","last_active":"2024-09-13T18:55:25.983681+0000","last_peered":"2024-09-13T18:55:25.983681+0000","last_clean":"2024-09-13T18:55:25.983681+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:55:25.983681+0000","last_undegraded":"2024-09-13T18:55:25.983681+0000","last_fullsized":"2024-09-13T18:55:25.983681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:35.790567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"28'4","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.968851+0000","last_change":"2024-09-13T18:26:32.355121+0000","last_active":"2024-09-13T18:55:25.968851+0000","last_peered":"2024-09-13T18:55:25.968851+0000","last_clean":"2024-09-13T18:55:25.968851+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:55:25.968851+0000","last_undegraded":"2024-09-13T18:55:25.968851+0000","last_fullsized":"2024-09-13T18:55:25.968851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:48.427332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"43'7","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:06.102071+0000","last_change":"2024-09-13T18:26:32.341497+0000","last_active":"2024-09-13T18:57:06.102071+0000","last_peered":"2024-09-13T18:57:06.102071+0000","last_clean":"2024-09-13T18:57:06.102071+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:57:06.102071+0000","last_undegraded":"2024-09-13T18:57:06.102071+0000","last_fullsized":"2024-09-13T18:57:06.102071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"28'2","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.978877+0000","last_change":"2024-09-13T18:26:32.350106+0000","last_active":"2024-09-13T18:55:25.978877+0000","last_peered":"2024-09-13T18:55:25.978877+0000","last_clean":"2024-09-13T18:55:25.978877+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:55:25.978877+0000","last_undegraded":"2024-09-13T18:55:25.978877+0000","last_fullsized":"2024-09-13T18:55:25.978877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:12.387451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"29'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:25.984165+0000","last_change":"2024-09-13T18:26:32.362196+0000","last_active":"2024-09-13T18:55:25.984165+0000","last_peered":"2024-09-13T18:55:25.984165+0000","last_clean":"2024-09-13T18:55:25.984165+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:55:25.984165+0000","last_undegraded":"2024-09-13T18:55:25.984165+0000","last_fullsized":"2024-09-13T18:55:25.984165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:14.946904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":7640507800,"num_objects":1997,"num_object_clones":0,"num_object_copies":7988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1997,"num_whiteouts":0,"num_read":797,"num_read_kb":0,"num_write":801603,"num_write_kb":16319459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6748651520,"data_stored":15521996800,"data_compressed":739285525,"data_compressed_allocated":1038176256,"data_compressed_original":9811521536,"omap_allocated":0,"internal_metadata":0},"log_size":174195,"ondisk_log_size":174195,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":517,"ondisk_log_size":517,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":16507087,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1362,"num_read_kb":52080,"num_write":3737,"num_write_kb":17664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22716416,"data_stored":33014174,"data_compressed":2441304,"data_compressed_allocated":3506176,"data_compressed_original":13950976,"omap_allocated":0,"internal_metadata":0},"log_size":3502,"ondisk_log_size":3502,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509811,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2017320,"kb_used_data":996256,"kb_used_omap":1,"kb_used_meta":1021054,"kb_avail":92354520,"statfs":{"total":96636764160,"available":94571028480,"internally_reserved":0,"allocated":1020166144,"data_stored":2371019484,"data_compressed":113136678,"data_compressed_allocated":158502912,"data_compressed_original":1509609472,"omap_allocated":1593,"internal_metadata":1045559751},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":1.3979999999999999,"15min":0.96599999999999997},"min":{"1min":0.224,"5min":0.223,"15min":0.223},"max":{"1min":9.0050000000000008,"5min":10.43,"15min":10.43},"last":0.38800000000000001},{"interface":"front","average":{"1min":2.5230000000000001,"5min":1.8320000000000001,"15min":1.141},"min":{"1min":0.25900000000000001,"5min":0.245,"15min":0.245},"max":{"1min":9.9190000000000005,"5min":16.600999999999999,"15min":16.600999999999999},"last":4.3109999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.9630000000000001,"5min":3.831,"15min":1.825},"min":{"1min":0.26000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":7.7409999999999997,"5min":52.290999999999997,"15min":52.290999999999997},"last":1.383},{"interface":"front","average":{"1min":1.6719999999999999,"5min":4.0910000000000002,"15min":1.9239999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":7.8410000000000002,"5min":52.149000000000001,"15min":52.149000000000001},"last":1.202}]},{"osd":2,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.518,"15min":1.0580000000000001},"min":{"1min":0.33500000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.8360000000000003,"5min":14.933,"15min":14.933},"last":0.316},{"interface":"front","average":{"1min":1.7509999999999999,"5min":1.4239999999999999,"15min":1.048},"min":{"1min":0.29099999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":6.7160000000000002,"5min":17.792000000000002,"15min":17.792000000000002},"last":0.441}]},{"osd":3,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":2.4580000000000002,"15min":1.3899999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":8.1809999999999992,"5min":15.622,"15min":15.622},"last":0.34699999999999998},{"interface":"front","average":{"1min":3.2429999999999999,"5min":3.2829999999999999,"15min":1.6799999999999999},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":14.993,"5min":28.248000000000001,"15min":28.248000000000001},"last":0.374}]},{"osd":4,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.304,"15min":1.0820000000000001},"min":{"1min":0.31,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.3570000000000002,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":4.4219999999999997},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.232,"15min":1.048},"min":{"1min":0.38700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":4.4569999999999999,"5min":7.9409999999999998,"15min":7.9409999999999998},"last":4.4450000000000003}]},{"osd":5,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.95699999999999996,"15min":0.97699999999999998},"min":{"1min":0.376,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.7410000000000001,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":4.3879999999999999},{"interface":"front","average":{"1min":1.099,"5min":0.96999999999999997,"15min":1.004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.2480000000000002,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":0.41899999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.0429999999999999,"15min":1.0720000000000001},"min":{"1min":0.39300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.8319999999999999,"5min":12.021000000000001,"15min":12.021000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.91300000000000003,"15min":0.98599999999999999},"min":{"1min":0.32500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":3.9430000000000001,"5min":9.0440000000000005,"15min":9.0440000000000005},"last":4.3600000000000003}]}]},{"osd":7,"up_from":15,"seq":64424509811,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1567988,"kb_used_data":682092,"kb_used_omap":1,"kb_used_meta":885886,"kb_avail":92803852,"statfs":{"total":96636764160,"available":95031144448,"internally_reserved":0,"allocated":698462208,"data_stored":1586761294,"data_compressed":75703497,"data_compressed_allocated":106385408,"data_compressed_original":994947072,"omap_allocated":1589,"internal_metadata":907147723},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":2.121,"15min":1.206},"min":{"1min":0.23000000000000001,"5min":0.217,"15min":0.217},"max":{"1min":14.778,"5min":24.414000000000001,"15min":24.414000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":2.1680000000000001,"5min":2.254,"15min":1.3460000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":14.814,"5min":24.469000000000001,"15min":24.469000000000001},"last":1.018}]},{"osd":1,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":2.4889999999999999,"5min":3.355,"15min":1.617},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":6.0490000000000004,"5min":46.161999999999999,"15min":46.161999999999999},"last":1.097},{"interface":"front","average":{"1min":5.1929999999999996,"5min":4.3570000000000002,"15min":2.0129999999999999},"min":{"1min":0.29099999999999998,"5min":0.252,"15min":0.252},"max":{"1min":39.853999999999999,"5min":45.713000000000001,"15min":45.713000000000001},"last":0.876}]},{"osd":2,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.228,"15min":0.98699999999999999},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":5.3230000000000004,"5min":7.4740000000000002,"15min":7.4740000000000002},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.329,"15min":0.97899999999999998},"min":{"1min":0.315,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.8559999999999999,"5min":9.1679999999999993,"15min":9.1679999999999993},"last":0.61799999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":3.4129999999999998,"5min":3.6960000000000002,"15min":1.8029999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":17.721,"5min":35.494999999999997,"15min":35.494999999999997},"last":0.98499999999999999},{"interface":"front","average":{"1min":2.278,"5min":2.976,"15min":1.6850000000000001},"min":{"1min":0.33800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":11.353999999999999,"5min":19.946000000000002,"15min":19.946000000000002},"last":1.3320000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.92600000000000005,"15min":0.91700000000000004},"min":{"1min":0.25800000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.2749999999999999,"5min":7.2880000000000003,"15min":7.2880000000000003},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.0820000000000001,"15min":1.0589999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":7.8810000000000002,"5min":7.8810000000000002,"15min":7.8810000000000002},"last":1.1899999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.80700000000000005,"15min":0.877},"min":{"1min":0.29399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":3.6200000000000001,"5min":6.7309999999999999,"15min":6.7309999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90100000000000002,"15min":0.96999999999999997},"min":{"1min":0.32000000000000001,"5min":0.318,"15min":0.318},"max":{"1min":3.5819999999999999,"5min":6.6699999999999999,"15min":6.6699999999999999},"last":1.0580000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:56:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.2889999999999999,"15min":1.137},"min":{"1min":0.35299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.4260000000000002,"5min":32.491,"15min":32.491},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":1.137,"15min":1.091},"min":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.82899999999999996,"5min":27.291,"15min":27.291},"last":1.216}]}]},{"osd":4,"up_from":15,"seq":64424509811,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1781648,"kb_used_data":818824,"kb_used_omap":1,"kb_used_meta":962814,"kb_avail":92590192,"statfs":{"total":96636764160,"available":94812356608,"internally_reserved":0,"allocated":838475776,"data_stored":1911410784,"data_compressed":90764139,"data_compressed_allocated":127696896,"data_compressed_original":1200885760,"omap_allocated":1594,"internal_metadata":985921990},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":2.2599999999999998,"5min":2.1259999999999999,"15min":1.214},"min":{"1min":0.26300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":8.8740000000000006,"5min":28.593,"15min":28.593},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.9850000000000001,"5min":1.837,"15min":1.1930000000000001},"min":{"1min":0.26400000000000001,"5min":0.255,"15min":0.255},"max":{"1min":10.711,"5min":28.559000000000001,"15min":28.559000000000001},"last":1.044}]},{"osd":1,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":2.6299999999999999,"5min":2.907,"15min":1.49},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":10.044,"5min":37.100999999999999,"15min":37.100999999999999},"last":1.409},{"interface":"front","average":{"1min":2.0089999999999999,"5min":3.4969999999999999,"15min":1.6910000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":8.3810000000000002,"5min":44.798000000000002,"15min":44.798000000000002},"last":0.77300000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":2.6789999999999998,"5min":1.845,"15min":1.1859999999999999},"min":{"1min":0.30199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":16.617000000000001,"5min":27.634,"15min":27.634},"last":0.81499999999999995},{"interface":"front","average":{"1min":2.8279999999999998,"5min":1.7170000000000001,"15min":1.2050000000000001},"min":{"1min":0.317,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":16.396999999999998,"5min":27.606000000000002,"15min":27.606000000000002},"last":1.337}]},{"osd":3,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":3.496,"5min":3.339,"15min":1.6499999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":18.503,"5min":40.676000000000002,"15min":40.676000000000002},"last":0.96299999999999997},{"interface":"front","average":{"1min":5.9610000000000003,"5min":3.8050000000000002,"15min":1.887},"min":{"1min":0.47999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":39.649000000000001,"5min":40.881999999999998,"15min":40.881999999999998},"last":1.1060000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:56:26 2024","interfaces":[{"interface":"back","average":{"1min":1.7769999999999999,"5min":0.876,"15min":0.88300000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":8.5649999999999995,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":1.538},{"interface":"front","average":{"1min":1.0820000000000001,"5min":0.91600000000000004,"15min":0.98499999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.0810000000000004,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":1.4810000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.167,"15min":1.0900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":5.984,"5min":9.4570000000000007,"15min":9.4570000000000007},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":0.996,"15min":1.0109999999999999},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":5.1059999999999999,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":1.5860000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8779999999999999,"5min":1.4279999999999999,"15min":1.177},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":10.101000000000001,"5min":10.101000000000001,"15min":10.101000000000001},"last":1.8260000000000001},{"interface":"front","average":{"1min":1.9239999999999999,"5min":1.492,"15min":1.274},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":10.127000000000001,"5min":10.127000000000001,"15min":10.127000000000001},"last":1.8939999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542515,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2269856,"kb_used_data":1071512,"kb_used_omap":1,"kb_used_meta":1198334,"kb_avail":92101984,"statfs":{"total":96636764160,"available":94312431616,"internally_reserved":0,"allocated":1097228288,"data_stored":2520138442,"data_compressed":120453364,"data_compressed_allocated":169361408,"data_compressed_original":1592528896,"omap_allocated":1591,"internal_metadata":1227094473},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:32 2024","interfaces":[{"interface":"back","average":{"1min":3.0569999999999999,"5min":3.3959999999999999,"15min":1.5109999999999999},"min":{"1min":0.20300000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":22.670000000000002,"5min":22.670000000000002,"15min":22.670000000000002},"last":10.571999999999999},{"interface":"front","average":{"1min":3.1179999999999999,"5min":3.5009999999999999,"15min":1.677},"min":{"1min":0.246,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":22.728000000000002,"5min":22.728000000000002,"15min":22.728000000000002},"last":10.512}]},{"osd":1,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":7.9240000000000004,"5min":6.282,"15min":2.4940000000000002},"min":{"1min":0.38500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":42.046999999999997,"5min":42.046999999999997,"15min":42.046999999999997},"last":11.987},{"interface":"front","average":{"1min":7.6500000000000004,"5min":6.1680000000000001,"15min":2.5630000000000002},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":42.024999999999999,"5min":42.024999999999999,"15min":42.024999999999999},"last":11.890000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":6.4550000000000001,"5min":4.2300000000000004,"15min":1.9259999999999999},"min":{"1min":0.27700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":41.909999999999997,"5min":41.909999999999997,"15min":41.909999999999997},"last":11.859},{"interface":"front","average":{"1min":5.1609999999999996,"5min":3.1819999999999999,"15min":1.5229999999999999},"min":{"1min":0.317,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":27.25,"5min":27.25,"15min":27.25},"last":1.4970000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:56:32 2024","interfaces":[{"interface":"back","average":{"1min":4.6550000000000002,"5min":3.3100000000000001,"15min":1.7430000000000001},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":42.003,"5min":42.003,"15min":42.003},"last":2.1589999999999998},{"interface":"front","average":{"1min":5.4329999999999998,"5min":3.9399999999999999,"15min":1.956},"min":{"1min":0.39300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":41.951000000000001,"5min":41.951000000000001,"15min":41.951000000000001},"last":11.754}]},{"osd":5,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":4.79,"5min":2.46,"15min":1.508},"min":{"1min":0.38100000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":22.988,"5min":22.988,"15min":22.988},"last":1.54},{"interface":"front","average":{"1min":3.2829999999999999,"5min":2.2799999999999998,"15min":1.462},"min":{"1min":0.35199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":23.030000000000001,"5min":23.030000000000001,"15min":23.030000000000001},"last":1.585}]},{"osd":6,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":4.0529999999999999,"5min":2.9689999999999999,"15min":1.6719999999999999},"min":{"1min":0.35799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":23.087,"5min":23.087,"15min":23.087},"last":1.5640000000000001},{"interface":"front","average":{"1min":5.4210000000000003,"5min":3.4540000000000002,"15min":1.843},"min":{"1min":0.52700000000000002,"5min":0.317,"15min":0.317},"max":{"1min":23.425000000000001,"5min":23.425000000000001,"15min":23.425000000000001},"last":2.032}]},{"osd":7,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":4.9379999999999997,"5min":3.1600000000000001,"15min":1.714},"min":{"1min":0.437,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":41.975999999999999,"5min":41.975999999999999,"15min":41.975999999999999},"last":11.833},{"interface":"front","average":{"1min":4.9550000000000001,"5min":2.956,"15min":1.6619999999999999},"min":{"1min":0.42699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":23.457999999999998,"5min":23.457999999999998,"15min":23.457999999999998},"last":2.073}]}]},{"osd":2,"up_from":14,"seq":60129542519,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1370500,"kb_used_data":633148,"kb_used_omap":1,"kb_used_meta":737342,"kb_avail":93001340,"statfs":{"total":96636764160,"available":95233372160,"internally_reserved":0,"allocated":648343552,"data_stored":1465060177,"data_compressed":69932537,"data_compressed_allocated":98414592,"data_compressed_original":915390464,"omap_allocated":1593,"internal_metadata":755038663},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":2.5489999999999999,"15min":1.2270000000000001},"min":{"1min":0.17799999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":5.8200000000000003,"5min":37.600000000000001,"15min":37.600000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.056,"5min":2.4569999999999999,"15min":1.266},"min":{"1min":0.218,"5min":0.19,"15min":0.19},"max":{"1min":6.0839999999999996,"5min":37.651000000000003,"15min":37.651000000000003},"last":0.69799999999999995}]},{"osd":1,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":2.5800000000000001,"5min":3.0449999999999999,"15min":1.5209999999999999},"min":{"1min":0.26500000000000001,"5min":0.255,"15min":0.255},"max":{"1min":12.244,"5min":18.055,"15min":18.055},"last":0.81799999999999995},{"interface":"front","average":{"1min":2.851,"5min":2.8900000000000001,"15min":1.431},"min":{"1min":0.28899999999999998,"5min":0.188,"15min":0.188},"max":{"1min":11.916,"5min":18.175999999999998,"15min":18.175999999999998},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":2.6179999999999999,"5min":3.5470000000000002,"15min":1.7010000000000001},"min":{"1min":0.313,"5min":0.249,"15min":0.249},"max":{"1min":12.888999999999999,"5min":25.832999999999998,"15min":25.832999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":2.621,"5min":3.5230000000000001,"15min":1.77},"min":{"1min":0.375,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":12.702999999999999,"5min":17.515000000000001,"15min":17.515000000000001},"last":0.75900000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.2190000000000001,"15min":0.996},"min":{"1min":0.27700000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.5590000000000002,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":1.03},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.242,"15min":1.0820000000000001},"min":{"1min":0.33700000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":5.0899999999999999,"5min":8.0999999999999996,"15min":8.0999999999999996},"last":0.94399999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.27,"15min":1.032},"min":{"1min":0.29799999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":8.4800000000000004,"5min":8.4800000000000004,"15min":8.4800000000000004},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.212,"5min":1.4319999999999999,"15min":1.1220000000000001},"min":{"1min":0.36499999999999999,"5min":0.309,"15min":0.309},"max":{"1min":3.8079999999999998,"5min":8.907,"15min":8.907},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":1.321,"15min":1.133},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.032,"5min":9.6720000000000006,"15min":9.6720000000000006},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.2070000000000001,"15min":1.069},"min":{"1min":0.35799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":5.3289999999999997,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Sep 13 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3560000000000001,"15min":1.129},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":4.7939999999999996,"5min":8.9250000000000007,"15min":8.9250000000000007},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.585,"15min":1.236},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":6.335,"5min":9.2330000000000005,"15min":9.2330000000000005},"last":1.1990000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575220,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1268524,"kb_used_data":583652,"kb_used_omap":1,"kb_used_meta":684862,"kb_avail":93103316,"statfs":{"total":96636764160,"available":95337795584,"internally_reserved":0,"allocated":597659648,"data_stored":1395494640,"data_compressed":66386477,"data_compressed_allocated":93138944,"data_compressed_original":891232256,"omap_allocated":1593,"internal_metadata":701299143},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":1.8879999999999999,"5min":1.8919999999999999,"15min":1.109},"min":{"1min":0.23799999999999999,"5min":0.224,"15min":0.224},"max":{"1min":8.2240000000000002,"5min":9.952,"15min":9.952},"last":3.6680000000000001},{"interface":"front","average":{"1min":1.95,"5min":2.0139999999999998,"15min":1.2350000000000001},"min":{"1min":0.29899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.2579999999999991,"5min":14.317,"15min":14.317},"last":3.6949999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":1.9550000000000001,"5min":2.29,"15min":1.3129999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":10.058,"5min":10.85,"15min":10.85},"last":0.33500000000000002},{"interface":"front","average":{"1min":2.1629999999999998,"5min":2.2719999999999998,"15min":1.3140000000000001},"min":{"1min":0.29299999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":10.462999999999999,"5min":10.868,"15min":10.868},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":3.8519999999999999,"5min":1.915,"15min":1.1759999999999999},"min":{"1min":0.27300000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":38.484000000000002,"5min":38.484000000000002,"15min":38.484000000000002},"last":8.5800000000000001},{"interface":"front","average":{"1min":4.0890000000000004,"5min":2.2309999999999999,"15min":1.286},"min":{"1min":0.27100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":40.686,"5min":40.686,"15min":40.686},"last":7.9169999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":3.9580000000000002,"5min":4.2320000000000002,"15min":1.9750000000000001},"min":{"1min":0.38100000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":19.093,"5min":72.534999999999997,"15min":72.534999999999997},"last":14.885999999999999},{"interface":"front","average":{"1min":3.5720000000000001,"5min":4.0199999999999996,"15min":1.9710000000000001},"min":{"1min":0.43099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":19.071000000000002,"5min":72.486999999999995,"15min":72.486999999999995},"last":14.91}]},{"osd":4,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":1.02,"15min":0.91900000000000004},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.2290000000000001,"5min":7.8710000000000004,"15min":7.8710000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.131,"15min":1.0760000000000001},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":8.7829999999999995,"5min":8.7829999999999995,"15min":8.7829999999999995},"last":0.68200000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:56:24 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":0.96399999999999997,"15min":0.997},"min":{"1min":0.28699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.04,"15min":1.0489999999999999},"min":{"1min":0.35599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.516,"5min":4.516,"15min":4.516},"last":1.0129999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.02,"15min":1.0209999999999999},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":4.4249999999999998,"5min":7.6020000000000003,"15min":7.6020000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.0169999999999999,"15min":1.101},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.1900000000000004,"5min":7.9109999999999996,"15min":7.9109999999999996},"last":0.51900000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575217,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2414128,"kb_used_data":1264416,"kb_used_omap":1,"kb_used_meta":1148350,"kb_avail":91957712,"statfs":{"total":96636764160,"available":94164697088,"internally_reserved":0,"allocated":1294761984,"data_stored":2998148414,"data_compressed":142679404,"data_compressed_allocated":200003584,"data_compressed_original":1903652864,"omap_allocated":1594,"internal_metadata":1175910854},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":3.2570000000000001,"15min":1.4910000000000001},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":7.9580000000000002,"5min":28.297000000000001,"15min":28.297000000000001},"last":0.24199999999999999},{"interface":"front","average":{"1min":3.1040000000000001,"5min":2.7749999999999999,"15min":1.341},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":19.785,"5min":19.785,"15min":19.785},"last":0.438}]},{"osd":2,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":2.8340000000000001,"15min":1.385},"min":{"1min":0.252,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":8.0630000000000006,"5min":25.609999999999999,"15min":25.609999999999999},"last":6.5709999999999997},{"interface":"front","average":{"1min":4.0890000000000004,"5min":3.097,"15min":1.536},"min":{"1min":0.38500000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":20.173999999999999,"5min":30.469999999999999,"15min":30.469999999999999},"last":6.7160000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":5.524,"5min":5.7249999999999996,"15min":2.383},"min":{"1min":0.58199999999999996,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":20.001000000000001,"5min":36.643999999999998,"15min":36.643999999999998},"last":4.6280000000000001},{"interface":"front","average":{"1min":5.5519999999999996,"5min":5.9119999999999999,"15min":2.5110000000000001},"min":{"1min":0.59899999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":20.030999999999999,"5min":36.670000000000002,"15min":36.670000000000002},"last":4.6870000000000003}]},{"osd":4,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":1.7549999999999999,"15min":1.1759999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.2249999999999996,"5min":7.6799999999999997,"15min":7.6799999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.26,"5min":2.1299999999999999,"15min":1.331},"min":{"1min":0.32800000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":6.5540000000000003,"5min":14.906000000000001,"15min":14.906000000000001},"last":0.68999999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":3.1880000000000002,"5min":1.9350000000000001,"15min":1.3149999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":9.0519999999999996,"5min":9.0519999999999996,"15min":9.0519999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":2.7149999999999999,"5min":1.9410000000000001,"15min":1.3300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.1050000000000004,"5min":12.27,"15min":12.27},"last":0.53000000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":2.577,"5min":2.4300000000000002,"15min":1.468},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":9.077,"5min":28.981999999999999,"15min":28.981999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":2.1819999999999999,"5min":2.1379999999999999,"15min":1.3799999999999999},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":15.727,"5min":15.727,"15min":15.727},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":2.379,"15min":1.4359999999999999},"min":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":7.9930000000000003,"5min":28.100000000000001,"15min":28.100000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":1.728,"5min":2.4860000000000002,"15min":1.518},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.0220000000000002,"5min":28.896999999999998,"15min":28.896999999999998},"last":0.54500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575219,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1547220,"kb_used_data":751436,"kb_used_omap":1,"kb_used_meta":795774,"kb_avail":92824620,"statfs":{"total":96636764160,"available":95052410880,"internally_reserved":0,"allocated":769470464,"data_stored":1771484795,"data_compressed":84640319,"data_compressed_allocated":118788096,"data_compressed_original":1121062912,"omap_allocated":1594,"internal_metadata":814873030},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":5.1710000000000003,"5min":4.0839999999999996,"15min":1.732},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":29.712,"5min":29.712,"15min":29.712},"last":5.9240000000000004},{"interface":"front","average":{"1min":5.4039999999999999,"5min":3.0979999999999999,"15min":1.4810000000000001},"min":{"1min":0.495,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":29.748000000000001,"5min":29.748000000000001,"15min":29.748000000000001},"last":5.9470000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":1.7849999999999999,"15min":1.014},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":17.611000000000001,"5min":17.611000000000001,"15min":17.611000000000001},"last":8.5660000000000007},{"interface":"front","average":{"1min":2.847,"5min":1.9570000000000001,"15min":1.1180000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":17.501999999999999,"5min":17.501999999999999,"15min":17.501999999999999},"last":9.2560000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":2.9580000000000002,"5min":3.4689999999999999,"15min":1.615},"min":{"1min":0.26200000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":17.260000000000002,"5min":17.260000000000002,"15min":17.260000000000002},"last":7.9340000000000002},{"interface":"front","average":{"1min":2.5459999999999998,"5min":3.0129999999999999,"15min":1.512},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":17.088000000000001,"5min":17.088000000000001,"15min":17.088000000000001},"last":5.8170000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":5.3200000000000003,"5min":2.3799999999999999,"15min":1.407},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":40.101999999999997,"5min":40.101999999999997,"15min":40.101999999999997},"last":8.0380000000000003},{"interface":"front","average":{"1min":5.2830000000000004,"5min":2.089,"15min":1.2989999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":40.981000000000002,"5min":40.981000000000002,"15min":40.981000000000002},"last":5.9009999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":4.5419999999999998,"5min":1.857,"15min":1.252},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":40.701999999999998,"5min":40.701999999999998,"15min":40.701999999999998},"last":2.4769999999999999},{"interface":"front","average":{"1min":4.8120000000000003,"5min":2.0230000000000001,"15min":1.3440000000000001},"min":{"1min":0.36899999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":40.130000000000003,"5min":40.130000000000003,"15min":40.130000000000003},"last":5.8559999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":5.7699999999999996,"5min":2.1499999999999999,"15min":1.3979999999999999},"min":{"1min":0.41399999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":51.719000000000001,"5min":51.719000000000001,"15min":51.719000000000001},"last":6.1440000000000001},{"interface":"front","average":{"1min":5.8150000000000004,"5min":2.1989999999999998,"15min":1.448},"min":{"1min":0.49099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":51.75,"5min":51.75,"15min":51.75},"last":7.9660000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":5.5179999999999998,"5min":2.3180000000000001,"15min":1.4770000000000001},"min":{"1min":0.36899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":58.810000000000002,"5min":58.810000000000002,"15min":58.810000000000002},"last":8.0120000000000005},{"interface":"front","average":{"1min":4.609,"5min":2.125,"15min":1.4370000000000001},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":39.622,"5min":39.622,"15min":39.622},"last":1.8680000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2072576,"data_stored":3824333,"data_compressed":409575,"data_compressed_allocated":573440,"data_compressed_original":2342912,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":94659,"data_compressed":1568,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5124096,"data_stored":8485180,"data_compressed":790472,"data_compressed_allocated":1138688,"data_compressed_original":4517888,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4136960,"data_stored":4196050,"data_compressed":20605,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1777664,"data_stored":4196686,"data_compressed":519875,"data_compressed_allocated":692224,"data_compressed_original":3125248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9449472,"data_stored":12214944,"data_compressed":699209,"data_compressed_allocated":1056768,"data_compressed_original":3842048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":744480768,"data_stored":1703915520,"data_compressed":81214636,"data_compressed_allocated":114057216,"data_compressed_original":1073491968,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1247416320,"data_stored":2911924224,"data_compressed":138596705,"data_compressed_allocated":194191360,"data_compressed_original":1858699264,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":635858944,"data_stored":1441656832,"data_compressed":68833641,"data_compressed_allocated":96866304,"data_compressed_original":902664192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1043156992,"data_stored":2381381632,"data_compressed":113492115,"data_compressed_allocated":159674368,"data_compressed_original":1497899008,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":796274688,"data_stored":1804193792,"data_compressed":85862232,"data_compressed_allocated":120893440,"data_compressed_original":1128812544,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":582737920,"data_stored":1353310208,"data_compressed":64396717,"data_compressed_allocated":90386432,"data_compressed_original":860958720,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1010257920,"data_stored":2351898624,"data_compressed":111918067,"data_compressed_allocated":156823552,"data_compressed_original":1498464256,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":688467968,"data_stored":1573715968,"data_compressed":74971412,"data_compressed_allocated":105283584,"data_compressed_original":990531584,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:57:12.862 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.862 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.862 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.862 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.862 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.862 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.862 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.862 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.863 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.863 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.863 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.863 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.863 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.863 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.863 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.863 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.864 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.864 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.864 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.864 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.864 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.864 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.865 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.865 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.865 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.865 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.865 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.865 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.865 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.865 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.865 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.866 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.866 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.866 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.866 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.866 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.866 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.866 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.866 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.866 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.867 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.867 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.867 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.867 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.867 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.867 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.867 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.867 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T18:26:24.880028+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=24, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.867 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.868 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.868 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.868 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.868 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.868 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.868 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.868 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.868 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.868 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.869 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.869 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.869 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.869 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.869 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.869 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.869 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.869 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.869 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.870 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.870 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.870 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.870 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.870 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.870 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.870 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.870 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.870 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.871 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.871 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.871 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.871 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.871 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.871 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.871 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.871 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.871 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.872 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.872 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.872 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.872 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.872 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.872 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.872 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.872 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.873 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.873 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.873 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.873 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.873 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.873 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.873 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.873 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.873 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.873 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.874 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.874 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.874 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.874 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.874 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.874 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.874 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.874 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.874 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.875 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.875 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.875 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.875 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.875 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.875 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.875 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.875 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.875 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.876 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.876 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.876 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.876 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.876 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.876 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.876 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.876 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:12.877 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T18:57:32.878 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:57:33.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 Processor -- start 2024-09-13T18:57:33.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 -- start start 2024-09-13T18:57:33.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbe7814fcf0 0x7fbe781563f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:33.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7806ada0 0x7fbe7806b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:33.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe780c6760 con 0x7fbe7806e250 2024-09-13T18:57:33.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe780c3180 con 0x7fbe7814fcf0 2024-09-13T18:57:33.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe78156930 con 0x7fbe7806ada0 2024-09-13T18:57:33.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7ce89700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fbe7806e250 0x7fbe780c69c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:51924/0 (socket says 172.21.15.146:51924) 2024-09-13T18:57:33.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7ce89700 1 -- 172.21.15.146:0/143354591 learned_addr learned my addr 172.21.15.146:0/143354591 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:57:33.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7de8b700 1 --2- 172.21.15.146:0/143354591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7806ada0 0x7fbe7806b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:33.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7d68a700 1 --2- 172.21.15.146:0/143354591 >> v2:172.21.15.154:3300/0 conn(0x7fbe7814fcf0 0x7fbe781563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:33.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe6ffff700 1 -- 172.21.15.146:0/143354591 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760732716 0 0) 0x7fbe780c6760 con 0x7fbe7806e250 2024-09-13T18:57:33.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe6ffff700 1 -- 172.21.15.146:0/143354591 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe640035d0 con 0x7fbe7806e250 2024-09-13T18:57:33.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7de8b700 1 -- 172.21.15.146:0/143354591 >> v2:172.21.15.154:3300/0 conn(0x7fbe7814fcf0 msgr2=0x7fbe781563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:33.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7de8b700 1 --2- 172.21.15.146:0/143354591 >> v2:172.21.15.154:3300/0 conn(0x7fbe7814fcf0 0x7fbe781563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:33.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7de8b700 1 -- 172.21.15.146:0/143354591 >> v1:172.21.15.146:6789/0 conn(0x7fbe7806e250 legacy=0x7fbe780c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:33.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7de8b700 1 -- 172.21.15.146:0/143354591 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78159e80 con 0x7fbe7806ada0 2024-09-13T18:57:33.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7de8b700 1 --2- 172.21.15.146:0/143354591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7806ada0 0x7fbe7806b1d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009850 tx=0x7fbe74009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bc17396e41302f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:33.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe6ffff700 1 -- 172.21.15.146:0/143354591 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74011830 con 0x7fbe7806ada0 2024-09-13T18:57:33.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe6ffff700 1 -- 172.21.15.146:0/143354591 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe74011990 con 0x7fbe7806ada0 2024-09-13T18:57:33.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe6ffff700 1 -- 172.21.15.146:0/143354591 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74011d00 con 0x7fbe7806ada0 2024-09-13T18:57:33.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 -- 172.21.15.146:0/143354591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7806ada0 msgr2=0x7fbe7806b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:33.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 --2- 172.21.15.146:0/143354591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7806ada0 0x7fbe7806b1d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009850 tx=0x7fbe74009b60 comp rx=0 tx=0).stop 2024-09-13T18:57:33.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 -- 172.21.15.146:0/143354591 shutdown_connections 2024-09-13T18:57:33.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 --2- 172.21.15.146:0/143354591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7806ada0 0x7fbe7806b1d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:33.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 --2- 172.21.15.146:0/143354591 >> v2:172.21.15.154:3300/0 conn(0x7fbe7814fcf0 0x7fbe781563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:33.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 -- 172.21.15.146:0/143354591 >> 172.21.15.146:0/143354591 conn(0x7fbe780723d0 msgr2=0x7fbe78066920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:33.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 -- 172.21.15.146:0/143354591 shutdown_connections 2024-09-13T18:57:33.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.016+0000 7fbe7e68c700 1 -- 172.21.15.146:0/143354591 wait complete. 2024-09-13T18:57:33.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7e68c700 1 Processor -- start 2024-09-13T18:57:33.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7e68c700 1 -- start start 2024-09-13T18:57:33.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7e68c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe7806ada0 0x7fbe781efc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:33.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7e68c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7806e250 0x7fbe78145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:33.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7e68c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe7814fcf0 0x7fbe7813ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:33.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7e68c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe78154860 con 0x7fbe7806ada0 2024-09-13T18:57:33.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7e68c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7814a580 con 0x7fbe7814fcf0 2024-09-13T18:57:33.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7e68c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe781554b0 con 0x7fbe7806e250 2024-09-13T18:57:33.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7d68a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe7806ada0 0x7fbe781efc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:33.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7d68a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe7806ada0 0x7fbe781efc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45730/0 (socket says 172.21.15.146:45730) 2024-09-13T18:57:33.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7d68a700 1 -- 172.21.15.146:0/734906915 learned_addr learned my addr 172.21.15.146:0/734906915 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:33.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7ce89700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7806e250 0x7fbe78145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:33.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7de8b700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe7814fcf0 0x7fbe7813ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:33.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7d68a700 1 -- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7806e250 msgr2=0x7fbe78145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:33.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7d68a700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7806e250 0x7fbe78145810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:33.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7d68a700 1 -- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe7814fcf0 msgr2=0x7fbe7813ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:33.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7d68a700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe7814fcf0 0x7fbe7813ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:33.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7d68a700 1 -- 172.21.15.146:0/734906915 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe740094b0 con 0x7fbe7806ada0 2024-09-13T18:57:33.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.020+0000 7fbe7d68a700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe7806ada0 0x7fbe781efc50 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000efc0 tx=0x7fbe7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:33.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.024+0000 7fbe6dffb700 1 -- 172.21.15.146:0/734906915 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70009b80 con 0x7fbe7806ada0 2024-09-13T18:57:33.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.024+0000 7fbe6dffb700 1 -- 172.21.15.146:0/734906915 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7000f040 con 0x7fbe7806ada0 2024-09-13T18:57:33.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.024+0000 7fbe7e68c700 1 -- 172.21.15.146:0/734906915 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe78154ef0 con 0x7fbe7806ada0 2024-09-13T18:57:33.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.024+0000 7fbe6dffb700 1 -- 172.21.15.146:0/734906915 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70014480 con 0x7fbe7806ada0 2024-09-13T18:57:33.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.024+0000 7fbe7e68c700 1 -- 172.21.15.146:0/734906915 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe78149f30 con 0x7fbe7806ada0 2024-09-13T18:57:33.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.024+0000 7fbe7e68c700 1 -- 172.21.15.146:0/734906915 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe78155d30 con 0x7fbe7806ada0 2024-09-13T18:57:33.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.024+0000 7fbe6dffb700 1 -- 172.21.15.146:0/734906915 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbe700145e0 con 0x7fbe7806ada0 2024-09-13T18:57:33.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.024+0000 7fbe6dffb700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbe5c066780 0x7fbe5c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:33.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.024+0000 7fbe6dffb700 1 -- 172.21.15.146:0/734906915 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fbe700854c0 con 0x7fbe7806ada0 2024-09-13T18:57:33.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.024+0000 7fbe7ce89700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbe5c066780 0x7fbe5c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:33.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.024+0000 7fbe7ce89700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbe5c066780 0x7fbe5c068c40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fbe78143f20 tx=0x7fbe68009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:33.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.032+0000 7fbe6dffb700 1 -- 172.21.15.146:0/734906915 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbe70051e60 con 0x7fbe7806ada0 2024-09-13T18:57:33.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.188+0000 7fbe7e68c700 1 -- 172.21.15.146:0/734906915 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe780c2770 con 0x7fbe5c066780 2024-09-13T18:57:33.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.208+0000 7fbe6dffb700 1 -- 172.21.15.146:0/734906915 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+368329 (secure 0 0 0) 0x7fbe68001c70 con 0x7fbe5c066780 2024-09-13T18:57:33.225 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:57:33.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.212+0000 7fbe7e68c700 1 -- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbe5c066780 msgr2=0x7fbe5c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:33.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.212+0000 7fbe7e68c700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbe5c066780 0x7fbe5c068c40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fbe78143f20 tx=0x7fbe68009040 comp rx=0 tx=0).stop 2024-09-13T18:57:33.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.212+0000 7fbe7e68c700 1 -- 172.21.15.146:0/734906915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe7806ada0 msgr2=0x7fbe781efc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:33.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.212+0000 7fbe7e68c700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe7806ada0 0x7fbe781efc50 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000efc0 tx=0x7fbe7000c3a0 comp rx=0 tx=0).stop 2024-09-13T18:57:33.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.212+0000 7fbe7e68c700 1 -- 172.21.15.146:0/734906915 shutdown_connections 2024-09-13T18:57:33.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.212+0000 7fbe7e68c700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fbe5c066780 0x7fbe5c068c40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:33.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.212+0000 7fbe7e68c700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe7814fcf0 0x7fbe7813ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:33.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.212+0000 7fbe7e68c700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7806e250 0x7fbe78145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:33.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.212+0000 7fbe7e68c700 1 --2- 172.21.15.146:0/734906915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe7806ada0 0x7fbe781efc50 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:33.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.212+0000 7fbe7e68c700 1 -- 172.21.15.146:0/734906915 >> 172.21.15.146:0/734906915 conn(0x7fbe780723d0 msgr2=0x7fbe7806cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:33.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.216+0000 7fbe7e68c700 1 -- 172.21.15.146:0/734906915 shutdown_connections 2024-09-13T18:57:33.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:33.216+0000 7fbe7e68c700 1 -- 172.21.15.146:0/734906915 wait complete. 2024-09-13T18:57:33.229 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:57:33.240 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":969,"stamp":"2024-09-13T18:57:31.605681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2932255995,"num_objects":772,"num_object_clones":0,"num_object_copies":2956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":772,"num_whiteouts":0,"num_read":2275,"num_read_kb":52180,"num_write":806793,"num_write_kb":16339540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181061,"ondisk_log_size":181061,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10687608,"kb_used_data":3053560,"kb_used_omap":12,"kb_used_meta":7633971,"kb_avail":744287112,"statfs":{"total":773094113280,"available":762150002688,"internally_reserved":0,"allocated":3126845440,"data_stored":6609551950,"data_compressed":315818702,"data_compressed_allocated":449458176,"data_compressed_original":3934232576,"omap_allocated":12741,"internal_metadata":7817186875},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2770695700,"num_objects":-811,"num_object_clones":0,"num_object_copies":-3244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":812,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004396"},"pg_stats":[{"pgid":"2.1f","version":"43'471","reported_seq":561,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.876243+0000","last_change":"2024-09-13T18:57:12.363177+0000","last_active":"2024-09-13T18:57:24.876243+0000","last_peered":"2024-09-13T18:57:24.876243+0000","last_clean":"2024-09-13T18:57:24.876243+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:57:24.876243+0000","last_undegraded":"2024-09-13T18:57:24.876243+0000","last_fullsized":"2024-09-13T18:57:24.876243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'469","last_scrub_stamp":"2024-09-13T18:57:12.363068+0000","last_deep_scrub":"43'469","last_deep_scrub_stamp":"2024-09-13T18:57:12.363068+0000","last_clean_scrub_stamp":"2024-09-13T18:57:12.363068+0000","objects_scrubbed":5,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:41:46.348669+0000","scrub_duration":0.044894295000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3824255,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":428,"num_write":456,"num_write_kb":3991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"43'5","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:16.089759+0000","last_change":"2024-09-13T18:57:16.090020+0000","last_active":"2024-09-13T18:57:16.089759+0000","last_peered":"2024-09-13T18:57:16.089759+0000","last_clean":"2024-09-13T18:57:16.089759+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:57:16.089759+0000","last_undegraded":"2024-09-13T18:57:16.089759+0000","last_fullsized":"2024-09-13T18:57:16.089759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:16.089709+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:16.089709+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.089709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.965527+0000","scrub_duration":0.0037314340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"43'5818","reported_seq":5897,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.624779+0000","last_change":"2024-09-13T18:26:36.407579+0000","last_active":"2024-09-13T18:57:30.624779+0000","last_peered":"2024-09-13T18:57:30.624779+0000","last_clean":"2024-09-13T18:57:30.624779+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:57:30.624779+0000","last_undegraded":"2024-09-13T18:57:30.624779+0000","last_fullsized":"2024-09-13T18:57:30.624779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5818,"log_dups_size":0,"ondisk_log_size":5818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":26747,"num_write_kb":537654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"43'16","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.390499+0000","last_change":"2024-09-13T18:57:25.919095+0000","last_active":"2024-09-13T18:57:27.390499+0000","last_peered":"2024-09-13T18:57:27.390499+0000","last_clean":"2024-09-13T18:57:27.390499+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:57:27.390499+0000","last_undegraded":"2024-09-13T18:57:27.390499+0000","last_fullsized":"2024-09-13T18:57:27.390499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:25.918713+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:25.918713+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.918713+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:35.551553+0000","scrub_duration":0.0032041489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"43'14","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.947308+0000","last_change":"2024-09-13T18:26:32.342690+0000","last_active":"2024-09-13T18:57:24.947308+0000","last_peered":"2024-09-13T18:57:24.947308+0000","last_clean":"2024-09-13T18:57:24.947308+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:57:24.947308+0000","last_undegraded":"2024-09-13T18:57:24.947308+0000","last_fullsized":"2024-09-13T18:57:24.947308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"43'5864","reported_seq":5936,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.471086+0000","last_change":"2024-09-13T18:26:36.396545+0000","last_active":"2024-09-13T18:57:30.471086+0000","last_peered":"2024-09-13T18:57:30.471086+0000","last_clean":"2024-09-13T18:57:30.471086+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:57:30.471086+0000","last_undegraded":"2024-09-13T18:57:30.471086+0000","last_fullsized":"2024-09-13T18:57:30.471086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5864,"log_dups_size":0,"ondisk_log_size":5864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":26759,"num_write_kb":545346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"43'25","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.848085+0000","last_change":"2024-09-13T18:26:31.977823+0000","last_active":"2024-09-13T18:57:24.848085+0000","last_peered":"2024-09-13T18:57:24.848085+0000","last_clean":"2024-09-13T18:57:24.848085+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:57:24.848085+0000","last_undegraded":"2024-09-13T18:57:24.848085+0000","last_fullsized":"2024-09-13T18:57:24.848085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":65,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"43'9","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.824681+0000","last_change":"2024-09-13T18:27:22.609917+0000","last_active":"2024-09-13T18:57:24.824681+0000","last_peered":"2024-09-13T18:57:24.824681+0000","last_clean":"2024-09-13T18:57:24.824681+0000","last_became_active":"2024-09-13T18:27:22.609625+0000","last_became_peered":"2024-09-13T18:27:22.609625+0000","last_unstale":"2024-09-13T18:57:24.824681+0000","last_undegraded":"2024-09-13T18:57:24.824681+0000","last_fullsized":"2024-09-13T18:57:24.824681+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"43'4946","reported_seq":5009,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.680471+0000","last_change":"2024-09-13T18:26:36.406675+0000","last_active":"2024-09-13T18:57:30.680471+0000","last_peered":"2024-09-13T18:57:30.680471+0000","last_clean":"2024-09-13T18:57:30.680471+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:57:30.680471+0000","last_undegraded":"2024-09-13T18:57:30.680471+0000","last_fullsized":"2024-09-13T18:57:30.680471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4946,"log_dups_size":0,"ondisk_log_size":4946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21604,"num_write_kb":467229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"43'8","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:05.494132+0000","last_change":"2024-09-13T18:26:31.979860+0000","last_active":"2024-09-13T18:57:05.494132+0000","last_peered":"2024-09-13T18:57:05.494132+0000","last_clean":"2024-09-13T18:57:05.494132+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:57:05.494132+0000","last_undegraded":"2024-09-13T18:57:05.494132+0000","last_fullsized":"2024-09-13T18:57:05.494132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":12,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"43'16","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.854618+0000","last_change":"2024-09-13T18:26:32.351494+0000","last_active":"2024-09-13T18:57:24.854618+0000","last_peered":"2024-09-13T18:57:24.854618+0000","last_clean":"2024-09-13T18:57:24.854618+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:57:24.854618+0000","last_undegraded":"2024-09-13T18:57:24.854618+0000","last_fullsized":"2024-09-13T18:57:24.854618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"43'4847","reported_seq":4910,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.733187+0000","last_change":"2024-09-13T18:26:36.398763+0000","last_active":"2024-09-13T18:57:30.733187+0000","last_peered":"2024-09-13T18:57:30.733187+0000","last_clean":"2024-09-13T18:57:30.733187+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:57:30.733187+0000","last_undegraded":"2024-09-13T18:57:30.733187+0000","last_fullsized":"2024-09-13T18:57:30.733187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4847,"log_dups_size":0,"ondisk_log_size":4847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21604,"num_write_kb":434281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"43'8","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:06.986872+0000","last_change":"2024-09-13T18:26:32.360976+0000","last_active":"2024-09-13T18:57:06.986872+0000","last_peered":"2024-09-13T18:57:06.986872+0000","last_clean":"2024-09-13T18:57:06.986872+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:57:06.986872+0000","last_undegraded":"2024-09-13T18:57:06.986872+0000","last_fullsized":"2024-09-13T18:57:06.986872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"43'1220","reported_seq":1293,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.901549+0000","last_change":"2024-09-13T18:26:31.977891+0000","last_active":"2024-09-13T18:57:24.901549+0000","last_peered":"2024-09-13T18:57:24.901549+0000","last_clean":"2024-09-13T18:57:24.901549+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:57:24.901549+0000","last_undegraded":"2024-09-13T18:57:24.901549+0000","last_fullsized":"2024-09-13T18:57:24.901549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":1220,"log_dups_size":0,"ondisk_log_size":1220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":10235,"num_write":1211,"num_write_kb":4423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"43'5408","reported_seq":5475,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.733256+0000","last_change":"2024-09-13T18:26:36.397484+0000","last_active":"2024-09-13T18:57:30.733256+0000","last_peered":"2024-09-13T18:57:30.733256+0000","last_clean":"2024-09-13T18:57:30.733256+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:57:30.733256+0000","last_undegraded":"2024-09-13T18:57:30.733256+0000","last_fullsized":"2024-09-13T18:57:30.733256+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5408,"log_dups_size":0,"ondisk_log_size":5408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":24105,"num_write_kb":505876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"43'11","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.870567+0000","last_change":"2024-09-13T18:26:32.345564+0000","last_active":"2024-09-13T18:57:24.870567+0000","last_peered":"2024-09-13T18:57:24.870567+0000","last_clean":"2024-09-13T18:57:24.870567+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:57:24.870567+0000","last_undegraded":"2024-09-13T18:57:24.870567+0000","last_fullsized":"2024-09-13T18:57:24.870567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"43'14","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.930897+0000","last_change":"2024-09-13T18:26:31.974107+0000","last_active":"2024-09-13T18:57:24.930897+0000","last_peered":"2024-09-13T18:57:24.930897+0000","last_clean":"2024-09-13T18:57:24.930897+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:57:24.930897+0000","last_undegraded":"2024-09-13T18:57:24.930897+0000","last_fullsized":"2024-09-13T18:57:24.930897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"43'5408","reported_seq":5478,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.678447+0000","last_change":"2024-09-13T18:26:36.405986+0000","last_active":"2024-09-13T18:57:30.678447+0000","last_peered":"2024-09-13T18:57:30.678447+0000","last_clean":"2024-09-13T18:57:30.678447+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:57:30.678447+0000","last_undegraded":"2024-09-13T18:57:30.678447+0000","last_fullsized":"2024-09-13T18:57:30.678447+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5408,"log_dups_size":0,"ondisk_log_size":5408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":24398,"num_write_kb":498911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"43'10","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:09.330678+0000","last_change":"2024-09-13T18:27:22.609984+0000","last_active":"2024-09-13T18:57:09.330678+0000","last_peered":"2024-09-13T18:57:09.330678+0000","last_clean":"2024-09-13T18:57:09.330678+0000","last_became_active":"2024-09-13T18:27:22.609642+0000","last_became_peered":"2024-09-13T18:27:22.609642+0000","last_unstale":"2024-09-13T18:57:09.330678+0000","last_undegraded":"2024-09-13T18:57:09.330678+0000","last_fullsized":"2024-09-13T18:57:09.330678+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"43'6","reported_seq":87,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:20.192184+0000","last_change":"2024-09-13T18:57:20.192184+0000","last_active":"2024-09-13T18:57:20.192184+0000","last_peered":"2024-09-13T18:57:20.192184+0000","last_clean":"2024-09-13T18:57:20.192184+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:57:20.192184+0000","last_undegraded":"2024-09-13T18:57:20.192184+0000","last_fullsized":"2024-09-13T18:57:20.192184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:20.191966+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:20.191966+0000","last_clean_scrub_stamp":"2024-09-13T18:57:20.191966+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:39.033583+0000","scrub_duration":0.0068871679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"43'6959","reported_seq":7029,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.521366+0000","last_change":"2024-09-13T18:27:22.610932+0000","last_active":"2024-09-13T18:57:30.521366+0000","last_peered":"2024-09-13T18:57:30.521366+0000","last_clean":"2024-09-13T18:57:30.521366+0000","last_became_active":"2024-09-13T18:27:22.610527+0000","last_became_peered":"2024-09-13T18:27:22.610527+0000","last_unstale":"2024-09-13T18:57:30.521366+0000","last_undegraded":"2024-09-13T18:57:30.521366+0000","last_fullsized":"2024-09-13T18:57:30.521366+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6959,"log_dups_size":0,"ondisk_log_size":6959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":31786,"num_write_kb":622755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"43'11","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.393739+0000","last_change":"2024-09-13T18:57:17.999669+0000","last_active":"2024-09-13T18:57:28.393739+0000","last_peered":"2024-09-13T18:57:28.393739+0000","last_clean":"2024-09-13T18:57:28.393739+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:57:28.393739+0000","last_undegraded":"2024-09-13T18:57:28.393739+0000","last_fullsized":"2024-09-13T18:57:28.393739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:17.999495+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:17.999495+0000","last_clean_scrub_stamp":"2024-09-13T18:57:17.999495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:58.108162+0000","scrub_duration":0.0026512860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"43'5","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:12.422278+0000","last_change":"2024-09-13T18:26:31.976872+0000","last_active":"2024-09-13T18:57:12.422278+0000","last_peered":"2024-09-13T18:57:12.422278+0000","last_clean":"2024-09-13T18:57:12.422278+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:57:12.422278+0000","last_undegraded":"2024-09-13T18:57:12.422278+0000","last_fullsized":"2024-09-13T18:57:12.422278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"43'5377","reported_seq":5455,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.589867+0000","last_change":"2024-09-13T18:27:22.648736+0000","last_active":"2024-09-13T18:57:30.589867+0000","last_peered":"2024-09-13T18:57:30.589867+0000","last_clean":"2024-09-13T18:57:30.589867+0000","last_became_active":"2024-09-13T18:27:22.648054+0000","last_became_peered":"2024-09-13T18:27:22.648054+0000","last_unstale":"2024-09-13T18:57:30.589867+0000","last_undegraded":"2024-09-13T18:57:30.589867+0000","last_fullsized":"2024-09-13T18:57:30.589867+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5377,"log_dups_size":0,"ondisk_log_size":5377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":23606,"num_write_kb":471297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"43'19","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.065688+0000","last_change":"2024-09-13T18:57:22.408978+0000","last_active":"2024-09-13T18:57:28.065688+0000","last_peered":"2024-09-13T18:57:28.065688+0000","last_clean":"2024-09-13T18:57:28.065688+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:57:28.065688+0000","last_undegraded":"2024-09-13T18:57:28.065688+0000","last_fullsized":"2024-09-13T18:57:28.065688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:22.408739+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:22.408739+0000","last_clean_scrub_stamp":"2024-09-13T18:57:22.408739+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:45:55.319773+0000","scrub_duration":0.003011952,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"43'9","reported_seq":92,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.906207+0000","last_change":"2024-09-13T18:57:18.823588+0000","last_active":"2024-09-13T18:57:24.906207+0000","last_peered":"2024-09-13T18:57:24.906207+0000","last_clean":"2024-09-13T18:57:24.906207+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:57:24.906207+0000","last_undegraded":"2024-09-13T18:57:24.906207+0000","last_fullsized":"2024-09-13T18:57:24.906207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'7","last_scrub_stamp":"2024-09-13T18:57:18.823152+0000","last_deep_scrub":"43'7","last_deep_scrub_stamp":"2024-09-13T18:57:18.823152+0000","last_clean_scrub_stamp":"2024-09-13T18:57:18.823152+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:52.572271+0000","scrub_duration":0.023118413000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"43'5381","reported_seq":5453,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.621076+0000","last_change":"2024-09-13T18:26:36.410028+0000","last_active":"2024-09-13T18:57:30.621076+0000","last_peered":"2024-09-13T18:57:30.621076+0000","last_clean":"2024-09-13T18:57:30.621076+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:57:30.621076+0000","last_undegraded":"2024-09-13T18:57:30.621076+0000","last_fullsized":"2024-09-13T18:57:30.621076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46137344,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":24441,"num_write_kb":507343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"43'12","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.878010+0000","last_change":"2024-09-13T18:57:17.064133+0000","last_active":"2024-09-13T18:57:24.878010+0000","last_peered":"2024-09-13T18:57:24.878010+0000","last_clean":"2024-09-13T18:57:24.878010+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:57:24.878010+0000","last_undegraded":"2024-09-13T18:57:24.878010+0000","last_fullsized":"2024-09-13T18:57:24.878010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:17.063923+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:17.063923+0000","last_clean_scrub_stamp":"2024-09-13T18:57:17.063923+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.609366+0000","scrub_duration":0.020682738999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"43'289","reported_seq":363,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.907282+0000","last_change":"2024-09-13T18:57:22.143854+0000","last_active":"2024-09-13T18:57:24.907282+0000","last_peered":"2024-09-13T18:57:24.907282+0000","last_clean":"2024-09-13T18:57:24.907282+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:57:24.907282+0000","last_undegraded":"2024-09-13T18:57:24.907282+0000","last_fullsized":"2024-09-13T18:57:24.907282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'286","last_scrub_stamp":"2024-09-13T18:57:22.143716+0000","last_deep_scrub":"43'286","last_deep_scrub_stamp":"2024-09-13T18:57:22.143716+0000","last_clean_scrub_stamp":"2024-09-13T18:57:22.143716+0000","objects_scrubbed":2,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:44.447938+0000","scrub_duration":0.0070881099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":52,"num_write":626,"num_write_kb":212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"43'6066","reported_seq":6148,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.502378+0000","last_change":"2024-09-13T18:27:22.645275+0000","last_active":"2024-09-13T18:57:30.502378+0000","last_peered":"2024-09-13T18:57:30.502378+0000","last_clean":"2024-09-13T18:57:30.502378+0000","last_became_active":"2024-09-13T18:27:22.644681+0000","last_became_peered":"2024-09-13T18:27:22.644681+0000","last_unstale":"2024-09-13T18:57:30.502378+0000","last_undegraded":"2024-09-13T18:57:30.502378+0000","last_fullsized":"2024-09-13T18:57:30.502378+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6066,"log_dups_size":0,"ondisk_log_size":6066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":27880,"num_write_kb":553947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"43'7","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:13.217203+0000","last_change":"2024-09-13T18:57:13.217203+0000","last_active":"2024-09-13T18:57:13.217203+0000","last_peered":"2024-09-13T18:57:13.217203+0000","last_clean":"2024-09-13T18:57:13.217203+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:57:13.217203+0000","last_undegraded":"2024-09-13T18:57:13.217203+0000","last_fullsized":"2024-09-13T18:57:13.217203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:13.216878+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:13.216878+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.216878+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:25:20.911582+0000","scrub_duration":0.0034586249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"43'6","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.418213+0000","last_change":"2024-09-13T18:26:31.978976+0000","last_active":"2024-09-13T18:57:26.418213+0000","last_peered":"2024-09-13T18:57:26.418213+0000","last_clean":"2024-09-13T18:57:26.418213+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:57:26.418213+0000","last_undegraded":"2024-09-13T18:57:26.418213+0000","last_fullsized":"2024-09-13T18:57:26.418213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"43'4743","reported_seq":4807,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.688743+0000","last_change":"2024-09-13T18:26:36.408844+0000","last_active":"2024-09-13T18:57:28.688743+0000","last_peered":"2024-09-13T18:57:28.688743+0000","last_clean":"2024-09-13T18:57:28.688743+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:57:28.688743+0000","last_undegraded":"2024-09-13T18:57:28.688743+0000","last_fullsized":"2024-09-13T18:57:28.688743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":22413,"num_write_kb":450115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"43'11","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.973250+0000","last_change":"2024-09-13T18:57:13.000734+0000","last_active":"2024-09-13T18:57:24.973250+0000","last_peered":"2024-09-13T18:57:24.973250+0000","last_clean":"2024-09-13T18:57:24.973250+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:57:24.973250+0000","last_undegraded":"2024-09-13T18:57:24.973250+0000","last_fullsized":"2024-09-13T18:57:24.973250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'7","last_scrub_stamp":"2024-09-13T18:57:13.000480+0000","last_deep_scrub":"28'7","last_deep_scrub_stamp":"2024-09-13T18:57:13.000480+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.000480+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:33.028238+0000","scrub_duration":0.0036870150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"43'1","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:14.228792+0000","last_change":"2024-09-13T18:57:14.228792+0000","last_active":"2024-09-13T18:57:14.228792+0000","last_peered":"2024-09-13T18:57:14.228792+0000","last_clean":"2024-09-13T18:57:14.228792+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:57:14.228792+0000","last_undegraded":"2024-09-13T18:57:14.228792+0000","last_fullsized":"2024-09-13T18:57:14.228792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:57:14.228519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:57:14.228519+0000","last_clean_scrub_stamp":"2024-09-13T18:57:14.228519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:55.913907+0000","scrub_duration":0.0032354189999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"43'6104","reported_seq":6174,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:25.870972+0000","last_change":"2024-09-13T18:26:36.398876+0000","last_active":"2024-09-13T18:57:25.870972+0000","last_peered":"2024-09-13T18:57:25.870972+0000","last_clean":"2024-09-13T18:57:25.870972+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:57:25.870972+0000","last_undegraded":"2024-09-13T18:57:25.870972+0000","last_fullsized":"2024-09-13T18:57:25.870972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6104,"log_dups_size":0,"ondisk_log_size":6104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146800640,"num_objects":35,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":28032,"num_write_kb":562911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"43'9","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.858553+0000","last_change":"2024-09-13T18:57:12.010586+0000","last_active":"2024-09-13T18:57:24.858553+0000","last_peered":"2024-09-13T18:57:24.858553+0000","last_clean":"2024-09-13T18:57:24.858553+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:57:24.858553+0000","last_undegraded":"2024-09-13T18:57:24.858553+0000","last_fullsized":"2024-09-13T18:57:24.858553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'6","last_scrub_stamp":"2024-09-13T18:57:12.010428+0000","last_deep_scrub":"28'6","last_deep_scrub_stamp":"2024-09-13T18:57:12.010428+0000","last_clean_scrub_stamp":"2024-09-13T18:57:12.010428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:15.823406+0000","scrub_duration":0.0039755040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"43'9","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.921523+0000","last_change":"2024-09-13T18:57:15.993201+0000","last_active":"2024-09-13T18:57:24.921523+0000","last_peered":"2024-09-13T18:57:24.921523+0000","last_clean":"2024-09-13T18:57:24.921523+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:57:24.921523+0000","last_undegraded":"2024-09-13T18:57:24.921523+0000","last_fullsized":"2024-09-13T18:57:24.921523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'6","last_scrub_stamp":"2024-09-13T18:57:15.992890+0000","last_deep_scrub":"41'6","last_deep_scrub_stamp":"2024-09-13T18:57:15.992890+0000","last_clean_scrub_stamp":"2024-09-13T18:57:15.992890+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:28:43.312440+0000","scrub_duration":0.0036839949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"43'5363","reported_seq":5436,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.443525+0000","last_change":"2024-09-13T18:26:36.401777+0000","last_active":"2024-09-13T18:57:26.443525+0000","last_peered":"2024-09-13T18:57:26.443525+0000","last_clean":"2024-09-13T18:57:26.443525+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:57:26.443525+0000","last_undegraded":"2024-09-13T18:57:26.443525+0000","last_fullsized":"2024-09-13T18:57:26.443525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5363,"log_dups_size":0,"ondisk_log_size":5363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113247261,"num_objects":29,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":24516,"num_write_kb":507684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"43'12","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.825850+0000","last_change":"2024-09-13T18:26:32.343791+0000","last_active":"2024-09-13T18:57:24.825850+0000","last_peered":"2024-09-13T18:57:24.825850+0000","last_clean":"2024-09-13T18:57:24.825850+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:57:24.825850+0000","last_undegraded":"2024-09-13T18:57:24.825850+0000","last_fullsized":"2024-09-13T18:57:24.825850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"43'131","reported_seq":317,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:16.151559+0000","last_change":"2024-09-13T18:57:16.151559+0000","last_active":"2024-09-13T18:57:16.151559+0000","last_peered":"2024-09-13T18:57:16.151559+0000","last_clean":"2024-09-13T18:57:16.151559+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:57:16.151559+0000","last_undegraded":"2024-09-13T18:57:16.151559+0000","last_fullsized":"2024-09-13T18:57:16.151559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'130","last_scrub_stamp":"2024-09-13T18:57:16.151397+0000","last_deep_scrub":"43'130","last_deep_scrub_stamp":"2024-09-13T18:57:16.151397+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.151397+0000","objects_scrubbed":3,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:19.471742+0000","scrub_duration":0.027827343000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93931,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":203,"num_write":119,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"43'6248","reported_seq":6319,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.582005+0000","last_change":"2024-09-13T18:26:36.401529+0000","last_active":"2024-09-13T18:57:26.582005+0000","last_peered":"2024-09-13T18:57:26.582005+0000","last_clean":"2024-09-13T18:57:26.582005+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:57:26.582005+0000","last_undegraded":"2024-09-13T18:57:26.582005+0000","last_fullsized":"2024-09-13T18:57:26.582005+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6248,"log_dups_size":0,"ondisk_log_size":6248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155189362,"num_objects":38,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":28265,"num_write_kb":563741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"43'8","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.899449+0000","last_change":"2024-09-13T18:57:14.329551+0000","last_active":"2024-09-13T18:57:24.899449+0000","last_peered":"2024-09-13T18:57:24.899449+0000","last_clean":"2024-09-13T18:57:24.899449+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:57:24.899449+0000","last_undegraded":"2024-09-13T18:57:24.899449+0000","last_fullsized":"2024-09-13T18:57:24.899449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:14.329297+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:14.329297+0000","last_clean_scrub_stamp":"2024-09-13T18:57:14.329297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:49.431750+0000","scrub_duration":0.0033135230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"43'17","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.912474+0000","last_change":"2024-09-13T18:57:26.912770+0000","last_active":"2024-09-13T18:57:26.912474+0000","last_peered":"2024-09-13T18:57:26.912474+0000","last_clean":"2024-09-13T18:57:26.912474+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:57:26.912474+0000","last_undegraded":"2024-09-13T18:57:26.912474+0000","last_fullsized":"2024-09-13T18:57:26.912474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'17","last_scrub_stamp":"2024-09-13T18:57:26.912423+0000","last_deep_scrub":"43'17","last_deep_scrub_stamp":"2024-09-13T18:57:26.912423+0000","last_clean_scrub_stamp":"2024-09-13T18:57:26.912423+0000","objects_scrubbed":1,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:00.997526+0000","scrub_duration":0.0065643969999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"43'4433","reported_seq":4504,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.870531+0000","last_change":"2024-09-13T18:26:36.405086+0000","last_active":"2024-09-13T18:57:28.870531+0000","last_peered":"2024-09-13T18:57:28.870531+0000","last_clean":"2024-09-13T18:57:28.870531+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:57:28.870531+0000","last_undegraded":"2024-09-13T18:57:28.870531+0000","last_fullsized":"2024-09-13T18:57:28.870531+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4433,"log_dups_size":0,"ondisk_log_size":4433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":19848,"num_write_kb":435588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"43'6","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:10.394438+0000","last_change":"2024-09-13T18:26:32.350510+0000","last_active":"2024-09-13T18:57:10.394438+0000","last_peered":"2024-09-13T18:57:10.394438+0000","last_clean":"2024-09-13T18:57:10.394438+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:57:10.394438+0000","last_undegraded":"2024-09-13T18:57:10.394438+0000","last_fullsized":"2024-09-13T18:57:10.394438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"43'11","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:23.965863+0000","last_change":"2024-09-13T18:57:23.965985+0000","last_active":"2024-09-13T18:57:23.965863+0000","last_peered":"2024-09-13T18:57:23.965863+0000","last_clean":"2024-09-13T18:57:23.965863+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:57:23.965863+0000","last_undegraded":"2024-09-13T18:57:23.965863+0000","last_fullsized":"2024-09-13T18:57:23.965863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:23.965843+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:23.965843+0000","last_clean_scrub_stamp":"2024-09-13T18:57:23.965843+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:51.676617+0000","scrub_duration":0.010225193,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":68,"num_write":9,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"43'119","reported_seq":215,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:19.933586+0000","last_change":"2024-09-13T18:57:19.933697+0000","last_active":"2024-09-13T18:57:19.933586+0000","last_peered":"2024-09-13T18:57:19.933586+0000","last_clean":"2024-09-13T18:57:19.933586+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:57:19.933586+0000","last_undegraded":"2024-09-13T18:57:19.933586+0000","last_fullsized":"2024-09-13T18:57:19.933586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'119","last_scrub_stamp":"2024-09-13T18:57:19.933561+0000","last_deep_scrub":"43'119","last_deep_scrub_stamp":"2024-09-13T18:57:19.933561+0000","last_clean_scrub_stamp":"2024-09-13T18:57:19.933561+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:36.963255+0000","scrub_duration":0.0099632039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"43'5948","reported_seq":6028,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:25.097945+0000","last_change":"2024-09-13T18:26:36.409827+0000","last_active":"2024-09-13T18:57:25.097945+0000","last_peered":"2024-09-13T18:57:25.097945+0000","last_clean":"2024-09-13T18:57:25.097945+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:57:25.097945+0000","last_undegraded":"2024-09-13T18:57:25.097945+0000","last_fullsized":"2024-09-13T18:57:25.097945+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5948,"log_dups_size":0,"ondisk_log_size":5948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138420445,"num_objects":35,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":27386,"num_write_kb":519330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"43'13","reported_seq":59,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:22.796812+0000","last_change":"2024-09-13T18:26:32.356850+0000","last_active":"2024-09-13T18:57:22.796812+0000","last_peered":"2024-09-13T18:57:22.796812+0000","last_clean":"2024-09-13T18:57:22.796812+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:57:22.796812+0000","last_undegraded":"2024-09-13T18:57:22.796812+0000","last_fullsized":"2024-09-13T18:57:22.796812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"43'8","reported_seq":81,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:22.774832+0000","last_change":"2024-09-13T18:57:21.187725+0000","last_active":"2024-09-13T18:57:22.774832+0000","last_peered":"2024-09-13T18:57:22.774832+0000","last_clean":"2024-09-13T18:57:22.774832+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:57:22.774832+0000","last_undegraded":"2024-09-13T18:57:22.774832+0000","last_fullsized":"2024-09-13T18:57:22.774832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:21.187525+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:21.187525+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.187525+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:42.675115+0000","scrub_duration":0.032458872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":35,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"43'5376","reported_seq":5441,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.505832+0000","last_change":"2024-09-13T18:26:36.409802+0000","last_active":"2024-09-13T18:57:26.505832+0000","last_peered":"2024-09-13T18:57:26.505832+0000","last_clean":"2024-09-13T18:57:26.505832+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:57:26.505832+0000","last_undegraded":"2024-09-13T18:57:26.505832+0000","last_fullsized":"2024-09-13T18:57:26.505832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5376,"log_dups_size":0,"ondisk_log_size":5376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857788,"num_objects":26,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":24041,"num_write_kb":513904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"43'13","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.970242+0000","last_change":"2024-09-13T18:27:22.610607+0000","last_active":"2024-09-13T18:57:24.970242+0000","last_peered":"2024-09-13T18:57:24.970242+0000","last_clean":"2024-09-13T18:57:24.970242+0000","last_became_active":"2024-09-13T18:27:22.610326+0000","last_became_peered":"2024-09-13T18:27:22.610326+0000","last_unstale":"2024-09-13T18:57:24.970242+0000","last_undegraded":"2024-09-13T18:57:24.970242+0000","last_fullsized":"2024-09-13T18:57:24.970242+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:08.808982+0000","last_change":"2024-09-13T18:26:31.966618+0000","last_active":"2024-09-13T18:57:08.808982+0000","last_peered":"2024-09-13T18:57:08.808982+0000","last_clean":"2024-09-13T18:57:08.808982+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:57:08.808982+0000","last_undegraded":"2024-09-13T18:57:08.808982+0000","last_fullsized":"2024-09-13T18:57:08.808982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"43'5121","reported_seq":5189,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.682144+0000","last_change":"2024-09-13T18:26:36.394344+0000","last_active":"2024-09-13T18:57:28.682144+0000","last_peered":"2024-09-13T18:57:28.682144+0000","last_clean":"2024-09-13T18:57:28.682144+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:57:28.682144+0000","last_undegraded":"2024-09-13T18:57:28.682144+0000","last_fullsized":"2024-09-13T18:57:28.682144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5121,"log_dups_size":0,"ondisk_log_size":5121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":22806,"num_write_kb":472029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"43'10","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:25.729365+0000","last_change":"2024-09-13T18:26:32.365655+0000","last_active":"2024-09-13T18:57:25.729365+0000","last_peered":"2024-09-13T18:57:25.729365+0000","last_clean":"2024-09-13T18:57:25.729365+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:57:25.729365+0000","last_undegraded":"2024-09-13T18:57:25.729365+0000","last_fullsized":"2024-09-13T18:57:25.729365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"43'10","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.389453+0000","last_change":"2024-09-13T18:57:25.103279+0000","last_active":"2024-09-13T18:57:27.389453+0000","last_peered":"2024-09-13T18:57:27.389453+0000","last_clean":"2024-09-13T18:57:27.389453+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:57:27.389453+0000","last_undegraded":"2024-09-13T18:57:27.389453+0000","last_fullsized":"2024-09-13T18:57:27.389453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:25.103144+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:25.103144+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.103144+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.183107+0000","scrub_duration":0.013012788000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"43'4960","reported_seq":5022,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.820116+0000","last_change":"2024-09-13T18:26:36.410280+0000","last_active":"2024-09-13T18:57:28.820116+0000","last_peered":"2024-09-13T18:57:28.820116+0000","last_clean":"2024-09-13T18:57:28.820116+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:57:28.820116+0000","last_undegraded":"2024-09-13T18:57:28.820116+0000","last_fullsized":"2024-09-13T18:57:28.820116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4960,"log_dups_size":0,"ondisk_log_size":4960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":17,"num_read_kb":0,"num_write":22761,"num_write_kb":463670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"43'4","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:12.422324+0000","last_change":"2024-09-13T18:26:32.361575+0000","last_active":"2024-09-13T18:57:12.422324+0000","last_peered":"2024-09-13T18:57:12.422324+0000","last_clean":"2024-09-13T18:57:12.422324+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:57:12.422324+0000","last_undegraded":"2024-09-13T18:57:12.422324+0000","last_fullsized":"2024-09-13T18:57:12.422324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"43'12","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:11.025964+0000","last_change":"2024-09-13T18:57:11.025964+0000","last_active":"2024-09-13T18:57:11.025964+0000","last_peered":"2024-09-13T18:57:11.025964+0000","last_clean":"2024-09-13T18:57:11.025964+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:57:11.025964+0000","last_undegraded":"2024-09-13T18:57:11.025964+0000","last_fullsized":"2024-09-13T18:57:11.025964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:11.025748+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:11.025748+0000","last_clean_scrub_stamp":"2024-09-13T18:57:11.025748+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:22:07.957036+0000","scrub_duration":0.0065355530000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"43'5113","reported_seq":5179,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.738029+0000","last_change":"2024-09-13T18:26:36.403479+0000","last_active":"2024-09-13T18:57:30.738029+0000","last_peered":"2024-09-13T18:57:30.738029+0000","last_clean":"2024-09-13T18:57:30.738029+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:57:30.738029+0000","last_undegraded":"2024-09-13T18:57:30.738029+0000","last_fullsized":"2024-09-13T18:57:30.738029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5113,"log_dups_size":0,"ondisk_log_size":5113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":22696,"num_write_kb":463382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"43'12","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:22.204108+0000","last_change":"2024-09-13T18:26:32.365143+0000","last_active":"2024-09-13T18:57:22.204108+0000","last_peered":"2024-09-13T18:57:22.204108+0000","last_clean":"2024-09-13T18:57:22.204108+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:57:22.204108+0000","last_undegraded":"2024-09-13T18:57:22.204108+0000","last_fullsized":"2024-09-13T18:57:22.204108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"43'32","reported_seq":149,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.902980+0000","last_change":"2024-09-13T18:57:20.154497+0000","last_active":"2024-09-13T18:57:24.902980+0000","last_peered":"2024-09-13T18:57:24.902980+0000","last_clean":"2024-09-13T18:57:24.902980+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:57:24.902980+0000","last_undegraded":"2024-09-13T18:57:24.902980+0000","last_fullsized":"2024-09-13T18:57:24.902980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'29","last_scrub_stamp":"2024-09-13T18:57:20.154238+0000","last_deep_scrub":"43'29","last_deep_scrub_stamp":"2024-09-13T18:57:20.154238+0000","last_clean_scrub_stamp":"2024-09-13T18:57:20.154238+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:10.657893+0000","scrub_duration":0.013053932000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":158,"num_read_kb":157,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"43'6023","reported_seq":6097,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.725418+0000","last_change":"2024-09-13T18:26:36.399235+0000","last_active":"2024-09-13T18:57:28.725418+0000","last_peered":"2024-09-13T18:57:28.725418+0000","last_clean":"2024-09-13T18:57:28.725418+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:57:28.725418+0000","last_undegraded":"2024-09-13T18:57:28.725418+0000","last_fullsized":"2024-09-13T18:57:28.725418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6023,"log_dups_size":0,"ondisk_log_size":6023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":27714,"num_write_kb":536907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"43'12","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:25.726347+0000","last_change":"2024-09-13T18:26:32.357403+0000","last_active":"2024-09-13T18:57:25.726347+0000","last_peered":"2024-09-13T18:57:25.726347+0000","last_clean":"2024-09-13T18:57:25.726347+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:57:25.726347+0000","last_undegraded":"2024-09-13T18:57:25.726347+0000","last_fullsized":"2024-09-13T18:57:25.726347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"43'15","reported_seq":87,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:21.771790+0000","last_change":"2024-09-13T18:57:21.771790+0000","last_active":"2024-09-13T18:57:21.771790+0000","last_peered":"2024-09-13T18:57:21.771790+0000","last_clean":"2024-09-13T18:57:21.771790+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:57:21.771790+0000","last_undegraded":"2024-09-13T18:57:21.771790+0000","last_fullsized":"2024-09-13T18:57:21.771790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:21.771519+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:21.771519+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.771519+0000","objects_scrubbed":2,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:44:46.293266+0000","scrub_duration":0.0083382200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1620,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"43'6119","reported_seq":6186,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.647064+0000","last_change":"2024-09-13T18:26:36.406177+0000","last_active":"2024-09-13T18:57:30.647064+0000","last_peered":"2024-09-13T18:57:30.647064+0000","last_clean":"2024-09-13T18:57:30.647064+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:57:30.647064+0000","last_undegraded":"2024-09-13T18:57:30.647064+0000","last_fullsized":"2024-09-13T18:57:30.647064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6119,"log_dups_size":0,"ondisk_log_size":6119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":28249,"num_write_kb":551294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"43'14","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.849841+0000","last_change":"2024-09-13T18:27:22.609309+0000","last_active":"2024-09-13T18:57:24.849841+0000","last_peered":"2024-09-13T18:57:24.849841+0000","last_clean":"2024-09-13T18:57:24.849841+0000","last_became_active":"2024-09-13T18:27:22.609101+0000","last_became_peered":"2024-09-13T18:27:22.609101+0000","last_unstale":"2024-09-13T18:57:24.849841+0000","last_undegraded":"2024-09-13T18:57:24.849841+0000","last_fullsized":"2024-09-13T18:57:24.849841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"43'13","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.642980+0000","last_change":"2024-09-13T18:26:31.982114+0000","last_active":"2024-09-13T18:57:27.642980+0000","last_peered":"2024-09-13T18:57:27.642980+0000","last_clean":"2024-09-13T18:57:27.642980+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:57:27.642980+0000","last_undegraded":"2024-09-13T18:57:27.642980+0000","last_fullsized":"2024-09-13T18:57:27.642980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"43'5912","reported_seq":5983,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.558742+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:57:30.558742+0000","last_peered":"2024-09-13T18:57:30.558742+0000","last_clean":"2024-09-13T18:57:30.558742+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:57:30.558742+0000","last_undegraded":"2024-09-13T18:57:30.558742+0000","last_fullsized":"2024-09-13T18:57:30.558742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5912,"log_dups_size":0,"ondisk_log_size":5912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27104,"num_write_kb":550847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"43'13","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:25.418872+0000","last_change":"2024-09-13T18:57:25.418872+0000","last_active":"2024-09-13T18:57:25.418872+0000","last_peered":"2024-09-13T18:57:25.418872+0000","last_clean":"2024-09-13T18:57:25.418872+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:57:25.418872+0000","last_undegraded":"2024-09-13T18:57:25.418872+0000","last_fullsized":"2024-09-13T18:57:25.418872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:57:25.418501+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:57:25.418501+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.418501+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:32.901503+0000","scrub_duration":0.003478723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"43'20","reported_seq":111,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.419225+0000","last_change":"2024-09-13T18:57:13.300861+0000","last_active":"2024-09-13T18:57:26.419225+0000","last_peered":"2024-09-13T18:57:26.419225+0000","last_clean":"2024-09-13T18:57:26.419225+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:57:26.419225+0000","last_undegraded":"2024-09-13T18:57:26.419225+0000","last_fullsized":"2024-09-13T18:57:26.419225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:13.300701+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:13.300701+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.300701+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:09.337133+0000","scrub_duration":0.0071490659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":93,"num_read_kb":112,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"43'5253","reported_seq":5319,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:29.739364+0000","last_change":"2024-09-13T18:26:36.410014+0000","last_active":"2024-09-13T18:57:29.739364+0000","last_peered":"2024-09-13T18:57:29.739364+0000","last_clean":"2024-09-13T18:57:29.739364+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:57:29.739364+0000","last_undegraded":"2024-09-13T18:57:29.739364+0000","last_fullsized":"2024-09-13T18:57:29.739364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5253,"log_dups_size":0,"ondisk_log_size":5253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":23220,"num_write_kb":490059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"29'9","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:55:36.402792+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:55:36.402792+0000","last_peered":"2024-09-13T18:55:36.402792+0000","last_clean":"2024-09-13T18:55:36.402792+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:55:36.402792+0000","last_undegraded":"2024-09-13T18:55:36.402792+0000","last_fullsized":"2024-09-13T18:55:36.402792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"43'389","reported_seq":488,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.908598+0000","last_change":"2024-09-13T18:57:13.774570+0000","last_active":"2024-09-13T18:57:24.908598+0000","last_peered":"2024-09-13T18:57:24.908598+0000","last_clean":"2024-09-13T18:57:24.908598+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:57:24.908598+0000","last_undegraded":"2024-09-13T18:57:24.908598+0000","last_fullsized":"2024-09-13T18:57:24.908598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'387","last_scrub_stamp":"2024-09-13T18:57:13.774429+0000","last_deep_scrub":"43'387","last_deep_scrub_stamp":"2024-09-13T18:57:13.774429+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.774429+0000","objects_scrubbed":5,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:38.043525+0000","scrub_duration":0.046187055999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":93,"num_read_kb":20568,"num_write":383,"num_write_kb":4318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"43'5414","reported_seq":5481,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.387576+0000","last_change":"2024-09-13T18:26:36.394080+0000","last_active":"2024-09-13T18:57:26.387576+0000","last_peered":"2024-09-13T18:57:26.387576+0000","last_clean":"2024-09-13T18:57:26.387576+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:57:26.387576+0000","last_undegraded":"2024-09-13T18:57:26.387576+0000","last_fullsized":"2024-09-13T18:57:26.387576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5414,"log_dups_size":0,"ondisk_log_size":5414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113343190,"num_objects":30,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":25004,"num_write_kb":485267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"43'10","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.055388+0000","last_change":"2024-09-13T18:57:16.962251+0000","last_active":"2024-09-13T18:57:28.055388+0000","last_peered":"2024-09-13T18:57:28.055388+0000","last_clean":"2024-09-13T18:57:28.055388+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:57:28.055388+0000","last_undegraded":"2024-09-13T18:57:28.055388+0000","last_fullsized":"2024-09-13T18:57:28.055388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:16.961847+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:16.961847+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.961847+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:54.684373+0000","scrub_duration":0.0040550839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"43'62","reported_seq":125,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:08.809096+0000","last_change":"2024-09-13T18:26:31.976401+0000","last_active":"2024-09-13T18:57:08.809096+0000","last_peered":"2024-09-13T18:57:08.809096+0000","last_clean":"2024-09-13T18:57:08.809096+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:57:08.809096+0000","last_undegraded":"2024-09-13T18:57:08.809096+0000","last_fullsized":"2024-09-13T18:57:08.809096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:30.927447+0000","last_clean_scrub_stamp":"2024-09-13T18:26:30.927447+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":63,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"43'5934","reported_seq":6005,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.639163+0000","last_change":"2024-09-13T18:26:36.406228+0000","last_active":"2024-09-13T18:57:30.639163+0000","last_peered":"2024-09-13T18:57:30.639163+0000","last_clean":"2024-09-13T18:57:30.639163+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:57:30.639163+0000","last_undegraded":"2024-09-13T18:57:30.639163+0000","last_fullsized":"2024-09-13T18:57:30.639163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5934,"log_dups_size":0,"ondisk_log_size":5934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27454,"num_write_kb":527755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"43'16","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:17.204016+0000","last_change":"2024-09-13T18:26:32.350841+0000","last_active":"2024-09-13T18:57:17.204016+0000","last_peered":"2024-09-13T18:57:17.204016+0000","last_clean":"2024-09-13T18:57:17.204016+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:57:17.204016+0000","last_undegraded":"2024-09-13T18:57:17.204016+0000","last_fullsized":"2024-09-13T18:57:17.204016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"43'6","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:22.784542+0000","last_change":"2024-09-13T18:57:18.948854+0000","last_active":"2024-09-13T18:57:22.784542+0000","last_peered":"2024-09-13T18:57:22.784542+0000","last_clean":"2024-09-13T18:57:22.784542+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:57:22.784542+0000","last_undegraded":"2024-09-13T18:57:22.784542+0000","last_fullsized":"2024-09-13T18:57:22.784542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4","last_scrub_stamp":"2024-09-13T18:57:18.948753+0000","last_deep_scrub":"43'4","last_deep_scrub_stamp":"2024-09-13T18:57:18.948753+0000","last_clean_scrub_stamp":"2024-09-13T18:57:18.948753+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:24:26.277291+0000","scrub_duration":0.0026056769999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"43'5465","reported_seq":5527,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.667220+0000","last_change":"2024-09-13T18:26:36.406090+0000","last_active":"2024-09-13T18:57:30.667220+0000","last_peered":"2024-09-13T18:57:30.667220+0000","last_clean":"2024-09-13T18:57:30.667220+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:57:30.667220+0000","last_undegraded":"2024-09-13T18:57:30.667220+0000","last_fullsized":"2024-09-13T18:57:30.667220+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5465,"log_dups_size":0,"ondisk_log_size":5465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":25024,"num_write_kb":489215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"43'13","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.044599+0000","last_change":"2024-09-13T18:26:32.357701+0000","last_active":"2024-09-13T18:57:28.044599+0000","last_peered":"2024-09-13T18:57:28.044599+0000","last_clean":"2024-09-13T18:57:28.044599+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:57:28.044599+0000","last_undegraded":"2024-09-13T18:57:28.044599+0000","last_fullsized":"2024-09-13T18:57:28.044599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"43'19","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:22.765983+0000","last_change":"2024-09-13T18:57:22.765983+0000","last_active":"2024-09-13T18:57:22.765983+0000","last_peered":"2024-09-13T18:57:22.765983+0000","last_clean":"2024-09-13T18:57:22.765983+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:57:22.765983+0000","last_undegraded":"2024-09-13T18:57:22.765983+0000","last_fullsized":"2024-09-13T18:57:22.765983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'18","last_scrub_stamp":"2024-09-13T18:57:22.765727+0000","last_deep_scrub":"43'18","last_deep_scrub_stamp":"2024-09-13T18:57:22.765727+0000","last_clean_scrub_stamp":"2024-09-13T18:57:22.765727+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:04.965728+0000","scrub_duration":0.0065666600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":18,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"43'5139","reported_seq":5206,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.328518+0000","last_change":"2024-09-13T18:26:36.406553+0000","last_active":"2024-09-13T18:57:30.328518+0000","last_peered":"2024-09-13T18:57:30.328518+0000","last_clean":"2024-09-13T18:57:30.328518+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:57:30.328518+0000","last_undegraded":"2024-09-13T18:57:30.328518+0000","last_fullsized":"2024-09-13T18:57:30.328518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5139,"log_dups_size":0,"ondisk_log_size":5139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":23325,"num_write_kb":507027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"43'6133","reported_seq":6204,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.902378+0000","last_change":"2024-09-13T18:26:36.406986+0000","last_active":"2024-09-13T18:57:28.902378+0000","last_peered":"2024-09-13T18:57:28.902378+0000","last_clean":"2024-09-13T18:57:28.902378+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:57:28.902378+0000","last_undegraded":"2024-09-13T18:57:28.902378+0000","last_fullsized":"2024-09-13T18:57:28.902378+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6133,"log_dups_size":0,"ondisk_log_size":6133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":28540,"num_write_kb":581141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"43'9","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.893097+0000","last_change":"2024-09-13T18:26:32.349081+0000","last_active":"2024-09-13T18:57:24.893097+0000","last_peered":"2024-09-13T18:57:24.893097+0000","last_clean":"2024-09-13T18:57:24.893097+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:57:24.893097+0000","last_undegraded":"2024-09-13T18:57:24.893097+0000","last_fullsized":"2024-09-13T18:57:24.893097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"43'8","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:21.964416+0000","last_change":"2024-09-13T18:57:21.964628+0000","last_active":"2024-09-13T18:57:21.964416+0000","last_peered":"2024-09-13T18:57:21.964416+0000","last_clean":"2024-09-13T18:57:21.964416+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:57:21.964416+0000","last_undegraded":"2024-09-13T18:57:21.964416+0000","last_fullsized":"2024-09-13T18:57:21.964416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:21.964364+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:21.964364+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.964364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:31:05.131047+0000","scrub_duration":0.003227541,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"43'5586","reported_seq":5659,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.600366+0000","last_change":"2024-09-13T18:26:36.396418+0000","last_active":"2024-09-13T18:57:26.600366+0000","last_peered":"2024-09-13T18:57:26.600366+0000","last_clean":"2024-09-13T18:57:26.600366+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:57:26.600366+0000","last_undegraded":"2024-09-13T18:57:26.600366+0000","last_fullsized":"2024-09-13T18:57:26.600366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5586,"log_dups_size":0,"ondisk_log_size":5586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100673806,"num_objects":26,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":25581,"num_write_kb":520259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"43'19","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.024605+0000","last_change":"2024-09-13T18:26:32.351369+0000","last_active":"2024-09-13T18:57:28.024605+0000","last_peered":"2024-09-13T18:57:28.024605+0000","last_clean":"2024-09-13T18:57:28.024605+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:57:28.024605+0000","last_undegraded":"2024-09-13T18:57:28.024605+0000","last_fullsized":"2024-09-13T18:57:28.024605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"43'6","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:21.960612+0000","last_change":"2024-09-13T18:57:21.960612+0000","last_active":"2024-09-13T18:57:21.960612+0000","last_peered":"2024-09-13T18:57:21.960612+0000","last_clean":"2024-09-13T18:57:21.960612+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:57:21.960612+0000","last_undegraded":"2024-09-13T18:57:21.960612+0000","last_fullsized":"2024-09-13T18:57:21.960612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:21.960203+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:21.960203+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.960203+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:59:06.847239+0000","scrub_duration":0.006849478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"43'11","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:16.794490+0000","last_change":"2024-09-13T18:57:16.794490+0000","last_active":"2024-09-13T18:57:16.794490+0000","last_peered":"2024-09-13T18:57:16.794490+0000","last_clean":"2024-09-13T18:57:16.794490+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:57:16.794490+0000","last_undegraded":"2024-09-13T18:57:16.794490+0000","last_fullsized":"2024-09-13T18:57:16.794490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:16.794241+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:16.794241+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.794241+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:35.699392+0000","scrub_duration":0.0056238700000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"43'9","reported_seq":53,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:08.026432+0000","last_change":"2024-09-13T18:26:32.345825+0000","last_active":"2024-09-13T18:57:08.026432+0000","last_peered":"2024-09-13T18:57:08.026432+0000","last_clean":"2024-09-13T18:57:08.026432+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:57:08.026432+0000","last_undegraded":"2024-09-13T18:57:08.026432+0000","last_fullsized":"2024-09-13T18:57:08.026432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"43'4168","reported_seq":4231,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.548826+0000","last_change":"2024-09-13T18:26:36.391520+0000","last_active":"2024-09-13T18:57:30.548826+0000","last_peered":"2024-09-13T18:57:30.548826+0000","last_clean":"2024-09-13T18:57:30.548826+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:57:30.548826+0000","last_undegraded":"2024-09-13T18:57:30.548826+0000","last_fullsized":"2024-09-13T18:57:30.548826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4168,"log_dups_size":0,"ondisk_log_size":4168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":19205,"num_write_kb":424944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"43'759","reported_seq":843,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.910037+0000","last_change":"2024-09-13T18:57:28.910129+0000","last_active":"2024-09-13T18:57:28.910037+0000","last_peered":"2024-09-13T18:57:28.910037+0000","last_clean":"2024-09-13T18:57:28.910037+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:57:28.910037+0000","last_undegraded":"2024-09-13T18:57:28.910037+0000","last_fullsized":"2024-09-13T18:57:28.910037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'759","last_scrub_stamp":"2024-09-13T18:57:28.910019+0000","last_deep_scrub":"43'759","last_deep_scrub_stamp":"2024-09-13T18:57:28.910019+0000","last_clean_scrub_stamp":"2024-09-13T18:57:28.910019+0000","objects_scrubbed":3,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:04.313816+0000","scrub_duration":0.042299119000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196273,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":19627,"num_write":735,"num_write_kb":4345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"43'13","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.967148+0000","last_change":"2024-09-13T18:57:20.986740+0000","last_active":"2024-09-13T18:57:24.967148+0000","last_peered":"2024-09-13T18:57:24.967148+0000","last_clean":"2024-09-13T18:57:24.967148+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:57:24.967148+0000","last_undegraded":"2024-09-13T18:57:24.967148+0000","last_fullsized":"2024-09-13T18:57:24.967148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:20.986473+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:20.986473+0000","last_clean_scrub_stamp":"2024-09-13T18:57:20.986473+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:56.996793+0000","scrub_duration":0.0033945970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"43'5979","reported_seq":6048,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.477623+0000","last_change":"2024-09-13T18:26:36.403817+0000","last_active":"2024-09-13T18:57:30.477623+0000","last_peered":"2024-09-13T18:57:30.477623+0000","last_clean":"2024-09-13T18:57:30.477623+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:57:30.477623+0000","last_undegraded":"2024-09-13T18:57:30.477623+0000","last_fullsized":"2024-09-13T18:57:30.477623+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5979,"log_dups_size":0,"ondisk_log_size":5979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":26780,"num_write_kb":557751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"43'15","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.930673+0000","last_change":"2024-09-13T18:26:32.357856+0000","last_active":"2024-09-13T18:57:24.930673+0000","last_peered":"2024-09-13T18:57:24.930673+0000","last_clean":"2024-09-13T18:57:24.930673+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:57:24.930673+0000","last_undegraded":"2024-09-13T18:57:24.930673+0000","last_fullsized":"2024-09-13T18:57:24.930673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"43'12","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:31.096999+0000","last_change":"2024-09-13T18:57:31.097096+0000","last_active":"2024-09-13T18:57:31.096999+0000","last_peered":"2024-09-13T18:57:31.096999+0000","last_clean":"2024-09-13T18:57:31.096999+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:57:31.096999+0000","last_undegraded":"2024-09-13T18:57:31.096999+0000","last_fullsized":"2024-09-13T18:57:31.096999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:57:31.096973+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:57:31.096973+0000","last_clean_scrub_stamp":"2024-09-13T18:57:31.096973+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:08.952276+0000","scrub_duration":0.0027320500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"43'15","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:17.299110+0000","last_change":"2024-09-13T18:26:32.355806+0000","last_active":"2024-09-13T18:57:17.299110+0000","last_peered":"2024-09-13T18:57:17.299110+0000","last_clean":"2024-09-13T18:57:17.299110+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:57:17.299110+0000","last_undegraded":"2024-09-13T18:57:17.299110+0000","last_fullsized":"2024-09-13T18:57:17.299110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"43'9","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:08.026490+0000","last_change":"2024-09-13T18:26:32.340965+0000","last_active":"2024-09-13T18:57:08.026490+0000","last_peered":"2024-09-13T18:57:08.026490+0000","last_clean":"2024-09-13T18:57:08.026490+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:57:08.026490+0000","last_undegraded":"2024-09-13T18:57:08.026490+0000","last_fullsized":"2024-09-13T18:57:08.026490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"43'10","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.421715+0000","last_change":"2024-09-13T18:57:19.006871+0000","last_active":"2024-09-13T18:57:26.421715+0000","last_peered":"2024-09-13T18:57:26.421715+0000","last_clean":"2024-09-13T18:57:26.421715+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:57:26.421715+0000","last_undegraded":"2024-09-13T18:57:26.421715+0000","last_fullsized":"2024-09-13T18:57:26.421715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'7","last_scrub_stamp":"2024-09-13T18:57:19.006675+0000","last_deep_scrub":"29'7","last_deep_scrub_stamp":"2024-09-13T18:57:19.006675+0000","last_clean_scrub_stamp":"2024-09-13T18:57:19.006675+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:53.159267+0000","scrub_duration":0.012523802000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"43'8","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:18.813158+0000","last_change":"2024-09-13T18:26:32.349170+0000","last_active":"2024-09-13T18:57:18.813158+0000","last_peered":"2024-09-13T18:57:18.813158+0000","last_clean":"2024-09-13T18:57:18.813158+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:57:18.813158+0000","last_undegraded":"2024-09-13T18:57:18.813158+0000","last_fullsized":"2024-09-13T18:57:18.813158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"43'7","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.709839+0000","last_change":"2024-09-13T18:57:24.709839+0000","last_active":"2024-09-13T18:57:24.709839+0000","last_peered":"2024-09-13T18:57:24.709839+0000","last_clean":"2024-09-13T18:57:24.709839+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:57:24.709839+0000","last_undegraded":"2024-09-13T18:57:24.709839+0000","last_fullsized":"2024-09-13T18:57:24.709839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:24.709588+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:24.709588+0000","last_clean_scrub_stamp":"2024-09-13T18:57:24.709588+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:56.958314+0000","scrub_duration":0.0036738090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"43'13","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.862606+0000","last_change":"2024-09-13T18:57:17.997698+0000","last_active":"2024-09-13T18:57:24.862606+0000","last_peered":"2024-09-13T18:57:24.862606+0000","last_clean":"2024-09-13T18:57:24.862606+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:57:24.862606+0000","last_undegraded":"2024-09-13T18:57:24.862606+0000","last_fullsized":"2024-09-13T18:57:24.862606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:17.997340+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:17.997340+0000","last_clean_scrub_stamp":"2024-09-13T18:57:17.997340+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:07.106051+0000","scrub_duration":0.0035120389999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"43'9","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.441501+0000","last_change":"2024-09-13T18:26:32.353359+0000","last_active":"2024-09-13T18:57:26.441501+0000","last_peered":"2024-09-13T18:57:26.441501+0000","last_clean":"2024-09-13T18:57:26.441501+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:57:26.441501+0000","last_undegraded":"2024-09-13T18:57:26.441501+0000","last_fullsized":"2024-09-13T18:57:26.441501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"43'14","reported_seq":60,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.910991+0000","last_change":"2024-09-13T18:26:32.349566+0000","last_active":"2024-09-13T18:57:24.910991+0000","last_peered":"2024-09-13T18:57:24.910991+0000","last_clean":"2024-09-13T18:57:24.910991+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:57:24.910991+0000","last_undegraded":"2024-09-13T18:57:24.910991+0000","last_fullsized":"2024-09-13T18:57:24.910991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"43'10","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:11.172691+0000","last_change":"2024-09-13T18:26:32.358640+0000","last_active":"2024-09-13T18:57:11.172691+0000","last_peered":"2024-09-13T18:57:11.172691+0000","last_clean":"2024-09-13T18:57:11.172691+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:57:11.172691+0000","last_undegraded":"2024-09-13T18:57:11.172691+0000","last_fullsized":"2024-09-13T18:57:11.172691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"43'17","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.022195+0000","last_change":"2024-09-13T18:26:32.353675+0000","last_active":"2024-09-13T18:57:28.022195+0000","last_peered":"2024-09-13T18:57:28.022195+0000","last_clean":"2024-09-13T18:57:28.022195+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:57:28.022195+0000","last_undegraded":"2024-09-13T18:57:28.022195+0000","last_fullsized":"2024-09-13T18:57:28.022195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"43'11","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.947071+0000","last_change":"2024-09-13T18:26:32.353223+0000","last_active":"2024-09-13T18:57:24.947071+0000","last_peered":"2024-09-13T18:57:24.947071+0000","last_clean":"2024-09-13T18:57:24.947071+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:57:24.947071+0000","last_undegraded":"2024-09-13T18:57:24.947071+0000","last_fullsized":"2024-09-13T18:57:24.947071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"43'12","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:15.857341+0000","last_change":"2024-09-13T18:57:15.857341+0000","last_active":"2024-09-13T18:57:15.857341+0000","last_peered":"2024-09-13T18:57:15.857341+0000","last_clean":"2024-09-13T18:57:15.857341+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:57:15.857341+0000","last_undegraded":"2024-09-13T18:57:15.857341+0000","last_fullsized":"2024-09-13T18:57:15.857341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:15.857084+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:15.857084+0000","last_clean_scrub_stamp":"2024-09-13T18:57:15.857084+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:39.110251+0000","scrub_duration":0.0034250190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"43'18","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.034005+0000","last_change":"2024-09-13T18:57:27.062756+0000","last_active":"2024-09-13T18:57:28.034005+0000","last_peered":"2024-09-13T18:57:28.034005+0000","last_clean":"2024-09-13T18:57:28.034005+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:57:28.034005+0000","last_undegraded":"2024-09-13T18:57:28.034005+0000","last_fullsized":"2024-09-13T18:57:28.034005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'15","last_scrub_stamp":"2024-09-13T18:57:27.062495+0000","last_deep_scrub":"43'15","last_deep_scrub_stamp":"2024-09-13T18:57:27.062495+0000","last_clean_scrub_stamp":"2024-09-13T18:57:27.062495+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:05:17.012914+0000","scrub_duration":0.0040785700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"43'12","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:22.792306+0000","last_change":"2024-09-13T18:26:32.356902+0000","last_active":"2024-09-13T18:57:22.792306+0000","last_peered":"2024-09-13T18:57:22.792306+0000","last_clean":"2024-09-13T18:57:22.792306+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:57:22.792306+0000","last_undegraded":"2024-09-13T18:57:22.792306+0000","last_fullsized":"2024-09-13T18:57:22.792306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"43'12","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:19.496003+0000","last_change":"2024-09-13T18:26:32.347047+0000","last_active":"2024-09-13T18:57:19.496003+0000","last_peered":"2024-09-13T18:57:19.496003+0000","last_clean":"2024-09-13T18:57:19.496003+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:57:19.496003+0000","last_undegraded":"2024-09-13T18:57:19.496003+0000","last_fullsized":"2024-09-13T18:57:19.496003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"43'16","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.936612+0000","last_change":"2024-09-13T18:26:32.349651+0000","last_active":"2024-09-13T18:57:24.936612+0000","last_peered":"2024-09-13T18:57:24.936612+0000","last_clean":"2024-09-13T18:57:24.936612+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:57:24.936612+0000","last_undegraded":"2024-09-13T18:57:24.936612+0000","last_fullsized":"2024-09-13T18:57:24.936612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"43'17","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.022015+0000","last_change":"2024-09-13T18:26:32.344607+0000","last_active":"2024-09-13T18:57:28.022015+0000","last_peered":"2024-09-13T18:57:28.022015+0000","last_clean":"2024-09-13T18:57:28.022015+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:57:28.022015+0000","last_undegraded":"2024-09-13T18:57:28.022015+0000","last_fullsized":"2024-09-13T18:57:28.022015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"43'12","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.080443+0000","last_change":"2024-09-13T18:26:32.347717+0000","last_active":"2024-09-13T18:57:28.080443+0000","last_peered":"2024-09-13T18:57:28.080443+0000","last_clean":"2024-09-13T18:57:28.080443+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:57:28.080443+0000","last_undegraded":"2024-09-13T18:57:28.080443+0000","last_fullsized":"2024-09-13T18:57:28.080443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"43'11","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:25.015223+0000","last_change":"2024-09-13T18:57:25.015223+0000","last_active":"2024-09-13T18:57:25.015223+0000","last_peered":"2024-09-13T18:57:25.015223+0000","last_clean":"2024-09-13T18:57:25.015223+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:57:25.015223+0000","last_undegraded":"2024-09-13T18:57:25.015223+0000","last_fullsized":"2024-09-13T18:57:25.015223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:25.015045+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:25.015045+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.015045+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:39.559328+0000","scrub_duration":0.013113774999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"43'13","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:11.675424+0000","last_change":"2024-09-13T18:26:32.357729+0000","last_active":"2024-09-13T18:57:11.675424+0000","last_peered":"2024-09-13T18:57:11.675424+0000","last_clean":"2024-09-13T18:57:11.675424+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:57:11.675424+0000","last_undegraded":"2024-09-13T18:57:11.675424+0000","last_fullsized":"2024-09-13T18:57:11.675424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"43'12","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.793255+0000","last_change":"2024-09-13T18:26:32.357980+0000","last_active":"2024-09-13T18:57:24.793255+0000","last_peered":"2024-09-13T18:57:24.793255+0000","last_clean":"2024-09-13T18:57:24.793255+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:57:24.793255+0000","last_undegraded":"2024-09-13T18:57:24.793255+0000","last_fullsized":"2024-09-13T18:57:24.793255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"43'16","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:12.422623+0000","last_change":"2024-09-13T18:26:32.365638+0000","last_active":"2024-09-13T18:57:12.422623+0000","last_peered":"2024-09-13T18:57:12.422623+0000","last_clean":"2024-09-13T18:57:12.422623+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:57:12.422623+0000","last_undegraded":"2024-09-13T18:57:12.422623+0000","last_fullsized":"2024-09-13T18:57:12.422623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"43'5","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.798756+0000","last_change":"2024-09-13T18:26:32.345496+0000","last_active":"2024-09-13T18:57:24.798756+0000","last_peered":"2024-09-13T18:57:24.798756+0000","last_clean":"2024-09-13T18:57:24.798756+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:57:24.798756+0000","last_undegraded":"2024-09-13T18:57:24.798756+0000","last_fullsized":"2024-09-13T18:57:24.798756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"43'6","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:12.422482+0000","last_change":"2024-09-13T18:26:32.356901+0000","last_active":"2024-09-13T18:57:12.422482+0000","last_peered":"2024-09-13T18:57:12.422482+0000","last_clean":"2024-09-13T18:57:12.422482+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:57:12.422482+0000","last_undegraded":"2024-09-13T18:57:12.422482+0000","last_fullsized":"2024-09-13T18:57:12.422482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"43'11","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:11.172752+0000","last_change":"2024-09-13T18:26:32.355203+0000","last_active":"2024-09-13T18:57:11.172752+0000","last_peered":"2024-09-13T18:57:11.172752+0000","last_clean":"2024-09-13T18:57:11.172752+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:57:11.172752+0000","last_undegraded":"2024-09-13T18:57:11.172752+0000","last_fullsized":"2024-09-13T18:57:11.172752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"43'12","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.964914+0000","last_change":"2024-09-13T18:26:32.358190+0000","last_active":"2024-09-13T18:57:24.964914+0000","last_peered":"2024-09-13T18:57:24.964914+0000","last_clean":"2024-09-13T18:57:24.964914+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:57:24.964914+0000","last_undegraded":"2024-09-13T18:57:24.964914+0000","last_fullsized":"2024-09-13T18:57:24.964914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"43'7","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.411788+0000","last_change":"2024-09-13T18:26:32.355121+0000","last_active":"2024-09-13T18:57:28.411788+0000","last_peered":"2024-09-13T18:57:28.411788+0000","last_clean":"2024-09-13T18:57:28.411788+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:57:28.411788+0000","last_undegraded":"2024-09-13T18:57:28.411788+0000","last_fullsized":"2024-09-13T18:57:28.411788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"43'11","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:19.341943+0000","last_change":"2024-09-13T18:57:11.323912+0000","last_active":"2024-09-13T18:57:19.341943+0000","last_peered":"2024-09-13T18:57:19.341943+0000","last_clean":"2024-09-13T18:57:19.341943+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:57:19.341943+0000","last_undegraded":"2024-09-13T18:57:19.341943+0000","last_fullsized":"2024-09-13T18:57:19.341943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:11.323767+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:11.323767+0000","last_clean_scrub_stamp":"2024-09-13T18:57:11.323767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:26.980778+0000","scrub_duration":0.0019515800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"43'4","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:22.193506+0000","last_change":"2024-09-13T18:26:32.350106+0000","last_active":"2024-09-13T18:57:22.193506+0000","last_peered":"2024-09-13T18:57:22.193506+0000","last_clean":"2024-09-13T18:57:22.193506+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:57:22.193506+0000","last_undegraded":"2024-09-13T18:57:22.193506+0000","last_fullsized":"2024-09-13T18:57:22.193506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"43'8","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.956536+0000","last_change":"2024-09-13T18:26:32.362196+0000","last_active":"2024-09-13T18:57:24.956536+0000","last_peered":"2024-09-13T18:57:24.956536+0000","last_clean":"2024-09-13T18:57:24.956536+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:57:24.956536+0000","last_undegraded":"2024-09-13T18:57:24.956536+0000","last_fullsized":"2024-09-13T18:57:24.956536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2915158540,"num_objects":706,"num_object_clones":0,"num_object_copies":2824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":706,"num_whiteouts":0,"num_read":797,"num_read_kb":0,"num_write":802894,"num_write_kb":16319459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2955124736,"data_stored":6273507328,"data_compressed":299049697,"data_compressed_allocated":425750528,"data_compressed_original":3744133120,"omap_allocated":0,"internal_metadata":0},"log_size":176608,"ondisk_log_size":176608,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":722,"ondisk_log_size":722,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":16507087,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1362,"num_read_kb":52080,"num_write":3740,"num_write_kb":17665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22716416,"data_stored":33014174,"data_compressed":2441304,"data_compressed_allocated":3506176,"data_compressed_original":13950976,"omap_allocated":0,"internal_metadata":0},"log_size":3612,"ondisk_log_size":3612,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509815,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1496748,"kb_used_data":447780,"kb_used_omap":1,"kb_used_meta":1048958,"kb_avail":92875092,"statfs":{"total":96636764160,"available":95104094208,"internally_reserved":0,"allocated":458526720,"data_stored":979907292,"data_compressed":47059645,"data_compressed_allocated":66723840,"data_compressed_original":588357632,"omap_allocated":1593,"internal_metadata":1074133447},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":1.3600000000000001,"15min":0.93999999999999995},"min":{"1min":0.247,"5min":0.223,"15min":0.223},"max":{"1min":0.84199999999999997,"5min":10.43,"15min":10.43},"last":0.247},{"interface":"front","average":{"1min":1.107,"5min":1.889,"15min":1.1599999999999999},"min":{"1min":0.27100000000000002,"5min":0.245,"15min":0.245},"max":{"1min":4.3109999999999999,"5min":16.600999999999999,"15min":16.600999999999999},"last":0.372}]},{"osd":1,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":3.5910000000000002,"15min":1.849},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":5.8780000000000001,"5min":52.290999999999997,"15min":52.290999999999997},"last":0.307},{"interface":"front","average":{"1min":1.2769999999999999,"5min":3.7690000000000001,"15min":1.9570000000000001},"min":{"1min":0.28199999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":5.4740000000000002,"5min":52.149000000000001,"15min":52.149000000000001},"last":0.28299999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":1.335,"15min":1.054},"min":{"1min":0.316,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.9399999999999999,"5min":14.933,"15min":14.933},"last":0.35699999999999998},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.264,"15min":1.052},"min":{"1min":0.35299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":7.266,"5min":9.2059999999999995,"15min":17.792000000000002},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":2.3849999999999998,"15min":1.4319999999999999},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":8.7910000000000004,"5min":15.622,"15min":15.622},"last":0.32400000000000001},{"interface":"front","average":{"1min":1.343,"5min":2.71,"15min":1.7170000000000001},"min":{"1min":0.374,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.1710000000000003,"5min":15.423,"15min":28.248000000000001},"last":0.38900000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":1.2090000000000001,"15min":1.073},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4219999999999997,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.242,"15min":1.054},"min":{"1min":0.34599999999999997,"5min":0.32900000000000001,"15min":0.23699999999999999},"max":{"1min":4.4450000000000003,"5min":7.9409999999999998,"15min":7.9409999999999998},"last":0.34599999999999997}]},{"osd":5,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.92000000000000004,"15min":0.97499999999999998},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.28000000000000003},"max":{"1min":4.3879999999999999,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.435},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.92300000000000004,"15min":0.97399999999999998},"min":{"1min":0.34599999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.143,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":0.46100000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.0640000000000001,"15min":1.054},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.8149999999999999,"5min":12.021000000000001,"15min":12.021000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.01,"15min":0.998},"min":{"1min":0.34999999999999998,"5min":0.32500000000000001,"15min":0.312},"max":{"1min":4.3600000000000003,"5min":9.0440000000000005,"15min":9.0440000000000005},"last":0.40400000000000003}]}]},{"osd":7,"up_from":15,"seq":64424509815,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1265508,"kb_used_data":375516,"kb_used_omap":1,"kb_used_meta":889982,"kb_avail":93106332,"statfs":{"total":96636764160,"available":95340883968,"internally_reserved":0,"allocated":384528384,"data_stored":818576974,"data_compressed":39090533,"data_compressed_allocated":55422976,"data_compressed_original":489734144,"omap_allocated":1589,"internal_metadata":911342027},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":2.121,"15min":1.206},"min":{"1min":0.23000000000000001,"5min":0.217,"15min":0.217},"max":{"1min":14.778,"5min":24.414000000000001,"15min":24.414000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":2.1680000000000001,"5min":2.254,"15min":1.3460000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":14.814,"5min":24.469000000000001,"15min":24.469000000000001},"last":1.0920000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":2.4889999999999999,"5min":3.355,"15min":1.617},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":6.0490000000000004,"5min":46.161999999999999,"15min":46.161999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":5.1929999999999996,"5min":4.3570000000000002,"15min":2.0129999999999999},"min":{"1min":0.29099999999999998,"5min":0.252,"15min":0.252},"max":{"1min":39.853999999999999,"5min":45.713000000000001,"15min":45.713000000000001},"last":0.95099999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.228,"15min":0.98699999999999999},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":5.3230000000000004,"5min":7.4740000000000002,"15min":7.4740000000000002},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.329,"15min":0.97899999999999998},"min":{"1min":0.315,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.8559999999999999,"5min":9.1679999999999993,"15min":9.1679999999999993},"last":1.069}]},{"osd":3,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":3.4129999999999998,"5min":3.6960000000000002,"15min":1.8029999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":17.721,"5min":35.494999999999997,"15min":35.494999999999997},"last":1.0680000000000001},{"interface":"front","average":{"1min":2.278,"5min":2.976,"15min":1.6850000000000001},"min":{"1min":0.33800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":11.353999999999999,"5min":19.946000000000002,"15min":19.946000000000002},"last":1.105}]},{"osd":4,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.92600000000000005,"15min":0.91700000000000004},"min":{"1min":0.25800000000000001,"5min":0.249,"15min":0.249},"max":{"1min":2.2749999999999999,"5min":7.2880000000000003,"15min":7.2880000000000003},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.0820000000000001,"15min":1.0589999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":7.8810000000000002,"5min":7.8810000000000002,"15min":7.8810000000000002},"last":1.161}]},{"osd":5,"last update":"Fri Sep 13 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.80700000000000005,"15min":0.877},"min":{"1min":0.29399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":3.6200000000000001,"5min":6.7309999999999999,"15min":6.7309999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90100000000000002,"15min":0.96999999999999997},"min":{"1min":0.32000000000000001,"5min":0.318,"15min":0.318},"max":{"1min":3.5819999999999999,"5min":6.6699999999999999,"15min":6.6699999999999999},"last":1.018}]},{"osd":6,"last update":"Fri Sep 13 18:56:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.2889999999999999,"15min":1.137},"min":{"1min":0.35299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.4260000000000002,"5min":32.491,"15min":32.491},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":1.137,"15min":1.091},"min":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.82899999999999996,"5min":27.291,"15min":27.291},"last":0.71499999999999997}]}]},{"osd":4,"up_from":15,"seq":64424509815,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1263276,"kb_used_data":308644,"kb_used_omap":1,"kb_used_meta":954622,"kb_avail":93108564,"statfs":{"total":96636764160,"available":95343169536,"internally_reserved":0,"allocated":316051456,"data_stored":658747488,"data_compressed":31206968,"data_compressed_allocated":44621824,"data_compressed_original":387571712,"omap_allocated":1594,"internal_metadata":977533382},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":2.0270000000000001,"15min":1.238},"min":{"1min":0.26300000000000001,"5min":0.24099999999999999,"15min":0.223},"max":{"1min":4.6299999999999999,"5min":28.593,"15min":28.593},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.7889999999999999,"5min":1.9890000000000001,"15min":1.256},"min":{"1min":0.33000000000000002,"5min":0.255,"15min":0.255},"max":{"1min":10.644,"5min":28.559000000000001,"15min":28.559000000000001},"last":0.65500000000000003}]},{"osd":1,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":3.7410000000000001,"5min":3.1200000000000001,"15min":1.6899999999999999},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":29.113,"5min":37.100999999999999,"15min":37.100999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":2.6909999999999998,"5min":3.468,"15min":1.8180000000000001},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":12.385999999999999,"5min":44.798000000000002,"15min":44.798000000000002},"last":0.84699999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.792,"15min":1.2130000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":10.506,"5min":27.634,"15min":27.634},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.7250000000000001,"15min":1.216},"min":{"1min":0.34200000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.4029999999999996,"5min":27.606000000000002,"15min":27.606000000000002},"last":0.92800000000000005}]},{"osd":3,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.641,"5min":3.1930000000000001,"15min":1.708},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":10.535,"5min":40.676000000000002,"15min":40.676000000000002},"last":0.81000000000000005},{"interface":"front","average":{"1min":2.1560000000000001,"5min":3.6699999999999999,"15min":1.97},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":10.548999999999999,"5min":40.881999999999998,"15min":40.881999999999998},"last":0.70299999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.93700000000000006,"15min":0.89200000000000002},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":4.9889999999999999,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":1.002},{"interface":"front","average":{"1min":1.0529999999999999,"5min":0.93999999999999995,"15min":0.98899999999999999},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.0170000000000003,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":1.0920000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2490000000000001,"15min":1.103},"min":{"1min":0.29799999999999999,"5min":0.251,"15min":0.251},"max":{"1min":8.9039999999999999,"5min":9.4570000000000007,"15min":9.4570000000000007},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.002,"15min":1.014},"min":{"1min":0.38600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":3.8559999999999999,"5min":5.1059999999999999,"15min":6.1319999999999997},"last":1.1579999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.464,"15min":1.1930000000000001},"min":{"1min":0.38600000000000001,"5min":0.307,"15min":0.307},"max":{"1min":6.1210000000000004,"5min":10.101000000000001,"15min":10.101000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.5269999999999999,"15min":1.286},"min":{"1min":0.41199999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.1509999999999998,"5min":10.127000000000001,"15min":10.127000000000001},"last":1.204}]}]},{"osd":3,"up_from":14,"seq":60129542519,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1750152,"kb_used_data":470400,"kb_used_omap":1,"kb_used_meta":1279742,"kb_avail":92621688,"statfs":{"total":96636764160,"available":94844608512,"internally_reserved":0,"allocated":481689600,"data_stored":1017455306,"data_compressed":48841722,"data_compressed_allocated":69697536,"data_compressed_original":605720576,"omap_allocated":1591,"internal_metadata":1310456265},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:32 2024","interfaces":[{"interface":"back","average":{"1min":3.0569999999999999,"5min":3.3959999999999999,"15min":1.5109999999999999},"min":{"1min":0.20300000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":22.670000000000002,"5min":22.670000000000002,"15min":22.670000000000002},"last":0.497},{"interface":"front","average":{"1min":3.1179999999999999,"5min":3.5009999999999999,"15min":1.677},"min":{"1min":0.246,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":22.728000000000002,"5min":22.728000000000002,"15min":22.728000000000002},"last":0.67700000000000005}]},{"osd":1,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":4.0330000000000004,"5min":5.657,"15min":2.7250000000000001},"min":{"1min":0.20499999999999999,"5min":0.191,"15min":0.191},"max":{"1min":19.170999999999999,"5min":42.046999999999997,"15min":42.046999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":3.5009999999999999,"5min":5.2409999999999997,"15min":2.7490000000000001},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":19.199999999999999,"5min":42.024999999999999,"15min":42.024999999999999},"last":0.57599999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":3.5950000000000002,"5min":3.734,"15min":2.113},"min":{"1min":0.29399999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":19.561,"5min":41.909999999999997,"15min":41.909999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.95399999999999996,"5min":2.7280000000000002,"15min":1.544},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.7570000000000001,"5min":27.25,"15min":27.25},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:56:32 2024","interfaces":[{"interface":"back","average":{"1min":4.6550000000000002,"5min":3.3100000000000001,"15min":1.7430000000000001},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":42.003,"5min":42.003,"15min":42.003},"last":0.81899999999999995},{"interface":"front","average":{"1min":5.4329999999999998,"5min":3.9399999999999999,"15min":1.956},"min":{"1min":0.39300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":41.951000000000001,"5min":41.951000000000001,"15min":41.951000000000001},"last":0.86899999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":2.3340000000000001,"15min":1.5489999999999999},"min":{"1min":0.29699999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":6.5679999999999996,"5min":22.988,"15min":22.988},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.155,"5min":2.0670000000000002,"15min":1.474},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.7330000000000001,"5min":23.030000000000001,"15min":23.030000000000001},"last":0.95699999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":2.5870000000000002,"15min":1.6859999999999999},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":3.6080000000000001,"5min":23.087,"15min":23.087},"last":1.048},{"interface":"front","average":{"1min":2.1190000000000002,"5min":3.2130000000000001,"15min":1.9159999999999999},"min":{"1min":0.39800000000000002,"5min":0.317,"15min":0.317},"max":{"1min":14.753,"5min":23.425000000000001,"15min":23.425000000000001},"last":0.83999999999999997}]},{"osd":7,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":3.4409999999999998,"5min":3.1819999999999999,"15min":1.8819999999999999},"min":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":19.588000000000001,"5min":41.975999999999999,"15min":41.975999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.956,"5min":2.7709999999999999,"15min":1.728},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34200000000000003},"max":{"1min":14.686999999999999,"5min":23.457999999999998,"15min":23.457999999999998},"last":0.88300000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542523,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":981492,"kb_used_data":240044,"kb_used_omap":1,"kb_used_meta":741438,"kb_avail":93390348,"statfs":{"total":96636764160,"available":95631716352,"internally_reserved":0,"allocated":245805056,"data_stored":518331217,"data_compressed":24695480,"data_compressed_allocated":35442688,"data_compressed_original":308228096,"omap_allocated":1593,"internal_metadata":759232967},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":2.5110000000000001,"15min":1.278},"min":{"1min":0.19600000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":4.7599999999999998,"5min":37.600000000000001,"15min":37.600000000000001},"last":0.216},{"interface":"front","average":{"1min":1.3540000000000001,"5min":2.5030000000000001,"15min":1.3149999999999999},"min":{"1min":0.20499999999999999,"5min":0.19,"15min":0.19},"max":{"1min":7.3949999999999996,"5min":37.651000000000003,"15min":37.651000000000003},"last":0.254}]},{"osd":1,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":3.4319999999999999,"5min":2.8719999999999999,"15min":1.7030000000000001},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":17.443999999999999,"5min":17.443999999999999,"15min":18.055},"last":0.372},{"interface":"front","average":{"1min":2.8220000000000001,"5min":2.6099999999999999,"15min":1.5800000000000001},"min":{"1min":0.28399999999999997,"5min":0.20000000000000001,"15min":0.188},"max":{"1min":15.132,"5min":15.94,"15min":18.175999999999998},"last":0.35299999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":3.3039999999999998,"15min":1.806},"min":{"1min":0.29099999999999998,"5min":0.249,"15min":0.249},"max":{"1min":17.613,"5min":25.832999999999998,"15min":25.832999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":1.627,"5min":3.2429999999999999,"15min":1.8220000000000001},"min":{"1min":0.35599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":17.59,"5min":17.59,"15min":17.59},"last":0.39500000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.1910000000000001,"15min":1.0309999999999999},"min":{"1min":0.32600000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.3559999999999999,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.435},{"interface":"front","average":{"1min":1.252,"5min":1.149,"15min":1.0940000000000001},"min":{"1min":0.35999999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.6980000000000004,"5min":8.0999999999999996,"15min":8.0999999999999996},"last":0.55000000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2669999999999999,"15min":1.0580000000000001},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29499999999999998},"max":{"1min":5.8600000000000003,"5min":8.4800000000000004,"15min":8.4800000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4470000000000001,"15min":1.1539999999999999},"min":{"1min":0.311,"5min":0.309,"15min":0.309},"max":{"1min":8.9949999999999992,"5min":8.9949999999999992,"15min":8.9949999999999992},"last":0.53300000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.2789999999999999,"15min":1.167},"min":{"1min":0.436,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.7370000000000001,"5min":9.6720000000000006,"15min":9.6720000000000006},"last":0.46300000000000002},{"interface":"front","average":{"1min":1.127,"5min":1.1599999999999999,"15min":1.0800000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":4.5129999999999999,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4039999999999999,"15min":1.169},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.3179999999999996,"5min":8.9250000000000007,"15min":8.9250000000000007},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.5029999999999999,"15min":1.2589999999999999},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":7.7569999999999997,"5min":9.2330000000000005,"15min":9.2330000000000005},"last":0.60199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575224,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":967932,"kb_used_data":260916,"kb_used_omap":1,"kb_used_meta":707006,"kb_avail":93403908,"statfs":{"total":96636764160,"available":95645601792,"internally_reserved":0,"allocated":267177984,"data_stored":562278128,"data_compressed":26771264,"data_compressed_allocated":38043648,"data_compressed_original":333402112,"omap_allocated":1593,"internal_metadata":723974599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":1.7170000000000001,"5min":1.8320000000000001,"15min":1.175},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":8.8219999999999992,"5min":9.952,"15min":9.952},"last":1.03},{"interface":"front","average":{"1min":1.796,"5min":1.948,"15min":1.3},"min":{"1min":0.29599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.8740000000000006,"5min":14.317,"15min":14.317},"last":0.872}]},{"osd":1,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.931,"15min":1.3240000000000001},"min":{"1min":0.32500000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":6.4930000000000003,"5min":10.85,"15min":10.85},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.917,"15min":1.3240000000000001},"min":{"1min":0.26000000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":3.113,"5min":10.868,"15min":10.868},"last":0.76200000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":2.641,"5min":2.222,"15min":1.3},"min":{"1min":0.30299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":9.9499999999999993,"5min":38.484000000000002,"15min":38.484000000000002},"last":0.80100000000000005},{"interface":"front","average":{"1min":2.0259999999999998,"5min":2.3730000000000002,"15min":1.365},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":8.0800000000000001,"5min":40.686,"15min":40.686},"last":0.621}]},{"osd":3,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":2.5819999999999999,"5min":4.0430000000000001,"15min":2.0880000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":14.885999999999999,"5min":72.534999999999997,"15min":72.534999999999997},"last":1.0720000000000001},{"interface":"front","average":{"1min":2.8029999999999999,"5min":3.9220000000000002,"15min":2.101},"min":{"1min":0.30499999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":14.91,"5min":72.486999999999995,"15min":72.486999999999995},"last":1.1020000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.93100000000000005,"15min":0.90800000000000003},"min":{"1min":0.28799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1970000000000001,"5min":7.8710000000000004,"15min":7.8710000000000004},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.1240000000000001,"15min":1.071},"min":{"1min":0.36099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.8399999999999999,"5min":8.7829999999999995,"15min":8.7829999999999995},"last":0.91200000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.97499999999999998,"15min":0.999},"min":{"1min":0.32400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.5179999999999998,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.0569999999999999,"15min":1.0509999999999999},"min":{"1min":0.38500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.54,"5min":4.516,"15min":4.516},"last":1.167}]},{"osd":7,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.02,"15min":1.0209999999999999},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":4.4249999999999998,"5min":7.6020000000000003,"15min":7.6020000000000003},"last":1.2470000000000001},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.0169999999999999,"15min":1.101},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.1900000000000004,"5min":7.9109999999999996,"15min":7.9109999999999996},"last":1.2869999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575221,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1793092,"kb_used_data":598780,"kb_used_omap":1,"kb_used_meta":1194302,"kb_avail":92578748,"statfs":{"total":96636764160,"available":94800637952,"internally_reserved":0,"allocated":613150720,"data_stored":1300917566,"data_compressed":62041686,"data_compressed_allocated":88047616,"data_compressed_original":776077312,"omap_allocated":1594,"internal_metadata":1222965702},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":3.2570000000000001,"15min":1.4910000000000001},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":7.9580000000000002,"5min":28.297000000000001,"15min":28.297000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":3.1040000000000001,"5min":2.7749999999999999,"15min":1.341},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":19.785,"5min":19.785,"15min":19.785},"last":0.92300000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":2.8340000000000001,"15min":1.385},"min":{"1min":0.252,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":8.0630000000000006,"5min":25.609999999999999,"15min":25.609999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":4.0890000000000004,"5min":3.097,"15min":1.536},"min":{"1min":0.38500000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":20.173999999999999,"5min":30.469999999999999,"15min":30.469999999999999},"last":1.0820000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":5.524,"5min":5.7249999999999996,"15min":2.383},"min":{"1min":0.58199999999999996,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":20.001000000000001,"5min":36.643999999999998,"15min":36.643999999999998},"last":0.76900000000000002},{"interface":"front","average":{"1min":5.5519999999999996,"5min":5.9119999999999999,"15min":2.5110000000000001},"min":{"1min":0.59899999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":20.030999999999999,"5min":36.670000000000002,"15min":36.670000000000002},"last":1.1279999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":1.7549999999999999,"15min":1.1759999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.2249999999999996,"5min":7.6799999999999997,"15min":7.6799999999999997},"last":1.044},{"interface":"front","average":{"1min":1.26,"5min":2.1299999999999999,"15min":1.331},"min":{"1min":0.32800000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":6.5540000000000003,"5min":14.906000000000001,"15min":14.906000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":3.1880000000000002,"5min":1.9350000000000001,"15min":1.3149999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":9.0519999999999996,"5min":9.0519999999999996,"15min":9.0519999999999996},"last":1.218},{"interface":"front","average":{"1min":2.7149999999999999,"5min":1.9410000000000001,"15min":1.3300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.1050000000000004,"5min":12.27,"15min":12.27},"last":1.1679999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":2.577,"5min":2.4300000000000002,"15min":1.468},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":9.077,"5min":28.981999999999999,"15min":28.981999999999999},"last":1.319},{"interface":"front","average":{"1min":2.1819999999999999,"5min":2.1379999999999999,"15min":1.3799999999999999},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":15.727,"5min":15.727,"15min":15.727},"last":1.117}]},{"osd":7,"last update":"Fri Sep 13 18:56:29 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":2.379,"15min":1.4359999999999999},"min":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":7.9930000000000003,"5min":28.100000000000001,"15min":28.100000000000001},"last":1.234},{"interface":"front","average":{"1min":1.728,"5min":2.4860000000000002,"15min":1.518},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.0220000000000002,"5min":28.896999999999998,"15min":28.896999999999998},"last":1.292}]}]},{"osd":0,"up_from":13,"seq":55834575223,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1169408,"kb_used_data":351480,"kb_used_omap":1,"kb_used_meta":817918,"kb_avail":93202432,"statfs":{"total":96636764160,"available":95439290368,"internally_reserved":0,"allocated":359915520,"data_stored":753337979,"data_compressed":36111404,"data_compressed_allocated":51458048,"data_compressed_original":445140992,"omap_allocated":1594,"internal_metadata":837548486},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.8009999999999999,"5min":3.573,"15min":1.8169999999999999},"min":{"1min":0.24399999999999999,"5min":0.183,"15min":0.183},"max":{"1min":10.925000000000001,"5min":29.712,"15min":29.712},"last":10.504},{"interface":"front","average":{"1min":1.877,"5min":2.6520000000000001,"15min":1.5640000000000001},"min":{"1min":0.29299999999999998,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":10.975,"5min":29.748000000000001,"15min":29.748000000000001},"last":10.539}]},{"osd":2,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.6519999999999999,"15min":1.0509999999999999},"min":{"1min":0.218,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":8.5660000000000007,"5min":17.611000000000001,"15min":17.611000000000001},"last":2.0880000000000001},{"interface":"front","average":{"1min":1.113,"5min":1.867,"15min":1.149},"min":{"1min":0.22700000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":9.2560000000000002,"5min":17.501999999999999,"15min":17.501999999999999},"last":2.0760000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":3.2879999999999998,"15min":1.6659999999999999},"min":{"1min":0.30099999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":7.9340000000000002,"5min":17.260000000000002,"15min":17.260000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.149,"5min":2.6269999999999998,"15min":1.536},"min":{"1min":0.27400000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":5.8170000000000002,"5min":17.088000000000001,"15min":17.088000000000001},"last":0.29799999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":2.286,"15min":1.4219999999999999},"min":{"1min":0.376,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":8.0380000000000003,"5min":40.101999999999997,"15min":40.101999999999997},"last":1.163},{"interface":"front","average":{"1min":1.0229999999999999,"5min":2.0600000000000001,"15min":1.3069999999999999},"min":{"1min":0.40100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":5.9009999999999998,"5min":40.981000000000002,"15min":40.981000000000002},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.8120000000000001,"15min":1.246},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.8900000000000001,"5min":40.701999999999998,"15min":40.701999999999998},"last":0.504},{"interface":"front","average":{"1min":1.0800000000000001,"5min":2.0499999999999998,"15min":1.3520000000000001},"min":{"1min":0.434,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.8559999999999999,"5min":40.130000000000003,"15min":40.130000000000003},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":2.1930000000000001,"15min":1.4159999999999999},"min":{"1min":0.40200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":6.1440000000000001,"5min":51.719000000000001,"15min":51.719000000000001},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":2.2559999999999998,"15min":1.4710000000000001},"min":{"1min":0.45500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":7.9660000000000002,"5min":51.75,"15min":51.75},"last":1.7729999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:56:33 2024","interfaces":[{"interface":"back","average":{"1min":5.5179999999999998,"5min":2.3180000000000001,"15min":1.4770000000000001},"min":{"1min":0.36899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":58.810000000000002,"5min":58.810000000000002,"15min":58.810000000000002},"last":4.5620000000000003},{"interface":"front","average":{"1min":4.609,"5min":2.125,"15min":1.4370000000000001},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":39.622,"5min":39.622,"15min":39.622},"last":5.508}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2072576,"data_stored":3824333,"data_compressed":409575,"data_compressed_allocated":573440,"data_compressed_original":2342912,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":94659,"data_compressed":1568,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5124096,"data_stored":8485180,"data_compressed":790472,"data_compressed_allocated":1138688,"data_compressed_original":4517888,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4136960,"data_stored":4196050,"data_compressed":20605,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1777664,"data_stored":4196686,"data_compressed":519875,"data_compressed_allocated":692224,"data_compressed_original":3125248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9449472,"data_stored":12214944,"data_compressed":699209,"data_compressed_allocated":1056768,"data_compressed_original":3842048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":331370496,"data_stored":689963008,"data_compressed":32874617,"data_compressed_allocated":46899200,"data_compressed_original":405491712,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":579702784,"data_stored":1231441920,"data_compressed":58750918,"data_compressed_allocated":83431424,"data_compressed_original":735170560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":245215232,"data_stored":517996544,"data_compressed":24693912,"data_compressed_allocated":35438592,"data_compressed_original":308219904,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":415129600,"data_stored":880803840,"data_compressed":41974213,"data_compressed_allocated":60002304,"data_compressed_original":525676544,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":311435264,"data_stored":654311424,"data_compressed":31186363,"data_compressed_allocated":44580864,"data_compressed_original":387457024,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":241475584,"data_stored":517996544,"data_compressed":24671456,"data_compressed_allocated":35045376,"data_compressed_original":311566336,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":456269824,"data_stored":975470592,"data_compressed":46539770,"data_compressed_allocated":66031616,"data_compressed_original":585232384,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":374525952,"data_stored":805523456,"data_compressed":38358448,"data_compressed_allocated":54321152,"data_compressed_original":485318656,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:57:33.247 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.247 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.247 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.247 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.247 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.247 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.247 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.248 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.248 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.248 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.248 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.248 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.248 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.248 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.248 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.248 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.249 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.249 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.249 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.249 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.249 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.249 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.249 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.250 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.250 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.250 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.250 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.250 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.250 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.250 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.251 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.251 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.251 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.251 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.251 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.251 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.251 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.252 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.252 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.252 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.252 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.252 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.252 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.252 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.252 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.253 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.253 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.253 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.253 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.253 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-13T18:26:30.927447+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=30, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.253 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.253 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.254 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.254 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.254 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.254 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.254 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.254 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.254 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.254 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.255 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.255 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.255 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.255 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.255 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.255 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.255 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.255 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.256 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.256 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.256 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.256 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.256 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.256 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.256 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.256 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.257 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.257 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.257 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.257 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.257 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.257 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.257 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.257 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.257 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.258 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:33.260 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T18:57:53.261 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:57:53.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e29f2700 1 Processor -- start 2024-09-13T18:57:53.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e29f2700 1 -- start start 2024-09-13T18:57:53.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e29f2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 0x7fd9dc0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:53.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e29f2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc15da20 con 0x7fd9dc071390 2024-09-13T18:57:53.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e29f2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc0ca750 con 0x7fd9dc0cbd50 2024-09-13T18:57:53.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e29f2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0caa30 con 0x7fd9dc0ca210 2024-09-13T18:57:53.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e21f1700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd9dc071390 0x7fd9dc163e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37042/0 (socket says 172.21.15.146:37042) 2024-09-13T18:57:53.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e21f1700 1 -- 172.21.15.146:0/3609902003 learned_addr learned my addr 172.21.15.146:0/3609902003 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:57:53.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e11ef700 1 --2- 172.21.15.146:0/3609902003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 0x7fd9dc0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:53.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537346221 0 0) 0x7fd9dc15da20 con 0x7fd9dc071390 2024-09-13T18:57:53.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9c80035d0 con 0x7fd9dc071390 2024-09-13T18:57:53.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1999139038 0 0) 0x7fd9c80035d0 con 0x7fd9dc071390 2024-09-13T18:57:53.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 msgr2=0x7fd9dc0b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:53.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 --2- 172.21.15.146:0/3609902003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 0x7fd9dc0b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:53.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 >> v1:172.21.15.154:6789/0 conn(0x7fd9dc0cbd50 legacy=0x7fd9dc0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:53.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc15cc60 con 0x7fd9dc071390 2024-09-13T18:57:53.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7fd9d40019f0 con 0x7fd9dc071390 2024-09-13T18:57:53.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 >> v1:172.21.15.146:6789/0 conn(0x7fd9dc071390 legacy=0x7fd9dc163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:53.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 --2- 172.21.15.146:0/3609902003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 0x7fd9c8003ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:53.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 --2- 172.21.15.146:0/3609902003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9c8004040 0x7fd9c8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:53.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 --2- 172.21.15.146:0/3609902003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9c8004db0 0x7fd9c8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:53.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0caa30 con 0x7fd9c8004db0 2024-09-13T18:57:53.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c800bb90 con 0x7fd9c8004040 2024-09-13T18:57:53.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c800bd50 con 0x7fd9dc0ca210 2024-09-13T18:57:53.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e19f0700 1 --2- 172.21.15.146:0/3609902003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 0x7fd9c8003ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:53.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e21f1700 1 --2- 172.21.15.146:0/3609902003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9c8004db0 0x7fd9c8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:53.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e11ef700 1 --2- 172.21.15.146:0/3609902003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9c8004040 0x7fd9c8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:53.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e19f0700 1 -- 172.21.15.146:0/3609902003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9c8004040 msgr2=0x7fd9c8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:53.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e19f0700 1 --2- 172.21.15.146:0/3609902003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9c8004040 0x7fd9c8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:53.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e19f0700 1 -- 172.21.15.146:0/3609902003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9c8004db0 msgr2=0x7fd9c8005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:57:53.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e19f0700 1 --2- 172.21.15.146:0/3609902003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9c8004db0 0x7fd9c8005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:53.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e19f0700 1 -- 172.21.15.146:0/3609902003 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c80035d0 con 0x7fd9dc0ca210 2024-09-13T18:57:53.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.772+0000 7fd9e21f1700 1 --2- 172.21.15.146:0/3609902003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9c8004db0 0x7fd9c8005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:57:53.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.776+0000 7fd9e19f0700 1 --2- 172.21.15.146:0/3609902003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 0x7fd9c8003ac0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80017e0 tx=0x7fd9c80058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1920d62d11174b6f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:53.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.776+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80052d0 con 0x7fd9dc0ca210 2024-09-13T18:57:53.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.776+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d8005430 con 0x7fd9dc0ca210 2024-09-13T18:57:53.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.776+0000 7fd9e09ee700 1 -- 172.21.15.146:0/3609902003 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80055a0 con 0x7fd9dc0ca210 2024-09-13T18:57:53.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.776+0000 7fd9e29f2700 1 -- 172.21.15.146:0/3609902003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 msgr2=0x7fd9c8003ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:53.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.776+0000 7fd9e29f2700 1 --2- 172.21.15.146:0/3609902003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 0x7fd9c8003ac0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80017e0 tx=0x7fd9c80058a0 comp rx=0 tx=0).stop 2024-09-13T18:57:53.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.776+0000 7fd9e21f1700 1 -- 172.21.15.146:0/3609902003 reap_dead start 2024-09-13T18:57:53.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.776+0000 7fd9e29f2700 1 -- 172.21.15.146:0/3609902003 shutdown_connections 2024-09-13T18:57:53.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.776+0000 7fd9e29f2700 1 -- 172.21.15.146:0/3609902003 >> 172.21.15.146:0/3609902003 conn(0x7fd9dc066270 msgr2=0x7fd9dc066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:53.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.776+0000 7fd9e29f2700 1 -- 172.21.15.146:0/3609902003 shutdown_connections 2024-09-13T18:57:53.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.776+0000 7fd9e29f2700 1 -- 172.21.15.146:0/3609902003 wait complete. 2024-09-13T18:57:53.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.776+0000 7fd9e29f2700 1 Processor -- start 2024-09-13T18:57:53.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.776+0000 7fd9e29f2700 1 -- start start 2024-09-13T18:57:53.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e29f2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 0x7fd9dc0c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:53.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e29f2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9dc1eff90 0x7fd9dc1f0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:53.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e29f2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9dc156ba0 0x7fd9dc0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:53.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e29f2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1ef2a0 con 0x7fd9dc156ba0 2024-09-13T18:57:53.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e29f2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0c8d00 con 0x7fd9dc1eff90 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e29f2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0c8f70 con 0x7fd9dc0ca210 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e21f1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9dc156ba0 0x7fd9dc0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e21f1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9dc156ba0 0x7fd9dc0cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57298/0 (socket says 172.21.15.146:57298) 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e21f1700 1 -- 172.21.15.146:0/2248178123 learned_addr learned my addr 172.21.15.146:0/2248178123 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e19f0700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 0x7fd9dc0c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e11ef700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9dc1eff90 0x7fd9dc1f0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e21f1700 1 -- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 msgr2=0x7fd9dc0c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e21f1700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 0x7fd9dc0c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e21f1700 1 -- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9dc1eff90 msgr2=0x7fd9dc1f0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e21f1700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9dc1eff90 0x7fd9dc1f0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e21f1700 1 -- 172.21.15.146:0/2248178123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc1ef4b0 con 0x7fd9dc156ba0 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e11ef700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9dc1eff90 0x7fd9dc1f0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e21f1700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9dc156ba0 0x7fd9dc0cf830 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc071870 tx=0x7fd9d4007290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:53.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9ce7fc700 1 -- 172.21.15.146:0/2248178123 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4001ea0 con 0x7fd9dc156ba0 2024-09-13T18:57:53.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9ce7fc700 1 -- 172.21.15.146:0/2248178123 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d4007a00 con 0x7fd9dc156ba0 2024-09-13T18:57:53.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e29f2700 1 -- 172.21.15.146:0/2248178123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc1ef790 con 0x7fd9dc156ba0 2024-09-13T18:57:53.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9ce7fc700 1 -- 172.21.15.146:0/2248178123 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4013630 con 0x7fd9dc156ba0 2024-09-13T18:57:53.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e29f2700 1 -- 172.21.15.146:0/2248178123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc1f0c50 con 0x7fd9dc156ba0 2024-09-13T18:57:53.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.780+0000 7fd9e29f2700 1 -- 172.21.15.146:0/2248178123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9dc1efcb0 con 0x7fd9dc156ba0 2024-09-13T18:57:53.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.784+0000 7fd9ce7fc700 1 -- 172.21.15.146:0/2248178123 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd9d4013790 con 0x7fd9dc156ba0 2024-09-13T18:57:53.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.784+0000 7fd9ce7fc700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd9c0066730 0x7fd9c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:57:53.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.784+0000 7fd9ce7fc700 1 -- 172.21.15.146:0/2248178123 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fd9d4085af0 con 0x7fd9dc156ba0 2024-09-13T18:57:53.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.784+0000 7fd9e19f0700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd9c0066730 0x7fd9c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:57:53.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.784+0000 7fd9e19f0700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd9c0066730 0x7fd9c0068bf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8015040 tx=0x7fd9d8009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:57:53.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.788+0000 7fd9ce7fc700 1 -- 172.21.15.146:0/2248178123 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd9d4055f30 con 0x7fd9dc156ba0 2024-09-13T18:57:53.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.960+0000 7fd9e29f2700 1 -- 172.21.15.146:0/2248178123 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9dc15be80 con 0x7fd9c0066730 2024-09-13T18:57:53.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.980+0000 7fd9ce7fc700 1 -- 172.21.15.146:0/2248178123 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370097 (secure 0 0 0) 0x7fd9dc15be80 con 0x7fd9c0066730 2024-09-13T18:57:53.997 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:57:54.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.988+0000 7fd9e29f2700 1 -- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd9c0066730 msgr2=0x7fd9c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:54.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.988+0000 7fd9e29f2700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd9c0066730 0x7fd9c0068bf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8015040 tx=0x7fd9d8009030 comp rx=0 tx=0).stop 2024-09-13T18:57:54.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.988+0000 7fd9e29f2700 1 -- 172.21.15.146:0/2248178123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9dc156ba0 msgr2=0x7fd9dc0cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:57:54.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.988+0000 7fd9e29f2700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9dc156ba0 0x7fd9dc0cf830 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc071870 tx=0x7fd9d4007290 comp rx=0 tx=0).stop 2024-09-13T18:57:54.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.988+0000 7fd9e29f2700 1 -- 172.21.15.146:0/2248178123 shutdown_connections 2024-09-13T18:57:54.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.988+0000 7fd9e29f2700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fd9c0066730 0x7fd9c0068bf0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:54.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.988+0000 7fd9e29f2700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9dc156ba0 0x7fd9dc0cf830 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:54.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.988+0000 7fd9e29f2700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9dc1eff90 0x7fd9dc1f0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:54.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.988+0000 7fd9e29f2700 1 --2- 172.21.15.146:0/2248178123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9dc0ca210 0x7fd9dc0c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:57:54.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.988+0000 7fd9e29f2700 1 -- 172.21.15.146:0/2248178123 >> 172.21.15.146:0/2248178123 conn(0x7fd9dc066270 msgr2=0x7fd9dc14e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:57:54.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.988+0000 7fd9e29f2700 1 -- 172.21.15.146:0/2248178123 shutdown_connections 2024-09-13T18:57:54.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:57:53.988+0000 7fd9e29f2700 1 -- 172.21.15.146:0/2248178123 wait complete. 2024-09-13T18:57:54.002 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:57:54.014 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":980,"stamp":"2024-09-13T18:57:53.615437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17097455,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":2275,"num_read_kb":52180,"num_write":807501,"num_write_kb":16339542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":182570,"ondisk_log_size":182570,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7792008,"kb_used_data":26056,"kb_used_omap":12,"kb_used_meta":7765875,"kb_avail":747182712,"statfs":{"total":773094113280,"available":765115097088,"internally_reserved":0,"allocated":26681344,"data_stored":36115022,"data_compressed":2507056,"data_compressed_allocated":3596288,"data_compressed_original":15097856,"omap_allocated":12741,"internal_metadata":7952256571},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-184549376,"num_objects":-44,"num_object_clones":0,"num_object_copies":-176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005640"},"pg_stats":[{"pgid":"2.1f","version":"43'471","reported_seq":562,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.638819+0000","last_change":"2024-09-13T18:57:12.363177+0000","last_active":"2024-09-13T18:57:27.638819+0000","last_peered":"2024-09-13T18:57:27.638819+0000","last_clean":"2024-09-13T18:57:27.638819+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:57:27.638819+0000","last_undegraded":"2024-09-13T18:57:27.638819+0000","last_fullsized":"2024-09-13T18:57:27.638819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'469","last_scrub_stamp":"2024-09-13T18:57:12.363068+0000","last_deep_scrub":"43'469","last_deep_scrub_stamp":"2024-09-13T18:57:12.363068+0000","last_clean_scrub_stamp":"2024-09-13T18:57:12.363068+0000","objects_scrubbed":5,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:41:46.348669+0000","scrub_duration":0.044894295000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3824255,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":428,"num_write":456,"num_write_kb":3991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"43'5","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:16.089759+0000","last_change":"2024-09-13T18:57:16.090020+0000","last_active":"2024-09-13T18:57:16.089759+0000","last_peered":"2024-09-13T18:57:16.089759+0000","last_clean":"2024-09-13T18:57:16.089759+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:57:16.089759+0000","last_undegraded":"2024-09-13T18:57:16.089759+0000","last_fullsized":"2024-09-13T18:57:16.089759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:16.089709+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:16.089709+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.089709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.965527+0000","scrub_duration":0.0037314340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"43'5858","reported_seq":5937,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.632785+0000","last_change":"2024-09-13T18:26:36.407579+0000","last_active":"2024-09-13T18:57:38.632785+0000","last_peered":"2024-09-13T18:57:38.632785+0000","last_clean":"2024-09-13T18:57:38.632785+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:57:38.632785+0000","last_undegraded":"2024-09-13T18:57:38.632785+0000","last_fullsized":"2024-09-13T18:57:38.632785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5858,"log_dups_size":0,"ondisk_log_size":5858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":26767,"num_write_kb":537654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"43'16","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.390499+0000","last_change":"2024-09-13T18:57:25.919095+0000","last_active":"2024-09-13T18:57:27.390499+0000","last_peered":"2024-09-13T18:57:27.390499+0000","last_clean":"2024-09-13T18:57:27.390499+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:57:27.390499+0000","last_undegraded":"2024-09-13T18:57:27.390499+0000","last_fullsized":"2024-09-13T18:57:27.390499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:25.918713+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:25.918713+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.918713+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:35.551553+0000","scrub_duration":0.0032041489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"43'14","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:33.385777+0000","last_change":"2024-09-13T18:57:33.385777+0000","last_active":"2024-09-13T18:57:33.385777+0000","last_peered":"2024-09-13T18:57:33.385777+0000","last_clean":"2024-09-13T18:57:33.385777+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:57:33.385777+0000","last_undegraded":"2024-09-13T18:57:33.385777+0000","last_fullsized":"2024-09-13T18:57:33.385777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:33.385394+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:33.385394+0000","last_clean_scrub_stamp":"2024-09-13T18:57:33.385394+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:52.913543+0000","scrub_duration":0.0034717820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"43'5909","reported_seq":5992,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:40.030582+0000","last_change":"2024-09-13T18:57:40.030905+0000","last_active":"2024-09-13T18:57:40.030582+0000","last_peered":"2024-09-13T18:57:40.030582+0000","last_clean":"2024-09-13T18:57:40.030582+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:57:40.030582+0000","last_undegraded":"2024-09-13T18:57:40.030582+0000","last_fullsized":"2024-09-13T18:57:40.030582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5909","last_scrub_stamp":"2024-09-13T18:57:40.030533+0000","last_deep_scrub":"43'5909","last_deep_scrub_stamp":"2024-09-13T18:57:40.030533+0000","last_clean_scrub_stamp":"2024-09-13T18:57:40.030533+0000","objects_scrubbed":0,"log_size":5909,"log_dups_size":0,"ondisk_log_size":5909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:18:12.112501+0000","scrub_duration":0.046110303999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":26783,"num_write_kb":545346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"43'29","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:37.893910+0000","last_change":"2024-09-13T18:57:37.894025+0000","last_active":"2024-09-13T18:57:37.893910+0000","last_peered":"2024-09-13T18:57:37.893910+0000","last_clean":"2024-09-13T18:57:37.893910+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:57:37.893910+0000","last_undegraded":"2024-09-13T18:57:37.893910+0000","last_fullsized":"2024-09-13T18:57:37.893910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'29","last_scrub_stamp":"2024-09-13T18:57:37.893891+0000","last_deep_scrub":"43'29","last_deep_scrub_stamp":"2024-09-13T18:57:37.893891+0000","last_clean_scrub_stamp":"2024-09-13T18:57:37.893891+0000","objects_scrubbed":2,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:54:57.603232+0000","scrub_duration":0.0099177959999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":65,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"43'9","reported_seq":59,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:44.043128+0000","last_change":"2024-09-13T18:57:44.043334+0000","last_active":"2024-09-13T18:57:44.043128+0000","last_peered":"2024-09-13T18:57:44.043128+0000","last_clean":"2024-09-13T18:57:44.043128+0000","last_became_active":"2024-09-13T18:27:22.609625+0000","last_became_peered":"2024-09-13T18:27:22.609625+0000","last_unstale":"2024-09-13T18:57:44.043128+0000","last_undegraded":"2024-09-13T18:57:44.043128+0000","last_fullsized":"2024-09-13T18:57:44.043128+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:57:44.043080+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:57:44.043080+0000","last_clean_scrub_stamp":"2024-09-13T18:57:44.043080+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:40:41.348901+0000","scrub_duration":0.0036812300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"43'4988","reported_seq":5062,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:50.959877+0000","last_change":"2024-09-13T18:57:50.960011+0000","last_active":"2024-09-13T18:57:50.959877+0000","last_peered":"2024-09-13T18:57:50.959877+0000","last_clean":"2024-09-13T18:57:50.959877+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:57:50.959877+0000","last_undegraded":"2024-09-13T18:57:50.959877+0000","last_fullsized":"2024-09-13T18:57:50.959877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4988","last_scrub_stamp":"2024-09-13T18:57:50.959856+0000","last_deep_scrub":"43'4988","last_deep_scrub_stamp":"2024-09-13T18:57:50.959856+0000","last_clean_scrub_stamp":"2024-09-13T18:57:50.959856+0000","objects_scrubbed":0,"log_size":4988,"log_dups_size":0,"ondisk_log_size":4988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:31.885945+0000","scrub_duration":0.040254119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21624,"num_write_kb":467229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"43'9","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:31.883126+0000","last_change":"2024-09-13T18:57:31.883350+0000","last_active":"2024-09-13T18:57:31.883126+0000","last_peered":"2024-09-13T18:57:31.883126+0000","last_clean":"2024-09-13T18:57:31.883126+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:57:31.883126+0000","last_undegraded":"2024-09-13T18:57:31.883126+0000","last_fullsized":"2024-09-13T18:57:31.883126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:57:31.883070+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:57:31.883070+0000","last_clean_scrub_stamp":"2024-09-13T18:57:31.883070+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:54.521068+0000","scrub_duration":0.0057825330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":12,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"43'16","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.854618+0000","last_change":"2024-09-13T18:26:32.351494+0000","last_active":"2024-09-13T18:57:24.854618+0000","last_peered":"2024-09-13T18:57:24.854618+0000","last_clean":"2024-09-13T18:57:24.854618+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:57:24.854618+0000","last_undegraded":"2024-09-13T18:57:24.854618+0000","last_fullsized":"2024-09-13T18:57:24.854618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"43'4887","reported_seq":4961,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:46.899456+0000","last_change":"2024-09-13T18:57:46.899691+0000","last_active":"2024-09-13T18:57:46.899456+0000","last_peered":"2024-09-13T18:57:46.899456+0000","last_clean":"2024-09-13T18:57:46.899456+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:57:46.899456+0000","last_undegraded":"2024-09-13T18:57:46.899456+0000","last_fullsized":"2024-09-13T18:57:46.899456+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4887","last_scrub_stamp":"2024-09-13T18:57:46.899410+0000","last_deep_scrub":"43'4887","last_deep_scrub_stamp":"2024-09-13T18:57:46.899410+0000","last_clean_scrub_stamp":"2024-09-13T18:57:46.899410+0000","objects_scrubbed":0,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:15.511519+0000","scrub_duration":0.04143703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21624,"num_write_kb":434281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"43'9","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:29.780911+0000","last_change":"2024-09-13T18:57:29.781039+0000","last_active":"2024-09-13T18:57:29.780911+0000","last_peered":"2024-09-13T18:57:29.780911+0000","last_clean":"2024-09-13T18:57:29.780911+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:57:29.780911+0000","last_undegraded":"2024-09-13T18:57:29.780911+0000","last_fullsized":"2024-09-13T18:57:29.780911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:57:29.780879+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:57:29.780879+0000","last_clean_scrub_stamp":"2024-09-13T18:57:29.780879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:44.093932+0000","scrub_duration":0.0034454730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"43'1223","reported_seq":1307,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.921461+0000","last_change":"2024-09-13T18:57:28.921602+0000","last_active":"2024-09-13T18:57:28.921461+0000","last_peered":"2024-09-13T18:57:28.921461+0000","last_clean":"2024-09-13T18:57:28.921461+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:57:28.921461+0000","last_undegraded":"2024-09-13T18:57:28.921461+0000","last_fullsized":"2024-09-13T18:57:28.921461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'1223","last_scrub_stamp":"2024-09-13T18:57:28.921440+0000","last_deep_scrub":"43'1223","last_deep_scrub_stamp":"2024-09-13T18:57:28.921440+0000","last_clean_scrub_stamp":"2024-09-13T18:57:28.921440+0000","objects_scrubbed":3,"log_size":1223,"log_dups_size":0,"ondisk_log_size":1223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:20.603136+0000","scrub_duration":0.029685530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":10235,"num_write":1211,"num_write_kb":4423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"43'5452","reported_seq":5530,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:47.934792+0000","last_change":"2024-09-13T18:57:47.934939+0000","last_active":"2024-09-13T18:57:47.934792+0000","last_peered":"2024-09-13T18:57:47.934792+0000","last_clean":"2024-09-13T18:57:47.934792+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:57:47.934792+0000","last_undegraded":"2024-09-13T18:57:47.934792+0000","last_fullsized":"2024-09-13T18:57:47.934792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5452","last_scrub_stamp":"2024-09-13T18:57:47.934770+0000","last_deep_scrub":"43'5452","last_deep_scrub_stamp":"2024-09-13T18:57:47.934770+0000","last_clean_scrub_stamp":"2024-09-13T18:57:47.934770+0000","objects_scrubbed":0,"log_size":5452,"log_dups_size":0,"ondisk_log_size":5452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:56:53.358515+0000","scrub_duration":0.035794448,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":24128,"num_write_kb":505876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"43'11","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:37.925394+0000","last_change":"2024-09-13T18:57:37.925499+0000","last_active":"2024-09-13T18:57:37.925394+0000","last_peered":"2024-09-13T18:57:37.925394+0000","last_clean":"2024-09-13T18:57:37.925394+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:57:37.925394+0000","last_undegraded":"2024-09-13T18:57:37.925394+0000","last_fullsized":"2024-09-13T18:57:37.925394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:37.925363+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:37.925363+0000","last_clean_scrub_stamp":"2024-09-13T18:57:37.925363+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:10.713761+0000","scrub_duration":0.0037509309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"43'16","reported_seq":82,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:32.292293+0000","last_change":"2024-09-13T18:57:32.292403+0000","last_active":"2024-09-13T18:57:32.292293+0000","last_peered":"2024-09-13T18:57:32.292293+0000","last_clean":"2024-09-13T18:57:32.292293+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:57:32.292293+0000","last_undegraded":"2024-09-13T18:57:32.292293+0000","last_fullsized":"2024-09-13T18:57:32.292293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:32.292267+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:32.292267+0000","last_clean_scrub_stamp":"2024-09-13T18:57:32.292267+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:41.284438+0000","scrub_duration":0.025296163999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"43'5442","reported_seq":5523,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:48.974134+0000","last_change":"2024-09-13T18:57:48.974304+0000","last_active":"2024-09-13T18:57:48.974134+0000","last_peered":"2024-09-13T18:57:48.974134+0000","last_clean":"2024-09-13T18:57:48.974134+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:57:48.974134+0000","last_undegraded":"2024-09-13T18:57:48.974134+0000","last_fullsized":"2024-09-13T18:57:48.974134+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5442","last_scrub_stamp":"2024-09-13T18:57:48.974110+0000","last_deep_scrub":"43'5442","last_deep_scrub_stamp":"2024-09-13T18:57:48.974110+0000","last_clean_scrub_stamp":"2024-09-13T18:57:48.974110+0000","objects_scrubbed":0,"log_size":5442,"log_dups_size":0,"ondisk_log_size":5442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:38.773027+0000","scrub_duration":0.039492270000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":24412,"num_write_kb":498911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"43'10","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:41.082668+0000","last_change":"2024-09-13T18:57:41.082668+0000","last_active":"2024-09-13T18:57:41.082668+0000","last_peered":"2024-09-13T18:57:41.082668+0000","last_clean":"2024-09-13T18:57:41.082668+0000","last_became_active":"2024-09-13T18:27:22.609642+0000","last_became_peered":"2024-09-13T18:27:22.609642+0000","last_unstale":"2024-09-13T18:57:41.082668+0000","last_undegraded":"2024-09-13T18:57:41.082668+0000","last_fullsized":"2024-09-13T18:57:41.082668+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:41.082525+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:41.082525+0000","last_clean_scrub_stamp":"2024-09-13T18:57:41.082525+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:52:04.123510+0000","scrub_duration":0.0029389820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"43'7","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.406801+0000","last_change":"2024-09-13T18:57:20.192184+0000","last_active":"2024-09-13T18:57:26.406801+0000","last_peered":"2024-09-13T18:57:26.406801+0000","last_clean":"2024-09-13T18:57:26.406801+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:57:26.406801+0000","last_undegraded":"2024-09-13T18:57:26.406801+0000","last_fullsized":"2024-09-13T18:57:26.406801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:20.191966+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:20.191966+0000","last_clean_scrub_stamp":"2024-09-13T18:57:20.191966+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:39.033583+0000","scrub_duration":0.0068871679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"43'7006","reported_seq":7087,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:43.055580+0000","last_change":"2024-09-13T18:57:43.055725+0000","last_active":"2024-09-13T18:57:43.055580+0000","last_peered":"2024-09-13T18:57:43.055580+0000","last_clean":"2024-09-13T18:57:43.055580+0000","last_became_active":"2024-09-13T18:27:22.610527+0000","last_became_peered":"2024-09-13T18:27:22.610527+0000","last_unstale":"2024-09-13T18:57:43.055580+0000","last_undegraded":"2024-09-13T18:57:43.055580+0000","last_fullsized":"2024-09-13T18:57:43.055580+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'7006","last_scrub_stamp":"2024-09-13T18:57:43.055548+0000","last_deep_scrub":"43'7006","last_deep_scrub_stamp":"2024-09-13T18:57:43.055548+0000","last_clean_scrub_stamp":"2024-09-13T18:57:43.055548+0000","objects_scrubbed":0,"log_size":7006,"log_dups_size":0,"ondisk_log_size":7006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:13:12.839547+0000","scrub_duration":0.042924489000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":31810,"num_write_kb":622755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"43'11","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.393739+0000","last_change":"2024-09-13T18:57:17.999669+0000","last_active":"2024-09-13T18:57:28.393739+0000","last_peered":"2024-09-13T18:57:28.393739+0000","last_clean":"2024-09-13T18:57:28.393739+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:57:28.393739+0000","last_undegraded":"2024-09-13T18:57:28.393739+0000","last_fullsized":"2024-09-13T18:57:28.393739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:17.999495+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:17.999495+0000","last_clean_scrub_stamp":"2024-09-13T18:57:17.999495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:58.108162+0000","scrub_duration":0.0026512860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"43'7","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.926150+0000","last_change":"2024-09-13T18:57:39.926414+0000","last_active":"2024-09-13T18:57:39.926150+0000","last_peered":"2024-09-13T18:57:39.926150+0000","last_clean":"2024-09-13T18:57:39.926150+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:57:39.926150+0000","last_undegraded":"2024-09-13T18:57:39.926150+0000","last_fullsized":"2024-09-13T18:57:39.926150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'7","last_scrub_stamp":"2024-09-13T18:57:39.926056+0000","last_deep_scrub":"43'7","last_deep_scrub_stamp":"2024-09-13T18:57:39.926056+0000","last_clean_scrub_stamp":"2024-09-13T18:57:39.926056+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:42.161980+0000","scrub_duration":0.0070368430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"43'5416","reported_seq":5505,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:42.965414+0000","last_change":"2024-09-13T18:57:42.965611+0000","last_active":"2024-09-13T18:57:42.965414+0000","last_peered":"2024-09-13T18:57:42.965414+0000","last_clean":"2024-09-13T18:57:42.965414+0000","last_became_active":"2024-09-13T18:27:22.648054+0000","last_became_peered":"2024-09-13T18:27:22.648054+0000","last_unstale":"2024-09-13T18:57:42.965414+0000","last_undegraded":"2024-09-13T18:57:42.965414+0000","last_fullsized":"2024-09-13T18:57:42.965414+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5416","last_scrub_stamp":"2024-09-13T18:57:42.965388+0000","last_deep_scrub":"43'5416","last_deep_scrub_stamp":"2024-09-13T18:57:42.965388+0000","last_clean_scrub_stamp":"2024-09-13T18:57:42.965388+0000","objects_scrubbed":0,"log_size":5416,"log_dups_size":0,"ondisk_log_size":5416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:18.488873+0000","scrub_duration":0.042479966000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":23625,"num_write_kb":471297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"43'19","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.065688+0000","last_change":"2024-09-13T18:57:22.408978+0000","last_active":"2024-09-13T18:57:28.065688+0000","last_peered":"2024-09-13T18:57:28.065688+0000","last_clean":"2024-09-13T18:57:28.065688+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:57:28.065688+0000","last_undegraded":"2024-09-13T18:57:28.065688+0000","last_fullsized":"2024-09-13T18:57:28.065688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:22.408739+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:22.408739+0000","last_clean_scrub_stamp":"2024-09-13T18:57:22.408739+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:45:55.319773+0000","scrub_duration":0.003011952,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"43'11","reported_seq":94,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.645812+0000","last_change":"2024-09-13T18:57:18.823588+0000","last_active":"2024-09-13T18:57:27.645812+0000","last_peered":"2024-09-13T18:57:27.645812+0000","last_clean":"2024-09-13T18:57:27.645812+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:57:27.645812+0000","last_undegraded":"2024-09-13T18:57:27.645812+0000","last_fullsized":"2024-09-13T18:57:27.645812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'7","last_scrub_stamp":"2024-09-13T18:57:18.823152+0000","last_deep_scrub":"43'7","last_deep_scrub_stamp":"2024-09-13T18:57:18.823152+0000","last_clean_scrub_stamp":"2024-09-13T18:57:18.823152+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:52.572271+0000","scrub_duration":0.023118413000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"43'5405","reported_seq":5489,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.568512+0000","last_change":"2024-09-13T18:57:36.167581+0000","last_active":"2024-09-13T18:57:38.568512+0000","last_peered":"2024-09-13T18:57:38.568512+0000","last_clean":"2024-09-13T18:57:38.568512+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:57:38.568512+0000","last_undegraded":"2024-09-13T18:57:38.568512+0000","last_fullsized":"2024-09-13T18:57:38.568512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5395","last_scrub_stamp":"2024-09-13T18:57:36.167414+0000","last_deep_scrub":"43'5395","last_deep_scrub_stamp":"2024-09-13T18:57:36.167414+0000","last_clean_scrub_stamp":"2024-09-13T18:57:36.167414+0000","objects_scrubbed":2,"log_size":5405,"log_dups_size":0,"ondisk_log_size":5405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:04:32.293024+0000","scrub_duration":0.24736883800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":24452,"num_write_kb":507343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"43'13","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.384061+0000","last_change":"2024-09-13T18:57:17.064133+0000","last_active":"2024-09-13T18:57:28.384061+0000","last_peered":"2024-09-13T18:57:28.384061+0000","last_clean":"2024-09-13T18:57:28.384061+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:57:28.384061+0000","last_undegraded":"2024-09-13T18:57:28.384061+0000","last_fullsized":"2024-09-13T18:57:28.384061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:17.063923+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:17.063923+0000","last_clean_scrub_stamp":"2024-09-13T18:57:17.063923+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.609366+0000","scrub_duration":0.020682738999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"43'289","reported_seq":363,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.907282+0000","last_change":"2024-09-13T18:57:22.143854+0000","last_active":"2024-09-13T18:57:24.907282+0000","last_peered":"2024-09-13T18:57:24.907282+0000","last_clean":"2024-09-13T18:57:24.907282+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:57:24.907282+0000","last_undegraded":"2024-09-13T18:57:24.907282+0000","last_fullsized":"2024-09-13T18:57:24.907282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'286","last_scrub_stamp":"2024-09-13T18:57:22.143716+0000","last_deep_scrub":"43'286","last_deep_scrub_stamp":"2024-09-13T18:57:22.143716+0000","last_clean_scrub_stamp":"2024-09-13T18:57:22.143716+0000","objects_scrubbed":2,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:44.447938+0000","scrub_duration":0.0070881099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":52,"num_write":626,"num_write_kb":212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"43'6116","reported_seq":6198,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.579563+0000","last_change":"2024-09-13T18:27:22.645275+0000","last_active":"2024-09-13T18:57:38.579563+0000","last_peered":"2024-09-13T18:57:38.579563+0000","last_clean":"2024-09-13T18:57:38.579563+0000","last_became_active":"2024-09-13T18:27:22.644681+0000","last_became_peered":"2024-09-13T18:27:22.644681+0000","last_unstale":"2024-09-13T18:57:38.579563+0000","last_undegraded":"2024-09-13T18:57:38.579563+0000","last_fullsized":"2024-09-13T18:57:38.579563+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6116,"log_dups_size":0,"ondisk_log_size":6116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":27902,"num_write_kb":553947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"43'9","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.134071+0000","last_change":"2024-09-13T18:57:13.217203+0000","last_active":"2024-09-13T18:57:28.134071+0000","last_peered":"2024-09-13T18:57:28.134071+0000","last_clean":"2024-09-13T18:57:28.134071+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:57:28.134071+0000","last_undegraded":"2024-09-13T18:57:28.134071+0000","last_fullsized":"2024-09-13T18:57:28.134071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:13.216878+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:13.216878+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.216878+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:25:20.911582+0000","scrub_duration":0.0034586249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"43'8","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:29.997531+0000","last_change":"2024-09-13T18:57:29.997531+0000","last_active":"2024-09-13T18:57:29.997531+0000","last_peered":"2024-09-13T18:57:29.997531+0000","last_clean":"2024-09-13T18:57:29.997531+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:57:29.997531+0000","last_undegraded":"2024-09-13T18:57:29.997531+0000","last_fullsized":"2024-09-13T18:57:29.997531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:29.997356+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:29.997356+0000","last_clean_scrub_stamp":"2024-09-13T18:57:29.997356+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:42:58.364306+0000","scrub_duration":0.076701386999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"43'4791","reported_seq":4855,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.580805+0000","last_change":"2024-09-13T18:26:36.408844+0000","last_active":"2024-09-13T18:57:38.580805+0000","last_peered":"2024-09-13T18:57:38.580805+0000","last_clean":"2024-09-13T18:57:38.580805+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:57:38.580805+0000","last_undegraded":"2024-09-13T18:57:38.580805+0000","last_fullsized":"2024-09-13T18:57:38.580805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4791,"log_dups_size":0,"ondisk_log_size":4791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":22435,"num_write_kb":450115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"43'12","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.070641+0000","last_change":"2024-09-13T18:57:13.000734+0000","last_active":"2024-09-13T18:57:28.070641+0000","last_peered":"2024-09-13T18:57:28.070641+0000","last_clean":"2024-09-13T18:57:28.070641+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:57:28.070641+0000","last_undegraded":"2024-09-13T18:57:28.070641+0000","last_fullsized":"2024-09-13T18:57:28.070641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'7","last_scrub_stamp":"2024-09-13T18:57:13.000480+0000","last_deep_scrub":"28'7","last_deep_scrub_stamp":"2024-09-13T18:57:13.000480+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.000480+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:33.028238+0000","scrub_duration":0.0036870150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"43'3","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.386642+0000","last_change":"2024-09-13T18:57:14.228792+0000","last_active":"2024-09-13T18:57:27.386642+0000","last_peered":"2024-09-13T18:57:27.386642+0000","last_clean":"2024-09-13T18:57:27.386642+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:57:27.386642+0000","last_undegraded":"2024-09-13T18:57:27.386642+0000","last_fullsized":"2024-09-13T18:57:27.386642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:57:14.228519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:57:14.228519+0000","last_clean_scrub_stamp":"2024-09-13T18:57:14.228519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:55.913907+0000","scrub_duration":0.0032354189999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"43'6174","reported_seq":6245,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.689282+0000","last_change":"2024-09-13T18:26:36.398876+0000","last_active":"2024-09-13T18:57:38.689282+0000","last_peered":"2024-09-13T18:57:38.689282+0000","last_clean":"2024-09-13T18:57:38.689282+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:57:38.689282+0000","last_undegraded":"2024-09-13T18:57:38.689282+0000","last_fullsized":"2024-09-13T18:57:38.689282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6174,"log_dups_size":0,"ondisk_log_size":6174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":28067,"num_write_kb":562911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"43'9","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.442033+0000","last_change":"2024-09-13T18:57:12.010586+0000","last_active":"2024-09-13T18:57:26.442033+0000","last_peered":"2024-09-13T18:57:26.442033+0000","last_clean":"2024-09-13T18:57:26.442033+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:57:26.442033+0000","last_undegraded":"2024-09-13T18:57:26.442033+0000","last_fullsized":"2024-09-13T18:57:26.442033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'6","last_scrub_stamp":"2024-09-13T18:57:12.010428+0000","last_deep_scrub":"28'6","last_deep_scrub_stamp":"2024-09-13T18:57:12.010428+0000","last_clean_scrub_stamp":"2024-09-13T18:57:12.010428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:15.823406+0000","scrub_duration":0.0039755040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"43'9","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.428380+0000","last_change":"2024-09-13T18:57:15.993201+0000","last_active":"2024-09-13T18:57:26.428380+0000","last_peered":"2024-09-13T18:57:26.428380+0000","last_clean":"2024-09-13T18:57:26.428380+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:57:26.428380+0000","last_undegraded":"2024-09-13T18:57:26.428380+0000","last_fullsized":"2024-09-13T18:57:26.428380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'6","last_scrub_stamp":"2024-09-13T18:57:15.992890+0000","last_deep_scrub":"41'6","last_deep_scrub_stamp":"2024-09-13T18:57:15.992890+0000","last_clean_scrub_stamp":"2024-09-13T18:57:15.992890+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:28:43.312440+0000","scrub_duration":0.0036839949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"43'5413","reported_seq":5486,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.366401+0000","last_change":"2024-09-13T18:26:36.401777+0000","last_active":"2024-09-13T18:57:38.366401+0000","last_peered":"2024-09-13T18:57:38.366401+0000","last_clean":"2024-09-13T18:57:38.366401+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:57:38.366401+0000","last_undegraded":"2024-09-13T18:57:38.366401+0000","last_fullsized":"2024-09-13T18:57:38.366401+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5413,"log_dups_size":0,"ondisk_log_size":5413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":24545,"num_write_kb":507684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"43'14","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:36.275994+0000","last_change":"2024-09-13T18:57:36.277633+0000","last_active":"2024-09-13T18:57:36.275994+0000","last_peered":"2024-09-13T18:57:36.275994+0000","last_clean":"2024-09-13T18:57:36.275994+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:57:36.275994+0000","last_undegraded":"2024-09-13T18:57:36.275994+0000","last_fullsized":"2024-09-13T18:57:36.275994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:36.275964+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:36.275964+0000","last_clean_scrub_stamp":"2024-09-13T18:57:36.275964+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:55:23.015452+0000","scrub_duration":0.027403200999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"43'131","reported_seq":318,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.095368+0000","last_change":"2024-09-13T18:57:16.151559+0000","last_active":"2024-09-13T18:57:28.095368+0000","last_peered":"2024-09-13T18:57:28.095368+0000","last_clean":"2024-09-13T18:57:28.095368+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:57:28.095368+0000","last_undegraded":"2024-09-13T18:57:28.095368+0000","last_fullsized":"2024-09-13T18:57:28.095368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'130","last_scrub_stamp":"2024-09-13T18:57:16.151397+0000","last_deep_scrub":"43'130","last_deep_scrub_stamp":"2024-09-13T18:57:16.151397+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.151397+0000","objects_scrubbed":3,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:19.471742+0000","scrub_duration":0.027827343000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93931,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":203,"num_write":119,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"43'6320","reported_seq":6391,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.629554+0000","last_change":"2024-09-13T18:26:36.401529+0000","last_active":"2024-09-13T18:57:38.629554+0000","last_peered":"2024-09-13T18:57:38.629554+0000","last_clean":"2024-09-13T18:57:38.629554+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:57:38.629554+0000","last_undegraded":"2024-09-13T18:57:38.629554+0000","last_fullsized":"2024-09-13T18:57:38.629554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6320,"log_dups_size":0,"ondisk_log_size":6320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":28303,"num_write_kb":563741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"43'8","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.026232+0000","last_change":"2024-09-13T18:57:14.329551+0000","last_active":"2024-09-13T18:57:28.026232+0000","last_peered":"2024-09-13T18:57:28.026232+0000","last_clean":"2024-09-13T18:57:28.026232+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:57:28.026232+0000","last_undegraded":"2024-09-13T18:57:28.026232+0000","last_fullsized":"2024-09-13T18:57:28.026232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:14.329297+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:14.329297+0000","last_clean_scrub_stamp":"2024-09-13T18:57:14.329297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:49.431750+0000","scrub_duration":0.0033135230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"43'18","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.912770+0000","last_change":"2024-09-13T18:57:26.912770+0000","last_active":"2024-09-13T18:57:26.912770+0000","last_peered":"2024-09-13T18:57:26.912770+0000","last_clean":"2024-09-13T18:57:26.912770+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:57:26.912770+0000","last_undegraded":"2024-09-13T18:57:26.912770+0000","last_fullsized":"2024-09-13T18:57:26.912770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'17","last_scrub_stamp":"2024-09-13T18:57:26.912423+0000","last_deep_scrub":"43'17","last_deep_scrub_stamp":"2024-09-13T18:57:26.912423+0000","last_clean_scrub_stamp":"2024-09-13T18:57:26.912423+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:00.997526+0000","scrub_duration":0.0065643969999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"43'4467","reported_seq":4538,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.545450+0000","last_change":"2024-09-13T18:26:36.405086+0000","last_active":"2024-09-13T18:57:38.545450+0000","last_peered":"2024-09-13T18:57:38.545450+0000","last_clean":"2024-09-13T18:57:38.545450+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:57:38.545450+0000","last_undegraded":"2024-09-13T18:57:38.545450+0000","last_fullsized":"2024-09-13T18:57:38.545450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4467,"log_dups_size":0,"ondisk_log_size":4467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":19862,"num_write_kb":435588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"43'8","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.860482+0000","last_change":"2024-09-13T18:57:39.860482+0000","last_active":"2024-09-13T18:57:39.860482+0000","last_peered":"2024-09-13T18:57:39.860482+0000","last_clean":"2024-09-13T18:57:39.860482+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:57:39.860482+0000","last_undegraded":"2024-09-13T18:57:39.860482+0000","last_fullsized":"2024-09-13T18:57:39.860482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:39.860244+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:39.860244+0000","last_clean_scrub_stamp":"2024-09-13T18:57:39.860244+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:08.621392+0000","scrub_duration":0.0035530589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"43'11","reported_seq":90,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:23.965985+0000","last_change":"2024-09-13T18:57:23.965985+0000","last_active":"2024-09-13T18:57:23.965985+0000","last_peered":"2024-09-13T18:57:23.965985+0000","last_clean":"2024-09-13T18:57:23.965985+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:57:23.965985+0000","last_undegraded":"2024-09-13T18:57:23.965985+0000","last_fullsized":"2024-09-13T18:57:23.965985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:23.965843+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:23.965843+0000","last_clean_scrub_stamp":"2024-09-13T18:57:23.965843+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:51.676617+0000","scrub_duration":0.010225193,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":68,"num_write":9,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"43'119","reported_seq":215,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:19.933586+0000","last_change":"2024-09-13T18:57:19.933697+0000","last_active":"2024-09-13T18:57:19.933586+0000","last_peered":"2024-09-13T18:57:19.933586+0000","last_clean":"2024-09-13T18:57:19.933586+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:57:19.933586+0000","last_undegraded":"2024-09-13T18:57:19.933586+0000","last_fullsized":"2024-09-13T18:57:19.933586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'119","last_scrub_stamp":"2024-09-13T18:57:19.933561+0000","last_deep_scrub":"43'119","last_deep_scrub_stamp":"2024-09-13T18:57:19.933561+0000","last_clean_scrub_stamp":"2024-09-13T18:57:19.933561+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:36.963255+0000","scrub_duration":0.0099632039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"43'6003","reported_seq":6083,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.486419+0000","last_change":"2024-09-13T18:26:36.409827+0000","last_active":"2024-09-13T18:57:38.486419+0000","last_peered":"2024-09-13T18:57:38.486419+0000","last_clean":"2024-09-13T18:57:38.486419+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:57:38.486419+0000","last_undegraded":"2024-09-13T18:57:38.486419+0000","last_fullsized":"2024-09-13T18:57:38.486419+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6003,"log_dups_size":0,"ondisk_log_size":6003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":27421,"num_write_kb":519330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"43'15","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.085572+0000","last_change":"2024-09-13T18:26:32.356850+0000","last_active":"2024-09-13T18:57:28.085572+0000","last_peered":"2024-09-13T18:57:28.085572+0000","last_clean":"2024-09-13T18:57:28.085572+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:57:28.085572+0000","last_undegraded":"2024-09-13T18:57:28.085572+0000","last_fullsized":"2024-09-13T18:57:28.085572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"43'8","reported_seq":81,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:22.774832+0000","last_change":"2024-09-13T18:57:21.187725+0000","last_active":"2024-09-13T18:57:22.774832+0000","last_peered":"2024-09-13T18:57:22.774832+0000","last_clean":"2024-09-13T18:57:22.774832+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:57:22.774832+0000","last_undegraded":"2024-09-13T18:57:22.774832+0000","last_fullsized":"2024-09-13T18:57:22.774832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:21.187525+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:21.187525+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.187525+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:42.675115+0000","scrub_duration":0.032458872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":35,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"43'5433","reported_seq":5498,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.371873+0000","last_change":"2024-09-13T18:26:36.409802+0000","last_active":"2024-09-13T18:57:38.371873+0000","last_peered":"2024-09-13T18:57:38.371873+0000","last_clean":"2024-09-13T18:57:38.371873+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:57:38.371873+0000","last_undegraded":"2024-09-13T18:57:38.371873+0000","last_fullsized":"2024-09-13T18:57:38.371873+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":24067,"num_write_kb":513904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"43'13","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:36.120773+0000","last_change":"2024-09-13T18:57:36.120879+0000","last_active":"2024-09-13T18:57:36.120773+0000","last_peered":"2024-09-13T18:57:36.120773+0000","last_clean":"2024-09-13T18:57:36.120773+0000","last_became_active":"2024-09-13T18:27:22.610326+0000","last_became_peered":"2024-09-13T18:27:22.610326+0000","last_unstale":"2024-09-13T18:57:36.120773+0000","last_undegraded":"2024-09-13T18:57:36.120773+0000","last_fullsized":"2024-09-13T18:57:36.120773+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:36.120741+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:36.120741+0000","last_clean_scrub_stamp":"2024-09-13T18:57:36.120741+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:03.902589+0000","scrub_duration":0.0026569660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.167737+0000","last_change":"2024-09-13T18:57:27.167920+0000","last_active":"2024-09-13T18:57:27.167737+0000","last_peered":"2024-09-13T18:57:27.167737+0000","last_clean":"2024-09-13T18:57:27.167737+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:57:27.167737+0000","last_undegraded":"2024-09-13T18:57:27.167737+0000","last_fullsized":"2024-09-13T18:57:27.167737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:27.167690+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:27.167690+0000","last_clean_scrub_stamp":"2024-09-13T18:57:27.167690+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:44.845487+0000","scrub_duration":0.0066826000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"43'5174","reported_seq":5242,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.554158+0000","last_change":"2024-09-13T18:26:36.394344+0000","last_active":"2024-09-13T18:57:38.554158+0000","last_peered":"2024-09-13T18:57:38.554158+0000","last_clean":"2024-09-13T18:57:38.554158+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:57:38.554158+0000","last_undegraded":"2024-09-13T18:57:38.554158+0000","last_fullsized":"2024-09-13T18:57:38.554158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":22830,"num_write_kb":472029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"43'11","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.416970+0000","last_change":"2024-09-13T18:26:32.365655+0000","last_active":"2024-09-13T18:57:26.416970+0000","last_peered":"2024-09-13T18:57:26.416970+0000","last_clean":"2024-09-13T18:57:26.416970+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:57:26.416970+0000","last_undegraded":"2024-09-13T18:57:26.416970+0000","last_fullsized":"2024-09-13T18:57:26.416970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"43'10","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.389453+0000","last_change":"2024-09-13T18:57:25.103279+0000","last_active":"2024-09-13T18:57:27.389453+0000","last_peered":"2024-09-13T18:57:27.389453+0000","last_clean":"2024-09-13T18:57:27.389453+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:57:27.389453+0000","last_undegraded":"2024-09-13T18:57:27.389453+0000","last_fullsized":"2024-09-13T18:57:27.389453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:25.103144+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:25.103144+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.103144+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.183107+0000","scrub_duration":0.013012788000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"43'4987","reported_seq":5049,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.563660+0000","last_change":"2024-09-13T18:26:36.410280+0000","last_active":"2024-09-13T18:57:38.563660+0000","last_peered":"2024-09-13T18:57:38.563660+0000","last_clean":"2024-09-13T18:57:38.563660+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:57:38.563660+0000","last_undegraded":"2024-09-13T18:57:38.563660+0000","last_fullsized":"2024-09-13T18:57:38.563660+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":4987,"log_dups_size":0,"ondisk_log_size":4987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":0,"num_write":22775,"num_write_kb":463670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"43'5","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.905690+0000","last_change":"2024-09-13T18:26:32.361575+0000","last_active":"2024-09-13T18:57:24.905690+0000","last_peered":"2024-09-13T18:57:24.905690+0000","last_clean":"2024-09-13T18:57:24.905690+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:57:24.905690+0000","last_undegraded":"2024-09-13T18:57:24.905690+0000","last_fullsized":"2024-09-13T18:57:24.905690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"43'12","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:11.025964+0000","last_change":"2024-09-13T18:57:11.025964+0000","last_active":"2024-09-13T18:57:11.025964+0000","last_peered":"2024-09-13T18:57:11.025964+0000","last_clean":"2024-09-13T18:57:11.025964+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:57:11.025964+0000","last_undegraded":"2024-09-13T18:57:11.025964+0000","last_fullsized":"2024-09-13T18:57:11.025964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:11.025748+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:11.025748+0000","last_clean_scrub_stamp":"2024-09-13T18:57:11.025748+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:22:07.957036+0000","scrub_duration":0.0065355530000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"43'5147","reported_seq":5213,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:37.851989+0000","last_change":"2024-09-13T18:26:36.403479+0000","last_active":"2024-09-13T18:57:37.851989+0000","last_peered":"2024-09-13T18:57:37.851989+0000","last_clean":"2024-09-13T18:57:37.851989+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:57:37.851989+0000","last_undegraded":"2024-09-13T18:57:37.851989+0000","last_fullsized":"2024-09-13T18:57:37.851989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5147,"log_dups_size":0,"ondisk_log_size":5147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":22715,"num_write_kb":463382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"43'12","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:22.204108+0000","last_change":"2024-09-13T18:26:32.365143+0000","last_active":"2024-09-13T18:57:22.204108+0000","last_peered":"2024-09-13T18:57:22.204108+0000","last_clean":"2024-09-13T18:57:22.204108+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:57:22.204108+0000","last_undegraded":"2024-09-13T18:57:22.204108+0000","last_fullsized":"2024-09-13T18:57:22.204108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"43'32","reported_seq":149,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.902980+0000","last_change":"2024-09-13T18:57:20.154497+0000","last_active":"2024-09-13T18:57:24.902980+0000","last_peered":"2024-09-13T18:57:24.902980+0000","last_clean":"2024-09-13T18:57:24.902980+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:57:24.902980+0000","last_undegraded":"2024-09-13T18:57:24.902980+0000","last_fullsized":"2024-09-13T18:57:24.902980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'29","last_scrub_stamp":"2024-09-13T18:57:20.154238+0000","last_deep_scrub":"43'29","last_deep_scrub_stamp":"2024-09-13T18:57:20.154238+0000","last_clean_scrub_stamp":"2024-09-13T18:57:20.154238+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:10.657893+0000","scrub_duration":0.013053932000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":158,"num_read_kb":157,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"43'6074","reported_seq":6148,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.495803+0000","last_change":"2024-09-13T18:26:36.399235+0000","last_active":"2024-09-13T18:57:38.495803+0000","last_peered":"2024-09-13T18:57:38.495803+0000","last_clean":"2024-09-13T18:57:38.495803+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:57:38.495803+0000","last_undegraded":"2024-09-13T18:57:38.495803+0000","last_fullsized":"2024-09-13T18:57:38.495803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6074,"log_dups_size":0,"ondisk_log_size":6074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":27740,"num_write_kb":536907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"43'13","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:32.703996+0000","last_change":"2024-09-13T18:57:32.704206+0000","last_active":"2024-09-13T18:57:32.703996+0000","last_peered":"2024-09-13T18:57:32.703996+0000","last_clean":"2024-09-13T18:57:32.703996+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:57:32.703996+0000","last_undegraded":"2024-09-13T18:57:32.703996+0000","last_fullsized":"2024-09-13T18:57:32.703996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:32.703957+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:32.703957+0000","last_clean_scrub_stamp":"2024-09-13T18:57:32.703957+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:39.171311+0000","scrub_duration":0.018593416000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"43'16","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.908758+0000","last_change":"2024-09-13T18:57:21.771790+0000","last_active":"2024-09-13T18:57:24.908758+0000","last_peered":"2024-09-13T18:57:24.908758+0000","last_clean":"2024-09-13T18:57:24.908758+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:57:24.908758+0000","last_undegraded":"2024-09-13T18:57:24.908758+0000","last_fullsized":"2024-09-13T18:57:24.908758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:21.771519+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:21.771519+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.771519+0000","objects_scrubbed":2,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:44:46.293266+0000","scrub_duration":0.0083382200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1620,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"43'6155","reported_seq":6222,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.504168+0000","last_change":"2024-09-13T18:26:36.406177+0000","last_active":"2024-09-13T18:57:38.504168+0000","last_peered":"2024-09-13T18:57:38.504168+0000","last_clean":"2024-09-13T18:57:38.504168+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:57:38.504168+0000","last_undegraded":"2024-09-13T18:57:38.504168+0000","last_fullsized":"2024-09-13T18:57:38.504168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6155,"log_dups_size":0,"ondisk_log_size":6155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":28268,"num_write_kb":551294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"43'15","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.640336+0000","last_change":"2024-09-13T18:27:22.609309+0000","last_active":"2024-09-13T18:57:27.640336+0000","last_peered":"2024-09-13T18:57:27.640336+0000","last_clean":"2024-09-13T18:57:27.640336+0000","last_became_active":"2024-09-13T18:27:22.609101+0000","last_became_peered":"2024-09-13T18:27:22.609101+0000","last_unstale":"2024-09-13T18:57:27.640336+0000","last_undegraded":"2024-09-13T18:57:27.640336+0000","last_fullsized":"2024-09-13T18:57:27.640336+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"43'14","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:35.005652+0000","last_change":"2024-09-13T18:57:35.005769+0000","last_active":"2024-09-13T18:57:35.005652+0000","last_peered":"2024-09-13T18:57:35.005652+0000","last_clean":"2024-09-13T18:57:35.005652+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:57:35.005652+0000","last_undegraded":"2024-09-13T18:57:35.005652+0000","last_fullsized":"2024-09-13T18:57:35.005652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:35.005633+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:35.005633+0000","last_clean_scrub_stamp":"2024-09-13T18:57:35.005633+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:24.849838+0000","scrub_duration":0.0050291620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"43'5953","reported_seq":6024,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.498590+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:57:38.498590+0000","last_peered":"2024-09-13T18:57:38.498590+0000","last_clean":"2024-09-13T18:57:38.498590+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:57:38.498590+0000","last_undegraded":"2024-09-13T18:57:38.498590+0000","last_fullsized":"2024-09-13T18:57:38.498590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5953,"log_dups_size":0,"ondisk_log_size":5953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27125,"num_write_kb":550847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"43'15","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:34.907335+0000","last_change":"2024-09-13T18:57:25.418872+0000","last_active":"2024-09-13T18:57:34.907335+0000","last_peered":"2024-09-13T18:57:34.907335+0000","last_clean":"2024-09-13T18:57:34.907335+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:57:34.907335+0000","last_undegraded":"2024-09-13T18:57:34.907335+0000","last_fullsized":"2024-09-13T18:57:34.907335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:57:25.418501+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:57:25.418501+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.418501+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:32.901503+0000","scrub_duration":0.003478723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"43'20","reported_seq":111,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.419225+0000","last_change":"2024-09-13T18:57:13.300861+0000","last_active":"2024-09-13T18:57:26.419225+0000","last_peered":"2024-09-13T18:57:26.419225+0000","last_clean":"2024-09-13T18:57:26.419225+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:57:26.419225+0000","last_undegraded":"2024-09-13T18:57:26.419225+0000","last_fullsized":"2024-09-13T18:57:26.419225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:13.300701+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:13.300701+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.300701+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:09.337133+0000","scrub_duration":0.0071490659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":93,"num_read_kb":112,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"43'5303","reported_seq":5369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.605322+0000","last_change":"2024-09-13T18:26:36.410014+0000","last_active":"2024-09-13T18:57:38.605322+0000","last_peered":"2024-09-13T18:57:38.605322+0000","last_clean":"2024-09-13T18:57:38.605322+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:57:38.605322+0000","last_undegraded":"2024-09-13T18:57:38.605322+0000","last_fullsized":"2024-09-13T18:57:38.605322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5303,"log_dups_size":0,"ondisk_log_size":5303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":23240,"num_write_kb":490059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"43'12","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:31.333814+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:57:31.333814+0000","last_peered":"2024-09-13T18:57:31.333814+0000","last_clean":"2024-09-13T18:57:31.333814+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:57:31.333814+0000","last_undegraded":"2024-09-13T18:57:31.333814+0000","last_fullsized":"2024-09-13T18:57:31.333814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"43'389","reported_seq":488,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.908598+0000","last_change":"2024-09-13T18:57:13.774570+0000","last_active":"2024-09-13T18:57:24.908598+0000","last_peered":"2024-09-13T18:57:24.908598+0000","last_clean":"2024-09-13T18:57:24.908598+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:57:24.908598+0000","last_undegraded":"2024-09-13T18:57:24.908598+0000","last_fullsized":"2024-09-13T18:57:24.908598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'387","last_scrub_stamp":"2024-09-13T18:57:13.774429+0000","last_deep_scrub":"43'387","last_deep_scrub_stamp":"2024-09-13T18:57:13.774429+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.774429+0000","objects_scrubbed":5,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:38.043525+0000","scrub_duration":0.046187055999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":93,"num_read_kb":20568,"num_write":383,"num_write_kb":4318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"43'5475","reported_seq":5542,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.528574+0000","last_change":"2024-09-13T18:26:36.394080+0000","last_active":"2024-09-13T18:57:38.528574+0000","last_peered":"2024-09-13T18:57:38.528574+0000","last_clean":"2024-09-13T18:57:38.528574+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:57:38.528574+0000","last_undegraded":"2024-09-13T18:57:38.528574+0000","last_fullsized":"2024-09-13T18:57:38.528574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":25034,"num_write_kb":485267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"43'10","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.055388+0000","last_change":"2024-09-13T18:57:16.962251+0000","last_active":"2024-09-13T18:57:28.055388+0000","last_peered":"2024-09-13T18:57:28.055388+0000","last_clean":"2024-09-13T18:57:28.055388+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:57:28.055388+0000","last_undegraded":"2024-09-13T18:57:28.055388+0000","last_fullsized":"2024-09-13T18:57:28.055388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:16.961847+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:16.961847+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.961847+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:54.684373+0000","scrub_duration":0.0040550839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"43'66","reported_seq":141,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.530024+0000","last_change":"2024-09-13T18:57:34.318022+0000","last_active":"2024-09-13T18:57:39.530024+0000","last_peered":"2024-09-13T18:57:39.530024+0000","last_clean":"2024-09-13T18:57:39.530024+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:57:39.530024+0000","last_undegraded":"2024-09-13T18:57:39.530024+0000","last_fullsized":"2024-09-13T18:57:39.530024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'65","last_scrub_stamp":"2024-09-13T18:57:34.317748+0000","last_deep_scrub":"43'65","last_deep_scrub_stamp":"2024-09-13T18:57:34.317748+0000","last_clean_scrub_stamp":"2024-09-13T18:57:34.317748+0000","objects_scrubbed":3,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:47.140265+0000","scrub_duration":0.0064888469999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":65,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"43'5962","reported_seq":6044,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:45.941460+0000","last_change":"2024-09-13T18:57:45.941610+0000","last_active":"2024-09-13T18:57:45.941460+0000","last_peered":"2024-09-13T18:57:45.941460+0000","last_clean":"2024-09-13T18:57:45.941460+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:57:45.941460+0000","last_undegraded":"2024-09-13T18:57:45.941460+0000","last_fullsized":"2024-09-13T18:57:45.941460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5962","last_scrub_stamp":"2024-09-13T18:57:45.941438+0000","last_deep_scrub":"43'5962","last_deep_scrub_stamp":"2024-09-13T18:57:45.941438+0000","last_clean_scrub_stamp":"2024-09-13T18:57:45.941438+0000","objects_scrubbed":0,"log_size":5962,"log_dups_size":0,"ondisk_log_size":5962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:34.601990+0000","scrub_duration":0.038953110999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27473,"num_write_kb":527755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"43'16","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:32.414816+0000","last_change":"2024-09-13T18:57:32.414926+0000","last_active":"2024-09-13T18:57:32.414816+0000","last_peered":"2024-09-13T18:57:32.414816+0000","last_clean":"2024-09-13T18:57:32.414816+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:57:32.414816+0000","last_undegraded":"2024-09-13T18:57:32.414816+0000","last_fullsized":"2024-09-13T18:57:32.414816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:32.414785+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:32.414785+0000","last_clean_scrub_stamp":"2024-09-13T18:57:32.414785+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:58:20.923011+0000","scrub_duration":0.0019847889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"43'6","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:22.784542+0000","last_change":"2024-09-13T18:57:18.948854+0000","last_active":"2024-09-13T18:57:22.784542+0000","last_peered":"2024-09-13T18:57:22.784542+0000","last_clean":"2024-09-13T18:57:22.784542+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:57:22.784542+0000","last_undegraded":"2024-09-13T18:57:22.784542+0000","last_fullsized":"2024-09-13T18:57:22.784542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4","last_scrub_stamp":"2024-09-13T18:57:18.948753+0000","last_deep_scrub":"43'4","last_deep_scrub_stamp":"2024-09-13T18:57:18.948753+0000","last_clean_scrub_stamp":"2024-09-13T18:57:18.948753+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:24:26.277291+0000","scrub_duration":0.0026056769999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"43'5502","reported_seq":5564,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.558626+0000","last_change":"2024-09-13T18:26:36.406090+0000","last_active":"2024-09-13T18:57:38.558626+0000","last_peered":"2024-09-13T18:57:38.558626+0000","last_clean":"2024-09-13T18:57:38.558626+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:57:38.558626+0000","last_undegraded":"2024-09-13T18:57:38.558626+0000","last_fullsized":"2024-09-13T18:57:38.558626+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5502,"log_dups_size":0,"ondisk_log_size":5502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":25045,"num_write_kb":489215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"43'13","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:45.146406+0000","last_change":"2024-09-13T18:57:45.146635+0000","last_active":"2024-09-13T18:57:45.146406+0000","last_peered":"2024-09-13T18:57:45.146406+0000","last_clean":"2024-09-13T18:57:45.146406+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:57:45.146406+0000","last_undegraded":"2024-09-13T18:57:45.146406+0000","last_fullsized":"2024-09-13T18:57:45.146406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:45.146357+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:45.146357+0000","last_clean_scrub_stamp":"2024-09-13T18:57:45.146357+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:12.892350+0000","scrub_duration":0.0033819689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"43'19","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:22.765983+0000","last_change":"2024-09-13T18:57:22.765983+0000","last_active":"2024-09-13T18:57:22.765983+0000","last_peered":"2024-09-13T18:57:22.765983+0000","last_clean":"2024-09-13T18:57:22.765983+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:57:22.765983+0000","last_undegraded":"2024-09-13T18:57:22.765983+0000","last_fullsized":"2024-09-13T18:57:22.765983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'18","last_scrub_stamp":"2024-09-13T18:57:22.765727+0000","last_deep_scrub":"43'18","last_deep_scrub_stamp":"2024-09-13T18:57:22.765727+0000","last_clean_scrub_stamp":"2024-09-13T18:57:22.765727+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:04.965728+0000","scrub_duration":0.0065666600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":18,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"43'5179","reported_seq":5248,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:37.619015+0000","last_change":"2024-09-13T18:26:36.406553+0000","last_active":"2024-09-13T18:57:37.619015+0000","last_peered":"2024-09-13T18:57:37.619015+0000","last_clean":"2024-09-13T18:57:37.619015+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:57:37.619015+0000","last_undegraded":"2024-09-13T18:57:37.619015+0000","last_fullsized":"2024-09-13T18:57:37.619015+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5179,"log_dups_size":0,"ondisk_log_size":5179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":23340,"num_write_kb":507027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"43'6189","reported_seq":6260,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.631829+0000","last_change":"2024-09-13T18:26:36.406986+0000","last_active":"2024-09-13T18:57:38.631829+0000","last_peered":"2024-09-13T18:57:38.631829+0000","last_clean":"2024-09-13T18:57:38.631829+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:57:38.631829+0000","last_undegraded":"2024-09-13T18:57:38.631829+0000","last_fullsized":"2024-09-13T18:57:38.631829+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6189,"log_dups_size":0,"ondisk_log_size":6189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":28563,"num_write_kb":581141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"43'10","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.452142+0000","last_change":"2024-09-13T18:26:32.349081+0000","last_active":"2024-09-13T18:57:26.452142+0000","last_peered":"2024-09-13T18:57:26.452142+0000","last_clean":"2024-09-13T18:57:26.452142+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:57:26.452142+0000","last_undegraded":"2024-09-13T18:57:26.452142+0000","last_fullsized":"2024-09-13T18:57:26.452142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"43'8","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:21.964416+0000","last_change":"2024-09-13T18:57:21.964628+0000","last_active":"2024-09-13T18:57:21.964416+0000","last_peered":"2024-09-13T18:57:21.964416+0000","last_clean":"2024-09-13T18:57:21.964416+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:57:21.964416+0000","last_undegraded":"2024-09-13T18:57:21.964416+0000","last_fullsized":"2024-09-13T18:57:21.964416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:21.964364+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:21.964364+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.964364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:31:05.131047+0000","scrub_duration":0.003227541,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"43'5646","reported_seq":5720,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.679721+0000","last_change":"2024-09-13T18:26:36.396418+0000","last_active":"2024-09-13T18:57:38.679721+0000","last_peered":"2024-09-13T18:57:38.679721+0000","last_clean":"2024-09-13T18:57:38.679721+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:57:38.679721+0000","last_undegraded":"2024-09-13T18:57:38.679721+0000","last_fullsized":"2024-09-13T18:57:38.679721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5646,"log_dups_size":0,"ondisk_log_size":5646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":25607,"num_write_kb":520259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"43'19","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:42.857148+0000","last_change":"2024-09-13T18:57:42.857256+0000","last_active":"2024-09-13T18:57:42.857148+0000","last_peered":"2024-09-13T18:57:42.857148+0000","last_clean":"2024-09-13T18:57:42.857148+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:57:42.857148+0000","last_undegraded":"2024-09-13T18:57:42.857148+0000","last_fullsized":"2024-09-13T18:57:42.857148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'19","last_scrub_stamp":"2024-09-13T18:57:42.857115+0000","last_deep_scrub":"43'19","last_deep_scrub_stamp":"2024-09-13T18:57:42.857115+0000","last_clean_scrub_stamp":"2024-09-13T18:57:42.857115+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:32.087665+0000","scrub_duration":0.0022791909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"43'6","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:21.960612+0000","last_change":"2024-09-13T18:57:21.960612+0000","last_active":"2024-09-13T18:57:21.960612+0000","last_peered":"2024-09-13T18:57:21.960612+0000","last_clean":"2024-09-13T18:57:21.960612+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:57:21.960612+0000","last_undegraded":"2024-09-13T18:57:21.960612+0000","last_fullsized":"2024-09-13T18:57:21.960612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:21.960203+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:21.960203+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.960203+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:59:06.847239+0000","scrub_duration":0.006849478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"43'13","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.647844+0000","last_change":"2024-09-13T18:57:16.794490+0000","last_active":"2024-09-13T18:57:27.647844+0000","last_peered":"2024-09-13T18:57:27.647844+0000","last_clean":"2024-09-13T18:57:27.647844+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:57:27.647844+0000","last_undegraded":"2024-09-13T18:57:27.647844+0000","last_fullsized":"2024-09-13T18:57:27.647844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:16.794241+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:16.794241+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.794241+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:35.699392+0000","scrub_duration":0.0056238700000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"43'10","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:42.246148+0000","last_change":"2024-09-13T18:57:42.246326+0000","last_active":"2024-09-13T18:57:42.246148+0000","last_peered":"2024-09-13T18:57:42.246148+0000","last_clean":"2024-09-13T18:57:42.246148+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:57:42.246148+0000","last_undegraded":"2024-09-13T18:57:42.246148+0000","last_fullsized":"2024-09-13T18:57:42.246148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:42.246101+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:42.246101+0000","last_clean_scrub_stamp":"2024-09-13T18:57:42.246101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:54.930475+0000","scrub_duration":0.0033923690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"43'4199","reported_seq":4275,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.531206+0000","last_change":"2024-09-13T18:57:32.974532+0000","last_active":"2024-09-13T18:57:38.531206+0000","last_peered":"2024-09-13T18:57:38.531206+0000","last_clean":"2024-09-13T18:57:38.531206+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:57:38.531206+0000","last_undegraded":"2024-09-13T18:57:38.531206+0000","last_fullsized":"2024-09-13T18:57:38.531206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4177","last_scrub_stamp":"2024-09-13T18:57:32.974351+0000","last_deep_scrub":"43'4177","last_deep_scrub_stamp":"2024-09-13T18:57:32.974351+0000","last_clean_scrub_stamp":"2024-09-13T18:57:32.974351+0000","objects_scrubbed":11,"log_size":4199,"log_dups_size":0,"ondisk_log_size":4199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:34.816537+0000","scrub_duration":0.80893831299999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":19219,"num_write_kb":424944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"43'759","reported_seq":843,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.910037+0000","last_change":"2024-09-13T18:57:28.910129+0000","last_active":"2024-09-13T18:57:28.910037+0000","last_peered":"2024-09-13T18:57:28.910037+0000","last_clean":"2024-09-13T18:57:28.910037+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:57:28.910037+0000","last_undegraded":"2024-09-13T18:57:28.910037+0000","last_fullsized":"2024-09-13T18:57:28.910037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'759","last_scrub_stamp":"2024-09-13T18:57:28.910019+0000","last_deep_scrub":"43'759","last_deep_scrub_stamp":"2024-09-13T18:57:28.910019+0000","last_clean_scrub_stamp":"2024-09-13T18:57:28.910019+0000","objects_scrubbed":3,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:04.313816+0000","scrub_duration":0.042299119000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196273,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":19627,"num_write":735,"num_write_kb":4345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"43'13","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.967148+0000","last_change":"2024-09-13T18:57:20.986740+0000","last_active":"2024-09-13T18:57:24.967148+0000","last_peered":"2024-09-13T18:57:24.967148+0000","last_clean":"2024-09-13T18:57:24.967148+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:57:24.967148+0000","last_undegraded":"2024-09-13T18:57:24.967148+0000","last_fullsized":"2024-09-13T18:57:24.967148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:20.986473+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:20.986473+0000","last_clean_scrub_stamp":"2024-09-13T18:57:20.986473+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:56.996793+0000","scrub_duration":0.0033945970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"43'6014","reported_seq":6083,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.651205+0000","last_change":"2024-09-13T18:26:36.403817+0000","last_active":"2024-09-13T18:57:38.651205+0000","last_peered":"2024-09-13T18:57:38.651205+0000","last_clean":"2024-09-13T18:57:38.651205+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:57:38.651205+0000","last_undegraded":"2024-09-13T18:57:38.651205+0000","last_fullsized":"2024-09-13T18:57:38.651205+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":6014,"log_dups_size":0,"ondisk_log_size":6014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":26799,"num_write_kb":557751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"43'16","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.583187+0000","last_change":"2024-09-13T18:26:32.357856+0000","last_active":"2024-09-13T18:57:27.583187+0000","last_peered":"2024-09-13T18:57:27.583187+0000","last_clean":"2024-09-13T18:57:27.583187+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:57:27.583187+0000","last_undegraded":"2024-09-13T18:57:27.583187+0000","last_fullsized":"2024-09-13T18:57:27.583187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"43'12","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:31.097096+0000","last_change":"2024-09-13T18:57:31.097096+0000","last_active":"2024-09-13T18:57:31.097096+0000","last_peered":"2024-09-13T18:57:31.097096+0000","last_clean":"2024-09-13T18:57:31.097096+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:57:31.097096+0000","last_undegraded":"2024-09-13T18:57:31.097096+0000","last_fullsized":"2024-09-13T18:57:31.097096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:57:31.096973+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:57:31.096973+0000","last_clean_scrub_stamp":"2024-09-13T18:57:31.096973+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:08.952276+0000","scrub_duration":0.0027320500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"43'17","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.122971+0000","last_change":"2024-09-13T18:57:39.122971+0000","last_active":"2024-09-13T18:57:39.122971+0000","last_peered":"2024-09-13T18:57:39.122971+0000","last_clean":"2024-09-13T18:57:39.122971+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:57:39.122971+0000","last_undegraded":"2024-09-13T18:57:39.122971+0000","last_fullsized":"2024-09-13T18:57:39.122971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'17","last_scrub_stamp":"2024-09-13T18:57:39.122544+0000","last_deep_scrub":"43'17","last_deep_scrub_stamp":"2024-09-13T18:57:39.122544+0000","last_clean_scrub_stamp":"2024-09-13T18:57:39.122544+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:32.935831+0000","scrub_duration":0.0032146100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"43'10","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:45.307811+0000","last_change":"2024-09-13T18:57:45.307991+0000","last_active":"2024-09-13T18:57:45.307811+0000","last_peered":"2024-09-13T18:57:45.307811+0000","last_clean":"2024-09-13T18:57:45.307811+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:57:45.307811+0000","last_undegraded":"2024-09-13T18:57:45.307811+0000","last_fullsized":"2024-09-13T18:57:45.307811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:45.307762+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:45.307762+0000","last_clean_scrub_stamp":"2024-09-13T18:57:45.307762+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:12.874226+0000","scrub_duration":0.0028431099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"43'10","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.421715+0000","last_change":"2024-09-13T18:57:19.006871+0000","last_active":"2024-09-13T18:57:26.421715+0000","last_peered":"2024-09-13T18:57:26.421715+0000","last_clean":"2024-09-13T18:57:26.421715+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:57:26.421715+0000","last_undegraded":"2024-09-13T18:57:26.421715+0000","last_fullsized":"2024-09-13T18:57:26.421715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'7","last_scrub_stamp":"2024-09-13T18:57:19.006675+0000","last_deep_scrub":"29'7","last_deep_scrub_stamp":"2024-09-13T18:57:19.006675+0000","last_clean_scrub_stamp":"2024-09-13T18:57:19.006675+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:53.159267+0000","scrub_duration":0.012523802000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"43'10","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:43.268378+0000","last_change":"2024-09-13T18:57:43.268610+0000","last_active":"2024-09-13T18:57:43.268378+0000","last_peered":"2024-09-13T18:57:43.268378+0000","last_clean":"2024-09-13T18:57:43.268378+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:57:43.268378+0000","last_undegraded":"2024-09-13T18:57:43.268378+0000","last_fullsized":"2024-09-13T18:57:43.268378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:43.268329+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:43.268329+0000","last_clean_scrub_stamp":"2024-09-13T18:57:43.268329+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:56.169188+0000","scrub_duration":0.0028286909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"43'8","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.922074+0000","last_change":"2024-09-13T18:57:24.709839+0000","last_active":"2024-09-13T18:57:24.922074+0000","last_peered":"2024-09-13T18:57:24.922074+0000","last_clean":"2024-09-13T18:57:24.922074+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:57:24.922074+0000","last_undegraded":"2024-09-13T18:57:24.922074+0000","last_fullsized":"2024-09-13T18:57:24.922074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:24.709588+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:24.709588+0000","last_clean_scrub_stamp":"2024-09-13T18:57:24.709588+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:56.958314+0000","scrub_duration":0.0036738090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"43'13","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.862606+0000","last_change":"2024-09-13T18:57:17.997698+0000","last_active":"2024-09-13T18:57:24.862606+0000","last_peered":"2024-09-13T18:57:24.862606+0000","last_clean":"2024-09-13T18:57:24.862606+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:57:24.862606+0000","last_undegraded":"2024-09-13T18:57:24.862606+0000","last_fullsized":"2024-09-13T18:57:24.862606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:17.997340+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:17.997340+0000","last_clean_scrub_stamp":"2024-09-13T18:57:17.997340+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:07.106051+0000","scrub_duration":0.0035120389999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"43'10","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:36.044747+0000","last_change":"2024-09-13T18:57:36.044747+0000","last_active":"2024-09-13T18:57:36.044747+0000","last_peered":"2024-09-13T18:57:36.044747+0000","last_clean":"2024-09-13T18:57:36.044747+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:57:36.044747+0000","last_undegraded":"2024-09-13T18:57:36.044747+0000","last_fullsized":"2024-09-13T18:57:36.044747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:57:36.044597+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:57:36.044597+0000","last_clean_scrub_stamp":"2024-09-13T18:57:36.044597+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:40.677133+0000","scrub_duration":0.010560052,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"43'15","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.892290+0000","last_change":"2024-09-13T18:57:38.892530+0000","last_active":"2024-09-13T18:57:38.892290+0000","last_peered":"2024-09-13T18:57:38.892290+0000","last_clean":"2024-09-13T18:57:38.892290+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:57:38.892290+0000","last_undegraded":"2024-09-13T18:57:38.892290+0000","last_fullsized":"2024-09-13T18:57:38.892290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'15","last_scrub_stamp":"2024-09-13T18:57:38.892242+0000","last_deep_scrub":"43'15","last_deep_scrub_stamp":"2024-09-13T18:57:38.892242+0000","last_clean_scrub_stamp":"2024-09-13T18:57:38.892242+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:17.674671+0000","scrub_duration":0.0037659220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"43'11","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.946397+0000","last_change":"2024-09-13T18:26:32.358640+0000","last_active":"2024-09-13T18:57:24.946397+0000","last_peered":"2024-09-13T18:57:24.946397+0000","last_clean":"2024-09-13T18:57:24.946397+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:57:24.946397+0000","last_undegraded":"2024-09-13T18:57:24.946397+0000","last_fullsized":"2024-09-13T18:57:24.946397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"43'18","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:47.274871+0000","last_change":"2024-09-13T18:57:47.275068+0000","last_active":"2024-09-13T18:57:47.274871+0000","last_peered":"2024-09-13T18:57:47.274871+0000","last_clean":"2024-09-13T18:57:47.274871+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:57:47.274871+0000","last_undegraded":"2024-09-13T18:57:47.274871+0000","last_fullsized":"2024-09-13T18:57:47.274871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'18","last_scrub_stamp":"2024-09-13T18:57:47.274823+0000","last_deep_scrub":"43'18","last_deep_scrub_stamp":"2024-09-13T18:57:47.274823+0000","last_clean_scrub_stamp":"2024-09-13T18:57:47.274823+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:23:23.332126+0000","scrub_duration":0.0026113270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"43'11","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:51.484385+0000","last_change":"2024-09-13T18:57:51.484615+0000","last_active":"2024-09-13T18:57:51.484385+0000","last_peered":"2024-09-13T18:57:51.484385+0000","last_clean":"2024-09-13T18:57:51.484385+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:57:51.484385+0000","last_undegraded":"2024-09-13T18:57:51.484385+0000","last_fullsized":"2024-09-13T18:57:51.484385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:51.484333+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:51.484333+0000","last_clean_scrub_stamp":"2024-09-13T18:57:51.484333+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:35.675512+0000","scrub_duration":0.0032285579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"43'15","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.397740+0000","last_change":"2024-09-13T18:57:15.857341+0000","last_active":"2024-09-13T18:57:28.397740+0000","last_peered":"2024-09-13T18:57:28.397740+0000","last_clean":"2024-09-13T18:57:28.397740+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:57:28.397740+0000","last_undegraded":"2024-09-13T18:57:28.397740+0000","last_fullsized":"2024-09-13T18:57:28.397740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:15.857084+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:15.857084+0000","last_clean_scrub_stamp":"2024-09-13T18:57:15.857084+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:39.110251+0000","scrub_duration":0.0034250190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"43'18","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.034005+0000","last_change":"2024-09-13T18:57:27.062756+0000","last_active":"2024-09-13T18:57:28.034005+0000","last_peered":"2024-09-13T18:57:28.034005+0000","last_clean":"2024-09-13T18:57:28.034005+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:57:28.034005+0000","last_undegraded":"2024-09-13T18:57:28.034005+0000","last_fullsized":"2024-09-13T18:57:28.034005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'15","last_scrub_stamp":"2024-09-13T18:57:27.062495+0000","last_deep_scrub":"43'15","last_deep_scrub_stamp":"2024-09-13T18:57:27.062495+0000","last_clean_scrub_stamp":"2024-09-13T18:57:27.062495+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:05:17.012914+0000","scrub_duration":0.0040785700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"43'14","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:48.240470+0000","last_change":"2024-09-13T18:57:48.240666+0000","last_active":"2024-09-13T18:57:48.240470+0000","last_peered":"2024-09-13T18:57:48.240470+0000","last_clean":"2024-09-13T18:57:48.240470+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:57:48.240470+0000","last_undegraded":"2024-09-13T18:57:48.240470+0000","last_fullsized":"2024-09-13T18:57:48.240470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:48.240421+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:48.240421+0000","last_clean_scrub_stamp":"2024-09-13T18:57:48.240421+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:11.977729+0000","scrub_duration":0.0030695620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"43'12","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:43.177441+0000","last_change":"2024-09-13T18:57:43.177686+0000","last_active":"2024-09-13T18:57:43.177441+0000","last_peered":"2024-09-13T18:57:43.177441+0000","last_clean":"2024-09-13T18:57:43.177441+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:57:43.177441+0000","last_undegraded":"2024-09-13T18:57:43.177441+0000","last_fullsized":"2024-09-13T18:57:43.177441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:57:43.177387+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:57:43.177387+0000","last_clean_scrub_stamp":"2024-09-13T18:57:43.177387+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:30.629006+0000","scrub_duration":0.0029135400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"43'16","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:37.308644+0000","last_change":"2024-09-13T18:57:37.308764+0000","last_active":"2024-09-13T18:57:37.308644+0000","last_peered":"2024-09-13T18:57:37.308644+0000","last_clean":"2024-09-13T18:57:37.308644+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:57:37.308644+0000","last_undegraded":"2024-09-13T18:57:37.308644+0000","last_fullsized":"2024-09-13T18:57:37.308644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:37.308610+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:37.308610+0000","last_clean_scrub_stamp":"2024-09-13T18:57:37.308610+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:24.954579+0000","scrub_duration":0.020444231,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"43'17","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:34.387125+0000","last_change":"2024-09-13T18:57:34.387338+0000","last_active":"2024-09-13T18:57:34.387125+0000","last_peered":"2024-09-13T18:57:34.387125+0000","last_clean":"2024-09-13T18:57:34.387125+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:57:34.387125+0000","last_undegraded":"2024-09-13T18:57:34.387125+0000","last_fullsized":"2024-09-13T18:57:34.387125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'17","last_scrub_stamp":"2024-09-13T18:57:34.387073+0000","last_deep_scrub":"43'17","last_deep_scrub_stamp":"2024-09-13T18:57:34.387073+0000","last_clean_scrub_stamp":"2024-09-13T18:57:34.387073+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:16.405083+0000","scrub_duration":0.002417584,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"43'13","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:44.875718+0000","last_change":"2024-09-13T18:57:44.875900+0000","last_active":"2024-09-13T18:57:44.875718+0000","last_peered":"2024-09-13T18:57:44.875718+0000","last_clean":"2024-09-13T18:57:44.875718+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:57:44.875718+0000","last_undegraded":"2024-09-13T18:57:44.875718+0000","last_fullsized":"2024-09-13T18:57:44.875718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:44.875664+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:44.875664+0000","last_clean_scrub_stamp":"2024-09-13T18:57:44.875664+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:19.462107+0000","scrub_duration":0.0036327349999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"43'12","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.070681+0000","last_change":"2024-09-13T18:57:25.015223+0000","last_active":"2024-09-13T18:57:28.070681+0000","last_peered":"2024-09-13T18:57:28.070681+0000","last_clean":"2024-09-13T18:57:28.070681+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:57:28.070681+0000","last_undegraded":"2024-09-13T18:57:28.070681+0000","last_fullsized":"2024-09-13T18:57:28.070681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:25.015045+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:25.015045+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.015045+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:39.559328+0000","scrub_duration":0.013113774999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"43'13","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.778813+0000","last_change":"2024-09-13T18:57:30.778903+0000","last_active":"2024-09-13T18:57:30.778813+0000","last_peered":"2024-09-13T18:57:30.778813+0000","last_clean":"2024-09-13T18:57:30.778813+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:57:30.778813+0000","last_undegraded":"2024-09-13T18:57:30.778813+0000","last_fullsized":"2024-09-13T18:57:30.778813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:30.778791+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:30.778791+0000","last_clean_scrub_stamp":"2024-09-13T18:57:30.778791+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:34.242313+0000","scrub_duration":0.00175041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"43'13","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.422475+0000","last_change":"2024-09-13T18:26:32.357980+0000","last_active":"2024-09-13T18:57:26.422475+0000","last_peered":"2024-09-13T18:57:26.422475+0000","last_clean":"2024-09-13T18:57:26.422475+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:57:26.422475+0000","last_undegraded":"2024-09-13T18:57:26.422475+0000","last_fullsized":"2024-09-13T18:57:26.422475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"43'16","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:12.422623+0000","last_change":"2024-09-13T18:26:32.365638+0000","last_active":"2024-09-13T18:57:12.422623+0000","last_peered":"2024-09-13T18:57:12.422623+0000","last_clean":"2024-09-13T18:57:12.422623+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:57:12.422623+0000","last_undegraded":"2024-09-13T18:57:12.422623+0000","last_fullsized":"2024-09-13T18:57:12.422623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"43'6","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.956696+0000","last_change":"2024-09-13T18:26:32.345496+0000","last_active":"2024-09-13T18:57:24.956696+0000","last_peered":"2024-09-13T18:57:24.956696+0000","last_clean":"2024-09-13T18:57:24.956696+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:57:24.956696+0000","last_undegraded":"2024-09-13T18:57:24.956696+0000","last_fullsized":"2024-09-13T18:57:24.956696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"43'9","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.097406+0000","last_change":"2024-09-13T18:26:32.356901+0000","last_active":"2024-09-13T18:57:28.097406+0000","last_peered":"2024-09-13T18:57:28.097406+0000","last_clean":"2024-09-13T18:57:28.097406+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:57:28.097406+0000","last_undegraded":"2024-09-13T18:57:28.097406+0000","last_fullsized":"2024-09-13T18:57:28.097406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"43'11","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:11.172752+0000","last_change":"2024-09-13T18:26:32.355203+0000","last_active":"2024-09-13T18:57:11.172752+0000","last_peered":"2024-09-13T18:57:11.172752+0000","last_clean":"2024-09-13T18:57:11.172752+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:57:11.172752+0000","last_undegraded":"2024-09-13T18:57:11.172752+0000","last_fullsized":"2024-09-13T18:57:11.172752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"43'13","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.393485+0000","last_change":"2024-09-13T18:26:32.358190+0000","last_active":"2024-09-13T18:57:27.393485+0000","last_peered":"2024-09-13T18:57:27.393485+0000","last_clean":"2024-09-13T18:57:27.393485+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:57:27.393485+0000","last_undegraded":"2024-09-13T18:57:27.393485+0000","last_fullsized":"2024-09-13T18:57:27.393485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"43'7","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.411788+0000","last_change":"2024-09-13T18:26:32.355121+0000","last_active":"2024-09-13T18:57:28.411788+0000","last_peered":"2024-09-13T18:57:28.411788+0000","last_clean":"2024-09-13T18:57:28.411788+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:57:28.411788+0000","last_undegraded":"2024-09-13T18:57:28.411788+0000","last_fullsized":"2024-09-13T18:57:28.411788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"43'11","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.862267+0000","last_change":"2024-09-13T18:57:11.323912+0000","last_active":"2024-09-13T18:57:24.862267+0000","last_peered":"2024-09-13T18:57:24.862267+0000","last_clean":"2024-09-13T18:57:24.862267+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:57:24.862267+0000","last_undegraded":"2024-09-13T18:57:24.862267+0000","last_fullsized":"2024-09-13T18:57:24.862267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:11.323767+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:11.323767+0000","last_clean_scrub_stamp":"2024-09-13T18:57:11.323767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:26.980778+0000","scrub_duration":0.0019515800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"43'6","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.252568+0000","last_change":"2024-09-13T18:57:39.252568+0000","last_active":"2024-09-13T18:57:39.252568+0000","last_peered":"2024-09-13T18:57:39.252568+0000","last_clean":"2024-09-13T18:57:39.252568+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:57:39.252568+0000","last_undegraded":"2024-09-13T18:57:39.252568+0000","last_fullsized":"2024-09-13T18:57:39.252568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:39.252320+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:39.252320+0000","last_clean_scrub_stamp":"2024-09-13T18:57:39.252320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:39.213931+0000","scrub_duration":0.002145554,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"43'9","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.021682+0000","last_change":"2024-09-13T18:26:32.362196+0000","last_active":"2024-09-13T18:57:28.021682+0000","last_peered":"2024-09-13T18:57:28.021682+0000","last_clean":"2024-09-13T18:57:28.021682+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:57:28.021682+0000","last_undegraded":"2024-09-13T18:57:28.021682+0000","last_fullsized":"2024-09-13T18:57:28.021682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":797,"num_read_kb":0,"num_write":803600,"num_write_kb":16319459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178039,"ondisk_log_size":178039,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":771,"ondisk_log_size":771,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":16507087,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1362,"num_read_kb":52080,"num_write":3742,"num_write_kb":17667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22716416,"data_stored":33014174,"data_compressed":2441304,"data_compressed_allocated":3506176,"data_compressed_original":13950976,"omap_allocated":0,"internal_metadata":0},"log_size":3641,"ondisk_log_size":3641,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509820,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1055268,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":1053054,"kb_avail":93316572,"statfs":{"total":96636764160,"available":95556169728,"internally_reserved":0,"allocated":2256896,"data_stored":4436700,"data_compressed":519875,"data_compressed_allocated":692224,"data_compressed_original":3125248,"omap_allocated":1593,"internal_metadata":1078327751},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":1.3600000000000001,"15min":0.93999999999999995},"min":{"1min":0.247,"5min":0.223,"15min":0.223},"max":{"1min":0.84199999999999997,"5min":10.43,"15min":10.43},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.107,"5min":1.889,"15min":1.1599999999999999},"min":{"1min":0.27100000000000002,"5min":0.245,"15min":0.245},"max":{"1min":4.3109999999999999,"5min":16.600999999999999,"15min":16.600999999999999},"last":0.61899999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":3.5910000000000002,"15min":1.849},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":5.8780000000000001,"5min":52.290999999999997,"15min":52.290999999999997},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.2769999999999999,"5min":3.7690000000000001,"15min":1.9570000000000001},"min":{"1min":0.28199999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":5.4740000000000002,"5min":52.149000000000001,"15min":52.149000000000001},"last":1.0209999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":1.335,"15min":1.054},"min":{"1min":0.316,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.9399999999999999,"5min":14.933,"15min":14.933},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.264,"15min":1.052},"min":{"1min":0.35299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":7.266,"5min":9.2059999999999995,"15min":17.792000000000002},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":2.3849999999999998,"15min":1.4319999999999999},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":8.7910000000000004,"5min":15.622,"15min":15.622},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.343,"5min":2.71,"15min":1.7170000000000001},"min":{"1min":0.374,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.1710000000000003,"5min":15.423,"15min":28.248000000000001},"last":1.115}]},{"osd":4,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":1.2090000000000001,"15min":1.073},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4219999999999997,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.242,"15min":1.054},"min":{"1min":0.34599999999999997,"5min":0.32900000000000001,"15min":0.23699999999999999},"max":{"1min":4.4450000000000003,"5min":7.9409999999999998,"15min":7.9409999999999998},"last":1.2}]},{"osd":5,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.92000000000000004,"15min":0.97499999999999998},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.28000000000000003},"max":{"1min":4.3879999999999999,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.92300000000000004,"15min":0.97399999999999998},"min":{"1min":0.34599999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.143,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":0.997}]},{"osd":7,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.0640000000000001,"15min":1.054},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.8149999999999999,"5min":12.021000000000001,"15min":12.021000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.01,"15min":0.998},"min":{"1min":0.34999999999999998,"5min":0.32500000000000001,"15min":0.312},"max":{"1min":4.3600000000000003,"5min":9.0440000000000005,"15min":9.0440000000000005},"last":1.1519999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509820,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":944040,"kb_used_data":9760,"kb_used_omap":1,"kb_used_meta":934270,"kb_avail":93427800,"statfs":{"total":96636764160,"available":95670067200,"internally_reserved":0,"allocated":9994240,"data_stored":13045326,"data_compressed":732085,"data_compressed_allocated":1101824,"data_compressed_original":4415488,"omap_allocated":1589,"internal_metadata":956692939},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.956,"15min":1.24},"min":{"1min":0.253,"5min":0.217,"15min":0.217},"max":{"1min":8.7949999999999999,"5min":24.414000000000001,"15min":24.414000000000001},"last":1.056},{"interface":"front","average":{"1min":1.3200000000000001,"5min":2.089,"15min":1.383},"min":{"1min":0.34000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":8.8480000000000008,"5min":24.469000000000001,"15min":24.469000000000001},"last":0.85799999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":3.1869999999999998,"15min":1.665},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":7.3250000000000002,"5min":46.161999999999999,"15min":46.161999999999999},"last":1.139},{"interface":"front","average":{"1min":1.333,"5min":3.9089999999999998,"15min":2.0459999999999998},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":7.548,"5min":45.713000000000001,"15min":45.713000000000001},"last":0.76200000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.208,"15min":0.98399999999999999},"min":{"1min":0.35499999999999998,"5min":0.29699999999999999,"15min":0.28999999999999998},"max":{"1min":3.4279999999999999,"5min":7.4740000000000002,"15min":7.4740000000000002},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.3340000000000001,"15min":0.99199999999999999},"min":{"1min":0.27800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.4699999999999998,"5min":9.1679999999999993,"15min":9.1679999999999993},"last":0.94299999999999995}]},{"osd":3,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.8169999999999999,"5min":3.0920000000000001,"15min":1.869},"min":{"1min":0.35399999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.8140000000000001,"5min":35.494999999999997,"15min":35.494999999999997},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.212,"5min":2.5289999999999999,"15min":1.7030000000000001},"min":{"1min":0.33600000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":5.0880000000000001,"5min":19.946000000000002,"15min":19.946000000000002},"last":1.194}]},{"osd":4,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.9690000000000001,"5min":1.198,"15min":0.98799999999999999},"min":{"1min":0.36499999999999999,"5min":0.249,"15min":0.249},"max":{"1min":10.477,"5min":10.477,"15min":10.477},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.2629999999999999,"15min":1.0980000000000001},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":10.499000000000001,"5min":10.499000000000001,"15min":10.499000000000001},"last":1.3109999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.0289999999999999,"15min":0.92300000000000004},"min":{"1min":0.30199999999999999,"5min":0.26600000000000001,"15min":0.26100000000000001},"max":{"1min":14.954000000000001,"5min":14.954000000000001,"15min":14.954000000000001},"last":0.997},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.0369999999999999,"15min":0.99399999999999999},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":9.0180000000000007,"5min":9.0180000000000007,"15min":9.0180000000000007},"last":1.109}]},{"osd":6,"last update":"Fri Sep 13 18:57:44 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.415,"15min":1.1639999999999999},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":7.383,"5min":32.491,"15min":32.491},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.786,"5min":1.3240000000000001,"15min":1.141},"min":{"1min":0.44700000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":8.8770000000000007,"5min":27.291,"15min":27.291},"last":1.2310000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509819,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":981284,"kb_used_data":4508,"kb_used_omap":1,"kb_used_meta":976766,"kb_avail":93390556,"statfs":{"total":96636764160,"available":95631929344,"internally_reserved":0,"allocated":4616192,"data_stored":4436064,"data_compressed":20605,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":1594,"internal_metadata":1000208838},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":2.0270000000000001,"15min":1.238},"min":{"1min":0.26300000000000001,"5min":0.24099999999999999,"15min":0.223},"max":{"1min":4.6299999999999999,"5min":28.593,"15min":28.593},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.7889999999999999,"5min":1.9890000000000001,"15min":1.256},"min":{"1min":0.33000000000000002,"5min":0.255,"15min":0.255},"max":{"1min":10.644,"5min":28.559000000000001,"15min":28.559000000000001},"last":0.85399999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":3.7410000000000001,"5min":3.1200000000000001,"15min":1.6899999999999999},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":29.113,"5min":37.100999999999999,"15min":37.100999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":2.6909999999999998,"5min":3.468,"15min":1.8180000000000001},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":12.385999999999999,"5min":44.798000000000002,"15min":44.798000000000002},"last":0.74099999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.792,"15min":1.2130000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":10.506,"5min":27.634,"15min":27.634},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.7250000000000001,"15min":1.216},"min":{"1min":0.34200000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.4029999999999996,"5min":27.606000000000002,"15min":27.606000000000002},"last":1.0960000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.641,"5min":3.1930000000000001,"15min":1.708},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":10.535,"5min":40.676000000000002,"15min":40.676000000000002},"last":0.79700000000000004},{"interface":"front","average":{"1min":2.1560000000000001,"5min":3.6699999999999999,"15min":1.97},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":10.548999999999999,"5min":40.881999999999998,"15min":40.881999999999998},"last":1.014}]},{"osd":5,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.93700000000000006,"15min":0.89200000000000002},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":4.9889999999999999,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":0.93999999999999995,"15min":0.98899999999999999},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.0170000000000003,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":1.171}]},{"osd":6,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2490000000000001,"15min":1.103},"min":{"1min":0.29799999999999999,"5min":0.251,"15min":0.251},"max":{"1min":8.9039999999999999,"5min":9.4570000000000007,"15min":9.4570000000000007},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.002,"15min":1.014},"min":{"1min":0.38600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":3.8559999999999999,"5min":5.1059999999999999,"15min":6.1319999999999997},"last":1.3049999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.464,"15min":1.1930000000000001},"min":{"1min":0.38600000000000001,"5min":0.307,"15min":0.307},"max":{"1min":6.1210000000000004,"5min":10.101000000000001,"15min":10.101000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.5269999999999999,"15min":1.286},"min":{"1min":0.41199999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.1509999999999998,"5min":10.127000000000001,"15min":10.127000000000001},"last":1.2749999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542523,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1272936,"kb_used_data":5472,"kb_used_omap":1,"kb_used_meta":1267454,"kb_avail":93098904,"statfs":{"total":96636764160,"available":95333277696,"internally_reserved":0,"allocated":5603328,"data_stored":8725194,"data_compressed":790472,"data_compressed_allocated":1138688,"data_compressed_original":4517888,"omap_allocated":1591,"internal_metadata":1297873353},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:32 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":2.9300000000000002,"15min":1.5609999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":10.571999999999999,"5min":22.670000000000002,"15min":22.670000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":1.266,"5min":2.9980000000000002,"15min":1.716},"min":{"1min":0.23000000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":10.512,"5min":22.728000000000002,"15min":22.728000000000002},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":4.0330000000000004,"5min":5.657,"15min":2.7250000000000001},"min":{"1min":0.20499999999999999,"5min":0.191,"15min":0.191},"max":{"1min":19.170999999999999,"5min":42.046999999999997,"15min":42.046999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":3.5009999999999999,"5min":5.2409999999999997,"15min":2.7490000000000001},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":19.199999999999999,"5min":42.024999999999999,"15min":42.024999999999999},"last":0.85299999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":3.5950000000000002,"5min":3.734,"15min":2.113},"min":{"1min":0.29399999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":19.561,"5min":41.909999999999997,"15min":41.909999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.95399999999999996,"5min":2.7280000000000002,"15min":1.544},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.7570000000000001,"5min":27.25,"15min":27.25},"last":0.624}]},{"osd":4,"last update":"Fri Sep 13 18:57:32 2024","interfaces":[{"interface":"back","average":{"1min":2.52,"5min":3.1509999999999998,"15min":1.8520000000000001},"min":{"1min":0.45000000000000001,"5min":0.30099999999999999,"15min":0.26300000000000001},"max":{"1min":19.533999999999999,"5min":42.003,"15min":42.003},"last":0.75900000000000001},{"interface":"front","average":{"1min":2.9780000000000002,"5min":3.46,"15min":2.0950000000000002},"min":{"1min":0.41699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":19.728999999999999,"5min":41.951000000000001,"15min":41.951000000000001},"last":0.80300000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":2.3340000000000001,"15min":1.5489999999999999},"min":{"1min":0.29699999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":6.5679999999999996,"5min":22.988,"15min":22.988},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.155,"5min":2.0670000000000002,"15min":1.474},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.7330000000000001,"5min":23.030000000000001,"15min":23.030000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":2.5870000000000002,"15min":1.6859999999999999},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":3.6080000000000001,"5min":23.087,"15min":23.087},"last":0.90300000000000002},{"interface":"front","average":{"1min":2.1190000000000002,"5min":3.2130000000000001,"15min":1.9159999999999999},"min":{"1min":0.39800000000000002,"5min":0.317,"15min":0.317},"max":{"1min":14.753,"5min":23.425000000000001,"15min":23.425000000000001},"last":0.91900000000000004}]},{"osd":7,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":3.4409999999999998,"5min":3.1819999999999999,"15min":1.8819999999999999},"min":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":19.588000000000001,"5min":41.975999999999999,"15min":41.975999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.956,"5min":2.7709999999999999,"15min":1.728},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34200000000000003},"max":{"1min":14.686999999999999,"5min":23.457999999999998,"15min":23.457999999999998},"last":1.0189999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542527,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746120,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":745534,"kb_avail":93625720,"statfs":{"total":96636764160,"available":95872737280,"internally_reserved":0,"allocated":589824,"data_stored":334673,"data_compressed":1568,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":763427271},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":2.5110000000000001,"15min":1.278},"min":{"1min":0.19600000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":4.7599999999999998,"5min":37.600000000000001,"15min":37.600000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":1.3540000000000001,"5min":2.5030000000000001,"15min":1.3149999999999999},"min":{"1min":0.20499999999999999,"5min":0.19,"15min":0.19},"max":{"1min":7.3949999999999996,"5min":37.651000000000003,"15min":37.651000000000003},"last":0.69699999999999995}]},{"osd":1,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":3.4319999999999999,"5min":2.8719999999999999,"15min":1.7030000000000001},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":17.443999999999999,"5min":17.443999999999999,"15min":18.055},"last":0.92400000000000004},{"interface":"front","average":{"1min":2.8220000000000001,"5min":2.6099999999999999,"15min":1.5800000000000001},"min":{"1min":0.28399999999999997,"5min":0.20000000000000001,"15min":0.188},"max":{"1min":15.132,"5min":15.94,"15min":18.175999999999998},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":3.3039999999999998,"15min":1.806},"min":{"1min":0.29099999999999998,"5min":0.249,"15min":0.249},"max":{"1min":17.613,"5min":25.832999999999998,"15min":25.832999999999998},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.627,"5min":3.2429999999999999,"15min":1.8220000000000001},"min":{"1min":0.35599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":17.59,"5min":17.59,"15min":17.59},"last":0.81100000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.1910000000000001,"15min":1.0309999999999999},"min":{"1min":0.32600000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.3559999999999999,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.149,"15min":1.0940000000000001},"min":{"1min":0.35999999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.6980000000000004,"5min":8.0999999999999996,"15min":8.0999999999999996},"last":1.0269999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2669999999999999,"15min":1.0580000000000001},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29499999999999998},"max":{"1min":5.8600000000000003,"5min":8.4800000000000004,"15min":8.4800000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4470000000000001,"15min":1.1539999999999999},"min":{"1min":0.311,"5min":0.309,"15min":0.309},"max":{"1min":8.9949999999999992,"5min":8.9949999999999992,"15min":8.9949999999999992},"last":0.875}]},{"osd":6,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.2789999999999999,"15min":1.167},"min":{"1min":0.436,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.7370000000000001,"5min":9.6720000000000006,"15min":9.6720000000000006},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.1599999999999999,"15min":1.0800000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":4.5129999999999999,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":0.90300000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4039999999999999,"15min":1.169},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.3179999999999996,"5min":8.9250000000000007,"15min":8.9250000000000007},"last":0.755},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.5029999999999999,"15min":1.2589999999999999},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":7.7569999999999997,"5min":9.2330000000000005,"15min":9.2330000000000005},"last":1.161}]}]},{"osd":5,"up_from":13,"seq":55834575228,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":726576,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":726078,"kb_avail":93645264,"statfs":{"total":96636764160,"available":95892750336,"internally_reserved":0,"allocated":499712,"data_stored":241392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":743504327},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":1.7170000000000001,"5min":1.8320000000000001,"15min":1.175},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":8.8219999999999992,"5min":9.952,"15min":9.952},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.796,"5min":1.948,"15min":1.3},"min":{"1min":0.29599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.8740000000000006,"5min":14.317,"15min":14.317},"last":1.1399999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.931,"15min":1.3240000000000001},"min":{"1min":0.32500000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":6.4930000000000003,"5min":10.85,"15min":10.85},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.917,"15min":1.3240000000000001},"min":{"1min":0.26000000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":3.113,"5min":10.868,"15min":10.868},"last":0.82899999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":2.641,"5min":2.222,"15min":1.3},"min":{"1min":0.30299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":9.9499999999999993,"5min":38.484000000000002,"15min":38.484000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":2.0259999999999998,"5min":2.3730000000000002,"15min":1.365},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":8.0800000000000001,"5min":40.686,"15min":40.686},"last":0.92400000000000004}]},{"osd":3,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":2.5819999999999999,"5min":4.0430000000000001,"15min":2.0880000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":14.885999999999999,"5min":72.534999999999997,"15min":72.534999999999997},"last":0.74099999999999999},{"interface":"front","average":{"1min":2.8029999999999999,"5min":3.9220000000000002,"15min":2.101},"min":{"1min":0.30499999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":14.91,"5min":72.486999999999995,"15min":72.486999999999995},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.93100000000000005,"15min":0.90800000000000003},"min":{"1min":0.28799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1970000000000001,"5min":7.8710000000000004,"15min":7.8710000000000004},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.1240000000000001,"15min":1.071},"min":{"1min":0.36099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.8399999999999999,"5min":8.7829999999999995,"15min":8.7829999999999995},"last":1.0049999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.97499999999999998,"15min":0.999},"min":{"1min":0.32400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.5179999999999998,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.095,"5min":1.0569999999999999,"15min":1.0509999999999999},"min":{"1min":0.38500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.54,"5min":4.516,"15min":4.516},"last":0.89900000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:57:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.1919999999999999,"15min":1.0640000000000001},"min":{"1min":0.46400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.5780000000000003,"5min":7.6020000000000003,"15min":7.6020000000000003},"last":1.208},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.179,"15min":1.139},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.9699999999999998,"5min":7.9109999999999996,"15min":7.9109999999999996},"last":1.268}]}]},{"osd":1,"up_from":13,"seq":55834575226,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1223156,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":1222654,"kb_avail":93148684,"statfs":{"total":96636764160,"available":95384252416,"internally_reserved":0,"allocated":503808,"data_stored":240958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1251998150},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":2.0539999999999998,"5min":2.7050000000000001,"15min":1.5920000000000001},"min":{"1min":0.214,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":10.279,"5min":28.297000000000001,"15min":28.297000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":2.464,"5min":2.7149999999999999,"15min":1.466},"min":{"1min":0.32400000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":10.162000000000001,"5min":19.785,"15min":19.785},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":2.5609999999999999,"15min":1.4359999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.17799999999999999},"max":{"1min":7.2380000000000004,"5min":25.609999999999999,"15min":25.609999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.905,"5min":2.891,"15min":1.619},"min":{"1min":0.29299999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":10.465,"5min":30.469999999999999,"15min":30.469999999999999},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":4.1040000000000001,"5min":5.694,"15min":2.6179999999999999},"min":{"1min":0.48499999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":18.695,"5min":36.643999999999998,"15min":36.643999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":4.2220000000000004,"5min":5.8979999999999997,"15min":2.754},"min":{"1min":0.53800000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":18.760000000000002,"5min":36.670000000000002,"15min":36.670000000000002},"last":0.93300000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:57:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":1.4399999999999999,"15min":1.1779999999999999},"min":{"1min":0.28399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.7110000000000003,"5min":7.4669999999999996,"15min":7.6799999999999997},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.766,"15min":1.339},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":4.508,"5min":14.906000000000001,"15min":14.906000000000001},"last":0.85399999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.986,"15min":1.3700000000000001},"min":{"1min":0.38800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.259,"5min":10.259,"15min":10.259},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.6910000000000001,"5min":2.0289999999999999,"15min":1.3799999999999999},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34100000000000003},"max":{"1min":10.231999999999999,"5min":12.27,"15min":12.27},"last":1.0149999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":2.093,"5min":2.4220000000000002,"15min":1.5449999999999999},"min":{"1min":0.432,"5min":0.371,"15min":0.316},"max":{"1min":10.538,"5min":28.981999999999999,"15min":28.981999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.8700000000000001,"15min":1.429},"min":{"1min":0.36599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":10.342000000000001,"5min":15.727,"15min":15.727},"last":1.004}]},{"osd":7,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":2.0630000000000002,"15min":1.466},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":7.2130000000000001,"5min":28.100000000000001,"15min":28.100000000000001},"last":1.103},{"interface":"front","average":{"1min":1.2889999999999999,"5min":2.0710000000000002,"15min":1.5389999999999999},"min":{"1min":0.41099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":7.173,"5min":28.896999999999998,"15min":28.896999999999998},"last":1.147}]}]},{"osd":0,"up_from":13,"seq":55834575227,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":842628,"kb_used_data":2556,"kb_used_omap":1,"kb_used_meta":840062,"kb_avail":93529212,"statfs":{"total":96636764160,"available":95773913088,"internally_reserved":0,"allocated":2617344,"data_stored":4654715,"data_compressed":442451,"data_compressed_allocated":618496,"data_compressed_original":2916352,"omap_allocated":1594,"internal_metadata":860223942},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.8009999999999999,"5min":3.573,"15min":1.8169999999999999},"min":{"1min":0.24399999999999999,"5min":0.183,"15min":0.183},"max":{"1min":10.925000000000001,"5min":29.712,"15min":29.712},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.877,"5min":2.6520000000000001,"15min":1.5640000000000001},"min":{"1min":0.29299999999999998,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":10.975,"5min":29.748000000000001,"15min":29.748000000000001},"last":0.72099999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.6519999999999999,"15min":1.0509999999999999},"min":{"1min":0.218,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":8.5660000000000007,"5min":17.611000000000001,"15min":17.611000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.113,"5min":1.867,"15min":1.149},"min":{"1min":0.22700000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":9.2560000000000002,"5min":17.501999999999999,"15min":17.501999999999999},"last":0.628}]},{"osd":3,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":3.2879999999999998,"15min":1.6659999999999999},"min":{"1min":0.30099999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":7.9340000000000002,"5min":17.260000000000002,"15min":17.260000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.149,"5min":2.6269999999999998,"15min":1.536},"min":{"1min":0.27400000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":5.8170000000000002,"5min":17.088000000000001,"15min":17.088000000000001},"last":0.67600000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":2.286,"15min":1.4219999999999999},"min":{"1min":0.376,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":8.0380000000000003,"5min":40.101999999999997,"15min":40.101999999999997},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.0229999999999999,"5min":2.0600000000000001,"15min":1.3069999999999999},"min":{"1min":0.40100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":5.9009999999999998,"5min":40.981000000000002,"15min":40.981000000000002},"last":0.83699999999999997}]},{"osd":5,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.8120000000000001,"15min":1.246},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.8900000000000001,"5min":40.701999999999998,"15min":40.701999999999998},"last":1.024},{"interface":"front","average":{"1min":1.0800000000000001,"5min":2.0499999999999998,"15min":1.3520000000000001},"min":{"1min":0.434,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.8559999999999999,"5min":40.130000000000003,"15min":40.130000000000003},"last":0.89500000000000002}]},{"osd":6,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":2.1930000000000001,"15min":1.4159999999999999},"min":{"1min":0.40200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":6.1440000000000001,"5min":51.719000000000001,"15min":51.719000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.4019999999999999,"5min":2.2559999999999998,"15min":1.4710000000000001},"min":{"1min":0.45500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":7.9660000000000002,"5min":51.75,"15min":51.75},"last":0.92600000000000005}]},{"osd":7,"last update":"Fri Sep 13 18:57:34 2024","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":2.4119999999999999,"15min":1.522},"min":{"1min":0.44400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":8.0120000000000005,"5min":58.810000000000002,"15min":58.810000000000002},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.6299999999999999,"5min":2.2389999999999999,"15min":1.4770000000000001},"min":{"1min":0.54100000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.508,"5min":39.622,"15min":39.622},"last":0.88400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2072576,"data_stored":3824333,"data_compressed":409575,"data_compressed_allocated":573440,"data_compressed_original":2342912,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":94659,"data_compressed":1568,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5124096,"data_stored":8485180,"data_compressed":790472,"data_compressed_allocated":1138688,"data_compressed_original":4517888,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4136960,"data_stored":4196050,"data_compressed":20605,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1777664,"data_stored":4196686,"data_compressed":519875,"data_compressed_allocated":692224,"data_compressed_original":3125248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9449472,"data_stored":12214944,"data_compressed":699209,"data_compressed_allocated":1056768,"data_compressed_original":3842048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:57:54.022 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.022 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.023 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.023 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.023 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.023 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.023 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.023 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.023 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.024 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.024 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.024 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.024 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.024 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.024 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.024 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.024 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.025 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.025 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.025 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.025 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.025 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.025 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.025 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.025 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.026 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.026 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.026 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.026 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.026 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.026 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.026 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.027 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.027 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.027 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.027 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.027 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.027 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.027 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.027 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:57:54.028 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T18:58:14.029 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:58:14.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.133+0000 7f4ad32a9700 1 Processor -- start 2024-09-13T18:58:14.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.133+0000 7f4ad32a9700 1 -- start start 2024-09-13T18:58:14.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.133+0000 7f4ad32a9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ac40dcd70 0x7f4ac4009260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:14.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.133+0000 7f4ad32a9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ac40e6c00 0x7f4ac4051ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:14.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.133+0000 7f4ad32a9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac404e800 con 0x7f4ac404e3e0 2024-09-13T18:58:14.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.133+0000 7f4ad32a9700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac40e7030 con 0x7f4ac40e6c00 2024-09-13T18:58:14.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.133+0000 7f4ad32a9700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac404dfb0 con 0x7f4ac40dcd70 2024-09-13T18:58:14.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.133+0000 7f4ad2aa8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ac40e6c00 0x7f4ac4051ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:14.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.133+0000 7f4ad2aa8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ac40e6c00 0x7f4ac4051ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38384/0 (socket says 172.21.15.146:38384) 2024-09-13T18:58:14.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.133+0000 7f4ad2aa8700 1 -- 172.21.15.146:0/2568734273 learned_addr learned my addr 172.21.15.146:0/2568734273 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:14.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.133+0000 7f4ad2aa8700 1 -- 172.21.15.146:0/2568734273 >> v2:172.21.15.154:3300/0 conn(0x7f4ac40dcd70 msgr2=0x7f4ac4009260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:14.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.133+0000 7f4ad22a7700 1 --2- 172.21.15.146:0/2568734273 >> v2:172.21.15.154:3300/0 conn(0x7f4ac40dcd70 0x7f4ac4009260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:14.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad2aa8700 1 --2- 172.21.15.146:0/2568734273 >> v2:172.21.15.154:3300/0 conn(0x7f4ac40dcd70 0x7f4ac4009260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:14.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad2aa8700 1 -- 172.21.15.146:0/2568734273 >> v1:172.21.15.154:6789/0 conn(0x7f4ac404e3e0 legacy=0x7f4ac404cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:14.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad2aa8700 1 -- 172.21.15.146:0/2568734273 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac40e6160 con 0x7f4ac40e6c00 2024-09-13T18:58:14.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad22a7700 1 --2- 172.21.15.146:0/2568734273 >> v2:172.21.15.154:3300/0 conn(0x7f4ac40dcd70 0x7f4ac4009260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:58:14.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad2aa8700 1 --2- 172.21.15.146:0/2568734273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac40e6c00 0x7f4ac4051ea0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0731e0 tx=0x7f4acc061ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7307e4c04a585f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:14.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad12a5700 1 -- 172.21.15.146:0/2568734273 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc0739c0 con 0x7f4ac40e6c00 2024-09-13T18:58:14.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad12a5700 1 -- 172.21.15.146:0/2568734273 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4acc075a30 con 0x7f4ac40e6c00 2024-09-13T18:58:14.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad12a5700 1 -- 172.21.15.146:0/2568734273 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc075de0 con 0x7f4ac40e6c00 2024-09-13T18:58:14.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 -- 172.21.15.146:0/2568734273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac40e6c00 msgr2=0x7f4ac4051ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:14.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 --2- 172.21.15.146:0/2568734273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac40e6c00 0x7f4ac4051ea0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0731e0 tx=0x7f4acc061ef0 comp rx=0 tx=0).stop 2024-09-13T18:58:14.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 -- 172.21.15.146:0/2568734273 shutdown_connections 2024-09-13T18:58:14.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 --2- 172.21.15.146:0/2568734273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac40e6c00 0x7f4ac4051ea0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:14.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 --2- 172.21.15.146:0/2568734273 >> v2:172.21.15.154:3300/0 conn(0x7f4ac40dcd70 0x7f4ac4009260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:14.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 -- 172.21.15.146:0/2568734273 >> 172.21.15.146:0/2568734273 conn(0x7f4ac400dc40 msgr2=0x7f4ac400e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:14.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 -- 172.21.15.146:0/2568734273 shutdown_connections 2024-09-13T18:58:14.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 -- 172.21.15.146:0/2568734273 wait complete. 2024-09-13T18:58:14.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 Processor -- start 2024-09-13T18:58:14.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 -- start start 2024-09-13T18:58:14.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac404e3e0 0x7f4ac40de600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:14.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ac40dcd70 0x7f4ac404b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:14.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac40e6c00 0x7f4ac417bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:14.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40dd600 con 0x7f4ac404e3e0 2024-09-13T18:58:14.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad22a7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac404e3e0 0x7f4ac40de600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:14.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad22a7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac404e3e0 0x7f4ac40de600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38388/0 (socket says 172.21.15.146:38388) 2024-09-13T18:58:14.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad22a7700 1 -- 172.21.15.146:0/3638593613 learned_addr learned my addr 172.21.15.146:0/3638593613 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:14.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 -- 172.21.15.146:0/3638593613 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40dd8c0 con 0x7f4ac40e6c00 2024-09-13T18:58:14.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.137+0000 7f4ad32a9700 1 -- 172.21.15.146:0/3638593613 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40df2b0 con 0x7f4ac40dcd70 2024-09-13T18:58:14.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ad1aa6700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ac40dcd70 0x7f4ac404b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:14.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ad2aa8700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac40e6c00 0x7f4ac417bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:14.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ad22a7700 1 -- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ac40dcd70 msgr2=0x7f4ac404b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:14.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ad22a7700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ac40dcd70 0x7f4ac404b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:14.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ad22a7700 1 -- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac40e6c00 msgr2=0x7f4ac417bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:14.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ad22a7700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac40e6c00 0x7f4ac417bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:14.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ad22a7700 1 -- 172.21.15.146:0/3638593613 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc04f5c0 con 0x7f4ac404e3e0 2024-09-13T18:58:14.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ad22a7700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac404e3e0 0x7f4ac40de600 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800c770 tx=0x7f4ac800ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:14.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ac2ffd700 1 -- 172.21.15.146:0/3638593613 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8011800 con 0x7f4ac404e3e0 2024-09-13T18:58:14.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ac2ffd700 1 -- 172.21.15.146:0/3638593613 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac8011960 con 0x7f4ac404e3e0 2024-09-13T18:58:14.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ad32a9700 1 -- 172.21.15.146:0/3638593613 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac40df590 con 0x7f4ac404e3e0 2024-09-13T18:58:14.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ac2ffd700 1 -- 172.21.15.146:0/3638593613 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8011d10 con 0x7f4ac404e3e0 2024-09-13T18:58:14.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ad32a9700 1 -- 172.21.15.146:0/3638593613 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac40dfb50 con 0x7f4ac404e3e0 2024-09-13T18:58:14.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ad32a9700 1 -- 172.21.15.146:0/3638593613 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac4051c40 con 0x7f4ac404e3e0 2024-09-13T18:58:14.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ac2ffd700 1 -- 172.21.15.146:0/3638593613 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4ac8018360 con 0x7f4ac404e3e0 2024-09-13T18:58:14.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ac2ffd700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ab0066760 0x7f4ab0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:14.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ac2ffd700 1 -- 172.21.15.146:0/3638593613 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f4ac8084910 con 0x7f4ac404e3e0 2024-09-13T18:58:14.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.141+0000 7f4ad1aa6700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ab0066760 0x7f4ab0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:14.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.145+0000 7f4ad1aa6700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ab0066760 0x7f4ab0068c20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40347f0 tx=0x7f4abc0090e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:14.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.149+0000 7f4ac2ffd700 1 -- 172.21.15.146:0/3638593613 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4ac80513f0 con 0x7f4ac404e3e0 2024-09-13T18:58:14.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.313+0000 7f4ad32a9700 1 -- 172.21.15.146:0/3638593613 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ac40e6400 con 0x7f4ab0066760 2024-09-13T18:58:14.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.333+0000 7f4ac2ffd700 1 -- 172.21.15.146:0/3638593613 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+372306 (secure 0 0 0) 0x7f4abc009e20 con 0x7f4ab0066760 2024-09-13T18:58:14.348 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:58:14.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.341+0000 7f4ad32a9700 1 -- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ab0066760 msgr2=0x7f4ab0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:14.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.341+0000 7f4ad32a9700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ab0066760 0x7f4ab0068c20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40347f0 tx=0x7f4abc0090e0 comp rx=0 tx=0).stop 2024-09-13T18:58:14.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.341+0000 7f4ad32a9700 1 -- 172.21.15.146:0/3638593613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac404e3e0 msgr2=0x7f4ac40de600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:14.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.341+0000 7f4ad32a9700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac404e3e0 0x7f4ac40de600 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800c770 tx=0x7f4ac800ca80 comp rx=0 tx=0).stop 2024-09-13T18:58:14.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.341+0000 7f4ad32a9700 1 -- 172.21.15.146:0/3638593613 shutdown_connections 2024-09-13T18:58:14.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.341+0000 7f4ad32a9700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f4ab0066760 0x7f4ab0068c20 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:14.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.341+0000 7f4ad32a9700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac40e6c00 0x7f4ac417bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:14.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.341+0000 7f4ad32a9700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ac40dcd70 0x7f4ac404b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:14.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.341+0000 7f4ad32a9700 1 --2- 172.21.15.146:0/3638593613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac404e3e0 0x7f4ac40de600 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:14.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.341+0000 7f4ad32a9700 1 -- 172.21.15.146:0/3638593613 >> 172.21.15.146:0/3638593613 conn(0x7f4ac400dc40 msgr2=0x7f4ac4051230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:14.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.341+0000 7f4ad32a9700 1 -- 172.21.15.146:0/3638593613 shutdown_connections 2024-09-13T18:58:14.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:14.341+0000 7f4ad32a9700 1 -- 172.21.15.146:0/3638593613 wait complete. 2024-09-13T18:58:14.352 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:58:14.365 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":990,"stamp":"2024-09-13T18:58:13.623831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17097455,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":2275,"num_read_kb":52180,"num_write":807501,"num_write_kb":16339542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":182570,"ondisk_log_size":182570,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7800200,"kb_used_data":26056,"kb_used_omap":12,"kb_used_meta":7774067,"kb_avail":747174520,"statfs":{"total":773094113280,"available":765106708480,"internally_reserved":0,"allocated":26681344,"data_stored":36115022,"data_compressed":2507056,"data_compressed_allocated":3596288,"data_compressed_original":15097856,"omap_allocated":12741,"internal_metadata":7960645179},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005372"},"pg_stats":[{"pgid":"2.1f","version":"43'471","reported_seq":562,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.638819+0000","last_change":"2024-09-13T18:57:12.363177+0000","last_active":"2024-09-13T18:57:27.638819+0000","last_peered":"2024-09-13T18:57:27.638819+0000","last_clean":"2024-09-13T18:57:27.638819+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:57:27.638819+0000","last_undegraded":"2024-09-13T18:57:27.638819+0000","last_fullsized":"2024-09-13T18:57:27.638819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'469","last_scrub_stamp":"2024-09-13T18:57:12.363068+0000","last_deep_scrub":"43'469","last_deep_scrub_stamp":"2024-09-13T18:57:12.363068+0000","last_clean_scrub_stamp":"2024-09-13T18:57:12.363068+0000","objects_scrubbed":5,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:41:46.348669+0000","scrub_duration":0.044894295000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3824255,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":428,"num_write":456,"num_write_kb":3991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"43'5","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:16.089759+0000","last_change":"2024-09-13T18:57:16.090020+0000","last_active":"2024-09-13T18:57:16.089759+0000","last_peered":"2024-09-13T18:57:16.089759+0000","last_clean":"2024-09-13T18:57:16.089759+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:57:16.089759+0000","last_undegraded":"2024-09-13T18:57:16.089759+0000","last_fullsized":"2024-09-13T18:57:16.089759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:16.089709+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:16.089709+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.089709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.965527+0000","scrub_duration":0.0037314340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"43'5858","reported_seq":5948,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:01.958558+0000","last_change":"2024-09-13T18:58:01.958678+0000","last_active":"2024-09-13T18:58:01.958558+0000","last_peered":"2024-09-13T18:58:01.958558+0000","last_clean":"2024-09-13T18:58:01.958558+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:58:01.958558+0000","last_undegraded":"2024-09-13T18:58:01.958558+0000","last_fullsized":"2024-09-13T18:58:01.958558+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5858","last_scrub_stamp":"2024-09-13T18:58:01.958539+0000","last_deep_scrub":"43'5858","last_deep_scrub_stamp":"2024-09-13T18:58:01.958539+0000","last_clean_scrub_stamp":"2024-09-13T18:58:01.958539+0000","objects_scrubbed":0,"log_size":5858,"log_dups_size":0,"ondisk_log_size":5858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:20.890340+0000","scrub_duration":0.043303400999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":26767,"num_write_kb":537654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"43'16","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.390499+0000","last_change":"2024-09-13T18:57:25.919095+0000","last_active":"2024-09-13T18:57:27.390499+0000","last_peered":"2024-09-13T18:57:27.390499+0000","last_clean":"2024-09-13T18:57:27.390499+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:57:27.390499+0000","last_undegraded":"2024-09-13T18:57:27.390499+0000","last_fullsized":"2024-09-13T18:57:27.390499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:25.918713+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:25.918713+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.918713+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:35.551553+0000","scrub_duration":0.0032041489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"43'14","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:33.385777+0000","last_change":"2024-09-13T18:57:33.385777+0000","last_active":"2024-09-13T18:57:33.385777+0000","last_peered":"2024-09-13T18:57:33.385777+0000","last_clean":"2024-09-13T18:57:33.385777+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:57:33.385777+0000","last_undegraded":"2024-09-13T18:57:33.385777+0000","last_fullsized":"2024-09-13T18:57:33.385777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:33.385394+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:33.385394+0000","last_clean_scrub_stamp":"2024-09-13T18:57:33.385394+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:52.913543+0000","scrub_duration":0.0034717820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"43'5909","reported_seq":5993,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:40.030905+0000","last_change":"2024-09-13T18:57:40.030905+0000","last_active":"2024-09-13T18:57:40.030905+0000","last_peered":"2024-09-13T18:57:40.030905+0000","last_clean":"2024-09-13T18:57:40.030905+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:57:40.030905+0000","last_undegraded":"2024-09-13T18:57:40.030905+0000","last_fullsized":"2024-09-13T18:57:40.030905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5909","last_scrub_stamp":"2024-09-13T18:57:40.030533+0000","last_deep_scrub":"43'5909","last_deep_scrub_stamp":"2024-09-13T18:57:40.030533+0000","last_clean_scrub_stamp":"2024-09-13T18:57:40.030533+0000","objects_scrubbed":0,"log_size":5909,"log_dups_size":0,"ondisk_log_size":5909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:18:12.112501+0000","scrub_duration":0.046110303999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":26783,"num_write_kb":545346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"43'29","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:37.893910+0000","last_change":"2024-09-13T18:57:37.894025+0000","last_active":"2024-09-13T18:57:37.893910+0000","last_peered":"2024-09-13T18:57:37.893910+0000","last_clean":"2024-09-13T18:57:37.893910+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:57:37.893910+0000","last_undegraded":"2024-09-13T18:57:37.893910+0000","last_fullsized":"2024-09-13T18:57:37.893910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'29","last_scrub_stamp":"2024-09-13T18:57:37.893891+0000","last_deep_scrub":"43'29","last_deep_scrub_stamp":"2024-09-13T18:57:37.893891+0000","last_clean_scrub_stamp":"2024-09-13T18:57:37.893891+0000","objects_scrubbed":2,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:54:57.603232+0000","scrub_duration":0.0099177959999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":65,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"43'9","reported_seq":59,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:44.043128+0000","last_change":"2024-09-13T18:57:44.043334+0000","last_active":"2024-09-13T18:57:44.043128+0000","last_peered":"2024-09-13T18:57:44.043128+0000","last_clean":"2024-09-13T18:57:44.043128+0000","last_became_active":"2024-09-13T18:27:22.609625+0000","last_became_peered":"2024-09-13T18:27:22.609625+0000","last_unstale":"2024-09-13T18:57:44.043128+0000","last_undegraded":"2024-09-13T18:57:44.043128+0000","last_fullsized":"2024-09-13T18:57:44.043128+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:57:44.043080+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:57:44.043080+0000","last_clean_scrub_stamp":"2024-09-13T18:57:44.043080+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:40:41.348901+0000","scrub_duration":0.0036812300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"43'4988","reported_seq":5062,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:50.959877+0000","last_change":"2024-09-13T18:57:50.960011+0000","last_active":"2024-09-13T18:57:50.959877+0000","last_peered":"2024-09-13T18:57:50.959877+0000","last_clean":"2024-09-13T18:57:50.959877+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:57:50.959877+0000","last_undegraded":"2024-09-13T18:57:50.959877+0000","last_fullsized":"2024-09-13T18:57:50.959877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4988","last_scrub_stamp":"2024-09-13T18:57:50.959856+0000","last_deep_scrub":"43'4988","last_deep_scrub_stamp":"2024-09-13T18:57:50.959856+0000","last_clean_scrub_stamp":"2024-09-13T18:57:50.959856+0000","objects_scrubbed":0,"log_size":4988,"log_dups_size":0,"ondisk_log_size":4988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:31.885945+0000","scrub_duration":0.040254119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21624,"num_write_kb":467229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"43'9","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:31.883126+0000","last_change":"2024-09-13T18:57:31.883350+0000","last_active":"2024-09-13T18:57:31.883126+0000","last_peered":"2024-09-13T18:57:31.883126+0000","last_clean":"2024-09-13T18:57:31.883126+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:57:31.883126+0000","last_undegraded":"2024-09-13T18:57:31.883126+0000","last_fullsized":"2024-09-13T18:57:31.883126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:57:31.883070+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:57:31.883070+0000","last_clean_scrub_stamp":"2024-09-13T18:57:31.883070+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:54.521068+0000","scrub_duration":0.0057825330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":12,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"43'16","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:54.218083+0000","last_change":"2024-09-13T18:57:54.218258+0000","last_active":"2024-09-13T18:57:54.218083+0000","last_peered":"2024-09-13T18:57:54.218083+0000","last_clean":"2024-09-13T18:57:54.218083+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:57:54.218083+0000","last_undegraded":"2024-09-13T18:57:54.218083+0000","last_fullsized":"2024-09-13T18:57:54.218083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:54.218043+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:54.218043+0000","last_clean_scrub_stamp":"2024-09-13T18:57:54.218043+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:34.235521+0000","scrub_duration":0.003853441,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"43'4887","reported_seq":4961,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:46.899456+0000","last_change":"2024-09-13T18:57:46.899691+0000","last_active":"2024-09-13T18:57:46.899456+0000","last_peered":"2024-09-13T18:57:46.899456+0000","last_clean":"2024-09-13T18:57:46.899456+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:57:46.899456+0000","last_undegraded":"2024-09-13T18:57:46.899456+0000","last_fullsized":"2024-09-13T18:57:46.899456+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4887","last_scrub_stamp":"2024-09-13T18:57:46.899410+0000","last_deep_scrub":"43'4887","last_deep_scrub_stamp":"2024-09-13T18:57:46.899410+0000","last_clean_scrub_stamp":"2024-09-13T18:57:46.899410+0000","objects_scrubbed":0,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:15.511519+0000","scrub_duration":0.04143703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21624,"num_write_kb":434281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"43'9","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:29.780911+0000","last_change":"2024-09-13T18:57:29.781039+0000","last_active":"2024-09-13T18:57:29.780911+0000","last_peered":"2024-09-13T18:57:29.780911+0000","last_clean":"2024-09-13T18:57:29.780911+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:57:29.780911+0000","last_undegraded":"2024-09-13T18:57:29.780911+0000","last_fullsized":"2024-09-13T18:57:29.780911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:57:29.780879+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:57:29.780879+0000","last_clean_scrub_stamp":"2024-09-13T18:57:29.780879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:44.093932+0000","scrub_duration":0.0034454730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"43'1223","reported_seq":1308,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.921602+0000","last_change":"2024-09-13T18:57:28.921602+0000","last_active":"2024-09-13T18:57:28.921602+0000","last_peered":"2024-09-13T18:57:28.921602+0000","last_clean":"2024-09-13T18:57:28.921602+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:57:28.921602+0000","last_undegraded":"2024-09-13T18:57:28.921602+0000","last_fullsized":"2024-09-13T18:57:28.921602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'1223","last_scrub_stamp":"2024-09-13T18:57:28.921440+0000","last_deep_scrub":"43'1223","last_deep_scrub_stamp":"2024-09-13T18:57:28.921440+0000","last_clean_scrub_stamp":"2024-09-13T18:57:28.921440+0000","objects_scrubbed":3,"log_size":1223,"log_dups_size":0,"ondisk_log_size":1223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:20.603136+0000","scrub_duration":0.029685530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":10235,"num_write":1211,"num_write_kb":4423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"43'5452","reported_seq":5531,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:47.934939+0000","last_change":"2024-09-13T18:57:47.934939+0000","last_active":"2024-09-13T18:57:47.934939+0000","last_peered":"2024-09-13T18:57:47.934939+0000","last_clean":"2024-09-13T18:57:47.934939+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:57:47.934939+0000","last_undegraded":"2024-09-13T18:57:47.934939+0000","last_fullsized":"2024-09-13T18:57:47.934939+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5452","last_scrub_stamp":"2024-09-13T18:57:47.934770+0000","last_deep_scrub":"43'5452","last_deep_scrub_stamp":"2024-09-13T18:57:47.934770+0000","last_clean_scrub_stamp":"2024-09-13T18:57:47.934770+0000","objects_scrubbed":0,"log_size":5452,"log_dups_size":0,"ondisk_log_size":5452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:56:53.358515+0000","scrub_duration":0.035794448,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":24128,"num_write_kb":505876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"43'11","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:37.925499+0000","last_change":"2024-09-13T18:57:37.925499+0000","last_active":"2024-09-13T18:57:37.925499+0000","last_peered":"2024-09-13T18:57:37.925499+0000","last_clean":"2024-09-13T18:57:37.925499+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:57:37.925499+0000","last_undegraded":"2024-09-13T18:57:37.925499+0000","last_fullsized":"2024-09-13T18:57:37.925499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:37.925363+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:37.925363+0000","last_clean_scrub_stamp":"2024-09-13T18:57:37.925363+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:10.713761+0000","scrub_duration":0.0037509309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"43'16","reported_seq":82,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:32.292293+0000","last_change":"2024-09-13T18:57:32.292403+0000","last_active":"2024-09-13T18:57:32.292293+0000","last_peered":"2024-09-13T18:57:32.292293+0000","last_clean":"2024-09-13T18:57:32.292293+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:57:32.292293+0000","last_undegraded":"2024-09-13T18:57:32.292293+0000","last_fullsized":"2024-09-13T18:57:32.292293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:32.292267+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:32.292267+0000","last_clean_scrub_stamp":"2024-09-13T18:57:32.292267+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:41.284438+0000","scrub_duration":0.025296163999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"43'5442","reported_seq":5524,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:48.974304+0000","last_change":"2024-09-13T18:57:48.974304+0000","last_active":"2024-09-13T18:57:48.974304+0000","last_peered":"2024-09-13T18:57:48.974304+0000","last_clean":"2024-09-13T18:57:48.974304+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:57:48.974304+0000","last_undegraded":"2024-09-13T18:57:48.974304+0000","last_fullsized":"2024-09-13T18:57:48.974304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5442","last_scrub_stamp":"2024-09-13T18:57:48.974110+0000","last_deep_scrub":"43'5442","last_deep_scrub_stamp":"2024-09-13T18:57:48.974110+0000","last_clean_scrub_stamp":"2024-09-13T18:57:48.974110+0000","objects_scrubbed":0,"log_size":5442,"log_dups_size":0,"ondisk_log_size":5442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:38.773027+0000","scrub_duration":0.039492270000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":24412,"num_write_kb":498911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"43'10","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:41.082668+0000","last_change":"2024-09-13T18:57:41.082668+0000","last_active":"2024-09-13T18:57:41.082668+0000","last_peered":"2024-09-13T18:57:41.082668+0000","last_clean":"2024-09-13T18:57:41.082668+0000","last_became_active":"2024-09-13T18:27:22.609642+0000","last_became_peered":"2024-09-13T18:27:22.609642+0000","last_unstale":"2024-09-13T18:57:41.082668+0000","last_undegraded":"2024-09-13T18:57:41.082668+0000","last_fullsized":"2024-09-13T18:57:41.082668+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:41.082525+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:41.082525+0000","last_clean_scrub_stamp":"2024-09-13T18:57:41.082525+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:52:04.123510+0000","scrub_duration":0.0029389820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"43'7","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.406801+0000","last_change":"2024-09-13T18:57:20.192184+0000","last_active":"2024-09-13T18:57:26.406801+0000","last_peered":"2024-09-13T18:57:26.406801+0000","last_clean":"2024-09-13T18:57:26.406801+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:57:26.406801+0000","last_undegraded":"2024-09-13T18:57:26.406801+0000","last_fullsized":"2024-09-13T18:57:26.406801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:20.191966+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:20.191966+0000","last_clean_scrub_stamp":"2024-09-13T18:57:20.191966+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:39.033583+0000","scrub_duration":0.0068871679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"43'7006","reported_seq":7087,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:43.055580+0000","last_change":"2024-09-13T18:57:43.055725+0000","last_active":"2024-09-13T18:57:43.055580+0000","last_peered":"2024-09-13T18:57:43.055580+0000","last_clean":"2024-09-13T18:57:43.055580+0000","last_became_active":"2024-09-13T18:27:22.610527+0000","last_became_peered":"2024-09-13T18:27:22.610527+0000","last_unstale":"2024-09-13T18:57:43.055580+0000","last_undegraded":"2024-09-13T18:57:43.055580+0000","last_fullsized":"2024-09-13T18:57:43.055580+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'7006","last_scrub_stamp":"2024-09-13T18:57:43.055548+0000","last_deep_scrub":"43'7006","last_deep_scrub_stamp":"2024-09-13T18:57:43.055548+0000","last_clean_scrub_stamp":"2024-09-13T18:57:43.055548+0000","objects_scrubbed":0,"log_size":7006,"log_dups_size":0,"ondisk_log_size":7006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:13:12.839547+0000","scrub_duration":0.042924489000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":31810,"num_write_kb":622755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"43'11","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.393739+0000","last_change":"2024-09-13T18:57:17.999669+0000","last_active":"2024-09-13T18:57:28.393739+0000","last_peered":"2024-09-13T18:57:28.393739+0000","last_clean":"2024-09-13T18:57:28.393739+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:57:28.393739+0000","last_undegraded":"2024-09-13T18:57:28.393739+0000","last_fullsized":"2024-09-13T18:57:28.393739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:17.999495+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:17.999495+0000","last_clean_scrub_stamp":"2024-09-13T18:57:17.999495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:58.108162+0000","scrub_duration":0.0026512860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"43'7","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.926150+0000","last_change":"2024-09-13T18:57:39.926414+0000","last_active":"2024-09-13T18:57:39.926150+0000","last_peered":"2024-09-13T18:57:39.926150+0000","last_clean":"2024-09-13T18:57:39.926150+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:57:39.926150+0000","last_undegraded":"2024-09-13T18:57:39.926150+0000","last_fullsized":"2024-09-13T18:57:39.926150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'7","last_scrub_stamp":"2024-09-13T18:57:39.926056+0000","last_deep_scrub":"43'7","last_deep_scrub_stamp":"2024-09-13T18:57:39.926056+0000","last_clean_scrub_stamp":"2024-09-13T18:57:39.926056+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:42.161980+0000","scrub_duration":0.0070368430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"43'5416","reported_seq":5506,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:42.965611+0000","last_change":"2024-09-13T18:57:42.965611+0000","last_active":"2024-09-13T18:57:42.965611+0000","last_peered":"2024-09-13T18:57:42.965611+0000","last_clean":"2024-09-13T18:57:42.965611+0000","last_became_active":"2024-09-13T18:27:22.648054+0000","last_became_peered":"2024-09-13T18:27:22.648054+0000","last_unstale":"2024-09-13T18:57:42.965611+0000","last_undegraded":"2024-09-13T18:57:42.965611+0000","last_fullsized":"2024-09-13T18:57:42.965611+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5416","last_scrub_stamp":"2024-09-13T18:57:42.965388+0000","last_deep_scrub":"43'5416","last_deep_scrub_stamp":"2024-09-13T18:57:42.965388+0000","last_clean_scrub_stamp":"2024-09-13T18:57:42.965388+0000","objects_scrubbed":0,"log_size":5416,"log_dups_size":0,"ondisk_log_size":5416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:18.488873+0000","scrub_duration":0.042479966000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":23625,"num_write_kb":471297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"43'19","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.065688+0000","last_change":"2024-09-13T18:57:22.408978+0000","last_active":"2024-09-13T18:57:28.065688+0000","last_peered":"2024-09-13T18:57:28.065688+0000","last_clean":"2024-09-13T18:57:28.065688+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:57:28.065688+0000","last_undegraded":"2024-09-13T18:57:28.065688+0000","last_fullsized":"2024-09-13T18:57:28.065688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:22.408739+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:22.408739+0000","last_clean_scrub_stamp":"2024-09-13T18:57:22.408739+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:45:55.319773+0000","scrub_duration":0.003011952,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"43'11","reported_seq":94,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.645812+0000","last_change":"2024-09-13T18:57:18.823588+0000","last_active":"2024-09-13T18:57:27.645812+0000","last_peered":"2024-09-13T18:57:27.645812+0000","last_clean":"2024-09-13T18:57:27.645812+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:57:27.645812+0000","last_undegraded":"2024-09-13T18:57:27.645812+0000","last_fullsized":"2024-09-13T18:57:27.645812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'7","last_scrub_stamp":"2024-09-13T18:57:18.823152+0000","last_deep_scrub":"43'7","last_deep_scrub_stamp":"2024-09-13T18:57:18.823152+0000","last_clean_scrub_stamp":"2024-09-13T18:57:18.823152+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:52.572271+0000","scrub_duration":0.023118413000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"43'5405","reported_seq":5489,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.568512+0000","last_change":"2024-09-13T18:57:36.167581+0000","last_active":"2024-09-13T18:57:38.568512+0000","last_peered":"2024-09-13T18:57:38.568512+0000","last_clean":"2024-09-13T18:57:38.568512+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:57:38.568512+0000","last_undegraded":"2024-09-13T18:57:38.568512+0000","last_fullsized":"2024-09-13T18:57:38.568512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5395","last_scrub_stamp":"2024-09-13T18:57:36.167414+0000","last_deep_scrub":"43'5395","last_deep_scrub_stamp":"2024-09-13T18:57:36.167414+0000","last_clean_scrub_stamp":"2024-09-13T18:57:36.167414+0000","objects_scrubbed":2,"log_size":5405,"log_dups_size":0,"ondisk_log_size":5405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:04:32.293024+0000","scrub_duration":0.24736883800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":24452,"num_write_kb":507343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"43'13","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.384061+0000","last_change":"2024-09-13T18:57:17.064133+0000","last_active":"2024-09-13T18:57:28.384061+0000","last_peered":"2024-09-13T18:57:28.384061+0000","last_clean":"2024-09-13T18:57:28.384061+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:57:28.384061+0000","last_undegraded":"2024-09-13T18:57:28.384061+0000","last_fullsized":"2024-09-13T18:57:28.384061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:17.063923+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:17.063923+0000","last_clean_scrub_stamp":"2024-09-13T18:57:17.063923+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.609366+0000","scrub_duration":0.020682738999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"43'289","reported_seq":363,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.907282+0000","last_change":"2024-09-13T18:57:22.143854+0000","last_active":"2024-09-13T18:57:24.907282+0000","last_peered":"2024-09-13T18:57:24.907282+0000","last_clean":"2024-09-13T18:57:24.907282+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:57:24.907282+0000","last_undegraded":"2024-09-13T18:57:24.907282+0000","last_fullsized":"2024-09-13T18:57:24.907282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'286","last_scrub_stamp":"2024-09-13T18:57:22.143716+0000","last_deep_scrub":"43'286","last_deep_scrub_stamp":"2024-09-13T18:57:22.143716+0000","last_clean_scrub_stamp":"2024-09-13T18:57:22.143716+0000","objects_scrubbed":2,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:44.447938+0000","scrub_duration":0.0070881099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":52,"num_write":626,"num_write_kb":212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"43'6116","reported_seq":6209,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:05.950781+0000","last_change":"2024-09-13T18:58:05.950968+0000","last_active":"2024-09-13T18:58:05.950781+0000","last_peered":"2024-09-13T18:58:05.950781+0000","last_clean":"2024-09-13T18:58:05.950781+0000","last_became_active":"2024-09-13T18:27:22.644681+0000","last_became_peered":"2024-09-13T18:27:22.644681+0000","last_unstale":"2024-09-13T18:58:05.950781+0000","last_undegraded":"2024-09-13T18:58:05.950781+0000","last_fullsized":"2024-09-13T18:58:05.950781+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6116","last_scrub_stamp":"2024-09-13T18:58:05.950737+0000","last_deep_scrub":"43'6116","last_deep_scrub_stamp":"2024-09-13T18:58:05.950737+0000","last_clean_scrub_stamp":"2024-09-13T18:58:05.950737+0000","objects_scrubbed":0,"log_size":6116,"log_dups_size":0,"ondisk_log_size":6116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:59:36.744911+0000","scrub_duration":0.042980222999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":27902,"num_write_kb":553947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"43'9","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.380487+0000","last_change":"2024-09-13T18:57:13.217203+0000","last_active":"2024-09-13T18:57:28.380487+0000","last_peered":"2024-09-13T18:57:28.380487+0000","last_clean":"2024-09-13T18:57:28.380487+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:57:28.380487+0000","last_undegraded":"2024-09-13T18:57:28.380487+0000","last_fullsized":"2024-09-13T18:57:28.380487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:13.216878+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:13.216878+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.216878+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:25:20.911582+0000","scrub_duration":0.0034586249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"43'8","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:29.997531+0000","last_change":"2024-09-13T18:57:29.997531+0000","last_active":"2024-09-13T18:57:29.997531+0000","last_peered":"2024-09-13T18:57:29.997531+0000","last_clean":"2024-09-13T18:57:29.997531+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:57:29.997531+0000","last_undegraded":"2024-09-13T18:57:29.997531+0000","last_fullsized":"2024-09-13T18:57:29.997531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:29.997356+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:29.997356+0000","last_clean_scrub_stamp":"2024-09-13T18:57:29.997356+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:42:58.364306+0000","scrub_duration":0.076701386999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"43'4791","reported_seq":4867,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:53.311285+0000","last_change":"2024-09-13T18:57:53.311285+0000","last_active":"2024-09-13T18:57:53.311285+0000","last_peered":"2024-09-13T18:57:53.311285+0000","last_clean":"2024-09-13T18:57:53.311285+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:57:53.311285+0000","last_undegraded":"2024-09-13T18:57:53.311285+0000","last_fullsized":"2024-09-13T18:57:53.311285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4791","last_scrub_stamp":"2024-09-13T18:57:53.311114+0000","last_deep_scrub":"43'4791","last_deep_scrub_stamp":"2024-09-13T18:57:53.311114+0000","last_clean_scrub_stamp":"2024-09-13T18:57:53.311114+0000","objects_scrubbed":0,"log_size":4791,"log_dups_size":0,"ondisk_log_size":4791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:13.675488+0000","scrub_duration":0.03002403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":22435,"num_write_kb":450115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"43'12","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.357797+0000","last_change":"2024-09-13T18:57:13.000734+0000","last_active":"2024-09-13T18:57:38.357797+0000","last_peered":"2024-09-13T18:57:38.357797+0000","last_clean":"2024-09-13T18:57:38.357797+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:57:38.357797+0000","last_undegraded":"2024-09-13T18:57:38.357797+0000","last_fullsized":"2024-09-13T18:57:38.357797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'7","last_scrub_stamp":"2024-09-13T18:57:13.000480+0000","last_deep_scrub":"28'7","last_deep_scrub_stamp":"2024-09-13T18:57:13.000480+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.000480+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:33.028238+0000","scrub_duration":0.0036870150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"43'3","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.389515+0000","last_change":"2024-09-13T18:57:14.228792+0000","last_active":"2024-09-13T18:57:27.389515+0000","last_peered":"2024-09-13T18:57:27.389515+0000","last_clean":"2024-09-13T18:57:27.389515+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:57:27.389515+0000","last_undegraded":"2024-09-13T18:57:27.389515+0000","last_fullsized":"2024-09-13T18:57:27.389515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:57:14.228519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:57:14.228519+0000","last_clean_scrub_stamp":"2024-09-13T18:57:14.228519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:55.913907+0000","scrub_duration":0.0032354189999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"43'6174","reported_seq":6256,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:56.237918+0000","last_change":"2024-09-13T18:57:56.238103+0000","last_active":"2024-09-13T18:57:56.237918+0000","last_peered":"2024-09-13T18:57:56.237918+0000","last_clean":"2024-09-13T18:57:56.237918+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:57:56.237918+0000","last_undegraded":"2024-09-13T18:57:56.237918+0000","last_fullsized":"2024-09-13T18:57:56.237918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6174","last_scrub_stamp":"2024-09-13T18:57:56.237894+0000","last_deep_scrub":"43'6174","last_deep_scrub_stamp":"2024-09-13T18:57:56.237894+0000","last_clean_scrub_stamp":"2024-09-13T18:57:56.237894+0000","objects_scrubbed":0,"log_size":6174,"log_dups_size":0,"ondisk_log_size":6174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:07:22.671920+0000","scrub_duration":0.045051943999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":28067,"num_write_kb":562911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"43'9","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.442033+0000","last_change":"2024-09-13T18:57:12.010586+0000","last_active":"2024-09-13T18:57:26.442033+0000","last_peered":"2024-09-13T18:57:26.442033+0000","last_clean":"2024-09-13T18:57:26.442033+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:57:26.442033+0000","last_undegraded":"2024-09-13T18:57:26.442033+0000","last_fullsized":"2024-09-13T18:57:26.442033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'6","last_scrub_stamp":"2024-09-13T18:57:12.010428+0000","last_deep_scrub":"28'6","last_deep_scrub_stamp":"2024-09-13T18:57:12.010428+0000","last_clean_scrub_stamp":"2024-09-13T18:57:12.010428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:15.823406+0000","scrub_duration":0.0039755040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"43'9","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.428380+0000","last_change":"2024-09-13T18:57:15.993201+0000","last_active":"2024-09-13T18:57:26.428380+0000","last_peered":"2024-09-13T18:57:26.428380+0000","last_clean":"2024-09-13T18:57:26.428380+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:57:26.428380+0000","last_undegraded":"2024-09-13T18:57:26.428380+0000","last_fullsized":"2024-09-13T18:57:26.428380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'6","last_scrub_stamp":"2024-09-13T18:57:15.992890+0000","last_deep_scrub":"41'6","last_deep_scrub_stamp":"2024-09-13T18:57:15.992890+0000","last_clean_scrub_stamp":"2024-09-13T18:57:15.992890+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:28:43.312440+0000","scrub_duration":0.0036839949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"43'5413","reported_seq":5498,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:54.240011+0000","last_change":"2024-09-13T18:57:54.240011+0000","last_active":"2024-09-13T18:57:54.240011+0000","last_peered":"2024-09-13T18:57:54.240011+0000","last_clean":"2024-09-13T18:57:54.240011+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:57:54.240011+0000","last_undegraded":"2024-09-13T18:57:54.240011+0000","last_fullsized":"2024-09-13T18:57:54.240011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5413","last_scrub_stamp":"2024-09-13T18:57:54.239842+0000","last_deep_scrub":"43'5413","last_deep_scrub_stamp":"2024-09-13T18:57:54.239842+0000","last_clean_scrub_stamp":"2024-09-13T18:57:54.239842+0000","objects_scrubbed":0,"log_size":5413,"log_dups_size":0,"ondisk_log_size":5413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:36.754881+0000","scrub_duration":0.035324452999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":24545,"num_write_kb":507684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"43'14","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:36.275994+0000","last_change":"2024-09-13T18:57:36.277633+0000","last_active":"2024-09-13T18:57:36.275994+0000","last_peered":"2024-09-13T18:57:36.275994+0000","last_clean":"2024-09-13T18:57:36.275994+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:57:36.275994+0000","last_undegraded":"2024-09-13T18:57:36.275994+0000","last_fullsized":"2024-09-13T18:57:36.275994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:36.275964+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:36.275964+0000","last_clean_scrub_stamp":"2024-09-13T18:57:36.275964+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:55:23.015452+0000","scrub_duration":0.027403200999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"43'131","reported_seq":318,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.095368+0000","last_change":"2024-09-13T18:57:16.151559+0000","last_active":"2024-09-13T18:57:28.095368+0000","last_peered":"2024-09-13T18:57:28.095368+0000","last_clean":"2024-09-13T18:57:28.095368+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:57:28.095368+0000","last_undegraded":"2024-09-13T18:57:28.095368+0000","last_fullsized":"2024-09-13T18:57:28.095368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'130","last_scrub_stamp":"2024-09-13T18:57:16.151397+0000","last_deep_scrub":"43'130","last_deep_scrub_stamp":"2024-09-13T18:57:16.151397+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.151397+0000","objects_scrubbed":3,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:19.471742+0000","scrub_duration":0.027827343000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93931,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":203,"num_write":119,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"43'6320","reported_seq":6402,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:01.115048+0000","last_change":"2024-09-13T18:58:01.115201+0000","last_active":"2024-09-13T18:58:01.115048+0000","last_peered":"2024-09-13T18:58:01.115048+0000","last_clean":"2024-09-13T18:58:01.115048+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:58:01.115048+0000","last_undegraded":"2024-09-13T18:58:01.115048+0000","last_fullsized":"2024-09-13T18:58:01.115048+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6320","last_scrub_stamp":"2024-09-13T18:58:01.115028+0000","last_deep_scrub":"43'6320","last_deep_scrub_stamp":"2024-09-13T18:58:01.115028+0000","last_clean_scrub_stamp":"2024-09-13T18:58:01.115028+0000","objects_scrubbed":0,"log_size":6320,"log_dups_size":0,"ondisk_log_size":6320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:12:25.681334+0000","scrub_duration":0.041107435999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":28303,"num_write_kb":563741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"43'8","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.026232+0000","last_change":"2024-09-13T18:57:14.329551+0000","last_active":"2024-09-13T18:57:28.026232+0000","last_peered":"2024-09-13T18:57:28.026232+0000","last_clean":"2024-09-13T18:57:28.026232+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:57:28.026232+0000","last_undegraded":"2024-09-13T18:57:28.026232+0000","last_fullsized":"2024-09-13T18:57:28.026232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:14.329297+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:14.329297+0000","last_clean_scrub_stamp":"2024-09-13T18:57:14.329297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:49.431750+0000","scrub_duration":0.0033135230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"43'18","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.912770+0000","last_change":"2024-09-13T18:57:26.912770+0000","last_active":"2024-09-13T18:57:26.912770+0000","last_peered":"2024-09-13T18:57:26.912770+0000","last_clean":"2024-09-13T18:57:26.912770+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:57:26.912770+0000","last_undegraded":"2024-09-13T18:57:26.912770+0000","last_fullsized":"2024-09-13T18:57:26.912770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'17","last_scrub_stamp":"2024-09-13T18:57:26.912423+0000","last_deep_scrub":"43'17","last_deep_scrub_stamp":"2024-09-13T18:57:26.912423+0000","last_clean_scrub_stamp":"2024-09-13T18:57:26.912423+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:00.997526+0000","scrub_duration":0.0065643969999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"43'4467","reported_seq":4549,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:49.291700+0000","last_change":"2024-09-13T18:57:49.291907+0000","last_active":"2024-09-13T18:57:49.291700+0000","last_peered":"2024-09-13T18:57:49.291700+0000","last_clean":"2024-09-13T18:57:49.291700+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:57:49.291700+0000","last_undegraded":"2024-09-13T18:57:49.291700+0000","last_fullsized":"2024-09-13T18:57:49.291700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4467","last_scrub_stamp":"2024-09-13T18:57:49.291657+0000","last_deep_scrub":"43'4467","last_deep_scrub_stamp":"2024-09-13T18:57:49.291657+0000","last_clean_scrub_stamp":"2024-09-13T18:57:49.291657+0000","objects_scrubbed":0,"log_size":4467,"log_dups_size":0,"ondisk_log_size":4467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:19.421097+0000","scrub_duration":0.030143559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":19862,"num_write_kb":435588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"43'8","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.860482+0000","last_change":"2024-09-13T18:57:39.860482+0000","last_active":"2024-09-13T18:57:39.860482+0000","last_peered":"2024-09-13T18:57:39.860482+0000","last_clean":"2024-09-13T18:57:39.860482+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:57:39.860482+0000","last_undegraded":"2024-09-13T18:57:39.860482+0000","last_fullsized":"2024-09-13T18:57:39.860482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:39.860244+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:39.860244+0000","last_clean_scrub_stamp":"2024-09-13T18:57:39.860244+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:08.621392+0000","scrub_duration":0.0035530589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"43'11","reported_seq":90,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:23.965985+0000","last_change":"2024-09-13T18:57:23.965985+0000","last_active":"2024-09-13T18:57:23.965985+0000","last_peered":"2024-09-13T18:57:23.965985+0000","last_clean":"2024-09-13T18:57:23.965985+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:57:23.965985+0000","last_undegraded":"2024-09-13T18:57:23.965985+0000","last_fullsized":"2024-09-13T18:57:23.965985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:23.965843+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:23.965843+0000","last_clean_scrub_stamp":"2024-09-13T18:57:23.965843+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:51.676617+0000","scrub_duration":0.010225193,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":68,"num_write":9,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"43'119","reported_seq":215,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:19.933586+0000","last_change":"2024-09-13T18:57:19.933697+0000","last_active":"2024-09-13T18:57:19.933586+0000","last_peered":"2024-09-13T18:57:19.933586+0000","last_clean":"2024-09-13T18:57:19.933586+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:57:19.933586+0000","last_undegraded":"2024-09-13T18:57:19.933586+0000","last_fullsized":"2024-09-13T18:57:19.933586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'119","last_scrub_stamp":"2024-09-13T18:57:19.933561+0000","last_deep_scrub":"43'119","last_deep_scrub_stamp":"2024-09-13T18:57:19.933561+0000","last_clean_scrub_stamp":"2024-09-13T18:57:19.933561+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:36.963255+0000","scrub_duration":0.0099632039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"43'6003","reported_seq":6095,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:03.635883+0000","last_change":"2024-09-13T18:58:03.636038+0000","last_active":"2024-09-13T18:58:03.635883+0000","last_peered":"2024-09-13T18:58:03.635883+0000","last_clean":"2024-09-13T18:58:03.635883+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:58:03.635883+0000","last_undegraded":"2024-09-13T18:58:03.635883+0000","last_fullsized":"2024-09-13T18:58:03.635883+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6003","last_scrub_stamp":"2024-09-13T18:58:03.635861+0000","last_deep_scrub":"43'6003","last_deep_scrub_stamp":"2024-09-13T18:58:03.635861+0000","last_clean_scrub_stamp":"2024-09-13T18:58:03.635861+0000","objects_scrubbed":0,"log_size":6003,"log_dups_size":0,"ondisk_log_size":6003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:38:50.326698+0000","scrub_duration":0.046218636,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":27421,"num_write_kb":519330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"43'15","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:10.113914+0000","last_change":"2024-09-13T18:58:10.114122+0000","last_active":"2024-09-13T18:58:10.113914+0000","last_peered":"2024-09-13T18:58:10.113914+0000","last_clean":"2024-09-13T18:58:10.113914+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:58:10.113914+0000","last_undegraded":"2024-09-13T18:58:10.113914+0000","last_fullsized":"2024-09-13T18:58:10.113914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'15","last_scrub_stamp":"2024-09-13T18:58:10.113857+0000","last_deep_scrub":"43'15","last_deep_scrub_stamp":"2024-09-13T18:58:10.113857+0000","last_clean_scrub_stamp":"2024-09-13T18:58:10.113857+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:13.910431+0000","scrub_duration":0.003224832,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"43'8","reported_seq":82,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.127170+0000","last_change":"2024-09-13T18:57:21.187725+0000","last_active":"2024-09-13T18:57:28.127170+0000","last_peered":"2024-09-13T18:57:28.127170+0000","last_clean":"2024-09-13T18:57:28.127170+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:57:28.127170+0000","last_undegraded":"2024-09-13T18:57:28.127170+0000","last_fullsized":"2024-09-13T18:57:28.127170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:21.187525+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:21.187525+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.187525+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:42.675115+0000","scrub_duration":0.032458872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":35,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"43'5433","reported_seq":5509,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:57.609674+0000","last_change":"2024-09-13T18:57:57.609833+0000","last_active":"2024-09-13T18:57:57.609674+0000","last_peered":"2024-09-13T18:57:57.609674+0000","last_clean":"2024-09-13T18:57:57.609674+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:57:57.609674+0000","last_undegraded":"2024-09-13T18:57:57.609674+0000","last_fullsized":"2024-09-13T18:57:57.609674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5433","last_scrub_stamp":"2024-09-13T18:57:57.609650+0000","last_deep_scrub":"43'5433","last_deep_scrub_stamp":"2024-09-13T18:57:57.609650+0000","last_clean_scrub_stamp":"2024-09-13T18:57:57.609650+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:39:53.104071+0000","scrub_duration":0.043346968999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":24067,"num_write_kb":513904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"43'13","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:36.120773+0000","last_change":"2024-09-13T18:57:36.120879+0000","last_active":"2024-09-13T18:57:36.120773+0000","last_peered":"2024-09-13T18:57:36.120773+0000","last_clean":"2024-09-13T18:57:36.120773+0000","last_became_active":"2024-09-13T18:27:22.610326+0000","last_became_peered":"2024-09-13T18:27:22.610326+0000","last_unstale":"2024-09-13T18:57:36.120773+0000","last_undegraded":"2024-09-13T18:57:36.120773+0000","last_fullsized":"2024-09-13T18:57:36.120773+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:36.120741+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:36.120741+0000","last_clean_scrub_stamp":"2024-09-13T18:57:36.120741+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:03.902589+0000","scrub_duration":0.0026569660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.167920+0000","last_change":"2024-09-13T18:57:27.167920+0000","last_active":"2024-09-13T18:57:27.167920+0000","last_peered":"2024-09-13T18:57:27.167920+0000","last_clean":"2024-09-13T18:57:27.167920+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:57:27.167920+0000","last_undegraded":"2024-09-13T18:57:27.167920+0000","last_fullsized":"2024-09-13T18:57:27.167920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:27.167690+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:27.167690+0000","last_clean_scrub_stamp":"2024-09-13T18:57:27.167690+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:44.845487+0000","scrub_duration":0.0066826000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"43'5174","reported_seq":5254,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:50.325810+0000","last_change":"2024-09-13T18:57:50.325810+0000","last_active":"2024-09-13T18:57:50.325810+0000","last_peered":"2024-09-13T18:57:50.325810+0000","last_clean":"2024-09-13T18:57:50.325810+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:57:50.325810+0000","last_undegraded":"2024-09-13T18:57:50.325810+0000","last_fullsized":"2024-09-13T18:57:50.325810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5174","last_scrub_stamp":"2024-09-13T18:57:50.325606+0000","last_deep_scrub":"43'5174","last_deep_scrub_stamp":"2024-09-13T18:57:50.325606+0000","last_clean_scrub_stamp":"2024-09-13T18:57:50.325606+0000","objects_scrubbed":0,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:33:51.857590+0000","scrub_duration":0.032878729000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":22830,"num_write_kb":472029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"43'11","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:59.101489+0000","last_change":"2024-09-13T18:57:59.101489+0000","last_active":"2024-09-13T18:57:59.101489+0000","last_peered":"2024-09-13T18:57:59.101489+0000","last_clean":"2024-09-13T18:57:59.101489+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:57:59.101489+0000","last_undegraded":"2024-09-13T18:57:59.101489+0000","last_fullsized":"2024-09-13T18:57:59.101489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:59.101212+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:59.101212+0000","last_clean_scrub_stamp":"2024-09-13T18:57:59.101212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:06.288499+0000","scrub_duration":0.003501742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"43'10","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.058571+0000","last_change":"2024-09-13T18:57:25.103279+0000","last_active":"2024-09-13T18:57:28.058571+0000","last_peered":"2024-09-13T18:57:28.058571+0000","last_clean":"2024-09-13T18:57:28.058571+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:57:28.058571+0000","last_undegraded":"2024-09-13T18:57:28.058571+0000","last_fullsized":"2024-09-13T18:57:28.058571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:25.103144+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:25.103144+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.103144+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.183107+0000","scrub_duration":0.013012788000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"43'4987","reported_seq":5060,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:04.306952+0000","last_change":"2024-09-13T18:58:04.307122+0000","last_active":"2024-09-13T18:58:04.306952+0000","last_peered":"2024-09-13T18:58:04.306952+0000","last_clean":"2024-09-13T18:58:04.306952+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:58:04.306952+0000","last_undegraded":"2024-09-13T18:58:04.306952+0000","last_fullsized":"2024-09-13T18:58:04.306952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4987","last_scrub_stamp":"2024-09-13T18:58:04.306926+0000","last_deep_scrub":"43'4987","last_deep_scrub_stamp":"2024-09-13T18:58:04.306926+0000","last_clean_scrub_stamp":"2024-09-13T18:58:04.306926+0000","objects_scrubbed":0,"log_size":4987,"log_dups_size":0,"ondisk_log_size":4987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:53:40.632719+0000","scrub_duration":0.035690187999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":0,"num_write":22775,"num_write_kb":463670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"43'5","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:58.077320+0000","last_change":"2024-09-13T18:57:58.077320+0000","last_active":"2024-09-13T18:57:58.077320+0000","last_peered":"2024-09-13T18:57:58.077320+0000","last_clean":"2024-09-13T18:57:58.077320+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:57:58.077320+0000","last_undegraded":"2024-09-13T18:57:58.077320+0000","last_fullsized":"2024-09-13T18:57:58.077320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:58.077036+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:58.077036+0000","last_clean_scrub_stamp":"2024-09-13T18:57:58.077036+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:31.127665+0000","scrub_duration":0.0029704359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"43'12","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:11.025964+0000","last_change":"2024-09-13T18:57:11.025964+0000","last_active":"2024-09-13T18:57:11.025964+0000","last_peered":"2024-09-13T18:57:11.025964+0000","last_clean":"2024-09-13T18:57:11.025964+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:57:11.025964+0000","last_undegraded":"2024-09-13T18:57:11.025964+0000","last_fullsized":"2024-09-13T18:57:11.025964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:11.025748+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:11.025748+0000","last_clean_scrub_stamp":"2024-09-13T18:57:11.025748+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:22:07.957036+0000","scrub_duration":0.0065355530000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"43'5147","reported_seq":5224,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:51.917449+0000","last_change":"2024-09-13T18:57:51.917748+0000","last_active":"2024-09-13T18:57:51.917449+0000","last_peered":"2024-09-13T18:57:51.917449+0000","last_clean":"2024-09-13T18:57:51.917449+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:57:51.917449+0000","last_undegraded":"2024-09-13T18:57:51.917449+0000","last_fullsized":"2024-09-13T18:57:51.917449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5147","last_scrub_stamp":"2024-09-13T18:57:51.917380+0000","last_deep_scrub":"43'5147","last_deep_scrub_stamp":"2024-09-13T18:57:51.917380+0000","last_clean_scrub_stamp":"2024-09-13T18:57:51.917380+0000","objects_scrubbed":0,"log_size":5147,"log_dups_size":0,"ondisk_log_size":5147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:11:21.117712+0000","scrub_duration":0.041709001000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":22715,"num_write_kb":463382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"43'12","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:56.116016+0000","last_change":"2024-09-13T18:57:56.116316+0000","last_active":"2024-09-13T18:57:56.116016+0000","last_peered":"2024-09-13T18:57:56.116016+0000","last_clean":"2024-09-13T18:57:56.116016+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:57:56.116016+0000","last_undegraded":"2024-09-13T18:57:56.116016+0000","last_fullsized":"2024-09-13T18:57:56.116016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:57:56.115968+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:57:56.115968+0000","last_clean_scrub_stamp":"2024-09-13T18:57:56.115968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:35.297035+0000","scrub_duration":0.0033167280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"43'32","reported_seq":149,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.902980+0000","last_change":"2024-09-13T18:57:20.154497+0000","last_active":"2024-09-13T18:57:24.902980+0000","last_peered":"2024-09-13T18:57:24.902980+0000","last_clean":"2024-09-13T18:57:24.902980+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:57:24.902980+0000","last_undegraded":"2024-09-13T18:57:24.902980+0000","last_fullsized":"2024-09-13T18:57:24.902980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'29","last_scrub_stamp":"2024-09-13T18:57:20.154238+0000","last_deep_scrub":"43'29","last_deep_scrub_stamp":"2024-09-13T18:57:20.154238+0000","last_clean_scrub_stamp":"2024-09-13T18:57:20.154238+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:10.657893+0000","scrub_duration":0.013053932000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":158,"num_read_kb":157,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"43'6074","reported_seq":6160,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:55.319773+0000","last_change":"2024-09-13T18:57:55.319773+0000","last_active":"2024-09-13T18:57:55.319773+0000","last_peered":"2024-09-13T18:57:55.319773+0000","last_clean":"2024-09-13T18:57:55.319773+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:57:55.319773+0000","last_undegraded":"2024-09-13T18:57:55.319773+0000","last_fullsized":"2024-09-13T18:57:55.319773+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6074","last_scrub_stamp":"2024-09-13T18:57:55.319608+0000","last_deep_scrub":"43'6074","last_deep_scrub_stamp":"2024-09-13T18:57:55.319608+0000","last_clean_scrub_stamp":"2024-09-13T18:57:55.319608+0000","objects_scrubbed":0,"log_size":6074,"log_dups_size":0,"ondisk_log_size":6074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:12.572658+0000","scrub_duration":0.041949826000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":27740,"num_write_kb":536907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"43'13","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:32.703996+0000","last_change":"2024-09-13T18:57:32.704206+0000","last_active":"2024-09-13T18:57:32.703996+0000","last_peered":"2024-09-13T18:57:32.703996+0000","last_clean":"2024-09-13T18:57:32.703996+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:57:32.703996+0000","last_undegraded":"2024-09-13T18:57:32.703996+0000","last_fullsized":"2024-09-13T18:57:32.703996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:32.703957+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:32.703957+0000","last_clean_scrub_stamp":"2024-09-13T18:57:32.703957+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:39.171311+0000","scrub_duration":0.018593416000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"43'16","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.908758+0000","last_change":"2024-09-13T18:57:21.771790+0000","last_active":"2024-09-13T18:57:24.908758+0000","last_peered":"2024-09-13T18:57:24.908758+0000","last_clean":"2024-09-13T18:57:24.908758+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:57:24.908758+0000","last_undegraded":"2024-09-13T18:57:24.908758+0000","last_fullsized":"2024-09-13T18:57:24.908758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:21.771519+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:21.771519+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.771519+0000","objects_scrubbed":2,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:44:46.293266+0000","scrub_duration":0.0083382200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1620,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"43'6155","reported_seq":6234,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:02.974017+0000","last_change":"2024-09-13T18:58:02.974017+0000","last_active":"2024-09-13T18:58:02.974017+0000","last_peered":"2024-09-13T18:58:02.974017+0000","last_clean":"2024-09-13T18:58:02.974017+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:58:02.974017+0000","last_undegraded":"2024-09-13T18:58:02.974017+0000","last_fullsized":"2024-09-13T18:58:02.974017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6155","last_scrub_stamp":"2024-09-13T18:58:02.973843+0000","last_deep_scrub":"43'6155","last_deep_scrub_stamp":"2024-09-13T18:58:02.973843+0000","last_clean_scrub_stamp":"2024-09-13T18:58:02.973843+0000","objects_scrubbed":0,"log_size":6155,"log_dups_size":0,"ondisk_log_size":6155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:07.662090+0000","scrub_duration":0.049362212000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":28268,"num_write_kb":551294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"43'15","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:03.046311+0000","last_change":"2024-09-13T18:58:03.046420+0000","last_active":"2024-09-13T18:58:03.046311+0000","last_peered":"2024-09-13T18:58:03.046311+0000","last_clean":"2024-09-13T18:58:03.046311+0000","last_became_active":"2024-09-13T18:27:22.609101+0000","last_became_peered":"2024-09-13T18:27:22.609101+0000","last_unstale":"2024-09-13T18:58:03.046311+0000","last_undegraded":"2024-09-13T18:58:03.046311+0000","last_fullsized":"2024-09-13T18:58:03.046311+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'15","last_scrub_stamp":"2024-09-13T18:58:03.046278+0000","last_deep_scrub":"43'15","last_deep_scrub_stamp":"2024-09-13T18:58:03.046278+0000","last_clean_scrub_stamp":"2024-09-13T18:58:03.046278+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:52:09.099898+0000","scrub_duration":0.00304259,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"43'14","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:35.005652+0000","last_change":"2024-09-13T18:57:35.005769+0000","last_active":"2024-09-13T18:57:35.005652+0000","last_peered":"2024-09-13T18:57:35.005652+0000","last_clean":"2024-09-13T18:57:35.005652+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:57:35.005652+0000","last_undegraded":"2024-09-13T18:57:35.005652+0000","last_fullsized":"2024-09-13T18:57:35.005652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:35.005633+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:35.005633+0000","last_clean_scrub_stamp":"2024-09-13T18:57:35.005633+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:24.849838+0000","scrub_duration":0.0050291620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"43'5953","reported_seq":6024,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.498590+0000","last_change":"2024-09-13T18:26:36.406371+0000","last_active":"2024-09-13T18:57:38.498590+0000","last_peered":"2024-09-13T18:57:38.498590+0000","last_clean":"2024-09-13T18:57:38.498590+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:57:38.498590+0000","last_undegraded":"2024-09-13T18:57:38.498590+0000","last_fullsized":"2024-09-13T18:57:38.498590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:35.350751+0000","last_clean_scrub_stamp":"2024-09-13T18:26:35.350751+0000","objects_scrubbed":0,"log_size":5953,"log_dups_size":0,"ondisk_log_size":5953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27125,"num_write_kb":550847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"43'15","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:34.907335+0000","last_change":"2024-09-13T18:57:25.418872+0000","last_active":"2024-09-13T18:57:34.907335+0000","last_peered":"2024-09-13T18:57:34.907335+0000","last_clean":"2024-09-13T18:57:34.907335+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:57:34.907335+0000","last_undegraded":"2024-09-13T18:57:34.907335+0000","last_fullsized":"2024-09-13T18:57:34.907335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:57:25.418501+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:57:25.418501+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.418501+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:32.901503+0000","scrub_duration":0.003478723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"43'20","reported_seq":111,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.419225+0000","last_change":"2024-09-13T18:57:13.300861+0000","last_active":"2024-09-13T18:57:26.419225+0000","last_peered":"2024-09-13T18:57:26.419225+0000","last_clean":"2024-09-13T18:57:26.419225+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:57:26.419225+0000","last_undegraded":"2024-09-13T18:57:26.419225+0000","last_fullsized":"2024-09-13T18:57:26.419225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:13.300701+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:13.300701+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.300701+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:09.337133+0000","scrub_duration":0.0071490659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":93,"num_read_kb":112,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"43'5303","reported_seq":5381,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:02.265627+0000","last_change":"2024-09-13T18:58:02.265627+0000","last_active":"2024-09-13T18:58:02.265627+0000","last_peered":"2024-09-13T18:58:02.265627+0000","last_clean":"2024-09-13T18:58:02.265627+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:58:02.265627+0000","last_undegraded":"2024-09-13T18:58:02.265627+0000","last_fullsized":"2024-09-13T18:58:02.265627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5303","last_scrub_stamp":"2024-09-13T18:58:02.265298+0000","last_deep_scrub":"43'5303","last_deep_scrub_stamp":"2024-09-13T18:58:02.265298+0000","last_clean_scrub_stamp":"2024-09-13T18:58:02.265298+0000","objects_scrubbed":0,"log_size":5303,"log_dups_size":0,"ondisk_log_size":5303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:41.041583+0000","scrub_duration":0.032593770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":23240,"num_write_kb":490059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"43'12","reported_seq":58,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:36.522619+0000","last_change":"2024-09-13T18:26:32.362858+0000","last_active":"2024-09-13T18:57:36.522619+0000","last_peered":"2024-09-13T18:57:36.522619+0000","last_clean":"2024-09-13T18:57:36.522619+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:57:36.522619+0000","last_undegraded":"2024-09-13T18:57:36.522619+0000","last_fullsized":"2024-09-13T18:57:36.522619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:26:31.936605+0000","last_clean_scrub_stamp":"2024-09-13T18:26:31.936605+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"43'389","reported_seq":488,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.908598+0000","last_change":"2024-09-13T18:57:13.774570+0000","last_active":"2024-09-13T18:57:24.908598+0000","last_peered":"2024-09-13T18:57:24.908598+0000","last_clean":"2024-09-13T18:57:24.908598+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:57:24.908598+0000","last_undegraded":"2024-09-13T18:57:24.908598+0000","last_fullsized":"2024-09-13T18:57:24.908598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'387","last_scrub_stamp":"2024-09-13T18:57:13.774429+0000","last_deep_scrub":"43'387","last_deep_scrub_stamp":"2024-09-13T18:57:13.774429+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.774429+0000","objects_scrubbed":5,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:38.043525+0000","scrub_duration":0.046187055999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":93,"num_read_kb":20568,"num_write":383,"num_write_kb":4318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"43'5475","reported_seq":5553,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:59.197285+0000","last_change":"2024-09-13T18:57:59.197483+0000","last_active":"2024-09-13T18:57:59.197285+0000","last_peered":"2024-09-13T18:57:59.197285+0000","last_clean":"2024-09-13T18:57:59.197285+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:57:59.197285+0000","last_undegraded":"2024-09-13T18:57:59.197285+0000","last_fullsized":"2024-09-13T18:57:59.197285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5475","last_scrub_stamp":"2024-09-13T18:57:59.197261+0000","last_deep_scrub":"43'5475","last_deep_scrub_stamp":"2024-09-13T18:57:59.197261+0000","last_clean_scrub_stamp":"2024-09-13T18:57:59.197261+0000","objects_scrubbed":0,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:37.225584+0000","scrub_duration":0.040786403999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":25034,"num_write_kb":485267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"43'10","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.055388+0000","last_change":"2024-09-13T18:57:16.962251+0000","last_active":"2024-09-13T18:57:28.055388+0000","last_peered":"2024-09-13T18:57:28.055388+0000","last_clean":"2024-09-13T18:57:28.055388+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:57:28.055388+0000","last_undegraded":"2024-09-13T18:57:28.055388+0000","last_fullsized":"2024-09-13T18:57:28.055388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:16.961847+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:16.961847+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.961847+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:54.684373+0000","scrub_duration":0.0040550839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"43'66","reported_seq":141,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.530024+0000","last_change":"2024-09-13T18:57:34.318022+0000","last_active":"2024-09-13T18:57:39.530024+0000","last_peered":"2024-09-13T18:57:39.530024+0000","last_clean":"2024-09-13T18:57:39.530024+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:57:39.530024+0000","last_undegraded":"2024-09-13T18:57:39.530024+0000","last_fullsized":"2024-09-13T18:57:39.530024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'65","last_scrub_stamp":"2024-09-13T18:57:34.317748+0000","last_deep_scrub":"43'65","last_deep_scrub_stamp":"2024-09-13T18:57:34.317748+0000","last_clean_scrub_stamp":"2024-09-13T18:57:34.317748+0000","objects_scrubbed":3,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:47.140265+0000","scrub_duration":0.0064888469999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":65,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"43'5962","reported_seq":6044,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:45.941460+0000","last_change":"2024-09-13T18:57:45.941610+0000","last_active":"2024-09-13T18:57:45.941460+0000","last_peered":"2024-09-13T18:57:45.941460+0000","last_clean":"2024-09-13T18:57:45.941460+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:57:45.941460+0000","last_undegraded":"2024-09-13T18:57:45.941460+0000","last_fullsized":"2024-09-13T18:57:45.941460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5962","last_scrub_stamp":"2024-09-13T18:57:45.941438+0000","last_deep_scrub":"43'5962","last_deep_scrub_stamp":"2024-09-13T18:57:45.941438+0000","last_clean_scrub_stamp":"2024-09-13T18:57:45.941438+0000","objects_scrubbed":0,"log_size":5962,"log_dups_size":0,"ondisk_log_size":5962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:34.601990+0000","scrub_duration":0.038953110999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27473,"num_write_kb":527755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"43'16","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:32.414926+0000","last_change":"2024-09-13T18:57:32.414926+0000","last_active":"2024-09-13T18:57:32.414926+0000","last_peered":"2024-09-13T18:57:32.414926+0000","last_clean":"2024-09-13T18:57:32.414926+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:57:32.414926+0000","last_undegraded":"2024-09-13T18:57:32.414926+0000","last_fullsized":"2024-09-13T18:57:32.414926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:32.414785+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:32.414785+0000","last_clean_scrub_stamp":"2024-09-13T18:57:32.414785+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:58:20.923011+0000","scrub_duration":0.0019847889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"43'6","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.921362+0000","last_change":"2024-09-13T18:57:18.948854+0000","last_active":"2024-09-13T18:57:24.921362+0000","last_peered":"2024-09-13T18:57:24.921362+0000","last_clean":"2024-09-13T18:57:24.921362+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:57:24.921362+0000","last_undegraded":"2024-09-13T18:57:24.921362+0000","last_fullsized":"2024-09-13T18:57:24.921362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4","last_scrub_stamp":"2024-09-13T18:57:18.948753+0000","last_deep_scrub":"43'4","last_deep_scrub_stamp":"2024-09-13T18:57:18.948753+0000","last_clean_scrub_stamp":"2024-09-13T18:57:18.948753+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:24:26.277291+0000","scrub_duration":0.0026056769999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"43'5502","reported_seq":5575,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:58.974602+0000","last_change":"2024-09-13T18:57:58.974930+0000","last_active":"2024-09-13T18:57:58.974602+0000","last_peered":"2024-09-13T18:57:58.974602+0000","last_clean":"2024-09-13T18:57:58.974602+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:57:58.974602+0000","last_undegraded":"2024-09-13T18:57:58.974602+0000","last_fullsized":"2024-09-13T18:57:58.974602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5502","last_scrub_stamp":"2024-09-13T18:57:58.974532+0000","last_deep_scrub":"43'5502","last_deep_scrub_stamp":"2024-09-13T18:57:58.974532+0000","last_clean_scrub_stamp":"2024-09-13T18:57:58.974532+0000","objects_scrubbed":0,"log_size":5502,"log_dups_size":0,"ondisk_log_size":5502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:31:36.209865+0000","scrub_duration":0.045783550999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":25045,"num_write_kb":489215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"43'13","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:45.146635+0000","last_change":"2024-09-13T18:57:45.146635+0000","last_active":"2024-09-13T18:57:45.146635+0000","last_peered":"2024-09-13T18:57:45.146635+0000","last_clean":"2024-09-13T18:57:45.146635+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:57:45.146635+0000","last_undegraded":"2024-09-13T18:57:45.146635+0000","last_fullsized":"2024-09-13T18:57:45.146635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:45.146357+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:45.146357+0000","last_clean_scrub_stamp":"2024-09-13T18:57:45.146357+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:12.892350+0000","scrub_duration":0.0033819689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"43'19","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.922065+0000","last_change":"2024-09-13T18:57:22.765983+0000","last_active":"2024-09-13T18:57:24.922065+0000","last_peered":"2024-09-13T18:57:24.922065+0000","last_clean":"2024-09-13T18:57:24.922065+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:57:24.922065+0000","last_undegraded":"2024-09-13T18:57:24.922065+0000","last_fullsized":"2024-09-13T18:57:24.922065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'18","last_scrub_stamp":"2024-09-13T18:57:22.765727+0000","last_deep_scrub":"43'18","last_deep_scrub_stamp":"2024-09-13T18:57:22.765727+0000","last_clean_scrub_stamp":"2024-09-13T18:57:22.765727+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:04.965728+0000","scrub_duration":0.0065666600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":18,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"43'5179","reported_seq":5259,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:53.977550+0000","last_change":"2024-09-13T18:57:53.977664+0000","last_active":"2024-09-13T18:57:53.977550+0000","last_peered":"2024-09-13T18:57:53.977550+0000","last_clean":"2024-09-13T18:57:53.977550+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:57:53.977550+0000","last_undegraded":"2024-09-13T18:57:53.977550+0000","last_fullsized":"2024-09-13T18:57:53.977550+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5179","last_scrub_stamp":"2024-09-13T18:57:53.977529+0000","last_deep_scrub":"43'5179","last_deep_scrub_stamp":"2024-09-13T18:57:53.977529+0000","last_clean_scrub_stamp":"2024-09-13T18:57:53.977529+0000","objects_scrubbed":0,"log_size":5179,"log_dups_size":0,"ondisk_log_size":5179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:35:59.485208+0000","scrub_duration":0.041690597000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":23340,"num_write_kb":507027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"43'6189","reported_seq":6271,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:57.311076+0000","last_change":"2024-09-13T18:57:57.311297+0000","last_active":"2024-09-13T18:57:57.311076+0000","last_peered":"2024-09-13T18:57:57.311076+0000","last_clean":"2024-09-13T18:57:57.311076+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:57:57.311076+0000","last_undegraded":"2024-09-13T18:57:57.311076+0000","last_fullsized":"2024-09-13T18:57:57.311076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6189","last_scrub_stamp":"2024-09-13T18:57:57.311029+0000","last_deep_scrub":"43'6189","last_deep_scrub_stamp":"2024-09-13T18:57:57.311029+0000","last_clean_scrub_stamp":"2024-09-13T18:57:57.311029+0000","objects_scrubbed":0,"log_size":6189,"log_dups_size":0,"ondisk_log_size":6189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:14.650943+0000","scrub_duration":0.039998120999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":28563,"num_write_kb":581141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"43'10","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:52.167207+0000","last_change":"2024-09-13T18:57:52.167207+0000","last_active":"2024-09-13T18:57:52.167207+0000","last_peered":"2024-09-13T18:57:52.167207+0000","last_clean":"2024-09-13T18:57:52.167207+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:57:52.167207+0000","last_undegraded":"2024-09-13T18:57:52.167207+0000","last_fullsized":"2024-09-13T18:57:52.167207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:52.166714+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:52.166714+0000","last_clean_scrub_stamp":"2024-09-13T18:57:52.166714+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:23.336388+0000","scrub_duration":0.0037465770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"43'8","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:21.964628+0000","last_change":"2024-09-13T18:57:21.964628+0000","last_active":"2024-09-13T18:57:21.964628+0000","last_peered":"2024-09-13T18:57:21.964628+0000","last_clean":"2024-09-13T18:57:21.964628+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:57:21.964628+0000","last_undegraded":"2024-09-13T18:57:21.964628+0000","last_fullsized":"2024-09-13T18:57:21.964628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:21.964364+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:21.964364+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.964364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:31:05.131047+0000","scrub_duration":0.003227541,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"43'5646","reported_seq":5731,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:06.699074+0000","last_change":"2024-09-13T18:58:06.699213+0000","last_active":"2024-09-13T18:58:06.699074+0000","last_peered":"2024-09-13T18:58:06.699074+0000","last_clean":"2024-09-13T18:58:06.699074+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:58:06.699074+0000","last_undegraded":"2024-09-13T18:58:06.699074+0000","last_fullsized":"2024-09-13T18:58:06.699074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5646","last_scrub_stamp":"2024-09-13T18:58:06.699052+0000","last_deep_scrub":"43'5646","last_deep_scrub_stamp":"2024-09-13T18:58:06.699052+0000","last_clean_scrub_stamp":"2024-09-13T18:58:06.699052+0000","objects_scrubbed":0,"log_size":5646,"log_dups_size":0,"ondisk_log_size":5646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:01.384154+0000","scrub_duration":0.038574091999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":25607,"num_write_kb":520259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"43'19","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:42.857256+0000","last_change":"2024-09-13T18:57:42.857256+0000","last_active":"2024-09-13T18:57:42.857256+0000","last_peered":"2024-09-13T18:57:42.857256+0000","last_clean":"2024-09-13T18:57:42.857256+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:57:42.857256+0000","last_undegraded":"2024-09-13T18:57:42.857256+0000","last_fullsized":"2024-09-13T18:57:42.857256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'19","last_scrub_stamp":"2024-09-13T18:57:42.857115+0000","last_deep_scrub":"43'19","last_deep_scrub_stamp":"2024-09-13T18:57:42.857115+0000","last_clean_scrub_stamp":"2024-09-13T18:57:42.857115+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:32.087665+0000","scrub_duration":0.0022791909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"43'6","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:21.960612+0000","last_change":"2024-09-13T18:57:21.960612+0000","last_active":"2024-09-13T18:57:21.960612+0000","last_peered":"2024-09-13T18:57:21.960612+0000","last_clean":"2024-09-13T18:57:21.960612+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:57:21.960612+0000","last_undegraded":"2024-09-13T18:57:21.960612+0000","last_fullsized":"2024-09-13T18:57:21.960612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:21.960203+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:21.960203+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.960203+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:59:06.847239+0000","scrub_duration":0.006849478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"43'13","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.647844+0000","last_change":"2024-09-13T18:57:16.794490+0000","last_active":"2024-09-13T18:57:27.647844+0000","last_peered":"2024-09-13T18:57:27.647844+0000","last_clean":"2024-09-13T18:57:27.647844+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:57:27.647844+0000","last_undegraded":"2024-09-13T18:57:27.647844+0000","last_fullsized":"2024-09-13T18:57:27.647844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:16.794241+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:16.794241+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.794241+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:35.699392+0000","scrub_duration":0.0056238700000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"43'10","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:42.246148+0000","last_change":"2024-09-13T18:57:42.246326+0000","last_active":"2024-09-13T18:57:42.246148+0000","last_peered":"2024-09-13T18:57:42.246148+0000","last_clean":"2024-09-13T18:57:42.246148+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:57:42.246148+0000","last_undegraded":"2024-09-13T18:57:42.246148+0000","last_fullsized":"2024-09-13T18:57:42.246148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:42.246101+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:42.246101+0000","last_clean_scrub_stamp":"2024-09-13T18:57:42.246101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:54.930475+0000","scrub_duration":0.0033923690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"43'4199","reported_seq":4275,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.531206+0000","last_change":"2024-09-13T18:57:32.974532+0000","last_active":"2024-09-13T18:57:38.531206+0000","last_peered":"2024-09-13T18:57:38.531206+0000","last_clean":"2024-09-13T18:57:38.531206+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:57:38.531206+0000","last_undegraded":"2024-09-13T18:57:38.531206+0000","last_fullsized":"2024-09-13T18:57:38.531206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4177","last_scrub_stamp":"2024-09-13T18:57:32.974351+0000","last_deep_scrub":"43'4177","last_deep_scrub_stamp":"2024-09-13T18:57:32.974351+0000","last_clean_scrub_stamp":"2024-09-13T18:57:32.974351+0000","objects_scrubbed":11,"log_size":4199,"log_dups_size":0,"ondisk_log_size":4199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:34.816537+0000","scrub_duration":0.80893831299999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":19219,"num_write_kb":424944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"43'759","reported_seq":843,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.910037+0000","last_change":"2024-09-13T18:57:28.910129+0000","last_active":"2024-09-13T18:57:28.910037+0000","last_peered":"2024-09-13T18:57:28.910037+0000","last_clean":"2024-09-13T18:57:28.910037+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:57:28.910037+0000","last_undegraded":"2024-09-13T18:57:28.910037+0000","last_fullsized":"2024-09-13T18:57:28.910037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'759","last_scrub_stamp":"2024-09-13T18:57:28.910019+0000","last_deep_scrub":"43'759","last_deep_scrub_stamp":"2024-09-13T18:57:28.910019+0000","last_clean_scrub_stamp":"2024-09-13T18:57:28.910019+0000","objects_scrubbed":3,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:04.313816+0000","scrub_duration":0.042299119000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196273,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":19627,"num_write":735,"num_write_kb":4345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"43'13","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.967148+0000","last_change":"2024-09-13T18:57:20.986740+0000","last_active":"2024-09-13T18:57:24.967148+0000","last_peered":"2024-09-13T18:57:24.967148+0000","last_clean":"2024-09-13T18:57:24.967148+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:57:24.967148+0000","last_undegraded":"2024-09-13T18:57:24.967148+0000","last_fullsized":"2024-09-13T18:57:24.967148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:20.986473+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:20.986473+0000","last_clean_scrub_stamp":"2024-09-13T18:57:20.986473+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:56.996793+0000","scrub_duration":0.0033945970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"43'6014","reported_seq":6094,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:54.990721+0000","last_change":"2024-09-13T18:57:54.990862+0000","last_active":"2024-09-13T18:57:54.990721+0000","last_peered":"2024-09-13T18:57:54.990721+0000","last_clean":"2024-09-13T18:57:54.990721+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:57:54.990721+0000","last_undegraded":"2024-09-13T18:57:54.990721+0000","last_fullsized":"2024-09-13T18:57:54.990721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6014","last_scrub_stamp":"2024-09-13T18:57:54.990699+0000","last_deep_scrub":"43'6014","last_deep_scrub_stamp":"2024-09-13T18:57:54.990699+0000","last_clean_scrub_stamp":"2024-09-13T18:57:54.990699+0000","objects_scrubbed":0,"log_size":6014,"log_dups_size":0,"ondisk_log_size":6014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:42:04.338574+0000","scrub_duration":0.044555913000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":26799,"num_write_kb":557751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"43'16","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:54.488901+0000","last_change":"2024-09-13T18:57:54.489110+0000","last_active":"2024-09-13T18:57:54.488901+0000","last_peered":"2024-09-13T18:57:54.488901+0000","last_clean":"2024-09-13T18:57:54.488901+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:57:54.488901+0000","last_undegraded":"2024-09-13T18:57:54.488901+0000","last_fullsized":"2024-09-13T18:57:54.488901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:54.488848+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:54.488848+0000","last_clean_scrub_stamp":"2024-09-13T18:57:54.488848+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:43:01.499699+0000","scrub_duration":0.0035375490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"43'12","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:31.097096+0000","last_change":"2024-09-13T18:57:31.097096+0000","last_active":"2024-09-13T18:57:31.097096+0000","last_peered":"2024-09-13T18:57:31.097096+0000","last_clean":"2024-09-13T18:57:31.097096+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:57:31.097096+0000","last_undegraded":"2024-09-13T18:57:31.097096+0000","last_fullsized":"2024-09-13T18:57:31.097096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:57:31.096973+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:57:31.096973+0000","last_clean_scrub_stamp":"2024-09-13T18:57:31.096973+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:08.952276+0000","scrub_duration":0.0027320500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"43'17","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.122971+0000","last_change":"2024-09-13T18:57:39.122971+0000","last_active":"2024-09-13T18:57:39.122971+0000","last_peered":"2024-09-13T18:57:39.122971+0000","last_clean":"2024-09-13T18:57:39.122971+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:57:39.122971+0000","last_undegraded":"2024-09-13T18:57:39.122971+0000","last_fullsized":"2024-09-13T18:57:39.122971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'17","last_scrub_stamp":"2024-09-13T18:57:39.122544+0000","last_deep_scrub":"43'17","last_deep_scrub_stamp":"2024-09-13T18:57:39.122544+0000","last_clean_scrub_stamp":"2024-09-13T18:57:39.122544+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:32.935831+0000","scrub_duration":0.0032146100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"43'10","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:45.307991+0000","last_change":"2024-09-13T18:57:45.307991+0000","last_active":"2024-09-13T18:57:45.307991+0000","last_peered":"2024-09-13T18:57:45.307991+0000","last_clean":"2024-09-13T18:57:45.307991+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:57:45.307991+0000","last_undegraded":"2024-09-13T18:57:45.307991+0000","last_fullsized":"2024-09-13T18:57:45.307991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:45.307762+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:45.307762+0000","last_clean_scrub_stamp":"2024-09-13T18:57:45.307762+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:12.874226+0000","scrub_duration":0.0028431099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"43'10","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.421715+0000","last_change":"2024-09-13T18:57:19.006871+0000","last_active":"2024-09-13T18:57:26.421715+0000","last_peered":"2024-09-13T18:57:26.421715+0000","last_clean":"2024-09-13T18:57:26.421715+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:57:26.421715+0000","last_undegraded":"2024-09-13T18:57:26.421715+0000","last_fullsized":"2024-09-13T18:57:26.421715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'7","last_scrub_stamp":"2024-09-13T18:57:19.006675+0000","last_deep_scrub":"29'7","last_deep_scrub_stamp":"2024-09-13T18:57:19.006675+0000","last_clean_scrub_stamp":"2024-09-13T18:57:19.006675+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:53.159267+0000","scrub_duration":0.012523802000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"43'10","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:43.268378+0000","last_change":"2024-09-13T18:57:43.268610+0000","last_active":"2024-09-13T18:57:43.268378+0000","last_peered":"2024-09-13T18:57:43.268378+0000","last_clean":"2024-09-13T18:57:43.268378+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:57:43.268378+0000","last_undegraded":"2024-09-13T18:57:43.268378+0000","last_fullsized":"2024-09-13T18:57:43.268378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:43.268329+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:43.268329+0000","last_clean_scrub_stamp":"2024-09-13T18:57:43.268329+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:56.169188+0000","scrub_duration":0.0028286909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"43'8","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.922074+0000","last_change":"2024-09-13T18:57:24.709839+0000","last_active":"2024-09-13T18:57:24.922074+0000","last_peered":"2024-09-13T18:57:24.922074+0000","last_clean":"2024-09-13T18:57:24.922074+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:57:24.922074+0000","last_undegraded":"2024-09-13T18:57:24.922074+0000","last_fullsized":"2024-09-13T18:57:24.922074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:24.709588+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:24.709588+0000","last_clean_scrub_stamp":"2024-09-13T18:57:24.709588+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:56.958314+0000","scrub_duration":0.0036738090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"43'13","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.862606+0000","last_change":"2024-09-13T18:57:17.997698+0000","last_active":"2024-09-13T18:57:24.862606+0000","last_peered":"2024-09-13T18:57:24.862606+0000","last_clean":"2024-09-13T18:57:24.862606+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:57:24.862606+0000","last_undegraded":"2024-09-13T18:57:24.862606+0000","last_fullsized":"2024-09-13T18:57:24.862606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:17.997340+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:17.997340+0000","last_clean_scrub_stamp":"2024-09-13T18:57:17.997340+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:07.106051+0000","scrub_duration":0.0035120389999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"43'10","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:36.044747+0000","last_change":"2024-09-13T18:57:36.044747+0000","last_active":"2024-09-13T18:57:36.044747+0000","last_peered":"2024-09-13T18:57:36.044747+0000","last_clean":"2024-09-13T18:57:36.044747+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:57:36.044747+0000","last_undegraded":"2024-09-13T18:57:36.044747+0000","last_fullsized":"2024-09-13T18:57:36.044747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:57:36.044597+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:57:36.044597+0000","last_clean_scrub_stamp":"2024-09-13T18:57:36.044597+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:40.677133+0000","scrub_duration":0.010560052,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"43'15","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.892530+0000","last_change":"2024-09-13T18:57:38.892530+0000","last_active":"2024-09-13T18:57:38.892530+0000","last_peered":"2024-09-13T18:57:38.892530+0000","last_clean":"2024-09-13T18:57:38.892530+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:57:38.892530+0000","last_undegraded":"2024-09-13T18:57:38.892530+0000","last_fullsized":"2024-09-13T18:57:38.892530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'15","last_scrub_stamp":"2024-09-13T18:57:38.892242+0000","last_deep_scrub":"43'15","last_deep_scrub_stamp":"2024-09-13T18:57:38.892242+0000","last_clean_scrub_stamp":"2024-09-13T18:57:38.892242+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:17.674671+0000","scrub_duration":0.0037659220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"43'11","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:58.186979+0000","last_change":"2024-09-13T18:57:58.186979+0000","last_active":"2024-09-13T18:57:58.186979+0000","last_peered":"2024-09-13T18:57:58.186979+0000","last_clean":"2024-09-13T18:57:58.186979+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:57:58.186979+0000","last_undegraded":"2024-09-13T18:57:58.186979+0000","last_fullsized":"2024-09-13T18:57:58.186979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:58.186729+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:58.186729+0000","last_clean_scrub_stamp":"2024-09-13T18:57:58.186729+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:41.470005+0000","scrub_duration":0.0030497200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"43'18","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:47.275068+0000","last_change":"2024-09-13T18:57:47.275068+0000","last_active":"2024-09-13T18:57:47.275068+0000","last_peered":"2024-09-13T18:57:47.275068+0000","last_clean":"2024-09-13T18:57:47.275068+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:57:47.275068+0000","last_undegraded":"2024-09-13T18:57:47.275068+0000","last_fullsized":"2024-09-13T18:57:47.275068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'18","last_scrub_stamp":"2024-09-13T18:57:47.274823+0000","last_deep_scrub":"43'18","last_deep_scrub_stamp":"2024-09-13T18:57:47.274823+0000","last_clean_scrub_stamp":"2024-09-13T18:57:47.274823+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:23:23.332126+0000","scrub_duration":0.0026113270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"43'11","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:51.484615+0000","last_change":"2024-09-13T18:57:51.484615+0000","last_active":"2024-09-13T18:57:51.484615+0000","last_peered":"2024-09-13T18:57:51.484615+0000","last_clean":"2024-09-13T18:57:51.484615+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:57:51.484615+0000","last_undegraded":"2024-09-13T18:57:51.484615+0000","last_fullsized":"2024-09-13T18:57:51.484615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:51.484333+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:51.484333+0000","last_clean_scrub_stamp":"2024-09-13T18:57:51.484333+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:35.675512+0000","scrub_duration":0.0032285579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"43'15","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:32.560841+0000","last_change":"2024-09-13T18:57:15.857341+0000","last_active":"2024-09-13T18:57:32.560841+0000","last_peered":"2024-09-13T18:57:32.560841+0000","last_clean":"2024-09-13T18:57:32.560841+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:57:32.560841+0000","last_undegraded":"2024-09-13T18:57:32.560841+0000","last_fullsized":"2024-09-13T18:57:32.560841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:15.857084+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:15.857084+0000","last_clean_scrub_stamp":"2024-09-13T18:57:15.857084+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:39.110251+0000","scrub_duration":0.0034250190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"43'18","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.034005+0000","last_change":"2024-09-13T18:57:27.062756+0000","last_active":"2024-09-13T18:57:28.034005+0000","last_peered":"2024-09-13T18:57:28.034005+0000","last_clean":"2024-09-13T18:57:28.034005+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:57:28.034005+0000","last_undegraded":"2024-09-13T18:57:28.034005+0000","last_fullsized":"2024-09-13T18:57:28.034005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'15","last_scrub_stamp":"2024-09-13T18:57:27.062495+0000","last_deep_scrub":"43'15","last_deep_scrub_stamp":"2024-09-13T18:57:27.062495+0000","last_clean_scrub_stamp":"2024-09-13T18:57:27.062495+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:05:17.012914+0000","scrub_duration":0.0040785700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"43'14","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:48.240470+0000","last_change":"2024-09-13T18:57:48.240666+0000","last_active":"2024-09-13T18:57:48.240470+0000","last_peered":"2024-09-13T18:57:48.240470+0000","last_clean":"2024-09-13T18:57:48.240470+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:57:48.240470+0000","last_undegraded":"2024-09-13T18:57:48.240470+0000","last_fullsized":"2024-09-13T18:57:48.240470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:48.240421+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:48.240421+0000","last_clean_scrub_stamp":"2024-09-13T18:57:48.240421+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:11.977729+0000","scrub_duration":0.0030695620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"43'12","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:43.177441+0000","last_change":"2024-09-13T18:57:43.177686+0000","last_active":"2024-09-13T18:57:43.177441+0000","last_peered":"2024-09-13T18:57:43.177441+0000","last_clean":"2024-09-13T18:57:43.177441+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:57:43.177441+0000","last_undegraded":"2024-09-13T18:57:43.177441+0000","last_fullsized":"2024-09-13T18:57:43.177441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:57:43.177387+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:57:43.177387+0000","last_clean_scrub_stamp":"2024-09-13T18:57:43.177387+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:30.629006+0000","scrub_duration":0.0029135400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"43'16","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:37.308644+0000","last_change":"2024-09-13T18:57:37.308764+0000","last_active":"2024-09-13T18:57:37.308644+0000","last_peered":"2024-09-13T18:57:37.308644+0000","last_clean":"2024-09-13T18:57:37.308644+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:57:37.308644+0000","last_undegraded":"2024-09-13T18:57:37.308644+0000","last_fullsized":"2024-09-13T18:57:37.308644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:37.308610+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:37.308610+0000","last_clean_scrub_stamp":"2024-09-13T18:57:37.308610+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:24.954579+0000","scrub_duration":0.020444231,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"43'17","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:34.387125+0000","last_change":"2024-09-13T18:57:34.387338+0000","last_active":"2024-09-13T18:57:34.387125+0000","last_peered":"2024-09-13T18:57:34.387125+0000","last_clean":"2024-09-13T18:57:34.387125+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:57:34.387125+0000","last_undegraded":"2024-09-13T18:57:34.387125+0000","last_fullsized":"2024-09-13T18:57:34.387125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'17","last_scrub_stamp":"2024-09-13T18:57:34.387073+0000","last_deep_scrub":"43'17","last_deep_scrub_stamp":"2024-09-13T18:57:34.387073+0000","last_clean_scrub_stamp":"2024-09-13T18:57:34.387073+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:16.405083+0000","scrub_duration":0.002417584,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"43'13","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:44.875900+0000","last_change":"2024-09-13T18:57:44.875900+0000","last_active":"2024-09-13T18:57:44.875900+0000","last_peered":"2024-09-13T18:57:44.875900+0000","last_clean":"2024-09-13T18:57:44.875900+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:57:44.875900+0000","last_undegraded":"2024-09-13T18:57:44.875900+0000","last_fullsized":"2024-09-13T18:57:44.875900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:44.875664+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:44.875664+0000","last_clean_scrub_stamp":"2024-09-13T18:57:44.875664+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:19.462107+0000","scrub_duration":0.0036327349999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"43'12","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:36.959082+0000","last_change":"2024-09-13T18:57:25.015223+0000","last_active":"2024-09-13T18:57:36.959082+0000","last_peered":"2024-09-13T18:57:36.959082+0000","last_clean":"2024-09-13T18:57:36.959082+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:57:36.959082+0000","last_undegraded":"2024-09-13T18:57:36.959082+0000","last_fullsized":"2024-09-13T18:57:36.959082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:25.015045+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:25.015045+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.015045+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:39.559328+0000","scrub_duration":0.013113774999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"43'13","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.778903+0000","last_change":"2024-09-13T18:57:30.778903+0000","last_active":"2024-09-13T18:57:30.778903+0000","last_peered":"2024-09-13T18:57:30.778903+0000","last_clean":"2024-09-13T18:57:30.778903+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:57:30.778903+0000","last_undegraded":"2024-09-13T18:57:30.778903+0000","last_fullsized":"2024-09-13T18:57:30.778903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:30.778791+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:30.778791+0000","last_clean_scrub_stamp":"2024-09-13T18:57:30.778791+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:34.242313+0000","scrub_duration":0.00175041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"43'13","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:55.513934+0000","last_change":"2024-09-13T18:57:55.513934+0000","last_active":"2024-09-13T18:57:55.513934+0000","last_peered":"2024-09-13T18:57:55.513934+0000","last_clean":"2024-09-13T18:57:55.513934+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:57:55.513934+0000","last_undegraded":"2024-09-13T18:57:55.513934+0000","last_fullsized":"2024-09-13T18:57:55.513934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:55.513649+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:55.513649+0000","last_clean_scrub_stamp":"2024-09-13T18:57:55.513649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:45.763242+0000","scrub_duration":0.0032588529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"43'16","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:01.133960+0000","last_change":"2024-09-13T18:58:01.133960+0000","last_active":"2024-09-13T18:58:01.133960+0000","last_peered":"2024-09-13T18:58:01.133960+0000","last_clean":"2024-09-13T18:58:01.133960+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:58:01.133960+0000","last_undegraded":"2024-09-13T18:58:01.133960+0000","last_fullsized":"2024-09-13T18:58:01.133960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:58:01.133537+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:58:01.133537+0000","last_clean_scrub_stamp":"2024-09-13T18:58:01.133537+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:06.020299+0000","scrub_duration":0.003245464,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"43'6","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:52.963821+0000","last_change":"2024-09-13T18:57:52.964032+0000","last_active":"2024-09-13T18:57:52.963821+0000","last_peered":"2024-09-13T18:57:52.963821+0000","last_clean":"2024-09-13T18:57:52.963821+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:57:52.963821+0000","last_undegraded":"2024-09-13T18:57:52.963821+0000","last_fullsized":"2024-09-13T18:57:52.963821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:52.963770+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:52.963770+0000","last_clean_scrub_stamp":"2024-09-13T18:57:52.963770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:59.872058+0000","scrub_duration":0.0035209389999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"43'9","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:07.047890+0000","last_change":"2024-09-13T18:58:07.048089+0000","last_active":"2024-09-13T18:58:07.047890+0000","last_peered":"2024-09-13T18:58:07.047890+0000","last_clean":"2024-09-13T18:58:07.047890+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:58:07.047890+0000","last_undegraded":"2024-09-13T18:58:07.047890+0000","last_fullsized":"2024-09-13T18:58:07.047890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:58:07.047843+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:58:07.047843+0000","last_clean_scrub_stamp":"2024-09-13T18:58:07.047843+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:00:30.056086+0000","scrub_duration":0.003715037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"43'11","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:01.217080+0000","last_change":"2024-09-13T18:58:01.217349+0000","last_active":"2024-09-13T18:58:01.217080+0000","last_peered":"2024-09-13T18:58:01.217080+0000","last_clean":"2024-09-13T18:58:01.217080+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:58:01.217080+0000","last_undegraded":"2024-09-13T18:58:01.217080+0000","last_fullsized":"2024-09-13T18:58:01.217080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:58:01.217028+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:58:01.217028+0000","last_clean_scrub_stamp":"2024-09-13T18:58:01.217028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:13:31.856600+0000","scrub_duration":0.0029497920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"43'13","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:05.140674+0000","last_change":"2024-09-13T18:58:05.141012+0000","last_active":"2024-09-13T18:58:05.140674+0000","last_peered":"2024-09-13T18:58:05.140674+0000","last_clean":"2024-09-13T18:58:05.140674+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:58:05.140674+0000","last_undegraded":"2024-09-13T18:58:05.140674+0000","last_fullsized":"2024-09-13T18:58:05.140674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:58:05.140565+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:58:05.140565+0000","last_clean_scrub_stamp":"2024-09-13T18:58:05.140565+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:10.286462+0000","scrub_duration":0.0034325750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"43'7","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:56.216315+0000","last_change":"2024-09-13T18:57:56.216516+0000","last_active":"2024-09-13T18:57:56.216315+0000","last_peered":"2024-09-13T18:57:56.216315+0000","last_clean":"2024-09-13T18:57:56.216315+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:57:56.216315+0000","last_undegraded":"2024-09-13T18:57:56.216315+0000","last_fullsized":"2024-09-13T18:57:56.216315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'7","last_scrub_stamp":"2024-09-13T18:57:56.216265+0000","last_deep_scrub":"43'7","last_deep_scrub_stamp":"2024-09-13T18:57:56.216265+0000","last_clean_scrub_stamp":"2024-09-13T18:57:56.216265+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:29.591142+0000","scrub_duration":0.0032929220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"43'11","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.862267+0000","last_change":"2024-09-13T18:57:11.323912+0000","last_active":"2024-09-13T18:57:24.862267+0000","last_peered":"2024-09-13T18:57:24.862267+0000","last_clean":"2024-09-13T18:57:24.862267+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:57:24.862267+0000","last_undegraded":"2024-09-13T18:57:24.862267+0000","last_fullsized":"2024-09-13T18:57:24.862267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:11.323767+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:11.323767+0000","last_clean_scrub_stamp":"2024-09-13T18:57:11.323767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:26.980778+0000","scrub_duration":0.0019515800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"43'6","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.252568+0000","last_change":"2024-09-13T18:57:39.252568+0000","last_active":"2024-09-13T18:57:39.252568+0000","last_peered":"2024-09-13T18:57:39.252568+0000","last_clean":"2024-09-13T18:57:39.252568+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:57:39.252568+0000","last_undegraded":"2024-09-13T18:57:39.252568+0000","last_fullsized":"2024-09-13T18:57:39.252568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:39.252320+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:39.252320+0000","last_clean_scrub_stamp":"2024-09-13T18:57:39.252320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:39.213931+0000","scrub_duration":0.002145554,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"43'9","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:06.094405+0000","last_change":"2024-09-13T18:58:06.094607+0000","last_active":"2024-09-13T18:58:06.094405+0000","last_peered":"2024-09-13T18:58:06.094405+0000","last_clean":"2024-09-13T18:58:06.094405+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:58:06.094405+0000","last_undegraded":"2024-09-13T18:58:06.094405+0000","last_fullsized":"2024-09-13T18:58:06.094405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:58:06.094356+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:58:06.094356+0000","last_clean_scrub_stamp":"2024-09-13T18:58:06.094356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:54.098829+0000","scrub_duration":0.0046552670000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":797,"num_read_kb":0,"num_write":803600,"num_write_kb":16319459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178039,"ondisk_log_size":178039,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":771,"ondisk_log_size":771,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":16507087,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1362,"num_read_kb":52080,"num_write":3742,"num_write_kb":17667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22716416,"data_stored":33014174,"data_compressed":2441304,"data_compressed_allocated":3506176,"data_compressed_original":13950976,"omap_allocated":0,"internal_metadata":0},"log_size":3641,"ondisk_log_size":3641,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509824,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1059364,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":1057150,"kb_avail":93312476,"statfs":{"total":96636764160,"available":95551975424,"internally_reserved":0,"allocated":2256896,"data_stored":4436700,"data_compressed":519875,"data_compressed_allocated":692224,"data_compressed_original":3125248,"omap_allocated":1593,"internal_metadata":1082522055},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":1.3600000000000001,"15min":0.93999999999999995},"min":{"1min":0.247,"5min":0.223,"15min":0.223},"max":{"1min":0.84199999999999997,"5min":10.43,"15min":10.43},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.889,"15min":1.1599999999999999},"min":{"1min":0.27100000000000002,"5min":0.245,"15min":0.245},"max":{"1min":4.3109999999999999,"5min":16.600999999999999,"15min":16.600999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":3.5910000000000002,"15min":1.849},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":5.8780000000000001,"5min":52.290999999999997,"15min":52.290999999999997},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":3.7690000000000001,"15min":1.9570000000000001},"min":{"1min":0.28199999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":5.4740000000000002,"5min":52.149000000000001,"15min":52.149000000000001},"last":0.83799999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":1.335,"15min":1.054},"min":{"1min":0.316,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.9399999999999999,"5min":14.933,"15min":14.933},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.264,"15min":1.052},"min":{"1min":0.35299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":7.266,"5min":9.2059999999999995,"15min":17.792000000000002},"last":0.77900000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":2.3849999999999998,"15min":1.4319999999999999},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":8.7910000000000004,"5min":15.622,"15min":15.622},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.343,"5min":2.71,"15min":1.7170000000000001},"min":{"1min":0.374,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.1710000000000003,"5min":15.423,"15min":28.248000000000001},"last":0.95999999999999996}]},{"osd":4,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":1.2090000000000001,"15min":1.073},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4219999999999997,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.242,"15min":1.054},"min":{"1min":0.34599999999999997,"5min":0.32900000000000001,"15min":0.23699999999999999},"max":{"1min":4.4450000000000003,"5min":7.9409999999999998,"15min":7.9409999999999998},"last":1.0149999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.92000000000000004,"15min":0.97499999999999998},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.28000000000000003},"max":{"1min":4.3879999999999999,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.92300000000000004,"15min":0.97399999999999998},"min":{"1min":0.34599999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.143,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":0.93000000000000005}]},{"osd":7,"last update":"Fri Sep 13 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.0640000000000001,"15min":1.054},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.8149999999999999,"5min":12.021000000000001,"15min":12.021000000000001},"last":1.048},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.01,"15min":0.998},"min":{"1min":0.34999999999999998,"5min":0.32500000000000001,"15min":0.312},"max":{"1min":4.3600000000000003,"5min":9.0440000000000005,"15min":9.0440000000000005},"last":1.123}]}]},{"osd":7,"up_from":15,"seq":64424509824,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":944040,"kb_used_data":9760,"kb_used_omap":1,"kb_used_meta":934270,"kb_avail":93427800,"statfs":{"total":96636764160,"available":95670067200,"internally_reserved":0,"allocated":9994240,"data_stored":13045326,"data_compressed":732085,"data_compressed_allocated":1101824,"data_compressed_original":4415488,"omap_allocated":1589,"internal_metadata":956692939},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.956,"15min":1.24},"min":{"1min":0.253,"5min":0.217,"15min":0.217},"max":{"1min":8.7949999999999999,"5min":24.414000000000001,"15min":24.414000000000001},"last":1.177},{"interface":"front","average":{"1min":1.3200000000000001,"5min":2.089,"15min":1.383},"min":{"1min":0.34000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":8.8480000000000008,"5min":24.469000000000001,"15min":24.469000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":3.1869999999999998,"15min":1.665},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":7.3250000000000002,"5min":46.161999999999999,"15min":46.161999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.333,"5min":3.9089999999999998,"15min":2.0459999999999998},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":7.548,"5min":45.713000000000001,"15min":45.713000000000001},"last":1.0660000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.208,"15min":0.98399999999999999},"min":{"1min":0.35499999999999998,"5min":0.29699999999999999,"15min":0.28999999999999998},"max":{"1min":3.4279999999999999,"5min":7.4740000000000002,"15min":7.4740000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.3340000000000001,"15min":0.99199999999999999},"min":{"1min":0.27800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.4699999999999998,"5min":9.1679999999999993,"15min":9.1679999999999993},"last":0.60999999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.8169999999999999,"5min":3.0920000000000001,"15min":1.869},"min":{"1min":0.35399999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.8140000000000001,"5min":35.494999999999997,"15min":35.494999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.212,"5min":2.5289999999999999,"15min":1.7030000000000001},"min":{"1min":0.33600000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":5.0880000000000001,"5min":19.946000000000002,"15min":19.946000000000002},"last":0.84799999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.9690000000000001,"5min":1.198,"15min":0.98799999999999999},"min":{"1min":0.36499999999999999,"5min":0.249,"15min":0.249},"max":{"1min":10.477,"5min":10.477,"15min":10.477},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.2629999999999999,"15min":1.0980000000000001},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":10.499000000000001,"5min":10.499000000000001,"15min":10.499000000000001},"last":1.0980000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.0289999999999999,"15min":0.92300000000000004},"min":{"1min":0.30199999999999999,"5min":0.26600000000000001,"15min":0.26100000000000001},"max":{"1min":14.954000000000001,"5min":14.954000000000001,"15min":14.954000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.0369999999999999,"15min":0.99399999999999999},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":9.0180000000000007,"5min":9.0180000000000007,"15min":9.0180000000000007},"last":0.89700000000000002}]},{"osd":6,"last update":"Fri Sep 13 18:57:44 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.415,"15min":1.1639999999999999},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":7.383,"5min":32.491,"15min":32.491},"last":1.002},{"interface":"front","average":{"1min":1.786,"5min":1.3240000000000001,"15min":1.141},"min":{"1min":0.44700000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":8.8770000000000007,"5min":27.291,"15min":27.291},"last":0.96199999999999997}]}]},{"osd":4,"up_from":15,"seq":64424509823,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":981284,"kb_used_data":4508,"kb_used_omap":1,"kb_used_meta":976766,"kb_avail":93390556,"statfs":{"total":96636764160,"available":95631929344,"internally_reserved":0,"allocated":4616192,"data_stored":4436064,"data_compressed":20605,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":1594,"internal_metadata":1000208838},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":2.0270000000000001,"15min":1.238},"min":{"1min":0.26300000000000001,"5min":0.24099999999999999,"15min":0.223},"max":{"1min":4.6299999999999999,"5min":28.593,"15min":28.593},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.7889999999999999,"5min":1.9890000000000001,"15min":1.256},"min":{"1min":0.33000000000000002,"5min":0.255,"15min":0.255},"max":{"1min":10.644,"5min":28.559000000000001,"15min":28.559000000000001},"last":0.71899999999999997}]},{"osd":1,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":3.7410000000000001,"5min":3.1200000000000001,"15min":1.6899999999999999},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":29.113,"5min":37.100999999999999,"15min":37.100999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":2.6909999999999998,"5min":3.468,"15min":1.8180000000000001},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":12.385999999999999,"5min":44.798000000000002,"15min":44.798000000000002},"last":1.044}]},{"osd":2,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.792,"15min":1.2130000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":10.506,"5min":27.634,"15min":27.634},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.137,"5min":1.7250000000000001,"15min":1.216},"min":{"1min":0.34200000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.4029999999999996,"5min":27.606000000000002,"15min":27.606000000000002},"last":1.554}]},{"osd":3,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.641,"5min":3.1930000000000001,"15min":1.708},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":10.535,"5min":40.676000000000002,"15min":40.676000000000002},"last":0.76300000000000001},{"interface":"front","average":{"1min":2.1560000000000001,"5min":3.6699999999999999,"15min":1.97},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":10.548999999999999,"5min":40.881999999999998,"15min":40.881999999999998},"last":1.1519999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.93700000000000006,"15min":0.89200000000000002},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":4.9889999999999999,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.0529999999999999,"5min":0.93999999999999995,"15min":0.98899999999999999},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.0170000000000003,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":1.6879999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2490000000000001,"15min":1.103},"min":{"1min":0.29799999999999999,"5min":0.251,"15min":0.251},"max":{"1min":8.9039999999999999,"5min":9.4570000000000007,"15min":9.4570000000000007},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.002,"15min":1.014},"min":{"1min":0.38600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":3.8559999999999999,"5min":5.1059999999999999,"15min":6.1319999999999997},"last":1.78}]},{"osd":7,"last update":"Fri Sep 13 18:57:20 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.464,"15min":1.1930000000000001},"min":{"1min":0.38600000000000001,"5min":0.307,"15min":0.307},"max":{"1min":6.1210000000000004,"5min":10.101000000000001,"15min":10.101000000000001},"last":1.581},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.5269999999999999,"15min":1.286},"min":{"1min":0.41199999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.1509999999999998,"5min":10.127000000000001,"15min":10.127000000000001},"last":1.647}]}]},{"osd":3,"up_from":14,"seq":60129542527,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1272936,"kb_used_data":5472,"kb_used_omap":1,"kb_used_meta":1267454,"kb_avail":93098904,"statfs":{"total":96636764160,"available":95333277696,"internally_reserved":0,"allocated":5603328,"data_stored":8725194,"data_compressed":790472,"data_compressed_allocated":1138688,"data_compressed_original":4517888,"omap_allocated":1591,"internal_metadata":1297873353},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:32 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":2.9300000000000002,"15min":1.5609999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":10.571999999999999,"5min":22.670000000000002,"15min":22.670000000000002},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.266,"5min":2.9980000000000002,"15min":1.716},"min":{"1min":0.23000000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":10.512,"5min":22.728000000000002,"15min":22.728000000000002},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":4.0330000000000004,"5min":5.657,"15min":2.7250000000000001},"min":{"1min":0.20499999999999999,"5min":0.191,"15min":0.191},"max":{"1min":19.170999999999999,"5min":42.046999999999997,"15min":42.046999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":3.5009999999999999,"5min":5.2409999999999997,"15min":2.7490000000000001},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":19.199999999999999,"5min":42.024999999999999,"15min":42.024999999999999},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":3.5950000000000002,"5min":3.734,"15min":2.113},"min":{"1min":0.29399999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":19.561,"5min":41.909999999999997,"15min":41.909999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.95399999999999996,"5min":2.7280000000000002,"15min":1.544},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.7570000000000001,"5min":27.25,"15min":27.25},"last":0.77000000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:57:32 2024","interfaces":[{"interface":"back","average":{"1min":2.52,"5min":3.1509999999999998,"15min":1.8520000000000001},"min":{"1min":0.45000000000000001,"5min":0.30099999999999999,"15min":0.26300000000000001},"max":{"1min":19.533999999999999,"5min":42.003,"15min":42.003},"last":1.1359999999999999},{"interface":"front","average":{"1min":2.9780000000000002,"5min":3.46,"15min":2.0950000000000002},"min":{"1min":0.41699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":19.728999999999999,"5min":41.951000000000001,"15min":41.951000000000001},"last":1.0669999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":2.3340000000000001,"15min":1.5489999999999999},"min":{"1min":0.29699999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":6.5679999999999996,"5min":22.988,"15min":22.988},"last":1.091},{"interface":"front","average":{"1min":1.155,"5min":2.0670000000000002,"15min":1.474},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.7330000000000001,"5min":23.030000000000001,"15min":23.030000000000001},"last":1.1659999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":2.5870000000000002,"15min":1.6859999999999999},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":3.6080000000000001,"5min":23.087,"15min":23.087},"last":0.97499999999999998},{"interface":"front","average":{"1min":2.1190000000000002,"5min":3.2130000000000001,"15min":1.9159999999999999},"min":{"1min":0.39800000000000002,"5min":0.317,"15min":0.317},"max":{"1min":14.753,"5min":23.425000000000001,"15min":23.425000000000001},"last":1.034}]},{"osd":7,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":3.4409999999999998,"5min":3.1819999999999999,"15min":1.8819999999999999},"min":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":19.588000000000001,"5min":41.975999999999999,"15min":41.975999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.956,"5min":2.7709999999999999,"15min":1.728},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34200000000000003},"max":{"1min":14.686999999999999,"5min":23.457999999999998,"15min":23.457999999999998},"last":1.107}]}]},{"osd":2,"up_from":14,"seq":60129542531,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746120,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":745534,"kb_avail":93625720,"statfs":{"total":96636764160,"available":95872737280,"internally_reserved":0,"allocated":589824,"data_stored":334673,"data_compressed":1568,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":763427271},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":2.5110000000000001,"15min":1.278},"min":{"1min":0.19600000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":4.7599999999999998,"5min":37.600000000000001,"15min":37.600000000000001},"last":0.498},{"interface":"front","average":{"1min":1.3540000000000001,"5min":2.5030000000000001,"15min":1.3149999999999999},"min":{"1min":0.20499999999999999,"5min":0.19,"15min":0.19},"max":{"1min":7.3949999999999996,"5min":37.651000000000003,"15min":37.651000000000003},"last":0.79900000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":3.4319999999999999,"5min":2.8719999999999999,"15min":1.7030000000000001},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":17.443999999999999,"5min":17.443999999999999,"15min":18.055},"last":0.63},{"interface":"front","average":{"1min":2.8220000000000001,"5min":2.6099999999999999,"15min":1.5800000000000001},"min":{"1min":0.28399999999999997,"5min":0.20000000000000001,"15min":0.188},"max":{"1min":15.132,"5min":15.94,"15min":18.175999999999998},"last":0.55200000000000005}]},{"osd":3,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":3.3039999999999998,"15min":1.806},"min":{"1min":0.29099999999999998,"5min":0.249,"15min":0.249},"max":{"1min":17.613,"5min":25.832999999999998,"15min":25.832999999999998},"last":1.282},{"interface":"front","average":{"1min":1.627,"5min":3.2429999999999999,"15min":1.8220000000000001},"min":{"1min":0.35599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":17.59,"5min":17.59,"15min":17.59},"last":1.3580000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.1910000000000001,"15min":1.0309999999999999},"min":{"1min":0.32600000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.3559999999999999,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.66400000000000003},{"interface":"front","average":{"1min":1.252,"5min":1.149,"15min":1.0940000000000001},"min":{"1min":0.35999999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.6980000000000004,"5min":8.0999999999999996,"15min":8.0999999999999996},"last":1.3280000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2669999999999999,"15min":1.0580000000000001},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29499999999999998},"max":{"1min":5.8600000000000003,"5min":8.4800000000000004,"15min":8.4800000000000004},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4470000000000001,"15min":1.1539999999999999},"min":{"1min":0.311,"5min":0.309,"15min":0.309},"max":{"1min":8.9949999999999992,"5min":8.9949999999999992,"15min":8.9949999999999992},"last":0.92600000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.2789999999999999,"15min":1.167},"min":{"1min":0.436,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.7370000000000001,"5min":9.6720000000000006,"15min":9.6720000000000006},"last":1.399},{"interface":"front","average":{"1min":1.127,"5min":1.1599999999999999,"15min":1.0800000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":4.5129999999999999,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":1.2410000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:57:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4039999999999999,"15min":1.169},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.3179999999999996,"5min":8.9250000000000007,"15min":8.9250000000000007},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.5029999999999999,"15min":1.2589999999999999},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":7.7569999999999997,"5min":9.2330000000000005,"15min":9.2330000000000005},"last":1.379}]}]},{"osd":5,"up_from":13,"seq":55834575232,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730672,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":730174,"kb_avail":93641168,"statfs":{"total":96636764160,"available":95888556032,"internally_reserved":0,"allocated":499712,"data_stored":241392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":747698631},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":1.7170000000000001,"5min":1.8320000000000001,"15min":1.175},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":8.8219999999999992,"5min":9.952,"15min":9.952},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.796,"5min":1.948,"15min":1.3},"min":{"1min":0.29599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.8740000000000006,"5min":14.317,"15min":14.317},"last":0.85699999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.931,"15min":1.3240000000000001},"min":{"1min":0.32500000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":6.4930000000000003,"5min":10.85,"15min":10.85},"last":1.089},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.917,"15min":1.3240000000000001},"min":{"1min":0.26000000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":3.113,"5min":10.868,"15min":10.868},"last":0.996}]},{"osd":2,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":2.641,"5min":2.222,"15min":1.3},"min":{"1min":0.30299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":9.9499999999999993,"5min":38.484000000000002,"15min":38.484000000000002},"last":0.747},{"interface":"front","average":{"1min":2.0259999999999998,"5min":2.3730000000000002,"15min":1.365},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":8.0800000000000001,"5min":40.686,"15min":40.686},"last":0.63100000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":2.5819999999999999,"5min":4.0430000000000001,"15min":2.0880000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":14.885999999999999,"5min":72.534999999999997,"15min":72.534999999999997},"last":0.82799999999999996},{"interface":"front","average":{"1min":2.8029999999999999,"5min":3.9220000000000002,"15min":2.101},"min":{"1min":0.30499999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":14.91,"5min":72.486999999999995,"15min":72.486999999999995},"last":0.80300000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.93100000000000005,"15min":0.90800000000000003},"min":{"1min":0.28799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1970000000000001,"5min":7.8710000000000004,"15min":7.8710000000000004},"last":1.014},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.1240000000000001,"15min":1.071},"min":{"1min":0.36099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.8399999999999999,"5min":8.7829999999999995,"15min":8.7829999999999995},"last":0.89300000000000002}]},{"osd":6,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.97499999999999998,"15min":0.999},"min":{"1min":0.32400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.5179999999999998,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.095,"5min":1.0569999999999999,"15min":1.0509999999999999},"min":{"1min":0.38500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.54,"5min":4.516,"15min":4.516},"last":1.0409999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:57:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.1919999999999999,"15min":1.0640000000000001},"min":{"1min":0.46400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.5780000000000003,"5min":7.6020000000000003,"15min":7.6020000000000003},"last":1.246},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.179,"15min":1.139},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.9699999999999998,"5min":7.9109999999999996,"15min":7.9109999999999996},"last":1.163}]}]},{"osd":1,"up_from":13,"seq":55834575230,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1223156,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":1222654,"kb_avail":93148684,"statfs":{"total":96636764160,"available":95384252416,"internally_reserved":0,"allocated":503808,"data_stored":240958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1251998150},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":2.0539999999999998,"5min":2.7050000000000001,"15min":1.5920000000000001},"min":{"1min":0.214,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":10.279,"5min":28.297000000000001,"15min":28.297000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":2.464,"5min":2.7149999999999999,"15min":1.466},"min":{"1min":0.32400000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":10.162000000000001,"5min":19.785,"15min":19.785},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":2.5609999999999999,"15min":1.4359999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.17799999999999999},"max":{"1min":7.2380000000000004,"5min":25.609999999999999,"15min":25.609999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.905,"5min":2.891,"15min":1.619},"min":{"1min":0.29299999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":10.465,"5min":30.469999999999999,"15min":30.469999999999999},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":4.1040000000000001,"5min":5.694,"15min":2.6179999999999999},"min":{"1min":0.48499999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":18.695,"5min":36.643999999999998,"15min":36.643999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":4.2220000000000004,"5min":5.8979999999999997,"15min":2.754},"min":{"1min":0.53800000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":18.760000000000002,"5min":36.670000000000002,"15min":36.670000000000002},"last":0.90200000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:57:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":1.4399999999999999,"15min":1.1779999999999999},"min":{"1min":0.28399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.7110000000000003,"5min":7.4669999999999996,"15min":7.6799999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.766,"15min":1.339},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":4.508,"5min":14.906000000000001,"15min":14.906000000000001},"last":0.70799999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.986,"15min":1.3700000000000001},"min":{"1min":0.38800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.259,"5min":10.259,"15min":10.259},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.6910000000000001,"5min":2.0289999999999999,"15min":1.3799999999999999},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34100000000000003},"max":{"1min":10.231999999999999,"5min":12.27,"15min":12.27},"last":0.83399999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":2.093,"5min":2.4220000000000002,"15min":1.5449999999999999},"min":{"1min":0.432,"5min":0.371,"15min":0.316},"max":{"1min":10.538,"5min":28.981999999999999,"15min":28.981999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.8700000000000001,"15min":1.429},"min":{"1min":0.36599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":10.342000000000001,"5min":15.727,"15min":15.727},"last":0.92000000000000004}]},{"osd":7,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":2.0630000000000002,"15min":1.466},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":7.2130000000000001,"5min":28.100000000000001,"15min":28.100000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.2889999999999999,"5min":2.0710000000000002,"15min":1.5389999999999999},"min":{"1min":0.41099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":7.173,"5min":28.896999999999998,"15min":28.896999999999998},"last":1.0129999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575231,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":842628,"kb_used_data":2556,"kb_used_omap":1,"kb_used_meta":840062,"kb_avail":93529212,"statfs":{"total":96636764160,"available":95773913088,"internally_reserved":0,"allocated":2617344,"data_stored":4654715,"data_compressed":442451,"data_compressed_allocated":618496,"data_compressed_original":2916352,"omap_allocated":1594,"internal_metadata":860223942},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.8009999999999999,"5min":3.573,"15min":1.8169999999999999},"min":{"1min":0.24399999999999999,"5min":0.183,"15min":0.183},"max":{"1min":10.925000000000001,"5min":29.712,"15min":29.712},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.877,"5min":2.6520000000000001,"15min":1.5640000000000001},"min":{"1min":0.29299999999999998,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":10.975,"5min":29.748000000000001,"15min":29.748000000000001},"last":0.76700000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.6519999999999999,"15min":1.0509999999999999},"min":{"1min":0.218,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":8.5660000000000007,"5min":17.611000000000001,"15min":17.611000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.113,"5min":1.867,"15min":1.149},"min":{"1min":0.22700000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":9.2560000000000002,"5min":17.501999999999999,"15min":17.501999999999999},"last":0.70399999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":3.2879999999999998,"15min":1.6659999999999999},"min":{"1min":0.30099999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":7.9340000000000002,"5min":17.260000000000002,"15min":17.260000000000002},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.149,"5min":2.6269999999999998,"15min":1.536},"min":{"1min":0.27400000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":5.8170000000000002,"5min":17.088000000000001,"15min":17.088000000000001},"last":0.84699999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":2.286,"15min":1.4219999999999999},"min":{"1min":0.376,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":8.0380000000000003,"5min":40.101999999999997,"15min":40.101999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":2.0600000000000001,"15min":1.3069999999999999},"min":{"1min":0.40100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":5.9009999999999998,"5min":40.981000000000002,"15min":40.981000000000002},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.8120000000000001,"15min":1.246},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.8900000000000001,"5min":40.701999999999998,"15min":40.701999999999998},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":2.0499999999999998,"15min":1.3520000000000001},"min":{"1min":0.434,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.8559999999999999,"5min":40.130000000000003,"15min":40.130000000000003},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Sep 13 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":2.1930000000000001,"15min":1.4159999999999999},"min":{"1min":0.40200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":6.1440000000000001,"5min":51.719000000000001,"15min":51.719000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.4019999999999999,"5min":2.2559999999999998,"15min":1.4710000000000001},"min":{"1min":0.45500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":7.9660000000000002,"5min":51.75,"15min":51.75},"last":1.046}]},{"osd":7,"last update":"Fri Sep 13 18:57:34 2024","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":2.4119999999999999,"15min":1.522},"min":{"1min":0.44400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":8.0120000000000005,"5min":58.810000000000002,"15min":58.810000000000002},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.6299999999999999,"5min":2.2389999999999999,"15min":1.4770000000000001},"min":{"1min":0.54100000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.508,"5min":39.622,"15min":39.622},"last":0.998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2072576,"data_stored":3824333,"data_compressed":409575,"data_compressed_allocated":573440,"data_compressed_original":2342912,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":94659,"data_compressed":1568,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5124096,"data_stored":8485180,"data_compressed":790472,"data_compressed_allocated":1138688,"data_compressed_original":4517888,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4136960,"data_stored":4196050,"data_compressed":20605,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1777664,"data_stored":4196686,"data_compressed":519875,"data_compressed_allocated":692224,"data_compressed_original":3125248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9449472,"data_stored":12214944,"data_compressed":699209,"data_compressed_allocated":1056768,"data_compressed_original":3842048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:58:14.372 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-13T18:26:35.350751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=35, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:58:14.372 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T18:26:31.936605+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=57, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:58:14.373 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T18:58:34.374 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:58:34.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f353615f700 1 Processor -- start 2024-09-13T18:58:34.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f353615f700 1 -- start start 2024-09-13T18:58:34.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f353615f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3530159e90 0x7f35300c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:34.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f353615f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f35300c6700 0x7f353014fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:34.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f353615f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35300c6b50 con 0x7f35300c7bc0 2024-09-13T18:58:34.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f353615f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35301591d0 con 0x7f35300c6700 2024-09-13T18:58:34.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f353615f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35301500e0 con 0x7f3530159e90 2024-09-13T18:58:34.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f352ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f35300c6700 0x7f353014fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:34.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f352ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f35300c6700 0x7f353014fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45508/0 (socket says 172.21.15.146:45508) 2024-09-13T18:58:34.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f352ffff700 1 -- 172.21.15.146:0/3806047191 learned_addr learned my addr 172.21.15.146:0/3806047191 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:34.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f352effd700 1 --2- 172.21.15.146:0/3806047191 >> v2:172.21.15.154:3300/0 conn(0x7f3530159e90 0x7f35300c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:34.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f352ffff700 1 -- 172.21.15.146:0/3806047191 >> v2:172.21.15.154:3300/0 conn(0x7f3530159e90 msgr2=0x7f35300c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:34.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f352ffff700 1 --2- 172.21.15.146:0/3806047191 >> v2:172.21.15.154:3300/0 conn(0x7f3530159e90 0x7f35300c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:34.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f352ffff700 1 -- 172.21.15.146:0/3806047191 >> v1:172.21.15.154:6789/0 conn(0x7f35300c7bc0 legacy=0x7f35300c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:34.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f352ffff700 1 -- 172.21.15.146:0/3806047191 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3530156700 con 0x7f35300c6700 2024-09-13T18:58:34.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f352ffff700 1 --2- 172.21.15.146:0/3806047191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35300c6700 0x7f353014fba0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f353006b9a0 tx=0x7f3524009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54807280a7ba37eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:34.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f352e7fc700 1 -- 172.21.15.146:0/3806047191 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524011830 con 0x7f35300c6700 2024-09-13T18:58:34.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f352e7fc700 1 -- 172.21.15.146:0/3806047191 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3524011990 con 0x7f35300c6700 2024-09-13T18:58:34.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.645+0000 7f352e7fc700 1 -- 172.21.15.146:0/3806047191 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524011d20 con 0x7f35300c6700 2024-09-13T18:58:34.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 -- 172.21.15.146:0/3806047191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35300c6700 msgr2=0x7f353014fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:34.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 --2- 172.21.15.146:0/3806047191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35300c6700 0x7f353014fba0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f353006b9a0 tx=0x7f3524009bf0 comp rx=0 tx=0).stop 2024-09-13T18:58:34.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 -- 172.21.15.146:0/3806047191 shutdown_connections 2024-09-13T18:58:34.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 --2- 172.21.15.146:0/3806047191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35300c6700 0x7f353014fba0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:34.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 --2- 172.21.15.146:0/3806047191 >> v2:172.21.15.154:3300/0 conn(0x7f3530159e90 0x7f35300c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:34.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 -- 172.21.15.146:0/3806047191 >> 172.21.15.146:0/3806047191 conn(0x7f353006ada0 msgr2=0x7f353006b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:34.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 -- 172.21.15.146:0/3806047191 shutdown_connections 2024-09-13T18:58:34.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 -- 172.21.15.146:0/3806047191 wait complete. 2024-09-13T18:58:34.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 Processor -- start 2024-09-13T18:58:34.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 -- start start 2024-09-13T18:58:34.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35300c6700 0x7f3530162d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:34.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35300c7bc0 0x7f35301543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:34.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3530159e90 0x7f3530155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:34.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3530153f10 con 0x7f3530159e90 2024-09-13T18:58:34.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353006ed20 con 0x7f35300c6700 2024-09-13T18:58:34.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f353615f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3530164130 con 0x7f35300c7bc0 2024-09-13T18:58:34.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f352ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3530159e90 0x7f3530155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:34.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f352ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3530159e90 0x7f3530155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45518/0 (socket says 172.21.15.146:45518) 2024-09-13T18:58:34.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.649+0000 7f352effd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35300c7bc0 0x7f35301543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:34.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f352f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35300c6700 0x7f3530162d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:34.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f352ffff700 1 -- 172.21.15.146:0/3674293704 learned_addr learned my addr 172.21.15.146:0/3674293704 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:34.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f352f7fe700 1 -- 172.21.15.146:0/3674293704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35300c7bc0 msgr2=0x7f35301543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:34.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f352f7fe700 1 --2- 172.21.15.146:0/3674293704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35300c7bc0 0x7f35301543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:34.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f352f7fe700 1 -- 172.21.15.146:0/3674293704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3530159e90 msgr2=0x7f3530155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:34.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f352f7fe700 1 --2- 172.21.15.146:0/3674293704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3530159e90 0x7f3530155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:34.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f352f7fe700 1 -- 172.21.15.146:0/3674293704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3524009570 con 0x7f35300c6700 2024-09-13T18:58:34.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f352ffff700 1 --2- 172.21.15.146:0/3674293704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3530159e90 0x7f3530155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:58:34.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f352f7fe700 1 --2- 172.21.15.146:0/3674293704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35300c6700 0x7f3530162d80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f352800cb90 tx=0x7f352800cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:34.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f3517fff700 1 -- 172.21.15.146:0/3674293704 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528003c40 con 0x7f35300c6700 2024-09-13T18:58:34.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f3517fff700 1 -- 172.21.15.146:0/3674293704 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3528003da0 con 0x7f35300c6700 2024-09-13T18:58:34.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f353615f700 1 -- 172.21.15.146:0/3674293704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35301643d0 con 0x7f35300c6700 2024-09-13T18:58:34.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f3517fff700 1 -- 172.21.15.146:0/3674293704 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352800aae0 con 0x7f35300c6700 2024-09-13T18:58:34.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f353615f700 1 -- 172.21.15.146:0/3674293704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35301538c0 con 0x7f35300c6700 2024-09-13T18:58:34.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.653+0000 7f353615f700 1 -- 172.21.15.146:0/3674293704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353014fba0 con 0x7f35300c6700 2024-09-13T18:58:34.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.657+0000 7f3517fff700 1 -- 172.21.15.146:0/3674293704 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f352800ac40 con 0x7f35300c6700 2024-09-13T18:58:34.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.657+0000 7f3517fff700 1 --2- 172.21.15.146:0/3674293704 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f351c066730 0x7f351c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:34.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.657+0000 7f3517fff700 1 -- 172.21.15.146:0/3674293704 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f35280850d0 con 0x7f35300c6700 2024-09-13T18:58:34.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.657+0000 7f352effd700 1 --2- 172.21.15.146:0/3674293704 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f351c066730 0x7f351c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:34.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.657+0000 7f352effd700 1 --2- 172.21.15.146:0/3674293704 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f351c066730 0x7f351c068bf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f35300c2150 tx=0x7f3520009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:34.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.665+0000 7f3517fff700 1 -- 172.21.15.146:0/3674293704 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3528051c00 con 0x7f35300c6700 2024-09-13T18:58:34.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.829+0000 7f353615f700 1 -- 172.21.15.146:0/3674293704 --> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f35300cb670 con 0x7f351c066730 2024-09-13T18:58:34.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.845+0000 7f3517fff700 1 -- 172.21.15.146:0/3674293704 <== mgr.4099 v2:172.21.15.154:6832/1235826889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+372521 (secure 0 0 0) 0x7f35300cb670 con 0x7f351c066730 2024-09-13T18:58:34.862 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:58:34.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.853+0000 7f353615f700 1 -- 172.21.15.146:0/3674293704 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f351c066730 msgr2=0x7f351c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:34.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.853+0000 7f353615f700 1 --2- 172.21.15.146:0/3674293704 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f351c066730 0x7f351c068bf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f35300c2150 tx=0x7f3520009240 comp rx=0 tx=0).stop 2024-09-13T18:58:34.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.853+0000 7f353615f700 1 -- 172.21.15.146:0/3674293704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35300c6700 msgr2=0x7f3530162d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:34.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.853+0000 7f353615f700 1 --2- 172.21.15.146:0/3674293704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35300c6700 0x7f3530162d80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f352800cb90 tx=0x7f352800cea0 comp rx=0 tx=0).stop 2024-09-13T18:58:34.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.853+0000 7f353615f700 1 -- 172.21.15.146:0/3674293704 shutdown_connections 2024-09-13T18:58:34.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.853+0000 7f353615f700 1 --2- 172.21.15.146:0/3674293704 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f351c066730 0x7f351c068bf0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:34.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.853+0000 7f353615f700 1 --2- 172.21.15.146:0/3674293704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3530159e90 0x7f3530155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:34.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.853+0000 7f353615f700 1 --2- 172.21.15.146:0/3674293704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35300c7bc0 0x7f35301543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:34.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.853+0000 7f353615f700 1 --2- 172.21.15.146:0/3674293704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35300c6700 0x7f3530162d80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:34.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.853+0000 7f353615f700 1 -- 172.21.15.146:0/3674293704 >> 172.21.15.146:0/3674293704 conn(0x7f353006ada0 msgr2=0x7f35300c88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:34.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.853+0000 7f353615f700 1 -- 172.21.15.146:0/3674293704 shutdown_connections 2024-09-13T18:58:34.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.853+0000 7f353615f700 1 -- 172.21.15.146:0/3674293704 wait complete. 2024-09-13T18:58:34.866 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-13T18:58:34.876 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":1000,"stamp":"2024-09-13T18:58:33.632425+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17097455,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":2275,"num_read_kb":52180,"num_write":807501,"num_write_kb":16339542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":182570,"ondisk_log_size":182570,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7800200,"kb_used_data":26056,"kb_used_omap":12,"kb_used_meta":7774067,"kb_avail":747174520,"statfs":{"total":773094113280,"available":765106708480,"internally_reserved":0,"allocated":26681344,"data_stored":36115022,"data_compressed":2507056,"data_compressed_allocated":3596288,"data_compressed_original":15097856,"omap_allocated":12741,"internal_metadata":7960645179},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005155"},"pg_stats":[{"pgid":"2.1f","version":"43'471","reported_seq":562,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.638819+0000","last_change":"2024-09-13T18:57:12.363177+0000","last_active":"2024-09-13T18:57:27.638819+0000","last_peered":"2024-09-13T18:57:27.638819+0000","last_clean":"2024-09-13T18:57:27.638819+0000","last_became_active":"2024-09-13T18:26:31.972547+0000","last_became_peered":"2024-09-13T18:26:31.972547+0000","last_unstale":"2024-09-13T18:57:27.638819+0000","last_undegraded":"2024-09-13T18:57:27.638819+0000","last_fullsized":"2024-09-13T18:57:27.638819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'469","last_scrub_stamp":"2024-09-13T18:57:12.363068+0000","last_deep_scrub":"43'469","last_deep_scrub_stamp":"2024-09-13T18:57:12.363068+0000","last_clean_scrub_stamp":"2024-09-13T18:57:12.363068+0000","objects_scrubbed":5,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:41:46.348669+0000","scrub_duration":0.044894295000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3824255,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":428,"num_write":456,"num_write_kb":3991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"43'5","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:16.089759+0000","last_change":"2024-09-13T18:57:16.090020+0000","last_active":"2024-09-13T18:57:16.089759+0000","last_peered":"2024-09-13T18:57:16.089759+0000","last_clean":"2024-09-13T18:57:16.089759+0000","last_became_active":"2024-09-13T18:26:32.348420+0000","last_became_peered":"2024-09-13T18:26:32.348420+0000","last_unstale":"2024-09-13T18:57:16.089759+0000","last_undegraded":"2024-09-13T18:57:16.089759+0000","last_fullsized":"2024-09-13T18:57:16.089759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:16.089709+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:16.089709+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.089709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.965527+0000","scrub_duration":0.0037314340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"43'5858","reported_seq":5949,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:01.958678+0000","last_change":"2024-09-13T18:58:01.958678+0000","last_active":"2024-09-13T18:58:01.958678+0000","last_peered":"2024-09-13T18:58:01.958678+0000","last_clean":"2024-09-13T18:58:01.958678+0000","last_became_active":"2024-09-13T18:26:36.407139+0000","last_became_peered":"2024-09-13T18:26:36.407139+0000","last_unstale":"2024-09-13T18:58:01.958678+0000","last_undegraded":"2024-09-13T18:58:01.958678+0000","last_fullsized":"2024-09-13T18:58:01.958678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5858","last_scrub_stamp":"2024-09-13T18:58:01.958539+0000","last_deep_scrub":"43'5858","last_deep_scrub_stamp":"2024-09-13T18:58:01.958539+0000","last_clean_scrub_stamp":"2024-09-13T18:58:01.958539+0000","objects_scrubbed":0,"log_size":5858,"log_dups_size":0,"ondisk_log_size":5858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:20.890340+0000","scrub_duration":0.043303400999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":26767,"num_write_kb":537654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"43'16","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.390499+0000","last_change":"2024-09-13T18:57:25.919095+0000","last_active":"2024-09-13T18:57:27.390499+0000","last_peered":"2024-09-13T18:57:27.390499+0000","last_clean":"2024-09-13T18:57:27.390499+0000","last_became_active":"2024-09-13T18:26:31.980469+0000","last_became_peered":"2024-09-13T18:26:31.980469+0000","last_unstale":"2024-09-13T18:57:27.390499+0000","last_undegraded":"2024-09-13T18:57:27.390499+0000","last_fullsized":"2024-09-13T18:57:27.390499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:25.918713+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:25.918713+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.918713+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:35.551553+0000","scrub_duration":0.0032041489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"43'14","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:33.385777+0000","last_change":"2024-09-13T18:57:33.385777+0000","last_active":"2024-09-13T18:57:33.385777+0000","last_peered":"2024-09-13T18:57:33.385777+0000","last_clean":"2024-09-13T18:57:33.385777+0000","last_became_active":"2024-09-13T18:26:32.342452+0000","last_became_peered":"2024-09-13T18:26:32.342452+0000","last_unstale":"2024-09-13T18:57:33.385777+0000","last_undegraded":"2024-09-13T18:57:33.385777+0000","last_fullsized":"2024-09-13T18:57:33.385777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:33.385394+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:33.385394+0000","last_clean_scrub_stamp":"2024-09-13T18:57:33.385394+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:52.913543+0000","scrub_duration":0.0034717820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"43'5909","reported_seq":5993,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:40.030905+0000","last_change":"2024-09-13T18:57:40.030905+0000","last_active":"2024-09-13T18:57:40.030905+0000","last_peered":"2024-09-13T18:57:40.030905+0000","last_clean":"2024-09-13T18:57:40.030905+0000","last_became_active":"2024-09-13T18:26:36.396198+0000","last_became_peered":"2024-09-13T18:26:36.396198+0000","last_unstale":"2024-09-13T18:57:40.030905+0000","last_undegraded":"2024-09-13T18:57:40.030905+0000","last_fullsized":"2024-09-13T18:57:40.030905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5909","last_scrub_stamp":"2024-09-13T18:57:40.030533+0000","last_deep_scrub":"43'5909","last_deep_scrub_stamp":"2024-09-13T18:57:40.030533+0000","last_clean_scrub_stamp":"2024-09-13T18:57:40.030533+0000","objects_scrubbed":0,"log_size":5909,"log_dups_size":0,"ondisk_log_size":5909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:18:12.112501+0000","scrub_duration":0.046110303999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":26783,"num_write_kb":545346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"43'29","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:37.893910+0000","last_change":"2024-09-13T18:57:37.894025+0000","last_active":"2024-09-13T18:57:37.893910+0000","last_peered":"2024-09-13T18:57:37.893910+0000","last_clean":"2024-09-13T18:57:37.893910+0000","last_became_active":"2024-09-13T18:26:31.977577+0000","last_became_peered":"2024-09-13T18:26:31.977577+0000","last_unstale":"2024-09-13T18:57:37.893910+0000","last_undegraded":"2024-09-13T18:57:37.893910+0000","last_fullsized":"2024-09-13T18:57:37.893910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'29","last_scrub_stamp":"2024-09-13T18:57:37.893891+0000","last_deep_scrub":"43'29","last_deep_scrub_stamp":"2024-09-13T18:57:37.893891+0000","last_clean_scrub_stamp":"2024-09-13T18:57:37.893891+0000","objects_scrubbed":2,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:54:57.603232+0000","scrub_duration":0.0099177959999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":65,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"43'9","reported_seq":59,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:44.043128+0000","last_change":"2024-09-13T18:57:44.043334+0000","last_active":"2024-09-13T18:57:44.043128+0000","last_peered":"2024-09-13T18:57:44.043128+0000","last_clean":"2024-09-13T18:57:44.043128+0000","last_became_active":"2024-09-13T18:27:22.609625+0000","last_became_peered":"2024-09-13T18:27:22.609625+0000","last_unstale":"2024-09-13T18:57:44.043128+0000","last_undegraded":"2024-09-13T18:57:44.043128+0000","last_fullsized":"2024-09-13T18:57:44.043128+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:57:44.043080+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:57:44.043080+0000","last_clean_scrub_stamp":"2024-09-13T18:57:44.043080+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:40:41.348901+0000","scrub_duration":0.0036812300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"43'4988","reported_seq":5062,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:50.959877+0000","last_change":"2024-09-13T18:57:50.960011+0000","last_active":"2024-09-13T18:57:50.959877+0000","last_peered":"2024-09-13T18:57:50.959877+0000","last_clean":"2024-09-13T18:57:50.959877+0000","last_became_active":"2024-09-13T18:26:36.406330+0000","last_became_peered":"2024-09-13T18:26:36.406330+0000","last_unstale":"2024-09-13T18:57:50.959877+0000","last_undegraded":"2024-09-13T18:57:50.959877+0000","last_fullsized":"2024-09-13T18:57:50.959877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4988","last_scrub_stamp":"2024-09-13T18:57:50.959856+0000","last_deep_scrub":"43'4988","last_deep_scrub_stamp":"2024-09-13T18:57:50.959856+0000","last_clean_scrub_stamp":"2024-09-13T18:57:50.959856+0000","objects_scrubbed":0,"log_size":4988,"log_dups_size":0,"ondisk_log_size":4988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:31.885945+0000","scrub_duration":0.040254119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21624,"num_write_kb":467229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"43'9","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:31.883126+0000","last_change":"2024-09-13T18:57:31.883350+0000","last_active":"2024-09-13T18:57:31.883126+0000","last_peered":"2024-09-13T18:57:31.883126+0000","last_clean":"2024-09-13T18:57:31.883126+0000","last_became_active":"2024-09-13T18:26:31.979470+0000","last_became_peered":"2024-09-13T18:26:31.979470+0000","last_unstale":"2024-09-13T18:57:31.883126+0000","last_undegraded":"2024-09-13T18:57:31.883126+0000","last_fullsized":"2024-09-13T18:57:31.883126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:57:31.883070+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:57:31.883070+0000","last_clean_scrub_stamp":"2024-09-13T18:57:31.883070+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:54.521068+0000","scrub_duration":0.0057825330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":12,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"43'16","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:54.218083+0000","last_change":"2024-09-13T18:57:54.218258+0000","last_active":"2024-09-13T18:57:54.218083+0000","last_peered":"2024-09-13T18:57:54.218083+0000","last_clean":"2024-09-13T18:57:54.218083+0000","last_became_active":"2024-09-13T18:26:32.351216+0000","last_became_peered":"2024-09-13T18:26:32.351216+0000","last_unstale":"2024-09-13T18:57:54.218083+0000","last_undegraded":"2024-09-13T18:57:54.218083+0000","last_fullsized":"2024-09-13T18:57:54.218083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:54.218043+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:54.218043+0000","last_clean_scrub_stamp":"2024-09-13T18:57:54.218043+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:34.235521+0000","scrub_duration":0.003853441,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"43'4887","reported_seq":4961,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:46.899456+0000","last_change":"2024-09-13T18:57:46.899691+0000","last_active":"2024-09-13T18:57:46.899456+0000","last_peered":"2024-09-13T18:57:46.899456+0000","last_clean":"2024-09-13T18:57:46.899456+0000","last_became_active":"2024-09-13T18:26:36.398364+0000","last_became_peered":"2024-09-13T18:26:36.398364+0000","last_unstale":"2024-09-13T18:57:46.899456+0000","last_undegraded":"2024-09-13T18:57:46.899456+0000","last_fullsized":"2024-09-13T18:57:46.899456+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4887","last_scrub_stamp":"2024-09-13T18:57:46.899410+0000","last_deep_scrub":"43'4887","last_deep_scrub_stamp":"2024-09-13T18:57:46.899410+0000","last_clean_scrub_stamp":"2024-09-13T18:57:46.899410+0000","objects_scrubbed":0,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:15.511519+0000","scrub_duration":0.04143703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":21624,"num_write_kb":434281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"43'9","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:29.780911+0000","last_change":"2024-09-13T18:57:29.781039+0000","last_active":"2024-09-13T18:57:29.780911+0000","last_peered":"2024-09-13T18:57:29.780911+0000","last_clean":"2024-09-13T18:57:29.780911+0000","last_became_active":"2024-09-13T18:26:32.360679+0000","last_became_peered":"2024-09-13T18:26:32.360679+0000","last_unstale":"2024-09-13T18:57:29.780911+0000","last_undegraded":"2024-09-13T18:57:29.780911+0000","last_fullsized":"2024-09-13T18:57:29.780911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:57:29.780879+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:57:29.780879+0000","last_clean_scrub_stamp":"2024-09-13T18:57:29.780879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:44.093932+0000","scrub_duration":0.0034454730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"43'1223","reported_seq":1308,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.921602+0000","last_change":"2024-09-13T18:57:28.921602+0000","last_active":"2024-09-13T18:57:28.921602+0000","last_peered":"2024-09-13T18:57:28.921602+0000","last_clean":"2024-09-13T18:57:28.921602+0000","last_became_active":"2024-09-13T18:26:31.977568+0000","last_became_peered":"2024-09-13T18:26:31.977568+0000","last_unstale":"2024-09-13T18:57:28.921602+0000","last_undegraded":"2024-09-13T18:57:28.921602+0000","last_fullsized":"2024-09-13T18:57:28.921602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'1223","last_scrub_stamp":"2024-09-13T18:57:28.921440+0000","last_deep_scrub":"43'1223","last_deep_scrub_stamp":"2024-09-13T18:57:28.921440+0000","last_clean_scrub_stamp":"2024-09-13T18:57:28.921440+0000","objects_scrubbed":3,"log_size":1223,"log_dups_size":0,"ondisk_log_size":1223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:20.603136+0000","scrub_duration":0.029685530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":10235,"num_write":1211,"num_write_kb":4423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"43'5452","reported_seq":5531,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:47.934939+0000","last_change":"2024-09-13T18:57:47.934939+0000","last_active":"2024-09-13T18:57:47.934939+0000","last_peered":"2024-09-13T18:57:47.934939+0000","last_clean":"2024-09-13T18:57:47.934939+0000","last_became_active":"2024-09-13T18:26:36.397107+0000","last_became_peered":"2024-09-13T18:26:36.397107+0000","last_unstale":"2024-09-13T18:57:47.934939+0000","last_undegraded":"2024-09-13T18:57:47.934939+0000","last_fullsized":"2024-09-13T18:57:47.934939+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5452","last_scrub_stamp":"2024-09-13T18:57:47.934770+0000","last_deep_scrub":"43'5452","last_deep_scrub_stamp":"2024-09-13T18:57:47.934770+0000","last_clean_scrub_stamp":"2024-09-13T18:57:47.934770+0000","objects_scrubbed":0,"log_size":5452,"log_dups_size":0,"ondisk_log_size":5452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:56:53.358515+0000","scrub_duration":0.035794448,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":24128,"num_write_kb":505876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"43'11","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:37.925499+0000","last_change":"2024-09-13T18:57:37.925499+0000","last_active":"2024-09-13T18:57:37.925499+0000","last_peered":"2024-09-13T18:57:37.925499+0000","last_clean":"2024-09-13T18:57:37.925499+0000","last_became_active":"2024-09-13T18:26:32.345306+0000","last_became_peered":"2024-09-13T18:26:32.345306+0000","last_unstale":"2024-09-13T18:57:37.925499+0000","last_undegraded":"2024-09-13T18:57:37.925499+0000","last_fullsized":"2024-09-13T18:57:37.925499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:37.925363+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:37.925363+0000","last_clean_scrub_stamp":"2024-09-13T18:57:37.925363+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:10.713761+0000","scrub_duration":0.0037509309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"43'16","reported_seq":82,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:32.292293+0000","last_change":"2024-09-13T18:57:32.292403+0000","last_active":"2024-09-13T18:57:32.292293+0000","last_peered":"2024-09-13T18:57:32.292293+0000","last_clean":"2024-09-13T18:57:32.292293+0000","last_became_active":"2024-09-13T18:26:31.973609+0000","last_became_peered":"2024-09-13T18:26:31.973609+0000","last_unstale":"2024-09-13T18:57:32.292293+0000","last_undegraded":"2024-09-13T18:57:32.292293+0000","last_fullsized":"2024-09-13T18:57:32.292293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:32.292267+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:32.292267+0000","last_clean_scrub_stamp":"2024-09-13T18:57:32.292267+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:41.284438+0000","scrub_duration":0.025296163999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"43'5442","reported_seq":5524,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:48.974304+0000","last_change":"2024-09-13T18:57:48.974304+0000","last_active":"2024-09-13T18:57:48.974304+0000","last_peered":"2024-09-13T18:57:48.974304+0000","last_clean":"2024-09-13T18:57:48.974304+0000","last_became_active":"2024-09-13T18:26:36.405574+0000","last_became_peered":"2024-09-13T18:26:36.405574+0000","last_unstale":"2024-09-13T18:57:48.974304+0000","last_undegraded":"2024-09-13T18:57:48.974304+0000","last_fullsized":"2024-09-13T18:57:48.974304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5442","last_scrub_stamp":"2024-09-13T18:57:48.974110+0000","last_deep_scrub":"43'5442","last_deep_scrub_stamp":"2024-09-13T18:57:48.974110+0000","last_clean_scrub_stamp":"2024-09-13T18:57:48.974110+0000","objects_scrubbed":0,"log_size":5442,"log_dups_size":0,"ondisk_log_size":5442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:38.773027+0000","scrub_duration":0.039492270000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":24412,"num_write_kb":498911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"43'10","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:41.082668+0000","last_change":"2024-09-13T18:57:41.082668+0000","last_active":"2024-09-13T18:57:41.082668+0000","last_peered":"2024-09-13T18:57:41.082668+0000","last_clean":"2024-09-13T18:57:41.082668+0000","last_became_active":"2024-09-13T18:27:22.609642+0000","last_became_peered":"2024-09-13T18:27:22.609642+0000","last_unstale":"2024-09-13T18:57:41.082668+0000","last_undegraded":"2024-09-13T18:57:41.082668+0000","last_fullsized":"2024-09-13T18:57:41.082668+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:41.082525+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:41.082525+0000","last_clean_scrub_stamp":"2024-09-13T18:57:41.082525+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:52:04.123510+0000","scrub_duration":0.0029389820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"43'7","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.406801+0000","last_change":"2024-09-13T18:57:20.192184+0000","last_active":"2024-09-13T18:57:26.406801+0000","last_peered":"2024-09-13T18:57:26.406801+0000","last_clean":"2024-09-13T18:57:26.406801+0000","last_became_active":"2024-09-13T18:26:31.968542+0000","last_became_peered":"2024-09-13T18:26:31.968542+0000","last_unstale":"2024-09-13T18:57:26.406801+0000","last_undegraded":"2024-09-13T18:57:26.406801+0000","last_fullsized":"2024-09-13T18:57:26.406801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:20.191966+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:20.191966+0000","last_clean_scrub_stamp":"2024-09-13T18:57:20.191966+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:39.033583+0000","scrub_duration":0.0068871679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"43'7006","reported_seq":7087,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:43.055580+0000","last_change":"2024-09-13T18:57:43.055725+0000","last_active":"2024-09-13T18:57:43.055580+0000","last_peered":"2024-09-13T18:57:43.055580+0000","last_clean":"2024-09-13T18:57:43.055580+0000","last_became_active":"2024-09-13T18:27:22.610527+0000","last_became_peered":"2024-09-13T18:27:22.610527+0000","last_unstale":"2024-09-13T18:57:43.055580+0000","last_undegraded":"2024-09-13T18:57:43.055580+0000","last_fullsized":"2024-09-13T18:57:43.055580+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'7006","last_scrub_stamp":"2024-09-13T18:57:43.055548+0000","last_deep_scrub":"43'7006","last_deep_scrub_stamp":"2024-09-13T18:57:43.055548+0000","last_clean_scrub_stamp":"2024-09-13T18:57:43.055548+0000","objects_scrubbed":0,"log_size":7006,"log_dups_size":0,"ondisk_log_size":7006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:13:12.839547+0000","scrub_duration":0.042924489000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":31810,"num_write_kb":622755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"43'11","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.393739+0000","last_change":"2024-09-13T18:57:17.999669+0000","last_active":"2024-09-13T18:57:28.393739+0000","last_peered":"2024-09-13T18:57:28.393739+0000","last_clean":"2024-09-13T18:57:28.393739+0000","last_became_active":"2024-09-13T18:26:32.360126+0000","last_became_peered":"2024-09-13T18:26:32.360126+0000","last_unstale":"2024-09-13T18:57:28.393739+0000","last_undegraded":"2024-09-13T18:57:28.393739+0000","last_fullsized":"2024-09-13T18:57:28.393739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:17.999495+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:17.999495+0000","last_clean_scrub_stamp":"2024-09-13T18:57:17.999495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:58.108162+0000","scrub_duration":0.0026512860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"43'7","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.926150+0000","last_change":"2024-09-13T18:57:39.926414+0000","last_active":"2024-09-13T18:57:39.926150+0000","last_peered":"2024-09-13T18:57:39.926150+0000","last_clean":"2024-09-13T18:57:39.926150+0000","last_became_active":"2024-09-13T18:26:31.976622+0000","last_became_peered":"2024-09-13T18:26:31.976622+0000","last_unstale":"2024-09-13T18:57:39.926150+0000","last_undegraded":"2024-09-13T18:57:39.926150+0000","last_fullsized":"2024-09-13T18:57:39.926150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'7","last_scrub_stamp":"2024-09-13T18:57:39.926056+0000","last_deep_scrub":"43'7","last_deep_scrub_stamp":"2024-09-13T18:57:39.926056+0000","last_clean_scrub_stamp":"2024-09-13T18:57:39.926056+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:42.161980+0000","scrub_duration":0.0070368430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"43'5416","reported_seq":5506,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:42.965611+0000","last_change":"2024-09-13T18:57:42.965611+0000","last_active":"2024-09-13T18:57:42.965611+0000","last_peered":"2024-09-13T18:57:42.965611+0000","last_clean":"2024-09-13T18:57:42.965611+0000","last_became_active":"2024-09-13T18:27:22.648054+0000","last_became_peered":"2024-09-13T18:27:22.648054+0000","last_unstale":"2024-09-13T18:57:42.965611+0000","last_undegraded":"2024-09-13T18:57:42.965611+0000","last_fullsized":"2024-09-13T18:57:42.965611+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5416","last_scrub_stamp":"2024-09-13T18:57:42.965388+0000","last_deep_scrub":"43'5416","last_deep_scrub_stamp":"2024-09-13T18:57:42.965388+0000","last_clean_scrub_stamp":"2024-09-13T18:57:42.965388+0000","objects_scrubbed":0,"log_size":5416,"log_dups_size":0,"ondisk_log_size":5416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:18.488873+0000","scrub_duration":0.042479966000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":23625,"num_write_kb":471297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"43'19","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.065688+0000","last_change":"2024-09-13T18:57:22.408978+0000","last_active":"2024-09-13T18:57:28.065688+0000","last_peered":"2024-09-13T18:57:28.065688+0000","last_clean":"2024-09-13T18:57:28.065688+0000","last_became_active":"2024-09-13T18:26:32.343744+0000","last_became_peered":"2024-09-13T18:26:32.343744+0000","last_unstale":"2024-09-13T18:57:28.065688+0000","last_undegraded":"2024-09-13T18:57:28.065688+0000","last_fullsized":"2024-09-13T18:57:28.065688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:22.408739+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:22.408739+0000","last_clean_scrub_stamp":"2024-09-13T18:57:22.408739+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:45:55.319773+0000","scrub_duration":0.003011952,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"43'11","reported_seq":94,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.645812+0000","last_change":"2024-09-13T18:57:18.823588+0000","last_active":"2024-09-13T18:57:27.645812+0000","last_peered":"2024-09-13T18:57:27.645812+0000","last_clean":"2024-09-13T18:57:27.645812+0000","last_became_active":"2024-09-13T18:26:31.979334+0000","last_became_peered":"2024-09-13T18:26:31.979334+0000","last_unstale":"2024-09-13T18:57:27.645812+0000","last_undegraded":"2024-09-13T18:57:27.645812+0000","last_fullsized":"2024-09-13T18:57:27.645812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'7","last_scrub_stamp":"2024-09-13T18:57:18.823152+0000","last_deep_scrub":"43'7","last_deep_scrub_stamp":"2024-09-13T18:57:18.823152+0000","last_clean_scrub_stamp":"2024-09-13T18:57:18.823152+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:52.572271+0000","scrub_duration":0.023118413000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"43'5405","reported_seq":5489,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.568512+0000","last_change":"2024-09-13T18:57:36.167581+0000","last_active":"2024-09-13T18:57:38.568512+0000","last_peered":"2024-09-13T18:57:38.568512+0000","last_clean":"2024-09-13T18:57:38.568512+0000","last_became_active":"2024-09-13T18:26:36.409345+0000","last_became_peered":"2024-09-13T18:26:36.409345+0000","last_unstale":"2024-09-13T18:57:38.568512+0000","last_undegraded":"2024-09-13T18:57:38.568512+0000","last_fullsized":"2024-09-13T18:57:38.568512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5395","last_scrub_stamp":"2024-09-13T18:57:36.167414+0000","last_deep_scrub":"43'5395","last_deep_scrub_stamp":"2024-09-13T18:57:36.167414+0000","last_clean_scrub_stamp":"2024-09-13T18:57:36.167414+0000","objects_scrubbed":2,"log_size":5405,"log_dups_size":0,"ondisk_log_size":5405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:04:32.293024+0000","scrub_duration":0.24736883800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":24452,"num_write_kb":507343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"43'13","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.384061+0000","last_change":"2024-09-13T18:57:17.064133+0000","last_active":"2024-09-13T18:57:28.384061+0000","last_peered":"2024-09-13T18:57:28.384061+0000","last_clean":"2024-09-13T18:57:28.384061+0000","last_became_active":"2024-09-13T18:26:32.358791+0000","last_became_peered":"2024-09-13T18:26:32.358791+0000","last_unstale":"2024-09-13T18:57:28.384061+0000","last_undegraded":"2024-09-13T18:57:28.384061+0000","last_fullsized":"2024-09-13T18:57:28.384061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:17.063923+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:17.063923+0000","last_clean_scrub_stamp":"2024-09-13T18:57:17.063923+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.609366+0000","scrub_duration":0.020682738999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"43'289","reported_seq":363,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.907282+0000","last_change":"2024-09-13T18:57:22.143854+0000","last_active":"2024-09-13T18:57:24.907282+0000","last_peered":"2024-09-13T18:57:24.907282+0000","last_clean":"2024-09-13T18:57:24.907282+0000","last_became_active":"2024-09-13T18:26:31.968664+0000","last_became_peered":"2024-09-13T18:26:31.968664+0000","last_unstale":"2024-09-13T18:57:24.907282+0000","last_undegraded":"2024-09-13T18:57:24.907282+0000","last_fullsized":"2024-09-13T18:57:24.907282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'286","last_scrub_stamp":"2024-09-13T18:57:22.143716+0000","last_deep_scrub":"43'286","last_deep_scrub_stamp":"2024-09-13T18:57:22.143716+0000","last_clean_scrub_stamp":"2024-09-13T18:57:22.143716+0000","objects_scrubbed":2,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:44.447938+0000","scrub_duration":0.0070881099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":52,"num_write":626,"num_write_kb":212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"43'6116","reported_seq":6209,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:05.950781+0000","last_change":"2024-09-13T18:58:05.950968+0000","last_active":"2024-09-13T18:58:05.950781+0000","last_peered":"2024-09-13T18:58:05.950781+0000","last_clean":"2024-09-13T18:58:05.950781+0000","last_became_active":"2024-09-13T18:27:22.644681+0000","last_became_peered":"2024-09-13T18:27:22.644681+0000","last_unstale":"2024-09-13T18:58:05.950781+0000","last_undegraded":"2024-09-13T18:58:05.950781+0000","last_fullsized":"2024-09-13T18:58:05.950781+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6116","last_scrub_stamp":"2024-09-13T18:58:05.950737+0000","last_deep_scrub":"43'6116","last_deep_scrub_stamp":"2024-09-13T18:58:05.950737+0000","last_clean_scrub_stamp":"2024-09-13T18:58:05.950737+0000","objects_scrubbed":0,"log_size":6116,"log_dups_size":0,"ondisk_log_size":6116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:59:36.744911+0000","scrub_duration":0.042980222999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":27902,"num_write_kb":553947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"43'9","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.380487+0000","last_change":"2024-09-13T18:57:13.217203+0000","last_active":"2024-09-13T18:57:28.380487+0000","last_peered":"2024-09-13T18:57:28.380487+0000","last_clean":"2024-09-13T18:57:28.380487+0000","last_became_active":"2024-09-13T18:26:32.342608+0000","last_became_peered":"2024-09-13T18:26:32.342608+0000","last_unstale":"2024-09-13T18:57:28.380487+0000","last_undegraded":"2024-09-13T18:57:28.380487+0000","last_fullsized":"2024-09-13T18:57:28.380487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:13.216878+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:13.216878+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.216878+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:25:20.911582+0000","scrub_duration":0.0034586249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"43'8","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:29.997531+0000","last_change":"2024-09-13T18:57:29.997531+0000","last_active":"2024-09-13T18:57:29.997531+0000","last_peered":"2024-09-13T18:57:29.997531+0000","last_clean":"2024-09-13T18:57:29.997531+0000","last_became_active":"2024-09-13T18:26:31.978672+0000","last_became_peered":"2024-09-13T18:26:31.978672+0000","last_unstale":"2024-09-13T18:57:29.997531+0000","last_undegraded":"2024-09-13T18:57:29.997531+0000","last_fullsized":"2024-09-13T18:57:29.997531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:29.997356+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:29.997356+0000","last_clean_scrub_stamp":"2024-09-13T18:57:29.997356+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:42:58.364306+0000","scrub_duration":0.076701386999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"43'4791","reported_seq":4867,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:53.311285+0000","last_change":"2024-09-13T18:57:53.311285+0000","last_active":"2024-09-13T18:57:53.311285+0000","last_peered":"2024-09-13T18:57:53.311285+0000","last_clean":"2024-09-13T18:57:53.311285+0000","last_became_active":"2024-09-13T18:26:36.408396+0000","last_became_peered":"2024-09-13T18:26:36.408396+0000","last_unstale":"2024-09-13T18:57:53.311285+0000","last_undegraded":"2024-09-13T18:57:53.311285+0000","last_fullsized":"2024-09-13T18:57:53.311285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4791","last_scrub_stamp":"2024-09-13T18:57:53.311114+0000","last_deep_scrub":"43'4791","last_deep_scrub_stamp":"2024-09-13T18:57:53.311114+0000","last_clean_scrub_stamp":"2024-09-13T18:57:53.311114+0000","objects_scrubbed":0,"log_size":4791,"log_dups_size":0,"ondisk_log_size":4791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:13.675488+0000","scrub_duration":0.03002403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":22435,"num_write_kb":450115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"43'12","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.357797+0000","last_change":"2024-09-13T18:57:13.000734+0000","last_active":"2024-09-13T18:57:38.357797+0000","last_peered":"2024-09-13T18:57:38.357797+0000","last_clean":"2024-09-13T18:57:38.357797+0000","last_became_active":"2024-09-13T18:26:32.354776+0000","last_became_peered":"2024-09-13T18:26:32.354776+0000","last_unstale":"2024-09-13T18:57:38.357797+0000","last_undegraded":"2024-09-13T18:57:38.357797+0000","last_fullsized":"2024-09-13T18:57:38.357797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'7","last_scrub_stamp":"2024-09-13T18:57:13.000480+0000","last_deep_scrub":"28'7","last_deep_scrub_stamp":"2024-09-13T18:57:13.000480+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.000480+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:33.028238+0000","scrub_duration":0.0036870150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"43'3","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.389515+0000","last_change":"2024-09-13T18:57:14.228792+0000","last_active":"2024-09-13T18:57:27.389515+0000","last_peered":"2024-09-13T18:57:27.389515+0000","last_clean":"2024-09-13T18:57:27.389515+0000","last_became_active":"2024-09-13T18:26:31.972634+0000","last_became_peered":"2024-09-13T18:26:31.972634+0000","last_unstale":"2024-09-13T18:57:27.389515+0000","last_undegraded":"2024-09-13T18:57:27.389515+0000","last_fullsized":"2024-09-13T18:57:27.389515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:57:14.228519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:57:14.228519+0000","last_clean_scrub_stamp":"2024-09-13T18:57:14.228519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:55.913907+0000","scrub_duration":0.0032354189999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"43'6174","reported_seq":6256,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:56.237918+0000","last_change":"2024-09-13T18:57:56.238103+0000","last_active":"2024-09-13T18:57:56.237918+0000","last_peered":"2024-09-13T18:57:56.237918+0000","last_clean":"2024-09-13T18:57:56.237918+0000","last_became_active":"2024-09-13T18:26:36.398463+0000","last_became_peered":"2024-09-13T18:26:36.398463+0000","last_unstale":"2024-09-13T18:57:56.237918+0000","last_undegraded":"2024-09-13T18:57:56.237918+0000","last_fullsized":"2024-09-13T18:57:56.237918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6174","last_scrub_stamp":"2024-09-13T18:57:56.237894+0000","last_deep_scrub":"43'6174","last_deep_scrub_stamp":"2024-09-13T18:57:56.237894+0000","last_clean_scrub_stamp":"2024-09-13T18:57:56.237894+0000","objects_scrubbed":0,"log_size":6174,"log_dups_size":0,"ondisk_log_size":6174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:07:22.671920+0000","scrub_duration":0.045051943999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":28067,"num_write_kb":562911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"43'9","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.442033+0000","last_change":"2024-09-13T18:57:12.010586+0000","last_active":"2024-09-13T18:57:26.442033+0000","last_peered":"2024-09-13T18:57:26.442033+0000","last_clean":"2024-09-13T18:57:26.442033+0000","last_became_active":"2024-09-13T18:26:32.352847+0000","last_became_peered":"2024-09-13T18:26:32.352847+0000","last_unstale":"2024-09-13T18:57:26.442033+0000","last_undegraded":"2024-09-13T18:57:26.442033+0000","last_fullsized":"2024-09-13T18:57:26.442033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'6","last_scrub_stamp":"2024-09-13T18:57:12.010428+0000","last_deep_scrub":"28'6","last_deep_scrub_stamp":"2024-09-13T18:57:12.010428+0000","last_clean_scrub_stamp":"2024-09-13T18:57:12.010428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:15.823406+0000","scrub_duration":0.0039755040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"43'9","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.428380+0000","last_change":"2024-09-13T18:57:15.993201+0000","last_active":"2024-09-13T18:57:26.428380+0000","last_peered":"2024-09-13T18:57:26.428380+0000","last_clean":"2024-09-13T18:57:26.428380+0000","last_became_active":"2024-09-13T18:26:31.975791+0000","last_became_peered":"2024-09-13T18:26:31.975791+0000","last_unstale":"2024-09-13T18:57:26.428380+0000","last_undegraded":"2024-09-13T18:57:26.428380+0000","last_fullsized":"2024-09-13T18:57:26.428380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'6","last_scrub_stamp":"2024-09-13T18:57:15.992890+0000","last_deep_scrub":"41'6","last_deep_scrub_stamp":"2024-09-13T18:57:15.992890+0000","last_clean_scrub_stamp":"2024-09-13T18:57:15.992890+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:28:43.312440+0000","scrub_duration":0.0036839949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"43'5413","reported_seq":5498,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:54.240011+0000","last_change":"2024-09-13T18:57:54.240011+0000","last_active":"2024-09-13T18:57:54.240011+0000","last_peered":"2024-09-13T18:57:54.240011+0000","last_clean":"2024-09-13T18:57:54.240011+0000","last_became_active":"2024-09-13T18:26:36.401407+0000","last_became_peered":"2024-09-13T18:26:36.401407+0000","last_unstale":"2024-09-13T18:57:54.240011+0000","last_undegraded":"2024-09-13T18:57:54.240011+0000","last_fullsized":"2024-09-13T18:57:54.240011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5413","last_scrub_stamp":"2024-09-13T18:57:54.239842+0000","last_deep_scrub":"43'5413","last_deep_scrub_stamp":"2024-09-13T18:57:54.239842+0000","last_clean_scrub_stamp":"2024-09-13T18:57:54.239842+0000","objects_scrubbed":0,"log_size":5413,"log_dups_size":0,"ondisk_log_size":5413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:36.754881+0000","scrub_duration":0.035324452999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":24545,"num_write_kb":507684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"43'14","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:36.275994+0000","last_change":"2024-09-13T18:57:36.277633+0000","last_active":"2024-09-13T18:57:36.275994+0000","last_peered":"2024-09-13T18:57:36.275994+0000","last_clean":"2024-09-13T18:57:36.275994+0000","last_became_active":"2024-09-13T18:26:32.343518+0000","last_became_peered":"2024-09-13T18:26:32.343518+0000","last_unstale":"2024-09-13T18:57:36.275994+0000","last_undegraded":"2024-09-13T18:57:36.275994+0000","last_fullsized":"2024-09-13T18:57:36.275994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:36.275964+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:36.275964+0000","last_clean_scrub_stamp":"2024-09-13T18:57:36.275964+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:55:23.015452+0000","scrub_duration":0.027403200999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"43'131","reported_seq":318,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.095368+0000","last_change":"2024-09-13T18:57:16.151559+0000","last_active":"2024-09-13T18:57:28.095368+0000","last_peered":"2024-09-13T18:57:28.095368+0000","last_clean":"2024-09-13T18:57:28.095368+0000","last_became_active":"2024-09-13T18:26:31.970482+0000","last_became_peered":"2024-09-13T18:26:31.970482+0000","last_unstale":"2024-09-13T18:57:28.095368+0000","last_undegraded":"2024-09-13T18:57:28.095368+0000","last_fullsized":"2024-09-13T18:57:28.095368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'130","last_scrub_stamp":"2024-09-13T18:57:16.151397+0000","last_deep_scrub":"43'130","last_deep_scrub_stamp":"2024-09-13T18:57:16.151397+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.151397+0000","objects_scrubbed":3,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:19.471742+0000","scrub_duration":0.027827343000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93931,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":203,"num_write":119,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"43'6320","reported_seq":6402,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:01.115048+0000","last_change":"2024-09-13T18:58:01.115201+0000","last_active":"2024-09-13T18:58:01.115048+0000","last_peered":"2024-09-13T18:58:01.115048+0000","last_clean":"2024-09-13T18:58:01.115048+0000","last_became_active":"2024-09-13T18:26:36.401100+0000","last_became_peered":"2024-09-13T18:26:36.401100+0000","last_unstale":"2024-09-13T18:58:01.115048+0000","last_undegraded":"2024-09-13T18:58:01.115048+0000","last_fullsized":"2024-09-13T18:58:01.115048+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6320","last_scrub_stamp":"2024-09-13T18:58:01.115028+0000","last_deep_scrub":"43'6320","last_deep_scrub_stamp":"2024-09-13T18:58:01.115028+0000","last_clean_scrub_stamp":"2024-09-13T18:58:01.115028+0000","objects_scrubbed":0,"log_size":6320,"log_dups_size":0,"ondisk_log_size":6320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:12:25.681334+0000","scrub_duration":0.041107435999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":28303,"num_write_kb":563741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"43'8","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.026232+0000","last_change":"2024-09-13T18:57:14.329551+0000","last_active":"2024-09-13T18:57:28.026232+0000","last_peered":"2024-09-13T18:57:28.026232+0000","last_clean":"2024-09-13T18:57:28.026232+0000","last_became_active":"2024-09-13T18:26:32.350224+0000","last_became_peered":"2024-09-13T18:26:32.350224+0000","last_unstale":"2024-09-13T18:57:28.026232+0000","last_undegraded":"2024-09-13T18:57:28.026232+0000","last_fullsized":"2024-09-13T18:57:28.026232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:14.329297+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:14.329297+0000","last_clean_scrub_stamp":"2024-09-13T18:57:14.329297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:49.431750+0000","scrub_duration":0.0033135230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"43'18","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.912770+0000","last_change":"2024-09-13T18:57:26.912770+0000","last_active":"2024-09-13T18:57:26.912770+0000","last_peered":"2024-09-13T18:57:26.912770+0000","last_clean":"2024-09-13T18:57:26.912770+0000","last_became_active":"2024-09-13T18:26:31.972281+0000","last_became_peered":"2024-09-13T18:26:31.972281+0000","last_unstale":"2024-09-13T18:57:26.912770+0000","last_undegraded":"2024-09-13T18:57:26.912770+0000","last_fullsized":"2024-09-13T18:57:26.912770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'17","last_scrub_stamp":"2024-09-13T18:57:26.912423+0000","last_deep_scrub":"43'17","last_deep_scrub_stamp":"2024-09-13T18:57:26.912423+0000","last_clean_scrub_stamp":"2024-09-13T18:57:26.912423+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:00.997526+0000","scrub_duration":0.0065643969999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"43'4467","reported_seq":4549,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:49.291700+0000","last_change":"2024-09-13T18:57:49.291907+0000","last_active":"2024-09-13T18:57:49.291700+0000","last_peered":"2024-09-13T18:57:49.291700+0000","last_clean":"2024-09-13T18:57:49.291700+0000","last_became_active":"2024-09-13T18:26:36.404724+0000","last_became_peered":"2024-09-13T18:26:36.404724+0000","last_unstale":"2024-09-13T18:57:49.291700+0000","last_undegraded":"2024-09-13T18:57:49.291700+0000","last_fullsized":"2024-09-13T18:57:49.291700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4467","last_scrub_stamp":"2024-09-13T18:57:49.291657+0000","last_deep_scrub":"43'4467","last_deep_scrub_stamp":"2024-09-13T18:57:49.291657+0000","last_clean_scrub_stamp":"2024-09-13T18:57:49.291657+0000","objects_scrubbed":0,"log_size":4467,"log_dups_size":0,"ondisk_log_size":4467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:19.421097+0000","scrub_duration":0.030143559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":19862,"num_write_kb":435588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"43'8","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.860482+0000","last_change":"2024-09-13T18:57:39.860482+0000","last_active":"2024-09-13T18:57:39.860482+0000","last_peered":"2024-09-13T18:57:39.860482+0000","last_clean":"2024-09-13T18:57:39.860482+0000","last_became_active":"2024-09-13T18:26:32.350271+0000","last_became_peered":"2024-09-13T18:26:32.350271+0000","last_unstale":"2024-09-13T18:57:39.860482+0000","last_undegraded":"2024-09-13T18:57:39.860482+0000","last_fullsized":"2024-09-13T18:57:39.860482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:39.860244+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:39.860244+0000","last_clean_scrub_stamp":"2024-09-13T18:57:39.860244+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:08.621392+0000","scrub_duration":0.0035530589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"43'11","reported_seq":90,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:23.965985+0000","last_change":"2024-09-13T18:57:23.965985+0000","last_active":"2024-09-13T18:57:23.965985+0000","last_peered":"2024-09-13T18:57:23.965985+0000","last_clean":"2024-09-13T18:57:23.965985+0000","last_became_active":"2024-09-13T18:26:31.978081+0000","last_became_peered":"2024-09-13T18:26:31.978081+0000","last_unstale":"2024-09-13T18:57:23.965985+0000","last_undegraded":"2024-09-13T18:57:23.965985+0000","last_fullsized":"2024-09-13T18:57:23.965985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:23.965843+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:23.965843+0000","last_clean_scrub_stamp":"2024-09-13T18:57:23.965843+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:51.676617+0000","scrub_duration":0.010225193,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":68,"num_write":9,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"43'119","reported_seq":216,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:19.933697+0000","last_change":"2024-09-13T18:57:19.933697+0000","last_active":"2024-09-13T18:57:19.933697+0000","last_peered":"2024-09-13T18:57:19.933697+0000","last_clean":"2024-09-13T18:57:19.933697+0000","last_became_active":"2024-09-13T18:26:26.929915+0000","last_became_peered":"2024-09-13T18:26:26.929915+0000","last_unstale":"2024-09-13T18:57:19.933697+0000","last_undegraded":"2024-09-13T18:57:19.933697+0000","last_fullsized":"2024-09-13T18:57:19.933697+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'119","last_scrub_stamp":"2024-09-13T18:57:19.933561+0000","last_deep_scrub":"43'119","last_deep_scrub_stamp":"2024-09-13T18:57:19.933561+0000","last_clean_scrub_stamp":"2024-09-13T18:57:19.933561+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:36.963255+0000","scrub_duration":0.0099632039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"43'6003","reported_seq":6095,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:03.635883+0000","last_change":"2024-09-13T18:58:03.636038+0000","last_active":"2024-09-13T18:58:03.635883+0000","last_peered":"2024-09-13T18:58:03.635883+0000","last_clean":"2024-09-13T18:58:03.635883+0000","last_became_active":"2024-09-13T18:26:36.409425+0000","last_became_peered":"2024-09-13T18:26:36.409425+0000","last_unstale":"2024-09-13T18:58:03.635883+0000","last_undegraded":"2024-09-13T18:58:03.635883+0000","last_fullsized":"2024-09-13T18:58:03.635883+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6003","last_scrub_stamp":"2024-09-13T18:58:03.635861+0000","last_deep_scrub":"43'6003","last_deep_scrub_stamp":"2024-09-13T18:58:03.635861+0000","last_clean_scrub_stamp":"2024-09-13T18:58:03.635861+0000","objects_scrubbed":0,"log_size":6003,"log_dups_size":0,"ondisk_log_size":6003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:38:50.326698+0000","scrub_duration":0.046218636,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":27421,"num_write_kb":519330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"43'15","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:10.113914+0000","last_change":"2024-09-13T18:58:10.114122+0000","last_active":"2024-09-13T18:58:10.113914+0000","last_peered":"2024-09-13T18:58:10.113914+0000","last_clean":"2024-09-13T18:58:10.113914+0000","last_became_active":"2024-09-13T18:26:32.354408+0000","last_became_peered":"2024-09-13T18:26:32.354408+0000","last_unstale":"2024-09-13T18:58:10.113914+0000","last_undegraded":"2024-09-13T18:58:10.113914+0000","last_fullsized":"2024-09-13T18:58:10.113914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'15","last_scrub_stamp":"2024-09-13T18:58:10.113857+0000","last_deep_scrub":"43'15","last_deep_scrub_stamp":"2024-09-13T18:58:10.113857+0000","last_clean_scrub_stamp":"2024-09-13T18:58:10.113857+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:13.910431+0000","scrub_duration":0.003224832,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"43'8","reported_seq":82,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.127170+0000","last_change":"2024-09-13T18:57:21.187725+0000","last_active":"2024-09-13T18:57:28.127170+0000","last_peered":"2024-09-13T18:57:28.127170+0000","last_clean":"2024-09-13T18:57:28.127170+0000","last_became_active":"2024-09-13T18:26:31.975637+0000","last_became_peered":"2024-09-13T18:26:31.975637+0000","last_unstale":"2024-09-13T18:57:28.127170+0000","last_undegraded":"2024-09-13T18:57:28.127170+0000","last_fullsized":"2024-09-13T18:57:28.127170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:21.187525+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:21.187525+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.187525+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:42.675115+0000","scrub_duration":0.032458872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":35,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"43'5433","reported_seq":5509,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:57.609674+0000","last_change":"2024-09-13T18:57:57.609833+0000","last_active":"2024-09-13T18:57:57.609674+0000","last_peered":"2024-09-13T18:57:57.609674+0000","last_clean":"2024-09-13T18:57:57.609674+0000","last_became_active":"2024-09-13T18:26:36.409369+0000","last_became_peered":"2024-09-13T18:26:36.409369+0000","last_unstale":"2024-09-13T18:57:57.609674+0000","last_undegraded":"2024-09-13T18:57:57.609674+0000","last_fullsized":"2024-09-13T18:57:57.609674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5433","last_scrub_stamp":"2024-09-13T18:57:57.609650+0000","last_deep_scrub":"43'5433","last_deep_scrub_stamp":"2024-09-13T18:57:57.609650+0000","last_clean_scrub_stamp":"2024-09-13T18:57:57.609650+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:39:53.104071+0000","scrub_duration":0.043346968999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":24067,"num_write_kb":513904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"43'13","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:36.120773+0000","last_change":"2024-09-13T18:57:36.120879+0000","last_active":"2024-09-13T18:57:36.120773+0000","last_peered":"2024-09-13T18:57:36.120773+0000","last_clean":"2024-09-13T18:57:36.120773+0000","last_became_active":"2024-09-13T18:27:22.610326+0000","last_became_peered":"2024-09-13T18:27:22.610326+0000","last_unstale":"2024-09-13T18:57:36.120773+0000","last_undegraded":"2024-09-13T18:57:36.120773+0000","last_fullsized":"2024-09-13T18:57:36.120773+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:36.120741+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:36.120741+0000","last_clean_scrub_stamp":"2024-09-13T18:57:36.120741+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:03.902589+0000","scrub_duration":0.0026569660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.167920+0000","last_change":"2024-09-13T18:57:27.167920+0000","last_active":"2024-09-13T18:57:27.167920+0000","last_peered":"2024-09-13T18:57:27.167920+0000","last_clean":"2024-09-13T18:57:27.167920+0000","last_became_active":"2024-09-13T18:26:31.966326+0000","last_became_peered":"2024-09-13T18:26:31.966326+0000","last_unstale":"2024-09-13T18:57:27.167920+0000","last_undegraded":"2024-09-13T18:57:27.167920+0000","last_fullsized":"2024-09-13T18:57:27.167920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:27.167690+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:27.167690+0000","last_clean_scrub_stamp":"2024-09-13T18:57:27.167690+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:44.845487+0000","scrub_duration":0.0066826000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"43'5174","reported_seq":5254,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:50.325810+0000","last_change":"2024-09-13T18:57:50.325810+0000","last_active":"2024-09-13T18:57:50.325810+0000","last_peered":"2024-09-13T18:57:50.325810+0000","last_clean":"2024-09-13T18:57:50.325810+0000","last_became_active":"2024-09-13T18:26:36.393644+0000","last_became_peered":"2024-09-13T18:26:36.393644+0000","last_unstale":"2024-09-13T18:57:50.325810+0000","last_undegraded":"2024-09-13T18:57:50.325810+0000","last_fullsized":"2024-09-13T18:57:50.325810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5174","last_scrub_stamp":"2024-09-13T18:57:50.325606+0000","last_deep_scrub":"43'5174","last_deep_scrub_stamp":"2024-09-13T18:57:50.325606+0000","last_clean_scrub_stamp":"2024-09-13T18:57:50.325606+0000","objects_scrubbed":0,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:33:51.857590+0000","scrub_duration":0.032878729000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":22830,"num_write_kb":472029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"43'11","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:59.101489+0000","last_change":"2024-09-13T18:57:59.101489+0000","last_active":"2024-09-13T18:57:59.101489+0000","last_peered":"2024-09-13T18:57:59.101489+0000","last_clean":"2024-09-13T18:57:59.101489+0000","last_became_active":"2024-09-13T18:26:32.365374+0000","last_became_peered":"2024-09-13T18:26:32.365374+0000","last_unstale":"2024-09-13T18:57:59.101489+0000","last_undegraded":"2024-09-13T18:57:59.101489+0000","last_fullsized":"2024-09-13T18:57:59.101489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:59.101212+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:59.101212+0000","last_clean_scrub_stamp":"2024-09-13T18:57:59.101212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:06.288499+0000","scrub_duration":0.003501742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"43'10","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.058571+0000","last_change":"2024-09-13T18:57:25.103279+0000","last_active":"2024-09-13T18:57:28.058571+0000","last_peered":"2024-09-13T18:57:28.058571+0000","last_clean":"2024-09-13T18:57:28.058571+0000","last_became_active":"2024-09-13T18:26:31.970361+0000","last_became_peered":"2024-09-13T18:26:31.970361+0000","last_unstale":"2024-09-13T18:57:28.058571+0000","last_undegraded":"2024-09-13T18:57:28.058571+0000","last_fullsized":"2024-09-13T18:57:28.058571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:25.103144+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:25.103144+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.103144+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.183107+0000","scrub_duration":0.013012788000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"43'4987","reported_seq":5060,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:04.306952+0000","last_change":"2024-09-13T18:58:04.307122+0000","last_active":"2024-09-13T18:58:04.306952+0000","last_peered":"2024-09-13T18:58:04.306952+0000","last_clean":"2024-09-13T18:58:04.306952+0000","last_became_active":"2024-09-13T18:26:36.409905+0000","last_became_peered":"2024-09-13T18:26:36.409905+0000","last_unstale":"2024-09-13T18:58:04.306952+0000","last_undegraded":"2024-09-13T18:58:04.306952+0000","last_fullsized":"2024-09-13T18:58:04.306952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4987","last_scrub_stamp":"2024-09-13T18:58:04.306926+0000","last_deep_scrub":"43'4987","last_deep_scrub_stamp":"2024-09-13T18:58:04.306926+0000","last_clean_scrub_stamp":"2024-09-13T18:58:04.306926+0000","objects_scrubbed":0,"log_size":4987,"log_dups_size":0,"ondisk_log_size":4987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:53:40.632719+0000","scrub_duration":0.035690187999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":0,"num_write":22775,"num_write_kb":463670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"43'5","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:58.077320+0000","last_change":"2024-09-13T18:57:58.077320+0000","last_active":"2024-09-13T18:57:58.077320+0000","last_peered":"2024-09-13T18:57:58.077320+0000","last_clean":"2024-09-13T18:57:58.077320+0000","last_became_active":"2024-09-13T18:26:32.361293+0000","last_became_peered":"2024-09-13T18:26:32.361293+0000","last_unstale":"2024-09-13T18:57:58.077320+0000","last_undegraded":"2024-09-13T18:57:58.077320+0000","last_fullsized":"2024-09-13T18:57:58.077320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:58.077036+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:58.077036+0000","last_clean_scrub_stamp":"2024-09-13T18:57:58.077036+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:31.127665+0000","scrub_duration":0.0029704359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"43'12","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.644779+0000","last_change":"2024-09-13T18:57:11.025964+0000","last_active":"2024-09-13T18:57:27.644779+0000","last_peered":"2024-09-13T18:57:27.644779+0000","last_clean":"2024-09-13T18:57:27.644779+0000","last_became_active":"2024-09-13T18:26:31.976736+0000","last_became_peered":"2024-09-13T18:26:31.976736+0000","last_unstale":"2024-09-13T18:57:27.644779+0000","last_undegraded":"2024-09-13T18:57:27.644779+0000","last_fullsized":"2024-09-13T18:57:27.644779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:11.025748+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:11.025748+0000","last_clean_scrub_stamp":"2024-09-13T18:57:11.025748+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:22:07.957036+0000","scrub_duration":0.0065355530000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"43'5147","reported_seq":5224,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:51.917449+0000","last_change":"2024-09-13T18:57:51.917748+0000","last_active":"2024-09-13T18:57:51.917449+0000","last_peered":"2024-09-13T18:57:51.917449+0000","last_clean":"2024-09-13T18:57:51.917449+0000","last_became_active":"2024-09-13T18:26:36.403013+0000","last_became_peered":"2024-09-13T18:26:36.403013+0000","last_unstale":"2024-09-13T18:57:51.917449+0000","last_undegraded":"2024-09-13T18:57:51.917449+0000","last_fullsized":"2024-09-13T18:57:51.917449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5147","last_scrub_stamp":"2024-09-13T18:57:51.917380+0000","last_deep_scrub":"43'5147","last_deep_scrub_stamp":"2024-09-13T18:57:51.917380+0000","last_clean_scrub_stamp":"2024-09-13T18:57:51.917380+0000","objects_scrubbed":0,"log_size":5147,"log_dups_size":0,"ondisk_log_size":5147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:11:21.117712+0000","scrub_duration":0.041709001000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":22715,"num_write_kb":463382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"43'12","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:56.116016+0000","last_change":"2024-09-13T18:57:56.116316+0000","last_active":"2024-09-13T18:57:56.116016+0000","last_peered":"2024-09-13T18:57:56.116016+0000","last_clean":"2024-09-13T18:57:56.116016+0000","last_became_active":"2024-09-13T18:26:32.364779+0000","last_became_peered":"2024-09-13T18:26:32.364779+0000","last_unstale":"2024-09-13T18:57:56.116016+0000","last_undegraded":"2024-09-13T18:57:56.116016+0000","last_fullsized":"2024-09-13T18:57:56.116016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:57:56.115968+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:57:56.115968+0000","last_clean_scrub_stamp":"2024-09-13T18:57:56.115968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:35.297035+0000","scrub_duration":0.0033167280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"43'32","reported_seq":149,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.902980+0000","last_change":"2024-09-13T18:57:20.154497+0000","last_active":"2024-09-13T18:57:24.902980+0000","last_peered":"2024-09-13T18:57:24.902980+0000","last_clean":"2024-09-13T18:57:24.902980+0000","last_became_active":"2024-09-13T18:26:31.965482+0000","last_became_peered":"2024-09-13T18:26:31.965482+0000","last_unstale":"2024-09-13T18:57:24.902980+0000","last_undegraded":"2024-09-13T18:57:24.902980+0000","last_fullsized":"2024-09-13T18:57:24.902980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'29","last_scrub_stamp":"2024-09-13T18:57:20.154238+0000","last_deep_scrub":"43'29","last_deep_scrub_stamp":"2024-09-13T18:57:20.154238+0000","last_clean_scrub_stamp":"2024-09-13T18:57:20.154238+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:10.657893+0000","scrub_duration":0.013053932000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":158,"num_read_kb":157,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"43'6074","reported_seq":6160,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:55.319773+0000","last_change":"2024-09-13T18:57:55.319773+0000","last_active":"2024-09-13T18:57:55.319773+0000","last_peered":"2024-09-13T18:57:55.319773+0000","last_clean":"2024-09-13T18:57:55.319773+0000","last_became_active":"2024-09-13T18:26:36.398871+0000","last_became_peered":"2024-09-13T18:26:36.398871+0000","last_unstale":"2024-09-13T18:57:55.319773+0000","last_undegraded":"2024-09-13T18:57:55.319773+0000","last_fullsized":"2024-09-13T18:57:55.319773+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6074","last_scrub_stamp":"2024-09-13T18:57:55.319608+0000","last_deep_scrub":"43'6074","last_deep_scrub_stamp":"2024-09-13T18:57:55.319608+0000","last_clean_scrub_stamp":"2024-09-13T18:57:55.319608+0000","objects_scrubbed":0,"log_size":6074,"log_dups_size":0,"ondisk_log_size":6074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:12.572658+0000","scrub_duration":0.041949826000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":27740,"num_write_kb":536907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"43'13","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:32.703996+0000","last_change":"2024-09-13T18:57:32.704206+0000","last_active":"2024-09-13T18:57:32.703996+0000","last_peered":"2024-09-13T18:57:32.703996+0000","last_clean":"2024-09-13T18:57:32.703996+0000","last_became_active":"2024-09-13T18:26:32.353947+0000","last_became_peered":"2024-09-13T18:26:32.353947+0000","last_unstale":"2024-09-13T18:57:32.703996+0000","last_undegraded":"2024-09-13T18:57:32.703996+0000","last_fullsized":"2024-09-13T18:57:32.703996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:32.703957+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:32.703957+0000","last_clean_scrub_stamp":"2024-09-13T18:57:32.703957+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:39.171311+0000","scrub_duration":0.018593416000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"43'16","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.908758+0000","last_change":"2024-09-13T18:57:21.771790+0000","last_active":"2024-09-13T18:57:24.908758+0000","last_peered":"2024-09-13T18:57:24.908758+0000","last_clean":"2024-09-13T18:57:24.908758+0000","last_became_active":"2024-09-13T18:26:31.975853+0000","last_became_peered":"2024-09-13T18:26:31.975853+0000","last_unstale":"2024-09-13T18:57:24.908758+0000","last_undegraded":"2024-09-13T18:57:24.908758+0000","last_fullsized":"2024-09-13T18:57:24.908758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:21.771519+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:21.771519+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.771519+0000","objects_scrubbed":2,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:44:46.293266+0000","scrub_duration":0.0083382200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1620,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"43'6155","reported_seq":6234,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:02.974017+0000","last_change":"2024-09-13T18:58:02.974017+0000","last_active":"2024-09-13T18:58:02.974017+0000","last_peered":"2024-09-13T18:58:02.974017+0000","last_clean":"2024-09-13T18:58:02.974017+0000","last_became_active":"2024-09-13T18:26:36.405724+0000","last_became_peered":"2024-09-13T18:26:36.405724+0000","last_unstale":"2024-09-13T18:58:02.974017+0000","last_undegraded":"2024-09-13T18:58:02.974017+0000","last_fullsized":"2024-09-13T18:58:02.974017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6155","last_scrub_stamp":"2024-09-13T18:58:02.973843+0000","last_deep_scrub":"43'6155","last_deep_scrub_stamp":"2024-09-13T18:58:02.973843+0000","last_clean_scrub_stamp":"2024-09-13T18:58:02.973843+0000","objects_scrubbed":0,"log_size":6155,"log_dups_size":0,"ondisk_log_size":6155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:07.662090+0000","scrub_duration":0.049362212000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":28268,"num_write_kb":551294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"43'15","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:03.046311+0000","last_change":"2024-09-13T18:58:03.046420+0000","last_active":"2024-09-13T18:58:03.046311+0000","last_peered":"2024-09-13T18:58:03.046311+0000","last_clean":"2024-09-13T18:58:03.046311+0000","last_became_active":"2024-09-13T18:27:22.609101+0000","last_became_peered":"2024-09-13T18:27:22.609101+0000","last_unstale":"2024-09-13T18:58:03.046311+0000","last_undegraded":"2024-09-13T18:58:03.046311+0000","last_fullsized":"2024-09-13T18:58:03.046311+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'15","last_scrub_stamp":"2024-09-13T18:58:03.046278+0000","last_deep_scrub":"43'15","last_deep_scrub_stamp":"2024-09-13T18:58:03.046278+0000","last_clean_scrub_stamp":"2024-09-13T18:58:03.046278+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:52:09.099898+0000","scrub_duration":0.00304259,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"43'14","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:35.005652+0000","last_change":"2024-09-13T18:57:35.005769+0000","last_active":"2024-09-13T18:57:35.005652+0000","last_peered":"2024-09-13T18:57:35.005652+0000","last_clean":"2024-09-13T18:57:35.005652+0000","last_became_active":"2024-09-13T18:26:31.981841+0000","last_became_peered":"2024-09-13T18:26:31.981841+0000","last_unstale":"2024-09-13T18:57:35.005652+0000","last_undegraded":"2024-09-13T18:57:35.005652+0000","last_fullsized":"2024-09-13T18:57:35.005652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:35.005633+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:35.005633+0000","last_clean_scrub_stamp":"2024-09-13T18:57:35.005633+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:24.849838+0000","scrub_duration":0.0050291620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"43'5953","reported_seq":6035,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:14.859250+0000","last_change":"2024-09-13T18:58:14.859406+0000","last_active":"2024-09-13T18:58:14.859250+0000","last_peered":"2024-09-13T18:58:14.859250+0000","last_clean":"2024-09-13T18:58:14.859250+0000","last_became_active":"2024-09-13T18:26:36.405646+0000","last_became_peered":"2024-09-13T18:26:36.405646+0000","last_unstale":"2024-09-13T18:58:14.859250+0000","last_undegraded":"2024-09-13T18:58:14.859250+0000","last_fullsized":"2024-09-13T18:58:14.859250+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5953","last_scrub_stamp":"2024-09-13T18:58:14.859225+0000","last_deep_scrub":"43'5953","last_deep_scrub_stamp":"2024-09-13T18:58:14.859225+0000","last_clean_scrub_stamp":"2024-09-13T18:58:14.859225+0000","objects_scrubbed":0,"log_size":5953,"log_dups_size":0,"ondisk_log_size":5953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:13.777477+0000","scrub_duration":0.043739180000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27125,"num_write_kb":550847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"43'15","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:34.907335+0000","last_change":"2024-09-13T18:57:25.418872+0000","last_active":"2024-09-13T18:57:34.907335+0000","last_peered":"2024-09-13T18:57:34.907335+0000","last_clean":"2024-09-13T18:57:34.907335+0000","last_became_active":"2024-09-13T18:26:32.350922+0000","last_became_peered":"2024-09-13T18:26:32.350922+0000","last_unstale":"2024-09-13T18:57:34.907335+0000","last_undegraded":"2024-09-13T18:57:34.907335+0000","last_fullsized":"2024-09-13T18:57:34.907335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:57:25.418501+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:57:25.418501+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.418501+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:32.901503+0000","scrub_duration":0.003478723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"43'20","reported_seq":111,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.419225+0000","last_change":"2024-09-13T18:57:13.300861+0000","last_active":"2024-09-13T18:57:26.419225+0000","last_peered":"2024-09-13T18:57:26.419225+0000","last_clean":"2024-09-13T18:57:26.419225+0000","last_became_active":"2024-09-13T18:26:31.960709+0000","last_became_peered":"2024-09-13T18:26:31.960709+0000","last_unstale":"2024-09-13T18:57:26.419225+0000","last_undegraded":"2024-09-13T18:57:26.419225+0000","last_fullsized":"2024-09-13T18:57:26.419225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:13.300701+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:13.300701+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.300701+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:09.337133+0000","scrub_duration":0.0071490659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":93,"num_read_kb":112,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"43'5303","reported_seq":5381,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:02.265627+0000","last_change":"2024-09-13T18:58:02.265627+0000","last_active":"2024-09-13T18:58:02.265627+0000","last_peered":"2024-09-13T18:58:02.265627+0000","last_clean":"2024-09-13T18:58:02.265627+0000","last_became_active":"2024-09-13T18:26:36.409651+0000","last_became_peered":"2024-09-13T18:26:36.409651+0000","last_unstale":"2024-09-13T18:58:02.265627+0000","last_undegraded":"2024-09-13T18:58:02.265627+0000","last_fullsized":"2024-09-13T18:58:02.265627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5303","last_scrub_stamp":"2024-09-13T18:58:02.265298+0000","last_deep_scrub":"43'5303","last_deep_scrub_stamp":"2024-09-13T18:58:02.265298+0000","last_clean_scrub_stamp":"2024-09-13T18:58:02.265298+0000","objects_scrubbed":0,"log_size":5303,"log_dups_size":0,"ondisk_log_size":5303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:41.041583+0000","scrub_duration":0.032593770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":23240,"num_write_kb":490059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"43'12","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:18.042305+0000","last_change":"2024-09-13T18:58:18.042531+0000","last_active":"2024-09-13T18:58:18.042305+0000","last_peered":"2024-09-13T18:58:18.042305+0000","last_clean":"2024-09-13T18:58:18.042305+0000","last_became_active":"2024-09-13T18:26:32.362626+0000","last_became_peered":"2024-09-13T18:26:32.362626+0000","last_unstale":"2024-09-13T18:58:18.042305+0000","last_undegraded":"2024-09-13T18:58:18.042305+0000","last_fullsized":"2024-09-13T18:58:18.042305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:58:18.042256+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:58:18.042256+0000","last_clean_scrub_stamp":"2024-09-13T18:58:18.042256+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:31.464208+0000","scrub_duration":0.003053058,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"43'389","reported_seq":488,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.908598+0000","last_change":"2024-09-13T18:57:13.774570+0000","last_active":"2024-09-13T18:57:24.908598+0000","last_peered":"2024-09-13T18:57:24.908598+0000","last_clean":"2024-09-13T18:57:24.908598+0000","last_became_active":"2024-09-13T18:26:31.973812+0000","last_became_peered":"2024-09-13T18:26:31.973812+0000","last_unstale":"2024-09-13T18:57:24.908598+0000","last_undegraded":"2024-09-13T18:57:24.908598+0000","last_fullsized":"2024-09-13T18:57:24.908598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'387","last_scrub_stamp":"2024-09-13T18:57:13.774429+0000","last_deep_scrub":"43'387","last_deep_scrub_stamp":"2024-09-13T18:57:13.774429+0000","last_clean_scrub_stamp":"2024-09-13T18:57:13.774429+0000","objects_scrubbed":5,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:38.043525+0000","scrub_duration":0.046187055999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":93,"num_read_kb":20568,"num_write":383,"num_write_kb":4318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"43'5475","reported_seq":5553,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:59.197285+0000","last_change":"2024-09-13T18:57:59.197483+0000","last_active":"2024-09-13T18:57:59.197285+0000","last_peered":"2024-09-13T18:57:59.197285+0000","last_clean":"2024-09-13T18:57:59.197285+0000","last_became_active":"2024-09-13T18:26:36.393663+0000","last_became_peered":"2024-09-13T18:26:36.393663+0000","last_unstale":"2024-09-13T18:57:59.197285+0000","last_undegraded":"2024-09-13T18:57:59.197285+0000","last_fullsized":"2024-09-13T18:57:59.197285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5475","last_scrub_stamp":"2024-09-13T18:57:59.197261+0000","last_deep_scrub":"43'5475","last_deep_scrub_stamp":"2024-09-13T18:57:59.197261+0000","last_clean_scrub_stamp":"2024-09-13T18:57:59.197261+0000","objects_scrubbed":0,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:37.225584+0000","scrub_duration":0.040786403999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":25034,"num_write_kb":485267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"43'10","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.055388+0000","last_change":"2024-09-13T18:57:16.962251+0000","last_active":"2024-09-13T18:57:28.055388+0000","last_peered":"2024-09-13T18:57:28.055388+0000","last_clean":"2024-09-13T18:57:28.055388+0000","last_became_active":"2024-09-13T18:26:32.362020+0000","last_became_peered":"2024-09-13T18:26:32.362020+0000","last_unstale":"2024-09-13T18:57:28.055388+0000","last_undegraded":"2024-09-13T18:57:28.055388+0000","last_fullsized":"2024-09-13T18:57:28.055388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:16.961847+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:16.961847+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.961847+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:54.684373+0000","scrub_duration":0.0040550839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"43'66","reported_seq":141,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.530024+0000","last_change":"2024-09-13T18:57:34.318022+0000","last_active":"2024-09-13T18:57:39.530024+0000","last_peered":"2024-09-13T18:57:39.530024+0000","last_clean":"2024-09-13T18:57:39.530024+0000","last_became_active":"2024-09-13T18:26:31.976108+0000","last_became_peered":"2024-09-13T18:26:31.976108+0000","last_unstale":"2024-09-13T18:57:39.530024+0000","last_undegraded":"2024-09-13T18:57:39.530024+0000","last_fullsized":"2024-09-13T18:57:39.530024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'65","last_scrub_stamp":"2024-09-13T18:57:34.317748+0000","last_deep_scrub":"43'65","last_deep_scrub_stamp":"2024-09-13T18:57:34.317748+0000","last_clean_scrub_stamp":"2024-09-13T18:57:34.317748+0000","objects_scrubbed":3,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:47.140265+0000","scrub_duration":0.0064888469999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":65,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"43'5962","reported_seq":6044,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:45.941460+0000","last_change":"2024-09-13T18:57:45.941610+0000","last_active":"2024-09-13T18:57:45.941460+0000","last_peered":"2024-09-13T18:57:45.941460+0000","last_clean":"2024-09-13T18:57:45.941460+0000","last_became_active":"2024-09-13T18:26:36.405869+0000","last_became_peered":"2024-09-13T18:26:36.405869+0000","last_unstale":"2024-09-13T18:57:45.941460+0000","last_undegraded":"2024-09-13T18:57:45.941460+0000","last_fullsized":"2024-09-13T18:57:45.941460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5962","last_scrub_stamp":"2024-09-13T18:57:45.941438+0000","last_deep_scrub":"43'5962","last_deep_scrub_stamp":"2024-09-13T18:57:45.941438+0000","last_clean_scrub_stamp":"2024-09-13T18:57:45.941438+0000","objects_scrubbed":0,"log_size":5962,"log_dups_size":0,"ondisk_log_size":5962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:34.601990+0000","scrub_duration":0.038953110999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":27473,"num_write_kb":527755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"43'16","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:32.414926+0000","last_change":"2024-09-13T18:57:32.414926+0000","last_active":"2024-09-13T18:57:32.414926+0000","last_peered":"2024-09-13T18:57:32.414926+0000","last_clean":"2024-09-13T18:57:32.414926+0000","last_became_active":"2024-09-13T18:26:32.350540+0000","last_became_peered":"2024-09-13T18:26:32.350540+0000","last_unstale":"2024-09-13T18:57:32.414926+0000","last_undegraded":"2024-09-13T18:57:32.414926+0000","last_fullsized":"2024-09-13T18:57:32.414926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:32.414785+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:32.414785+0000","last_clean_scrub_stamp":"2024-09-13T18:57:32.414785+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:58:20.923011+0000","scrub_duration":0.0019847889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"43'6","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.921362+0000","last_change":"2024-09-13T18:57:18.948854+0000","last_active":"2024-09-13T18:57:24.921362+0000","last_peered":"2024-09-13T18:57:24.921362+0000","last_clean":"2024-09-13T18:57:24.921362+0000","last_became_active":"2024-09-13T18:26:31.977910+0000","last_became_peered":"2024-09-13T18:26:31.977910+0000","last_unstale":"2024-09-13T18:57:24.921362+0000","last_undegraded":"2024-09-13T18:57:24.921362+0000","last_fullsized":"2024-09-13T18:57:24.921362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4","last_scrub_stamp":"2024-09-13T18:57:18.948753+0000","last_deep_scrub":"43'4","last_deep_scrub_stamp":"2024-09-13T18:57:18.948753+0000","last_clean_scrub_stamp":"2024-09-13T18:57:18.948753+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:24:26.277291+0000","scrub_duration":0.0026056769999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"43'5502","reported_seq":5575,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:58.974602+0000","last_change":"2024-09-13T18:57:58.974930+0000","last_active":"2024-09-13T18:57:58.974602+0000","last_peered":"2024-09-13T18:57:58.974602+0000","last_clean":"2024-09-13T18:57:58.974602+0000","last_became_active":"2024-09-13T18:26:36.405406+0000","last_became_peered":"2024-09-13T18:26:36.405406+0000","last_unstale":"2024-09-13T18:57:58.974602+0000","last_undegraded":"2024-09-13T18:57:58.974602+0000","last_fullsized":"2024-09-13T18:57:58.974602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5502","last_scrub_stamp":"2024-09-13T18:57:58.974532+0000","last_deep_scrub":"43'5502","last_deep_scrub_stamp":"2024-09-13T18:57:58.974532+0000","last_clean_scrub_stamp":"2024-09-13T18:57:58.974532+0000","objects_scrubbed":0,"log_size":5502,"log_dups_size":0,"ondisk_log_size":5502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:31:36.209865+0000","scrub_duration":0.045783550999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":25045,"num_write_kb":489215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"43'13","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:45.146635+0000","last_change":"2024-09-13T18:57:45.146635+0000","last_active":"2024-09-13T18:57:45.146635+0000","last_peered":"2024-09-13T18:57:45.146635+0000","last_clean":"2024-09-13T18:57:45.146635+0000","last_became_active":"2024-09-13T18:26:32.357348+0000","last_became_peered":"2024-09-13T18:26:32.357348+0000","last_unstale":"2024-09-13T18:57:45.146635+0000","last_undegraded":"2024-09-13T18:57:45.146635+0000","last_fullsized":"2024-09-13T18:57:45.146635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:45.146357+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:45.146357+0000","last_clean_scrub_stamp":"2024-09-13T18:57:45.146357+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:12.892350+0000","scrub_duration":0.0033819689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"43'19","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.922065+0000","last_change":"2024-09-13T18:57:22.765983+0000","last_active":"2024-09-13T18:57:24.922065+0000","last_peered":"2024-09-13T18:57:24.922065+0000","last_clean":"2024-09-13T18:57:24.922065+0000","last_became_active":"2024-09-13T18:26:31.979695+0000","last_became_peered":"2024-09-13T18:26:31.979695+0000","last_unstale":"2024-09-13T18:57:24.922065+0000","last_undegraded":"2024-09-13T18:57:24.922065+0000","last_fullsized":"2024-09-13T18:57:24.922065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'18","last_scrub_stamp":"2024-09-13T18:57:22.765727+0000","last_deep_scrub":"43'18","last_deep_scrub_stamp":"2024-09-13T18:57:22.765727+0000","last_clean_scrub_stamp":"2024-09-13T18:57:22.765727+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:04.965728+0000","scrub_duration":0.0065666600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":18,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"43'5179","reported_seq":5259,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:53.977550+0000","last_change":"2024-09-13T18:57:53.977664+0000","last_active":"2024-09-13T18:57:53.977550+0000","last_peered":"2024-09-13T18:57:53.977550+0000","last_clean":"2024-09-13T18:57:53.977550+0000","last_became_active":"2024-09-13T18:26:36.405364+0000","last_became_peered":"2024-09-13T18:26:36.405364+0000","last_unstale":"2024-09-13T18:57:53.977550+0000","last_undegraded":"2024-09-13T18:57:53.977550+0000","last_fullsized":"2024-09-13T18:57:53.977550+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5179","last_scrub_stamp":"2024-09-13T18:57:53.977529+0000","last_deep_scrub":"43'5179","last_deep_scrub_stamp":"2024-09-13T18:57:53.977529+0000","last_clean_scrub_stamp":"2024-09-13T18:57:53.977529+0000","objects_scrubbed":0,"log_size":5179,"log_dups_size":0,"ondisk_log_size":5179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:35:59.485208+0000","scrub_duration":0.041690597000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":23340,"num_write_kb":507027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"43'6189","reported_seq":6271,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:57.311076+0000","last_change":"2024-09-13T18:57:57.311297+0000","last_active":"2024-09-13T18:57:57.311076+0000","last_peered":"2024-09-13T18:57:57.311076+0000","last_clean":"2024-09-13T18:57:57.311076+0000","last_became_active":"2024-09-13T18:26:36.406661+0000","last_became_peered":"2024-09-13T18:26:36.406661+0000","last_unstale":"2024-09-13T18:57:57.311076+0000","last_undegraded":"2024-09-13T18:57:57.311076+0000","last_fullsized":"2024-09-13T18:57:57.311076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6189","last_scrub_stamp":"2024-09-13T18:57:57.311029+0000","last_deep_scrub":"43'6189","last_deep_scrub_stamp":"2024-09-13T18:57:57.311029+0000","last_clean_scrub_stamp":"2024-09-13T18:57:57.311029+0000","objects_scrubbed":0,"log_size":6189,"log_dups_size":0,"ondisk_log_size":6189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:14.650943+0000","scrub_duration":0.039998120999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":28563,"num_write_kb":581141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"43'10","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:52.167207+0000","last_change":"2024-09-13T18:57:52.167207+0000","last_active":"2024-09-13T18:57:52.167207+0000","last_peered":"2024-09-13T18:57:52.167207+0000","last_clean":"2024-09-13T18:57:52.167207+0000","last_became_active":"2024-09-13T18:26:32.348763+0000","last_became_peered":"2024-09-13T18:26:32.348763+0000","last_unstale":"2024-09-13T18:57:52.167207+0000","last_undegraded":"2024-09-13T18:57:52.167207+0000","last_fullsized":"2024-09-13T18:57:52.167207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:52.166714+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:52.166714+0000","last_clean_scrub_stamp":"2024-09-13T18:57:52.166714+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:23.336388+0000","scrub_duration":0.0037465770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"43'8","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:21.964628+0000","last_change":"2024-09-13T18:57:21.964628+0000","last_active":"2024-09-13T18:57:21.964628+0000","last_peered":"2024-09-13T18:57:21.964628+0000","last_clean":"2024-09-13T18:57:21.964628+0000","last_became_active":"2024-09-13T18:26:31.968569+0000","last_became_peered":"2024-09-13T18:26:31.968569+0000","last_unstale":"2024-09-13T18:57:21.964628+0000","last_undegraded":"2024-09-13T18:57:21.964628+0000","last_fullsized":"2024-09-13T18:57:21.964628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:21.964364+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:21.964364+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.964364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:31:05.131047+0000","scrub_duration":0.003227541,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"43'5646","reported_seq":5731,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:06.699074+0000","last_change":"2024-09-13T18:58:06.699213+0000","last_active":"2024-09-13T18:58:06.699074+0000","last_peered":"2024-09-13T18:58:06.699074+0000","last_clean":"2024-09-13T18:58:06.699074+0000","last_became_active":"2024-09-13T18:26:36.396039+0000","last_became_peered":"2024-09-13T18:26:36.396039+0000","last_unstale":"2024-09-13T18:58:06.699074+0000","last_undegraded":"2024-09-13T18:58:06.699074+0000","last_fullsized":"2024-09-13T18:58:06.699074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5646","last_scrub_stamp":"2024-09-13T18:58:06.699052+0000","last_deep_scrub":"43'5646","last_deep_scrub_stamp":"2024-09-13T18:58:06.699052+0000","last_clean_scrub_stamp":"2024-09-13T18:58:06.699052+0000","objects_scrubbed":0,"log_size":5646,"log_dups_size":0,"ondisk_log_size":5646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:01.384154+0000","scrub_duration":0.038574091999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":25607,"num_write_kb":520259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"43'19","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:42.857256+0000","last_change":"2024-09-13T18:57:42.857256+0000","last_active":"2024-09-13T18:57:42.857256+0000","last_peered":"2024-09-13T18:57:42.857256+0000","last_clean":"2024-09-13T18:57:42.857256+0000","last_became_active":"2024-09-13T18:26:32.350007+0000","last_became_peered":"2024-09-13T18:26:32.350007+0000","last_unstale":"2024-09-13T18:57:42.857256+0000","last_undegraded":"2024-09-13T18:57:42.857256+0000","last_fullsized":"2024-09-13T18:57:42.857256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'19","last_scrub_stamp":"2024-09-13T18:57:42.857115+0000","last_deep_scrub":"43'19","last_deep_scrub_stamp":"2024-09-13T18:57:42.857115+0000","last_clean_scrub_stamp":"2024-09-13T18:57:42.857115+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:32.087665+0000","scrub_duration":0.0022791909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"43'6","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.792890+0000","last_change":"2024-09-13T18:57:21.960612+0000","last_active":"2024-09-13T18:57:24.792890+0000","last_peered":"2024-09-13T18:57:24.792890+0000","last_clean":"2024-09-13T18:57:24.792890+0000","last_became_active":"2024-09-13T18:26:31.971696+0000","last_became_peered":"2024-09-13T18:26:31.971696+0000","last_unstale":"2024-09-13T18:57:24.792890+0000","last_undegraded":"2024-09-13T18:57:24.792890+0000","last_fullsized":"2024-09-13T18:57:24.792890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'5","last_scrub_stamp":"2024-09-13T18:57:21.960203+0000","last_deep_scrub":"43'5","last_deep_scrub_stamp":"2024-09-13T18:57:21.960203+0000","last_clean_scrub_stamp":"2024-09-13T18:57:21.960203+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:59:06.847239+0000","scrub_duration":0.006849478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"43'13","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:27.647844+0000","last_change":"2024-09-13T18:57:16.794490+0000","last_active":"2024-09-13T18:57:27.647844+0000","last_peered":"2024-09-13T18:57:27.647844+0000","last_clean":"2024-09-13T18:57:27.647844+0000","last_became_active":"2024-09-13T18:26:31.974940+0000","last_became_peered":"2024-09-13T18:26:31.974940+0000","last_unstale":"2024-09-13T18:57:27.647844+0000","last_undegraded":"2024-09-13T18:57:27.647844+0000","last_fullsized":"2024-09-13T18:57:27.647844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:16.794241+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:16.794241+0000","last_clean_scrub_stamp":"2024-09-13T18:57:16.794241+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:35.699392+0000","scrub_duration":0.0056238700000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"43'10","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:42.246148+0000","last_change":"2024-09-13T18:57:42.246326+0000","last_active":"2024-09-13T18:57:42.246148+0000","last_peered":"2024-09-13T18:57:42.246148+0000","last_clean":"2024-09-13T18:57:42.246148+0000","last_became_active":"2024-09-13T18:26:32.345581+0000","last_became_peered":"2024-09-13T18:26:32.345581+0000","last_unstale":"2024-09-13T18:57:42.246148+0000","last_undegraded":"2024-09-13T18:57:42.246148+0000","last_fullsized":"2024-09-13T18:57:42.246148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:42.246101+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:42.246101+0000","last_clean_scrub_stamp":"2024-09-13T18:57:42.246101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:54.930475+0000","scrub_duration":0.0033923690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"43'4199","reported_seq":4275,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.531206+0000","last_change":"2024-09-13T18:57:32.974532+0000","last_active":"2024-09-13T18:57:38.531206+0000","last_peered":"2024-09-13T18:57:38.531206+0000","last_clean":"2024-09-13T18:57:38.531206+0000","last_became_active":"2024-09-13T18:26:36.391152+0000","last_became_peered":"2024-09-13T18:26:36.391152+0000","last_unstale":"2024-09-13T18:57:38.531206+0000","last_undegraded":"2024-09-13T18:57:38.531206+0000","last_fullsized":"2024-09-13T18:57:38.531206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'4177","last_scrub_stamp":"2024-09-13T18:57:32.974351+0000","last_deep_scrub":"43'4177","last_deep_scrub_stamp":"2024-09-13T18:57:32.974351+0000","last_clean_scrub_stamp":"2024-09-13T18:57:32.974351+0000","objects_scrubbed":11,"log_size":4199,"log_dups_size":0,"ondisk_log_size":4199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:34.816537+0000","scrub_duration":0.80893831299999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":19219,"num_write_kb":424944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"43'759","reported_seq":843,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.910037+0000","last_change":"2024-09-13T18:57:28.910129+0000","last_active":"2024-09-13T18:57:28.910037+0000","last_peered":"2024-09-13T18:57:28.910037+0000","last_clean":"2024-09-13T18:57:28.910037+0000","last_became_active":"2024-09-13T18:26:31.976340+0000","last_became_peered":"2024-09-13T18:26:31.976340+0000","last_unstale":"2024-09-13T18:57:28.910037+0000","last_undegraded":"2024-09-13T18:57:28.910037+0000","last_fullsized":"2024-09-13T18:57:28.910037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'759","last_scrub_stamp":"2024-09-13T18:57:28.910019+0000","last_deep_scrub":"43'759","last_deep_scrub_stamp":"2024-09-13T18:57:28.910019+0000","last_clean_scrub_stamp":"2024-09-13T18:57:28.910019+0000","objects_scrubbed":3,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:04.313816+0000","scrub_duration":0.042299119000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196273,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":19627,"num_write":735,"num_write_kb":4345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"43'13","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.967148+0000","last_change":"2024-09-13T18:57:20.986740+0000","last_active":"2024-09-13T18:57:24.967148+0000","last_peered":"2024-09-13T18:57:24.967148+0000","last_clean":"2024-09-13T18:57:24.967148+0000","last_became_active":"2024-09-13T18:26:32.352707+0000","last_became_peered":"2024-09-13T18:26:32.352707+0000","last_unstale":"2024-09-13T18:57:24.967148+0000","last_undegraded":"2024-09-13T18:57:24.967148+0000","last_fullsized":"2024-09-13T18:57:24.967148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:20.986473+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:20.986473+0000","last_clean_scrub_stamp":"2024-09-13T18:57:20.986473+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:56.996793+0000","scrub_duration":0.0033945970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"43'6014","reported_seq":6094,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:54.990721+0000","last_change":"2024-09-13T18:57:54.990862+0000","last_active":"2024-09-13T18:57:54.990721+0000","last_peered":"2024-09-13T18:57:54.990721+0000","last_clean":"2024-09-13T18:57:54.990721+0000","last_became_active":"2024-09-13T18:26:36.403385+0000","last_became_peered":"2024-09-13T18:26:36.403385+0000","last_unstale":"2024-09-13T18:57:54.990721+0000","last_undegraded":"2024-09-13T18:57:54.990721+0000","last_fullsized":"2024-09-13T18:57:54.990721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6014","last_scrub_stamp":"2024-09-13T18:57:54.990699+0000","last_deep_scrub":"43'6014","last_deep_scrub_stamp":"2024-09-13T18:57:54.990699+0000","last_clean_scrub_stamp":"2024-09-13T18:57:54.990699+0000","objects_scrubbed":0,"log_size":6014,"log_dups_size":0,"ondisk_log_size":6014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:42:04.338574+0000","scrub_duration":0.044555913000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":26799,"num_write_kb":557751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"43'16","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:54.488901+0000","last_change":"2024-09-13T18:57:54.489110+0000","last_active":"2024-09-13T18:57:54.488901+0000","last_peered":"2024-09-13T18:57:54.488901+0000","last_clean":"2024-09-13T18:57:54.488901+0000","last_became_active":"2024-09-13T18:26:32.357581+0000","last_became_peered":"2024-09-13T18:26:32.357581+0000","last_unstale":"2024-09-13T18:57:54.488901+0000","last_undegraded":"2024-09-13T18:57:54.488901+0000","last_fullsized":"2024-09-13T18:57:54.488901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:54.488848+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:54.488848+0000","last_clean_scrub_stamp":"2024-09-13T18:57:54.488848+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:43:01.499699+0000","scrub_duration":0.0035375490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"43'12","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:31.097096+0000","last_change":"2024-09-13T18:57:31.097096+0000","last_active":"2024-09-13T18:57:31.097096+0000","last_peered":"2024-09-13T18:57:31.097096+0000","last_clean":"2024-09-13T18:57:31.097096+0000","last_became_active":"2024-09-13T18:26:32.350724+0000","last_became_peered":"2024-09-13T18:26:32.350724+0000","last_unstale":"2024-09-13T18:57:31.097096+0000","last_undegraded":"2024-09-13T18:57:31.097096+0000","last_fullsized":"2024-09-13T18:57:31.097096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:57:31.096973+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:57:31.096973+0000","last_clean_scrub_stamp":"2024-09-13T18:57:31.096973+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:08.952276+0000","scrub_duration":0.0027320500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"43'17","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.122971+0000","last_change":"2024-09-13T18:57:39.122971+0000","last_active":"2024-09-13T18:57:39.122971+0000","last_peered":"2024-09-13T18:57:39.122971+0000","last_clean":"2024-09-13T18:57:39.122971+0000","last_became_active":"2024-09-13T18:26:32.355490+0000","last_became_peered":"2024-09-13T18:26:32.355490+0000","last_unstale":"2024-09-13T18:57:39.122971+0000","last_undegraded":"2024-09-13T18:57:39.122971+0000","last_fullsized":"2024-09-13T18:57:39.122971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'17","last_scrub_stamp":"2024-09-13T18:57:39.122544+0000","last_deep_scrub":"43'17","last_deep_scrub_stamp":"2024-09-13T18:57:39.122544+0000","last_clean_scrub_stamp":"2024-09-13T18:57:39.122544+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:32.935831+0000","scrub_duration":0.0032146100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"43'10","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:45.307991+0000","last_change":"2024-09-13T18:57:45.307991+0000","last_active":"2024-09-13T18:57:45.307991+0000","last_peered":"2024-09-13T18:57:45.307991+0000","last_clean":"2024-09-13T18:57:45.307991+0000","last_became_active":"2024-09-13T18:26:32.340624+0000","last_became_peered":"2024-09-13T18:26:32.340624+0000","last_unstale":"2024-09-13T18:57:45.307991+0000","last_undegraded":"2024-09-13T18:57:45.307991+0000","last_fullsized":"2024-09-13T18:57:45.307991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:45.307762+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:45.307762+0000","last_clean_scrub_stamp":"2024-09-13T18:57:45.307762+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:12.874226+0000","scrub_duration":0.0028431099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"43'10","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:26.421715+0000","last_change":"2024-09-13T18:57:19.006871+0000","last_active":"2024-09-13T18:57:26.421715+0000","last_peered":"2024-09-13T18:57:26.421715+0000","last_clean":"2024-09-13T18:57:26.421715+0000","last_became_active":"2024-09-13T18:26:32.349381+0000","last_became_peered":"2024-09-13T18:26:32.349381+0000","last_unstale":"2024-09-13T18:57:26.421715+0000","last_undegraded":"2024-09-13T18:57:26.421715+0000","last_fullsized":"2024-09-13T18:57:26.421715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'7","last_scrub_stamp":"2024-09-13T18:57:19.006675+0000","last_deep_scrub":"29'7","last_deep_scrub_stamp":"2024-09-13T18:57:19.006675+0000","last_clean_scrub_stamp":"2024-09-13T18:57:19.006675+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:53.159267+0000","scrub_duration":0.012523802000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"43'10","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:43.268378+0000","last_change":"2024-09-13T18:57:43.268610+0000","last_active":"2024-09-13T18:57:43.268378+0000","last_peered":"2024-09-13T18:57:43.268378+0000","last_clean":"2024-09-13T18:57:43.268378+0000","last_became_active":"2024-09-13T18:26:32.348790+0000","last_became_peered":"2024-09-13T18:26:32.348790+0000","last_unstale":"2024-09-13T18:57:43.268378+0000","last_undegraded":"2024-09-13T18:57:43.268378+0000","last_fullsized":"2024-09-13T18:57:43.268378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:43.268329+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:43.268329+0000","last_clean_scrub_stamp":"2024-09-13T18:57:43.268329+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:56.169188+0000","scrub_duration":0.0028286909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"43'8","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.922074+0000","last_change":"2024-09-13T18:57:24.709839+0000","last_active":"2024-09-13T18:57:24.922074+0000","last_peered":"2024-09-13T18:57:24.922074+0000","last_clean":"2024-09-13T18:57:24.922074+0000","last_became_active":"2024-09-13T18:26:32.364495+0000","last_became_peered":"2024-09-13T18:26:32.364495+0000","last_unstale":"2024-09-13T18:57:24.922074+0000","last_undegraded":"2024-09-13T18:57:24.922074+0000","last_fullsized":"2024-09-13T18:57:24.922074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:24.709588+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:24.709588+0000","last_clean_scrub_stamp":"2024-09-13T18:57:24.709588+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:56.958314+0000","scrub_duration":0.0036738090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"43'13","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.862606+0000","last_change":"2024-09-13T18:57:17.997698+0000","last_active":"2024-09-13T18:57:24.862606+0000","last_peered":"2024-09-13T18:57:24.862606+0000","last_clean":"2024-09-13T18:57:24.862606+0000","last_became_active":"2024-09-13T18:26:32.347340+0000","last_became_peered":"2024-09-13T18:26:32.347340+0000","last_unstale":"2024-09-13T18:57:24.862606+0000","last_undegraded":"2024-09-13T18:57:24.862606+0000","last_fullsized":"2024-09-13T18:57:24.862606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:17.997340+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:17.997340+0000","last_clean_scrub_stamp":"2024-09-13T18:57:17.997340+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:07.106051+0000","scrub_duration":0.0035120389999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"43'10","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:36.044747+0000","last_change":"2024-09-13T18:57:36.044747+0000","last_active":"2024-09-13T18:57:36.044747+0000","last_peered":"2024-09-13T18:57:36.044747+0000","last_clean":"2024-09-13T18:57:36.044747+0000","last_became_active":"2024-09-13T18:26:32.353122+0000","last_became_peered":"2024-09-13T18:26:32.353122+0000","last_unstale":"2024-09-13T18:57:36.044747+0000","last_undegraded":"2024-09-13T18:57:36.044747+0000","last_fullsized":"2024-09-13T18:57:36.044747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:57:36.044597+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:57:36.044597+0000","last_clean_scrub_stamp":"2024-09-13T18:57:36.044597+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:40.677133+0000","scrub_duration":0.010560052,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"43'15","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:38.892530+0000","last_change":"2024-09-13T18:57:38.892530+0000","last_active":"2024-09-13T18:57:38.892530+0000","last_peered":"2024-09-13T18:57:38.892530+0000","last_clean":"2024-09-13T18:57:38.892530+0000","last_became_active":"2024-09-13T18:26:32.349310+0000","last_became_peered":"2024-09-13T18:26:32.349310+0000","last_unstale":"2024-09-13T18:57:38.892530+0000","last_undegraded":"2024-09-13T18:57:38.892530+0000","last_fullsized":"2024-09-13T18:57:38.892530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'15","last_scrub_stamp":"2024-09-13T18:57:38.892242+0000","last_deep_scrub":"43'15","last_deep_scrub_stamp":"2024-09-13T18:57:38.892242+0000","last_clean_scrub_stamp":"2024-09-13T18:57:38.892242+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:17.674671+0000","scrub_duration":0.0037659220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"43'11","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:58.186979+0000","last_change":"2024-09-13T18:57:58.186979+0000","last_active":"2024-09-13T18:57:58.186979+0000","last_peered":"2024-09-13T18:57:58.186979+0000","last_clean":"2024-09-13T18:57:58.186979+0000","last_became_active":"2024-09-13T18:26:32.358330+0000","last_became_peered":"2024-09-13T18:26:32.358330+0000","last_unstale":"2024-09-13T18:57:58.186979+0000","last_undegraded":"2024-09-13T18:57:58.186979+0000","last_fullsized":"2024-09-13T18:57:58.186979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:58.186729+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:58.186729+0000","last_clean_scrub_stamp":"2024-09-13T18:57:58.186729+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:41.470005+0000","scrub_duration":0.0030497200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"43'18","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:47.275068+0000","last_change":"2024-09-13T18:57:47.275068+0000","last_active":"2024-09-13T18:57:47.275068+0000","last_peered":"2024-09-13T18:57:47.275068+0000","last_clean":"2024-09-13T18:57:47.275068+0000","last_became_active":"2024-09-13T18:26:32.353442+0000","last_became_peered":"2024-09-13T18:26:32.353442+0000","last_unstale":"2024-09-13T18:57:47.275068+0000","last_undegraded":"2024-09-13T18:57:47.275068+0000","last_fullsized":"2024-09-13T18:57:47.275068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'18","last_scrub_stamp":"2024-09-13T18:57:47.274823+0000","last_deep_scrub":"43'18","last_deep_scrub_stamp":"2024-09-13T18:57:47.274823+0000","last_clean_scrub_stamp":"2024-09-13T18:57:47.274823+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:23:23.332126+0000","scrub_duration":0.0026113270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"43'11","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:51.484615+0000","last_change":"2024-09-13T18:57:51.484615+0000","last_active":"2024-09-13T18:57:51.484615+0000","last_peered":"2024-09-13T18:57:51.484615+0000","last_clean":"2024-09-13T18:57:51.484615+0000","last_became_active":"2024-09-13T18:26:32.352772+0000","last_became_peered":"2024-09-13T18:26:32.352772+0000","last_unstale":"2024-09-13T18:57:51.484615+0000","last_undegraded":"2024-09-13T18:57:51.484615+0000","last_fullsized":"2024-09-13T18:57:51.484615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:51.484333+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:51.484333+0000","last_clean_scrub_stamp":"2024-09-13T18:57:51.484333+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:35.675512+0000","scrub_duration":0.0032285579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"43'15","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:32.560841+0000","last_change":"2024-09-13T18:57:15.857341+0000","last_active":"2024-09-13T18:57:32.560841+0000","last_peered":"2024-09-13T18:57:32.560841+0000","last_clean":"2024-09-13T18:57:32.560841+0000","last_became_active":"2024-09-13T18:26:32.357116+0000","last_became_peered":"2024-09-13T18:26:32.357116+0000","last_unstale":"2024-09-13T18:57:32.560841+0000","last_undegraded":"2024-09-13T18:57:32.560841+0000","last_fullsized":"2024-09-13T18:57:32.560841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:57:15.857084+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:57:15.857084+0000","last_clean_scrub_stamp":"2024-09-13T18:57:15.857084+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:39.110251+0000","scrub_duration":0.0034250190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"43'18","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:28.034005+0000","last_change":"2024-09-13T18:57:27.062756+0000","last_active":"2024-09-13T18:57:28.034005+0000","last_peered":"2024-09-13T18:57:28.034005+0000","last_clean":"2024-09-13T18:57:28.034005+0000","last_became_active":"2024-09-13T18:26:32.348591+0000","last_became_peered":"2024-09-13T18:26:32.348591+0000","last_unstale":"2024-09-13T18:57:28.034005+0000","last_undegraded":"2024-09-13T18:57:28.034005+0000","last_fullsized":"2024-09-13T18:57:28.034005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'15","last_scrub_stamp":"2024-09-13T18:57:27.062495+0000","last_deep_scrub":"43'15","last_deep_scrub_stamp":"2024-09-13T18:57:27.062495+0000","last_clean_scrub_stamp":"2024-09-13T18:57:27.062495+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:05:17.012914+0000","scrub_duration":0.0040785700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"43'14","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:48.240470+0000","last_change":"2024-09-13T18:57:48.240666+0000","last_active":"2024-09-13T18:57:48.240470+0000","last_peered":"2024-09-13T18:57:48.240470+0000","last_clean":"2024-09-13T18:57:48.240470+0000","last_became_active":"2024-09-13T18:26:32.356634+0000","last_became_peered":"2024-09-13T18:26:32.356634+0000","last_unstale":"2024-09-13T18:57:48.240470+0000","last_undegraded":"2024-09-13T18:57:48.240470+0000","last_fullsized":"2024-09-13T18:57:48.240470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'14","last_scrub_stamp":"2024-09-13T18:57:48.240421+0000","last_deep_scrub":"43'14","last_deep_scrub_stamp":"2024-09-13T18:57:48.240421+0000","last_clean_scrub_stamp":"2024-09-13T18:57:48.240421+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:11.977729+0000","scrub_duration":0.0030695620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"43'12","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:43.177441+0000","last_change":"2024-09-13T18:57:43.177686+0000","last_active":"2024-09-13T18:57:43.177441+0000","last_peered":"2024-09-13T18:57:43.177441+0000","last_clean":"2024-09-13T18:57:43.177441+0000","last_became_active":"2024-09-13T18:26:32.346702+0000","last_became_peered":"2024-09-13T18:26:32.346702+0000","last_unstale":"2024-09-13T18:57:43.177441+0000","last_undegraded":"2024-09-13T18:57:43.177441+0000","last_fullsized":"2024-09-13T18:57:43.177441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'12","last_scrub_stamp":"2024-09-13T18:57:43.177387+0000","last_deep_scrub":"43'12","last_deep_scrub_stamp":"2024-09-13T18:57:43.177387+0000","last_clean_scrub_stamp":"2024-09-13T18:57:43.177387+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:30.629006+0000","scrub_duration":0.0029135400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"43'16","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:37.308644+0000","last_change":"2024-09-13T18:57:37.308764+0000","last_active":"2024-09-13T18:57:37.308644+0000","last_peered":"2024-09-13T18:57:37.308644+0000","last_clean":"2024-09-13T18:57:37.308644+0000","last_became_active":"2024-09-13T18:26:32.349366+0000","last_became_peered":"2024-09-13T18:26:32.349366+0000","last_unstale":"2024-09-13T18:57:37.308644+0000","last_undegraded":"2024-09-13T18:57:37.308644+0000","last_fullsized":"2024-09-13T18:57:37.308644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:57:37.308610+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:57:37.308610+0000","last_clean_scrub_stamp":"2024-09-13T18:57:37.308610+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:36:24.954579+0000","scrub_duration":0.020444231,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"43'17","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:34.387125+0000","last_change":"2024-09-13T18:57:34.387338+0000","last_active":"2024-09-13T18:57:34.387125+0000","last_peered":"2024-09-13T18:57:34.387125+0000","last_clean":"2024-09-13T18:57:34.387125+0000","last_became_active":"2024-09-13T18:26:32.344366+0000","last_became_peered":"2024-09-13T18:26:32.344366+0000","last_unstale":"2024-09-13T18:57:34.387125+0000","last_undegraded":"2024-09-13T18:57:34.387125+0000","last_fullsized":"2024-09-13T18:57:34.387125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'17","last_scrub_stamp":"2024-09-13T18:57:34.387073+0000","last_deep_scrub":"43'17","last_deep_scrub_stamp":"2024-09-13T18:57:34.387073+0000","last_clean_scrub_stamp":"2024-09-13T18:57:34.387073+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:16.405083+0000","scrub_duration":0.002417584,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"43'13","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:44.875900+0000","last_change":"2024-09-13T18:57:44.875900+0000","last_active":"2024-09-13T18:57:44.875900+0000","last_peered":"2024-09-13T18:57:44.875900+0000","last_clean":"2024-09-13T18:57:44.875900+0000","last_became_active":"2024-09-13T18:26:32.347467+0000","last_became_peered":"2024-09-13T18:26:32.347467+0000","last_unstale":"2024-09-13T18:57:44.875900+0000","last_undegraded":"2024-09-13T18:57:44.875900+0000","last_fullsized":"2024-09-13T18:57:44.875900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:44.875664+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:44.875664+0000","last_clean_scrub_stamp":"2024-09-13T18:57:44.875664+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:19.462107+0000","scrub_duration":0.0036327349999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"43'12","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:36.959082+0000","last_change":"2024-09-13T18:57:25.015223+0000","last_active":"2024-09-13T18:57:36.959082+0000","last_peered":"2024-09-13T18:57:36.959082+0000","last_clean":"2024-09-13T18:57:36.959082+0000","last_became_active":"2024-09-13T18:26:32.353276+0000","last_became_peered":"2024-09-13T18:26:32.353276+0000","last_unstale":"2024-09-13T18:57:36.959082+0000","last_undegraded":"2024-09-13T18:57:36.959082+0000","last_fullsized":"2024-09-13T18:57:36.959082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'10","last_scrub_stamp":"2024-09-13T18:57:25.015045+0000","last_deep_scrub":"43'10","last_deep_scrub_stamp":"2024-09-13T18:57:25.015045+0000","last_clean_scrub_stamp":"2024-09-13T18:57:25.015045+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:39.559328+0000","scrub_duration":0.013113774999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"43'13","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:30.778903+0000","last_change":"2024-09-13T18:57:30.778903+0000","last_active":"2024-09-13T18:57:30.778903+0000","last_peered":"2024-09-13T18:57:30.778903+0000","last_clean":"2024-09-13T18:57:30.778903+0000","last_became_active":"2024-09-13T18:26:32.354286+0000","last_became_peered":"2024-09-13T18:26:32.354286+0000","last_unstale":"2024-09-13T18:57:30.778903+0000","last_undegraded":"2024-09-13T18:57:30.778903+0000","last_fullsized":"2024-09-13T18:57:30.778903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:30.778791+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:30.778791+0000","last_clean_scrub_stamp":"2024-09-13T18:57:30.778791+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:34.242313+0000","scrub_duration":0.00175041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"43'13","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:55.513934+0000","last_change":"2024-09-13T18:57:55.513934+0000","last_active":"2024-09-13T18:57:55.513934+0000","last_peered":"2024-09-13T18:57:55.513934+0000","last_clean":"2024-09-13T18:57:55.513934+0000","last_became_active":"2024-09-13T18:26:32.357403+0000","last_became_peered":"2024-09-13T18:26:32.357403+0000","last_unstale":"2024-09-13T18:57:55.513934+0000","last_undegraded":"2024-09-13T18:57:55.513934+0000","last_fullsized":"2024-09-13T18:57:55.513934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:57:55.513649+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:57:55.513649+0000","last_clean_scrub_stamp":"2024-09-13T18:57:55.513649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:45.763242+0000","scrub_duration":0.0032588529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"43'16","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:01.133960+0000","last_change":"2024-09-13T18:58:01.133960+0000","last_active":"2024-09-13T18:58:01.133960+0000","last_peered":"2024-09-13T18:58:01.133960+0000","last_clean":"2024-09-13T18:58:01.133960+0000","last_became_active":"2024-09-13T18:26:32.365232+0000","last_became_peered":"2024-09-13T18:26:32.365232+0000","last_unstale":"2024-09-13T18:58:01.133960+0000","last_undegraded":"2024-09-13T18:58:01.133960+0000","last_fullsized":"2024-09-13T18:58:01.133960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'16","last_scrub_stamp":"2024-09-13T18:58:01.133537+0000","last_deep_scrub":"43'16","last_deep_scrub_stamp":"2024-09-13T18:58:01.133537+0000","last_clean_scrub_stamp":"2024-09-13T18:58:01.133537+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:06.020299+0000","scrub_duration":0.003245464,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"43'6","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:52.963821+0000","last_change":"2024-09-13T18:57:52.964032+0000","last_active":"2024-09-13T18:57:52.963821+0000","last_peered":"2024-09-13T18:57:52.963821+0000","last_clean":"2024-09-13T18:57:52.963821+0000","last_became_active":"2024-09-13T18:26:32.345243+0000","last_became_peered":"2024-09-13T18:26:32.345243+0000","last_unstale":"2024-09-13T18:57:52.963821+0000","last_undegraded":"2024-09-13T18:57:52.963821+0000","last_fullsized":"2024-09-13T18:57:52.963821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:52.963770+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:52.963770+0000","last_clean_scrub_stamp":"2024-09-13T18:57:52.963770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:59.872058+0000","scrub_duration":0.0035209389999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"43'9","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:07.047890+0000","last_change":"2024-09-13T18:58:07.048089+0000","last_active":"2024-09-13T18:58:07.047890+0000","last_peered":"2024-09-13T18:58:07.047890+0000","last_clean":"2024-09-13T18:58:07.047890+0000","last_became_active":"2024-09-13T18:26:32.354474+0000","last_became_peered":"2024-09-13T18:26:32.354474+0000","last_unstale":"2024-09-13T18:58:07.047890+0000","last_undegraded":"2024-09-13T18:58:07.047890+0000","last_fullsized":"2024-09-13T18:58:07.047890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:58:07.047843+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:58:07.047843+0000","last_clean_scrub_stamp":"2024-09-13T18:58:07.047843+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:00:30.056086+0000","scrub_duration":0.003715037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"43'11","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:01.217080+0000","last_change":"2024-09-13T18:58:01.217349+0000","last_active":"2024-09-13T18:58:01.217080+0000","last_peered":"2024-09-13T18:58:01.217080+0000","last_clean":"2024-09-13T18:58:01.217080+0000","last_became_active":"2024-09-13T18:26:32.354821+0000","last_became_peered":"2024-09-13T18:26:32.354821+0000","last_unstale":"2024-09-13T18:58:01.217080+0000","last_undegraded":"2024-09-13T18:58:01.217080+0000","last_fullsized":"2024-09-13T18:58:01.217080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'11","last_scrub_stamp":"2024-09-13T18:58:01.217028+0000","last_deep_scrub":"43'11","last_deep_scrub_stamp":"2024-09-13T18:58:01.217028+0000","last_clean_scrub_stamp":"2024-09-13T18:58:01.217028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:13:31.856600+0000","scrub_duration":0.0029497920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"43'13","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:05.140674+0000","last_change":"2024-09-13T18:58:05.141012+0000","last_active":"2024-09-13T18:58:05.140674+0000","last_peered":"2024-09-13T18:58:05.140674+0000","last_clean":"2024-09-13T18:58:05.140674+0000","last_became_active":"2024-09-13T18:26:32.357668+0000","last_became_peered":"2024-09-13T18:26:32.357668+0000","last_unstale":"2024-09-13T18:58:05.140674+0000","last_undegraded":"2024-09-13T18:58:05.140674+0000","last_fullsized":"2024-09-13T18:58:05.140674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'13","last_scrub_stamp":"2024-09-13T18:58:05.140565+0000","last_deep_scrub":"43'13","last_deep_scrub_stamp":"2024-09-13T18:58:05.140565+0000","last_clean_scrub_stamp":"2024-09-13T18:58:05.140565+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:10.286462+0000","scrub_duration":0.0034325750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"43'7","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:56.216315+0000","last_change":"2024-09-13T18:57:56.216516+0000","last_active":"2024-09-13T18:57:56.216315+0000","last_peered":"2024-09-13T18:57:56.216315+0000","last_clean":"2024-09-13T18:57:56.216315+0000","last_became_active":"2024-09-13T18:26:32.353352+0000","last_became_peered":"2024-09-13T18:26:32.353352+0000","last_unstale":"2024-09-13T18:57:56.216315+0000","last_undegraded":"2024-09-13T18:57:56.216315+0000","last_fullsized":"2024-09-13T18:57:56.216315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'7","last_scrub_stamp":"2024-09-13T18:57:56.216265+0000","last_deep_scrub":"43'7","last_deep_scrub_stamp":"2024-09-13T18:57:56.216265+0000","last_clean_scrub_stamp":"2024-09-13T18:57:56.216265+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:29.591142+0000","scrub_duration":0.0032929220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"43'11","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:24.862267+0000","last_change":"2024-09-13T18:57:11.323912+0000","last_active":"2024-09-13T18:57:24.862267+0000","last_peered":"2024-09-13T18:57:24.862267+0000","last_clean":"2024-09-13T18:57:24.862267+0000","last_became_active":"2024-09-13T18:26:32.341227+0000","last_became_peered":"2024-09-13T18:26:32.341227+0000","last_unstale":"2024-09-13T18:57:24.862267+0000","last_undegraded":"2024-09-13T18:57:24.862267+0000","last_fullsized":"2024-09-13T18:57:24.862267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'8","last_scrub_stamp":"2024-09-13T18:57:11.323767+0000","last_deep_scrub":"43'8","last_deep_scrub_stamp":"2024-09-13T18:57:11.323767+0000","last_clean_scrub_stamp":"2024-09-13T18:57:11.323767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:26.980778+0000","scrub_duration":0.0019515800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"43'6","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:57:39.252568+0000","last_change":"2024-09-13T18:57:39.252568+0000","last_active":"2024-09-13T18:57:39.252568+0000","last_peered":"2024-09-13T18:57:39.252568+0000","last_clean":"2024-09-13T18:57:39.252568+0000","last_became_active":"2024-09-13T18:26:32.349861+0000","last_became_peered":"2024-09-13T18:26:32.349861+0000","last_unstale":"2024-09-13T18:57:39.252568+0000","last_undegraded":"2024-09-13T18:57:39.252568+0000","last_fullsized":"2024-09-13T18:57:39.252568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'6","last_scrub_stamp":"2024-09-13T18:57:39.252320+0000","last_deep_scrub":"43'6","last_deep_scrub_stamp":"2024-09-13T18:57:39.252320+0000","last_clean_scrub_stamp":"2024-09-13T18:57:39.252320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:39.213931+0000","scrub_duration":0.002145554,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"43'9","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-13T18:58:06.094405+0000","last_change":"2024-09-13T18:58:06.094607+0000","last_active":"2024-09-13T18:58:06.094405+0000","last_peered":"2024-09-13T18:58:06.094405+0000","last_clean":"2024-09-13T18:58:06.094405+0000","last_became_active":"2024-09-13T18:26:32.361914+0000","last_became_peered":"2024-09-13T18:26:32.361914+0000","last_unstale":"2024-09-13T18:58:06.094405+0000","last_undegraded":"2024-09-13T18:58:06.094405+0000","last_fullsized":"2024-09-13T18:58:06.094405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'9","last_scrub_stamp":"2024-09-13T18:58:06.094356+0000","last_deep_scrub":"43'9","last_deep_scrub_stamp":"2024-09-13T18:58:06.094356+0000","last_clean_scrub_stamp":"2024-09-13T18:58:06.094356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:54.098829+0000","scrub_duration":0.0046552670000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":797,"num_read_kb":0,"num_write":803600,"num_write_kb":16319459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178039,"ondisk_log_size":178039,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":771,"ondisk_log_size":771,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":16507087,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1362,"num_read_kb":52080,"num_write":3742,"num_write_kb":17667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22716416,"data_stored":33014174,"data_compressed":2441304,"data_compressed_allocated":3506176,"data_compressed_original":13950976,"omap_allocated":0,"internal_metadata":0},"log_size":3641,"ondisk_log_size":3641,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65752,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509828,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1059364,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":1057150,"kb_avail":93312476,"statfs":{"total":96636764160,"available":95551975424,"internally_reserved":0,"allocated":2256896,"data_stored":4436700,"data_compressed":519875,"data_compressed_allocated":692224,"data_compressed_original":3125248,"omap_allocated":1593,"internal_metadata":1082522055},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":1.28,"15min":0.94199999999999995},"min":{"1min":0.47399999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.98799999999999999,"5min":10.43,"15min":10.43},"last":1.016},{"interface":"front","average":{"1min":0.68799999999999994,"5min":1.8029999999999999,"15min":1.153},"min":{"1min":0.48899999999999999,"5min":0.245,"15min":0.245},"max":{"1min":1.103,"5min":16.600999999999999,"15min":16.600999999999999},"last":0.73499999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":2.9279999999999999,"15min":1.8500000000000001},"min":{"1min":0.54700000000000004,"5min":0.252,"15min":0.252},"max":{"1min":1.0620000000000001,"5min":52.290999999999997,"15min":52.290999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.90600000000000003,"5min":3.0720000000000001,"15min":1.962},"min":{"1min":0.55000000000000004,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.3120000000000001,"5min":52.149000000000001,"15min":52.149000000000001},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":1.1679999999999999,"15min":1.052},"min":{"1min":0.47699999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1259999999999999,"5min":8.4819999999999993,"15min":14.933},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":1.252,"15min":1.04},"min":{"1min":0.495,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0640000000000001,"5min":9.2059999999999995,"15min":17.792000000000002},"last":0.90100000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":1.8759999999999999,"15min":1.4359999999999999},"min":{"1min":0.55200000000000005,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.077,"5min":9.6690000000000005,"15min":15.622},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":2.1560000000000001,"15min":1.7250000000000001},"min":{"1min":0.626,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.2010000000000001,"5min":14.993,"15min":28.248000000000001},"last":0.84999999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":1.1539999999999999,"15min":1.0620000000000001},"min":{"1min":0.58099999999999996,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0529999999999999,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.268,"15min":1.0609999999999999},"min":{"1min":0.83699999999999997,"5min":0.34599999999999997,"15min":0.23699999999999999},"max":{"1min":1.2370000000000001,"5min":7.9409999999999998,"15min":7.9409999999999998},"last":1.048}]},{"osd":5,"last update":"Fri Sep 13 18:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.92300000000000004,"15min":0.97899999999999998},"min":{"1min":0.67900000000000005,"5min":0.35399999999999998,"15min":0.28000000000000003},"max":{"1min":1.194,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.91000000000000003,"15min":0.96799999999999997},"min":{"1min":0.67700000000000005,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.171,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":1.25}]},{"osd":7,"last update":"Fri Sep 13 18:58:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.1040000000000001,"15min":1.0489999999999999},"min":{"1min":0.79500000000000004,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.278,"5min":12.021000000000001,"15min":12.021000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.028,"15min":1.0029999999999999},"min":{"1min":0.78400000000000003,"5min":0.32500000000000001,"15min":0.312},"max":{"1min":1.1619999999999999,"5min":9.0440000000000005,"15min":9.0440000000000005},"last":1.111}]}]},{"osd":7,"up_from":15,"seq":64424509828,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":944040,"kb_used_data":9760,"kb_used_omap":1,"kb_used_meta":934270,"kb_avail":93427800,"statfs":{"total":96636764160,"available":95670067200,"internally_reserved":0,"allocated":9994240,"data_stored":13045326,"data_compressed":732085,"data_compressed_allocated":1101824,"data_compressed_original":4415488,"omap_allocated":1589,"internal_metadata":956692939},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.956,"15min":1.24},"min":{"1min":0.253,"5min":0.217,"15min":0.217},"max":{"1min":8.7949999999999999,"5min":24.414000000000001,"15min":24.414000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.3200000000000001,"5min":2.089,"15min":1.383},"min":{"1min":0.34000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":8.8480000000000008,"5min":24.469000000000001,"15min":24.469000000000001},"last":0.67700000000000005}]},{"osd":1,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":3.1869999999999998,"15min":1.665},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":7.3250000000000002,"5min":46.161999999999999,"15min":46.161999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.333,"5min":3.9089999999999998,"15min":2.0459999999999998},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":7.548,"5min":45.713000000000001,"15min":45.713000000000001},"last":0.755}]},{"osd":2,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.208,"15min":0.98399999999999999},"min":{"1min":0.35499999999999998,"5min":0.29699999999999999,"15min":0.28999999999999998},"max":{"1min":3.4279999999999999,"5min":7.4740000000000002,"15min":7.4740000000000002},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.3340000000000001,"15min":0.99199999999999999},"min":{"1min":0.27800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.4699999999999998,"5min":9.1679999999999993,"15min":9.1679999999999993},"last":0.99299999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.8169999999999999,"5min":3.0920000000000001,"15min":1.869},"min":{"1min":0.35399999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.8140000000000001,"5min":35.494999999999997,"15min":35.494999999999997},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.212,"5min":2.5289999999999999,"15min":1.7030000000000001},"min":{"1min":0.33600000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":5.0880000000000001,"5min":19.946000000000002,"15min":19.946000000000002},"last":0.90500000000000003}]},{"osd":4,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.9690000000000001,"5min":1.198,"15min":0.98799999999999999},"min":{"1min":0.36499999999999999,"5min":0.249,"15min":0.249},"max":{"1min":10.477,"5min":10.477,"15min":10.477},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.2629999999999999,"15min":1.0980000000000001},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":10.499000000000001,"5min":10.499000000000001,"15min":10.499000000000001},"last":1.1839999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.0289999999999999,"15min":0.92300000000000004},"min":{"1min":0.30199999999999999,"5min":0.26600000000000001,"15min":0.26100000000000001},"max":{"1min":14.954000000000001,"5min":14.954000000000001,"15min":14.954000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.0369999999999999,"15min":0.99399999999999999},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":9.0180000000000007,"5min":9.0180000000000007,"15min":9.0180000000000007},"last":1.0269999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:57:44 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.415,"15min":1.1639999999999999},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":7.383,"5min":32.491,"15min":32.491},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.786,"5min":1.3240000000000001,"15min":1.141},"min":{"1min":0.44700000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":8.8770000000000007,"5min":27.291,"15min":27.291},"last":1.266}]}]},{"osd":4,"up_from":15,"seq":64424509827,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":981284,"kb_used_data":4508,"kb_used_omap":1,"kb_used_meta":976766,"kb_avail":93390556,"statfs":{"total":96636764160,"available":95631929344,"internally_reserved":0,"allocated":4616192,"data_stored":4436064,"data_compressed":20605,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":1594,"internal_metadata":1000208838},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:58:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":1.8979999999999999,"15min":1.2390000000000001},"min":{"1min":0.30099999999999999,"5min":0.24099999999999999,"15min":0.223},"max":{"1min":1.0229999999999999,"5min":28.593,"15min":28.593},"last":1.103},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.9219999999999999,"15min":1.2529999999999999},"min":{"1min":0.32100000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.268,"5min":28.559000000000001,"15min":28.559000000000001},"last":1.232}]},{"osd":1,"last update":"Fri Sep 13 18:58:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":2.9089999999999998,"15min":1.6950000000000001},"min":{"1min":0.42099999999999999,"5min":0.245,"15min":0.245},"max":{"1min":1.2330000000000001,"5min":37.100999999999999,"15min":37.100999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":3.246,"15min":1.8240000000000001},"min":{"1min":0.503,"5min":0.253,"15min":0.253},"max":{"1min":1.0629999999999999,"5min":44.798000000000002,"15min":44.798000000000002},"last":0.88500000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:58:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.8109999999999999,"15min":1.2330000000000001},"min":{"1min":0.51500000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.4349999999999996,"5min":27.634,"15min":27.634},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.6970000000000001,"15min":1.2190000000000001},"min":{"1min":0.54900000000000004,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.6930000000000001,"5min":27.606000000000002,"15min":27.606000000000002},"last":1.0740000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:58:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":2.7850000000000001,"15min":1.706},"min":{"1min":0.41199999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.028,"5min":40.676000000000002,"15min":40.676000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1839999999999999,"5min":3.4489999999999998,"15min":1.988},"min":{"1min":0.59899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.2649999999999997,"5min":40.881999999999998,"15min":40.881999999999998},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:58:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96499999999999997,"15min":0.90100000000000002},"min":{"1min":0.38,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.4279999999999999,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.119,"5min":0.96599999999999997,"15min":0.999},"min":{"1min":0.42899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.6879999999999999,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":1.024}]},{"osd":6,"last update":"Fri Sep 13 18:58:21 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.258,"15min":1.1020000000000001},"min":{"1min":0.44800000000000001,"5min":0.251,"15min":0.251},"max":{"1min":1.4890000000000001,"5min":9.4570000000000007,"15min":9.4570000000000007},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.0189999999999999,"15min":1.0289999999999999},"min":{"1min":0.46800000000000003,"5min":0.317,"15min":0.317},"max":{"1min":1.78,"5min":5.1059999999999999,"15min":6.1319999999999997},"last":0.98499999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:58:21 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.4750000000000001,"15min":1.216},"min":{"1min":0.48399999999999999,"5min":0.307,"15min":0.307},"max":{"1min":4.2629999999999999,"5min":10.101000000000001,"15min":10.101000000000001},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.5369999999999999,"15min":1.3069999999999999},"min":{"1min":0.53300000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.2930000000000001,"5min":10.127000000000001,"15min":10.127000000000001},"last":1.5740000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542531,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1272936,"kb_used_data":5472,"kb_used_omap":1,"kb_used_meta":1267454,"kb_avail":93098904,"statfs":{"total":96636764160,"available":95333277696,"internally_reserved":0,"allocated":5603328,"data_stored":8725194,"data_compressed":790472,"data_compressed_allocated":1138688,"data_compressed_original":4517888,"omap_allocated":1591,"internal_metadata":1297873353},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:32 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":2.9300000000000002,"15min":1.5609999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":10.571999999999999,"5min":22.670000000000002,"15min":22.670000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.266,"5min":2.9980000000000002,"15min":1.716},"min":{"1min":0.23000000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":10.512,"5min":22.728000000000002,"15min":22.728000000000002},"last":0.70199999999999996}]},{"osd":1,"last update":"Fri Sep 13 18:58:21 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":4.6689999999999996,"15min":2.7509999999999999},"min":{"1min":0.36499999999999999,"5min":0.191,"15min":0.191},"max":{"1min":6.5570000000000004,"5min":42.046999999999997,"15min":42.046999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.085,"5min":4.3419999999999996,"15min":2.762},"min":{"1min":0.47799999999999998,"5min":0.223,"15min":0.223},"max":{"1min":6.6159999999999997,"5min":42.024999999999999,"15min":42.024999999999999},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:58:21 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":3.3530000000000002,"15min":2.1080000000000001},"min":{"1min":0.40600000000000003,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.5270000000000001,"5min":41.909999999999997,"15min":41.909999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.0249999999999999,"5min":2.5470000000000002,"15min":1.571},"min":{"1min":0.24299999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":6.7009999999999996,"5min":27.25,"15min":27.25},"last":0.83599999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:57:32 2024","interfaces":[{"interface":"back","average":{"1min":2.52,"5min":3.1509999999999998,"15min":1.8520000000000001},"min":{"1min":0.45000000000000001,"5min":0.30099999999999999,"15min":0.26300000000000001},"max":{"1min":19.533999999999999,"5min":42.003,"15min":42.003},"last":0.93799999999999994},{"interface":"front","average":{"1min":2.9780000000000002,"5min":3.46,"15min":2.0950000000000002},"min":{"1min":0.41699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":19.728999999999999,"5min":41.951000000000001,"15min":41.951000000000001},"last":0.875}]},{"osd":5,"last update":"Fri Sep 13 18:58:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":2.1120000000000001,"15min":1.534},"min":{"1min":0.38400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.1519999999999999,"5min":22.988,"15min":22.988},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.921,"15min":1.4630000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.228,"5min":23.030000000000001,"15min":23.030000000000001},"last":0.97999999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:58:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":2.4529999999999998,"15min":1.6970000000000001},"min":{"1min":0.41799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.2689999999999999,"5min":23.087,"15min":23.087},"last":1.04},{"interface":"front","average":{"1min":0.93899999999999995,"5min":2.9169999999999998,"15min":1.8979999999999999},"min":{"1min":0.38500000000000001,"5min":0.317,"15min":0.317},"max":{"1min":1.304,"5min":23.425000000000001,"15min":23.425000000000001},"last":1.0649999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:58:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":2.9260000000000002,"15min":1.8979999999999999},"min":{"1min":0.64700000000000002,"5min":0.35899999999999999,"15min":0.32600000000000001},"max":{"1min":1.335,"5min":41.975999999999999,"15min":41.975999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":2.5619999999999998,"15min":1.728},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34200000000000003},"max":{"1min":1.3540000000000001,"5min":23.457999999999998,"15min":23.457999999999998},"last":0.89800000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542535,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746120,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":745534,"kb_avail":93625720,"statfs":{"total":96636764160,"available":95872737280,"internally_reserved":0,"allocated":589824,"data_stored":334673,"data_compressed":1568,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":763427271},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":2.4740000000000002,"15min":1.292},"min":{"1min":0.216,"5min":0.17799999999999999,"15min":0.16700000000000001},"max":{"1min":4.1050000000000004,"5min":37.600000000000001,"15min":37.600000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":2.4470000000000001,"15min":1.323},"min":{"1min":0.254,"5min":0.19,"15min":0.19},"max":{"1min":3.996,"5min":37.651000000000003,"15min":37.651000000000003},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":2.681,"15min":1.7030000000000001},"min":{"1min":0.372,"5min":0.26000000000000001,"15min":0.255},"max":{"1min":1.407,"5min":17.443999999999999,"15min":18.055},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":2.4820000000000002,"15min":1.5840000000000001},"min":{"1min":0.35299999999999998,"5min":0.20000000000000001,"15min":0.188},"max":{"1min":1.371,"5min":15.94,"15min":18.175999999999998},"last":0.88300000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":2.899,"15min":1.804},"min":{"1min":0.252,"5min":0.252,"15min":0.249},"max":{"1min":1.282,"5min":25.832999999999998,"15min":25.832999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":2.8159999999999998,"15min":1.8149999999999999},"min":{"1min":0.29199999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.3580000000000001,"5min":17.59,"15min":17.59},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":1.198,"15min":1.024},"min":{"1min":0.435,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.226,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.175,"15min":1.0880000000000001},"min":{"1min":0.41899999999999998,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":1.397,"5min":8.0999999999999996,"15min":8.0999999999999996},"last":0.79300000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":1.2649999999999999,"15min":1.0580000000000001},"min":{"1min":0.38100000000000001,"5min":0.29799999999999999,"15min":0.29499999999999998},"max":{"1min":1.375,"5min":8.4800000000000004,"15min":8.4800000000000004},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.387,"15min":1.1539999999999999},"min":{"1min":0.51000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.415,"5min":8.9949999999999992,"15min":8.9949999999999992},"last":0.96999999999999997}]},{"osd":6,"last update":"Fri Sep 13 18:58:22 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.236,"15min":1.169},"min":{"1min":0.35799999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.4450000000000001,"5min":9.6720000000000006,"15min":9.6720000000000006},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.167,"15min":1.0780000000000001},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":1.258,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":1.0089999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.383,"15min":1.1659999999999999},"min":{"1min":0.56599999999999995,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2729999999999999,"5min":8.9250000000000007,"15min":8.9250000000000007},"last":1.071},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.5349999999999999,"15min":1.25},"min":{"1min":0.39200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4039999999999999,"5min":9.2330000000000005,"15min":9.2330000000000005},"last":0.92800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575236,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730672,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":730174,"kb_avail":93641168,"statfs":{"total":96636764160,"available":95888556032,"internally_reserved":0,"allocated":499712,"data_stored":241392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":747698631},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:58:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.7789999999999999,"15min":1.179},"min":{"1min":0.248,"5min":0.219,"15min":0.219},"max":{"1min":1.052,"5min":9.952,"15min":9.952},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.905,"15min":1.3100000000000001},"min":{"1min":0.39800000000000002,"5min":0.29599999999999999,"15min":0.29399999999999998},"max":{"1min":1.206,"5min":14.317,"15min":14.317},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:58:26 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.8779999999999999,"15min":1.355},"min":{"1min":0.496,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":7.5999999999999996,"5min":10.85,"15min":10.85},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.843,"15min":1.3540000000000001},"min":{"1min":0.34899999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.665,"5min":10.868,"15min":10.868},"last":1.038}]},{"osd":2,"last update":"Fri Sep 13 18:58:26 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":2.093,"15min":1.2969999999999999},"min":{"1min":0.371,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.097,"5min":38.484000000000002,"15min":38.484000000000002},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":2.161,"15min":1.367},"min":{"1min":0.52800000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.012,"5min":40.686,"15min":40.686},"last":0.88300000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:58:26 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":3.7879999999999998,"15min":2.0859999999999999},"min":{"1min":0.33100000000000002,"5min":0.318,"15min":0.318},"max":{"1min":1.194,"5min":72.534999999999997,"15min":72.534999999999997},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.94599999999999995,"5min":3.6469999999999998,"15min":2.1019999999999999},"min":{"1min":0.438,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.1739999999999999,"5min":72.486999999999995,"15min":72.486999999999995},"last":0.73699999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:58:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.97899999999999998,"15min":0.92100000000000004},"min":{"1min":0.42599999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.3180000000000001,"5min":7.8710000000000004,"15min":7.8710000000000004},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.1619999999999999,"15min":1.0720000000000001},"min":{"1min":0.44400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.3089999999999999,"5min":8.7829999999999995,"15min":8.7829999999999995},"last":1.0680000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:57:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.97499999999999998,"15min":0.999},"min":{"1min":0.32400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.5179999999999998,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.0569999999999999,"15min":1.0509999999999999},"min":{"1min":0.38500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.54,"5min":4.516,"15min":4.516},"last":1.337}]},{"osd":7,"last update":"Fri Sep 13 18:57:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.1919999999999999,"15min":1.0640000000000001},"min":{"1min":0.46400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.5780000000000003,"5min":7.6020000000000003,"15min":7.6020000000000003},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.179,"15min":1.139},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.9699999999999998,"5min":7.9109999999999996,"15min":7.9109999999999996},"last":1.3899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575234,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1223156,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":1222654,"kb_avail":93148684,"statfs":{"total":96636764160,"available":95384252416,"internally_reserved":0,"allocated":503808,"data_stored":240958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1251998150},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":2.0539999999999998,"5min":2.7050000000000001,"15min":1.5920000000000001},"min":{"1min":0.214,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":10.279,"5min":28.297000000000001,"15min":28.297000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":2.464,"5min":2.7149999999999999,"15min":1.466},"min":{"1min":0.32400000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":10.162000000000001,"5min":19.785,"15min":19.785},"last":0.64200000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":2.5609999999999999,"15min":1.4359999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.17799999999999999},"max":{"1min":7.2380000000000004,"5min":25.609999999999999,"15min":25.609999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.905,"5min":2.891,"15min":1.619},"min":{"1min":0.29299999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":10.465,"5min":30.469999999999999,"15min":30.469999999999999},"last":1.0189999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":4.1040000000000001,"5min":5.694,"15min":2.6179999999999999},"min":{"1min":0.48499999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":18.695,"5min":36.643999999999998,"15min":36.643999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":4.2220000000000004,"5min":5.8979999999999997,"15min":2.754},"min":{"1min":0.53800000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":18.760000000000002,"5min":36.670000000000002,"15min":36.670000000000002},"last":0.80300000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:57:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":1.4399999999999999,"15min":1.1779999999999999},"min":{"1min":0.28399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.7110000000000003,"5min":7.4669999999999996,"15min":7.6799999999999997},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.766,"15min":1.339},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":4.508,"5min":14.906000000000001,"15min":14.906000000000001},"last":1.0640000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.986,"15min":1.3700000000000001},"min":{"1min":0.38800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.259,"5min":10.259,"15min":10.259},"last":1.369},{"interface":"front","average":{"1min":1.6910000000000001,"5min":2.0289999999999999,"15min":1.3799999999999999},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34100000000000003},"max":{"1min":10.231999999999999,"5min":12.27,"15min":12.27},"last":1.3100000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":2.093,"5min":2.4220000000000002,"15min":1.5449999999999999},"min":{"1min":0.432,"5min":0.371,"15min":0.316},"max":{"1min":10.538,"5min":28.981999999999999,"15min":28.981999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.8700000000000001,"15min":1.429},"min":{"1min":0.36599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":10.342000000000001,"5min":15.727,"15min":15.727},"last":0.96999999999999997}]},{"osd":7,"last update":"Fri Sep 13 18:57:30 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":2.0630000000000002,"15min":1.466},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":7.2130000000000001,"5min":28.100000000000001,"15min":28.100000000000001},"last":1},{"interface":"front","average":{"1min":1.2889999999999999,"5min":2.0710000000000002,"15min":1.5389999999999999},"min":{"1min":0.41099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":7.173,"5min":28.896999999999998,"15min":28.896999999999998},"last":1.085}]}]},{"osd":0,"up_from":13,"seq":55834575235,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":842628,"kb_used_data":2556,"kb_used_omap":1,"kb_used_meta":840062,"kb_avail":93529212,"statfs":{"total":96636764160,"available":95773913088,"internally_reserved":0,"allocated":2617344,"data_stored":4654715,"data_compressed":442451,"data_compressed_allocated":618496,"data_compressed_original":2916352,"omap_allocated":1594,"internal_metadata":860223942},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:58:23 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":3.4460000000000002,"15min":1.8500000000000001},"min":{"1min":0.45600000000000002,"5min":0.183,"15min":0.183},"max":{"1min":10.504,"5min":29.712,"15min":29.712},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.248,"5min":2.7069999999999999,"15min":1.6000000000000001},"min":{"1min":0.56599999999999995,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":10.539,"5min":29.748000000000001,"15min":29.748000000000001},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:58:23 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.54,"15min":1.0580000000000001},"min":{"1min":0.503,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":2.0880000000000001,"5min":17.611000000000001,"15min":17.611000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":1.74,"15min":1.1499999999999999},"min":{"1min":0.34499999999999997,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":2.0760000000000001,"5min":17.501999999999999,"15min":17.501999999999999},"last":0.79200000000000004}]},{"osd":3,"last update":"Fri Sep 13 18:58:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":2.819,"15min":1.671},"min":{"1min":0.36099999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.1040000000000001,"5min":17.260000000000002,"15min":17.260000000000002},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":2.3999999999999999,"15min":1.5369999999999999},"min":{"1min":0.29799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.208,"5min":17.088000000000001,"15min":17.088000000000001},"last":0.85299999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:58:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":2.306,"15min":1.429},"min":{"1min":0.61599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.6840000000000002,"5min":40.101999999999997,"15min":40.101999999999997},"last":1.1659999999999999},{"interface":"front","average":{"1min":0.86299999999999999,"5min":2.0249999999999999,"15min":1.304},"min":{"1min":0.44500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.333,"5min":40.981000000000002,"15min":40.981000000000002},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Sep 13 18:58:23 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.823,"15min":1.246},"min":{"1min":0.504,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1850000000000001,"5min":40.701999999999998,"15min":40.701999999999998},"last":1.091},{"interface":"front","average":{"1min":0.91700000000000004,"5min":2.0310000000000001,"15min":1.3400000000000001},"min":{"1min":0.47499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.3879999999999999,"5min":40.130000000000003,"15min":40.130000000000003},"last":1.115}]},{"osd":6,"last update":"Fri Sep 13 18:58:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":2.1629999999999998,"15min":1.4059999999999999},"min":{"1min":0.39700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.7430000000000001,"5min":51.719000000000001,"15min":51.719000000000001},"last":1.214},{"interface":"front","average":{"1min":1.1950000000000001,"5min":2.2690000000000001,"15min":1.4730000000000001},"min":{"1min":0.82599999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.641,"5min":51.75,"15min":51.75},"last":1.2529999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:57:34 2024","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":2.4119999999999999,"15min":1.522},"min":{"1min":0.44400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":8.0120000000000005,"5min":58.810000000000002,"15min":58.810000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.6299999999999999,"5min":2.2389999999999999,"15min":1.4770000000000001},"min":{"1min":0.54100000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.508,"5min":39.622,"15min":39.622},"last":1.298}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32876,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2072576,"data_stored":3824333,"data_compressed":409575,"data_compressed_allocated":573440,"data_compressed_original":2342912,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":94659,"data_compressed":1568,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5124096,"data_stored":8485180,"data_compressed":790472,"data_compressed_allocated":1138688,"data_compressed_original":4517888,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4136960,"data_stored":4196050,"data_compressed":20605,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1777664,"data_stored":4196686,"data_compressed":519875,"data_compressed_allocated":692224,"data_compressed_original":3125248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9449472,"data_stored":12214944,"data_compressed":699209,"data_compressed_allocated":1056768,"data_compressed_original":3842048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:58:34.885 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T18:58:34.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffeafa700 1 Processor -- start 2024-09-13T18:58:34.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffeafa700 1 -- start start 2024-09-13T18:58:34.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffeafa700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6ff80cc870 0x7f6ff80c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:34.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffeafa700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6ff80c79d0 0x7f6ff80c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:34.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffeafa700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff806f8c0 con 0x7f6ff80cc490 2024-09-13T18:58:34.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffeafa700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff80c5860 con 0x7f6ff80c79d0 2024-09-13T18:58:34.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffeafa700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff806ea40 con 0x7f6ff80cc870 2024-09-13T18:58:34.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffdaf8700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6ff80cc490 0x7f6ff806e230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57816/0 (socket says 172.21.15.146:57816) 2024-09-13T18:58:34.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffdaf8700 1 -- 172.21.15.146:0/4225855396 learned_addr learned my addr 172.21.15.146:0/4225855396 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:58:34.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffe2f9700 1 --2- 172.21.15.146:0/4225855396 >> v2:172.21.15.146:3300/0 conn(0x7f6ff80c79d0 0x7f6ff80c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:34.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffd2f7700 1 --2- 172.21.15.146:0/4225855396 >> v2:172.21.15.154:3300/0 conn(0x7f6ff80cc870 0x7f6ff80c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:34.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffe2f9700 1 -- 172.21.15.146:0/4225855396 >> v2:172.21.15.154:3300/0 conn(0x7f6ff80cc870 msgr2=0x7f6ff80c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:34.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffe2f9700 1 --2- 172.21.15.146:0/4225855396 >> v2:172.21.15.154:3300/0 conn(0x7f6ff80cc870 0x7f6ff80c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:34.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffe2f9700 1 -- 172.21.15.146:0/4225855396 >> v1:172.21.15.146:6789/0 conn(0x7f6ff80cc490 legacy=0x7f6ff806e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:34.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffe2f9700 1 -- 172.21.15.146:0/4225855396 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff806fb80 con 0x7f6ff80c79d0 2024-09-13T18:58:34.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffcaf6700 1 -- 172.21.15.146:0/4225855396 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383789675 0 0) 0x7f6ff806f8c0 con 0x7f6ff80cc490 2024-09-13T18:58:34.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffe2f9700 1 --2- 172.21.15.146:0/4225855396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ff80c79d0 0x7f6ff80c5cb0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009d90 tx=0x7f6ff000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb006546821e1b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:34.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffcaf6700 1 -- 172.21.15.146:0/4225855396 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000e070 con 0x7f6ff80c79d0 2024-09-13T18:58:34.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffcaf6700 1 -- 172.21.15.146:0/4225855396 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff000bb90 con 0x7f6ff80c79d0 2024-09-13T18:58:34.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.985+0000 7f6ffcaf6700 1 -- 172.21.15.146:0/4225855396 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0012430 con 0x7f6ff80c79d0 2024-09-13T18:58:34.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.989+0000 7f6ffeafa700 1 -- 172.21.15.146:0/4225855396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ff80c79d0 msgr2=0x7f6ff80c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:34.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.989+0000 7f6ffeafa700 1 --2- 172.21.15.146:0/4225855396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ff80c79d0 0x7f6ff80c5cb0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009d90 tx=0x7f6ff000b330 comp rx=0 tx=0).stop 2024-09-13T18:58:35.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.989+0000 7f6ffeafa700 1 -- 172.21.15.146:0/4225855396 shutdown_connections 2024-09-13T18:58:35.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.989+0000 7f6ffeafa700 1 --2- 172.21.15.146:0/4225855396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ff80c79d0 0x7f6ff80c5cb0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.989+0000 7f6ffeafa700 1 --2- 172.21.15.146:0/4225855396 >> v2:172.21.15.154:3300/0 conn(0x7f6ff80cc870 0x7f6ff80c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.989+0000 7f6ffeafa700 1 -- 172.21.15.146:0/4225855396 >> 172.21.15.146:0/4225855396 conn(0x7f6ff8071ed0 msgr2=0x7f6ff80722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:35.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.989+0000 7f6ffeafa700 1 -- 172.21.15.146:0/4225855396 shutdown_connections 2024-09-13T18:58:35.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.989+0000 7f6ffeafa700 1 -- 172.21.15.146:0/4225855396 wait complete. 2024-09-13T18:58:35.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.989+0000 7f6ffeafa700 1 Processor -- start 2024-09-13T18:58:35.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.989+0000 7f6ffeafa700 1 -- start start 2024-09-13T18:58:35.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffeafa700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ff80c79d0 0x7f6ff814cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffeafa700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ff80cc490 0x7f6ff80cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffeafa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ff80cc870 0x7f6ff80cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffeafa700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff80caf80 con 0x7f6ff80c79d0 2024-09-13T18:58:35.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffeafa700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff80c4490 con 0x7f6ff80cc870 2024-09-13T18:58:35.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffeafa700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff80c4090 con 0x7f6ff80cc490 2024-09-13T18:58:35.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffdaf8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ff80c79d0 0x7f6ff814cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffdaf8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ff80c79d0 0x7f6ff814cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45530/0 (socket says 172.21.15.146:45530) 2024-09-13T18:58:35.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffdaf8700 1 -- 172.21.15.146:0/2146103148 learned_addr learned my addr 172.21.15.146:0/2146103148 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:35.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffd2f7700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ff80cc490 0x7f6ff80cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffe2f9700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ff80cc870 0x7f6ff80cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffdaf8700 1 -- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ff80cc490 msgr2=0x7f6ff80cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffdaf8700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ff80cc490 0x7f6ff80cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffdaf8700 1 -- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ff80cc870 msgr2=0x7f6ff80cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffdaf8700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ff80cc870 0x7f6ff80cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffdaf8700 1 -- 172.21.15.146:0/2146103148 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0009a70 con 0x7f6ff80c79d0 2024-09-13T18:58:35.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffe2f9700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ff80cc870 0x7f6ff80cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:58:35.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffdaf8700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ff80c79d0 0x7f6ff814cca0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400c5b0 tx=0x7f6ff400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:35.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6fee7fc700 1 -- 172.21.15.146:0/2146103148 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4004ca0 con 0x7f6ff80c79d0 2024-09-13T18:58:35.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffeafa700 1 -- 172.21.15.146:0/2146103148 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff814eb20 con 0x7f6ff80c79d0 2024-09-13T18:58:35.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6fee7fc700 1 -- 172.21.15.146:0/2146103148 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff4004e20 con 0x7f6ff80c79d0 2024-09-13T18:58:35.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6fee7fc700 1 -- 172.21.15.146:0/2146103148 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4007310 con 0x7f6ff80c79d0 2024-09-13T18:58:35.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffeafa700 1 -- 172.21.15.146:0/2146103148 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff814f070 con 0x7f6ff80c79d0 2024-09-13T18:58:35.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6ffeafa700 1 -- 172.21.15.146:0/2146103148 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff80c8860 con 0x7f6ff80c79d0 2024-09-13T18:58:35.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.993+0000 7f6fee7fc700 1 -- 172.21.15.146:0/2146103148 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6ff4007470 con 0x7f6ff80c79d0 2024-09-13T18:58:35.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.997+0000 7f6fee7fc700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6fdc066730 0x7f6fdc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.997+0000 7f6fee7fc700 1 -- 172.21.15.146:0/2146103148 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f6ff4084840 con 0x7f6ff80c79d0 2024-09-13T18:58:35.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.997+0000 7f6ffd2f7700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6fdc066730 0x7f6fdc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:34.997+0000 7f6ffd2f7700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6fdc066730 0x7f6fdc068bf0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8072420 tx=0x7f6fe8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:35.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.005+0000 7f6fee7fc700 1 -- 172.21.15.146:0/2146103148 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6ff4051210 con 0x7f6ff80c79d0 2024-09-13T18:58:35.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.173+0000 7f6ffeafa700 1 -- 172.21.15.146:0/2146103148 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f6ff80c6960 con 0x7f6ff80c79d0 2024-09-13T18:58:35.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.181+0000 7f6fee7fc700 1 -- 172.21.15.146:0/2146103148 <== mon.0 v2:172.21.15.146:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6ff40012d0 con 0x7f6ff80c79d0 2024-09-13T18:58:35.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.181+0000 7f6fee7fc700 1 -- 172.21.15.146:0/2146103148 <== mon.0 v2:172.21.15.146:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6ff4007720 con 0x7f6ff80c79d0 2024-09-13T18:58:35.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.185+0000 7f6ffeafa700 1 -- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6fdc066730 msgr2=0x7f6fdc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.185+0000 7f6ffeafa700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6fdc066730 0x7f6fdc068bf0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8072420 tx=0x7f6fe8008040 comp rx=0 tx=0).stop 2024-09-13T18:58:35.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.185+0000 7f6ffeafa700 1 -- 172.21.15.146:0/2146103148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ff80c79d0 msgr2=0x7f6ff814cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.185+0000 7f6ffeafa700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ff80c79d0 0x7f6ff814cca0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400c5b0 tx=0x7f6ff400c970 comp rx=0 tx=0).stop 2024-09-13T18:58:35.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.185+0000 7f6ffeafa700 1 -- 172.21.15.146:0/2146103148 shutdown_connections 2024-09-13T18:58:35.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.185+0000 7f6ffeafa700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f6fdc066730 0x7f6fdc068bf0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.185+0000 7f6ffeafa700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ff80cc870 0x7f6ff80cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.185+0000 7f6ffeafa700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ff80cc490 0x7f6ff80cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.185+0000 7f6ffeafa700 1 --2- 172.21.15.146:0/2146103148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ff80c79d0 0x7f6ff814cca0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.185+0000 7f6ffeafa700 1 -- 172.21.15.146:0/2146103148 >> 172.21.15.146:0/2146103148 conn(0x7f6ff8071ed0 msgr2=0x7f6ff80c5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:35.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.185+0000 7f6ffeafa700 1 -- 172.21.15.146:0/2146103148 shutdown_connections 2024-09-13T18:58:35.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.185+0000 7f6ffeafa700 1 -- 172.21.15.146:0/2146103148 wait complete. 2024-09-13T18:58:35.208 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T18:58:35.208 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:58:35.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.309+0000 7fe0c2c6e700 1 Processor -- start 2024-09-13T18:58:35.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 -- start start 2024-09-13T18:58:35.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe0bc150230 0x7fe0bc06ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe0bc150610 0x7fe0bc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc071410 con 0x7fe0bc071610 2024-09-13T18:58:35.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0bc1596b0 con 0x7fe0bc150230 2024-09-13T18:58:35.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0bc156090 con 0x7fe0bc150610 2024-09-13T18:58:35.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c1c6c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe0bc150230 0x7fe0bc06ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c1c6c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe0bc150230 0x7fe0bc06ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45544/0 (socket says 172.21.15.146:45544) 2024-09-13T18:58:35.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c1c6c700 1 -- 172.21.15.146:0/23077856 learned_addr learned my addr 172.21.15.146:0/23077856 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:35.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c146b700 1 --2- 172.21.15.146:0/23077856 >> v2:172.21.15.154:3300/0 conn(0x7fe0bc150610 0x7fe0bc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c1c6c700 1 -- 172.21.15.146:0/23077856 >> v2:172.21.15.154:3300/0 conn(0x7fe0bc150610 msgr2=0x7fe0bc06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:35.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c1c6c700 1 --2- 172.21.15.146:0/23077856 >> v2:172.21.15.154:3300/0 conn(0x7fe0bc150610 0x7fe0bc06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c1c6c700 1 -- 172.21.15.146:0/23077856 >> v1:172.21.15.154:6789/0 conn(0x7fe0bc071610 legacy=0x7fe0bc159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c1c6c700 1 -- 172.21.15.146:0/23077856 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc156350 con 0x7fe0bc150230 2024-09-13T18:58:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c1c6c700 1 --2- 172.21.15.146:0/23077856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0bc150230 0x7fe0bc06ebd0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc06e950 tx=0x7fe0b8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb341d6e68236e5e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c0c6a700 1 -- 172.21.15.146:0/23077856 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8011830 con 0x7fe0bc150230 2024-09-13T18:58:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c0c6a700 1 -- 172.21.15.146:0/23077856 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe0b8011990 con 0x7fe0bc150230 2024-09-13T18:58:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c0c6a700 1 -- 172.21.15.146:0/23077856 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8011d40 con 0x7fe0bc150230 2024-09-13T18:58:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/23077856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0bc150230 msgr2=0x7fe0bc06ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 --2- 172.21.15.146:0/23077856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0bc150230 0x7fe0bc06ebd0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc06e950 tx=0x7fe0b8009bf0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/23077856 shutdown_connections 2024-09-13T18:58:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 --2- 172.21.15.146:0/23077856 >> v2:172.21.15.154:3300/0 conn(0x7fe0bc150610 0x7fe0bc06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 --2- 172.21.15.146:0/23077856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0bc150230 0x7fe0bc06ebd0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/23077856 >> 172.21.15.146:0/23077856 conn(0x7fe0bc071ed0 msgr2=0x7fe0bc0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:35.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/23077856 shutdown_connections 2024-09-13T18:58:35.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.313+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/23077856 wait complete. 2024-09-13T18:58:35.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c2c6e700 1 Processor -- start 2024-09-13T18:58:35.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c2c6e700 1 -- start start 2024-09-13T18:58:35.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c2c6e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0bc071610 0x7fe0bc152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c2c6e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0bc150230 0x7fe0bc157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c2c6e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0bc151b80 0x7fe0bc157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c2c6e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc151870 con 0x7fe0bc151b80 2024-09-13T18:58:35.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c2c6e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc1536c0 con 0x7fe0bc071610 2024-09-13T18:58:35.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c2c6e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc1555d0 con 0x7fe0bc150230 2024-09-13T18:58:35.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c246d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0bc151b80 0x7fe0bc157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c246d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0bc151b80 0x7fe0bc157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45558/0 (socket says 172.21.15.146:45558) 2024-09-13T18:58:35.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c246d700 1 -- 172.21.15.146:0/2662828348 learned_addr learned my addr 172.21.15.146:0/2662828348 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:35.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c146b700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0bc150230 0x7fe0bc157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c1c6c700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0bc071610 0x7fe0bc152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c246d700 1 -- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0bc150230 msgr2=0x7fe0bc157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c246d700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0bc150230 0x7fe0bc157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c246d700 1 -- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0bc071610 msgr2=0x7fe0bc152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c246d700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0bc071610 0x7fe0bc152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c246d700 1 -- 172.21.15.146:0/2662828348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b8009570 con 0x7fe0bc151b80 2024-09-13T18:58:35.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c1c6c700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0bc071610 0x7fe0bc152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:58:35.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c246d700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0bc151b80 0x7fe0bc157d70 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400ee20 tx=0x7fe0b400c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:35.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0b27fc700 1 -- 172.21.15.146:0/2662828348 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4003a20 con 0x7fe0bc151b80 2024-09-13T18:58:35.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0b27fc700 1 -- 172.21.15.146:0/2662828348 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe0b4003b80 con 0x7fe0bc151b80 2024-09-13T18:58:35.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/2662828348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0bc1ec260 con 0x7fe0bc151b80 2024-09-13T18:58:35.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0b27fc700 1 -- 172.21.15.146:0/2662828348 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40050d0 con 0x7fe0bc151b80 2024-09-13T18:58:35.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.317+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/2662828348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc1552f0 con 0x7fe0bc151b80 2024-09-13T18:58:35.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.321+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/2662828348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0bc153870 con 0x7fe0bc151b80 2024-09-13T18:58:35.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.321+0000 7fe0b27fc700 1 -- 172.21.15.146:0/2662828348 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe0b4005230 con 0x7fe0bc151b80 2024-09-13T18:58:35.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.321+0000 7fe0b27fc700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe0a0066730 0x7fe0a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.321+0000 7fe0b27fc700 1 -- 172.21.15.146:0/2662828348 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fe0b4087230 con 0x7fe0bc151b80 2024-09-13T18:58:35.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.321+0000 7fe0c1c6c700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe0a0066730 0x7fe0a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.321+0000 7fe0c1c6c700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe0a0066730 0x7fe0a0068bf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8000c00 tx=0x7fe0b8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:35.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.329+0000 7fe0b27fc700 1 -- 172.21.15.146:0/2662828348 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe0bc153870 con 0x7fe0bc151b80 2024-09-13T18:58:35.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.521+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/2662828348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0bc153870 con 0x7fe0bc151b80 2024-09-13T18:58:35.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.521+0000 7fe0b27fc700 1 -- 172.21.15.146:0/2662828348 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+6297 (secure 0 0 0) 0x7fe0bc153870 con 0x7fe0bc151b80 2024-09-13T18:58:35.531 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:58:35.531 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":118,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6323,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2261933208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2261933208},{"type":"v1","addr":"172.21.15.154:6839","nonce":2261933208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:55:47.963024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":7224,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7224":{"gid":7224,"name":"c","rank":1,"incarnation":113,"state":"up:active","state_seq":25,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:58:35.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.525+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe0a0066730 msgr2=0x7fe0a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.525+0000 7fe0c2c6e700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe0a0066730 0x7fe0a0068bf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8000c00 tx=0x7fe0b8008030 comp rx=0 tx=0).stop 2024-09-13T18:58:35.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.525+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/2662828348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0bc151b80 msgr2=0x7fe0bc157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.525+0000 7fe0c2c6e700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0bc151b80 0x7fe0bc157d70 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400ee20 tx=0x7fe0b400c550 comp rx=0 tx=0).stop 2024-09-13T18:58:35.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.525+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/2662828348 shutdown_connections 2024-09-13T18:58:35.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.525+0000 7fe0c2c6e700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fe0a0066730 0x7fe0a0068bf0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.525+0000 7fe0c2c6e700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0bc151b80 0x7fe0bc157d70 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.525+0000 7fe0c2c6e700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0bc150230 0x7fe0bc157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.525+0000 7fe0c2c6e700 1 --2- 172.21.15.146:0/2662828348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0bc071610 0x7fe0bc152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.525+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/2662828348 >> 172.21.15.146:0/2662828348 conn(0x7fe0bc071ed0 msgr2=0x7fe0bc153cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:35.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.525+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/2662828348 shutdown_connections 2024-09-13T18:58:35.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.525+0000 7fe0c2c6e700 1 -- 172.21.15.146:0/2662828348 wait complete. 2024-09-13T18:58:35.535 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 118 2024-09-13T18:58:35.546 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:58:35.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f96724700 1 Processor -- start 2024-09-13T18:58:35.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f96724700 1 -- start start 2024-09-13T18:58:35.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f96724700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f90151840 0x7f7f900c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f96724700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f900c8380 0x7f7f90161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f96724700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f900c9890 con 0x7f7f90157fc0 2024-09-13T18:58:35.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f96724700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f900c36f0 con 0x7f7f90151840 2024-09-13T18:58:35.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f96724700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f901572d0 con 0x7f7f900c8380 2024-09-13T18:58:35.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f94f21700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f7f90157fc0 0x7f7f9006c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57824/0 (socket says 172.21.15.146:57824) 2024-09-13T18:58:35.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f94f21700 1 -- 172.21.15.146:0/3887440039 learned_addr learned my addr 172.21.15.146:0/3887440039 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:58:35.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f95f23700 1 --2- 172.21.15.146:0/3887440039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f900c8380 0x7f7f90161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f95722700 1 --2- 172.21.15.146:0/3887440039 >> v2:172.21.15.154:3300/0 conn(0x7f7f90151840 0x7f7f900c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f87fff700 1 -- 172.21.15.146:0/3887440039 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236195844 0 0) 0x7f7f900c9890 con 0x7f7f90157fc0 2024-09-13T18:58:35.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f87fff700 1 -- 172.21.15.146:0/3887440039 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f7c0035d0 con 0x7f7f90157fc0 2024-09-13T18:58:35.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f95f23700 1 -- 172.21.15.146:0/3887440039 >> v2:172.21.15.154:3300/0 conn(0x7f7f90151840 msgr2=0x7f7f900c1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:35.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f95f23700 1 --2- 172.21.15.146:0/3887440039 >> v2:172.21.15.154:3300/0 conn(0x7f7f90151840 0x7f7f900c1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f95f23700 1 -- 172.21.15.146:0/3887440039 >> v1:172.21.15.146:6789/0 conn(0x7f7f90157fc0 legacy=0x7f7f9006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f95f23700 1 -- 172.21.15.146:0/3887440039 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f900c9510 con 0x7f7f900c8380 2024-09-13T18:58:35.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f95f23700 1 --2- 172.21.15.146:0/3887440039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f900c8380 0x7f7f90161b70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009d90 tx=0x7f7f8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42239a43563db4c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:35.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f87fff700 1 -- 172.21.15.146:0/3887440039 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8800e070 con 0x7f7f900c8380 2024-09-13T18:58:35.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f87fff700 1 -- 172.21.15.146:0/3887440039 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7f8800bb20 con 0x7f7f900c8380 2024-09-13T18:58:35.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.641+0000 7f7f87fff700 1 -- 172.21.15.146:0/3887440039 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88012430 con 0x7f7f900c8380 2024-09-13T18:58:35.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.645+0000 7f7f96724700 1 -- 172.21.15.146:0/3887440039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f900c8380 msgr2=0x7f7f90161b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.645+0000 7f7f96724700 1 --2- 172.21.15.146:0/3887440039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f900c8380 0x7f7f90161b70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009d90 tx=0x7f7f8800b330 comp rx=0 tx=0).stop 2024-09-13T18:58:35.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.645+0000 7f7f96724700 1 -- 172.21.15.146:0/3887440039 shutdown_connections 2024-09-13T18:58:35.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.645+0000 7f7f96724700 1 --2- 172.21.15.146:0/3887440039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f900c8380 0x7f7f90161b70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.645+0000 7f7f96724700 1 --2- 172.21.15.146:0/3887440039 >> v2:172.21.15.154:3300/0 conn(0x7f7f90151840 0x7f7f900c1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.645+0000 7f7f96724700 1 -- 172.21.15.146:0/3887440039 >> 172.21.15.146:0/3887440039 conn(0x7f7f900ccf70 msgr2=0x7f7f900cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:35.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.645+0000 7f7f96724700 1 -- 172.21.15.146:0/3887440039 shutdown_connections 2024-09-13T18:58:35.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.645+0000 7f7f96724700 1 -- 172.21.15.146:0/3887440039 wait complete. 2024-09-13T18:58:35.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.645+0000 7f7f96724700 1 Processor -- start 2024-09-13T18:58:35.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.645+0000 7f7f96724700 1 -- start start 2024-09-13T18:58:35.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.645+0000 7f7f96724700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f900c8380 0x7f7f900cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f96724700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f90151840 0x7f7f9014a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f96724700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f90157fc0 0x7f7f9014abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f96724700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f901ed8d0 con 0x7f7f90157fc0 2024-09-13T18:58:35.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f96724700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9014c7f0 con 0x7f7f90151840 2024-09-13T18:58:35.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f96724700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f901ec890 con 0x7f7f900c8380 2024-09-13T18:58:35.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f95f23700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f90157fc0 0x7f7f9014abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f95f23700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f90157fc0 0x7f7f9014abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45568/0 (socket says 172.21.15.146:45568) 2024-09-13T18:58:35.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f95f23700 1 -- 172.21.15.146:0/4065681067 learned_addr learned my addr 172.21.15.146:0/4065681067 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:35.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f95722700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f900c8380 0x7f7f900cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f94f21700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f90151840 0x7f7f9014a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f95f23700 1 -- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f900c8380 msgr2=0x7f7f900cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f95f23700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f900c8380 0x7f7f900cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f95f23700 1 -- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f90151840 msgr2=0x7f7f9014a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f95f23700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f90151840 0x7f7f9014a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f95f23700 1 -- 172.21.15.146:0/4065681067 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f88009a70 con 0x7f7f90157fc0 2024-09-13T18:58:35.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f94f21700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f90151840 0x7f7f9014a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:58:35.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f95722700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f900c8380 0x7f7f900cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:58:35.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f95f23700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f90157fc0 0x7f7f9014abc0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f7f88014040 tx=0x7f7f88004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:35.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f85ffb700 1 -- 172.21.15.146:0/4065681067 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88012430 con 0x7f7f90157fc0 2024-09-13T18:58:35.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f85ffb700 1 -- 172.21.15.146:0/4065681067 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7f88020070 con 0x7f7f90157fc0 2024-09-13T18:58:35.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f85ffb700 1 -- 172.21.15.146:0/4065681067 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880042a0 con 0x7f7f90157fc0 2024-09-13T18:58:35.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f96724700 1 -- 172.21.15.146:0/4065681067 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f900c7d10 con 0x7f7f90157fc0 2024-09-13T18:58:35.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f96724700 1 -- 172.21.15.146:0/4065681067 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f901ecdf0 con 0x7f7f90157fc0 2024-09-13T18:58:35.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f96724700 1 -- 172.21.15.146:0/4065681067 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f90067a60 con 0x7f7f90157fc0 2024-09-13T18:58:35.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f85ffb700 1 -- 172.21.15.146:0/4065681067 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7f88004400 con 0x7f7f90157fc0 2024-09-13T18:58:35.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.649+0000 7f7f85ffb700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7f74066760 0x7f7f74068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:35.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.653+0000 7f7f85ffb700 1 -- 172.21.15.146:0/4065681067 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f7f88022070 con 0x7f7f90157fc0 2024-09-13T18:58:35.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.653+0000 7f7f95722700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7f74066760 0x7f7f74068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:35.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.653+0000 7f7f95722700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7f74066760 0x7f7f74068c20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009c40 tx=0x7f7f8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:35.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.657+0000 7f7f85ffb700 1 -- 172.21.15.146:0/4065681067 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7f88052d10 con 0x7f7f90157fc0 2024-09-13T18:58:35.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.865+0000 7f7f96724700 1 -- 172.21.15.146:0/4065681067 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f900c7470 con 0x7f7f90157fc0 2024-09-13T18:58:35.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.865+0000 7f7f85ffb700 1 -- 172.21.15.146:0/4065681067 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+6297 (secure 0 0 0) 0x7f7f900c7470 con 0x7f7f90157fc0 2024-09-13T18:58:35.876 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:58:35.876 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":118,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1534137749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1534137749},{"type":"v1","addr":"172.21.15.154:6838","nonce":1534137749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/763096821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":763096821},{"type":"v1","addr":"172.21.15.154:6835","nonce":763096821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6323,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2261933208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6837","nonce":2261933208},{"type":"v1","addr":"172.21.15.154:6839","nonce":2261933208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:26:32.323712+0000","modified":"2024-09-13T18:55:47.963024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6984,"mds_1":7224,"mds_2":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":28,"addr":"172.21.15.146:6835/2934322065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2934322065},{"type":"v1","addr":"172.21.15.146:6835","nonce":2934322065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7098":{"gid":7098,"name":"e","rank":2,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6833/354253522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":354253522},{"type":"v1","addr":"172.21.15.146:6833","nonce":354253522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7224":{"gid":7224,"name":"c","rank":1,"incarnation":113,"state":"up:active","state_seq":25,"addr":"172.21.15.146:6837/3683135798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3683135798},{"type":"v1","addr":"172.21.15.146:6837","nonce":3683135798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:58:35.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.869+0000 7f7f96724700 1 -- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7f74066760 msgr2=0x7f7f74068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.869+0000 7f7f96724700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7f74066760 0x7f7f74068c20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009c40 tx=0x7f7f8c008040 comp rx=0 tx=0).stop 2024-09-13T18:58:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.869+0000 7f7f96724700 1 -- 172.21.15.146:0/4065681067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f90157fc0 msgr2=0x7f7f9014abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.869+0000 7f7f96724700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f90157fc0 0x7f7f9014abc0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f7f88014040 tx=0x7f7f88004af0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.869+0000 7f7f96724700 1 -- 172.21.15.146:0/4065681067 shutdown_connections 2024-09-13T18:58:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.869+0000 7f7f96724700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7f74066760 0x7f7f74068c20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.869+0000 7f7f96724700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f90157fc0 0x7f7f9014abc0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.869+0000 7f7f96724700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f90151840 0x7f7f9014a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.869+0000 7f7f96724700 1 --2- 172.21.15.146:0/4065681067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f900c8380 0x7f7f900cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.869+0000 7f7f96724700 1 -- 172.21.15.146:0/4065681067 >> 172.21.15.146:0/4065681067 conn(0x7f7f900ccf70 msgr2=0x7f7f900c1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.869+0000 7f7f96724700 1 -- 172.21.15.146:0/4065681067 shutdown_connections 2024-09-13T18:58:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.869+0000 7f7f96724700 1 -- 172.21.15.146:0/4065681067 wait complete. 2024-09-13T18:58:35.880 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 118 2024-09-13T18:58:35.890 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:58:36.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.989+0000 7fee8a43a700 1 Processor -- start 2024-09-13T18:58:36.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.989+0000 7fee8a43a700 1 -- start start 2024-09-13T18:58:36.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee8a43a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fee840cfb70 0x7fee840c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee8a43a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee841541b0 con 0x7fee84158090 2024-09-13T18:58:36.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee8a43a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee8415bba0 con 0x7fee840cf790 2024-09-13T18:58:36.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee8a43a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee8415cb00 con 0x7fee840cfb70 2024-09-13T18:58:36.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee7bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fee840cfb70 0x7fee840c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee889d7700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fee84158090 0x7fee8415ae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57840/0 (socket says 172.21.15.146:57840) 2024-09-13T18:58:36.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee889d7700 1 -- 172.21.15.146:0/532248466 learned_addr learned my addr 172.21.15.146:0/532248466 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:58:36.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee837fe700 1 -- 172.21.15.146:0/532248466 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058380071 0 0) 0x7fee841541b0 con 0x7fee84158090 2024-09-13T18:58:36.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee837fe700 1 -- 172.21.15.146:0/532248466 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee700035d0 con 0x7fee84158090 2024-09-13T18:58:36.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee7bfff700 1 -- 172.21.15.146:0/532248466 >> v1:172.21.15.154:6789/0 conn(0x7fee840cf790 legacy=0x7fee84157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee7bfff700 1 -- 172.21.15.146:0/532248466 >> v1:172.21.15.146:6789/0 conn(0x7fee84158090 legacy=0x7fee8415ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee7bfff700 1 -- 172.21.15.146:0/532248466 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8415be80 con 0x7fee840cfb70 2024-09-13T18:58:36.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee7bfff700 1 --2- 172.21.15.146:0/532248466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee840cfb70 0x7fee840c8ba0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009880 tx=0x7fee6c009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60237d45735e2fcd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:36.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee837fe700 1 -- 172.21.15.146:0/532248466 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c011830 con 0x7fee840cfb70 2024-09-13T18:58:36.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee837fe700 1 -- 172.21.15.146:0/532248466 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fee6c011990 con 0x7fee840cfb70 2024-09-13T18:58:36.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee837fe700 1 -- 172.21.15.146:0/532248466 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c011d40 con 0x7fee840cfb70 2024-09-13T18:58:36.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee8a43a700 1 -- 172.21.15.146:0/532248466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee840cfb70 msgr2=0x7fee840c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee8a43a700 1 --2- 172.21.15.146:0/532248466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee840cfb70 0x7fee840c8ba0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009880 tx=0x7fee6c009b90 comp rx=0 tx=0).stop 2024-09-13T18:58:36.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee8a43a700 1 -- 172.21.15.146:0/532248466 shutdown_connections 2024-09-13T18:58:36.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee8a43a700 1 --2- 172.21.15.146:0/532248466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee840cfb70 0x7fee840c8ba0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee8a43a700 1 -- 172.21.15.146:0/532248466 >> 172.21.15.146:0/532248466 conn(0x7fee84071d90 msgr2=0x7fee840721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:36.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee8a43a700 1 -- 172.21.15.146:0/532248466 shutdown_connections 2024-09-13T18:58:36.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee8a43a700 1 -- 172.21.15.146:0/532248466 wait complete. 2024-09-13T18:58:36.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee8a43a700 1 Processor -- start 2024-09-13T18:58:36.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.993+0000 7fee8a43a700 1 -- start start 2024-09-13T18:58:36.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee8a43a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee840cf790 0x7fee840ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee8a43a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee840cfb70 0x7fee8414f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee8a43a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee84158090 0x7fee8414d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee8a43a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee840c28b0 con 0x7fee84158090 2024-09-13T18:58:36.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee8a43a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee84154760 con 0x7fee840cfb70 2024-09-13T18:58:36.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee8a43a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee841544c0 con 0x7fee840cf790 2024-09-13T18:58:36.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee889d7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee84158090 0x7fee8414d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee889d7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee84158090 0x7fee8414d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45574/0 (socket says 172.21.15.146:45574) 2024-09-13T18:58:36.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee889d7700 1 -- 172.21.15.146:0/3545670431 learned_addr learned my addr 172.21.15.146:0/3545670431 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:36.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee83fff700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee840cf790 0x7fee840ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee7bfff700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee840cfb70 0x7fee8414f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee889d7700 1 -- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee840cf790 msgr2=0x7fee840ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee889d7700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee840cf790 0x7fee840ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee889d7700 1 -- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee840cfb70 msgr2=0x7fee8414f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee889d7700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee840cfb70 0x7fee8414f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee889d7700 1 -- 172.21.15.146:0/3545670431 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c0094b0 con 0x7fee84158090 2024-09-13T18:58:36.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee889d7700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee84158090 0x7fee8414d6a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00caa0 tx=0x7fee7c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:36.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee817fa700 1 -- 172.21.15.146:0/3545670431 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c00a7b0 con 0x7fee84158090 2024-09-13T18:58:36.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee817fa700 1 -- 172.21.15.146:0/3545670431 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fee7c00a910 con 0x7fee84158090 2024-09-13T18:58:36.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee817fa700 1 -- 172.21.15.146:0/3545670431 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c00acc0 con 0x7fee84158090 2024-09-13T18:58:36.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee8a43a700 1 -- 172.21.15.146:0/3545670431 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee840cb9b0 con 0x7fee84158090 2024-09-13T18:58:36.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee8a43a700 1 -- 172.21.15.146:0/3545670431 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee840b31a0 con 0x7fee84158090 2024-09-13T18:58:36.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:35.997+0000 7fee817fa700 1 -- 172.21.15.146:0/3545670431 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fee7c01b3b0 con 0x7fee84158090 2024-09-13T18:58:36.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.001+0000 7fee8a43a700 1 -- 172.21.15.146:0/3545670431 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee84073700 con 0x7fee84158090 2024-09-13T18:58:36.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.001+0000 7fee817fa700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fee64066730 0x7fee64068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.001+0000 7fee817fa700 1 -- 172.21.15.146:0/3545670431 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7fee7c0846f0 con 0x7fee84158090 2024-09-13T18:58:36.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.001+0000 7fee83fff700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fee64066730 0x7fee64068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.001+0000 7fee83fff700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fee64066730 0x7fee64068bf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fee74000f60 tx=0x7fee74009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:36.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.005+0000 7fee817fa700 1 -- 172.21.15.146:0/3545670431 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fee7c050fa0 con 0x7fee84158090 2024-09-13T18:58:36.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.169+0000 7fee8a43a700 1 -- 172.21.15.146:0/3545670431 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee84072170 con 0x7fee84158090 2024-09-13T18:58:36.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.173+0000 7fee817fa700 1 -- 172.21.15.146:0/3545670431 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+20825 (secure 0 0 0) 0x7fee7c017090 con 0x7fee84158090 2024-09-13T18:58:36.182 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-13T18:58:36.182 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":43,"fsid":"509e851e-cdbc-4e57-a792-d3d1aeeeba86","created":"2024-09-13T18:26:15.618349+0000","modified":"2024-09-13T18:55:25.944150+0000","last_up_change":"2024-09-13T18:26:25.888175+0000","last_in_change":"2024-09-13T18:26:17.734910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:26:24.867515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:26:30.782269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:26:31.323277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:26:34.393386+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e2f8470a-1ce4-4248-bed6-4636f5d458e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6817","nonce":2811789208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6819","nonce":2811789208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6823","nonce":2811789208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2811789208},{"type":"v1","addr":"172.21.15.146:6821","nonce":2811789208}]},"public_addr":"172.21.15.146:6817/2811789208","cluster_addr":"172.21.15.146:6819/2811789208","heartbeat_back_addr":"172.21.15.146:6823/2811789208","heartbeat_front_addr":"172.21.15.146:6821/2811789208","state":["exists","up"]},{"osd":1,"uuid":"78bcc5f7-e643-4530-9f06-c9669e2db81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6806","nonce":3907005797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6810","nonce":3907005797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6815","nonce":3907005797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3907005797},{"type":"v1","addr":"172.21.15.146:6813","nonce":3907005797}]},"public_addr":"172.21.15.146:6806/3907005797","cluster_addr":"172.21.15.146:6810/3907005797","heartbeat_back_addr":"172.21.15.146:6815/3907005797","heartbeat_front_addr":"172.21.15.146:6813/3907005797","state":["exists","up"]},{"osd":2,"uuid":"ad13f832-2def-402e-b36b-e68088f63502","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6801","nonce":3571336781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6803","nonce":3571336781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6811","nonce":3571336781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3571336781},{"type":"v1","addr":"172.21.15.146:6807","nonce":3571336781}]},"public_addr":"172.21.15.146:6801/3571336781","cluster_addr":"172.21.15.146:6803/3571336781","heartbeat_back_addr":"172.21.15.146:6811/3571336781","heartbeat_front_addr":"172.21.15.146:6807/3571336781","state":["exists","up"]},{"osd":3,"uuid":"905c1019-7640-42d4-8c40-1363a8039b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6825","nonce":3480660705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6827","nonce":3480660705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6831","nonce":3480660705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3480660705},{"type":"v1","addr":"172.21.15.146:6829","nonce":3480660705}]},"public_addr":"172.21.15.146:6825/3480660705","cluster_addr":"172.21.15.146:6827/3480660705","heartbeat_back_addr":"172.21.15.146:6831/3480660705","heartbeat_front_addr":"172.21.15.146:6829/3480660705","state":["exists","up"]},{"osd":4,"uuid":"e1178506-9350-4efa-8e99-27d251a1d966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6809","nonce":2872450676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6811","nonce":2872450676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6815","nonce":2872450676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2872450676},{"type":"v1","addr":"172.21.15.154:6813","nonce":2872450676}]},"public_addr":"172.21.15.154:6809/2872450676","cluster_addr":"172.21.15.154:6811/2872450676","heartbeat_back_addr":"172.21.15.154:6815/2872450676","heartbeat_front_addr":"172.21.15.154:6813/2872450676","state":["exists","up"]},{"osd":5,"uuid":"cef9a2a4-ce47-4ebf-a64d-fb332e0637a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6801","nonce":363322763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6803","nonce":363322763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6807","nonce":363322763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":363322763},{"type":"v1","addr":"172.21.15.154:6805","nonce":363322763}]},"public_addr":"172.21.15.154:6801/363322763","cluster_addr":"172.21.15.154:6803/363322763","heartbeat_back_addr":"172.21.15.154:6807/363322763","heartbeat_front_addr":"172.21.15.154:6805/363322763","state":["exists","up"]},{"osd":6,"uuid":"5aef7701-3a54-4e2d-af07-6fe296078938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6818","nonce":2272517392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6822","nonce":2272517392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6830","nonce":2272517392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2272517392},{"type":"v1","addr":"172.21.15.154:6826","nonce":2272517392}]},"public_addr":"172.21.15.154:6818/2272517392","cluster_addr":"172.21.15.154:6822/2272517392","heartbeat_back_addr":"172.21.15.154:6830/2272517392","heartbeat_front_addr":"172.21.15.154:6826/2272517392","state":["exists","up"]},{"osd":7,"uuid":"0aae359a-7aaf-44be-b0d7-0f89c1b6cd3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6819","nonce":3339245911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6823","nonce":3339245911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6831","nonce":3339245911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":3339245911},{"type":"v1","addr":"172.21.15.154:6827","nonce":3339245911}]},"public_addr":"172.21.15.154:6819/3339245911","cluster_addr":"172.21.15.154:6823/3339245911","heartbeat_back_addr":"172.21.15.154:6831/3339245911","heartbeat_front_addr":"172.21.15.154:6827/3339245911","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.769090+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.535222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.011082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.937430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.939059+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.687211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:23.132338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:26:22.914310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.13","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/2053734262":"2024-09-14T18:55:25.944088+0000","172.21.15.154:6839/3274220802":"2024-09-14T18:36:10.677826+0000","172.21.15.146:6835/294014284":"2024-09-14T18:50:50.876658+0000","172.21.15.154:6837/3274220802":"2024-09-14T18:36:10.677826+0000","172.21.15.146:6836/2169954140":"2024-09-14T18:53:50.924269+0000","172.21.15.146:6832/1498985282":"2024-09-14T18:34:25.647701+0000","172.21.15.146:6835/1642500214":"2024-09-14T18:30:30.598692+0000","172.21.15.146:6836/31041557":"2024-09-14T18:32:55.627334+0000","172.21.15.146:6832/3023779168":"2024-09-14T18:52:35.902095+0000","172.21.15.146:6834/1642500214":"2024-09-14T18:30:30.598692+0000","172.21.15.146:6836/3234109877":"2024-09-14T18:45:55.807881+0000","172.21.15.146:6837/2169954140":"2024-09-14T18:53:50.924269+0000","172.21.15.146:6833/1498985282":"2024-09-14T18:34:25.647701+0000","172.21.15.146:6837/31041557":"2024-09-14T18:32:55.627334+0000","172.21.15.154:6839/776638146":"2024-09-14T18:44:35.790651+0000","172.21.15.154:6839/3978828499":"2024-09-14T18:29:25.578693+0000","172.21.15.146:6832/2835746745":"2024-09-14T18:28:15.564530+0000","172.21.15.146:6834/1652676664":"2024-09-14T18:38:00.699180+0000","172.21.15.154:6837/776638146":"2024-09-14T18:44:35.790651+0000","172.21.15.154:6837/925899756":"2024-09-14T18:48:20.844616+0000","172.21.15.146:6833/2835746745":"2024-09-14T18:28:15.564530+0000","172.21.15.146:6835/1652676664":"2024-09-14T18:38:00.699180+0000","172.21.15.154:6839/2053734262":"2024-09-14T18:55:25.944088+0000","172.21.15.154:6839/925899756":"2024-09-14T18:48:20.844616+0000","172.21.15.154:6837/3978828499":"2024-09-14T18:29:25.578693+0000","172.21.15.146:6836/3926931671":"2024-09-14T18:40:45.735995+0000","172.21.15.146:6837/3926931671":"2024-09-14T18:40:45.735995+0000","172.21.15.146:6833/3901685195":"2024-09-14T18:43:15.768366+0000","172.21.15.146:6837/3234109877":"2024-09-14T18:45:55.807881+0000","172.21.15.146:6834/294014284":"2024-09-14T18:50:50.876658+0000","172.21.15.146:6832/3901685195":"2024-09-14T18:43:15.768366+0000","172.21.15.146:6833/3023779168":"2024-09-14T18:52:35.902095+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:58:36.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.173+0000 7fee8a43a700 1 -- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fee64066730 msgr2=0x7fee64068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.173+0000 7fee8a43a700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fee64066730 0x7fee64068bf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fee74000f60 tx=0x7fee74009040 comp rx=0 tx=0).stop 2024-09-13T18:58:36.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.173+0000 7fee8a43a700 1 -- 172.21.15.146:0/3545670431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee84158090 msgr2=0x7fee8414d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.173+0000 7fee8a43a700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee84158090 0x7fee8414d6a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00caa0 tx=0x7fee7c00cb80 comp rx=0 tx=0).stop 2024-09-13T18:58:36.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.173+0000 7fee8a43a700 1 -- 172.21.15.146:0/3545670431 shutdown_connections 2024-09-13T18:58:36.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.173+0000 7fee8a43a700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7fee64066730 0x7fee64068bf0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.173+0000 7fee8a43a700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee84158090 0x7fee8414d6a0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.173+0000 7fee8a43a700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee840cfb70 0x7fee8414f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.173+0000 7fee8a43a700 1 --2- 172.21.15.146:0/3545670431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee840cf790 0x7fee840ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.173+0000 7fee8a43a700 1 -- 172.21.15.146:0/3545670431 >> 172.21.15.146:0/3545670431 conn(0x7fee84071d90 msgr2=0x7fee84158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:36.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.173+0000 7fee8a43a700 1 -- 172.21.15.146:0/3545670431 shutdown_connections 2024-09-13T18:58:36.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.173+0000 7fee8a43a700 1 -- 172.21.15.146:0/3545670431 wait complete. 2024-09-13T18:58:36.203 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T18:58:36.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.293+0000 7f70b359e700 1 Processor -- start 2024-09-13T18:58:36.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.293+0000 7f70b359e700 1 -- start start 2024-09-13T18:58:36.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70b40c7900 0x7f70b406bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f70b406c490 0x7f70b40c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b359e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b4153b40 con 0x7f70b4153760 2024-09-13T18:58:36.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b415a440 con 0x7f70b406c490 2024-09-13T18:58:36.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b415cd50 con 0x7f70b40c7900 2024-09-13T18:58:36.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b1d9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70b40c7900 0x7f70b406bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b1d9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70b40c7900 0x7f70b406bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:59806/0 (socket says 172.21.15.146:59806) 2024-09-13T18:58:36.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b2d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f70b406c490 0x7f70b40c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b1d9b700 1 -- 172.21.15.146:0/1424930197 learned_addr learned my addr 172.21.15.146:0/1424930197 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:36.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b1d9b700 1 -- 172.21.15.146:0/1424930197 >> v2:172.21.15.154:3300/0 conn(0x7f70b406c490 msgr2=0x7f70b40c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:36.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b1d9b700 1 --2- 172.21.15.146:0/1424930197 >> v2:172.21.15.154:3300/0 conn(0x7f70b406c490 0x7f70b40c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b1d9b700 1 -- 172.21.15.146:0/1424930197 >> v1:172.21.15.154:6789/0 conn(0x7f70b4153760 legacy=0x7f70b406cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b1d9b700 1 -- 172.21.15.146:0/1424930197 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b415d030 con 0x7f70b40c7900 2024-09-13T18:58:36.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b2d9d700 1 --2- 172.21.15.146:0/1424930197 >> v2:172.21.15.154:3300/0 conn(0x7f70b406c490 0x7f70b40c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:58:36.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b1d9b700 1 --2- 172.21.15.146:0/1424930197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70b40c7900 0x7f70b406bf50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009910 tx=0x7f70a8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc398a5d5a73ac62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:36.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b159a700 1 -- 172.21.15.146:0/1424930197 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8011830 con 0x7f70b40c7900 2024-09-13T18:58:36.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b159a700 1 -- 172.21.15.146:0/1424930197 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f70a8011990 con 0x7f70b40c7900 2024-09-13T18:58:36.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b159a700 1 -- 172.21.15.146:0/1424930197 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8011d60 con 0x7f70b40c7900 2024-09-13T18:58:36.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b359e700 1 -- 172.21.15.146:0/1424930197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70b40c7900 msgr2=0x7f70b406bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b359e700 1 --2- 172.21.15.146:0/1424930197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70b40c7900 0x7f70b406bf50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009910 tx=0x7f70a8009c20 comp rx=0 tx=0).stop 2024-09-13T18:58:36.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b359e700 1 -- 172.21.15.146:0/1424930197 shutdown_connections 2024-09-13T18:58:36.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b359e700 1 --2- 172.21.15.146:0/1424930197 >> v2:172.21.15.154:3300/0 conn(0x7f70b406c490 0x7f70b40c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b359e700 1 --2- 172.21.15.146:0/1424930197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70b40c7900 0x7f70b406bf50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b359e700 1 -- 172.21.15.146:0/1424930197 >> 172.21.15.146:0/1424930197 conn(0x7f70b40ccf00 msgr2=0x7f70b40cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:36.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b359e700 1 -- 172.21.15.146:0/1424930197 shutdown_connections 2024-09-13T18:58:36.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.297+0000 7f70b359e700 1 -- 172.21.15.146:0/1424930197 wait complete. 2024-09-13T18:58:36.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b359e700 1 Processor -- start 2024-09-13T18:58:36.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b359e700 1 -- start start 2024-09-13T18:58:36.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70b406c490 0x7f70b415b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b40c7900 0x7f70b40c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70b4153760 0x7f70b4071a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40726e0 con 0x7f70b40c7900 2024-09-13T18:58:36.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b41e2df0 con 0x7f70b406c490 2024-09-13T18:58:36.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b41e30d0 con 0x7f70b4153760 2024-09-13T18:58:36.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b40c7900 0x7f70b40c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b40c7900 0x7f70b40c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45576/0 (socket says 172.21.15.146:45576) 2024-09-13T18:58:36.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b1d9b700 1 -- 172.21.15.146:0/2358810056 learned_addr learned my addr 172.21.15.146:0/2358810056 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:36.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b259c700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70b406c490 0x7f70b415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b2d9d700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70b4153760 0x7f70b4071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b1d9b700 1 -- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70b4153760 msgr2=0x7f70b4071a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b1d9b700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70b4153760 0x7f70b4071a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b1d9b700 1 -- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70b406c490 msgr2=0x7f70b415b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b1d9b700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70b406c490 0x7f70b415b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b1d9b700 1 -- 172.21.15.146:0/2358810056 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac009a70 con 0x7f70b40c7900 2024-09-13T18:58:36.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b1d9b700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b40c7900 0x7f70b40c5cd0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f70a8000c00 tx=0x7f70a8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:36.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70a2ffd700 1 -- 172.21.15.146:0/2358810056 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a80045d0 con 0x7f70b40c7900 2024-09-13T18:58:36.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70a2ffd700 1 -- 172.21.15.146:0/2358810056 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f70a80036c0 con 0x7f70b40c7900 2024-09-13T18:58:36.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70a2ffd700 1 -- 172.21.15.146:0/2358810056 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8003a90 con 0x7f70b40c7900 2024-09-13T18:58:36.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b359e700 1 -- 172.21.15.146:0/2358810056 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a8009570 con 0x7f70b40c7900 2024-09-13T18:58:36.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.301+0000 7f70b359e700 1 -- 172.21.15.146:0/2358810056 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b415a660 con 0x7f70b40c7900 2024-09-13T18:58:36.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.305+0000 7f70b359e700 1 -- 172.21.15.146:0/2358810056 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b4067a60 con 0x7f70b40c7900 2024-09-13T18:58:36.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.305+0000 7f70a2ffd700 1 -- 172.21.15.146:0/2358810056 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f70a8003bf0 con 0x7f70b40c7900 2024-09-13T18:58:36.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.305+0000 7f70a2ffd700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7094066730 0x7f7094068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.305+0000 7f70a2ffd700 1 -- 172.21.15.146:0/2358810056 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8039+0+0 (secure 0 0 0) 0x7f70a8021030 con 0x7f70b40c7900 2024-09-13T18:58:36.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.305+0000 7f70b259c700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7094066730 0x7f7094068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.305+0000 7f70b259c700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7094066730 0x7f7094068bf0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f70a4000f60 tx=0x7f70a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:36.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.313+0000 7f70a2ffd700 1 -- 172.21.15.146:0/2358810056 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f70a8052fd0 con 0x7f70b40c7900 2024-09-13T18:58:36.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.513+0000 7f70b359e700 1 -- 172.21.15.146:0/2358810056 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f70b40cdd00 con 0x7f70b40c7900 2024-09-13T18:58:36.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.533+0000 7f70a2ffd700 1 -- 172.21.15.146:0/2358810056 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v119) v1 ==== 161+0+0 (secure 0 0 0) 0x7f70a802b070 con 0x7f70b40c7900 2024-09-13T18:58:36.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.537+0000 7f70b359e700 1 -- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7094066730 msgr2=0x7f7094068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.537+0000 7f70b359e700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7094066730 0x7f7094068bf0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f70a4000f60 tx=0x7f70a4009040 comp rx=0 tx=0).stop 2024-09-13T18:58:36.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.537+0000 7f70b359e700 1 -- 172.21.15.146:0/2358810056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b40c7900 msgr2=0x7f70b40c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.537+0000 7f70b359e700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b40c7900 0x7f70b40c5cd0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f70a8000c00 tx=0x7f70a8004770 comp rx=0 tx=0).stop 2024-09-13T18:58:36.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.537+0000 7f70b359e700 1 -- 172.21.15.146:0/2358810056 shutdown_connections 2024-09-13T18:58:36.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.537+0000 7f70b359e700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f7094066730 0x7f7094068bf0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.537+0000 7f70b359e700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70b4153760 0x7f70b4071a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.537+0000 7f70b359e700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b40c7900 0x7f70b40c5cd0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.537+0000 7f70b359e700 1 --2- 172.21.15.146:0/2358810056 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70b406c490 0x7f70b415b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.537+0000 7f70b359e700 1 -- 172.21.15.146:0/2358810056 >> 172.21.15.146:0/2358810056 conn(0x7f70b40ccf00 msgr2=0x7f70b4150f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:36.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.537+0000 7f70b359e700 1 -- 172.21.15.146:0/2358810056 shutdown_connections 2024-09-13T18:58:36.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.537+0000 7f70b359e700 1 -- 172.21.15.146:0/2358810056 wait complete. 2024-09-13T18:58:36.556 INFO:teuthology.orchestra.run.smithi146.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T18:58:36.568 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T18:58:36.639 INFO:tasks.ceph.mds.e.smithi146.stdout:starting mds.e at 2024-09-13T18:58:36.640 INFO:tasks.ceph.mds.c.smithi146.stdout:starting mds.c at 2024-09-13T18:58:36.641 INFO:tasks.ceph.mds.a.smithi146.stdout:starting mds.a at 2024-09-13T18:58:36.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febdcdad700 1 Processor -- start 2024-09-13T18:58:36.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febdcdad700 1 -- start start 2024-09-13T18:58:36.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febdcdad700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7febd8151ae0 0x7febd80c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febdcdad700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd815b7d0 0x7febd80c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febdcdad700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd80c5d00 con 0x7febd806c7a0 2024-09-13T18:58:36.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febdcdad700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd8158170 con 0x7febd8151ae0 2024-09-13T18:58:36.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febdcdad700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd80c5fa0 con 0x7febd815b7d0 2024-09-13T18:58:36.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febd659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7febd8151ae0 0x7febd80c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febd6d9d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd815b7d0 0x7febd80c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febd5d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7febd806c7a0 0x7febd815b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.146:58304/0 (socket says 172.21.15.146:58304) 2024-09-13T18:58:36.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febd5d9b700 1 -- 172.21.15.146:0/1879165290 learned_addr learned my addr 172.21.15.146:0/1879165290 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:58:36.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febd659c700 1 -- 172.21.15.146:0/1879165290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd815b7d0 msgr2=0x7febd80c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:36.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febd659c700 1 --2- 172.21.15.146:0/1879165290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd815b7d0 0x7febd80c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febd659c700 1 -- 172.21.15.146:0/1879165290 >> v1:172.21.15.154:6789/0 conn(0x7febd806c7a0 legacy=0x7febd815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febd659c700 1 -- 172.21.15.146:0/1879165290 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd815aba0 con 0x7febd8151ae0 2024-09-13T18:58:36.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febd659c700 1 --2- 172.21.15.146:0/1879165290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd8151ae0 0x7febd80c7b60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7febd0009d90 tx=0x7febd000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a6ed2fa9a002ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:36.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febd559a700 1 -- 172.21.15.146:0/1879165290 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd000e070 con 0x7febd8151ae0 2024-09-13T18:58:36.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febd559a700 1 -- 172.21.15.146:0/1879165290 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7febd000bb20 con 0x7febd8151ae0 2024-09-13T18:58:36.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.661+0000 7febd559a700 1 -- 172.21.15.146:0/1879165290 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0012430 con 0x7febd8151ae0 2024-09-13T18:58:36.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.665+0000 7febdcdad700 1 -- 172.21.15.146:0/1879165290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd8151ae0 msgr2=0x7febd80c7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.665+0000 7febdcdad700 1 --2- 172.21.15.146:0/1879165290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd8151ae0 0x7febd80c7b60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7febd0009d90 tx=0x7febd000b330 comp rx=0 tx=0).stop 2024-09-13T18:58:36.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.665+0000 7febdcdad700 1 -- 172.21.15.146:0/1879165290 shutdown_connections 2024-09-13T18:58:36.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.665+0000 7febdcdad700 1 --2- 172.21.15.146:0/1879165290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd815b7d0 0x7febd80c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.665+0000 7febdcdad700 1 --2- 172.21.15.146:0/1879165290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd8151ae0 0x7febd80c7b60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.665+0000 7febdcdad700 1 -- 172.21.15.146:0/1879165290 >> 172.21.15.146:0/1879165290 conn(0x7febd806f5c0 msgr2=0x7febd806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:36.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.665+0000 7febdcdad700 1 -- 172.21.15.146:0/1879165290 shutdown_connections 2024-09-13T18:58:36.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.665+0000 7febdcdad700 1 -- 172.21.15.146:0/1879165290 wait complete. 2024-09-13T18:58:36.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.665+0000 7febdcdad700 1 Processor -- start 2024-09-13T18:58:36.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.665+0000 7febdcdad700 1 -- start start 2024-09-13T18:58:36.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febdcdad700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd806c7a0 0x7febd80ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febdcdad700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd8151ae0 0x7febd81e0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febdcdad700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd815b7d0 0x7febd81e41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febdcdad700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd81efa80 con 0x7febd806c7a0 2024-09-13T18:58:36.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febdcdad700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd81ef1d0 con 0x7febd815b7d0 2024-09-13T18:58:36.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febdcdad700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd80cae40 con 0x7febd8151ae0 2024-09-13T18:58:36.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febd659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd806c7a0 0x7febd80ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febd659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd806c7a0 0x7febd80ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45638/0 (socket says 172.21.15.146:45638) 2024-09-13T18:58:36.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febd659c700 1 -- 172.21.15.146:0/1878062756 learned_addr learned my addr 172.21.15.146:0/1878062756 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:36.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febd6d9d700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd815b7d0 0x7febd81e41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febd5d9b700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd8151ae0 0x7febd81e0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febd659c700 1 -- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd8151ae0 msgr2=0x7febd81e0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febd659c700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd8151ae0 0x7febd81e0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febd659c700 1 -- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd815b7d0 msgr2=0x7febd81e41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:36.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febd659c700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd815b7d0 0x7febd81e41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:36.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febd659c700 1 -- 172.21.15.146:0/1878062756 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd0009a70 con 0x7febd806c7a0 2024-09-13T18:58:36.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febd5d9b700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd8151ae0 0x7febd81e0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:58:36.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febd659c700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd806c7a0 0x7febd80ca900 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7febd80c8820 tx=0x7febd0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:36.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febceffd700 1 -- 172.21.15.146:0/1878062756 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0012430 con 0x7febd806c7a0 2024-09-13T18:58:36.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febceffd700 1 -- 172.21.15.146:0/1878062756 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7febd0020070 con 0x7febd806c7a0 2024-09-13T18:58:36.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febdcdad700 1 -- 172.21.15.146:0/1878062756 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd81e1010 con 0x7febd806c7a0 2024-09-13T18:58:36.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febceffd700 1 -- 172.21.15.146:0/1878062756 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd00042a0 con 0x7febd806c7a0 2024-09-13T18:58:36.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febdcdad700 1 -- 172.21.15.146:0/1878062756 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd81f34d0 con 0x7febd806c7a0 2024-09-13T18:58:36.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.669+0000 7febdcdad700 1 -- 172.21.15.146:0/1878062756 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd80ade70 con 0x7febd806c7a0 2024-09-13T18:58:36.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.673+0000 7febceffd700 1 -- 172.21.15.146:0/1878062756 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7febd0004400 con 0x7febd806c7a0 2024-09-13T18:58:36.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.673+0000 7febceffd700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7febb8066730 0x7febb8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:36.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.673+0000 7febceffd700 1 -- 172.21.15.146:0/1878062756 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 8297+0+0 (secure 0 0 0) 0x7febd0022070 con 0x7febd806c7a0 2024-09-13T18:58:36.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.673+0000 7febd5d9b700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7febb8066730 0x7febb8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:36.682 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.673+0000 7febd5d9b700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7febb8066730 0x7febb8068bf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7febc8007fd0 tx=0x7febc8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:36.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.677+0000 7febceffd700 1 -- 172.21.15.146:0/1878062756 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7febd00535d0 con 0x7febd806c7a0 2024-09-13T18:58:36.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:36.889+0000 7febdcdad700 1 -- 172.21.15.146:0/1878062756 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7febd80c72d0 con 0x7febd806c7a0 2024-09-13T18:58:37.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.537+0000 7febceffd700 1 -- 172.21.15.146:0/1878062756 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v120) v1 ==== 104+0+0 (secure 0 0 0) 0x7febd002c070 con 0x7febd806c7a0 2024-09-13T18:58:37.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.541+0000 7febdcdad700 1 -- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7febb8066730 msgr2=0x7febb8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:37.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.541+0000 7febdcdad700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7febb8066730 0x7febb8068bf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7febc8007fd0 tx=0x7febc8009040 comp rx=0 tx=0).stop 2024-09-13T18:58:37.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.541+0000 7febdcdad700 1 -- 172.21.15.146:0/1878062756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd806c7a0 msgr2=0x7febd80ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:37.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.541+0000 7febdcdad700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd806c7a0 0x7febd80ca900 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7febd80c8820 tx=0x7febd0004af0 comp rx=0 tx=0).stop 2024-09-13T18:58:37.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.541+0000 7febdcdad700 1 -- 172.21.15.146:0/1878062756 shutdown_connections 2024-09-13T18:58:37.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.541+0000 7febdcdad700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7febb8066730 0x7febb8068bf0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:37.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.541+0000 7febdcdad700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd815b7d0 0x7febd81e41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:37.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.541+0000 7febdcdad700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd8151ae0 0x7febd81e0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:37.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.541+0000 7febdcdad700 1 --2- 172.21.15.146:0/1878062756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febd806c7a0 0x7febd80ca900 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:37.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.541+0000 7febdcdad700 1 -- 172.21.15.146:0/1878062756 >> 172.21.15.146:0/1878062756 conn(0x7febd806f5c0 msgr2=0x7febd80c7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:37.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.541+0000 7febdcdad700 1 -- 172.21.15.146:0/1878062756 shutdown_connections 2024-09-13T18:58:37.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.541+0000 7febdcdad700 1 -- 172.21.15.146:0/1878062756 wait complete. 2024-09-13T18:58:37.565 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T18:58:37.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.665+0000 7f5cdcee3700 1 Processor -- start 2024-09-13T18:58:37.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.665+0000 7f5cdcee3700 1 -- start start 2024-09-13T18:58:37.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.665+0000 7f5cdcee3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5cd8151560 0x7f5cd815afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.665+0000 7f5cdcee3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd815b510 0x7f5cd80c7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.665+0000 7f5cdcee3700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd815a860 con 0x7f5cd80b2e10 2024-09-13T18:58:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.665+0000 7f5cdcee3700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd80c3960 con 0x7f5cd8151560 2024-09-13T18:58:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.665+0000 7f5cdcee3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd81572f0 con 0x7f5cd815b510 2024-09-13T18:58:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cd6ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd815b510 0x7f5cd80c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cd6ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd815b510 0x7f5cd80c7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.146:59926/0 (socket says 172.21.15.146:59926) 2024-09-13T18:58:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cd6ffd700 1 -- 172.21.15.146:0/1595498627 learned_addr learned my addr 172.21.15.146:0/1595498627 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cd77fe700 1 --2- 172.21.15.146:0/1595498627 >> v2:172.21.15.154:3300/0 conn(0x7f5cd8151560 0x7f5cd815afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cd6ffd700 1 -- 172.21.15.146:0/1595498627 >> v2:172.21.15.154:3300/0 conn(0x7f5cd8151560 msgr2=0x7f5cd815afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cd6ffd700 1 --2- 172.21.15.146:0/1595498627 >> v2:172.21.15.154:3300/0 conn(0x7f5cd8151560 0x7f5cd815afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cd6ffd700 1 -- 172.21.15.146:0/1595498627 >> v1:172.21.15.154:6789/0 conn(0x7f5cd80b2e10 legacy=0x7f5cd80c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:37.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cd6ffd700 1 -- 172.21.15.146:0/1595498627 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd80c1f20 con 0x7f5cd815b510 2024-09-13T18:58:37.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cd67fc700 1 -- 172.21.15.146:0/1595498627 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285509658 0 0) 0x7f5cd815a860 con 0x7f5cd80b2e10 2024-09-13T18:58:37.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cd6ffd700 1 --2- 172.21.15.146:0/1595498627 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd815b510 0x7f5cd80c7900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009940 tx=0x7f5ccc009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50ae97cb2a261b09 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:37.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cd67fc700 1 -- 172.21.15.146:0/1595498627 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc011830 con 0x7f5cd815b510 2024-09-13T18:58:37.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cd67fc700 1 -- 172.21.15.146:0/1595498627 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5ccc011990 con 0x7f5cd815b510 2024-09-13T18:58:37.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cd67fc700 1 -- 172.21.15.146:0/1595498627 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc011d40 con 0x7f5cd815b510 2024-09-13T18:58:37.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 -- 172.21.15.146:0/1595498627 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd815b510 msgr2=0x7f5cd80c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:37.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 --2- 172.21.15.146:0/1595498627 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd815b510 0x7f5cd80c7900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009940 tx=0x7f5ccc009c50 comp rx=0 tx=0).stop 2024-09-13T18:58:37.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 -- 172.21.15.146:0/1595498627 shutdown_connections 2024-09-13T18:58:37.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 --2- 172.21.15.146:0/1595498627 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd815b510 0x7f5cd80c7900 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:37.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 --2- 172.21.15.146:0/1595498627 >> v2:172.21.15.154:3300/0 conn(0x7f5cd8151560 0x7f5cd815afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 -- 172.21.15.146:0/1595498627 >> 172.21.15.146:0/1595498627 conn(0x7f5cd80ccf00 msgr2=0x7f5cd80cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 -- 172.21.15.146:0/1595498627 shutdown_connections 2024-09-13T18:58:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 -- 172.21.15.146:0/1595498627 wait complete. 2024-09-13T18:58:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 Processor -- start 2024-09-13T18:58:37.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 -- start start 2024-09-13T18:58:37.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd80b2e10 0x7f5cd814b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:37.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd8151560 0x7f5cd8071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:37.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd815b510 0x7f5cd8071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd8148470 con 0x7f5cd80b2e10 2024-09-13T18:58:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd8156eb0 con 0x7f5cd8151560 2024-09-13T18:58:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.669+0000 7f5cdcee3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd80cb540 con 0x7f5cd815b510 2024-09-13T18:58:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cd77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd80b2e10 0x7f5cd814b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cd77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd80b2e10 0x7f5cd814b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45644/0 (socket says 172.21.15.146:45644) 2024-09-13T18:58:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cd77fe700 1 -- 172.21.15.146:0/3350106537 learned_addr learned my addr 172.21.15.146:0/3350106537 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cd6ffd700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd8151560 0x7f5cd8071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cd7fff700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd815b510 0x7f5cd8071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cd77fe700 1 -- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd815b510 msgr2=0x7f5cd8071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cd77fe700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd815b510 0x7f5cd8071dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cd77fe700 1 -- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd8151560 msgr2=0x7f5cd8071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:37.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cd77fe700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd8151560 0x7f5cd8071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:37.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cd77fe700 1 -- 172.21.15.146:0/3350106537 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc009570 con 0x7f5cd80b2e10 2024-09-13T18:58:37.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cd77fe700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd80b2e10 0x7f5cd814b020 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800ea00 tx=0x7f5cc800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:37.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cbffff700 1 -- 172.21.15.146:0/3350106537 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8009720 con 0x7f5cd80b2e10 2024-09-13T18:58:37.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cdcee3700 1 -- 172.21.15.146:0/3350106537 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd8156c10 con 0x7f5cd80b2e10 2024-09-13T18:58:37.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cdcee3700 1 -- 172.21.15.146:0/3350106537 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd80caf80 con 0x7f5cd80b2e10 2024-09-13T18:58:37.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cbffff700 1 -- 172.21.15.146:0/3350106537 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5cc800eea0 con 0x7f5cd80b2e10 2024-09-13T18:58:37.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cbffff700 1 -- 172.21.15.146:0/3350106537 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc800ca70 con 0x7f5cd80b2e10 2024-09-13T18:58:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cdcee3700 1 -- 172.21.15.146:0/3350106537 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd8067a60 con 0x7f5cd80b2e10 2024-09-13T18:58:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.673+0000 7f5cbffff700 1 -- 172.21.15.146:0/3350106537 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5cc800cbd0 con 0x7f5cd80b2e10 2024-09-13T18:58:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.677+0000 7f5cbffff700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cc4066730 0x7f5cc4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.677+0000 7f5cbffff700 1 -- 172.21.15.146:0/3350106537 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 8297+0+0 (secure 0 0 0) 0x7f5cc8085000 con 0x7f5cd80b2e10 2024-09-13T18:58:37.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.677+0000 7f5cd6ffd700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cc4066730 0x7f5cc4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:37.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.677+0000 7f5cd6ffd700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cc4066730 0x7f5cc4068bf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009940 tx=0x7f5ccc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:37.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.685+0000 7f5cbffff700 1 -- 172.21.15.146:0/3350106537 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5cc8051880 con 0x7f5cd80b2e10 2024-09-13T18:58:37.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:37.853+0000 7f5cdcee3700 1 -- 172.21.15.146:0/3350106537 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5cd806b950 con 0x7f5cd80b2e10 2024-09-13T18:58:38.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.545+0000 7f5cbffff700 1 -- 172.21.15.146:0/3350106537 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5cc8054ea0 con 0x7f5cd80b2e10 2024-09-13T18:58:38.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.553+0000 7f5cdcee3700 1 -- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cc4066730 msgr2=0x7f5cc4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:38.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.553+0000 7f5cdcee3700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cc4066730 0x7f5cc4068bf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009940 tx=0x7f5ccc008030 comp rx=0 tx=0).stop 2024-09-13T18:58:38.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.553+0000 7f5cdcee3700 1 -- 172.21.15.146:0/3350106537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd80b2e10 msgr2=0x7f5cd814b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:38.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.553+0000 7f5cdcee3700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd80b2e10 0x7f5cd814b020 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800ea00 tx=0x7f5cc800ed10 comp rx=0 tx=0).stop 2024-09-13T18:58:38.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.553+0000 7f5cdcee3700 1 -- 172.21.15.146:0/3350106537 shutdown_connections 2024-09-13T18:58:38.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.553+0000 7f5cdcee3700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f5cc4066730 0x7f5cc4068bf0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:38.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.553+0000 7f5cdcee3700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cd815b510 0x7f5cd8071dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:38.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.553+0000 7f5cdcee3700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd8151560 0x7f5cd8071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:38.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.553+0000 7f5cdcee3700 1 --2- 172.21.15.146:0/3350106537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cd80b2e10 0x7f5cd814b020 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:38.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.553+0000 7f5cdcee3700 1 -- 172.21.15.146:0/3350106537 >> 172.21.15.146:0/3350106537 conn(0x7f5cd80ccf00 msgr2=0x7f5cd81558a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:38.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.553+0000 7f5cdcee3700 1 -- 172.21.15.146:0/3350106537 shutdown_connections 2024-09-13T18:58:38.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.553+0000 7f5cdcee3700 1 -- 172.21.15.146:0/3350106537 wait complete. 2024-09-13T18:58:38.565 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_metadata' removed 2024-09-13T18:58:38.580 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T18:58:38.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.677+0000 7f81fffff700 1 Processor -- start 2024-09-13T18:58:38.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.677+0000 7f81fffff700 1 -- start start 2024-09-13T18:58:38.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82000c11f0 0x7f82000c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:38.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f82000c15d0 0x7f8200159a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:38.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82000c2e40 0x7f8200159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:38.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8200159630 con 0x7f82000c15d0 2024-09-13T18:58:38.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820006e480 con 0x7f82000c2e40 2024-09-13T18:58:38.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820006e720 con 0x7f82000c11f0 2024-09-13T18:58:38.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fe7fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f82000c15d0 0x7f8200159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:38.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fe7fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f82000c15d0 0x7f8200159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45656/0 (socket says 172.21.15.146:45656) 2024-09-13T18:58:38.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fe7fc700 1 -- 172.21.15.146:0/4144344667 learned_addr learned my addr 172.21.15.146:0/4144344667 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:38.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81ff7fe700 1 --2- 172.21.15.146:0/4144344667 >> v2:172.21.15.154:3300/0 conn(0x7f82000c2e40 0x7f8200159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:38.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81feffd700 1 --2- 172.21.15.146:0/4144344667 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82000c11f0 0x7f82000c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:38.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fe7fc700 1 -- 172.21.15.146:0/4144344667 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82000c11f0 msgr2=0x7f82000c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:38.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fe7fc700 1 --2- 172.21.15.146:0/4144344667 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82000c11f0 0x7f82000c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:38.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fe7fc700 1 -- 172.21.15.146:0/4144344667 >> v2:172.21.15.154:3300/0 conn(0x7f82000c2e40 msgr2=0x7f8200159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:38.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fe7fc700 1 --2- 172.21.15.146:0/4144344667 >> v2:172.21.15.154:3300/0 conn(0x7f82000c2e40 0x7f8200159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:38.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fe7fc700 1 -- 172.21.15.146:0/4144344667 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820006ea00 con 0x7f82000c15d0 2024-09-13T18:58:38.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fe7fc700 1 --2- 172.21.15.146:0/4144344667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82000c15d0 0x7f8200159a80 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f8200071ca0 tx=0x7f81f4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214087aea2162959 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:38.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fdffb700 1 -- 172.21.15.146:0/4144344667 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f400b8a0 con 0x7f82000c15d0 2024-09-13T18:58:38.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fdffb700 1 -- 172.21.15.146:0/4144344667 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f81f400ba00 con 0x7f82000c15d0 2024-09-13T18:58:38.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fdffb700 1 -- 172.21.15.146:0/4144344667 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f400bdb0 con 0x7f82000c15d0 2024-09-13T18:58:38.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 -- 172.21.15.146:0/4144344667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82000c15d0 msgr2=0x7f8200159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:38.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 --2- 172.21.15.146:0/4144344667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82000c15d0 0x7f8200159a80 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f8200071ca0 tx=0x7f81f4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:58:38.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 -- 172.21.15.146:0/4144344667 shutdown_connections 2024-09-13T18:58:38.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 --2- 172.21.15.146:0/4144344667 >> v2:172.21.15.154:3300/0 conn(0x7f82000c2e40 0x7f8200159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:38.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 --2- 172.21.15.146:0/4144344667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82000c15d0 0x7f8200159a80 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:38.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 --2- 172.21.15.146:0/4144344667 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82000c11f0 0x7f82000c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:38.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 -- 172.21.15.146:0/4144344667 >> 172.21.15.146:0/4144344667 conn(0x7f8200071ed0 msgr2=0x7f82000722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:38.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 -- 172.21.15.146:0/4144344667 shutdown_connections 2024-09-13T18:58:38.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.681+0000 7f81fffff700 1 -- 172.21.15.146:0/4144344667 wait complete. 2024-09-13T18:58:38.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81fffff700 1 Processor -- start 2024-09-13T18:58:38.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81fffff700 1 -- start start 2024-09-13T18:58:38.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81fffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82000c11f0 0x7f82000c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81fffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82000c15d0 0x7f82001e6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81fffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82000c2e40 0x7f82001f2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81fffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82000c4570 con 0x7f82000c2e40 2024-09-13T18:58:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81fffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820006f5c0 con 0x7f82000c11f0 2024-09-13T18:58:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81fffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82000c3360 con 0x7f82000c15d0 2024-09-13T18:58:38.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81ff7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82000c2e40 0x7f82001f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:38.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81ff7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82000c2e40 0x7f82001f2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45658/0 (socket says 172.21.15.146:45658) 2024-09-13T18:58:38.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81ff7fe700 1 -- 172.21.15.146:0/2424480539 learned_addr learned my addr 172.21.15.146:0/2424480539 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:38.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81feffd700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82000c11f0 0x7f82000c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:38.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81fe7fc700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82000c15d0 0x7f82001e6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:38.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81ff7fe700 1 -- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82000c15d0 msgr2=0x7f82001e6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:38.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81ff7fe700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82000c15d0 0x7f82001e6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:38.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81ff7fe700 1 -- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82000c11f0 msgr2=0x7f82000c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81ff7fe700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82000c11f0 0x7f82000c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81ff7fe700 1 -- 172.21.15.146:0/2424480539 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f4009570 con 0x7f82000c2e40 2024-09-13T18:58:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81ff7fe700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82000c2e40 0x7f82001f2640 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f81f8007cf0 tx=0x7f81f800e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81e7fff700 1 -- 172.21.15.146:0/2424480539 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f800eb70 con 0x7f82000c2e40 2024-09-13T18:58:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81e7fff700 1 -- 172.21.15.146:0/2424480539 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f81f800ecf0 con 0x7f82000c2e40 2024-09-13T18:58:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81fffff700 1 -- 172.21.15.146:0/2424480539 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820006f810 con 0x7f82000c2e40 2024-09-13T18:58:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81e7fff700 1 -- 172.21.15.146:0/2424480539 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8014680 con 0x7f82000c2e40 2024-09-13T18:58:38.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.685+0000 7f81fffff700 1 -- 172.21.15.146:0/2424480539 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82000c35b0 con 0x7f82000c2e40 2024-09-13T18:58:38.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.689+0000 7f81fffff700 1 -- 172.21.15.146:0/2424480539 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82000c7ae0 con 0x7f82000c2e40 2024-09-13T18:58:38.704 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.689+0000 7f81e7fff700 1 -- 172.21.15.146:0/2424480539 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f81f80147e0 con 0x7f82000c2e40 2024-09-13T18:58:38.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.689+0000 7f81e7fff700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f81ec066730 0x7f81ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:38.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.689+0000 7f81e7fff700 1 -- 172.21.15.146:0/2424480539 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 7859+0+0 (secure 0 0 0) 0x7f81f8085060 con 0x7f82000c2e40 2024-09-13T18:58:38.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.689+0000 7f81feffd700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f81ec066730 0x7f81ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:38.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.689+0000 7f81feffd700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f81ec066730 0x7f81ec068bf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009970 tx=0x7f81f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:38.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.697+0000 7f81e7fff700 1 -- 172.21.15.146:0/2424480539 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f81f8051a40 con 0x7f82000c2e40 2024-09-13T18:58:38.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:38.865+0000 7f81fffff700 1 -- 172.21.15.146:0/2424480539 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f82000b0e00 con 0x7f82000c2e40 2024-09-13T18:58:39.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.553+0000 7f81e7fff700 1 -- 172.21.15.146:0/2424480539 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) v1 ==== 169+0+0 (secure 0 0 0) 0x7f81f800ee60 con 0x7f82000c2e40 2024-09-13T18:58:39.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.561+0000 7f81fffff700 1 -- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f81ec066730 msgr2=0x7f81ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:39.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.561+0000 7f81fffff700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f81ec066730 0x7f81ec068bf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009970 tx=0x7f81f0008040 comp rx=0 tx=0).stop 2024-09-13T18:58:39.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.561+0000 7f81fffff700 1 -- 172.21.15.146:0/2424480539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82000c2e40 msgr2=0x7f82001f2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:39.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.561+0000 7f81fffff700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82000c2e40 0x7f82001f2640 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f81f8007cf0 tx=0x7f81f800e0e0 comp rx=0 tx=0).stop 2024-09-13T18:58:39.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.561+0000 7f81fffff700 1 -- 172.21.15.146:0/2424480539 shutdown_connections 2024-09-13T18:58:39.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.561+0000 7f81fffff700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f81ec066730 0x7f81ec068bf0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:39.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.561+0000 7f81fffff700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82000c2e40 0x7f82001f2640 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:39.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.561+0000 7f81fffff700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82000c15d0 0x7f82001e6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:39.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.561+0000 7f81fffff700 1 --2- 172.21.15.146:0/2424480539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82000c11f0 0x7f82000c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:39.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.561+0000 7f81fffff700 1 -- 172.21.15.146:0/2424480539 >> 172.21.15.146:0/2424480539 conn(0x7f8200071ed0 msgr2=0x7f8200159940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:39.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.561+0000 7f81fffff700 1 -- 172.21.15.146:0/2424480539 shutdown_connections 2024-09-13T18:58:39.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.561+0000 7f81fffff700 1 -- 172.21.15.146:0/2424480539 wait complete. 2024-09-13T18:58:39.570 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_data' removed 2024-09-13T18:58:39.583 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-13T18:58:39.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.697+0000 7f20eaf32700 1 Processor -- start 2024-09-13T18:58:39.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.697+0000 7f20eaf32700 1 -- start start 2024-09-13T18:58:39.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.697+0000 7f20eaf32700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20e40c11f0 0x7f20e40c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:39.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.697+0000 7f20eaf32700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e406e480 con 0x7f20e415a190 2024-09-13T18:58:39.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.697+0000 7f20eaf32700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e406e720 con 0x7f20e40c15d0 2024-09-13T18:58:39.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.697+0000 7f20eaf32700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4159630 con 0x7f20e40c11f0 2024-09-13T18:58:39.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.697+0000 7f20ea731700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f20e415a190 0x7f20e41567b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57868/0 (socket says 172.21.15.146:57868) 2024-09-13T18:58:39.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.697+0000 7f20ea731700 1 -- 172.21.15.146:0/1413720437 learned_addr learned my addr 172.21.15.146:0/1413720437 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-13T18:58:39.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.697+0000 7f20e9f30700 1 --2- 172.21.15.146:0/1413720437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20e40c11f0 0x7f20e40c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:39.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.697+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519566852 0 0) 0x7f20e406e480 con 0x7f20e415a190 2024-09-13T18:58:39.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.697+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20d00035d0 con 0x7f20e415a190 2024-09-13T18:58:39.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.697+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 680542829 0 0) 0x7f20e406e720 con 0x7f20e40c15d0 2024-09-13T18:58:39.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.697+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e406e480 con 0x7f20e40c15d0 2024-09-13T18:58:39.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1572265235 0 0) 0x7f20d00035d0 con 0x7f20e415a190 2024-09-13T18:58:39.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20e40c11f0 msgr2=0x7f20e40c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:39.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 --2- 172.21.15.146:0/1413720437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20e40c11f0 0x7f20e40c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:39.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 >> v1:172.21.15.154:6789/0 conn(0x7f20e40c15d0 legacy=0x7f20e4159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:39.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e406ea00 con 0x7f20e415a190 2024-09-13T18:58:39.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2759382160 0 0) 0x7f20dc0019f0 con 0x7f20e415a190 2024-09-13T18:58:39.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 >> v1:172.21.15.146:6789/0 conn(0x7f20e415a190 legacy=0x7f20e41567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:39.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 --2- 172.21.15.146:0/1413720437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d0003c50 0x7f20d0004050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:39.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 --2- 172.21.15.146:0/1413720437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d0004870 0x7f20d0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:39.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 --2- 172.21.15.146:0/1413720437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d0005230 0x7f20d000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:39.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4159630 con 0x7f20d0005230 2024-09-13T18:58:39.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d000c050 con 0x7f20d0004870 2024-09-13T18:58:39.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d000c210 con 0x7f20d0003c50 2024-09-13T18:58:39.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e9f30700 1 --2- 172.21.15.146:0/1413720437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20e40c11f0 0x7f20e40c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:58:39.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e972f700 1 --2- 172.21.15.146:0/1413720437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d0004870 0x7f20d0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:39.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20ea731700 1 --2- 172.21.15.146:0/1413720437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d0005230 0x7f20d000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:39.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e9f30700 1 --2- 172.21.15.146:0/1413720437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d0003c50 0x7f20d0004050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:39.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e972f700 1 -- 172.21.15.146:0/1413720437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d0003c50 msgr2=0x7f20d0004050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:39.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e972f700 1 --2- 172.21.15.146:0/1413720437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d0003c50 0x7f20d0004050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:39.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e972f700 1 -- 172.21.15.146:0/1413720437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d0005230 msgr2=0x7f20d000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:58:39.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e972f700 1 --2- 172.21.15.146:0/1413720437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d0005230 0x7f20d000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:39.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e972f700 1 -- 172.21.15.146:0/1413720437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d00035d0 con 0x7f20d0004870 2024-09-13T18:58:39.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e972f700 1 --2- 172.21.15.146:0/1413720437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d0004870 0x7f20d0004cf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f20d4007fd0 tx=0x7f20d00058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8252ccac0ad9dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:39.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4001440 con 0x7f20d0004870 2024-09-13T18:58:39.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f20d40052d0 con 0x7f20d0004870 2024-09-13T18:58:39.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20e8f2e700 1 -- 172.21.15.146:0/1413720437 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4001d50 con 0x7f20d0004870 2024-09-13T18:58:39.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20eaf32700 1 -- 172.21.15.146:0/1413720437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d0004870 msgr2=0x7f20d0004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:39.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20eaf32700 1 --2- 172.21.15.146:0/1413720437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d0004870 0x7f20d0004cf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f20d4007fd0 tx=0x7f20d00058a0 comp rx=0 tx=0).stop 2024-09-13T18:58:39.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20ea731700 1 -- 172.21.15.146:0/1413720437 reap_dead start 2024-09-13T18:58:39.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20eaf32700 1 -- 172.21.15.146:0/1413720437 shutdown_connections 2024-09-13T18:58:39.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20eaf32700 1 -- 172.21.15.146:0/1413720437 >> 172.21.15.146:0/1413720437 conn(0x7f20e4071ed0 msgr2=0x7f20e40722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:39.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20eaf32700 1 -- 172.21.15.146:0/1413720437 shutdown_connections 2024-09-13T18:58:39.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20eaf32700 1 -- 172.21.15.146:0/1413720437 wait complete. 2024-09-13T18:58:39.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20eaf32700 1 Processor -- start 2024-09-13T18:58:39.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.701+0000 7f20eaf32700 1 -- start start 2024-09-13T18:58:39.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20eaf32700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d0004870 0x7f20e41df920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:39.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20eaf32700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20e40711c0 0x7f20e41eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:39.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20eaf32700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20e4145960 0x7f20e4145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:39.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20eaf32700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e41e0480 con 0x7f20d0004870 2024-09-13T18:58:39.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20eaf32700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40709d0 con 0x7f20e4145960 2024-09-13T18:58:39.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20eaf32700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e41e3de0 con 0x7f20e40711c0 2024-09-13T18:58:39.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20e9f30700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d0004870 0x7f20e41df920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:39.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20ea731700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20e4145960 0x7f20e4145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:39.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20e972f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20e40711c0 0x7f20e41eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:39.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20e9f30700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d0004870 0x7f20e41df920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45686/0 (socket says 172.21.15.146:45686) 2024-09-13T18:58:39.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20e9f30700 1 -- 172.21.15.146:0/56419135 learned_addr learned my addr 172.21.15.146:0/56419135 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-13T18:58:39.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20e9f30700 1 -- 172.21.15.146:0/56419135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20e40711c0 msgr2=0x7f20e41eefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:39.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20e9f30700 1 --2- 172.21.15.146:0/56419135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20e40711c0 0x7f20e41eefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:39.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20e9f30700 1 -- 172.21.15.146:0/56419135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20e4145960 msgr2=0x7f20e4145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:39.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20e9f30700 1 --2- 172.21.15.146:0/56419135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20e4145960 0x7f20e4145de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:39.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20e9f30700 1 -- 172.21.15.146:0/56419135 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e4070f00 con 0x7f20d0004870 2024-09-13T18:58:39.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20e9f30700 1 --2- 172.21.15.146:0/56419135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d0004870 0x7f20e41df920 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f20e0011c30 tx=0x7f20e0011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:39.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20da7fc700 1 -- 172.21.15.146:0/56419135 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0006e80 con 0x7f20d0004870 2024-09-13T18:58:39.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20da7fc700 1 -- 172.21.15.146:0/56419135 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f20e000f9a0 con 0x7f20d0004870 2024-09-13T18:58:39.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20eaf32700 1 -- 172.21.15.146:0/56419135 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e4146320 con 0x7f20d0004870 2024-09-13T18:58:39.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20da7fc700 1 -- 172.21.15.146:0/56419135 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000fd10 con 0x7f20d0004870 2024-09-13T18:58:39.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20eaf32700 1 -- 172.21.15.146:0/56419135 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e40cbb90 con 0x7f20d0004870 2024-09-13T18:58:39.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.705+0000 7f20eaf32700 1 -- 172.21.15.146:0/56419135 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e406e170 con 0x7f20d0004870 2024-09-13T18:58:39.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.709+0000 7f20da7fc700 1 -- 172.21.15.146:0/56419135 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f20e0014440 con 0x7f20d0004870 2024-09-13T18:58:39.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.709+0000 7f20da7fc700 1 --2- 172.21.15.146:0/56419135 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f20c8066730 0x7f20c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:58:39.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.709+0000 7f20da7fc700 1 -- 172.21.15.146:0/56419135 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7329+0+0 (secure 0 0 0) 0x7f20e0084dc0 con 0x7f20d0004870 2024-09-13T18:58:39.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.709+0000 7f20e972f700 1 --2- 172.21.15.146:0/56419135 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f20c8066730 0x7f20c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:58:39.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.709+0000 7f20e972f700 1 --2- 172.21.15.146:0/56419135 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f20c8066730 0x7f20c8068bf0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f20d4015040 tx=0x7f20d4009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:58:39.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.713+0000 7f20da7fc700 1 -- 172.21.15.146:0/56419135 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f20e0051ad0 con 0x7f20d0004870 2024-09-13T18:58:39.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:39.889+0000 7f20eaf32700 1 -- 172.21.15.146:0/56419135 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f20e40b0e00 con 0x7f20d0004870 2024-09-13T18:58:40.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:40.569+0000 7f20da7fc700 1 -- 172.21.15.146:0/56419135 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v47) v1 ==== 178+0+0 (secure 0 0 0) 0x7f20e00550f0 con 0x7f20d0004870 2024-09-13T18:58:40.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:40.573+0000 7f20eaf32700 1 -- 172.21.15.146:0/56419135 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f20c8066730 msgr2=0x7f20c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:40.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:40.573+0000 7f20eaf32700 1 --2- 172.21.15.146:0/56419135 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f20c8066730 0x7f20c8068bf0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f20d4015040 tx=0x7f20d4009030 comp rx=0 tx=0).stop 2024-09-13T18:58:40.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:40.573+0000 7f20eaf32700 1 -- 172.21.15.146:0/56419135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d0004870 msgr2=0x7f20e41df920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:58:40.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:40.573+0000 7f20eaf32700 1 --2- 172.21.15.146:0/56419135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d0004870 0x7f20e41df920 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f20e0011c30 tx=0x7f20e0011f40 comp rx=0 tx=0).stop 2024-09-13T18:58:40.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:40.573+0000 7f20eaf32700 1 -- 172.21.15.146:0/56419135 shutdown_connections 2024-09-13T18:58:40.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:40.573+0000 7f20eaf32700 1 --2- 172.21.15.146:0/56419135 >> [v2:172.21.15.154:6832/1235826889,v1:172.21.15.154:6833/1235826889] conn(0x7f20c8066730 0x7f20c8068bf0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:40.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:40.573+0000 7f20eaf32700 1 --2- 172.21.15.146:0/56419135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20e4145960 0x7f20e4145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:40.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:40.573+0000 7f20eaf32700 1 --2- 172.21.15.146:0/56419135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20e40711c0 0x7f20e41eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:40.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:40.573+0000 7f20eaf32700 1 --2- 172.21.15.146:0/56419135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d0004870 0x7f20e41df920 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:58:40.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:40.573+0000 7f20eaf32700 1 -- 172.21.15.146:0/56419135 >> 172.21.15.146:0/56419135 conn(0x7f20e4071ed0 msgr2=0x7f20e4151790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:58:40.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:40.573+0000 7f20eaf32700 1 -- 172.21.15.146:0/56419135 shutdown_connections 2024-09-13T18:58:40.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-13T18:58:40.573+0000 7f20eaf32700 1 -- 172.21.15.146:0/56419135 wait complete. 2024-09-13T18:58:40.585 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_data_ec' removed 2024-09-13T18:58:40.608 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T18:58:40.609 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T18:58:40.609 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:58:40.620 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T18:58:40.620 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T18:58:40.620 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:58:40.629 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T18:58:40.630 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-09-13T18:58:40.630 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:58:40.682 INFO:tasks.ceph.mds.e:Stopped 2024-09-13T18:58:40.682 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T18:58:40.682 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:58:40.702 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T18:58:40.703 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T18:58:40.703 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:58:40.714 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T18:58:40.714 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-09-13T18:58:40.714 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:58:40.781 INFO:tasks.ceph.mds.f:Stopped 2024-09-13T18:58:40.781 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T18:58:40.782 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T18:58:40.782 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:58:46.884 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T18:58:46.884 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T18:58:46.884 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:58:47.618 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-13T18:58:52.986 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T18:58:52.987 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T18:58:52.987 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:58:53.728 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-13T18:58:53.728 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-13T18:58:59.089 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T18:58:59.090 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T18:58:59.090 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:58:59.737 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-13T18:58:59.737 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-13T18:59:05.193 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T18:59:05.193 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T18:59:05.193 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:59:05.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-13T18:59:05.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-13T18:59:11.296 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T18:59:11.296 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T18:59:11.296 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:59:11.453 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-13T18:59:11.453 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-13T18:59:17.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-13T18:59:17.160 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-13T18:59:17.398 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T18:59:17.399 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T18:59:17.399 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:59:22.766 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-13T18:59:22.766 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-13T18:59:23.500 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T18:59:23.501 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T18:59:23.501 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:59:28.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-13T18:59:28.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-13T18:59:29.603 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T18:59:29.603 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T18:59:29.603 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T18:59:29.604 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:59:29.629 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T18:59:29.629 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T18:59:29.629 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:59:29.658 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T18:59:29.659 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T18:59:29.659 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T18:59:29.659 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:59:29.705 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T18:59:29.705 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T18:59:29.705 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:59:29.758 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T18:59:29.758 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T18:59:29.758 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:59:29.849 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T18:59:29.850 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T18:59:29.850 DEBUG:teuthology.orchestra.run.smithi146:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v 'Replacing daemon mds' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-13T18:59:29.949 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-13T18:59:29.949 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T18:59:31.015 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-13T18:59:31.015 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T18:59:31.261 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-13T18:59:31.261 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T18:59:31.541 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-13T18:59:31.541 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T18:59:31.785 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-13T18:59:31.786 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T18:59:32.121 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-13T18:59:32.122 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T18:59:32.372 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-13T18:59:32.372 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T18:59:32.576 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-13T18:59:32.576 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T18:59:32.807 INFO:tasks.ceph:Archiving mon data... 2024-09-13T18:59:32.808 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903977/data/mon.a.tgz 2024-09-13T18:59:32.808 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-09-13T18:59:32.829 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.7DM34LYiQR 2024-09-13T18:59:32.829 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.7DM34LYiQR 2024-09-13T18:59:32.935 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.7DM34LYiQR 2024-09-13T18:59:33.044 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.7DM34LYiQR is 434KB 2024-09-13T18:59:33.111 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.7DM34LYiQR 2024-09-13T18:59:33.118 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903977/data/mon.b.tgz 2024-09-13T18:59:33.118 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-09-13T18:59:33.150 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.1yEcS7ovcd 2024-09-13T18:59:33.150 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.1yEcS7ovcd 2024-09-13T18:59:33.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-13T18:59:33.285 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.1yEcS7ovcd 2024-09-13T18:59:33.334 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.1yEcS7ovcd is 434KB 2024-09-13T18:59:33.356 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.1yEcS7ovcd 2024-09-13T18:59:33.361 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903977/data/mon.c.tgz 2024-09-13T18:59:33.362 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-09-13T18:59:33.408 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.Iv0pFkI5QE 2024-09-13T18:59:33.409 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Iv0pFkI5QE 2024-09-13T18:59:33.513 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.Iv0pFkI5QE 2024-09-13T18:59:33.535 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.Iv0pFkI5QE is 434KB 2024-09-13T18:59:33.601 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.Iv0pFkI5QE 2024-09-13T18:59:33.607 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T18:59:33.608 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T18:59:33.610 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T18:59:33.653 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T18:59:33.653 DEBUG:teuthology.orchestra.run.smithi146:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T18:59:33.674 DEBUG:teuthology.orchestra.run.smithi154:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T18:59:33.702 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T18:59:33.705 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T18:59:33.712 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T18:59:33.714 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903977/remote/smithi146/crash 2024-09-13T18:59:33.715 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T18:59:33.734 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903977/remote/smithi154/crash 2024-09-13T18:59:33.734 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T18:59:33.770 INFO:tasks.ceph:Shutting down logrotate 2024-09-13T18:59:49.248 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T18:59:49.268 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T18:59:49.285 INFO:tasks.ceph:Compressing logs... 2024-09-13T18:59:49.285 DEBUG:teuthology.orchestra.run.smithi146:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:59:49.316 DEBUG:teuthology.orchestra.run.smithi154:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T19:07:59.276 INFO:tasks.ceph:Archiving logs... 2024-09-13T19:07:59.277 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903977/remote/smithi146/log 2024-09-13T19:07:59.278 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T19:09:19.070 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903977/remote/smithi154/log 2024-09-13T19:09:19.071 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T19:10:11.041 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T19:10:11.051 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T19:10:11.051 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T19:10:11.054 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T19:10:11.144 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T19:10:11.155 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T19:10:11.156 DEBUG:teuthology.orchestra.run.smithi146:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T19:10:11.158 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T19:10:11.229 INFO:teuthology.orchestra.run.smithi146.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T19:10:11.229 INFO:teuthology.orchestra.run.smithi146.stdout:============================================================================== 2024-09-13T19:10:11.229 INFO:teuthology.orchestra.run.smithi146.stdout:*hv01.front.sepi 67.205.162.81 3 u 54 64 377 0.097 6.281 0.447 2024-09-13T19:10:11.229 INFO:teuthology.orchestra.run.smithi146.stdout:+hv02.front.sepi 96.245.170.99 3 u 39 64 377 0.091 1.459 0.577 2024-09-13T19:10:11.230 INFO:teuthology.orchestra.run.smithi146.stdout:+hv03.front.sepi 74.6.168.72 3 u 52 64 377 0.093 6.147 0.612 2024-09-13T19:10:11.230 INFO:teuthology.orchestra.run.smithi146.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-09-13T19:10:11.248 INFO:teuthology.orchestra.run.smithi154.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T19:10:11.248 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================== 2024-09-13T19:10:11.248 INFO:teuthology.orchestra.run.smithi154.stdout:*hv01.front.sepi 67.205.162.81 3 u 40 128 377 0.093 1.559 0.118 2024-09-13T19:10:11.248 INFO:teuthology.orchestra.run.smithi154.stdout:+hv02.front.sepi 96.245.170.99 3 u 110 128 377 0.114 -2.869 0.115 2024-09-13T19:10:11.248 INFO:teuthology.orchestra.run.smithi154.stdout:+hv03.front.sepi 74.6.168.72 3 u 114 128 377 0.108 1.862 0.139 2024-09-13T19:10:11.248 INFO:teuthology.orchestra.run.smithi154.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-09-13T19:10:11.249 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T19:10:11.259 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T19:10:11.260 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T19:10:11.269 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T19:10:11.279 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T19:10:11.291 INFO:teuthology.task.internal:Duration was 3204.903389 seconds 2024-09-13T19:10:11.291 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T19:10:11.306 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T19:10:11.306 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T19:10:11.309 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T19:10:11.459 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T19:10:11.459 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi146.front.sepia.ceph.com 2024-09-13T19:10:11.459 DEBUG:teuthology.orchestra.run.smithi146:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T19:10:11.474 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2024-09-13T19:10:11.474 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T19:10:11.489 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T19:10:11.489 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T19:10:11.492 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T19:10:11.556 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T19:10:11.556 DEBUG:teuthology.orchestra.run.smithi146:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T19:10:11.559 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T19:10:11.758 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T19:10:11.767 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T19:10:11.768 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T19:10:11.793 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T19:10:11.822 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T19:10:11.833 DEBUG:teuthology.orchestra.run.smithi146:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T19:10:11.836 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T19:10:11.859 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2024-09-13T19:10:11.869 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = core 2024-09-13T19:10:11.912 DEBUG:teuthology.orchestra.run.smithi146:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T19:10:11.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T19:10:11.962 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T19:10:11.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T19:10:11.968 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T19:10:11.978 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T19:10:11.979 DEBUG:teuthology.misc:Transferring archived files from smithi146:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903977/remote/smithi146 2024-09-13T19:10:11.980 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T19:10:12.007 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903977/remote/smithi154 2024-09-13T19:10:12.008 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T19:10:12.032 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T19:10:12.032 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T19:10:12.034 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T19:10:12.040 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T19:10:12.051 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T19:10:12.051 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T19:10:12.061 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T19:10:12.061 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T19:10:12.084 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T19:10:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 19:10 /home/ubuntu/cephtest 2024-09-13T19:10:12.092 INFO:teuthology.orchestra.run.smithi154.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 19:10 /home/ubuntu/cephtest 2024-09-13T19:10:12.113 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T19:10:12.123 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T19:10:12.149 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} duration: 3204.9033885002136 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-13T19:10:12.149 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T19:10:12.248 INFO:teuthology.run:pass